2024-04-25T09:48:33.125 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:48:33.125 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040 --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --verbose -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/orig.config.yaml 2024-04-25T09:48:33.167 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:48:33.250 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673040' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi082.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHN7JvV1wTrMW5BDv21p8/DFaAdvrNvn+Rt9PcIjCHMKwFMep4ZwBmAkGkl63VW9vmKO1FSqTBRE+TZWzTO4CZw= smithi196.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+M9uE/UVSWDnE37gtjNlQSL9UjkrinuUY/QPsLENLe/7sIWyRDLndrtq+p0JzsceXDgS9RF8XFSSqoC3wyJaQ= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:48:33.251 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:48:33.251 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:48:33.251 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:48:33.253 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:48:33.277 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:48:33.277 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:48:33.277 INFO:teuthology.packaging:ref: None 2024-04-25T09:48:33.277 INFO:teuthology.packaging:tag: None 2024-04-25T09:48:33.277 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:48:33.278 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:48:33.278 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:48:33.425 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:48:33.426 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:48:33.433 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:48:33.433 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:48:33.503 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:48:33.520 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:48:33.526 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:48:33.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi082.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:43:11.883610', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIENY+wdZlpBk/bB6jZvAz+EuXheTsVWZeCc3bzypik0x'} 2024-04-25T09:48:33.568 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi196.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:43:11.885015', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDCuIaOJorPj+q+s2KFIw/J7P+krUV1oiGu7D/lkAjQO'} 2024-04-25T09:48:33.568 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:48:33.574 INFO:teuthology.task.internal:roles: ubuntu@smithi082.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:48:33.574 INFO:teuthology.task.internal:roles: ubuntu@smithi196.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:48:33.574 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:48:33.636 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7facea7d30d0>, signals=[15]) 2024-04-25T09:48:33.636 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:48:33.643 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:48:33.643 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T09:48:33.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:48:33.725 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T09:48:33.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:48:33.807 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:48:33.814 DEBUG:teuthology.orchestra.run.smithi082:> uname -m 2024-04-25T09:48:33.820 INFO:teuthology.orchestra.run.smithi082.stdout:x86_64 2024-04-25T09:48:33.820 DEBUG:teuthology.orchestra.run.smithi082:> cat /etc/os-release 2024-04-25T09:48:33.869 INFO:teuthology.orchestra.run.smithi082.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:48:33.869 INFO:teuthology.orchestra.run.smithi082.stdout:NAME="Ubuntu" 2024-04-25T09:48:33.869 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_ID="22.04" 2024-04-25T09:48:33.869 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:VERSION_CODENAME=jammy 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:ID=ubuntu 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:ID_LIKE=debian 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:48:33.870 INFO:teuthology.orchestra.run.smithi082.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:48:33.871 INFO:teuthology.lock.ops:Updating smithi082.front.sepia.ceph.com on lock server 2024-04-25T09:48:33.899 DEBUG:teuthology.orchestra.run.smithi196:> uname -m 2024-04-25T09:48:33.905 INFO:teuthology.orchestra.run.smithi196.stdout:x86_64 2024-04-25T09:48:33.905 DEBUG:teuthology.orchestra.run.smithi196:> cat /etc/os-release 2024-04-25T09:48:33.951 INFO:teuthology.orchestra.run.smithi196.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T09:48:33.951 INFO:teuthology.orchestra.run.smithi196.stdout:NAME="Ubuntu" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_ID="22.04" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:VERSION_CODENAME=jammy 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:ID=ubuntu 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:ID_LIKE=debian 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T09:48:33.952 INFO:teuthology.orchestra.run.smithi196.stdout:UBUNTU_CODENAME=jammy 2024-04-25T09:48:33.952 INFO:teuthology.lock.ops:Updating smithi196.front.sepia.ceph.com on lock server 2024-04-25T09:48:33.973 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:48:33.983 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:48:33.988 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:48:33.989 DEBUG:teuthology.orchestra.run.smithi082:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:48:33.991 DEBUG:teuthology.orchestra.run.smithi196:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:48:33.999 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:48:34.005 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:48:34.005 DEBUG:teuthology.orchestra.run.smithi082:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:48:34.038 DEBUG:teuthology.orchestra.run.smithi196:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:48:34.049 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:48:34.095 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:48:34.111 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T09:48:34.111 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T09:48:34.111 DEBUG:teuthology.orchestra.run.smithi082:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:48:34.112 DEBUG:teuthology.orchestra.run.smithi196:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:48:34.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:34.116 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-25T09:48:34.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:48:34.117 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-25T09:48:34.162 INFO:teuthology.orchestra.run.smithi082.stdout:5.15.0-56-generic 2024-04-25T09:48:34.162 INFO:teuthology.task.kernel:Running kernel on smithi082: 5.15.0-56-generic 2024-04-25T09:48:34.162 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get clean 2024-04-25T09:48:34.164 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-56-generic 2024-04-25T09:48:34.164 INFO:teuthology.task.kernel:Running kernel on smithi196: 5.15.0-56-generic 2024-04-25T09:48:34.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-25T09:48:34.292 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get update 2024-04-25T09:48:34.308 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T09:48:34.559 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:48:34.578 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:48:34.580 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:48:34.589 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T09:48:34.623 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:48:34.639 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T09:48:34.988 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:48:34.995 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T09:48:35.004 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:48:35.006 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T09:48:35.127 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:48:35.134 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:48:35.352 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T09:48:35.430 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:48:35.436 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T09:48:35.447 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T09:48:35.447 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:48:35.454 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T09:48:35.457 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:48:35.457 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:48:35.459 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:48:35.463 INFO:teuthology.orchestra.run.smithi082.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T09:48:35.463 INFO:teuthology.orchestra.run.smithi082.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T09:48:35.465 INFO:teuthology.orchestra.run.smithi082.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T09:48:35.478 INFO:teuthology.orchestra.run.smithi082.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:48:35.487 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T09:48:35.493 INFO:teuthology.orchestra.run.smithi082.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:48:35.494 INFO:teuthology.orchestra.run.smithi082.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:48:35.502 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T09:48:35.503 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:48:35.504 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T09:48:35.553 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:48:35.556 INFO:teuthology.orchestra.run.smithi082.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T09:48:35.557 INFO:teuthology.orchestra.run.smithi082.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:48:35.575 INFO:teuthology.orchestra.run.smithi082.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T09:48:35.575 INFO:teuthology.orchestra.run.smithi082.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:48:35.580 INFO:teuthology.orchestra.run.smithi082.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:48:35.581 INFO:teuthology.orchestra.run.smithi082.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:48:35.597 INFO:teuthology.orchestra.run.smithi082.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:48:35.600 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T09:48:35.600 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:48:35.601 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T09:48:35.609 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T09:48:35.618 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T09:48:35.740 INFO:teuthology.orchestra.run.smithi082.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:48:35.774 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T09:48:35.775 INFO:teuthology.orchestra.run.smithi082.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:48:35.794 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T09:48:36.185 INFO:teuthology.orchestra.run.smithi082.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:48:36.190 INFO:teuthology.orchestra.run.smithi082.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:48:36.204 INFO:teuthology.orchestra.run.smithi082.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:48:36.205 INFO:teuthology.orchestra.run.smithi082.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:48:36.205 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T09:48:36.207 INFO:teuthology.orchestra.run.smithi082.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:48:36.211 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T09:48:36.225 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T09:48:36.226 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T09:48:36.229 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T09:48:36.375 INFO:teuthology.orchestra.run.smithi082.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:48:36.404 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T09:48:36.413 INFO:teuthology.orchestra.run.smithi082.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:48:36.426 INFO:teuthology.orchestra.run.smithi082.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:48:36.427 INFO:teuthology.orchestra.run.smithi082.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:48:36.429 INFO:teuthology.orchestra.run.smithi082.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:48:36.430 INFO:teuthology.orchestra.run.smithi082.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:48:36.430 INFO:teuthology.orchestra.run.smithi082.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:48:36.431 INFO:teuthology.orchestra.run.smithi082.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:48:36.458 INFO:teuthology.orchestra.run.smithi082.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:48:36.462 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T09:48:36.466 INFO:teuthology.orchestra.run.smithi082.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:48:36.481 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T09:48:36.483 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T09:48:36.487 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T09:48:36.487 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T09:48:36.488 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T09:48:36.488 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T09:48:36.492 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T09:48:36.497 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T09:48:36.631 INFO:teuthology.orchestra.run.smithi082.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:48:36.666 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T09:48:36.714 INFO:teuthology.orchestra.run.smithi082.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:48:36.749 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T09:48:36.798 INFO:teuthology.orchestra.run.smithi082.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:48:36.837 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T09:48:36.879 INFO:teuthology.orchestra.run.smithi082.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:48:36.911 INFO:teuthology.orchestra.run.smithi082.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:48:36.923 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T09:48:36.957 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T09:48:41.137 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 13.3 MB in 3s (5,215 kB/s) 2024-04-25T09:48:41.196 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 13.3 MB in 3s (5,145 kB/s) 2024-04-25T09:48:42.479 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:48:42.495 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:48:42.497 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:42.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:48:42.551 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:48:42.564 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:48:42.730 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T09:48:42.730 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T09:48:42.742 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T09:48:42.742 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T09:48:42.886 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout: python2.7-minimal 2024-04-25T09:48:42.887 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:42.888 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T09:48:42.888 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:42.888 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:48:42.889 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:42.890 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T09:48:42.890 INFO:teuthology.orchestra.run.smithi196.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:48:42.890 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-25T09:48:42.890 INFO:teuthology.orchestra.run.smithi196.stdout: thermald 2024-04-25T09:48:42.894 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:48:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:48:42.894 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:48:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:48:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T09:48:42.895 INFO:teuthology.orchestra.run.smithi082.stdout: python2.7-minimal 2024-04-25T09:48:42.895 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:48:42.896 INFO:teuthology.orchestra.run.smithi082.stdout:The following additional packages will be installed: 2024-04-25T09:48:42.897 INFO:teuthology.orchestra.run.smithi082.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:42.897 INFO:teuthology.orchestra.run.smithi082.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T09:48:42.897 INFO:teuthology.orchestra.run.smithi082.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:42.898 INFO:teuthology.orchestra.run.smithi082.stdout:Suggested packages: 2024-04-25T09:48:42.898 INFO:teuthology.orchestra.run.smithi082.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T09:48:42.898 INFO:teuthology.orchestra.run.smithi082.stdout:Recommended packages: 2024-04-25T09:48:42.898 INFO:teuthology.orchestra.run.smithi082.stdout: thermald 2024-04-25T09:48:42.941 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T09:48:42.941 INFO:teuthology.orchestra.run.smithi196.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:42.941 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:48:42.942 INFO:teuthology.orchestra.run.smithi196.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:42.943 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-25T09:48:42.943 INFO:teuthology.orchestra.run.smithi196.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:48:42.948 INFO:teuthology.orchestra.run.smithi082.stdout:The following NEW packages will be installed: 2024-04-25T09:48:42.948 INFO:teuthology.orchestra.run.smithi082.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T09:48:42.948 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T09:48:42.948 INFO:teuthology.orchestra.run.smithi082.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T09:48:42.949 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages will be upgraded: 2024-04-25T09:48:42.950 INFO:teuthology.orchestra.run.smithi082.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T09:48:43.132 INFO:teuthology.orchestra.run.smithi196.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:43.132 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 113 MB of archives. 2024-04-25T09:48:43.132 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:48:43.132 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:48:43.133 INFO:teuthology.orchestra.run.smithi082.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:48:43.133 INFO:teuthology.orchestra.run.smithi082.stdout:Need to get 113 MB of archives. 2024-04-25T09:48:43.133 INFO:teuthology.orchestra.run.smithi082.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T09:48:43.134 INFO:teuthology.orchestra.run.smithi082.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T09:48:44.394 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:48:44.457 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T09:48:44.655 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:48:44.774 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T09:48:46.426 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:48:46.426 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:48:46.427 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:48:46.666 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:48:46.726 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:48:46.735 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T09:48:46.735 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T09:48:46.735 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T09:48:46.979 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T09:48:47.038 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T09:48:47.107 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 113 MB in 4s (30.0 MB/s) 2024-04-25T09:48:47.265 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:48:47.435 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 113 MB in 4s (27.8 MB/s) 2024-04-25T09:48:47.603 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T09:48:49.570 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:48:49.575 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:49.672 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T09:48:49.677 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:49.703 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:49.942 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:51.771 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:48:51.792 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:51.885 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:52.094 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T09:48:52.116 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:52.214 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:52.324 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:48:52.345 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:52.373 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:52.663 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T09:48:52.684 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:48:52.721 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:48:58.755 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:58.882 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:59.057 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:59.166 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:59.232 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:59.371 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:59.498 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:48:59.619 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:48:59.657 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:48:59.668 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:48:59.707 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:48:59.897 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T09:48:59.919 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T09:48:59.964 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:49:05.107 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:49:05.143 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:49:05.181 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:05.575 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T09:49:05.612 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T09:49:05.650 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:08.005 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:49:08.141 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:49:08.269 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T09:49:08.394 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T09:49:08.474 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:49:08.595 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:08.730 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:49:08.736 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T09:49:08.858 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:08.858 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:08.984 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T09:49:09.120 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:10.272 INFO:teuthology.orchestra.run.smithi082.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:10.273 INFO:teuthology.orchestra.run.smithi082.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:10.273 INFO:teuthology.orchestra.run.smithi082.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:10.273 INFO:teuthology.orchestra.run.smithi082.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:10.473 INFO:teuthology.orchestra.run.smithi196.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:10.698 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:10.785 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:12.083 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:12.212 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:13.484 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:49:13.587 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T09:49:13.609 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:49:13.705 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T09:49:13.738 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:13.803 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:49:13.803 INFO:teuthology.orchestra.run.smithi196.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:13.839 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T09:49:13.899 INFO:teuthology.orchestra.run.smithi082.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T09:49:13.899 INFO:teuthology.orchestra.run.smithi082.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:27.545 INFO:teuthology.orchestra.run.smithi196.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:49:27.546 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:49:27.556 INFO:teuthology.orchestra.run.smithi196.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:49:27.573 INFO:teuthology.orchestra.run.smithi196.stdout:Generating grub configuration file ... 2024-04-25T09:49:27.746 INFO:teuthology.orchestra.run.smithi082.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T09:49:27.746 INFO:teuthology.orchestra.run.smithi082.stdout:Sourcing file `/etc/default/grub' 2024-04-25T09:49:27.755 INFO:teuthology.orchestra.run.smithi082.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:49:27.777 INFO:teuthology.orchestra.run.smithi082.stdout:Generating grub configuration file ... 2024-04-25T09:49:27.915 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:27.931 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:28.056 INFO:teuthology.orchestra.run.smithi082.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:28.074 INFO:teuthology.orchestra.run.smithi082.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:28.211 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:28.217 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:28.294 INFO:teuthology.orchestra.run.smithi196.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:49:28.299 INFO:teuthology.orchestra.run.smithi196.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:49:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:28.346 INFO:teuthology.orchestra.run.smithi082.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:28.410 INFO:teuthology.orchestra.run.smithi082.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:49:28.416 INFO:teuthology.orchestra.run.smithi082.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:49:28.483 INFO:teuthology.orchestra.run.smithi196.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:49:28.483 INFO:teuthology.orchestra.run.smithi196.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:49:28.483 INFO:teuthology.orchestra.run.smithi196.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:49:28.528 INFO:teuthology.orchestra.run.smithi196.stdout:done 2024-04-25T09:49:28.566 INFO:teuthology.orchestra.run.smithi082.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:49:28.567 INFO:teuthology.orchestra.run.smithi082.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:49:28.567 INFO:teuthology.orchestra.run.smithi082.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:49:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:done 2024-04-25T09:49:29.001 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-25T09:49:29.021 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-25T09:49:29.021 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.105.102 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:49:29.022 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-25T09:49:29.023 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:49:29.023 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:49:29.023 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:49:29.023 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T09:49:29.023 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi196.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:49:29.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get clean 2024-04-25T09:49:29.112 DEBUG:teuthology.orchestra.run.smithi082:> dpkg -s linux-image-generic 2024-04-25T09:49:29.135 INFO:teuthology.orchestra.run.smithi082.stdout:Package: linux-image-generic 2024-04-25T09:49:29.135 INFO:teuthology.orchestra.run.smithi082.stdout:Status: install ok installed 2024-04-25T09:49:29.135 INFO:teuthology.orchestra.run.smithi082.stdout:Priority: optional 2024-04-25T09:49:29.135 INFO:teuthology.orchestra.run.smithi082.stdout:Section: kernel 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Installed-Size: 21 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: amd64 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Source: linux-meta 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Version: 5.15.0.105.102 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Recommends: thermald 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout:Description: Generic Linux kernel image 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:49:29.136 INFO:teuthology.orchestra.run.smithi082.stdout: available. 2024-04-25T09:49:29.136 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T09:49:29.136 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T09:49:29.137 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T09:49:29.137 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T09:49:29.137 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi082.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T09:49:29.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get clean 2024-04-25T09:49:29.207 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T09:49:29.300 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get update 2024-04-25T09:49:29.324 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:49:29.424 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:49:29.424 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:49:29.437 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:49:29.447 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:49:29.457 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:49:29.523 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:49:29.615 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:49:30.673 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:49:30.690 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:49:30.730 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:49:30.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T09:49:30.749 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:49:30.812 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:49:30.950 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T09:49:30.950 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T09:49:31.013 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T09:49:31.014 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T09:49:31.126 INFO:teuthology.orchestra.run.smithi196.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:49:31.127 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:49:31.172 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:49:31.175 DEBUG:teuthology.orchestra.run.smithi196:> dpkg -s linux-image-generic 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Package: linux-image-generic 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Status: install ok installed 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Priority: optional 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Section: kernel 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Installed-Size: 21 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Architecture: amd64 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Source: linux-meta 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Version: 5.15.0.105.102 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:49:31.187 INFO:teuthology.orchestra.run.smithi196.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:49:31.188 INFO:teuthology.orchestra.run.smithi196.stdout:Recommends: thermald 2024-04-25T09:49:31.188 INFO:teuthology.orchestra.run.smithi196.stdout:Description: Generic Linux kernel image 2024-04-25T09:49:31.188 INFO:teuthology.orchestra.run.smithi196.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:49:31.188 INFO:teuthology.orchestra.run.smithi196.stdout: available. 2024-04-25T09:49:31.188 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T09:49:31.194 INFO:teuthology.orchestra.run.smithi082.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T09:49:31.194 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:49:31.194 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:49:31.194 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:49:31.194 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:49:31.195 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:49:31.195 INFO:teuthology.orchestra.run.smithi082.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:49:31.195 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:49:31.235 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.uexhYsr86U 2024-04-25T09:49:31.235 DEBUG:teuthology.orchestra.run.smithi196:> sudo cp /boot/grub/grub.cfg /tmp/tmp.uexhYsr86U 2024-04-25T09:49:31.238 INFO:teuthology.orchestra.run.smithi082.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T09:49:31.240 DEBUG:teuthology.orchestra.run.smithi082:> dpkg -s linux-image-generic 2024-04-25T09:49:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:Package: linux-image-generic 2024-04-25T09:49:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:Status: install ok installed 2024-04-25T09:49:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:Priority: optional 2024-04-25T09:49:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:Section: kernel 2024-04-25T09:49:31.255 INFO:teuthology.orchestra.run.smithi082.stdout:Installed-Size: 21 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Architecture: amd64 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Source: linux-meta 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Version: 5.15.0.105.102 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Recommends: thermald 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout:Description: Generic Linux kernel image 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout: This package will always depend on the latest generic kernel image 2024-04-25T09:49:31.256 INFO:teuthology.orchestra.run.smithi082.stdout: available. 2024-04-25T09:49:31.256 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-04-25T09:49:31.293 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.uexhYsr86U 2024-04-25T09:49:31.301 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.nTIwrvlCHD 2024-04-25T09:49:31.301 DEBUG:teuthology.orchestra.run.smithi082:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nTIwrvlCHD 2024-04-25T09:49:31.358 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.nTIwrvlCHD 2024-04-25T09:49:31.406 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.uexhYsr86U is 10KB 2024-04-25T09:49:31.420 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.uexhYsr86U 2024-04-25T09:49:31.425 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.478 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T09:49:31.479 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.479 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.481 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.nTIwrvlCHD is 10KB 2024-04-25T09:49:31.494 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.nTIwrvlCHD 2024-04-25T09:49:31.500 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.542 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:49:31.542 DEBUG:teuthology.orchestra.run.smithi196:> sudo update-grub 2024-04-25T09:49:31.553 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T09:49:31.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T09:49:31.627 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T09:49:31.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo update-grub 2024-04-25T09:49:32.698 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:49:32.699 INFO:teuthology.orchestra.run.smithi082.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:49:32.701 INFO:teuthology.orchestra.run.smithi082.stderr:Generating grub configuration file ... 2024-04-25T09:49:32.742 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub' 2024-04-25T09:49:32.743 INFO:teuthology.orchestra.run.smithi196.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T09:49:32.745 INFO:teuthology.orchestra.run.smithi196.stderr:Generating grub configuration file ... 2024-04-25T09:49:32.900 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:32.905 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:32.925 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T09:49:32.930 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T09:49:33.150 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:33.156 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:33.157 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T09:49:33.163 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T09:49:33.199 INFO:teuthology.orchestra.run.smithi082.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:49:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:49:33.205 INFO:teuthology.orchestra.run.smithi196.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T09:49:33.210 INFO:teuthology.orchestra.run.smithi196.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T09:49:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:49:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:49:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:49:33.334 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T09:49:33.334 INFO:teuthology.orchestra.run.smithi196.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T09:49:33.335 INFO:teuthology.orchestra.run.smithi196.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T09:49:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:done 2024-04-25T09:49:33.353 DEBUG:teuthology.orchestra.run.smithi082:> sudo shutdown -r now 2024-04-25T09:49:33.353 INFO:teuthology.orchestra.run.smithi196.stderr:done 2024-04-25T09:49:33.361 DEBUG:teuthology.orchestra.run.smithi196:> sudo shutdown -r now 2024-04-25T09:50:03.390 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T09:50:03.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T09:50:03.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:50:03.399 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T09:50:03.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T09:50:03.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:03.455 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:51:03.456 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T09:51:12.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T09:51:12.464 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:12.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T09:51:12.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:13.026 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-25T09:51:13.189 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-25T09:51:13.868 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T09:51:13.868 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T09:51:13.869 DEBUG:teuthology.orchestra.run.smithi082:> uname -r 2024-04-25T09:51:13.916 INFO:teuthology.orchestra.run.smithi082.stdout:5.15.0-105-generic 2024-04-25T09:51:13.916 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:51:13.916 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:51:13.916 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:51:14.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T09:51:14.115 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T09:51:14.115 DEBUG:teuthology.orchestra.run.smithi196:> uname -r 2024-04-25T09:51:14.166 INFO:teuthology.orchestra.run.smithi196.stdout:5.15.0-105-generic 2024-04-25T09:51:14.166 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T09:51:14.166 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:51:14.166 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T09:51:14.917 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T09:51:14.918 DEBUG:teuthology.orchestra.run.smithi082:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:51:15.103 INFO:teuthology.orchestra.run.smithi082.stdout:ttyS1 2024-04-25T09:51:15.135 DEBUG:teuthology.parallel:result is None 2024-04-25T09:51:15.167 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T09:51:15.167 DEBUG:teuthology.orchestra.run.smithi196:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T09:51:15.580 INFO:teuthology.orchestra.run.smithi196.stdout:ttyS1 2024-04-25T09:51:15.600 DEBUG:teuthology.parallel:result is None 2024-04-25T09:51:15.600 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:51:15.607 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:51:15.608 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:51:15.610 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:51:15.616 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:51:15.622 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:51:15.636 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:51:15.636 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:51:15.657 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:51:15.721 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:51:15.728 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:51:15.729 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:51:15.732 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:51:15.754 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:51:15.759 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:51:15.762 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:51:15.767 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:51:15.768 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:51:15.775 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:51:15.775 DEBUG:teuthology.orchestra.run.smithi082:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:51:15.805 DEBUG:teuthology.orchestra.run.smithi196:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:51:15.843 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:51:15.853 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:51:15.853 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:51:15.861 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:51:15.890 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:51:15.916 DEBUG:teuthology.orchestra.run.smithi082:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:51:15.964 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T09:51:15.964 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:51:16.030 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:51:16.036 DEBUG:teuthology.orchestra.run.smithi196:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:51:16.082 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T09:51:16.082 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:51:16.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo service rsyslog restart 2024-04-25T09:51:16.174 DEBUG:teuthology.orchestra.run.smithi196:> sudo service rsyslog restart 2024-04-25T09:51:16.295 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:51:16.303 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:51:16.303 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:51:16.334 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:51:16.342 DEBUG:teuthology.task.selinux:Excluding smithi082: OS 'ubuntu' does not support SELinux 2024-04-25T09:51:16.342 DEBUG:teuthology.task.selinux:Excluding smithi196: OS 'ubuntu' does not support SELinux 2024-04-25T09:51:16.342 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:51:16.343 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T09:51:16.343 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:51:16.343 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:51:16.352 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:51:16.375 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:51:16.376 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi082.front.sepia.ceph.com,smithi196.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:58:18.283 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi082.front.sepia.ceph.com'), Remote(name='ubuntu@smithi196.front.sepia.ceph.com')] 2024-04-25T09:58:18.284 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T09:58:18.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi082.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:18.368 DEBUG:teuthology.orchestra.run.smithi082:> true 2024-04-25T09:58:18.446 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi082.front.sepia.ceph.com' 2024-04-25T09:58:18.446 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T09:58:18.447 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi196.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:18.530 DEBUG:teuthology.orchestra.run.smithi196:> true 2024-04-25T09:58:18.610 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi196.front.sepia.ceph.com' 2024-04-25T09:58:18.611 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:58:18.621 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:58:18.622 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:18.622 DEBUG:teuthology.orchestra.run.smithi082:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:18.624 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:18.625 DEBUG:teuthology.orchestra.run.smithi196:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:18.650 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T09:58:18.650 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Command line: ntpd -gq 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: ---------------------------------------------------- 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: corporation. Support and training for ntp-4 are 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: available at https://www.nwtime.org/support 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: ---------------------------------------------------- 2024-04-25T09:58:18.651 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: proto: precision = 0.045 usec (-24) 2024-04-25T09:58:18.652 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: basedate set to 2022-02-04 2024-04-25T09:58:18.652 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: gps base set to 2022-02-06 (week 2196) 2024-04-25T09:58:18.652 INFO:teuthology.orchestra.run.smithi082.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:18.652 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:18.653 INFO:teuthology.orchestra.run.smithi082.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:18.653 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:18.653 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen normally on 3 enp3s0f1 172.21.15.82:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen normally on 4 lo [::1]:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d3ff%5]:123 2024-04-25T09:58:18.654 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:18 ntpd[17199]: Listening on routing socket on fd #22 for interface updates 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Command line: ntpd -gq 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: ---------------------------------------------------- 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: corporation. Support and training for ntp-4 are 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: available at https://www.nwtime.org/support 2024-04-25T09:58:18.681 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: ---------------------------------------------------- 2024-04-25T09:58:18.682 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: proto: precision = 0.046 usec (-24) 2024-04-25T09:58:18.682 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: basedate set to 2022-02-04 2024-04-25T09:58:18.682 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: gps base set to 2022-02-06 (week 2196) 2024-04-25T09:58:18.682 INFO:teuthology.orchestra.run.smithi196.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:18.683 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T09:58:18.683 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:18.683 INFO:teuthology.orchestra.run.smithi196.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen normally on 3 enp3s0f1 172.21.15.196:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen normally on 4 lo [::1]:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f47%5]:123 2024-04-25T09:58:18.684 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:18 ntpd[17212]: Listening on routing socket on fd #22 for interface updates 2024-04-25T09:58:19.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:19.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:19.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:19.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:19.655 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:19.655 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:19 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:19.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:19.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:19.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:19.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:19.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:19.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:19 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:20.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:20 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:20.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:20 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:21.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:21.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:21 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:21.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:21.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:21.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:21.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:21.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:21.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:21.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:21.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:21 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:22 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:22.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:22.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:22.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:22.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:22 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:23.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:23.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:23 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:23.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:23.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:23 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:24.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:24.653 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:24 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:24.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:24 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stdout:25 Apr 09:58:25 ntpd[17199]: ntpd: time slew -0.002890 s 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stdout:ntpd: time slew -0.002890s 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T09:58:25.652 INFO:teuthology.orchestra.run.smithi082.stderr:25 Apr 09:58:25 ntpd[17199]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T09:58:25.682 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T09:58:25.683 INFO:teuthology.orchestra.run.smithi196.stdout:25 Apr 09:58:25 ntpd[17212]: ntpd: time slew -0.001708 s 2024-04-25T09:58:25.683 INFO:teuthology.orchestra.run.smithi196.stdout:ntpd: time slew -0.001708s 2024-04-25T09:58:25.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T09:58:25.683 INFO:teuthology.orchestra.run.smithi196.stderr:25 Apr 09:58:25 ntpd[17212]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================== 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.701 INFO:teuthology.orchestra.run.smithi082.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.733 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T09:58:25.734 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:58:25.742 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:58:25.742 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:58:25.742 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:58:25.743 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:58:25.748 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:58:25.748 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:58:25.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-key list | grep Ceph 2024-04-25T09:58:25.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-key list | grep Ceph 2024-04-25T09:58:25.809 INFO:teuthology.orchestra.run.smithi082.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T09:58:25.827 INFO:teuthology.orchestra.run.smithi196.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T09:58:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T09:58:25.840 INFO:teuthology.orchestra.run.smithi082.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T09:58:25.840 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T09:58:25.840 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T09:58:25.840 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T09:58:25.853 INFO:teuthology.orchestra.run.smithi196.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T09:58:25.853 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T09:58:25.853 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T09:58:25.854 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:25.994 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T09:58:25.994 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:25.997 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T09:58:25.997 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:26.135 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T09:58:26.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T09:58:26.140 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T09:58:26.140 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T09:58:26.149 DEBUG:teuthology.orchestra.run.smithi196:> sudo apt-get update 2024-04-25T09:58:26.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo apt-get update 2024-04-25T09:58:26.322 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:58:26.322 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:58:26.325 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T09:58:26.335 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:58:26.338 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T09:58:26.355 INFO:teuthology.orchestra.run.smithi196.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:58:26.358 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T09:58:26.369 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T09:58:26.390 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T09:58:26.395 INFO:teuthology.orchestra.run.smithi082.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T09:58:26.412 INFO:teuthology.orchestra.run.smithi196.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T09:58:26.421 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T09:58:26.433 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T09:58:26.444 INFO:teuthology.orchestra.run.smithi082.stdout:Hit:6 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T09:58:26.448 INFO:teuthology.orchestra.run.smithi082.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T09:58:26.460 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T09:58:26.475 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T09:58:26.506 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T09:58:26.665 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 28.9 kB in 0s (87.5 kB/s) 2024-04-25T09:58:26.674 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 28.9 kB in 0s (82.9 kB/s) 2024-04-25T09:58:27.601 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:58:27.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:27.635 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:58:27.652 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T09:58:27.679 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T09:58:27.715 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T09:58:27.879 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T09:58:27.880 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T09:58:27.914 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T09:58:27.915 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T09:58:28.075 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:58:28.075 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:58:28.075 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:58:28.075 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:58:28.076 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:58:28.076 INFO:teuthology.orchestra.run.smithi082.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:58:28.076 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout:The following additional packages will be installed: 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T09:58:28.078 INFO:teuthology.orchestra.run.smithi082.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T09:58:28.079 INFO:teuthology.orchestra.run.smithi082.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout:Suggested packages: 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T09:58:28.080 INFO:teuthology.orchestra.run.smithi082.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T09:58:28.081 INFO:teuthology.orchestra.run.smithi082.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T09:58:28.081 INFO:teuthology.orchestra.run.smithi082.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T09:58:28.081 INFO:teuthology.orchestra.run.smithi082.stdout:Recommended packages: 2024-04-25T09:58:28.081 INFO:teuthology.orchestra.run.smithi082.stdout: btrfs-tools 2024-04-25T09:58:28.108 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T09:58:28.108 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T09:58:28.108 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T09:58:28.108 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T09:58:28.109 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T09:58:28.111 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T09:58:28.112 INFO:teuthology.orchestra.run.smithi196.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout:Recommended packages: 2024-04-25T09:58:28.113 INFO:teuthology.orchestra.run.smithi196.stdout: btrfs-tools 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout:The following NEW packages will be installed: 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:28.185 INFO:teuthology.orchestra.run.smithi082.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T09:58:28.186 INFO:teuthology.orchestra.run.smithi082.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T09:58:28.187 INFO:teuthology.orchestra.run.smithi082.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T09:58:28.188 INFO:teuthology.orchestra.run.smithi082.stdout: socat unzip xmlstarlet zip 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T09:58:28.219 INFO:teuthology.orchestra.run.smithi196.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T09:58:28.220 INFO:teuthology.orchestra.run.smithi196.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T09:58:28.221 INFO:teuthology.orchestra.run.smithi196.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T09:58:28.222 INFO:teuthology.orchestra.run.smithi196.stdout: socat unzip xmlstarlet zip 2024-04-25T09:58:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T09:58:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:Need to get 176 MB of archives. 2024-04-25T09:58:28.340 INFO:teuthology.orchestra.run.smithi082.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T09:58:28.341 INFO:teuthology.orchestra.run.smithi082.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T09:58:28.374 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T09:58:28.375 INFO:teuthology.orchestra.run.smithi196.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T09:58:28.375 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 176 MB of archives. 2024-04-25T09:58:28.375 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T09:58:28.375 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T09:58:28.407 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T09:58:28.457 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T09:58:28.490 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T09:58:28.503 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T09:58:28.535 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T09:58:28.673 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T09:58:28.706 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T09:58:28.708 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T09:58:28.730 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T09:58:28.801 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T09:58:28.814 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T09:58:28.828 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T09:58:28.848 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T09:58:28.855 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T09:58:28.865 INFO:teuthology.orchestra.run.smithi082.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T09:58:28.897 INFO:teuthology.orchestra.run.smithi082.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T09:58:28.904 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T09:58:28.910 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T09:58:28.924 INFO:teuthology.orchestra.run.smithi082.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T09:58:28.926 INFO:teuthology.orchestra.run.smithi082.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T09:58:28.932 INFO:teuthology.orchestra.run.smithi082.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T09:58:28.935 INFO:teuthology.orchestra.run.smithi082.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T09:58:28.953 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T09:58:28.963 INFO:teuthology.orchestra.run.smithi082.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T09:58:28.985 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T09:58:28.986 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T09:58:28.992 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T09:58:29.014 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T09:58:29.027 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T09:58:29.040 INFO:teuthology.orchestra.run.smithi082.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T09:58:29.048 INFO:teuthology.orchestra.run.smithi082.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T09:58:29.076 INFO:teuthology.orchestra.run.smithi082.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T09:58:29.084 INFO:teuthology.orchestra.run.smithi082.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T09:58:29.183 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T09:58:29.197 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T09:58:29.229 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T09:58:29.242 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T09:58:29.251 INFO:teuthology.orchestra.run.smithi082.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T09:58:29.372 INFO:teuthology.orchestra.run.smithi082.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T09:58:29.377 INFO:teuthology.orchestra.run.smithi082.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T09:58:29.409 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T09:58:29.418 INFO:teuthology.orchestra.run.smithi082.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T09:58:29.467 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T09:58:29.469 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T09:58:29.519 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T09:58:29.822 INFO:teuthology.orchestra.run.smithi082.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T09:58:29.840 INFO:teuthology.orchestra.run.smithi082.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T09:58:29.858 INFO:teuthology.orchestra.run.smithi082.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T09:58:29.867 INFO:teuthology.orchestra.run.smithi082.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T09:58:29.867 INFO:teuthology.orchestra.run.smithi082.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T09:58:29.886 INFO:teuthology.orchestra.run.smithi082.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T09:58:29.916 INFO:teuthology.orchestra.run.smithi082.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T09:58:29.930 INFO:teuthology.orchestra.run.smithi082.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T09:58:29.934 INFO:teuthology.orchestra.run.smithi082.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T09:58:29.934 INFO:teuthology.orchestra.run.smithi082.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T09:58:29.984 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T09:58:29.999 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T09:58:30.015 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T09:58:30.024 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T09:58:30.024 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T09:58:30.041 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T09:58:30.079 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T09:58:30.092 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T09:58:30.096 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T09:58:30.096 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T09:58:30.104 INFO:teuthology.orchestra.run.smithi082.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T09:58:30.187 INFO:teuthology.orchestra.run.smithi082.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T09:58:30.204 INFO:teuthology.orchestra.run.smithi082.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T09:58:30.268 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T09:58:30.270 INFO:teuthology.orchestra.run.smithi082.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T09:58:30.299 INFO:teuthology.orchestra.run.smithi082.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T09:58:30.318 INFO:teuthology.orchestra.run.smithi082.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T09:58:30.343 INFO:teuthology.orchestra.run.smithi082.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T09:58:30.351 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T09:58:30.357 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T09:58:30.433 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T09:58:30.458 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T09:58:30.475 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T09:58:30.490 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T09:58:30.522 INFO:teuthology.orchestra.run.smithi082.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T09:58:30.538 INFO:teuthology.orchestra.run.smithi082.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T09:58:30.546 INFO:teuthology.orchestra.run.smithi082.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T09:58:30.550 INFO:teuthology.orchestra.run.smithi082.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T09:58:30.680 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T09:58:30.696 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T09:58:30.703 INFO:teuthology.orchestra.run.smithi196.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T09:58:30.708 INFO:teuthology.orchestra.run.smithi196.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T09:58:30.735 INFO:teuthology.orchestra.run.smithi082.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T09:58:30.818 INFO:teuthology.orchestra.run.smithi082.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T09:58:30.832 INFO:teuthology.orchestra.run.smithi082.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T09:58:30.892 INFO:teuthology.orchestra.run.smithi196.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T09:58:30.976 INFO:teuthology.orchestra.run.smithi196.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T09:58:30.992 INFO:teuthology.orchestra.run.smithi196.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T09:58:31.033 INFO:teuthology.orchestra.run.smithi082.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T09:58:31.035 INFO:teuthology.orchestra.run.smithi082.stdout:Get:50 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T09:58:31.055 INFO:teuthology.orchestra.run.smithi082.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T09:58:31.067 INFO:teuthology.orchestra.run.smithi082.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T09:58:31.211 INFO:teuthology.orchestra.run.smithi196.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T09:58:31.227 INFO:teuthology.orchestra.run.smithi082.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T09:58:31.229 INFO:teuthology.orchestra.run.smithi082.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T09:58:31.233 INFO:teuthology.orchestra.run.smithi082.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T09:58:31.238 INFO:teuthology.orchestra.run.smithi082.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T09:58:31.251 INFO:teuthology.orchestra.run.smithi082.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T09:58:31.263 INFO:teuthology.orchestra.run.smithi082.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T09:58:31.269 INFO:teuthology.orchestra.run.smithi196.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T09:58:31.269 INFO:teuthology.orchestra.run.smithi196.stdout:Get:51 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T09:58:31.270 INFO:teuthology.orchestra.run.smithi082.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T09:58:31.270 INFO:teuthology.orchestra.run.smithi196.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T09:58:31.273 INFO:teuthology.orchestra.run.smithi082.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T09:58:31.358 INFO:teuthology.orchestra.run.smithi082.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T09:58:31.401 INFO:teuthology.orchestra.run.smithi196.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T09:58:31.402 INFO:teuthology.orchestra.run.smithi196.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T09:58:31.407 INFO:teuthology.orchestra.run.smithi196.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T09:58:31.412 INFO:teuthology.orchestra.run.smithi196.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T09:58:31.425 INFO:teuthology.orchestra.run.smithi196.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T09:58:31.439 INFO:teuthology.orchestra.run.smithi082.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T09:58:31.522 INFO:teuthology.orchestra.run.smithi196.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T09:58:31.543 INFO:teuthology.orchestra.run.smithi196.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T09:58:31.547 INFO:teuthology.orchestra.run.smithi196.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T09:58:31.611 INFO:teuthology.orchestra.run.smithi196.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T09:58:31.657 INFO:teuthology.orchestra.run.smithi196.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T09:58:31.693 INFO:teuthology.orchestra.run.smithi082.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T09:58:31.758 INFO:teuthology.orchestra.run.smithi082.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T09:58:31.805 INFO:teuthology.orchestra.run.smithi082.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T09:58:31.848 INFO:teuthology.orchestra.run.smithi082.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T09:58:31.878 INFO:teuthology.orchestra.run.smithi196.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T09:58:31.945 INFO:teuthology.orchestra.run.smithi082.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T09:58:31.961 INFO:teuthology.orchestra.run.smithi082.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T09:58:31.977 INFO:teuthology.orchestra.run.smithi082.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T09:58:31.994 INFO:teuthology.orchestra.run.smithi196.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T09:58:32.033 INFO:teuthology.orchestra.run.smithi082.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T09:58:32.037 INFO:teuthology.orchestra.run.smithi196.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T09:58:32.068 INFO:teuthology.orchestra.run.smithi196.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T09:58:32.125 INFO:teuthology.orchestra.run.smithi196.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T09:58:32.135 INFO:teuthology.orchestra.run.smithi196.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T09:58:32.149 INFO:teuthology.orchestra.run.smithi196.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T09:58:32.192 INFO:teuthology.orchestra.run.smithi196.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T09:58:32.369 INFO:teuthology.orchestra.run.smithi082.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T09:58:32.371 INFO:teuthology.orchestra.run.smithi082.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T09:58:32.373 INFO:teuthology.orchestra.run.smithi082.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T09:58:32.457 INFO:teuthology.orchestra.run.smithi196.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T09:58:32.458 INFO:teuthology.orchestra.run.smithi196.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T09:58:32.461 INFO:teuthology.orchestra.run.smithi196.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T09:58:32.539 INFO:teuthology.orchestra.run.smithi082.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T09:58:32.627 INFO:teuthology.orchestra.run.smithi196.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T09:58:32.866 INFO:teuthology.orchestra.run.smithi082.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T09:58:32.953 INFO:teuthology.orchestra.run.smithi196.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T09:58:33.311 INFO:teuthology.orchestra.run.smithi082.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T09:58:33.311 INFO:teuthology.orchestra.run.smithi082.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T09:58:33.311 INFO:teuthology.orchestra.run.smithi082.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T09:58:33.316 INFO:teuthology.orchestra.run.smithi082.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T09:58:33.317 INFO:teuthology.orchestra.run.smithi082.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T09:58:33.329 INFO:teuthology.orchestra.run.smithi082.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T09:58:33.362 INFO:teuthology.orchestra.run.smithi196.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T09:58:33.368 INFO:teuthology.orchestra.run.smithi082.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T09:58:33.370 INFO:teuthology.orchestra.run.smithi082.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T09:58:33.372 INFO:teuthology.orchestra.run.smithi196.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T09:58:33.373 INFO:teuthology.orchestra.run.smithi196.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T09:58:33.393 INFO:teuthology.orchestra.run.smithi082.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T09:58:33.394 INFO:teuthology.orchestra.run.smithi082.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T09:58:33.396 INFO:teuthology.orchestra.run.smithi082.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T09:58:33.427 INFO:teuthology.orchestra.run.smithi082.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T09:58:33.442 INFO:teuthology.orchestra.run.smithi196.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T09:58:33.443 INFO:teuthology.orchestra.run.smithi196.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T09:58:33.445 INFO:teuthology.orchestra.run.smithi196.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T09:58:33.446 INFO:teuthology.orchestra.run.smithi196.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T09:58:33.447 INFO:teuthology.orchestra.run.smithi196.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T09:58:33.470 INFO:teuthology.orchestra.run.smithi082.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T09:58:33.474 INFO:teuthology.orchestra.run.smithi082.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T09:58:33.564 INFO:teuthology.orchestra.run.smithi082.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T09:58:33.605 INFO:teuthology.orchestra.run.smithi082.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T09:58:33.614 INFO:teuthology.orchestra.run.smithi196.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T09:58:33.658 INFO:teuthology.orchestra.run.smithi082.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T09:58:33.671 INFO:teuthology.orchestra.run.smithi196.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T09:58:33.672 INFO:teuthology.orchestra.run.smithi196.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T09:58:33.689 INFO:teuthology.orchestra.run.smithi196.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T09:58:33.700 INFO:teuthology.orchestra.run.smithi196.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T09:58:33.756 INFO:teuthology.orchestra.run.smithi082.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T09:58:33.796 INFO:teuthology.orchestra.run.smithi196.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T09:58:33.818 INFO:teuthology.orchestra.run.smithi196.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T09:58:33.880 INFO:teuthology.orchestra.run.smithi196.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T09:58:33.885 INFO:teuthology.orchestra.run.smithi196.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T09:58:33.995 INFO:teuthology.orchestra.run.smithi196.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T09:58:34.004 INFO:teuthology.orchestra.run.smithi082.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T09:58:34.038 INFO:teuthology.orchestra.run.smithi196.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T09:58:34.047 INFO:teuthology.orchestra.run.smithi082.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T09:58:34.071 INFO:teuthology.orchestra.run.smithi082.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T09:58:34.080 INFO:teuthology.orchestra.run.smithi082.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T09:58:34.082 INFO:teuthology.orchestra.run.smithi196.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T09:58:34.106 INFO:teuthology.orchestra.run.smithi196.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T09:58:34.114 INFO:teuthology.orchestra.run.smithi196.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T09:58:34.126 INFO:teuthology.orchestra.run.smithi082.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T09:58:34.152 INFO:teuthology.orchestra.run.smithi082.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T09:58:34.154 INFO:teuthology.orchestra.run.smithi082.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T09:58:34.159 INFO:teuthology.orchestra.run.smithi196.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T09:58:34.160 INFO:teuthology.orchestra.run.smithi082.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T09:58:34.173 INFO:teuthology.orchestra.run.smithi082.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T09:58:34.173 INFO:teuthology.orchestra.run.smithi082.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T09:58:34.184 INFO:teuthology.orchestra.run.smithi196.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T09:58:34.186 INFO:teuthology.orchestra.run.smithi196.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T09:58:34.193 INFO:teuthology.orchestra.run.smithi196.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T09:58:34.328 INFO:teuthology.orchestra.run.smithi082.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T09:58:34.359 INFO:teuthology.orchestra.run.smithi196.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T09:58:34.498 INFO:teuthology.orchestra.run.smithi196.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T09:58:34.499 INFO:teuthology.orchestra.run.smithi196.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T09:58:34.863 INFO:teuthology.orchestra.run.smithi082.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T09:58:34.872 INFO:teuthology.orchestra.run.smithi082.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T09:58:34.896 INFO:teuthology.orchestra.run.smithi196.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T09:58:34.902 INFO:teuthology.orchestra.run.smithi196.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T09:58:34.933 INFO:teuthology.orchestra.run.smithi082.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T09:58:34.936 INFO:teuthology.orchestra.run.smithi082.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T09:58:34.944 INFO:teuthology.orchestra.run.smithi082.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T09:58:34.948 INFO:teuthology.orchestra.run.smithi082.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T09:58:34.964 INFO:teuthology.orchestra.run.smithi196.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T09:58:34.965 INFO:teuthology.orchestra.run.smithi196.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T09:58:34.967 INFO:teuthology.orchestra.run.smithi082.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T09:58:34.971 INFO:teuthology.orchestra.run.smithi196.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T09:58:34.974 INFO:teuthology.orchestra.run.smithi196.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T09:58:34.977 INFO:teuthology.orchestra.run.smithi082.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T09:58:34.994 INFO:teuthology.orchestra.run.smithi196.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T09:58:35.003 INFO:teuthology.orchestra.run.smithi196.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T09:58:35.051 INFO:teuthology.orchestra.run.smithi082.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T09:58:35.052 INFO:teuthology.orchestra.run.smithi082.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T09:58:35.079 INFO:teuthology.orchestra.run.smithi196.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T09:58:35.081 INFO:teuthology.orchestra.run.smithi196.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T09:58:35.217 INFO:teuthology.orchestra.run.smithi082.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T09:58:35.245 INFO:teuthology.orchestra.run.smithi196.stdout:Get:115 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T09:58:35.650 INFO:teuthology.orchestra.run.smithi082.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T09:58:35.666 INFO:teuthology.orchestra.run.smithi082.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T09:58:35.686 INFO:teuthology.orchestra.run.smithi196.stdout:Get:116 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T09:58:35.703 INFO:teuthology.orchestra.run.smithi196.stdout:Get:117 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T09:58:37.874 INFO:teuthology.orchestra.run.smithi196.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T09:58:37.876 INFO:teuthology.orchestra.run.smithi196.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T09:58:37.881 INFO:teuthology.orchestra.run.smithi196.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T09:58:37.882 INFO:teuthology.orchestra.run.smithi196.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T09:58:37.883 INFO:teuthology.orchestra.run.smithi196.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T09:58:37.884 INFO:teuthology.orchestra.run.smithi196.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T09:58:37.914 INFO:teuthology.orchestra.run.smithi082.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T09:58:37.917 INFO:teuthology.orchestra.run.smithi082.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T09:58:37.921 INFO:teuthology.orchestra.run.smithi082.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T09:58:37.922 INFO:teuthology.orchestra.run.smithi082.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T09:58:37.923 INFO:teuthology.orchestra.run.smithi082.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T09:58:37.924 INFO:teuthology.orchestra.run.smithi082.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T09:58:38.560 INFO:teuthology.orchestra.run.smithi196.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T09:58:38.572 INFO:teuthology.orchestra.run.smithi082.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T09:58:39.168 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 176 MB in 10s (17.1 MB/s) 2024-04-25T09:58:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 176 MB in 10s (17.0 MB/s) 2024-04-25T09:58:39.286 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T09:58:39.335 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T09:58:39.350 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T09:58:39.355 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T09:58:39.394 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T09:58:39.432 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T09:58:39.437 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T09:58:39.475 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T09:58:39.663 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T09:58:39.689 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T09:58:39.730 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:58:39.769 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T09:58:39.795 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T09:58:39.836 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:58:40.066 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T09:58:40.093 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T09:58:40.133 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T09:58:40.156 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T09:58:40.182 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T09:58:40.222 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T09:58:40.486 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T09:58:40.511 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T09:58:40.594 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:58:40.775 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T09:58:40.801 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T09:58:40.867 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:58:40.972 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T09:58:40.999 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T09:58:41.048 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:58:41.204 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T09:58:41.229 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T09:58:41.270 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:58:41.350 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T09:58:41.377 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T09:58:41.417 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:58:41.556 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T09:58:41.585 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T09:58:41.615 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:58:41.870 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T09:58:41.896 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:41.945 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:42.043 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T09:58:42.069 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:42.118 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:42.324 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T09:58:42.350 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:42.390 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:42.505 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T09:58:42.531 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:42.571 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:42.727 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T09:58:42.753 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:42.793 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:42.866 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T09:58:42.892 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T09:58:42.932 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:58:43.154 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T09:58:43.180 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T09:58:43.221 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:58:43.277 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T09:58:43.303 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T09:58:43.343 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T09:58:43.549 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librados2. 2024-04-25T09:58:43.575 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:43.615 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:43.672 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librados2. 2024-04-25T09:58:43.698 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:43.738 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:44.027 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librbd1. 2024-04-25T09:58:44.053 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:44.093 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:44.157 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librbd1. 2024-04-25T09:58:44.172 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:44.207 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:44.546 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T09:58:44.572 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:44.612 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:44.669 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T09:58:44.695 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:44.735 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:44.915 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rados. 2024-04-25T09:58:44.941 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:44.981 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.012 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rados. 2024-04-25T09:58:45.038 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:45.079 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.284 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T09:58:45.310 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:45.351 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.399 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T09:58:45.425 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:45.465 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.586 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T09:58:45.601 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:45.644 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.718 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T09:58:45.733 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:45.767 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:45.964 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T09:58:45.990 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:46.031 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:46.187 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T09:58:46.213 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:58:46.253 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:46.359 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T09:58:46.386 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T09:58:46.425 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T09:58:46.574 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T09:58:46.600 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T09:58:46.640 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T09:58:46.686 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T09:58:46.701 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:46.744 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:46.901 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T09:58:46.927 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:46.967 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:47.287 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T09:58:47.313 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T09:58:47.345 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:58:47.606 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T09:58:47.632 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T09:58:47.673 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:58:47.698 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T09:58:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T09:58:47.765 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:58:47.976 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T09:58:48.002 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T09:58:48.042 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:58:48.050 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T09:58:48.065 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T09:58:48.108 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:58:48.371 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua5.1. 2024-04-25T09:58:48.436 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T09:58:48.437 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T09:58:48.437 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:58:48.446 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T09:58:48.486 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:58:48.732 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-any. 2024-04-25T09:58:48.758 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T09:58:48.798 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T09:58:48.814 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua5.1. 2024-04-25T09:58:48.840 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T09:58:48.889 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:58:49.051 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package zip. 2024-04-25T09:58:49.077 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T09:58:49.117 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T09:58:49.192 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-any. 2024-04-25T09:58:49.219 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T09:58:49.259 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T09:58:49.429 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package unzip. 2024-04-25T09:58:49.455 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T09:58:49.486 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package zip. 2024-04-25T09:58:49.502 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T09:58:49.504 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:58:49.545 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T09:58:49.823 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package unzip. 2024-04-25T09:58:49.838 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T09:58:49.841 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package luarocks. 2024-04-25T09:58:49.867 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T09:58:49.881 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:58:49.899 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:58:50.249 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package librgw2. 2024-04-25T09:58:50.275 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:50.293 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package luarocks. 2024-04-25T09:58:50.315 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:50.319 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T09:58:50.360 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T09:58:50.717 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package librgw2. 2024-04-25T09:58:50.744 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:50.784 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:50.859 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T09:58:50.885 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:50.926 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:51.261 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T09:58:51.284 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:51.319 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:51.395 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T09:58:51.422 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T09:58:51.462 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:58:51.664 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T09:58:51.690 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T09:58:51.730 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:58:51.731 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T09:58:51.746 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:51.781 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:52.042 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T09:58:52.068 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:52.108 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:52.201 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-common. 2024-04-25T09:58:52.228 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:52.267 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:52.453 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-common. 2024-04-25T09:58:52.480 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:52.511 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:53.548 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-base. 2024-04-25T09:58:53.574 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:53.624 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:53.792 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-base. 2024-04-25T09:58:53.818 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:58:53.867 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:58:54.160 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T09:58:54.187 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T09:58:54.227 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:58:54.429 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T09:58:54.455 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T09:58:54.487 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:58:54.496 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T09:58:54.523 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T09:58:54.563 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:58:54.824 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T09:58:54.850 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T09:58:54.874 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T09:58:54.890 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:58:54.901 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T09:58:54.941 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:58:55.218 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T09:58:55.244 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T09:58:55.244 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T09:58:55.270 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T09:58:55.284 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:58:55.310 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:58:55.587 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T09:58:55.588 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T09:58:55.613 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T09:58:55.615 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T09:58:55.653 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:58:55.655 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:58:55.942 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T09:58:55.958 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T09:58:55.968 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T09:58:55.984 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T09:58:56.008 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:58:56.024 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T09:58:56.294 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T09:58:56.321 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T09:58:56.327 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-portend. 2024-04-25T09:58:56.353 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T09:58:56.361 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T09:58:56.402 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T09:58:56.689 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-portend. 2024-04-25T09:58:56.696 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T09:58:56.715 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T09:58:56.723 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T09:58:56.755 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T09:58:56.756 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T09:58:57.025 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T09:58:57.051 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T09:58:57.058 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T09:58:57.083 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T09:58:57.084 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T09:58:57.124 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:58:57.394 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T09:58:57.420 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T09:58:57.444 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T09:58:57.460 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:58:57.471 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T09:58:57.511 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T09:58:57.814 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T09:58:57.822 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T09:58:57.841 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T09:58:57.849 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T09:58:57.880 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T09:58:57.889 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T09:58:58.192 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T09:58:58.192 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T09:58:58.218 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T09:58:58.218 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T09:58:58.258 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:58:58.259 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T09:58:58.536 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T09:58:58.551 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T09:58:58.561 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-mako. 2024-04-25T09:58:58.586 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:58:58.587 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T09:58:58.628 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:58:58.855 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-mako. 2024-04-25T09:58:58.882 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T09:58:58.922 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:58:58.923 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T09:58:58.949 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T09:58:58.981 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:58:59.242 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T09:58:59.250 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T09:58:59.268 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T09:58:59.277 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T09:58:59.317 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:58:59.334 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:58:59.628 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-webob. 2024-04-25T09:58:59.636 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T09:58:59.654 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T09:58:59.663 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T09:58:59.703 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:58:59.714 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:00.005 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webob. 2024-04-25T09:59:00.032 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T09:59:00.072 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:00.072 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T09:59:00.099 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T09:59:00.142 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:00.392 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T09:59:00.418 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T09:59:00.442 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T09:59:00.461 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:00.469 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T09:59:00.508 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:00.761 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T09:59:00.788 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T09:59:00.811 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-paste. 2024-04-25T09:59:00.827 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:00.838 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T09:59:00.894 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:01.130 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-paste. 2024-04-25T09:59:01.157 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T09:59:01.197 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:01.223 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T09:59:01.244 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T09:59:01.423 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:01.508 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T09:59:01.535 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T09:59:01.574 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:02.493 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T09:59:02.520 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T09:59:02.559 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:02.862 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T09:59:02.889 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T09:59:02.937 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T09:59:02.961 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T09:59:02.988 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T09:59:03.028 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:03.190 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T09:59:03.206 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T09:59:03.240 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:59:03.305 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T09:59:03.321 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T09:59:03.355 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T09:59:03.603 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T09:59:03.617 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T09:59:03.630 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T09:59:03.643 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T09:59:03.670 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:03.684 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T09:59:03.940 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-idna. 2024-04-25T09:59:03.966 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T09:59:03.997 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T09:59:04.038 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T09:59:04.065 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:04.310 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T09:59:04.335 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T09:59:04.375 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:04.399 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-idna. 2024-04-25T09:59:04.426 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T09:59:04.465 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T09:59:04.687 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-requests. 2024-04-25T09:59:04.714 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T09:59:04.753 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:04.777 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T09:59:04.803 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T09:59:04.843 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:05.091 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T09:59:05.118 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T09:59:05.163 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests. 2024-04-25T09:59:05.164 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:05.189 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T09:59:05.237 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:05.451 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T09:59:05.466 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:05.509 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:05.542 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T09:59:05.567 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T09:59:05.607 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:05.856 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T09:59:05.882 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:05.893 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T09:59:05.909 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:05.913 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:05.943 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:06.183 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T09:59:06.210 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T09:59:06.249 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:06.331 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T09:59:06.358 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:06.398 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:06.552 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T09:59:06.579 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T09:59:06.619 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T09:59:06.683 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T09:59:06.710 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T09:59:06.750 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:06.964 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T09:59:06.991 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:07.020 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T09:59:07.030 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.046 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T09:59:07.086 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T09:59:07.367 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T09:59:07.393 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:07.425 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.431 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T09:59:07.457 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:07.497 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:07.900 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T09:59:07.927 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:07.967 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:08.020 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T09:59:08.046 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:08.086 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:08.579 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T09:59:08.605 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:08.645 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:09.131 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph. 2024-04-25T09:59:09.158 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:09.214 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:09.458 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T09:59:09.486 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:09.516 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:09.690 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph. 2024-04-25T09:59:09.717 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:09.756 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:09.903 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T09:59:09.929 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:09.969 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:10.051 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T09:59:10.067 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:10.117 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:10.364 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephadm. 2024-04-25T09:59:10.391 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:10.420 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T09:59:10.430 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:10.447 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:10.478 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:10.700 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T09:59:10.723 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T09:59:10.758 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:10.865 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephadm. 2024-04-25T09:59:10.892 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:10.931 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:11.052 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:59:11.110 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:11.259 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T09:59:11.286 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T09:59:11.325 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:11.620 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T09:59:11.646 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:59:11.678 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:12.199 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-babel. 2024-04-25T09:59:12.226 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:59:12.265 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:12.569 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T09:59:12.595 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T09:59:12.635 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:12.716 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-babel. 2024-04-25T09:59:12.743 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T09:59:12.782 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:12.988 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T09:59:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:13.054 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:13.111 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T09:59:13.137 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T09:59:13.360 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:13.399 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T09:59:13.426 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T09:59:13.466 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:13.680 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T09:59:13.695 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:13.730 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:13.768 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-routes. 2024-04-25T09:59:13.795 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T09:59:13.835 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:14.033 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T09:59:14.060 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T09:59:14.091 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:14.155 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T09:59:14.182 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:14.221 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:14.394 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-routes. 2024-04-25T09:59:14.421 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T09:59:14.460 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:14.608 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T09:59:14.623 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T09:59:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:14.797 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T09:59:14.824 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:14.863 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:15.144 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T09:59:15.172 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T09:59:15.211 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:15.316 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T09:59:15.332 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T09:59:15.374 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:15.575 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T09:59:15.602 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T09:59:15.726 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:15.845 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T09:59:15.872 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T09:59:15.903 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:15.996 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T09:59:16.023 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T09:59:16.054 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:16.256 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T09:59:16.283 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T09:59:16.425 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:16.728 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T09:59:16.737 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T09:59:16.753 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:16.754 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T09:59:16.794 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:16.795 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:17.494 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T09:59:17.522 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:17.561 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:17.741 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T09:59:17.767 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T09:59:17.807 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:18.076 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T09:59:18.089 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T09:59:18.118 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:18.396 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T09:59:18.423 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T09:59:18.463 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:18.505 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T09:59:18.533 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T09:59:18.572 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:18.774 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T09:59:18.801 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T09:59:18.834 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T09:59:18.875 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T09:59:18.902 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T09:59:18.941 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:19.136 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T09:59:19.163 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T09:59:19.202 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T09:59:19.261 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T09:59:19.277 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T09:59:19.311 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:19.505 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T09:59:19.532 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T09:59:19.572 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:19.606 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T09:59:19.633 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T09:59:19.664 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T09:59:19.883 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T09:59:19.910 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T09:59:19.950 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T09:59:19.951 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T09:59:19.978 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T09:59:20.017 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T09:59:20.261 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T09:59:20.288 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T09:59:20.320 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T09:59:20.327 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:20.347 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T09:59:20.387 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:20.698 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T09:59:20.725 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T09:59:20.764 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T09:59:21.008 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T09:59:21.032 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:21.068 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T09:59:21.074 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:21.095 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T09:59:21.134 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T09:59:21.402 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T09:59:21.429 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T09:59:21.493 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:21.796 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T09:59:21.798 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T09:59:21.814 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:21.824 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:21.848 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:21.862 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:22.107 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package jq. 2024-04-25T09:59:22.134 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:22.142 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T09:59:22.169 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T09:59:22.173 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:22.209 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:22.587 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T09:59:22.614 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:22.651 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package socat. 2024-04-25T09:59:22.653 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:22.666 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T09:59:22.710 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:22.914 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package jq. 2024-04-25T09:59:22.930 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T09:59:22.972 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:23.016 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T09:59:23.043 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T09:59:23.137 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:23.191 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package socat. 2024-04-25T09:59:23.218 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T09:59:23.249 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:23.491 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-test. 2024-04-25T09:59:23.502 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T09:59:23.507 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:23.518 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T09:59:23.549 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:23.552 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:23.898 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-test. 2024-04-25T09:59:23.914 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:23.948 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:25.795 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T09:59:25.822 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:25.861 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:26.119 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T09:59:26.146 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:26.182 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T09:59:26.185 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:26.210 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:26.240 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:26.551 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T09:59:26.556 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T09:59:26.579 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T09:59:26.583 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:26.617 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T09:59:26.622 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:26.878 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T09:59:26.905 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:26.941 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T09:59:26.944 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:26.969 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T09:59:27.016 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T09:59:27.197 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T09:59:27.224 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:27.244 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T09:59:27.261 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:27.263 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:27.294 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:27.516 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T09:59:27.544 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:27.554 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T09:59:27.570 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T09:59:27.583 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:27.613 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:27.882 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T09:59:27.909 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:27.910 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T09:59:27.938 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T09:59:27.948 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:27.977 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:28.281 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T09:59:28.309 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T09:59:28.356 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:28.443 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T09:59:28.470 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T09:59:28.518 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:28.693 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T09:59:28.720 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T09:59:28.759 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:29.089 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T09:59:29.116 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T09:59:29.147 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package pkg-config. 2024-04-25T09:59:29.155 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:29.163 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T09:59:29.198 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:29.434 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T09:59:29.450 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T09:59:29.492 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:29.534 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T09:59:29.561 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T09:59:29.600 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:29.886 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T09:59:29.902 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T09:59:29.944 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:30.031 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pkg-config. 2024-04-25T09:59:30.059 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T09:59:30.097 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:30.247 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T09:59:30.275 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T09:59:30.313 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:30.349 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T09:59:30.365 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T09:59:30.399 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:30.625 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T09:59:30.653 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T09:59:30.692 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:30.836 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T09:59:30.863 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T09:59:30.902 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T09:59:31.031 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T09:59:31.069 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:31.205 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T09:59:31.232 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T09:59:31.272 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T09:59:31.355 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-py. 2024-04-25T09:59:31.371 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T09:59:31.405 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T09:59:31.616 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T09:59:31.645 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T09:59:31.683 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:31.683 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T09:59:31.700 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T09:59:31.733 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:31.961 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T09:59:31.989 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T09:59:32.019 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:32.187 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T09:59:32.215 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T09:59:32.253 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:32.372 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-py. 2024-04-25T09:59:32.400 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T09:59:32.447 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T09:59:32.539 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-toml. 2024-04-25T09:59:32.567 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T09:59:32.606 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T09:59:32.766 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T09:59:32.794 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T09:59:32.833 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:33.100 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T09:59:33.128 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T09:59:33.158 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:33.289 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T09:59:33.316 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T09:59:33.355 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:33.495 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T09:59:33.523 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T09:59:33.561 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:33.649 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-toml. 2024-04-25T09:59:33.678 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T09:59:33.707 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T09:59:33.856 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T09:59:33.883 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T09:59:33.922 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:34.010 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T09:59:34.038 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T09:59:34.077 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:34.455 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T09:59:34.472 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T09:59:34.513 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:34.643 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package radosgw. 2024-04-25T09:59:34.670 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:34.709 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:34.783 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T09:59:34.811 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T09:59:34.849 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:35.511 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T09:59:35.535 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:35.578 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:35.587 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package radosgw. 2024-04-25T09:59:35.615 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:35.653 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:35.866 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:36.116 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:59:36.242 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:36.456 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T09:59:36.484 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T09:59:36.504 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T09:59:36.522 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:36.845 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T09:59:36.860 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T09:59:37.111 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T09:59:37.212 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T09:59:37.290 INFO:teuthology.orchestra.run.smithi082.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T09:59:37.369 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:37.471 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T09:59:37.826 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T09:59:38.194 INFO:teuthology.orchestra.run.smithi082.stdout:Adding system user cephadm....done 2024-04-25T09:59:38.287 INFO:teuthology.orchestra.run.smithi196.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T09:59:38.351 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:38.355 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:38.635 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:59:38.879 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user cephadm....done 2024-04-25T09:59:38.886 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:38.994 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T09:59:39.004 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:59:39.245 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T09:59:39.255 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:39.480 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T09:59:39.506 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:59:39.606 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T09:59:39.632 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T09:59:39.857 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T09:59:39.959 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:40.116 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T09:59:40.234 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T09:59:40.327 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:40.519 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T09:59:40.604 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T09:59:40.838 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T09:59:40.871 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T09:59:41.098 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:59:41.122 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T09:59:41.341 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:41.390 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T09:59:41.609 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:41.666 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T09:59:41.918 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T09:59:41.935 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:42.160 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T09:59:42.219 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:42.403 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T09:59:42.470 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:42.571 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:42.689 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T09:59:42.712 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:42.931 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:59:42.972 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T09:59:43.057 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:59:43.098 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T09:59:43.224 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T09:59:43.483 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T09:59:43.508 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:59:43.609 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T09:59:43.860 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T09:59:43.869 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T09:59:43.897 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T09:59:43.961 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:59:44.087 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:44.221 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T09:59:44.257 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T09:59:44.322 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T09:59:44.438 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:44.556 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T09:59:44.747 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T09:59:44.823 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:45.083 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:45.115 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:45.241 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T09:59:45.384 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T09:59:45.500 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T09:59:45.510 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:45.752 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T09:59:46.036 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T09:59:46.162 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T09:59:46.262 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:46.496 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:46.597 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:46.723 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:59:46.839 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T09:59:47.024 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:47.107 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T09:59:47.233 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T09:59:47.342 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T09:59:47.425 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:47.618 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T09:59:47.676 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:47.970 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T09:59:48.077 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:59:48.229 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T09:59:48.362 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T09:59:48.606 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T09:59:48.663 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:48.882 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T09:59:49.014 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:59:49.183 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T09:59:49.282 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:49.408 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:49.517 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T09:59:49.709 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T09:59:49.744 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T09:59:49.853 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T09:59:50.035 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:50.137 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T09:59:50.455 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T09:59:51.216 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:51.467 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:51.553 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T09:59:51.601 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T09:59:51.821 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:51.869 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:59:51.955 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T09:59:51.978 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:52.113 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:52.223 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T09:59:52.349 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T09:59:52.397 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:52.466 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T09:59:52.648 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T09:59:52.751 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T09:59:52.899 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:52.994 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T09:59:53.025 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:53.237 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:53.352 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:59:53.362 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T09:59:53.611 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T09:59:53.655 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T09:59:53.862 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T09:59:53.907 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T09:59:54.113 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:59:54.166 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T09:59:54.372 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T09:59:54.392 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T09:59:54.640 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:59:54.660 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T09:59:54.766 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T09:59:54.913 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T09:59:55.037 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T09:59:55.067 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:55.176 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:59:55.313 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T09:59:55.439 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T09:59:55.461 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:55.734 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T09:59:55.770 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:59:56.030 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:59:56.050 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T09:59:56.147 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:56.273 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:56.284 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T09:59:56.361 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T09:59:56.385 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T09:59:56.486 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T09:59:56.574 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T09:59:56.600 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:56.725 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:56.813 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T09:59:56.939 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T09:59:57.110 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:57.323 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T09:59:57.361 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T09:59:57.462 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T09:59:57.574 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T09:59:57.675 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T09:59:57.705 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:57.831 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:57.902 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T09:59:58.003 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T09:59:58.149 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:58.329 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T09:59:58.449 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:58.630 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T09:59:58.726 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:58.914 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T09:59:58.937 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T09:59:59.126 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T09:59:59.196 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T09:59:59.369 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T09:59:59.473 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:59.620 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T09:59:59.732 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T09:59:59.871 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T10:00:00.100 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:00:00.250 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T10:00:00.369 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:00:00.532 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T10:00:00.696 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:00:00.822 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:00.859 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T10:00:00.948 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:00.977 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.074 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:00:01.128 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.279 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T10:00:01.360 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.477 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.612 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:00:01.649 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.775 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.890 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:01.900 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T10:00:02.014 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.165 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.177 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.303 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.377 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:00:02.378 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:00:02.429 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.642 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:00:02.643 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T10:00:02.875 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:02.976 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.077 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.155 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.194 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.281 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.322 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:00:03.407 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.533 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:03.659 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T10:00:04.509 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:04.619 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:04.745 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:04.870 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:04.879 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:04.997 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.005 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.123 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.148 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.274 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.400 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.509 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:05.631 INFO:teuthology.orchestra.run.smithi082.stdout:Adding group ceph....done 2024-04-25T10:00:05.942 INFO:teuthology.orchestra.run.smithi082.stdout:Adding system user ceph....done 2024-04-25T10:00:06.027 INFO:teuthology.orchestra.run.smithi196.stdout:Adding group ceph....done 2024-04-25T10:00:06.077 INFO:teuthology.orchestra.run.smithi082.stdout:Setting system user ceph properties....done 2024-04-25T10:00:06.098 INFO:teuthology.orchestra.run.smithi082.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:00:06.234 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:00:06.403 INFO:teuthology.orchestra.run.smithi196.stdout:Adding system user ceph....done 2024-04-25T10:00:06.574 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:00:06.580 INFO:teuthology.orchestra.run.smithi196.stdout:Setting system user ceph properties....done 2024-04-25T10:00:06.595 INFO:teuthology.orchestra.run.smithi196.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T10:00:06.730 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T10:00:07.057 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T10:00:07.091 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:07.289 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:07.289 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:07.560 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:07.758 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:07.789 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:07.789 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:07.893 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:08.303 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:08.348 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:08.349 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:08.471 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:08.836 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:08.944 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:08.944 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:09.181 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:00:09.431 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:09.672 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:09.780 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T10:00:09.905 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:09.905 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:10.266 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:10.398 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:10.523 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:10.616 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:10.616 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:11.009 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:11.066 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:11.206 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:11.206 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:11.338 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:11.338 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:11.729 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:11.768 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:11.894 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.045 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.115 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:12.115 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:12.258 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:12.258 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:12.621 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.738 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.747 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.889 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:12.898 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:13.015 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:13.113 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:13.113 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:13.166 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:13.511 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:00:13.600 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:13.619 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:00:13.759 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:13.763 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:00:13.860 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:14.011 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T10:00:14.399 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:00:14.508 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:00:14.660 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:00:14.882 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:00:15.521 INFO:teuthology.orchestra.run.smithi082.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:00:15.524 DEBUG:teuthology.orchestra.run.smithi082:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:00:15.585 INFO:teuthology.orchestra.run.smithi082.stdout:Reading package lists... 2024-04-25T10:00:15.787 INFO:teuthology.orchestra.run.smithi082.stdout:Building dependency tree... 2024-04-25T10:00:15.788 INFO:teuthology.orchestra.run.smithi082.stdout:Reading state information... 2024-04-25T10:00:15.851 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:00:15.983 INFO:teuthology.orchestra.run.smithi082.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:00:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:00:15.984 INFO:teuthology.orchestra.run.smithi082.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout:The following additional packages will be installed: 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:00:15.985 INFO:teuthology.orchestra.run.smithi082.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:15.986 INFO:teuthology.orchestra.run.smithi082.stdout:Suggested packages: 2024-04-25T10:00:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:00:15.986 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:00:15.987 INFO:teuthology.orchestra.run.smithi082.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:00:16.026 INFO:teuthology.orchestra.run.smithi082.stdout:The following NEW packages will be installed: 2024-04-25T10:00:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:00:16.027 INFO:teuthology.orchestra.run.smithi082.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:00:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:00:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:00:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:00:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:00:16.028 INFO:teuthology.orchestra.run.smithi082.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:16.029 INFO:teuthology.orchestra.run.smithi082.stdout:The following packages will be upgraded: 2024-04-25T10:00:16.029 INFO:teuthology.orchestra.run.smithi082.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:00:16.211 INFO:teuthology.orchestra.run.smithi082.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T10:00:16.211 INFO:teuthology.orchestra.run.smithi082.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:00:16.211 INFO:teuthology.orchestra.run.smithi082.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:00:16.211 INFO:teuthology.orchestra.run.smithi082.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:00:16.504 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:00:16.508 DEBUG:teuthology.orchestra.run.smithi196:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T10:00:16.563 INFO:teuthology.orchestra.run.smithi196.stdout:Reading package lists... 2024-04-25T10:00:16.764 INFO:teuthology.orchestra.run.smithi196.stdout:Building dependency tree... 2024-04-25T10:00:16.764 INFO:teuthology.orchestra.run.smithi196.stdout:Reading state information... 2024-04-25T10:00:16.839 INFO:teuthology.orchestra.run.smithi082.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:00:16.858 INFO:teuthology.orchestra.run.smithi082.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T10:00:16.956 INFO:teuthology.orchestra.run.smithi196.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T10:00:16.957 INFO:teuthology.orchestra.run.smithi196.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T10:00:16.957 INFO:teuthology.orchestra.run.smithi196.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T10:00:16.957 INFO:teuthology.orchestra.run.smithi196.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout:The following additional packages will be installed: 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T10:00:16.958 INFO:teuthology.orchestra.run.smithi196.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:16.959 INFO:teuthology.orchestra.run.smithi196.stdout:Suggested packages: 2024-04-25T10:00:16.959 INFO:teuthology.orchestra.run.smithi196.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T10:00:16.959 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T10:00:16.959 INFO:teuthology.orchestra.run.smithi196.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T10:00:16.981 INFO:teuthology.orchestra.run.smithi082.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:00:16.982 INFO:teuthology.orchestra.run.smithi082.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:00:16.984 INFO:teuthology.orchestra.run.smithi082.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:00:17.000 INFO:teuthology.orchestra.run.smithi196.stdout:The following NEW packages will be installed: 2024-04-25T10:00:17.000 INFO:teuthology.orchestra.run.smithi196.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T10:00:17.000 INFO:teuthology.orchestra.run.smithi196.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T10:00:17.001 INFO:teuthology.orchestra.run.smithi196.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T10:00:17.001 INFO:teuthology.orchestra.run.smithi196.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T10:00:17.001 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T10:00:17.001 INFO:teuthology.orchestra.run.smithi196.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T10:00:17.001 INFO:teuthology.orchestra.run.smithi196.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T10:00:17.002 INFO:teuthology.orchestra.run.smithi196.stdout:The following packages will be upgraded: 2024-04-25T10:00:17.003 INFO:teuthology.orchestra.run.smithi196.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T10:00:17.012 INFO:teuthology.orchestra.run.smithi082.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:00:17.014 INFO:teuthology.orchestra.run.smithi082.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:00:17.018 INFO:teuthology.orchestra.run.smithi082.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:00:17.021 INFO:teuthology.orchestra.run.smithi082.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:00:17.022 INFO:teuthology.orchestra.run.smithi082.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:00:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T10:00:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:Need to get 55.7 MB of archives. 2024-04-25T10:00:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T10:00:17.061 INFO:teuthology.orchestra.run.smithi196.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T10:00:17.188 INFO:teuthology.orchestra.run.smithi082.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:00:17.217 INFO:teuthology.orchestra.run.smithi196.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T10:00:17.224 INFO:teuthology.orchestra.run.smithi196.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T10:00:17.264 INFO:teuthology.orchestra.run.smithi196.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T10:00:17.266 INFO:teuthology.orchestra.run.smithi196.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T10:00:17.266 INFO:teuthology.orchestra.run.smithi196.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T10:00:17.277 INFO:teuthology.orchestra.run.smithi196.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T10:00:17.277 INFO:teuthology.orchestra.run.smithi196.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T10:00:17.280 INFO:teuthology.orchestra.run.smithi196.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T10:00:17.281 INFO:teuthology.orchestra.run.smithi196.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T10:00:17.282 INFO:teuthology.orchestra.run.smithi196.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T10:00:17.283 INFO:teuthology.orchestra.run.smithi196.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T10:00:17.286 INFO:teuthology.orchestra.run.smithi196.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:00:17.304 INFO:teuthology.orchestra.run.smithi196.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:00:17.304 INFO:teuthology.orchestra.run.smithi196.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:00:17.305 INFO:teuthology.orchestra.run.smithi196.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:00:17.481 INFO:teuthology.orchestra.run.smithi082.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T10:00:17.490 INFO:teuthology.orchestra.run.smithi082.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T10:00:17.494 INFO:teuthology.orchestra.run.smithi082.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T10:00:17.580 INFO:teuthology.orchestra.run.smithi082.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T10:00:17.629 INFO:teuthology.orchestra.run.smithi196.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:00:17.631 INFO:teuthology.orchestra.run.smithi196.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:00:17.631 INFO:teuthology.orchestra.run.smithi196.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:00:17.634 INFO:teuthology.orchestra.run.smithi196.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:00:17.634 INFO:teuthology.orchestra.run.smithi196.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:00:17.639 INFO:teuthology.orchestra.run.smithi196.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:00:17.664 INFO:teuthology.orchestra.run.smithi196.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:00:17.671 INFO:teuthology.orchestra.run.smithi196.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:00:17.671 INFO:teuthology.orchestra.run.smithi196.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:00:17.694 INFO:teuthology.orchestra.run.smithi196.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:00:17.694 INFO:teuthology.orchestra.run.smithi196.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:00:17.704 INFO:teuthology.orchestra.run.smithi196.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:00:17.724 INFO:teuthology.orchestra.run.smithi196.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:00:17.727 INFO:teuthology.orchestra.run.smithi196.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:00:17.727 INFO:teuthology.orchestra.run.smithi196.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:00:17.728 INFO:teuthology.orchestra.run.smithi196.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:00:17.728 INFO:teuthology.orchestra.run.smithi196.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:00:17.745 INFO:teuthology.orchestra.run.smithi196.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:00:17.749 INFO:teuthology.orchestra.run.smithi196.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:00:18.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:00:18.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:00:18.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:00:18.076 INFO:teuthology.orchestra.run.smithi196.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:00:18.077 INFO:teuthology.orchestra.run.smithi196.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:00:18.077 INFO:teuthology.orchestra.run.smithi196.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:00:18.092 INFO:teuthology.orchestra.run.smithi196.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:00:18.093 INFO:teuthology.orchestra.run.smithi196.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:00:18.495 INFO:teuthology.orchestra.run.smithi196.stdout:Preconfiguring packages ... 2024-04-25T10:00:18.651 INFO:teuthology.orchestra.run.smithi196.stdout:Fetched 55.7 MB in 1s (51.5 MB/s) 2024-04-25T10:00:18.673 INFO:teuthology.orchestra.run.smithi082.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T10:00:18.677 INFO:teuthology.orchestra.run.smithi082.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T10:00:18.677 INFO:teuthology.orchestra.run.smithi082.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T10:00:18.684 INFO:teuthology.orchestra.run.smithi082.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T10:00:18.684 INFO:teuthology.orchestra.run.smithi082.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T10:00:18.694 INFO:teuthology.orchestra.run.smithi082.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T10:00:18.864 INFO:teuthology.orchestra.run.smithi082.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T10:00:18.972 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:18.977 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:19.123 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:19.457 INFO:teuthology.orchestra.run.smithi082.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T10:00:19.459 INFO:teuthology.orchestra.run.smithi082.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T10:00:19.639 INFO:teuthology.orchestra.run.smithi082.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T10:00:19.639 INFO:teuthology.orchestra.run.smithi082.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T10:00:19.678 INFO:teuthology.orchestra.run.smithi082.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T10:00:19.736 INFO:teuthology.orchestra.run.smithi082.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T10:00:19.743 INFO:teuthology.orchestra.run.smithi082.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T10:00:19.744 INFO:teuthology.orchestra.run.smithi082.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T10:00:19.745 INFO:teuthology.orchestra.run.smithi082.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T10:00:19.746 INFO:teuthology.orchestra.run.smithi082.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T10:00:19.761 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:19.955 INFO:teuthology.orchestra.run.smithi082.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T10:00:20.357 INFO:teuthology.orchestra.run.smithi082.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T10:00:20.745 INFO:teuthology.orchestra.run.smithi196.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:00:21.045 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package flex. 2024-04-25T10:00:21.120 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:21.126 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:21.160 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:00:21.205 INFO:teuthology.orchestra.run.smithi082.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T10:00:21.206 INFO:teuthology.orchestra.run.smithi082.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T10:00:21.206 INFO:teuthology.orchestra.run.smithi082.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T10:00:21.206 INFO:teuthology.orchestra.run.smithi082.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T10:00:21.207 INFO:teuthology.orchestra.run.smithi082.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T10:00:21.207 INFO:teuthology.orchestra.run.smithi082.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T10:00:21.234 INFO:teuthology.orchestra.run.smithi082.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T10:00:21.235 INFO:teuthology.orchestra.run.smithi082.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T10:00:21.618 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:21.641 INFO:teuthology.orchestra.run.smithi082.stdout:Preconfiguring packages ... 2024-04-25T10:00:21.775 INFO:teuthology.orchestra.run.smithi082.stdout:Fetched 55.7 MB in 5s (10.7 MB/s) 2024-04-25T10:00:21.782 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:00:22.094 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:22.100 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:22.204 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:22.256 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:22.759 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:22.764 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:22.859 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:22.892 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:23.284 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:23.390 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:23.700 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:23.845 INFO:teuthology.orchestra.run.smithi082.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T10:00:24.109 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package flex. 2024-04-25T10:00:24.115 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package bison. 2024-04-25T10:00:24.204 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T10:00:24.210 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:24.221 INFO:teuthology.orchestra.run.smithi196.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:24.227 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:24.276 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:24.284 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T10:00:24.677 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:00:24.704 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:00:24.743 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:24.800 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:24.973 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T10:00:25.063 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package dump. 2024-04-25T10:00:25.090 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:00:25.129 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:00:25.376 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:25.425 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package indent. 2024-04-25T10:00:25.440 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:25.483 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:25.744 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:00:25.771 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:00:25.810 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:25.894 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:25.899 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:26.031 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:26.232 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:00:26.260 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:00:26.299 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:26.432 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T10:00:26.551 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:00:26.571 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T10:00:26.579 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:00:26.618 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:26.898 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:26.930 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:00:26.957 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:26.996 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:27.216 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:00:27.243 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:27.282 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:27.305 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package bison. 2024-04-25T10:00:27.409 INFO:teuthology.orchestra.run.smithi082.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T10:00:27.414 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:27.466 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:27.535 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:00:27.551 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:00:27.593 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:00:27.833 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T10:00:27.861 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T10:00:27.900 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:27.904 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:00:27.932 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:00:27.971 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:28.170 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package dump. 2024-04-25T10:00:28.186 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T10:00:28.219 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T10:00:28.482 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package indent. 2024-04-25T10:00:28.510 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:28.548 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:28.801 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T10:00:28.817 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T10:00:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:29.147 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T10:00:29.174 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T10:00:29.205 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:29.500 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T10:00:29.517 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T10:00:29.558 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:29.616 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:00:29.631 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:29.665 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:29.895 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T10:00:29.923 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:29.943 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:00:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:29.970 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:30.010 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:30.181 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T10:00:30.197 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T10:00:30.231 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:30.338 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:00:30.365 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:30.404 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:30.484 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T10:00:30.500 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T10:00:30.542 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T10:00:30.774 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:00:30.801 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:00:30.845 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T10:00:30.849 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:30.873 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T10:00:30.912 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:31.320 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:00:31.347 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:31.386 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:31.764 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:00:31.792 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:31.831 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:32.125 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:00:32.153 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:32.192 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:32.495 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:00:32.511 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:32.553 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:32.632 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T10:00:32.648 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:32.681 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:32.797 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:00:32.813 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:32.855 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:32.926 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T10:00:32.942 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T10:00:32.984 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:33.312 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T10:00:33.340 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:33.387 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:33.443 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:00:33.470 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:00:33.517 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:33.699 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T10:00:33.714 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T10:00:33.748 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:33.887 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:00:33.915 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:33.954 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:34.010 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T10:00:34.037 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:34.076 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:34.556 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T10:00:34.583 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:34.585 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:00:34.613 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:34.615 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:34.660 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:34.918 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T10:00:34.946 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:34.985 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:35.293 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package ppp. 2024-04-25T10:00:35.320 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:00:35.321 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T10:00:35.337 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:35.380 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:35.596 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:35.615 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T10:00:35.631 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T10:00:35.665 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:35.914 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:00:35.942 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:00:35.981 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:36.144 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T10:00:36.172 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T10:00:36.211 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:36.418 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:00:36.446 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:00:36.485 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:36.564 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package modemmanager. 2024-04-25T10:00:36.591 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T10:00:36.630 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:36.746 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:00:36.774 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:00:36.813 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:00:37.102 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:00:37.127 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:37.166 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:37.254 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package network-manager. 2024-04-25T10:00:37.281 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T10:00:37.337 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:37.755 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:00:37.783 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:00:37.840 INFO:teuthology.orchestra.run.smithi196.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:00:37.869 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:00:37.945 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package ppp. 2024-04-25T10:00:37.972 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T10:00:38.242 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:38.278 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:00:38.306 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:38.340 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:38.633 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T10:00:38.649 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T10:00:38.691 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:38.935 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T10:00:38.952 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T10:00:38.994 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:39.272 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T10:00:39.299 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T10:00:39.338 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T10:00:39.642 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T10:00:39.658 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:39.700 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:39.814 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:00:39.831 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:00:39.873 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:00:40.116 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:00:40.144 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:00:40.183 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:00:40.289 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T10:00:40.317 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T10:00:40.366 INFO:teuthology.orchestra.run.smithi082.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T10:00:40.386 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-common (238) ... 2024-04-25T10:00:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:00:40.451 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:00:40.493 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:00:40.812 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:00:40.838 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T10:00:40.841 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:40.865 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T10:00:40.879 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:40.916 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:41.157 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package tcl. 2024-04-25T10:00:41.185 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:00:41.228 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:00:41.509 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:00:41.537 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:00:41.567 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:41.947 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:00:41.975 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:00:42.013 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:42.207 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql. 2024-04-25T10:00:42.224 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T10:00:42.265 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql (14+238) ... 2024-04-25T10:00:42.400 INFO:teuthology.orchestra.run.smithi196.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:00:42.428 INFO:teuthology.orchestra.run.smithi196.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:00:42.474 INFO:teuthology.orchestra.run.smithi196.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:42.510 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T10:00:42.537 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T10:00:42.576 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T10:00:42.754 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:00:42.828 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T10:00:42.856 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T10:00:42.895 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T10:00:42.964 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:00:43.090 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:43.181 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T10:00:43.209 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T10:00:43.216 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:43.248 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:43.350 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:43.501 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:43.559 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package tcl. 2024-04-25T10:00:43.587 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T10:00:43.630 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T10:00:43.652 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:43.978 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T10:00:44.006 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T10:00:44.045 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:44.142 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:00:44.241 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:44.366 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T10:00:44.394 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T10:00:44.432 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:44.670 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:44.670 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:44.811 INFO:teuthology.orchestra.run.smithi082.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T10:00:44.839 INFO:teuthology.orchestra.run.smithi082.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T10:00:44.885 INFO:teuthology.orchestra.run.smithi082.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:45.165 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T10:00:45.177 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:45.324 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:45.375 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T10:00:45.463 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:45.484 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T10:00:45.602 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:45.614 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:45.728 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T10:00:45.765 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:45.856 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T10:00:45.916 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:45.982 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:46.058 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:46.184 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:46.461 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T10:00:46.537 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T10:00:46.917 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:46.917 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T10:00:47.382 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T10:00:47.516 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T10:00:47.634 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T10:00:47.640 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:47.783 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:00:47.810 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T10:00:47.894 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:00:47.968 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:48.119 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:48.270 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:00:48.421 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:00:48.547 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:48.638 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:00:48.699 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:48.737 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T10:00:48.833 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:48.888 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:49.014 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T10:00:49.043 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:49.140 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T10:00:49.269 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:49.420 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:49.605 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:00:49.731 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:49.770 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T10:00:49.896 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T10:00:49.990 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T10:00:50.040 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:50.157 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout:- enp3s0f1 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:00:50.227 INFO:teuthology.orchestra.run.smithi196.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:00:50.228 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:50.283 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T10:00:50.339 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:00:50.409 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T10:00:50.535 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T10:00:50.625 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T10:00:50.645 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:00:50.687 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T10:00:50.813 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T10:00:50.981 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T10:00:50.986 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:00:51.082 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:51.208 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T10:00:51.376 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T10:00:51.500 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:51.502 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T10:00:51.660 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:51.786 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:51.912 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout:which means they are currently configured by ifupdown: 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout:- enp3s0f1 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T10:00:51.996 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:52.124 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T10:00:52.451 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T10:00:52.778 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T10:00:52.779 INFO:teuthology.orchestra.run.smithi196.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:00:53.000 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:53.001 INFO:teuthology.orchestra.run.smithi196.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:00:53.189 INFO:teuthology.orchestra.run.smithi196.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:00:53.189 INFO:teuthology.orchestra.run.smithi196.stdout:Removing obsolete dictionary files: 2024-04-25T10:00:53.262 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T10:00:53.398 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T10:00:53.524 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T10:00:53.703 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-common (238) ... 2024-04-25T10:00:53.754 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:00:54.155 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:54.477 INFO:teuthology.orchestra.run.smithi196.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:00:54.493 INFO:teuthology.orchestra.run.smithi196.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:00:54.532 INFO:teuthology.orchestra.run.smithi196.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:00:54.532 INFO:teuthology.orchestra.run.smithi196.stdout:This user must also own the server process. 2024-04-25T10:00:54.532 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:The default text search configuration will be set to "english". 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:Data page checksums are disabled. 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:creating subdirectories ... ok 2024-04-25T10:00:54.533 INFO:teuthology.orchestra.run.smithi196.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:00:54.548 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default max_connections ... 100 2024-04-25T10:00:54.568 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:00:54.578 INFO:teuthology.orchestra.run.smithi196.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:00:54.578 INFO:teuthology.orchestra.run.smithi196.stdout:creating configuration files ... ok 2024-04-25T10:00:54.687 INFO:teuthology.orchestra.run.smithi082.stdout:Adding user postgres to group ssl-cert 2024-04-25T10:00:54.693 INFO:teuthology.orchestra.run.smithi196.stdout:running bootstrap script ... ok 2024-04-25T10:00:54.925 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:54.926 INFO:teuthology.orchestra.run.smithi082.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T10:00:55.113 INFO:teuthology.orchestra.run.smithi082.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T10:00:55.113 INFO:teuthology.orchestra.run.smithi082.stdout:Removing obsolete dictionary files: 2024-04-25T10:00:55.308 INFO:teuthology.orchestra.run.smithi196.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:00:55.633 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T10:00:55.975 INFO:teuthology.orchestra.run.smithi196.stdout:syncing data to disk ... ok 2024-04-25T10:00:56.034 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T10:00:56.360 INFO:teuthology.orchestra.run.smithi082.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T10:00:56.369 INFO:teuthology.orchestra.run.smithi196.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:00:56.378 INFO:teuthology.orchestra.run.smithi082.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T10:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T10:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:This user must also own the server process. 2024-04-25T10:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T10:00:56.421 INFO:teuthology.orchestra.run.smithi082.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:The default text search configuration will be set to "english". 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:Data page checksums are disabled. 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T10:00:56.422 INFO:teuthology.orchestra.run.smithi082.stdout:creating subdirectories ... ok 2024-04-25T10:00:56.423 INFO:teuthology.orchestra.run.smithi082.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T10:00:56.436 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default max_connections ... 100 2024-04-25T10:00:56.455 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default shared_buffers ... 128MB 2024-04-25T10:00:56.464 INFO:teuthology.orchestra.run.smithi082.stdout:selecting default time zone ... Etc/UTC 2024-04-25T10:00:56.465 INFO:teuthology.orchestra.run.smithi082.stdout:creating configuration files ... ok 2024-04-25T10:00:56.579 INFO:teuthology.orchestra.run.smithi082.stdout:running bootstrap script ... ok 2024-04-25T10:00:57.193 INFO:teuthology.orchestra.run.smithi082.stdout:performing post-bootstrap initialization ... ok 2024-04-25T10:00:57.796 INFO:teuthology.orchestra.run.smithi082.stdout:syncing data to disk ... ok 2024-04-25T10:00:58.202 INFO:teuthology.orchestra.run.smithi082.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T10:00:59.178 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:59.312 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:00:59.438 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:00:59.564 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:00:59.690 INFO:teuthology.orchestra.run.smithi196.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:00:59.925 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:00:59.925 INFO:teuthology.orchestra.run.smithi196.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:01:00.467 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:01:00.560 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:01:00.879 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:01:00.949 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:01:01.048 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:01:01.075 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T10:01:01.201 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up postgresql (14+238) ... 2024-04-25T10:01:01.327 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T10:01:01.453 INFO:teuthology.orchestra.run.smithi082.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T10:01:01.704 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:01:01.705 INFO:teuthology.orchestra.run.smithi082.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T10:01:02.264 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T10:01:02.349 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T10:01:02.375 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:01:02.475 INFO:teuthology.orchestra.run.smithi196.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:01:02.634 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T10:01:02.794 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T10:01:02.981 INFO:teuthology.orchestra.run.smithi196.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:01:02.987 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:04.163 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T10:01:04.246 INFO:teuthology.orchestra.run.smithi082.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T10:01:04.711 INFO:teuthology.orchestra.run.smithi082.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T10:01:04.716 DEBUG:teuthology.parallel:result is None 2024-04-25T10:01:04.716 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:04.879 DEBUG:teuthology.orchestra.run.smithi082:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:01:04.897 INFO:teuthology.orchestra.run.smithi082.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:04.898 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:04.898 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:01:04.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:01:05.045 DEBUG:teuthology.orchestra.run.smithi196:> dpkg-query -W -f '${Version}' ceph 2024-04-25T10:01:05.065 INFO:teuthology.orchestra.run.smithi196.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:05.065 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T10:01:05.066 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T10:01:05.068 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:01:05.068 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:05.068 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:01:05.083 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:05.083 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:01:05.120 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:01:05.121 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:05.121 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:01:05.137 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:01:05.189 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:05.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:01:05.203 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:01:05.259 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:01:05.259 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:05.259 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:01:05.272 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:01:05.325 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:05.326 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:01:05.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:01:05.395 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:01:05.395 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:05.395 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:01:05.408 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:01:05.462 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:05.462 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:01:05.475 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:01:05.531 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:01:05.623 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:01:05.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 777 /var/log/ceph 2024-04-25T10:01:05.626 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 777 /var/log/ceph 2024-04-25T10:01:05.638 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:01:05.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:01:05.684 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:01:05.696 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:01:05.696 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:01:05.740 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:01:05.760 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:01:05.760 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:01:05.761 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:01:05.761 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:05.761 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:01:05.795 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:05.870 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:05.870 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:01:05.876 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:05.956 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:01:05.956 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:01:05.957 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673040', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi082.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHN7JvV1wTrMW5BDv21p8/DFaAdvrNvn+Rt9PcIjCHMKwFMep4ZwBmAkGkl63VW9vmKO1FSqTBRE+TZWzTO4CZw=', 'smithi196.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC+M9uE/UVSWDnE37gtjNlQSL9UjkrinuUY/QPsLENLe/7sIWyRDLndrtq+p0JzsceXDgS9RF8XFSSqoC3wyJaQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:01:05.957 DEBUG:teuthology.orchestra.run.smithi082:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:01:05.959 DEBUG:teuthology.orchestra.run.smithi196:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:01:06.001 DEBUG:teuthology.orchestra.run.smithi082:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:01:06.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:01:06.055 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:06.056 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:01:06.071 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:01:06.071 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_1 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 883 Links: 1 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 09:54:59.405392176 +0000 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 09:54:59.157394391 +0000 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 09:54:59.157394391 +0000 2024-04-25T10:01:06.119 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-04-25T10:01:06.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:01:06.174 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T10:01:06.174 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T10:01:06.174 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000381 s, 1.3 MB/s 2024-04-25T10:01:06.175 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:01:06.224 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_2 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 09:54:59.909387674 +0000 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 09:54:59.665389854 +0000 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 09:54:59.665389854 +0000 2024-04-25T10:01:06.271 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-04-25T10:01:06.272 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T10:01:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T10:01:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000417258 s, 1.2 MB/s 2024-04-25T10:01:06.327 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:01:06.376 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_3 2024-04-25T10:01:06.423 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:01:06.423 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.423 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 914 Links: 1 2024-04-25T10:01:06.424 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.424 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 09:55:00.409383208 +0000 2024-04-25T10:01:06.424 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 09:55:00.165385387 +0000 2024-04-25T10:01:06.424 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 09:55:00.165385387 +0000 2024-04-25T10:01:06.424 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-04-25T10:01:06.424 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:01:06.477 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T10:01:06.477 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T10:01:06.477 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000448497 s, 1.1 MB/s 2024-04-25T10:01:06.478 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:01:06.528 DEBUG:teuthology.orchestra.run.smithi082:> stat /dev/vg_nvme/lv_4 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout:Access: 2024-04-25 09:55:00.641381136 +0000 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout:Modify: 2024-04-25 09:55:00.637381171 +0000 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout:Change: 2024-04-25 09:55:00.637381171 +0000 2024-04-25T10:01:06.575 INFO:teuthology.orchestra.run.smithi082.stdout: Birth: - 2024-04-25T10:01:06.576 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:01:06.629 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records in 2024-04-25T10:01:06.629 INFO:teuthology.orchestra.run.smithi082.stderr:1+0 records out 2024-04-25T10:01:06.629 INFO:teuthology.orchestra.run.smithi082.stderr:512 bytes copied, 0.000275335 s, 1.9 MB/s 2024-04-25T10:01:06.630 DEBUG:teuthology.orchestra.run.smithi082:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:01:06.680 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:06.680 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:06.680 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:01:06.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:01:06.686 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_1 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 875 Links: 1 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 09:55:00.923388621 +0000 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 09:55:00.659390979 +0000 2024-04-25T10:01:06.733 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 09:55:00.659390979 +0000 2024-04-25T10:01:06.734 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T10:01:06.734 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:01:06.787 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T10:01:06.787 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T10:01:06.787 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.00045282 s, 1.1 MB/s 2024-04-25T10:01:06.788 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:01:06.838 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_2 2024-04-25T10:01:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:01:06.884 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:06.884 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-04-25T10:01:06.884 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:06.884 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 09:55:01.391384440 +0000 2024-04-25T10:01:06.885 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 09:55:01.147386620 +0000 2024-04-25T10:01:06.885 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 09:55:01.147386620 +0000 2024-04-25T10:01:06.885 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T10:01:06.885 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:01:06.938 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T10:01:06.938 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T10:01:06.939 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000411318 s, 1.2 MB/s 2024-04-25T10:01:06.940 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:01:06.989 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_3 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 911 Links: 1 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 09:55:01.855380295 +0000 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 09:55:01.607382510 +0000 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 09:55:01.607382510 +0000 2024-04-25T10:01:07.036 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T10:01:07.037 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:01:07.091 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T10:01:07.091 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T10:01:07.091 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000434207 s, 1.2 MB/s 2024-04-25T10:01:07.092 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:01:07.141 DEBUG:teuthology.orchestra.run.smithi196:> stat /dev/vg_nvme/lv_4 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout:Device: 5h/5d Inode: 927 Links: 1 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout:Access: 2024-04-25 09:55:02.331376044 +0000 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout:Modify: 2024-04-25 09:55:02.079378294 +0000 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout:Change: 2024-04-25 09:55:02.079378294 +0000 2024-04-25T10:01:07.188 INFO:teuthology.orchestra.run.smithi196.stdout: Birth: - 2024-04-25T10:01:07.189 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:01:07.242 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records in 2024-04-25T10:01:07.242 INFO:teuthology.orchestra.run.smithi196.stderr:1+0 records out 2024-04-25T10:01:07.242 INFO:teuthology.orchestra.run.smithi196.stderr:512 bytes copied, 0.000391037 s, 1.3 MB/s 2024-04-25T10:01:07.243 DEBUG:teuthology.orchestra.run.smithi196:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:01:07.293 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:07.293 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:01:07.293 INFO:tasks.ceph:Generating config... 2024-04-25T10:01:07.294 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:01:07.294 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:01:07.294 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:01:07.295 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:01:07.295 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:01:07.295 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:01:07.295 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:01:07.296 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:01:07.296 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:01:07.296 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:01:07.296 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:01:07.296 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:01:07.297 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:01:07.297 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:01:07.297 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:01:07.297 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:07.333 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:01:07.349 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:01:07.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:01:07.478 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.82'), ('mon.b', '172.21.15.196'), ('mon.c', '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]')] 2024-04-25T10:01:07.479 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:01:07.479 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:07.480 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:01:07.527 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.82 --add b 172.21.15.196 --addv c '[v2:172.21.15.196:3301,v1:172.21.15.196:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:07.611 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: generated fsid 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:setting min_mon_release = quincy 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:epoch 0 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:fsid 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:last_changed 2024-04-25T10:01:07.608924+0000 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:created 2024-04-25T10:01:07.608924+0000 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:min_mon_release 17 (quincy) 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:election_strategy: 1 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:0: [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] mon.a 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:1: [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] mon.b 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:2: [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] mon.c 2024-04-25T10:01:07.612 INFO:teuthology.orchestra.run.smithi082.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:01:07.613 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:01:07.659 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 923528c9-0d4c-4379-be0f-2d1b434d3c6c... 2024-04-25T10:01:07.660 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:01:07.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:01:07.732 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T10:01:07.732 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T10:01:07.732 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:01:07.732 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T10:01:07.733 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:01:07.734 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:01:07.735 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T10:01:07.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T10:01:07.737 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:01:07.738 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T10:01:07.739 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:01:07.740 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.741 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T10:01:07.742 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:01:07.742 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:01:07.743 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T10:01:07.744 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:01:07.745 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T10:01:07.746 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T10:01:07.747 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T10:01:07.748 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T10:01:07.749 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.750 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:01:07.751 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:01:07.752 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:01:07.753 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.754 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:01:07.755 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T10:01:07.756 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:01:07.757 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:01:07.757 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:01:07.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:01:07.838 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:01:07.838 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:07.838 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:01:07.843 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:07.844 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:01:07.890 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:01:07.890 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:07.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:01:07.890 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:01:07.955 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:07.955 DEBUG:teuthology.orchestra.run.smithi082:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:08.007 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:01:08.007 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:08.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:01:08.007 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:01:08.032 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:08.032 DEBUG:teuthology.orchestra.run.smithi196:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:08.084 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:01:08.084 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:01:08.085 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:01:08.128 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:01:08.131 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:01:08.173 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:01:08.193 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:01:08.193 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:01:08.236 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:01:08.240 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:01:08.295 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:01:08.380 INFO:teuthology.orchestra.run.smithi082.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:01:08.383 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:01:08.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:01:08.484 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:01:08.487 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:01:08.499 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:01:08.583 INFO:teuthology.orchestra.run.smithi196.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:01:08.586 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:01:08.640 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:01:08.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:01:08.677 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:01:08.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:01:08.769 INFO:teuthology.orchestra.run.smithi082.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:01:08.779 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:01:08.816 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:01:08.828 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:01:08.904 INFO:teuthology.orchestra.run.smithi196.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:01:08.914 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:01:08.915 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi082.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi196.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:01:08.915 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:08.928 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:08.929 INFO:tasks.ceph:role: osd.0 2024-04-25T10:01:08.929 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:01:08.929 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:08.992 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:08.992 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:08.992 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:08.992 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:08.993 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:09.450 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T10:01:09.452 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T10:01:09.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:09.546 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:09.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:09.556 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:09.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:09.608 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:09.608 INFO:tasks.ceph:role: osd.1 2024-04-25T10:01:09.608 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:01:09.609 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:09.669 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:09.670 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:09.670 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:09.670 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:09.670 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:10.131 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T10:01:10.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T10:01:10.133 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:10.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:10.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:10.286 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:10.343 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:10.343 INFO:tasks.ceph:role: osd.2 2024-04-25T10:01:10.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:01:10.343 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:10.406 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:10.407 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:10.407 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:10.407 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:10.407 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:10.407 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:10.886 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T10:01:10.888 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T10:01:10.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:11.038 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:11.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:11.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:11.103 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:11.103 INFO:tasks.ceph:role: osd.3 2024-04-25T10:01:11.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:01:11.103 DEBUG:teuthology.orchestra.run.smithi082:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:11.166 INFO:teuthology.orchestra.run.smithi082.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:11.167 INFO:teuthology.orchestra.run.smithi082.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:11.610 INFO:teuthology.orchestra.run.smithi082.stdout:Discarding blocks...Done. 2024-04-25T10:01:11.612 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi082.front.sepia.ceph.com -o noatime 2024-04-25T10:01:11.612 DEBUG:teuthology.orchestra.run.smithi082:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:11.714 DEBUG:teuthology.orchestra.run.smithi082:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:11.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:11.819 INFO:teuthology.orchestra.run.smithi082.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:11.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.932+0000 7f257bfc37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:01:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.936+0000 7f257bfc37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:01:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.936+0000 7f257bfc37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.936+0000 7f257bfc37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.936+0000 7f257bfc37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:11.944+0000 7f257bfc37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:01:14.927 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:14.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:01:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:01:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:15.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:15.012+0000 7f83824467c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:01:17.981 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:18.035 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:18.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.108+0000 7f6feb2c37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:01:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.108+0000 7f6feb2c37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:01:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.108+0000 7f6feb2c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.108+0000 7f6feb2c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.108+0000 7f6feb2c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:18.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:18.112+0000 7f6feb2c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:01:21.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:21.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.236+0000 7fc952eae7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:01:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.236+0000 7fc952eae7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:01:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.236+0000 7fc952eae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.236+0000 7fc952eae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.236+0000 7fc952eae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:21.240+0000 7fc952eae7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:01:24.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:24.315 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:24.329 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:24.329 INFO:tasks.ceph:role: osd.4 2024-04-25T10:01:24.329 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:01:24.329 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:24.391 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:24.391 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:24.391 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:24.391 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:24.392 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:24.848 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T10:01:24.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T10:01:24.850 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:24.955 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:24.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:24.966 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:24.966 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:25.017 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:25.017 INFO:tasks.ceph:role: osd.5 2024-04-25T10:01:25.018 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:01:25.018 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:25.078 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:25.079 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:25.557 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T10:01:25.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T10:01:25.559 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:25.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:25.736 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:25.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:25.737 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:25.791 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:25.792 INFO:tasks.ceph:role: osd.6 2024-04-25T10:01:25.792 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:01:25.792 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:25.855 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:25.856 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:25.856 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:26.326 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T10:01:26.328 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T10:01:26.328 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:26.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:26.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:26.493 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:26.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:26.547 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:26.548 INFO:tasks.ceph:role: osd.7 2024-04-25T10:01:26.548 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:01:26.548 DEBUG:teuthology.orchestra.run.smithi196:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:26.614 INFO:teuthology.orchestra.run.smithi196.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:26.615 INFO:teuthology.orchestra.run.smithi196.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:26.615 INFO:teuthology.orchestra.run.smithi196.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T10:01:26.615 INFO:teuthology.orchestra.run.smithi196.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:26.615 INFO:teuthology.orchestra.run.smithi196.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:27.050 INFO:teuthology.orchestra.run.smithi196.stdout:Discarding blocks...Done. 2024-04-25T10:01:27.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi196.front.sepia.ceph.com -o noatime 2024-04-25T10:01:27.052 DEBUG:teuthology.orchestra.run.smithi196:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:27.147 DEBUG:teuthology.orchestra.run.smithi196:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:27.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:01:27.159 INFO:teuthology.orchestra.run.smithi196.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T10:01:27.159 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:27.270 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.266+0000 7fcc9e8097c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:01:27.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.270+0000 7fcc9e8097c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:01:27.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.270+0000 7fcc9e8097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:27.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.270+0000 7fcc9e8097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:27.275 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.270+0000 7fcc9e8097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:27.279 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:27.274+0000 7fcc9e8097c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:01:30.277 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:30.291 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:30.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.370+0000 7fcb8d6387c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:01:30.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.374+0000 7fcb8d6387c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:01:30.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.374+0000 7fcb8d6387c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:30.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.374+0000 7fcb8d6387c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:30.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.374+0000 7fcb8d6387c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:30.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:30.374+0000 7fcb8d6387c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:01:33.339 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:33.396 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:33.478 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.474+0000 7fb35d6fe7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:01:33.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.474+0000 7fb35d6fe7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:01:33.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.474+0000 7fb35d6fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:33.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.474+0000 7fb35d6fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:33.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.474+0000 7fb35d6fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:33.479 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:33.478+0000 7fb35d6fe7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:01:35.958 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:35.961 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:36.464 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:36.481 DEBUG:teuthology.orchestra.run.smithi196:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:36.562 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.558+0000 7f2945aeb7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.558+0000 7f2945aeb7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.558+0000 7f2945aeb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.558+0000 7f2945aeb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.558+0000 7f2945aeb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:36.563 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:36.562+0000 7f2945aeb7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:01:39.526 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:39.580 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:01:39.580 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:01:39.594 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.594 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:01:39.650 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.650 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:01:39.706 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.706 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:01:39.763 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:01:39.817 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.817 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:01:39.869 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:39.870 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:01:39.925 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:39.926 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:01:39.939 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:39.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:01:39.994 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:39.994 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:01:40.048 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.048 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:01:40.106 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.106 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:01:40.159 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.160 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:01:40.215 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.215 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:01:40.271 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:40.271 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:01:40.277 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:40.277 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:01:40.323 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.323 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:01:40.329 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:40.329 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:01:40.380 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:01:40.380 DEBUG:teuthology.orchestra.run.smithi082:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:40.383 DEBUG:teuthology.orchestra.run.smithi196:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.y] 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA== 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ== 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA== 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.0] 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw== 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.1] 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA== 2024-04-25T10:01:40.426 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.2] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.3] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr.x] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.4] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg== 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.5] 2024-04-25T10:01:40.427 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.6] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[osd.7] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.2] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.1] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw== 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout:[client.3] 2024-04-25T10:01:40.428 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ== 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.y] 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA== 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ== 2024-04-25T10:01:40.434 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.0] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.1] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.2] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.3] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr.x] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw== 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:01:40.435 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.4] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.5] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.6] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[osd.7] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[client.0] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[client.2] 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg== 2024-04-25T10:01:40.436 INFO:teuthology.orchestra.run.smithi196.stdout:[client.1] 2024-04-25T10:01:40.437 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw== 2024-04-25T10:01:40.437 INFO:teuthology.orchestra.run.smithi196.stdout:[client.3] 2024-04-25T10:01:40.437 INFO:teuthology.orchestra.run.smithi196.stdout: key = AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ== 2024-04-25T10:01:40.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:40.472 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:40.527 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:40.530 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:40.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:40.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:40.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.664 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.746 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.749 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.839 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.841 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:40.930 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.011 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:41.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:41.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:41.102 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:41.148 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:41.150 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:41.198 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.239 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.320 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.362 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.407 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.452 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.491 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:41.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.572 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.617 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.619 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.704 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.790 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.792 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:41.837 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:01:41.837 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:41.884 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:41.976 DEBUG:teuthology.orchestra.run.smithi082:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:41.987 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:42.000 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:42.097 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:42.108 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:42.161 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:42.253 DEBUG:teuthology.orchestra.run.smithi196:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:42.263 DEBUG:teuthology.orchestra.run.smithi082:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:42.266 DEBUG:teuthology.orchestra.run.smithi196:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:42.310 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:01:42.311 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:01:42.311 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:01:42.313 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:01:42.313 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:01:42.313 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:01:42.357 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:01:42.357 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:01:42.357 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:01:42.359 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:01:42.359 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:01:42.359 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:01:42.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:01:42.362 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:01:42.362 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:01:42.362 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:01:42.364 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:01:42.365 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:01:42.365 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:01:42.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 Processor -- start 2024-04-25T10:01:42.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 -- start start 2024-04-25T10:01:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc06b710 con 0x7fe1bc15d180 2024-04-25T10:01:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1bc1617b0 con 0x7fe1bc06fc50 2024-04-25T10:01:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c30d6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1bc161560 con 0x7fe1bc0cf840 2024-04-25T10:01:42.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c1673640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe1bc15d180 0x7fe1bc15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34116/0 (socket says 172.21.15.82:34116) 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1c1673640 1 -- 172.21.15.82:0/1063386594 learned_addr learned my addr 172.21.15.82:0/1063386594 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1bbfff640 1 -- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 msgr2=0x7fe1bc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1bbfff640 1 -- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 msgr2=0x7fe1bc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:42.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.472+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:42.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.676+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.676+0000 7fe1bbfff640 1 -- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 msgr2=0x7fe1bc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T10:01:42.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.676+0000 7fe1bbfff640 1 -- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 msgr2=0x7fe1bc0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:42.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.676+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:42.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.676+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T10:01:42.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.740+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268517515 0 0) 0x7fe1bc06b710 con 0x7fe1bc15d180 2024-04-25T10:01:42.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.740+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a4003530 con 0x7fe1bc15d180 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 66183844 0 0) 0x7fe1a4003530 con 0x7fe1bc15d180 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 msgr2=0x7fe1bc0cfcd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 --2- 172.21.15.82:0/1063386594 >> v2:172.21.15.196:3300/0 conn(0x7fe1bc0cf840 0x7fe1bc0cfcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 >> v1:172.21.15.196:6789/0 conn(0x7fe1bc06fc50 legacy=0x7fe1bc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc161a90 con 0x7fe1bc15d180 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe1b4001a00 con 0x7fe1bc15d180 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 >> v1:172.21.15.82:6789/0 conn(0x7fe1bc15d180 legacy=0x7fe1bc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1a4004810 0x7fe1a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40051d0 0x7fe1a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc161560 con 0x7fe1a4004810 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a400c100 con 0x7fe1a4003a30 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a400c300 con 0x7fe1a40051d0 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1a4004810 0x7fe1a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c1673640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40051d0 0x7fe1a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 -- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40051d0 msgr2=0x7fe1a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1a40051d0 0x7fe1a400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 -- 172.21.15.82:0/1063386594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1a4004810 msgr2=0x7fe1a4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1a4004810 0x7fe1a4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c0e72640 1 -- 172.21.15.82:0/1063386594 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a4003530 con 0x7fe1a4003a30 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.744+0000 7fe1c1673640 1 -- 172.21.15.82:0/1063386594 reap_dead start 2024-04-25T10:01:42.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.748+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1a4004810 0x7fe1a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:42.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.752+0000 7fe1c0e72640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1a4003e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac01ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea11073a5fc39bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.752+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac0b5070 con 0x7fe1a4003a30 2024-04-25T10:01:42.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.752+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ac0a8830 con 0x7fe1a4003a30 2024-04-25T10:01:42.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.752+0000 7fe1bb7fe640 1 -- 172.21.15.82:0/1063386594 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ac001460 con 0x7fe1a4003a30 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 msgr2=0x7fe1a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1a4003e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac01ec10 comp rx=0 tx=0).stop 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- 172.21.15.82:0/1063386594 shutdown_connections 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/1063386594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1a4003e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- 172.21.15.82:0/1063386594 >> 172.21.15.82:0/1063386594 conn(0x7fe1bc099a70 msgr2=0x7fe1bc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- 172.21.15.82:0/1063386594 shutdown_connections 2024-04-25T10:01:42.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- 172.21.15.82:0/1063386594 wait complete. 2024-04-25T10:01:42.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 Processor -- start 2024-04-25T10:01:42.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- start start 2024-04-25T10:01:42.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1bc0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc0b19e0 0x7fe1bc0b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc070030 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0d0e30 con 0x7fe1a4003a30 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c30d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1bc0701b0 con 0x7fe1bc0b19e0 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35940/0 (socket says 172.21.15.82:35940) 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1bbfff640 1 -- 172.21.15.82:0/2418230648 learned_addr learned my addr 172.21.15.82:0/2418230648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c0e72640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1bc0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.756+0000 7fe1c1673640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc0b19e0 0x7fe1bc0b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 -- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc0b19e0 msgr2=0x7fe1bc0b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc0b19e0 0x7fe1bc0b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 -- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 msgr2=0x7fe1bc0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1bc0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 -- 172.21.15.82:0/2418230648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0c6e50 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1bbfff640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0013910 tx=0x7fe1b0013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b001a070 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0004440 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1bc1e8680 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000f040 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1bc1e8940 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe1b001d070 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe1b00039a0 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.760+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1bc04efa0 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.764+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe1b00059a0 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.828+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe1bc0cc6b0 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1b97fa640 1 -- 172.21.15.82:0/2418230648 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe1b0003bf0 con 0x7fe1bc0c69d0 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:adjusted tunables profile to default 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 msgr2=0x7fe1bc0afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0013910 tx=0x7fe1b0013dd0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 shutdown_connections 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1bc0b19e0 0x7fe1bc0b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1bc0c69d0 0x7fe1bc0afff0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 --2- 172.21.15.82:0/2418230648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1a4003a30 0x7fe1bc0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 >> 172.21.15.82:0/2418230648 conn(0x7fe1bc099a70 msgr2=0x7fe1bc0cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 shutdown_connections 2024-04-25T10:01:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:42.832+0000 7fe1c30d6640 1 -- 172.21.15.82:0/2418230648 wait complete. 2024-04-25T10:01:42.859 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:01:42.859 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:01:42.859 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:42.860 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:01:42.871 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:42.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:01:42.925 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:42.925 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:01:42.941 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:42.932+0000 7f1c0921b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:42.950 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:01:42.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:01:43.008 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:43.008 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:01:43.023 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:43.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:01:43.081 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:43.081 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:01:43.135 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:01:43.135 DEBUG:teuthology.orchestra.run.smithi196:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:01:43.196 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 246b7925-be82-4cc8-a55d-b2faaec118ea 0 2024-04-25T10:01:43.304 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:43.302+0000 7f2c9abd9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:43.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 Processor -- start 2024-04-25T10:01:43.365 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 -- start start 2024-04-25T10:01:43.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa95409a180 0x7fa9540b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9540cbec0 con 0x7fa9540b2c30 2024-04-25T10:01:43.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9540cc1a0 con 0x7fa95409a7b0 2024-04-25T10:01:43.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95c1c5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9540cc480 con 0x7fa95409a180 2024-04-25T10:01:43.368 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95a762640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa95409a7b0 0x7fa9540ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:60846/0 (socket says 172.21.15.196:60846) 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa95a762640 1 -- 172.21.15.196:0/2483469849 learned_addr learned my addr 172.21.15.196:0/2483469849 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa959760640 1 --2- 172.21.15.196:0/2483469849 >> v2:172.21.15.82:3300/0 conn(0x7fa95409a180 0x7fa9540b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565186920 0 0) 0x7fa9540cc1a0 con 0x7fa95409a7b0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.362+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa940003530 con 0x7fa95409a7b0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 136674395 0 0) 0x7fa940003530 con 0x7fa95409a7b0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 >> v2:172.21.15.82:3300/0 conn(0x7fa95409a180 msgr2=0x7fa9540b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 --2- 172.21.15.196:0/2483469849 >> v2:172.21.15.82:3300/0 conn(0x7fa95409a180 0x7fa9540b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 >> v1:172.21.15.82:6789/0 conn(0x7fa9540b2c30 legacy=0x7fa954099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9540cc760 con 0x7fa95409a7b0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa94c001a00 con 0x7fa95409a7b0 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 >> v1:172.21.15.196:6789/0 conn(0x7fa95409a7b0 legacy=0x7fa9540ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa940003a30 0x7fa940003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa940004810 0x7fa940004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 0x7fa94000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.369 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540cc480 con 0x7fa940003a30 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94000c100 con 0x7fa9400051d0 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94000c300 con 0x7fa940004810 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 0x7fa94000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa959760640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa940004810 0x7fa940004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa959f61640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa940003a30 0x7fa940003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 -- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa940004810 msgr2=0x7fa940004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa940004810 0x7fa940004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 -- 172.21.15.196:0/2483469849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa940003a30 msgr2=0x7fa940003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa940003a30 0x7fa940003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 -- 172.21.15.196:0/2483469849 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940003530 con 0x7fa9400051d0 2024-04-25T10:01:43.370 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa959f61640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa940003a30 0x7fa940003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 -- 172.21.15.196:0/2483469849 reap_dead start 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95a762640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 0x7fa94000b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00a850 tx=0x7fa94c01d340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83e1e93639a65e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c00a4a0 con 0x7fa9400051d0 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c0a5e50 con 0x7fa9400051d0 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa958f5f640 1 -- 172.21.15.196:0/2483469849 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0ad890 con 0x7fa9400051d0 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95c1c5640 1 -- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 msgr2=0x7fa94000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 0x7fa94000b9d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa94c00a850 tx=0x7fa94c01d340 comp rx=0 tx=0).stop 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95c1c5640 1 -- 172.21.15.196:0/2483469849 shutdown_connections 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/2483469849 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9400051d0 0x7fa94000b9d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.366+0000 7fa95c1c5640 1 -- 172.21.15.196:0/2483469849 >> 172.21.15.196:0/2483469849 conn(0x7fa9540b3010 msgr2=0x7fa9540b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- 172.21.15.196:0/2483469849 shutdown_connections 2024-04-25T10:01:43.372 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- 172.21.15.196:0/2483469849 wait complete. 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 Processor -- start 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- start start 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa954157430 0x7fa9540c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954153300 0x7fa954153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95406c380 con 0x7fa954157430 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95406c200 con 0x7fa954153300 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9540b0ba0 con 0x7fa9400051d0 2024-04-25T10:01:43.375 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:54926/0 (socket says 172.21.15.196:54926) 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 -- 172.21.15.196:0/4120540912 learned_addr learned my addr 172.21.15.196:0/4120540912 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95a762640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954153300 0x7fa954153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959760640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa954157430 0x7fa9540c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 -- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954153300 msgr2=0x7fa954153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954153300 0x7fa954153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 -- 172.21.15.196:0/4120540912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa954157430 msgr2=0x7fa9540c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa954157430 0x7fa9540c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 -- 172.21.15.196:0/4120540912 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9541610d0 con 0x7fa9400051d0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa959f61640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d770 tx=0x7fa95000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa950005270 con 0x7fa9400051d0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa950017070 con 0x7fa9400051d0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa954153cc0 con 0x7fa9400051d0 2024-04-25T10:01:43.376 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000b6b0 con 0x7fa9400051d0 2024-04-25T10:01:43.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954160b10 con 0x7fa9400051d0 2024-04-25T10:01:43.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa95000b850 con 0x7fa9400051d0 2024-04-25T10:01:43.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa95001c460 con 0x7fa9400051d0 2024-04-25T10:01:43.377 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.370+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95404efa0 con 0x7fa9400051d0 2024-04-25T10:01:43.382 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.378+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa950003da0 con 0x7fa9400051d0 2024-04-25T10:01:43.411 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:43.404+0000 7f1c0921b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:43.451 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.450+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "246b7925-be82-4cc8-a55d-b2faaec118ea", "id": 0} v 0) v1 -- 0x7fa9540d12d0 con 0x7fa9400051d0 2024-04-25T10:01:43.459 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa94a7fc640 1 -- 172.21.15.196:0/4120540912 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "246b7925-be82-4cc8-a55d-b2faaec118ea", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa95000bd20 con 0x7fa9400051d0 2024-04-25T10:01:43.460 INFO:teuthology.orchestra.run.smithi196.stdout:0 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 msgr2=0x7fa9540c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa95000d770 tx=0x7fa95000dc30 comp rx=0 tx=0).stop 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 shutdown_connections 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa954153300 0x7fa954153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa954157430 0x7fa9540c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 --2- 172.21.15.196:0/4120540912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa9400051d0 0x7fa9540c7930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 >> 172.21.15.196:0/4120540912 conn(0x7fa9540b3010 msgr2=0x7fa9540cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 shutdown_connections 2024-04-25T10:01:43.461 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.458+0000 7fa95c1c5640 1 -- 172.21.15.196:0/4120540912 wait complete. 2024-04-25T10:01:43.477 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new bbb18f09-b87b-47ec-a70e-f53bd22c7e66 1 2024-04-25T10:01:43.572 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:43.564+0000 7f1c0921b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:43.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 Processor -- start 2024-04-25T10:01:43.594 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 -- start start 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f891006bca0 0x7f89100cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8910066e30 0x7f891006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f891006ec30 con 0x7f89100af370 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100669a0 con 0x7f891006bca0 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8917721640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89100cd5f0 con 0x7f8910066e30 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.590+0000 7f8914cbc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8910066e30 0x7f891006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8910066e30 0x7f891006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43620/0 (socket says 172.21.15.196:43620) 2024-04-25T10:01:43.596 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 -- 172.21.15.196:0/3953126540 learned_addr learned my addr 172.21.15.196:0/3953126540 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:43.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f89154bd640 1 --2- 172.21.15.196:0/3953126540 >> v2:172.21.15.82:3300/0 conn(0x7f891006bca0 0x7f89100cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 -- 172.21.15.196:0/3953126540 >> v2:172.21.15.82:3300/0 conn(0x7f891006bca0 msgr2=0x7f89100cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 --2- 172.21.15.196:0/3953126540 >> v2:172.21.15.82:3300/0 conn(0x7f891006bca0 0x7f89100cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 -- 172.21.15.196:0/3953126540 >> v1:172.21.15.82:6789/0 conn(0x7f89100af370 legacy=0x7f891006d870 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:01:43.597 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 -- 172.21.15.196:0/3953126540 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891006c5c0 con 0x7f8910066e30 2024-04-25T10:01:43.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8914cbc640 1 --2- 172.21.15.196:0/3953126540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8910066e30 0x7f891006c080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89100731a0 tx=0x7f890001c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e494b4182b28138a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8907fff640 1 -- 172.21.15.196:0/3953126540 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000b0070 con 0x7f8910066e30 2024-04-25T10:01:43.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8907fff640 1 -- 172.21.15.196:0/3953126540 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89000ab600 con 0x7f8910066e30 2024-04-25T10:01:43.598 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.594+0000 7f8907fff640 1 -- 172.21.15.196:0/3953126540 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000ab9d0 con 0x7f8910066e30 2024-04-25T10:01:43.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/3953126540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8910066e30 msgr2=0x7f891006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.599 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- 172.21.15.196:0/3953126540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8910066e30 0x7f891006c080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f89100731a0 tx=0x7f890001c390 comp rx=0 tx=0).stop 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/3953126540 shutdown_connections 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- 172.21.15.196:0/3953126540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8910066e30 0x7f891006c080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- 172.21.15.196:0/3953126540 >> v2:172.21.15.82:3300/0 conn(0x7f891006bca0 0x7f89100cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/3953126540 >> 172.21.15.196:0/3953126540 conn(0x7f8910071810 msgr2=0x7f8910071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/3953126540 shutdown_connections 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/3953126540 wait complete. 2024-04-25T10:01:43.600 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 Processor -- start 2024-04-25T10:01:43.601 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- start start 2024-04-25T10:01:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 0x7f8910148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.603 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 0x7f89100adad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891015cc80 con 0x7f8910066e30 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f891015cb00 con 0x7f89100af370 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8910168630 con 0x7f891006bca0 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8915cbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 0x7f89100adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:54940/0 (socket says 172.21.15.196:54940) 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 -- 172.21.15.196:0/1289257003 learned_addr learned my addr 172.21.15.196:0/1289257003 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f89154bd640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 0x7f8910148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 -- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 msgr2=0x7f89100adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 0x7f89100adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 -- 172.21.15.196:0/1289257003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 msgr2=0x7f8910148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 0x7f8910148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 -- 172.21.15.196:0/1289257003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89101506c0 con 0x7f891006bca0 2024-04-25T10:01:43.604 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8915cbe640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 0x7f89100adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f89154bd640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 0x7f8910148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8914cbc640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f89000ae040 tx=0x7f8900019130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000b0040 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89000b9070 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89000b4660 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89101509a0 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.598+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8910150f60 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.602+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f89000b4800 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.602+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f89000bec50 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.602+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f891004efa0 con 0x7f891006bca0 2024-04-25T10:01:43.605 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.602+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f89000160d0 con 0x7f891006bca0 2024-04-25T10:01:43.670 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.666+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bbb18f09-b87b-47ec-a70e-f53bd22c7e66", "id": 1} v 0) v1 -- 0x7f8910072910 con 0x7f891006bca0 2024-04-25T10:01:43.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.674+0000 7f8905ffb640 1 -- 172.21.15.196:0/1289257003 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bbb18f09-b87b-47ec-a70e-f53bd22c7e66", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f89000b4c30 con 0x7f891006bca0 2024-04-25T10:01:43.679 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T10:01:43.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 msgr2=0x7f89100ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f89000ae040 tx=0x7f8900019130 comp rx=0 tx=0).stop 2024-04-25T10:01:43.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 shutdown_connections 2024-04-25T10:01:43.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f891006bca0 0x7f89100ad590 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89100af370 0x7f89100adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 --2- 172.21.15.196:0/1289257003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8910066e30 0x7f8910148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 >> 172.21.15.196:0/1289257003 conn(0x7f8910071810 msgr2=0x7f89100cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 shutdown_connections 2024-04-25T10:01:43.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.678+0000 7f8917721640 1 -- 172.21.15.196:0/1289257003 wait complete. 2024-04-25T10:01:43.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 917e2894-19a0-4a70-b172-c039392e5436 2 2024-04-25T10:01:43.793 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:43.790+0000 7f2c9abd9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:43.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 Processor -- start 2024-04-25T10:01:43.818 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 -- start start 2024-04-25T10:01:43.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89fc0cf0e0 0x7f89fc15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.820 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0ce320 con 0x7f89fc15aac0 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc0ce600 con 0x7f89fc0cd350 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f8a02da8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc0ce0d0 con 0x7f89fc0cf0e0 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f89fbfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f89fc0cd350 0x7f89fc15aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:60854/0 (socket says 172.21.15.196:60854) 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.814+0000 7f89fbfff640 1 -- 172.21.15.196:0/1919557102 learned_addr learned my addr 172.21.15.196:0/1919557102 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 --2- 172.21.15.196:0/1919557102 >> v2:172.21.15.82:3300/0 conn(0x7f89fc0cf0e0 0x7f89fc15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842564767 0 0) 0x7f89fc0ce600 con 0x7f89fc0cd350 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e4003530 con 0x7f89fc0cd350 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 591538761 0 0) 0x7f89e4003530 con 0x7f89fc0cd350 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 >> v2:172.21.15.82:3300/0 conn(0x7f89fc0cf0e0 msgr2=0x7f89fc15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 --2- 172.21.15.196:0/1919557102 >> v2:172.21.15.82:3300/0 conn(0x7f89fc0cf0e0 0x7f89fc15b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 >> v1:172.21.15.82:6789/0 conn(0x7f89fc15aac0 legacy=0x7f89fc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0ce8e0 con 0x7f89fc0cd350 2024-04-25T10:01:43.821 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f89f0001680 con 0x7f89fc0cd350 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 >> v1:172.21.15.196:6789/0 conn(0x7f89fc0cd350 legacy=0x7f89fc15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4003a30 0x7f89e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e4004810 0x7f89e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 0x7f89e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0ce0d0 con 0x7f89e4003a30 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c100 con 0x7f89e40051d0 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400c300 con 0x7f89e4004810 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 0x7f89e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fbfff640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e4004810 0x7f89e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 -- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e4004810 msgr2=0x7f89e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e4004810 0x7f89e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 -- 172.21.15.196:0/1919557102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4003a30 msgr2=0x7f89e4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.822 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e4003a30 0x7f89e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.823 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 -- 172.21.15.196:0/1919557102 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4003530 con 0x7f89e40051d0 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 -- 172.21.15.196:0/1919557102 reap_dead start 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a01345640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 0x7f89e400b9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009ae0 tx=0x7f89ec01cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c621d97798145ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0096b0 con 0x7f89e40051d0 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec004020 con 0x7f89e40051d0 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f89fb7fe640 1 -- 172.21.15.196:0/1919557102 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec004410 con 0x7f89e40051d0 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 -- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 msgr2=0x7f89e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.824 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 0x7f89e400b9d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009ae0 tx=0x7f89ec01cab0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 -- 172.21.15.196:0/1919557102 shutdown_connections 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 --2- 172.21.15.196:0/1919557102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40051d0 0x7f89e400b9d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 -- 172.21.15.196:0/1919557102 >> 172.21.15.196:0/1919557102 conn(0x7f89fc099a70 msgr2=0x7f89fc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.818+0000 7f8a02da8640 1 -- 172.21.15.196:0/1919557102 shutdown_connections 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- 172.21.15.196:0/1919557102 wait complete. 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 Processor -- start 2024-04-25T10:01:43.825 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- start start 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40051d0 0x7f89fc1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1643b0 0x7f89fc0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0ccfd0 con 0x7f89e40051d0 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cce50 con 0x7f89fc1643b0 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0cd150 con 0x7f89fc15ccd0 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:54958/0 (socket says 172.21.15.196:54958) 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 -- 172.21.15.196:0/465608219 learned_addr learned my addr 172.21.15.196:0/465608219 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89fbfff640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1643b0 0x7f89fc0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.827 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a00b44640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40051d0 0x7f89fc1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 -- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1643b0 msgr2=0x7f89fc0c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1643b0 0x7f89fc0c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 -- 172.21.15.196:0/465608219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40051d0 msgr2=0x7f89fc1559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40051d0 0x7f89fc1559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 -- 172.21.15.196:0/465608219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0c7e20 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a01345640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89ec01d350 tx=0x7f89ec019850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0b7070 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0af070 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec003860 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0c8100 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc162740 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f89ec0ba070 con 0x7f89fc15ccd0 2024-04-25T10:01:43.828 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f89ec0a6bd0 con 0x7f89fc15ccd0 2024-04-25T10:01:43.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.822+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc04efa0 con 0x7f89fc15ccd0 2024-04-25T10:01:43.829 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.826+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f89ec0d3dd0 con 0x7f89fc15ccd0 2024-04-25T10:01:43.897 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.894+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "917e2894-19a0-4a70-b172-c039392e5436", "id": 2} v 0) v1 -- 0x7f89fc0cdc90 con 0x7f89fc15ccd0 2024-04-25T10:01:43.905 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f89f97fa640 1 -- 172.21.15.196:0/465608219 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "917e2894-19a0-4a70-b172-c039392e5436", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f89ec0c2070 con 0x7f89fc15ccd0 2024-04-25T10:01:43.906 INFO:teuthology.orchestra.run.smithi196.stdout:2 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 msgr2=0x7f89fc15d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f89ec01d350 tx=0x7f89ec019850 comp rx=0 tx=0).stop 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 shutdown_connections 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15ccd0 0x7f89fc15d150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc1643b0 0x7f89fc0c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 --2- 172.21.15.196:0/465608219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40051d0 0x7f89fc1559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 >> 172.21.15.196:0/465608219 conn(0x7f89fc099a70 msgr2=0x7f89fc158bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 shutdown_connections 2024-04-25T10:01:43.907 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:43.902+0000 7f8a02da8640 1 -- 172.21.15.196:0/465608219 wait complete. 2024-04-25T10:01:43.922 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 54695f07-cce4-4a88-b28f-a1d141b52f49 3 2024-04-25T10:01:43.980 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:43.978+0000 7f2c9abd9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 Processor -- start 2024-04-25T10:01:44.034 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 -- start start 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f7718099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771809a950 con 0x7f7718099fb0 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771809ac30 con 0x7f77180b0440 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.030+0000 7f771e84e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771809a400 con 0x7f771815b920 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7717fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f7718099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f77177fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f77180b0440 0x7f77180aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:60862/0 (socket says 172.21.15.196:60862) 2024-04-25T10:01:44.035 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f77177fe640 1 -- 172.21.15.196:0/2713127005 learned_addr learned my addr 172.21.15.196:0/2713127005 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T10:01:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7716ffd640 1 -- 172.21.15.196:0/2713127005 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954772286 0 0) 0x7f771809ac30 con 0x7f77180b0440 2024-04-25T10:01:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7717fff640 1 -- 172.21.15.196:0/2713127005 >> v1:172.21.15.196:6789/0 conn(0x7f77180b0440 legacy=0x7f77180aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7717fff640 1 -- 172.21.15.196:0/2713127005 >> v1:172.21.15.82:6789/0 conn(0x7f7718099fb0 legacy=0x7f77180af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.036 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7717fff640 1 -- 172.21.15.196:0/2713127005 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180ae380 con 0x7f771815b920 2024-04-25T10:01:44.037 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7717fff640 1 --2- 172.21.15.196:0/2713127005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f7718099a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77080097f0 tx=0x7f770801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721e05d0d9a95eb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7716ffd640 1 -- 172.21.15.196:0/2713127005 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080b0070 con 0x7f771815b920 2024-04-25T10:01:44.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7716ffd640 1 -- 172.21.15.196:0/2713127005 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080ab600 con 0x7f771815b920 2024-04-25T10:01:44.038 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f7716ffd640 1 -- 172.21.15.196:0/2713127005 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080ab9d0 con 0x7f771815b920 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 -- 172.21.15.196:0/2713127005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 msgr2=0x7f7718099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 --2- 172.21.15.196:0/2713127005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f7718099a70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f77080097f0 tx=0x7f770801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 -- 172.21.15.196:0/2713127005 shutdown_connections 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 --2- 172.21.15.196:0/2713127005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f7718099a70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 -- 172.21.15.196:0/2713127005 >> 172.21.15.196:0/2713127005 conn(0x7f77180b3010 msgr2=0x7f77180b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 -- 172.21.15.196:0/2713127005 shutdown_connections 2024-04-25T10:01:44.039 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.034+0000 7f771e84e640 1 -- 172.21.15.196:0/2713127005 wait complete. 2024-04-25T10:01:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 Processor -- start 2024-04-25T10:01:44.040 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- start start 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7718099fb0 0x7f77180cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f77180c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718169ce0 con 0x7f7718099fb0 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180cd550 con 0x7f77180b0440 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7718169e60 con 0x7f771815b920 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f77177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f77180c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.042 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f77177fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43662/0 (socket says 172.21.15.196:43662) 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f77177fe640 1 -- 172.21.15.196:0/2599945712 learned_addr learned my addr 172.21.15.196:0/2599945712 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 -- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 msgr2=0x7f771815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 -- 172.21.15.196:0/2599945712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7718099fb0 msgr2=0x7f77180cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7718099fb0 0x7f77180cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 -- 172.21.15.196:0/2599945712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180cb2f0 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f77177fe640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771cdeb640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f77180c93b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f771000be30 tx=0x7f771000bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710019070 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710014440 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710014830 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77180c85f0 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180c8bb0 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f771001c070 con 0x7f771815b920 2024-04-25T10:01:44.043 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f77100045a0 con 0x7f771815b920 2024-04-25T10:01:44.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.038+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771804efa0 con 0x7f771815b920 2024-04-25T10:01:44.044 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.042+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7710004110 con 0x7f771815b920 2024-04-25T10:01:44.112 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.104+0000 7f1c0921b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:44.112 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.110+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "54695f07-cce4-4a88-b28f-a1d141b52f49", "id": 3} v 0) v1 -- 0x7f77180b2bc0 con 0x7f771815b920 2024-04-25T10:01:44.118 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.114+0000 7f7714ff9640 1 -- 172.21.15.196:0/2599945712 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "54695f07-cce4-4a88-b28f-a1d141b52f49", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7710003980 con 0x7f771815b920 2024-04-25T10:01:44.119 INFO:teuthology.orchestra.run.smithi196.stdout:3 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 msgr2=0x7f77180c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f77180c93b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f771000be30 tx=0x7f771000bf20 comp rx=0 tx=0).stop 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 shutdown_connections 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f771815b920 0x7f77180c93b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77180b0440 0x7f771815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 --2- 172.21.15.196:0/2599945712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7718099fb0 0x7f77180cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 >> 172.21.15.196:0/2599945712 conn(0x7f77180b3010 msgr2=0x7f77180cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 shutdown_connections 2024-04-25T10:01:44.120 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.118+0000 7f771e84e640 1 -- 172.21.15.196:0/2599945712 wait complete. 2024-04-25T10:01:44.134 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f 4 2024-04-25T10:01:44.166 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.156+0000 7f1c0921b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:44.218 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.208+0000 7f1c0921b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 Processor -- start 2024-04-25T10:01:44.245 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 -- start start 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2cc0cd180 0x7fb2cc15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2cc0ce320 con 0x7fb2cc0cf0e0 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0cd5d0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2d151a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2cc0ce0d0 con 0x7fb2cc15aac0 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43672/0 (socket says 172.21.15.196:43672) 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 -- 172.21.15.196:0/1872594527 learned_addr learned my addr 172.21.15.196:0/1872594527 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 -- 172.21.15.196:0/1872594527 >> v2:172.21.15.82:3300/0 conn(0x7fb2cc0cd180 msgr2=0x7fb2cc15b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.247 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 --2- 172.21.15.196:0/1872594527 >> v2:172.21.15.82:3300/0 conn(0x7fb2cc0cd180 0x7fb2cc15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 -- 172.21.15.196:0/1872594527 >> v1:172.21.15.82:6789/0 conn(0x7fb2cc0cf0e0 legacy=0x7fb2cc15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.248 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.242+0000 7fb2caffd640 1 -- 172.21.15.196:0/1872594527 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0ce600 con 0x7fb2cc15aac0 2024-04-25T10:01:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2caffd640 1 --2- 172.21.15.196:0/1872594527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc072f80 tx=0x7fb2bc01c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bbba18b07e837df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2c9ffb640 1 -- 172.21.15.196:0/1872594527 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0b1070 con 0x7fb2cc15aac0 2024-04-25T10:01:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2c9ffb640 1 -- 172.21.15.196:0/1872594527 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc0ac630 con 0x7fb2cc15aac0 2024-04-25T10:01:44.249 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2c9ffb640 1 -- 172.21.15.196:0/1872594527 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0a8050 con 0x7fb2cc15aac0 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- 172.21.15.196:0/1872594527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc15aac0 msgr2=0x7fb2cc0ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 --2- 172.21.15.196:0/1872594527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc072f80 tx=0x7fb2bc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- 172.21.15.196:0/1872594527 shutdown_connections 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 --2- 172.21.15.196:0/1872594527 >> v2:172.21.15.82:3300/0 conn(0x7fb2cc0cd180 0x7fb2cc15b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 --2- 172.21.15.196:0/1872594527 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc0ccc40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- 172.21.15.196:0/1872594527 >> 172.21.15.196:0/1872594527 conn(0x7fb2cc099a70 msgr2=0x7fb2cc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- 172.21.15.196:0/1872594527 shutdown_connections 2024-04-25T10:01:44.250 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- 172.21.15.196:0/1872594527 wait complete. 2024-04-25T10:01:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 Processor -- start 2024-04-25T10:01:44.251 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.246+0000 7fb2d151a640 1 -- start start 2024-04-25T10:01:44.253 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc0cf0e0 0x7fb2cc151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc1f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0cb9a0 con 0x7fb2cc15aac0 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0cb820 con 0x7fb2cc0cf0e0 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2cc0cbb20 con 0x7fb2cc0cd180 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2ca7fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc0cf0e0 0x7fb2cc151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:54984/0 (socket says 172.21.15.196:54984) 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 -- 172.21.15.196:0/2869073239 learned_addr learned my addr 172.21.15.196:0/2869073239 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2cb7fe640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc1f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 -- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc0cf0e0 msgr2=0x7fb2cc151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc0cf0e0 0x7fb2cc151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.254 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 -- 172.21.15.196:0/2869073239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2cc15aac0 msgr2=0x7fb2cc1f7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc1f7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 -- 172.21.15.196:0/2869073239 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc1fa780 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2caffd640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0af040 tx=0x7fb2bc019280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0b1040 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc0ba070 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2cc1faa60 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0a8040 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2cc151950 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb2bc0b58a0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb2bc0bfba0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.255 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.250+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2cc04efa0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.256 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.254+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb2bc0160d0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.294 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.288+0000 7f1c0921b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:44.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.326+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f", "id": 4} v 0) v1 -- 0x7fb2cc0cdc90 con 0x7fb2cc0cd180 2024-04-25T10:01:44.336 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2c37fe640 1 -- 172.21.15.196:0/2869073239 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb2bc0b5cd0 con 0x7fb2cc0cd180 2024-04-25T10:01:44.336 INFO:teuthology.orchestra.run.smithi196.stdout:4 2024-04-25T10:01:44.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 msgr2=0x7fb2cc1fa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.337 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc0af040 tx=0x7fb2bc019280 comp rx=0 tx=0).stop 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 shutdown_connections 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2cc15aac0 0x7fb2cc1f7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2cc0cf0e0 0x7fb2cc151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 --2- 172.21.15.196:0/2869073239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2cc0cd180 0x7fb2cc1fa0a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 >> 172.21.15.196:0/2869073239 conn(0x7fb2cc099a70 msgr2=0x7fb2cc15aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 shutdown_connections 2024-04-25T10:01:44.338 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.334+0000 7fb2d151a640 1 -- 172.21.15.196:0/2869073239 wait complete. 2024-04-25T10:01:44.346 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.340+0000 7f1c0921b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:44.353 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 2a95d237-1bf2-424b-b5d4-b3e5ae3ee189 5 2024-04-25T10:01:44.422 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.416+0000 7f1c0921b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:44.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.466+0000 7f5fabfbd640 1 Processor -- start 2024-04-25T10:01:44.470 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.466+0000 7f5fabfbd640 1 -- start start 2024-04-25T10:01:44.475 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.466+0000 7f5fabfbd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 0x7f5fa40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.466+0000 7f5fabfbd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5fa40d1cd0 0x7f5fa40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fabfbd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa406be60 con 0x7f5fa40d0620 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fabfbd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa406c440 con 0x7f5fa40d1cd0 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fabfbd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa415de60 con 0x7f5fa415da80 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5faa55a640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5fa40d0620 0x7f5fa40ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:60878/0 (socket says 172.21.15.196:60878) 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5faa55a640 1 -- 172.21.15.196:0/1299946658 learned_addr learned my addr 172.21.15.196:0/1299946658 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/1299946658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 0x7f5fa40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9558640 1 --2- 172.21.15.196:0/1299946658 >> v2:172.21.15.82:3300/0 conn(0x7f5fa40d1cd0 0x7f5fa40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa8d57640 1 -- 172.21.15.196:0/1299946658 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369053242 0 0) 0x7f5fa406be60 con 0x7f5fa40d0620 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa8d57640 1 -- 172.21.15.196:0/1299946658 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f90003530 con 0x7f5fa40d0620 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 -- 172.21.15.196:0/1299946658 >> v2:172.21.15.82:3300/0 conn(0x7f5fa40d1cd0 msgr2=0x7f5fa40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/1299946658 >> v2:172.21.15.82:3300/0 conn(0x7f5fa40d1cd0 0x7f5fa40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.476 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 -- 172.21.15.196:0/1299946658 >> v1:172.21.15.196:6789/0 conn(0x7f5fa40d0620 legacy=0x7f5fa40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 -- 172.21.15.196:0/1299946658 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa40d2450 con 0x7f5fa415da80 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/1299946658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 0x7f5fa40d4230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097c0 tx=0x7f5fa001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=980f00f721d75590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa8d57640 1 -- 172.21.15.196:0/1299946658 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00b0070 con 0x7f5fa415da80 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa8d57640 1 -- 172.21.15.196:0/1299946658 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa00ab600 con 0x7f5fa415da80 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.470+0000 7f5fa8d57640 1 -- 172.21.15.196:0/1299946658 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00ab9d0 con 0x7f5fa415da80 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- 172.21.15.196:0/1299946658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 msgr2=0x7f5fa40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/1299946658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 0x7f5fa40d4230 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5fa00097c0 tx=0x7f5fa001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- 172.21.15.196:0/1299946658 shutdown_connections 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/1299946658 >> v2:172.21.15.82:3300/0 conn(0x7f5fa40d1cd0 0x7f5fa40d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/1299946658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa415da80 0x7f5fa40d4230 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- 172.21.15.196:0/1299946658 >> 172.21.15.196:0/1299946658 conn(0x7f5fa408b000 msgr2=0x7f5fa40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- 172.21.15.196:0/1299946658 shutdown_connections 2024-04-25T10:01:44.477 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- 172.21.15.196:0/1299946658 wait complete. 2024-04-25T10:01:44.478 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.468+0000 7f1c0921b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:44.480 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 Processor -- start 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- start start 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa40d1cd0 0x7f5fa40c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 0x7f5fa4155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa416a730 con 0x7f5fa40d1cd0 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa416a5b0 con 0x7f5fa415da80 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fabfbd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa416a8b0 con 0x7f5fa40d0620 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fa9d59640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fa9d59640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:55008/0 (socket says 172.21.15.196:55008) 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5faa55a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 0x7f5fa4155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fa9d59640 1 -- 172.21.15.196:0/2335669152 learned_addr learned my addr 172.21.15.196:0/2335669152 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.474+0000 7f5fa9558640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa40d1cd0 0x7f5fa40c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.481 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 -- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 msgr2=0x7f5fa4155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 0x7f5fa4155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 -- 172.21.15.196:0/2335669152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa40d1cd0 msgr2=0x7f5fa40c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa40d1cd0 0x7f5fa40c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 -- 172.21.15.196:0/2335669152 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa4158e20 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5faa55a640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 0x7f5fa4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fa9d59640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5fa001a0d0 tx=0x7f5fa0019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa00b0040 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa00bd070 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0006e20 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa41593b0 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa4159970 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5fa00b3070 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5fa00c29e0 con 0x7f5fa40d0620 2024-04-25T10:01:44.482 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa404efa0 con 0x7f5fa40d0620 2024-04-25T10:01:44.483 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.478+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5fa0016090 con 0x7f5fa40d0620 2024-04-25T10:01:44.547 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.546+0000 7f2c9abd9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:44.550 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.546+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2a95d237-1bf2-424b-b5d4-b3e5ae3ee189", "id": 5} v 0) v1 -- 0x7f5fa40d39d0 con 0x7f5fa40d0620 2024-04-25T10:01:44.556 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5f9a7fc640 1 -- 172.21.15.196:0/2335669152 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2a95d237-1bf2-424b-b5d4-b3e5ae3ee189", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5fa00c2460 con 0x7f5fa40d0620 2024-04-25T10:01:44.556 INFO:teuthology.orchestra.run.smithi196.stdout:5 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 msgr2=0x7f5fa4158510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5fa001a0d0 tx=0x7f5fa0019240 comp rx=0 tx=0).stop 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 shutdown_connections 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5fa415da80 0x7f5fa4155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5fa40d1cd0 0x7f5fa40c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 --2- 172.21.15.196:0/2335669152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5fa40d0620 0x7f5fa4158510 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 >> 172.21.15.196:0/2335669152 conn(0x7f5fa408b000 msgr2=0x7f5fa40b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 shutdown_connections 2024-04-25T10:01:44.558 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.554+0000 7f5fabfbd640 1 -- 172.21.15.196:0/2335669152 wait complete. 2024-04-25T10:01:44.573 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new da7f4dc6-f8de-4cce-9987-dd7b65760a8f 6 2024-04-25T10:01:44.604 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.602+0000 7f2c9abd9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:44.622 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.616+0000 7f1c0921b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:44.659 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.654+0000 7f2c9abd9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:44.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 Processor -- start 2024-04-25T10:01:44.677 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 -- start start 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6c415aac0 0x7fe6c40ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40ce0d0 con 0x7fe6c415aac0 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6c40ce320 con 0x7fe6c415aea0 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c984d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c40ce600 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c37fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c37fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43708/0 (socket says 172.21.15.196:43708) 2024-04-25T10:01:44.679 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c37fe640 1 -- 172.21.15.196:0/4041361914 learned_addr learned my addr 172.21.15.196:0/4041361914 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c2ffd640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.82:3300/0 conn(0x7fe6c415aac0 0x7fe6c40ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 -- 172.21.15.196:0/4041361914 >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 msgr2=0x7fe6c415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 -- 172.21.15.196:0/4041361914 >> v2:172.21.15.82:3300/0 conn(0x7fe6c415aac0 msgr2=0x7fe6c40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.82:3300/0 conn(0x7fe6c415aac0 0x7fe6c40ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6ba7fc640 1 -- 172.21.15.196:0/4041361914 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c40ce8e0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.680 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.674+0000 7fe6c37fe640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:44.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6ba7fc640 1 --2- 172.21.15.196:0/4041361914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40cd180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6c415e1b0 tx=0x7fe6ac01eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4b535a056d61cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c27fc640 1 -- 172.21.15.196:0/4041361914 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0b4070 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c27fc640 1 -- 172.21.15.196:0/4041361914 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0a7af0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.681 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c27fc640 1 -- 172.21.15.196:0/4041361914 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0af440 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- 172.21.15.196:0/4041361914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 msgr2=0x7fe6c40cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 --2- 172.21.15.196:0/4041361914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40cd180 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe6c415e1b0 tx=0x7fe6ac01eb50 comp rx=0 tx=0).stop 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- 172.21.15.196:0/4041361914 shutdown_connections 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.196:3300/0 conn(0x7fe6c415aea0 0x7fe6c415b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 --2- 172.21.15.196:0/4041361914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40cd180 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 --2- 172.21.15.196:0/4041361914 >> v2:172.21.15.82:3300/0 conn(0x7fe6c415aac0 0x7fe6c40ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- 172.21.15.196:0/4041361914 >> 172.21.15.196:0/4041361914 conn(0x7fe6c4099a70 msgr2=0x7fe6c40d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- 172.21.15.196:0/4041361914 shutdown_connections 2024-04-25T10:01:44.682 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- 172.21.15.196:0/4041361914 wait complete. 2024-04-25T10:01:44.683 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:44.676+0000 7f1c0921b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:44.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 Processor -- start 2024-04-25T10:01:44.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.678+0000 7fe6c984d640 1 -- start start 2024-04-25T10:01:44.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c415aac0 0x7fe6c41500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.685 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 0x7fe6c40b45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415ee10 con 0x7fe6c415aac0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415ec90 con 0x7fe6c415aea0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c415ef90 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c37fe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 0x7fe6c40b45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:55016/0 (socket says 172.21.15.196:55016) 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 -- 172.21.15.196:0/672222614 learned_addr learned my addr 172.21.15.196:0/672222614 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6ba7fc640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c415aac0 0x7fe6c41500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 -- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 msgr2=0x7fe6c40b45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 0x7fe6c40b45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 -- 172.21.15.196:0/672222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c415aac0 msgr2=0x7fe6c41500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c415aac0 0x7fe6c41500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.686 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 -- 172.21.15.196:0/672222614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c415b380 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c37fe640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 0x7fe6c40b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c2ffd640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400d800 tx=0x7fe6b400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400b660 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4019070 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c40b5090 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b400ba00 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c40ad260 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe6b400bba0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fe6b4014b10 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.687 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.682+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c404efa0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.688 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.686+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe6b4014dc0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.739 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.738+0000 7f2c9abd9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:44.753 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.750+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_command({"prefix": "osd new", "uuid": "da7f4dc6-f8de-4cce-9987-dd7b65760a8f", "id": 6} v 0) v1 -- 0x7fe6c40b6480 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.759 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.754+0000 7fe6c0ff9640 1 -- 172.21.15.196:0/672222614 <== mon.2 v2:172.21.15.196:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da7f4dc6-f8de-4cce-9987-dd7b65760a8f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe6b40145f0 con 0x7fe6c40cf0e0 2024-04-25T10:01:44.759 INFO:teuthology.orchestra.run.smithi196.stdout:6 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 msgr2=0x7fe6c40add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400d800 tx=0x7fe6b400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 shutdown_connections 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6c415aea0 0x7fe6c40b45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6c415aac0 0x7fe6c41500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 --2- 172.21.15.196:0/672222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6c40cf0e0 0x7fe6c40add40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 >> 172.21.15.196:0/672222614 conn(0x7fe6c4099a70 msgr2=0x7fe6c415be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.760 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 shutdown_connections 2024-04-25T10:01:44.761 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.758+0000 7fe6c984d640 1 -- 172.21.15.196:0/672222614 wait complete. 2024-04-25T10:01:44.775 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --cluster ceph osd new 0c42460a-e6a1-45e0-95a9-2f8ffb3ff016 7 2024-04-25T10:01:44.795 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.794+0000 7f2c9abd9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:44.876 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.873+0000 7f2c9abd9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:44.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 Processor -- start 2024-04-25T10:01:44.914 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 -- start start 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e84066e30 0x7f7e8406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8406ec30 con 0x7f7e8406bca0 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e8406c5c0 con 0x7f7e840af370 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.909+0000 7f7e8a050640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e840669a0 con 0x7f7e84066e30 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e83fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7e840af370 0x7f7e840ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.196:60880/0 (socket says 172.21.15.196:60880) 2024-04-25T10:01:44.917 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e83fff640 1 -- 172.21.15.196:0/3804094954 learned_addr learned my addr 172.21.15.196:0/3804094954 (peer_addr_for_me v1:172.21.15.196:0/0) 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/3804094954 >> v2:172.21.15.82:3300/0 conn(0x7f7e84066e30 0x7f7e8406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960522434 0 0) 0x7f7e8406c5c0 con 0x7f7e840af370 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e6c003530 con 0x7f7e840af370 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2262443883 0 0) 0x7f7e6c003530 con 0x7f7e840af370 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 >> v2:172.21.15.82:3300/0 conn(0x7f7e84066e30 msgr2=0x7f7e8406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 --2- 172.21.15.196:0/3804094954 >> v2:172.21.15.82:3300/0 conn(0x7f7e84066e30 0x7f7e8406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 >> v1:172.21.15.82:6789/0 conn(0x7f7e8406bca0 legacy=0x7f7e840cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8406f230 con 0x7f7e840af370 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/3804094954 >> v2:172.21.15.82:3300/0 conn(0x7f7e84066e30 0x7f7e8406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7e74001680 con 0x7f7e840af370 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 >> v1:172.21.15.196:6789/0 conn(0x7f7e840af370 legacy=0x7f7e840ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 0x7f7e6c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.918 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e6c004810 0x7f7e6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c005380 0x7f7e6c00b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840669a0 con 0x7f7e6c004810 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c00c2d0 con 0x7f7e6c003a30 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c00c4d0 con 0x7f7e6c005380 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 0x7f7e6c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e83fff640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c005380 0x7f7e6c00b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e6c004810 0x7f7e6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 -- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c005380 msgr2=0x7f7e6c00b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c005380 0x7f7e6c00b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 -- 172.21.15.196:0/3804094954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e6c004810 msgr2=0x7f7e6c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e6c004810 0x7f7e6c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 -- 172.21.15.196:0/3804094954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c003530 con 0x7f7e6c003a30 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e83fff640 1 -- 172.21.15.196:0/3804094954 reap_dead start 2024-04-25T10:01:44.919 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e6c004810 0x7f7e6c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e837fe640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 0x7f7e6c003e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c001e70 tx=0x7f7e7c01c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f20f9339f0f6fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c001a90 con 0x7f7e6c003a30 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c003b80 con 0x7f7e6c003a30 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.913+0000 7f7e827fc640 1 -- 172.21.15.196:0/3804094954 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c005080 con 0x7f7e6c003a30 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 msgr2=0x7f7e6c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 0x7f7e6c003e50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c001e70 tx=0x7f7e7c01c730 comp rx=0 tx=0).stop 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/3804094954 shutdown_connections 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 --2- 172.21.15.196:0/3804094954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e6c003a30 0x7f7e6c003e50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/3804094954 >> 172.21.15.196:0/3804094954 conn(0x7f7e84071810 msgr2=0x7f7e84071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.923 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/3804094954 shutdown_connections 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/3804094954 wait complete. 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 Processor -- start 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- start start 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e84151c60 0x7f7e84154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8415cc80 con 0x7f7e84151c60 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8415cb00 con 0x7f7e84155f50 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8406e080 con 0x7f7e6c003a30 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e837fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.196:43726/0 (socket says 172.21.15.196:43726) 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e837fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.196:55048/0 (socket says 172.21.15.196:55048) 2024-04-25T10:01:44.924 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 -- 172.21.15.196:0/238057448 learned_addr learned my addr 172.21.15.196:0/238057448 (peer_addr_for_me v2:172.21.15.196:0/0) 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e83fff640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e84151c60 0x7f7e84154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 -- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 msgr2=0x7f7e8414e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 -- 172.21.15.196:0/238057448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e84151c60 msgr2=0x7f7e84154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e84151c60 0x7f7e84154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 -- 172.21.15.196:0/238057448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e841552d0 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e837fe640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e82ffd640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e780108a0 tx=0x7f7e78010d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7801a020 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7800eb60 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e841555b0 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7800f040 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84154ed0 con 0x7f7e84155f50 2024-04-25T10:01:44.925 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7e7801d020 con 0x7f7e84155f50 2024-04-25T10:01:44.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f7e78016cb0 con 0x7f7e84155f50 2024-04-25T10:01:44.926 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.917+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8404efa0 con 0x7f7e84155f50 2024-04-25T10:01:44.927 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.921+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7e78024020 con 0x7f7e84155f50 2024-04-25T10:01:44.933 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:44.929+0000 7f2c9abd9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:44.991 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.985+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0c42460a-e6a1-45e0-95a9-2f8ffb3ff016", "id": 7} v 0) v1 -- 0x7f7e84072400 con 0x7f7e84155f50 2024-04-25T10:01:44.996 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e6bfff640 1 -- 172.21.15.196:0/238057448 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0c42460a-e6a1-45e0-95a9-2f8ffb3ff016", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7e780159e0 con 0x7f7e84155f50 2024-04-25T10:01:44.996 INFO:teuthology.orchestra.run.smithi196.stdout:7 2024-04-25T10:01:44.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 msgr2=0x7f7e84154450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:44.997 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7e780108a0 tx=0x7f7e78010d60 comp rx=0 tx=0).stop 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 shutdown_connections 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e84151c60 0x7f7e84154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e84155f50 0x7f7e84154450 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 --2- 172.21.15.196:0/238057448 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e6c003a30 0x7f7e8414e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 >> 172.21.15.196:0/238057448 conn(0x7f7e84071810 msgr2=0x7f7e84158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 shutdown_connections 2024-04-25T10:01:44.998 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:01:44.993+0000 7f7e8a050640 1 -- 172.21.15.196:0/238057448 wait complete. 2024-04-25T10:01:45.012 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:01:45.013 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:01:45.015 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:01:45.015 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:01:45.015 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:01:45.018 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:01:45.018 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:01:45.018 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:01:45.020 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:01:45.021 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:01:45.021 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:01:45.023 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:01:45.023 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:01:45.023 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:01:45.025 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:01:45.025 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:01:45.025 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:01:45.027 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:01:45.028 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:01:45.028 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:01:45.030 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:01:45.030 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:01:45.030 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:01:45.032 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:01:45.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:45.095 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:45.089+0000 7f2c9abd9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:45.160 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:45.157+0000 7f2c9abd9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.200+0000 7f6e1bdf8640 1 Processor -- start 2024-04-25T10:01:45.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.200+0000 7f6e1bdf8640 1 -- start start 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1bdf8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6e1415b920 0x7f6e14099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1bdf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e140cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1bdf8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e1409a3c0 con 0x7f6e140b0440 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1bdf8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e140cd680 con 0x7f6e1415b920 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1bdf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1409a950 con 0x7f6e14099fb0 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6e140b0440 0x7f6e140aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44416/0 (socket says 172.21.15.82:44416) 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 -- 172.21.15.82:0/2755012691 learned_addr learned my addr 172.21.15.82:0/2755012691 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1a395640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e140cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19b94640 1 --2- 172.21.15.82:0/2755012691 >> v2:172.21.15.196:3300/0 conn(0x7f6e1415b920 0x7f6e14099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113178658 0 0) 0x7f6e10001120 con 0x7f6e140b0440 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e00003530 con 0x7f6e140b0440 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3170960210 0 0) 0x7f6e00003530 con 0x7f6e140b0440 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 msgr2=0x7f6e140cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e140cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 >> v2:172.21.15.196:3300/0 conn(0x7f6e1415b920 msgr2=0x7f6e14099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 --2- 172.21.15.82:0/2755012691 >> v2:172.21.15.196:3300/0 conn(0x7f6e1415b920 0x7f6e14099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e1409ac30 con 0x7f6e140b0440 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6e10001850 con 0x7f6e140b0440 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 >> v1:172.21.15.82:6789/0 conn(0x7f6e140b0440 legacy=0x7f6e140aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e00003a30 0x7f6e00003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140cd680 con 0x7f6e00004810 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1409a950 con 0x7f6e00003a30 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0000b9d0 con 0x7f6e14099fb0 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e1a395640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19b94640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e00003a30 0x7f6e00003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 -- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 msgr2=0x7f6e000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14099fb0 0x7f6e000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 -- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e00003a30 msgr2=0x7f6e00003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e00003a30 0x7f6e00003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 -- 172.21.15.82:0/2755012691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00003530 con 0x7f6e00004810 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.204+0000 7f6e19393640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e00004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000c410 tx=0x7f6e1001f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d93c1ea88fa14f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1000c0e0 con 0x7f6e00004810 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e10005ae0 con 0x7f6e00004810 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e18b92640 1 -- 172.21.15.82:0/2755012691 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10007070 con 0x7f6e00004810 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/2755012691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 msgr2=0x7f6e00004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 --2- 172.21.15.82:0/2755012691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e00004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6e1000c410 tx=0x7f6e1001f400 comp rx=0 tx=0).stop 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1a395640 1 -- 172.21.15.82:0/2755012691 reap_dead start 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/2755012691 shutdown_connections 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/2755012691 >> 172.21.15.82:0/2755012691 conn(0x7f6e140b3010 msgr2=0x7f6e140b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/2755012691 shutdown_connections 2024-04-25T10:01:45.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/2755012691 wait complete. 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 Processor -- start 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- start start 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 0x7f6e140b6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 0x7f6e1415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1415bd00 con 0x7f6e00004810 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140ae660 con 0x7f6e14159c20 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e140cd140 con 0x7f6e14159100 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47728/0 (socket says 172.21.15.82:47728) 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 -- 172.21.15.82:0/4176373293 learned_addr learned my addr 172.21.15.82:0/4176373293 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19393640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 0x7f6e140b6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1a395640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 0x7f6e1415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 -- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 msgr2=0x7f6e140b6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 0x7f6e140b6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 -- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 msgr2=0x7f6e1415a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 0x7f6e1415a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 -- 172.21.15.82:0/4176373293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e140b4720 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1a395640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 0x7f6e1415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19393640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 0x7f6e140b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e19b94640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e040115e0 tx=0x7f6e04011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04015870 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e04015a10 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e04015dc0 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e140b4a00 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e140b4fc0 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f6e0400f480 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.208+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6e0400f770 con 0x7f6e00004810 2024-04-25T10:01:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.212+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1415a5e0 con 0x7f6e00004810 2024-04-25T10:01:45.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.212+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6e04019030 con 0x7f6e00004810 2024-04-25T10:01:45.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e1404efa0 con 0x7f6e00004810 2024-04-25T10:01:45.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e0a7fc640 1 -- 172.21.15.82:0/4176373293 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f6e0402a020 con 0x7f6e00004810 2024-04-25T10:01:45.276 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:01:45.276 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":10,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:44.990012+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 msgr2=0x7f6e140b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e040115e0 tx=0x7f6e04011aa0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 shutdown_connections 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e14159100 0x7f6e140b6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e14159c20 0x7f6e1415a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 --2- 172.21.15.82:0/4176373293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e00004810 0x7f6e140b4140 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:45.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 >> 172.21.15.82:0/4176373293 conn(0x7f6e140b3010 msgr2=0x7f6e140c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:45.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 shutdown_connections 2024-04-25T10:01:45.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:01:45.268+0000 7f6e1bdf8640 1 -- 172.21.15.82:0/4176373293 wait complete. 2024-04-25T10:01:45.291 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:01:45.291 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:01:45.381 INFO:tasks.ceph.mgr.y.smithi082.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:01:45.381 INFO:tasks.ceph.mgr.y.smithi082.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:01:45.381 INFO:tasks.ceph.mgr.y.smithi082.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:01:45.421 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:45.412+0000 7f1c0921b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:45.439 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-04-25T10:01:45.432+0000 7f58f5f6f7c0 -1 Falling back to public interface 2024-04-25T10:01:45.439 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-04-25T10:01:45.432+0000 7f585c6c37c0 -1 Falling back to public interface 2024-04-25T10:01:45.565 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:45.556+0000 7f1c0921b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:45.645 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T10:01:45.641+0000 7f8a1a8a87c0 -1 Falling back to public interface 2024-04-25T10:01:45.645 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T10:01:45.641+0000 7feaf77747c0 -1 Falling back to public interface 2024-04-25T10:01:45.645 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T10:01:45.641+0000 7feaf6734640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T10:01:45.665 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T10:01:45.661+0000 7f6848ebe7c0 -1 Falling back to public interface 2024-04-25T10:01:45.668 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T10:01:45.665+0000 7f24c50847c0 -1 Falling back to public interface 2024-04-25T10:01:45.683 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-04-25T10:01:45.676+0000 7fc9e9de07c0 -1 Falling back to public interface 2024-04-25T10:01:45.703 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-04-25T10:01:45.696+0000 7f12199d27c0 -1 Falling back to public interface 2024-04-25T10:01:45.718 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:45.712+0000 7f1c0921b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:45.783 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:45.776+0000 7f1c0921b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:45.914 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:45.908+0000 7f1c0921b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:45.986 INFO:tasks.ceph.mgr.x.smithi196.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T10:01:45.986 INFO:tasks.ceph.mgr.x.smithi196.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T10:01:45.986 INFO:tasks.ceph.mgr.x.smithi196.stderr: from numpy import show_config as show_numpy_config 2024-04-25T10:01:46.039 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:46.033+0000 7f2c9abd9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:46.096 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:46.188 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:46.185+0000 7f2c9abd9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:46.201 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.192+0000 7f1ee517f640 1 Processor -- start 2024-04-25T10:01:46.202 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.192+0000 7f1ee517f640 1 -- start start 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ee517f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 0x7f1ee00ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ee517f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ee006bca0 0x7f1ee006c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ee517f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee015a120 con 0x7f1ee0159d40 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ee517f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ee006c640 con 0x7f1ee006bca0 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ee517f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00aea90 con 0x7f1ee00b3710 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ee006bca0 0x7f1ee006c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ee006bca0 0x7f1ee006c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47734/0 (socket says 172.21.15.82:47734) 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 -- 172.21.15.82:0/695839585 learned_addr learned my addr 172.21.15.82:0/695839585 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ede59c640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 0x7f1ee00ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 -- 172.21.15.82:0/695839585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 msgr2=0x7f1ee00ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 0x7f1ee00ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.205 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 -- 172.21.15.82:0/695839585 >> v1:172.21.15.196:6789/0 conn(0x7f1ee0159d40 legacy=0x7f1ee00aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:46.205 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1edf59e640 1 -- 172.21.15.82:0/695839585 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee00ade80 con 0x7f1ee006bca0 2024-04-25T10:01:46.205 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.196+0000 7f1ede59c640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 0x7f1ee00ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:46.207 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1edf59e640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee006bca0 0x7f1ee006c100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d560b8632c53a2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.207 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1eddd9b640 1 -- 172.21.15.82:0/695839585 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80a56c0 con 0x7f1ee006bca0 2024-04-25T10:01:46.207 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1eddd9b640 1 -- 172.21.15.82:0/695839585 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed80a5860 con 0x7f1ee006bca0 2024-04-25T10:01:46.207 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1eddd9b640 1 -- 172.21.15.82:0/695839585 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed80a5c30 con 0x7f1ee006bca0 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- 172.21.15.82:0/695839585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee006bca0 msgr2=0x7f1ee006c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee006bca0 0x7f1ee006c100 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8009e50 tx=0x7f1ed801c970 comp rx=0 tx=0).stop 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- 172.21.15.82:0/695839585 shutdown_connections 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee00b3710 0x7f1ee00ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 --2- 172.21.15.82:0/695839585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee006bca0 0x7f1ee006c100 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- 172.21.15.82:0/695839585 >> 172.21.15.82:0/695839585 conn(0x7f1ee00718f0 msgr2=0x7f1ee0071d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- 172.21.15.82:0/695839585 shutdown_connections 2024-04-25T10:01:46.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- 172.21.15.82:0/695839585 wait complete. 2024-04-25T10:01:46.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 Processor -- start 2024-04-25T10:01:46.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.200+0000 7f1ee517f640 1 -- start start 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee006bca0 0x7f1ee0142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 0x7f1ee01452d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00adc60 con 0x7f1ee00b3710 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee00adae0 con 0x7f1ee0159d40 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee006d800 con 0x7f1ee006bca0 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47746/0 (socket says 172.21.15.82:47746) 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 -- 172.21.15.82:0/1717494616 learned_addr learned my addr 172.21.15.82:0/1717494616 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1eded9d640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee006bca0 0x7f1ee0142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.212 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1edf59e640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 0x7f1ee01452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 -- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee006bca0 msgr2=0x7f1ee0142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee006bca0 0x7f1ee0142710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 -- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 msgr2=0x7f1ee01452d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 0x7f1ee01452d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 -- 172.21.15.82:0/1717494616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee015d250 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1edf59e640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 0x7f1ee01452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ede59c640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400d6f0 tx=0x7f1ed400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400b040 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed400b9c0 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee015cf70 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed400bdb0 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee0143250 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f1ed4011460 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1ed40233f0 con 0x7f1ee00b3710 2024-04-25T10:01:46.213 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.204+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee01484b0 con 0x7f1ee00b3710 2024-04-25T10:01:46.214 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.208+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f1ed4015030 con 0x7f1ee00b3710 2024-04-25T10:01:46.270 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee0074d80 con 0x7f1ee00b3710 2024-04-25T10:01:46.271 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ec77fe640 1 -- 172.21.15.82:0/1717494616 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f1ed4011930 con 0x7f1ee00b3710 2024-04-25T10:01:46.271 INFO:teuthology.misc.health.smithi082.stdout: 2024-04-25T10:01:46.271 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":10,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:44.990012+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:46.272 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 msgr2=0x7f1ee01f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400d6f0 tx=0x7f1ed400dbb0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 shutdown_connections 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ee0159d40 0x7f1ee01452d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ee00b3710 0x7f1ee01f49f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 --2- 172.21.15.82:0/1717494616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ee006bca0 0x7f1ee0142710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 >> 172.21.15.82:0/1717494616 conn(0x7f1ee00718f0 msgr2=0x7f1ee01489c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 shutdown_connections 2024-04-25T10:01:46.273 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:46.264+0000 7f1ee517f640 1 -- 172.21.15.82:0/1717494616 wait complete. 2024-04-25T10:01:46.287 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:46.343 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:46.337+0000 7f2c9abd9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:46.416 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:46.413+0000 7f2c9abd9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:46.549 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:46.545+0000 7f2c9abd9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:46.556 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:46.548+0000 7f1c0921b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:46.631 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:46.624+0000 7f1c0921b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:46.811 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:46.804+0000 7f1c0921b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:46.885 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:46.876+0000 7f1c0921b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:47.172 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:47.164+0000 7f1c0921b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:47.175 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.173+0000 7f2c9abd9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:47.233 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:47.224+0000 7f1c0921b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:47.251 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.245+0000 7f2c9abd9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:47.435 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.429+0000 7f2c9abd9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:47.508 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.505+0000 7f2c9abd9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:47.799 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.793+0000 7f2c9abd9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:47.861 INFO:tasks.ceph.mgr.x.smithi196.stderr:2024-04-25T10:01:47.857+0000 7f2c9abd9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:48.270 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:48.264+0000 7f1bb0769640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:01:48.271 INFO:tasks.ceph.mgr.y.smithi082.stderr:2024-04-25T10:01:48.264+0000 7f1bb0769640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:01:49.029 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-04-25T10:01:49.020+0000 7f58f5f6f7c0 -1 osd.0 0 log_to_monitors true 2024-04-25T10:01:49.257 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-04-25T10:01:49.248+0000 7f585c6c37c0 -1 osd.2 0 log_to_monitors true 2024-04-25T10:01:49.548 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-04-25T10:01:49.540+0000 7fc9e9de07c0 -1 osd.1 0 log_to_monitors true 2024-04-25T10:01:49.554 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T10:01:49.549+0000 7f6848ebe7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T10:01:49.564 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-04-25T10:01:49.556+0000 7f12199d27c0 -1 osd.3 0 log_to_monitors true 2024-04-25T10:01:49.565 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T10:01:49.561+0000 7f24c50847c0 -1 osd.7 0 log_to_monitors true 2024-04-25T10:01:49.593 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T10:01:49.589+0000 7f8a1a8a87c0 -1 osd.5 0 log_to_monitors true 2024-04-25T10:01:49.594 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T10:01:49.589+0000 7feaf77747c0 -1 osd.6 0 log_to_monitors true 2024-04-25T10:01:52.955 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T10:01:52.949+0000 7f6844e67640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:01:52.978 INFO:tasks.ceph.osd.4.smithi196.stderr:2024-04-25T10:01:52.973+0000 7f683cc9c640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:52.981 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T10:01:52.977+0000 7f8a16851640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:01:52.991 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T10:01:52.989+0000 7feaf371d640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:01:53.016 INFO:tasks.ceph.osd.5.smithi196.stderr:2024-04-25T10:01:53.013+0000 7f8a0e686640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.020 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-04-25T10:01:53.012+0000 7f58f1f18640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:01:53.040 INFO:tasks.ceph.osd.6.smithi196.stderr:2024-04-25T10:01:53.037+0000 7feaeb552640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.045 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-04-25T10:01:53.036+0000 7fc9e5d89640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:01:53.052 INFO:tasks.ceph.osd.0.smithi082.stderr:2024-04-25T10:01:53.044+0000 7f58e9d4d640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.060 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T10:01:53.057+0000 7f24c102d640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:01:53.070 INFO:tasks.ceph.osd.1.smithi082.stderr:2024-04-25T10:01:53.064+0000 7fc9ddbbe640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.084 INFO:tasks.ceph.osd.7.smithi196.stderr:2024-04-25T10:01:53.081+0000 7f24b8e62640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.093 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:53.106 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-04-25T10:01:53.100+0000 7f585866c640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:01:53.126 INFO:tasks.ceph.osd.2.smithi082.stderr:2024-04-25T10:01:53.120+0000 7f58504a1640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.157 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-04-25T10:01:53.148+0000 7f121597b640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:01:53.198 INFO:tasks.ceph.osd.3.smithi082.stderr:2024-04-25T10:01:53.192+0000 7f120d7b0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:53.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 Processor -- start 2024-04-25T10:01:53.204 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 -- start start 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20b006e870 0x7f20b015a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f20b00b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b00b44d0 con 0x7f20b00cbd90 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b00b3c30 con 0x7f20b006e870 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b822c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b3f10 con 0x7f20b006ec50 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b67c9640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f20b00cbd90 0x7f20b015d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44424/0 (socket says 172.21.15.82:44424) 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.196+0000 7f20b67c9640 1 -- 172.21.15.82:0/2055487657 learned_addr learned my addr 172.21.15.82:0/2055487657 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2055487657 >> v2:172.21.15.196:3300/0 conn(0x7f20b006e870 0x7f20b015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f20b00b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.208 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436234066 0 0) 0x7f20a8001120 con 0x7f20b00cbd90 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2098003450 con 0x7f20b00cbd90 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1890193510 0 0) 0x7f2098003450 con 0x7f20b00cbd90 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 msgr2=0x7f20b00b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f20b00b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 >> v2:172.21.15.196:3300/0 conn(0x7f20b006e870 msgr2=0x7f20b015a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 --2- 172.21.15.82:0/2055487657 >> v2:172.21.15.196:3300/0 conn(0x7f20b006e870 0x7f20b015a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00b41f0 con 0x7f20b00cbd90 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f20a8001850 con 0x7f20b00cbd90 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 >> v1:172.21.15.82:6789/0 conn(0x7f20b00cbd90 legacy=0x7f20b015d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2098003ab0 0x7f2098003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2098008860 0x7f209800ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.209 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f209800d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b3c30 con 0x7f2098008860 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00b3f10 con 0x7f2098003ab0 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209800db60 con 0x7f20b006ec50 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2098008860 0x7f209800ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2098003ab0 0x7f2098003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b67c9640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f209800d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 -- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 msgr2=0x7f209800d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f209800d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 -- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2098003ab0 msgr2=0x7f2098003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2098003ab0 0x7f2098003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 -- 172.21.15.82:0/2055487657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098003450 con 0x7f2098008860 2024-04-25T10:01:53.210 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b67c9640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b006ec50 0x7f209800d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:53.214 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2098008860 0x7f209800ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009fa0 tx=0x7f20a001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53383d3bd5c5e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b2070 con 0x7f2098008860 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a00a5c00 con 0x7f2098008860 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b4fc6640 1 -- 172.21.15.82:0/2055487657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0009bf0 con 0x7f2098008860 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 -- 172.21.15.82:0/2055487657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2098008860 msgr2=0x7f209800ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 --2- 172.21.15.82:0/2055487657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2098008860 0x7f209800ac90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009fa0 tx=0x7f20a001cc40 comp rx=0 tx=0).stop 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b67c9640 1 -- 172.21.15.82:0/2055487657 reap_dead start 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 -- 172.21.15.82:0/2055487657 shutdown_connections 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 -- 172.21.15.82:0/2055487657 >> 172.21.15.82:0/2055487657 conn(0x7f20b00cfb90 msgr2=0x7f20b00cff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 -- 172.21.15.82:0/2055487657 shutdown_connections 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.200+0000 7f20b822c640 1 -- 172.21.15.82:0/2055487657 wait complete. 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 Processor -- start 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- start start 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 0x7f20b00c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.215 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 0x7f20b014ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006c4e0 con 0x7f20b01f4a10 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006c360 con 0x7f20b015a5f0 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b006c660 con 0x7f2098008860 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47860/0 (socket says 172.21.15.82:47860) 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 -- 172.21.15.82:0/2246615131 learned_addr learned my addr 172.21.15.82:0/2246615131 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 0x7f20b00c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 0x7f20b014ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 -- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 msgr2=0x7f20b00c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 0x7f20b00c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 -- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 msgr2=0x7f20b014ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.216 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 0x7f20b014ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 -- 172.21.15.82:0/2246615131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00c93e0 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b57c7640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 0x7f20b014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 0x7f20b00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b67c9640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b2e10 tx=0x7f20a8010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a801a070 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8011040 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b00c96c0 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a800ead0 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b00c9c80 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.204+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b0074d80 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.208+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7f20a801d070 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.208+0000 7f20ae7fc640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 0x7f209c069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.208+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f20a80868b0 con 0x7f20b01f4a10 2024-04-25T10:01:53.217 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.208+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 0x7f209c069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:53.218 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.208+0000 7f20b5fc8640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 0x7f209c069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009b70 tx=0x7f20a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:53.219 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.212+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20b00b44d0 con 0x7f20b01f4a10 2024-04-25T10:01:53.378 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20b00d0980 con 0x7f20b01f4a10 2024-04-25T10:01:53.379 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20ae7fc640 1 -- 172.21.15.82:0/2246615131 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f20a8058040 con 0x7f20b01f4a10 2024-04-25T10:01:53.379 INFO:teuthology.misc.health.smithi082.stdout: 2024-04-25T10:01:53.380 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":12,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:50.762317+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 msgr2=0x7f209c069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 0x7f209c069390 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009b70 tx=0x7f20a4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 msgr2=0x7f20b00ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f20b00b2e10 tx=0x7f20a8010c30 comp rx=0 tx=0).stop 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 shutdown_connections 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f209c066ed0 0x7f209c069390 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b01f4a10 0x7f20b00ca480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.382 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b015a5f0 0x7f20b014ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.383 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 --2- 172.21.15.82:0/2246615131 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2098008860 0x7f20b00c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:53.383 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 >> 172.21.15.82:0/2246615131 conn(0x7f20b00cfb90 msgr2=0x7f20b01e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:53.383 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 shutdown_connections 2024-04-25T10:01:53.383 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:01:53.372+0000 7f20b822c640 1 -- 172.21.15.82:0/2246615131 wait complete. 2024-04-25T10:01:53.397 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:02:00.204 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:00.321 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.312+0000 7f0c0dc58640 1 Processor -- start 2024-04-25T10:02:00.322 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.312+0000 7f0c0dc58640 1 -- start start 2024-04-25T10:02:00.323 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c0dc58640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0c080d2400 0x7f0c080cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.323 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c0dc58640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c080ce700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.323 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c0dc58640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0806c240 con 0x7f0c0815daa0 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c0dc58640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c080d0f80 con 0x7f0c080d2400 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c0dc58640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080d0400 con 0x7f0c080b2b00 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c077fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f0c0815daa0 0x7f0c080d1cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:55090/0 (socket says 172.21.15.82:55090) 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c077fe640 1 -- 172.21.15.82:0/882178526 learned_addr learned my addr 172.21.15.82:0/882178526 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c07fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c080ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.324 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c06ffd640 1 --2- 172.21.15.82:0/882178526 >> v2:172.21.15.196:3300/0 conn(0x7f0c080d2400 0x7f0c080cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c067fc640 1 -- 172.21.15.82:0/882178526 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911629364 0 0) 0x7f0c0806c240 con 0x7f0c0815daa0 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c067fc640 1 -- 172.21.15.82:0/882178526 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf0003530 con 0x7f0c0815daa0 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c07fff640 1 -- 172.21.15.82:0/882178526 >> v2:172.21.15.196:3300/0 conn(0x7f0c080d2400 msgr2=0x7f0c080cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c07fff640 1 --2- 172.21.15.82:0/882178526 >> v2:172.21.15.196:3300/0 conn(0x7f0c080d2400 0x7f0c080cfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c07fff640 1 -- 172.21.15.82:0/882178526 >> v1:172.21.15.196:6789/0 conn(0x7f0c0815daa0 legacy=0x7f0c080d1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.325 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.316+0000 7f0c07fff640 1 -- 172.21.15.82:0/882178526 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080d06e0 con 0x7f0c080b2b00 2024-04-25T10:02:00.327 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c07fff640 1 --2- 172.21.15.82:0/882178526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c080ce700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c0001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb5dbfd9a26aebb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.327 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c067fc640 1 -- 172.21.15.82:0/882178526 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000b2070 con 0x7f0c080b2b00 2024-04-25T10:02:00.327 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c067fc640 1 -- 172.21.15.82:0/882178526 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c000a59a0 con 0x7f0c080b2b00 2024-04-25T10:02:00.327 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c067fc640 1 -- 172.21.15.82:0/882178526 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c000a5d50 con 0x7f0c080b2b00 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 -- 172.21.15.82:0/882178526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 msgr2=0x7f0c080ce700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/882178526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c080ce700 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009d90 tx=0x7f0c0001c8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 -- 172.21.15.82:0/882178526 shutdown_connections 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/882178526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c080ce700 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/882178526 >> v2:172.21.15.196:3300/0 conn(0x7f0c080d2400 0x7f0c080cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 -- 172.21.15.82:0/882178526 >> 172.21.15.82:0/882178526 conn(0x7f0c0809ca80 msgr2=0x7f0c0808afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 -- 172.21.15.82:0/882178526 shutdown_connections 2024-04-25T10:02:00.328 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.320+0000 7f0c0dc58640 1 -- 172.21.15.82:0/882178526 wait complete. 2024-04-25T10:02:00.330 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 Processor -- start 2024-04-25T10:02:00.330 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- start start 2024-04-25T10:02:00.332 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c08156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.332 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.332 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c0815daa0 0x7f0c080c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.332 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080cf450 con 0x7f0c080d2400 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0815de80 con 0x7f0c0815daa0 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c080cf5d0 con 0x7f0c080b2b00 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44658/0 (socket says 172.21.15.82:44658) 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 -- 172.21.15.82:0/1848036066 learned_addr learned my addr 172.21.15.82:0/1848036066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c077fe640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c08156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c07fff640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c0815daa0 0x7f0c080c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 -- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 msgr2=0x7f0c08156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c08156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 -- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c0815daa0 msgr2=0x7f0c080c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c0815daa0 0x7f0c080c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 -- 172.21.15.82:0/1848036066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c081621f0 con 0x7f0c080d2400 2024-04-25T10:02:00.333 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c077fe640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c08156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c06ffd640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e930 tx=0x7f0bf800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8017070 con 0x7f0c080d2400 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bf800ca00 con 0x7f0c080d2400 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800cdb0 con 0x7f0c080d2400 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c080c72e0 con 0x7f0c080d2400 2024-04-25T10:02:00.334 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c080cd4f0 con 0x7f0c080d2400 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c08074d80 con 0x7f0c080d2400 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.324+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0bf801a070 con 0x7f0c080d2400 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.328+0000 7f0beffff640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 0x7f0bf4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.328+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0bf8084d00 con 0x7f0c080d2400 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.328+0000 7f0c077fe640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 0x7f0bf4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.335 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.328+0000 7f0c077fe640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 0x7f0bf4069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c080b17f0 tx=0x7f0bfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.339 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.332+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0bf8056120 con 0x7f0c080d2400 2024-04-25T10:02:00.500 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.492+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c0806c020 con 0x7f0c080d2400 2024-04-25T10:02:00.501 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.492+0000 7f0beffff640 1 -- 172.21.15.82:0/1848036066 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f0bf8055a20 con 0x7f0c080d2400 2024-04-25T10:02:00.502 INFO:teuthology.misc.health.smithi082.stdout: 2024-04-25T10:02:00.502 INFO:teuthology.misc.health.smithi082.stdout:{"epoch":17,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:58.028534+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 msgr2=0x7f0bf4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 0x7f0bf4069450 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c080b17f0 tx=0x7f0bfc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 msgr2=0x7f0c080ca7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e930 tx=0x7f0bf800edf0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 shutdown_connections 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bf4066f90 0x7f0bf4069450 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c0815daa0 0x7f0c080c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c080d2400 0x7f0c080ca7e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.504 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 --2- 172.21.15.82:0/1848036066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c080b2b00 0x7f0c08156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.505 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 >> 172.21.15.82:0/1848036066 conn(0x7f0c0809ca80 msgr2=0x7f0c081613b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.505 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 shutdown_connections 2024-04-25T10:02:00.505 INFO:teuthology.misc.health.smithi082.stderr:2024-04-25T10:02:00.496+0000 7f0c0dc58640 1 -- 172.21.15.82:0/1848036066 wait complete. 2024-04-25T10:02:00.520 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:02:00.520 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:02:00.520 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:02:00.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:02:00.522 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:02:00.523 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:02:00.523 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:02:00.524 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:02:00.525 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:02:00.525 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:02:00.527 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:02:00.527 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:02:00.527 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:02:00.530 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:02:00.531 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:02:00.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:00.601 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:02:00.602 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:02:00.647 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:02:00.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 Processor -- start 2024-04-25T10:02:00.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 -- start start 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1bdc15b920 0x7f1bdc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bdc09a950 con 0x7f1bdc0aeb80 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bdc09ac30 con 0x7f1bdc15b920 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be475f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc0ae380 con 0x7f1bdc0b0440 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34934/0 (socket says 172.21.15.82:34934) 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 -- 172.21.15.82:0/585928963 learned_addr learned my addr 172.21.15.82:0/585928963 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be24fb640 1 --2- 172.21.15.82:0/585928963 >> v2:172.21.15.196:3300/0 conn(0x7f1bdc15b920 0x7f1bdc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 -- 172.21.15.82:0/585928963 >> v2:172.21.15.196:3300/0 conn(0x7f1bdc15b920 msgr2=0x7f1bdc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 --2- 172.21.15.82:0/585928963 >> v2:172.21.15.196:3300/0 conn(0x7f1bdc15b920 0x7f1bdc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 -- 172.21.15.82:0/585928963 >> v1:172.21.15.196:6789/0 conn(0x7f1bdc0aeb80 legacy=0x7f1bdc0aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.640+0000 7f1be1cfa640 1 -- 172.21.15.82:0/585928963 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc0ae660 con 0x7f1bdc0b0440 2024-04-25T10:02:00.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be1cfa640 1 --2- 172.21.15.82:0/585928963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009870 tx=0x7f1bcc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa7065fec8ecafe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be14f9640 1 -- 172.21.15.82:0/585928963 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0b1070 con 0x7f1bdc0b0440 2024-04-25T10:02:00.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be14f9640 1 -- 172.21.15.82:0/585928963 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0ac410 con 0x7f1bdc0b0440 2024-04-25T10:02:00.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be14f9640 1 -- 172.21.15.82:0/585928963 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0ac7e0 con 0x7f1bdc0b0440 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- 172.21.15.82:0/585928963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 msgr2=0x7f1bdc099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 --2- 172.21.15.82:0/585928963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc009870 tx=0x7f1bcc01c560 comp rx=0 tx=0).stop 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- 172.21.15.82:0/585928963 shutdown_connections 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 --2- 172.21.15.82:0/585928963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0b0440 0x7f1bdc099fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 --2- 172.21.15.82:0/585928963 >> v2:172.21.15.196:3300/0 conn(0x7f1bdc15b920 0x7f1bdc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- 172.21.15.82:0/585928963 >> 172.21.15.82:0/585928963 conn(0x7f1bdc0b3010 msgr2=0x7f1bdc0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- 172.21.15.82:0/585928963 shutdown_connections 2024-04-25T10:02:00.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- 172.21.15.82:0/585928963 wait complete. 2024-04-25T10:02:00.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 Processor -- start 2024-04-25T10:02:00.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.644+0000 7f1be475f640 1 -- start start 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0aeb80 0x7f1bdc070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bdc15b920 0x7f1bdc06f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc168850 con 0x7f1bdc0b0440 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1686d0 con 0x7f1bdc15b920 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bdc1689d0 con 0x7f1bdc0aeb80 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44684/0 (socket says 172.21.15.82:44684) 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 -- 172.21.15.82:0/165028758 learned_addr learned my addr 172.21.15.82:0/165028758 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be24fb640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0aeb80 0x7f1bdc070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be2cfc640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bdc15b920 0x7f1bdc06f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 -- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0aeb80 msgr2=0x7f1bdc070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0aeb80 0x7f1bdc070500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 -- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bdc15b920 msgr2=0x7f1bdc06f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bdc15b920 0x7f1bdc06f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 -- 172.21.15.82:0/165028758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bdc15e380 con 0x7f1bdc0b0440 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be1cfa640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0af040 tx=0x7f1bcc019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0bc070 con 0x7f1bdc0b0440 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bcc0b7440 con 0x7f1bdc0b0440 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bdc0c9b30 con 0x7f1bdc0b0440 2024-04-25T10:02:00.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc0b1070 con 0x7f1bdc0b0440 2024-04-25T10:02:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bdc0ca0f0 con 0x7f1bdc0b0440 2024-04-25T10:02:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bdc04efa0 con 0x7f1bdc0b0440 2024-04-25T10:02:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bcc0bf070 con 0x7f1bdc0b0440 2024-04-25T10:02:00.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.648+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1bcc11e360 con 0x7f1bdc0b0440 2024-04-25T10:02:00.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.652+0000 7f1be24fb640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.652+0000 7f1be24fb640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 0x7f1bc0069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009ac0 tx=0x7f1bd8009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.652+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bcc016150 con 0x7f1bdc0b0440 2024-04-25T10:02:00.662 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:02:00.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.800+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bdc15eb30 con 0x7f1bdc0b0440 2024-04-25T10:02:00.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.800+0000 7f1bd2ffd640 1 -- 172.21.15.82:0/165028758 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f1bcc0ef1e0 con 0x7f1bdc0b0440 2024-04-25T10:02:00.810 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:00.810 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:58.028534+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 msgr2=0x7f1bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 0x7f1bc0069420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009ac0 tx=0x7f1bd8009190 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 msgr2=0x7f1bdc0c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0af040 tx=0x7f1bcc019300 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 shutdown_connections 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1bc0066f60 0x7f1bc0069420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1bdc15b920 0x7f1bdc06f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1bdc0b0440 0x7f1bdc0c9220 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 --2- 172.21.15.82:0/165028758 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1bdc0aeb80 0x7f1bdc070500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 >> 172.21.15.82:0/165028758 conn(0x7f1bdc0b3010 msgr2=0x7f1bdc0b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 shutdown_connections 2024-04-25T10:02:00.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.804+0000 7f1be475f640 1 -- 172.21.15.82:0/165028758 wait complete. 2024-04-25T10:02:00.826 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:00.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 Processor -- start 2024-04-25T10:02:00.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 -- start start 2024-04-25T10:02:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2410066e30 0x7f241006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f24100ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241006ec30 con 0x7f241006bca0 2024-04-25T10:02:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24100669a0 con 0x7f2410066e30 2024-04-25T10:02:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2416075640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241006c5c0 con 0x7f24100af370 2024-04-25T10:02:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f24100ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2410066e30 0x7f241006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240f7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f241006bca0 0x7f24100cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:55130/0 (socket says 172.21.15.82:55130) 2024-04-25T10:02:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f24100ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34966/0 (socket says 172.21.15.82:34966) 2024-04-25T10:02:00.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2410066e30 0x7f241006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:36934/0 (socket says 172.21.15.82:36934) 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240f7fe640 1 -- 172.21.15.82:0/3344136604 learned_addr learned my addr 172.21.15.82:0/3344136604 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240effd640 1 -- 172.21.15.82:0/3344136604 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367652926 0 0) 0x7f241006ec30 con 0x7f241006bca0 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f240effd640 1 -- 172.21.15.82:0/3344136604 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23fc003530 con 0x7f241006bca0 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 -- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 msgr2=0x7f24100ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 --2- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f24100ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 -- 172.21.15.82:0/3344136604 >> v1:172.21.15.196:6789/0 conn(0x7f241006bca0 legacy=0x7f24100cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.944+0000 7f2407fff640 1 -- 172.21.15.82:0/3344136604 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241006f230 con 0x7f2410066e30 2024-04-25T10:02:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2407fff640 1 --2- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f241006c080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097c0 tx=0x7f23f801c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d80386edbf1438dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f240effd640 1 -- 172.21.15.82:0/3344136604 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80b0070 con 0x7f2410066e30 2024-04-25T10:02:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f240effd640 1 -- 172.21.15.82:0/3344136604 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f80ab600 con 0x7f2410066e30 2024-04-25T10:02:00.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f240effd640 1 -- 172.21.15.82:0/3344136604 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80ab9d0 con 0x7f2410066e30 2024-04-25T10:02:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 -- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 msgr2=0x7f241006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 --2- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f241006c080 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f23f80097c0 tx=0x7f23f801c350 comp rx=0 tx=0).stop 2024-04-25T10:02:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 -- 172.21.15.82:0/3344136604 shutdown_connections 2024-04-25T10:02:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 --2- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f24100ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 --2- 172.21.15.82:0/3344136604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f241006c080 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 -- 172.21.15.82:0/3344136604 >> 172.21.15.82:0/3344136604 conn(0x7f2410071810 msgr2=0x7f2410071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 -- 172.21.15.82:0/3344136604 shutdown_connections 2024-04-25T10:02:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.948+0000 7f2416075640 1 -- 172.21.15.82:0/3344136604 wait complete. 2024-04-25T10:02:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 Processor -- start 2024-04-25T10:02:00.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- start start 2024-04-25T10:02:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f24101487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f241006b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101687b0 con 0x7f241006bca0 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2410168630 con 0x7f2410066e30 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2410168930 con 0x7f24100af370 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44686/0 (socket says 172.21.15.82:44686) 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 -- 172.21.15.82:0/85768546 learned_addr learned my addr 172.21.15.82:0/85768546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240f7fe640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f24101487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240ffff640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f241006b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 -- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 msgr2=0x7f241006b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f241006b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 -- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 msgr2=0x7f24101487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f24101487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 -- 172.21.15.82:0/85768546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410151530 con 0x7f241006bca0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240f7fe640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f24101487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2407fff640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23f8019f80 tx=0x7f23f80190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80b8020 con 0x7f241006bca0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f80b4410 con 0x7f241006bca0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f80b0040 con 0x7f241006bca0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2410151810 con 0x7f241006bca0 2024-04-25T10:02:00.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24100aca80 con 0x7f241006bca0 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.952+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241004efa0 con 0x7f241006bca0 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.956+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23f80bb020 con 0x7f241006bca0 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.956+0000 7f240cff9640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 0x7f23f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.956+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23f811e710 con 0x7f241006bca0 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.956+0000 7f240f7fe640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 0x7f23f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.956+0000 7f240f7fe640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 0x7f23f0069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2400005890 tx=0x7f2400009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:00.960+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23f80c2030 con 0x7f241006bca0 2024-04-25T10:02:01.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.112+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2410072400 con 0x7f241006bca0 2024-04-25T10:02:01.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f240cff9640 1 -- 172.21.15.82:0/85768546 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23f8016090 con 0x7f241006bca0 2024-04-25T10:02:01.122 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:01.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 msgr2=0x7f23f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 0x7f23f0069420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2400005890 tx=0x7f2400009040 comp rx=0 tx=0).stop 2024-04-25T10:02:01.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 msgr2=0x7f241006ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23f8019f80 tx=0x7f23f80190f0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 shutdown_connections 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f23f0066f60 0x7f23f0069420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24100af370 0x7f241006b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f241006bca0 0x7f241006ae90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 --2- 172.21.15.82:0/85768546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2410066e30 0x7f24101487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 >> 172.21.15.82:0/85768546 conn(0x7f2410071810 msgr2=0x7f24100b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 shutdown_connections 2024-04-25T10:02:01.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.116+0000 7f2416075640 1 -- 172.21.15.82:0/85768546 wait complete. 2024-04-25T10:02:01.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:01.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.236+0000 7feb8a736640 1 Processor -- start 2024-04-25T10:02:01.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.236+0000 7feb8a736640 1 -- start start 2024-04-25T10:02:01.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb8a736640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feb8415da80 0x7feb840d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb8a736640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feb840d07f0 0x7feb8406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb8a736640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8406be60 con 0x7feb840d1cd0 2024-04-25T10:02:01.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb8a736640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8406c440 con 0x7feb8415da80 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb8a736640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb8415de60 con 0x7feb840d07f0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feb8415da80 0x7feb840d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb837fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feb840d1cd0 0x7feb840d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50922/0 (socket says 172.21.15.82:50922) 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb837fe640 1 -- 172.21.15.82:0/2355620137 learned_addr learned my addr 172.21.15.82:0/2355620137 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2717510318 0 0) 0x7feb8406be60 con 0x7feb840d1cd0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb6c003530 con 0x7feb840d1cd0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb88cd3640 1 --2- 172.21.15.82:0/2355620137 >> v2:172.21.15.196:3300/0 conn(0x7feb840d07f0 0x7feb8406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 -- 172.21.15.82:0/2355620137 >> v2:172.21.15.196:3300/0 conn(0x7feb840d07f0 msgr2=0x7feb8406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 --2- 172.21.15.82:0/2355620137 >> v2:172.21.15.196:3300/0 conn(0x7feb840d07f0 0x7feb8406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2018149189 0 0) 0x7feb6c003530 con 0x7feb840d1cd0 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 -- 172.21.15.82:0/2355620137 >> v1:172.21.15.82:6789/0 conn(0x7feb840d1cd0 legacy=0x7feb840d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 -- 172.21.15.82:0/2355620137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840d2450 con 0x7feb8415da80 2024-04-25T10:02:01.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb83fff640 1 --2- 172.21.15.82:0/2355620137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb8415da80 0x7feb840d4230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d90 tx=0x7feb7c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67595d5d1a9d4623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0b2070 con 0x7feb8415da80 2024-04-25T10:02:01.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c0a5640 con 0x7feb8415da80 2024-04-25T10:02:01.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.240+0000 7feb82ffd640 1 -- 172.21.15.82:0/2355620137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb78001800 con 0x7feb8415da80 2024-04-25T10:02:01.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/2355620137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb8415da80 msgr2=0x7feb840d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- 172.21.15.82:0/2355620137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb8415da80 0x7feb840d4230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009d90 tx=0x7feb7c01c820 comp rx=0 tx=0).stop 2024-04-25T10:02:01.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/2355620137 shutdown_connections 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- 172.21.15.82:0/2355620137 >> v2:172.21.15.196:3300/0 conn(0x7feb840d07f0 0x7feb8406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- 172.21.15.82:0/2355620137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb8415da80 0x7feb840d4230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/2355620137 >> 172.21.15.82:0/2355620137 conn(0x7feb8408b000 msgr2=0x7feb840d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/2355620137 shutdown_connections 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/2355620137 wait complete. 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 Processor -- start 2024-04-25T10:02:01.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- start start 2024-04-25T10:02:01.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb840d1cd0 0x7feb840b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8415da80 0x7feb840c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8406d7a0 con 0x7feb840d07f0 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb8406d620 con 0x7feb8415da80 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb8406d920 con 0x7feb840d1cd0 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44714/0 (socket says 172.21.15.82:44714) 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 -- 172.21.15.82:0/66443349 learned_addr learned my addr 172.21.15.82:0/66443349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 -- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb840d1cd0 msgr2=0x7feb840b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb88cd3640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8415da80 0x7feb840c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb840d1cd0 0x7feb840b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 -- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8415da80 msgr2=0x7feb840c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8415da80 0x7feb840c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 -- 172.21.15.82:0/66443349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840c76c0 con 0x7feb840d07f0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb83fff640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0b0040 tx=0x7feb7c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c0b2050 con 0x7feb840d07f0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb7c0be070 con 0x7feb840d07f0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb840c79a0 con 0x7feb840d07f0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb7c009450 con 0x7feb840d07f0 2024-04-25T10:02:01.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.244+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb840c7f60 con 0x7feb840d07f0 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8404efa0 con 0x7feb840d07f0 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb7c0ada70 con 0x7feb840d07f0 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb80ff9640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 0x7feb70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feb7c11fe30 con 0x7feb840d07f0 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb837fe640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 0x7feb70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.248+0000 7feb837fe640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 0x7feb70069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb78000f00 tx=0x7feb78001de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.252+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb7c016480 con 0x7feb840d07f0 2024-04-25T10:02:01.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb841570f0 con 0x7feb840d07f0 2024-04-25T10:02:01.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb80ff9640 1 -- 172.21.15.82:0/66443349 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7feb7c0f1190 con 0x7feb840d07f0 2024-04-25T10:02:01.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:01.444 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T10:02:01.444 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 2 2024-04-25T10:02:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 msgr2=0x7feb70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 0x7feb70069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feb78000f00 tx=0x7feb78001de0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 msgr2=0x7feb84156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feb7c0b0040 tx=0x7feb7c0195c0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 shutdown_connections 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feb70066f60 0x7feb70069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feb840d1cd0 0x7feb840b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feb8415da80 0x7feb840c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 --2- 172.21.15.82:0/66443349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feb840d07f0 0x7feb84156bb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 >> 172.21.15.82:0/66443349 conn(0x7feb8408b000 msgr2=0x7feb840b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 shutdown_connections 2024-04-25T10:02:01.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:01.436+0000 7feb8a736640 1 -- 172.21.15.82:0/66443349 wait complete. 2024-04-25T10:02:03.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:03.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.584+0000 7fdf78a7c640 1 Processor -- start 2024-04-25T10:02:03.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.584+0000 7fdf78a7c640 1 -- start start 2024-04-25T10:02:03.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdf740ce510 0x7fdf740ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf74144ca0 0x7fdf7406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf7406ec60 0x7fdf74142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf740cc2b0 con 0x7fdf7406ec60 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7406e290 con 0x7fdf740ce510 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740cee30 con 0x7fdf74144ca0 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf7406ec60 0x7fdf74142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdf7406ec60 0x7fdf74142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44728/0 (socket says 172.21.15.82:44728) 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1166309833 learned_addr learned my addr 172.21.15.82:0/1166309833 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf7259c640 1 --2- 172.21.15.82:0/1166309833 >> v2:172.21.15.196:3300/0 conn(0x7fdf740ce510 0x7fdf740ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf71d9b640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf74144ca0 0x7fdf7406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1166309833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf74144ca0 msgr2=0x7fdf7406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf74144ca0 0x7fdf7406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1166309833 >> v2:172.21.15.196:3300/0 conn(0x7fdf740ce510 msgr2=0x7fdf740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1166309833 >> v2:172.21.15.196:3300/0 conn(0x7fdf740ce510 0x7fdf740ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1166309833 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740ccc70 con 0x7fdf7406ec60 2024-04-25T10:02:03.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf7406ec60 0x7fdf74142680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf74144c50 tx=0x7fdf6c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1587b241a971632b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf7159a640 1 -- 172.21.15.82:0/1166309833 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0b4070 con 0x7fdf7406ec60 2024-04-25T10:02:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf7159a640 1 -- 172.21.15.82:0/1166309833 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0af400 con 0x7fdf7406ec60 2024-04-25T10:02:03.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf7159a640 1 -- 172.21.15.82:0/1166309833 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0af7d0 con 0x7fdf7406ec60 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1166309833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf7406ec60 msgr2=0x7fdf74142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf7406ec60 0x7fdf74142680 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdf74144c50 tx=0x7fdf6c01f1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1166309833 shutdown_connections 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf7406ec60 0x7fdf74142680 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1166309833 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf74144ca0 0x7fdf7406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1166309833 >> v2:172.21.15.196:3300/0 conn(0x7fdf740ce510 0x7fdf740ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1166309833 >> 172.21.15.82:0/1166309833 conn(0x7fdf740b11a0 msgr2=0x7fdf740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1166309833 shutdown_connections 2024-04-25T10:02:03.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.588+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1166309833 wait complete. 2024-04-25T10:02:03.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 Processor -- start 2024-04-25T10:02:03.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- start start 2024-04-25T10:02:03.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf7406ec60 0x7fdf740a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf740ce510 0x7fdf741439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf740cc900 con 0x7fdf74144ca0 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7406df20 con 0x7fdf740ce510 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf741451f0 con 0x7fdf7406ec60 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44742/0 (socket says 172.21.15.82:44742) 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1357187673 learned_addr learned my addr 172.21.15.82:0/1357187673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf7259c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf7406ec60 0x7fdf740a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf71d9b640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf740ce510 0x7fdf741439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf7406ec60 msgr2=0x7fdf740a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf7406ec60 0x7fdf740a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf740ce510 msgr2=0x7fdf741439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf740ce510 0x7fdf741439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 -- 172.21.15.82:0/1357187673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf740a8a10 con 0x7fdf74144ca0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf72d9d640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c01fa80 tx=0x7fdf6c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0c0070 con 0x7fdf74144ca0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0090d0 con 0x7fdf74144ca0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf74130ce0 con 0x7fdf74144ca0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0b4040 con 0x7fdf74144ca0 2024-04-25T10:02:03.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf741312a0 con 0x7fdf74144ca0 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.592+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7404efa0 con 0x7fdf74144ca0 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.596+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf6c0c3070 con 0x7fdf74144ca0 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.596+0000 7fdf62ffd640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 0x7fdf54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.596+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdf6c1228a0 con 0x7fdf74144ca0 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.596+0000 7fdf7259c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 0x7fdf54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.596+0000 7fdf7259c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 0x7fdf54069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.600+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf6c0ca080 con 0x7fdf74144ca0 2024-04-25T10:02:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf740b1cc0 con 0x7fdf74144ca0 2024-04-25T10:02:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf62ffd640 1 -- 172.21.15.82:0/1357187673 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7fdf6c018d40 con 0x7fdf74144ca0 2024-04-25T10:02:03.799 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:03.799 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:02:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 3 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 msgr2=0x7fdf54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 0x7fdf54069420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdf64006fd0 tx=0x7fdf64008040 comp rx=0 tx=0).stop 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 msgr2=0x7fdf74132bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c01fa80 tx=0x7fdf6c01bf80 comp rx=0 tx=0).stop 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 shutdown_connections 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf54066f60 0x7fdf54069420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf74144ca0 0x7fdf74132bd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf740ce510 0x7fdf741439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 --2- 172.21.15.82:0/1357187673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf7406ec60 0x7fdf740a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 >> 172.21.15.82:0/1357187673 conn(0x7fdf740b11a0 msgr2=0x7fdf740c61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 shutdown_connections 2024-04-25T10:02:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.792+0000 7fdf78a7c640 1 -- 172.21.15.82:0/1357187673 wait complete. 2024-04-25T10:02:03.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:03.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 Processor -- start 2024-04-25T10:02:03.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 -- start start 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c900d0870 0x7f9c900d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c9015dc30 0x7f9c900d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900ce6b0 con 0x7f9c9015d850 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c900ce990 con 0x7f9c9015dc30 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c97451640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c900cec70 con 0x7f9c900d0870 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c9015dc30 0x7f9c900d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9c9015dc30 0x7f9c900d2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44750/0 (socket says 172.21.15.82:44750) 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 -- 172.21.15.82:0/308589348 learned_addr learned my addr 172.21.15.82:0/308589348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:03.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c949ec640 1 --2- 172.21.15.82:0/308589348 >> v2:172.21.15.196:3300/0 conn(0x7f9c900d0870 0x7f9c900d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 -- 172.21.15.82:0/308589348 >> v2:172.21.15.196:3300/0 conn(0x7f9c900d0870 msgr2=0x7f9c900d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 --2- 172.21.15.82:0/308589348 >> v2:172.21.15.196:3300/0 conn(0x7f9c900d0870 0x7f9c900d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 -- 172.21.15.82:0/308589348 >> v1:172.21.15.82:6789/0 conn(0x7f9c9015d850 legacy=0x7f9c900d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.928+0000 7f9c959ee640 1 -- 172.21.15.82:0/308589348 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c900cef50 con 0x7f9c9015dc30 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 --2- 172.21.15.82:0/308589348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900d2320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c9008b930 tx=0x7f9c8801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ecfb5caf24addc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c87fff640 1 -- 172.21.15.82:0/308589348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880a57d0 con 0x7f9c9015dc30 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c87fff640 1 -- 172.21.15.82:0/308589348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c880a5970 con 0x7f9c9015dc30 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c87fff640 1 -- 172.21.15.82:0/308589348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880a5d40 con 0x7f9c9015dc30 2024-04-25T10:02:03.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- 172.21.15.82:0/308589348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 msgr2=0x7f9c900d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- 172.21.15.82:0/308589348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900d2320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9c9008b930 tx=0x7f9c8801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- 172.21.15.82:0/308589348 shutdown_connections 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- 172.21.15.82:0/308589348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900d2320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- 172.21.15.82:0/308589348 >> v2:172.21.15.196:3300/0 conn(0x7f9c900d0870 0x7f9c900d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- 172.21.15.82:0/308589348 >> 172.21.15.82:0/308589348 conn(0x7f9c9008b000 msgr2=0x7f9c900d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- 172.21.15.82:0/308589348 shutdown_connections 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- 172.21.15.82:0/308589348 wait complete. 2024-04-25T10:02:03.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 Processor -- start 2024-04-25T10:02:03.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- start start 2024-04-25T10:02:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c900d0870 0x7f9c900b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9015d850 0x7f9c900b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9016a4c0 con 0x7f9c9015dc30 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9016a340 con 0x7f9c900d0870 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c97451640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9016a640 con 0x7f9c9015d850 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44756/0 (socket says 172.21.15.82:44756) 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 -- 172.21.15.82:0/2454618319 learned_addr learned my addr 172.21.15.82:0/2454618319 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c949ec640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9015d850 0x7f9c900b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 -- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9015d850 msgr2=0x7f9c900b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9015d850 0x7f9c900b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 -- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c900d0870 msgr2=0x7f9c900b2870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c900d0870 0x7f9c900b2870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.932+0000 7f9c959ee640 1 -- 172.21.15.82:0/2454618319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c901536e0 con 0x7f9c9015dc30 2024-04-25T10:02:03.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c959ee640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c88019600 tx=0x7f9c88019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880b8470 con 0x7f9c9015dc30 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c880b8630 con 0x7f9c9015dc30 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c901539c0 con 0x7f9c9015dc30 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880b8a20 con 0x7f9c9015dc30 2024-04-25T10:02:03.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9015b340 con 0x7f9c9015dc30 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9004efa0 con 0x7f9c9015dc30 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c880c0020 con 0x7f9c9015dc30 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 0x7f9c74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9c8811f9e0 con 0x7f9c9015dc30 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c951ed640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 0x7f9c74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.936+0000 7f9c951ed640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 0x7f9c74069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c001470 tx=0x7f9c8c0079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:03.940+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c880165e0 con 0x7f9c9015dc30 2024-04-25T10:02:04.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c900d0e60 con 0x7f9c9015dc30 2024-04-25T10:02:04.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c85ffb640 1 -- 172.21.15.82:0/2454618319 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12896 (secure 0 0 0) 0x7f9c880f0d40 con 0x7f9c9015dc30 2024-04-25T10:02:04.107 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:04.107 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:01:58.028534+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 msgr2=0x7f9c74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 0x7f9c74069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c001470 tx=0x7f9c8c0079e0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 msgr2=0x7f9c900b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c88019600 tx=0x7f9c88019790 comp rx=0 tx=0).stop 2024-04-25T10:02:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 shutdown_connections 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c74066f60 0x7f9c74069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c9015d850 0x7f9c900b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c9015dc30 0x7f9c900b6440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 --2- 172.21.15.82:0/2454618319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c900d0870 0x7f9c900b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 >> 172.21.15.82:0/2454618319 conn(0x7f9c9008b000 msgr2=0x7f9c90156810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 shutdown_connections 2024-04-25T10:02:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.100+0000 7f9c97451640 1 -- 172.21.15.82:0/2454618319 wait complete. 2024-04-25T10:02:04.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:04.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:04.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 Processor -- start 2024-04-25T10:02:04.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 -- start start 2024-04-25T10:02:04.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 0x7fc3dc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3dc06be60 con 0x7fc3dc15da80 2024-04-25T10:02:04.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3dc06c440 con 0x7fc3dc0d07f0 2024-04-25T10:02:04.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3e20ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc15de60 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:35036/0 (socket says 172.21.15.82:35036) 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 -- 172.21.15.82:0/4221449103 learned_addr learned my addr 172.21.15.82:0/4221449103 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3daffd640 1 --2- 172.21.15.82:0/4221449103 >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 0x7fc3dc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3da7fc640 1 -- 172.21.15.82:0/4221449103 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592737246 0 0) 0x7fc3dc06be60 con 0x7fc3dc15da80 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3da7fc640 1 -- 172.21.15.82:0/4221449103 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3c4003530 con 0x7fc3dc15da80 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 -- 172.21.15.82:0/4221449103 >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 msgr2=0x7fc3dc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 --2- 172.21.15.82:0/4221449103 >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 0x7fc3dc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 -- 172.21.15.82:0/4221449103 >> v1:172.21.15.196:6789/0 conn(0x7fc3dc15da80 legacy=0x7fc3dc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3dbfff640 1 -- 172.21.15.82:0/4221449103 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0d2450 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.244+0000 7fc3daffd640 1 --2- 172.21.15.82:0/4221449103 >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 0x7fc3dc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:04.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.248+0000 7fc3dbfff640 1 --2- 172.21.15.82:0/4221449103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e50 tx=0x7fc3d401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d3ae81cb7f0bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3da7fc640 1 -- 172.21.15.82:0/4221449103 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40a5890 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3da7fc640 1 -- 172.21.15.82:0/4221449103 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d40a5a30 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3da7fc640 1 -- 172.21.15.82:0/4221449103 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d40a5de0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- 172.21.15.82:0/4221449103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 msgr2=0x7fc3dc06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/4221449103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4009e50 tx=0x7fc3d401ca00 comp rx=0 tx=0).stop 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- 172.21.15.82:0/4221449103 shutdown_connections 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/4221449103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d1cd0 0x7fc3dc06c8d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/4221449103 >> v2:172.21.15.196:3300/0 conn(0x7fc3dc0d07f0 0x7fc3dc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- 172.21.15.82:0/4221449103 >> 172.21.15.82:0/4221449103 conn(0x7fc3dc08b000 msgr2=0x7fc3dc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- 172.21.15.82:0/4221449103 shutdown_connections 2024-04-25T10:02:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- 172.21.15.82:0/4221449103 wait complete. 2024-04-25T10:02:04.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 Processor -- start 2024-04-25T10:02:04.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- start start 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d07f0 0x7fc3dc154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3dc15da80 0x7fc3dc1f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc06d7a0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc06d620 con 0x7fc3dc15da80 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3e20ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc06d920 con 0x7fc3dc0d07f0 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3daffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3daffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44770/0 (socket says 172.21.15.82:44770) 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3daffd640 1 -- 172.21.15.82:0/2723372571 learned_addr learned my addr 172.21.15.82:0/2723372571 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3dbfff640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3dc15da80 0x7fc3dc1f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.252+0000 7fc3db7fe640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d07f0 0x7fc3dc154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 -- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d07f0 msgr2=0x7fc3dc154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d07f0 0x7fc3dc154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 -- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3dc15da80 msgr2=0x7fc3dc1f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3dc15da80 0x7fc3dc1f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 -- 172.21.15.82:0/2723372571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc159350 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3daffd640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ec10 tx=0x7fc3d000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d000cc30 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d000cdd0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc159630 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0005bc0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc15ac00 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc04efa0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3d0006030 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 0x7fc3c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc3d00850d0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3db7fe640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 0x7fc3c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.256+0000 7fc3db7fe640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 0x7fc3c8069450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc160b50 tx=0x7fc3cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.260+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3d0017020 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3dc15ae40 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3c3fff640 1 -- 172.21.15.82:0/2723372571 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc3d00564c0 con 0x7fc3dc0d1cd0 2024-04-25T10:02:04.428 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 msgr2=0x7fc3c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 0x7fc3c8069450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc160b50 tx=0x7fc3cc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 msgr2=0x7fc3dc1f9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000ec10 tx=0x7fc3d000c520 comp rx=0 tx=0).stop 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 shutdown_connections 2024-04-25T10:02:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc3c8066f90 0x7fc3c8069450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc3dc15da80 0x7fc3dc1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc3dc0d1cd0 0x7fc3dc1f9070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 --2- 172.21.15.82:0/2723372571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc3dc0d07f0 0x7fc3dc154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 >> 172.21.15.82:0/2723372571 conn(0x7fc3dc08b000 msgr2=0x7fc3dc0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 shutdown_connections 2024-04-25T10:02:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.420+0000 7fc3e20ca640 1 -- 172.21.15.82:0/2723372571 wait complete. 2024-04-25T10:02:04.445 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:02:04.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.540+0000 7f39e2be7640 1 Processor -- start 2024-04-25T10:02:04.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e2be7640 1 -- start start 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e2be7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f39dc0d07f0 0x7f39dc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e2be7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc06be60 con 0x7f39dc15da80 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e2be7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39dc15de60 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e2be7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39dc06c440 con 0x7f39dc0d07f0 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f39dc0d07f0 0x7f39dc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e0983640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f39dc15da80 0x7f39dc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50940/0 (socket says 172.21.15.82:50940) 2024-04-25T10:02:04.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39e0983640 1 -- 172.21.15.82:0/4062306990 learned_addr learned my addr 172.21.15.82:0/4062306990 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:04.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d3fff640 1 -- 172.21.15.82:0/4062306990 >> v1:172.21.15.196:6789/0 conn(0x7f39dc0d1cd0 legacy=0x7f39dc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d3fff640 1 -- 172.21.15.82:0/4062306990 >> v1:172.21.15.82:6789/0 conn(0x7f39dc15da80 legacy=0x7f39dc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d3fff640 1 -- 172.21.15.82:0/4062306990 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc0d2450 con 0x7f39dc0d07f0 2024-04-25T10:02:04.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d37fe640 1 -- 172.21.15.82:0/4062306990 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469794896 0 0) 0x7f39dc06be60 con 0x7f39dc15da80 2024-04-25T10:02:04.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.544+0000 7f39d3fff640 1 --2- 172.21.15.82:0/4062306990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d07f0 0x7f39dc0d4230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0097a0 tx=0x7f39cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574b01a21d85bef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d37fe640 1 -- 172.21.15.82:0/4062306990 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0ab430 con 0x7f39dc0d07f0 2024-04-25T10:02:04.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d37fe640 1 -- 172.21.15.82:0/4062306990 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc0b0070 con 0x7f39dc0d07f0 2024-04-25T10:02:04.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d37fe640 1 -- 172.21.15.82:0/4062306990 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0ab7f0 con 0x7f39dc0d07f0 2024-04-25T10:02:04.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/4062306990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d07f0 msgr2=0x7f39dc0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 --2- 172.21.15.82:0/4062306990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d07f0 0x7f39dc0d4230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39cc0097a0 tx=0x7f39cc01c410 comp rx=0 tx=0).stop 2024-04-25T10:02:04.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/4062306990 shutdown_connections 2024-04-25T10:02:04.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 --2- 172.21.15.82:0/4062306990 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d07f0 0x7f39dc0d4230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/4062306990 >> 172.21.15.82:0/4062306990 conn(0x7f39dc08b000 msgr2=0x7f39dc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/4062306990 shutdown_connections 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/4062306990 wait complete. 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 Processor -- start 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- start start 2024-04-25T10:02:04.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0d07f0 0x7f39dc15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc15da80 0x7f39dc155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc06d970 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc06d7f0 con 0x7f39dc0d07f0 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc06daf0 con 0x7f39dc15da80 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44790/0 (socket says 172.21.15.82:44790) 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 -- 172.21.15.82:0/1503250992 learned_addr learned my addr 172.21.15.82:0/1503250992 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 -- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc15da80 msgr2=0x7f39dc155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc15da80 0x7f39dc155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 -- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0d07f0 msgr2=0x7f39dc15b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0d07f0 0x7f39dc15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 -- 172.21.15.82:0/1503250992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc15ca40 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d3fff640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39cc019020 tx=0x7f39cc01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0bf610 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc0bf7d0 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc15bec0 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc0bfbc0 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.548+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc15c480 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc04efa0 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39cc0bd020 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39d17fa640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 0x7f39c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f39cc0a8030 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39e0983640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 0x7f39c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.552+0000 7f39e0983640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 0x7f39c8069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39d80011f0 tx=0x7f39d8007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.556+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39cc0c4060 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.704+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f39dc15e5a0 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.704+0000 7f39d17fa640 1 -- 172.21.15.82:0/1503250992 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f39cc016030 con 0x7f39dc0d1cd0 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:02:04.713 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:02:04.714 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:02:04.714 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:02:04.714 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:02:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 msgr2=0x7f39c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 0x7f39c8069420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39d80011f0 tx=0x7f39d8007930 comp rx=0 tx=0).stop 2024-04-25T10:02:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 msgr2=0x7f39dc154a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f39cc019020 tx=0x7f39cc01ac50 comp rx=0 tx=0).stop 2024-04-25T10:02:04.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 shutdown_connections 2024-04-25T10:02:04.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39c8066f60 0x7f39c8069420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39dc15da80 0x7f39dc155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39dc0d1cd0 0x7f39dc154a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 --2- 172.21.15.82:0/1503250992 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39dc0d07f0 0x7f39dc15b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 >> 172.21.15.82:0/1503250992 conn(0x7f39dc08b000 msgr2=0x7f39dc15cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 shutdown_connections 2024-04-25T10:02:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.708+0000 7f39e2be7640 1 -- 172.21.15.82:0/1503250992 wait complete. 2024-04-25T10:02:04.729 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:02:04.730 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:02:04.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.840+0000 7f95a0978640 1 Processor -- start 2024-04-25T10:02:04.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.840+0000 7f95a0978640 1 -- start start 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f95a0978640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f959c0b0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f95a0978640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0ce870 con 0x7f959c0d21f0 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f95a0978640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959c0ceb50 con 0x7f959c161050 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f95a0978640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c161430 con 0x7f959c08b4c0 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f9599d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f959c0d21f0 0x7f959c0b0280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57892/0 (socket says 172.21.15.82:57892) 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f9599d9b640 1 -- 172.21.15.82:0/2911868838 learned_addr learned my addr 172.21.15.82:0/2911868838 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f959c0b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 649922150 0 0) 0x7f959c0ce870 con 0x7f959c0d21f0 2024-04-25T10:02:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9588003530 con 0x7f959c0d21f0 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135434361 0 0) 0x7f9588003530 con 0x7f959c0d21f0 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 msgr2=0x7f959c0b0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f959c0b0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 >> v1:172.21.15.196:6789/0 conn(0x7f959c161050 legacy=0x7f959c0d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0cee30 con 0x7f959c0d21f0 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9584001680 con 0x7f959c0d21f0 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 >> v1:172.21.15.82:6789/0 conn(0x7f959c0d21f0 legacy=0x7f959c0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95880039f0 0x7f9588003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f9588004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9588004d50 0x7f95880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c161430 con 0x7f9588004d50 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958800ba80 con 0x7f95880039f0 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958800bc80 con 0x7f959c08b4c0 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9588004d50 0x7f95880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959a59c640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95880039f0 0x7f9588003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f9599d9b640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f9588004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 -- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 msgr2=0x7f9588004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f9588004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 -- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95880039f0 msgr2=0x7f9588003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95880039f0 0x7f9588003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959ad9d640 1 -- 172.21.15.82:0/2911868838 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9588003530 con 0x7f9588004d50 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f959a59c640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95880039f0 0x7f9588003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.844+0000 7f9599d9b640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c08b4c0 0x7f9588004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959ad9d640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9588004d50 0x7f95880051d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f959c08b8c0 tx=0x7f959401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b026711e97021031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95940b2070 con 0x7f9588004d50 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95940a5600 con 0x7f9588004d50 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959959a640 1 -- 172.21.15.82:0/2911868838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95940096b0 con 0x7f9588004d50 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- 172.21.15.82:0/2911868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9588004d50 msgr2=0x7f95880051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 --2- 172.21.15.82:0/2911868838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9588004d50 0x7f95880051d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f959c08b8c0 tx=0x7f959401ca10 comp rx=0 tx=0).stop 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959ad9d640 1 -- 172.21.15.82:0/2911868838 reap_dead start 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- 172.21.15.82:0/2911868838 shutdown_connections 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- 172.21.15.82:0/2911868838 >> 172.21.15.82:0/2911868838 conn(0x7f959c08b000 msgr2=0x7f959c0d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- 172.21.15.82:0/2911868838 shutdown_connections 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- 172.21.15.82:0/2911868838 wait complete. 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 Processor -- start 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- start start 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f959c15f4c0 0x7f959c155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c146e70 0x7f959c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cf910 con 0x7f959c15f0e0 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0cf3f0 con 0x7f959c15f4c0 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f95a0978640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959c0ce390 con 0x7f959c146e70 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46792/0 (socket says 172.21.15.82:46792) 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 -- 172.21.15.82:0/1034710109 learned_addr learned my addr 172.21.15.82:0/1034710109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 -- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c146e70 msgr2=0x7f959c1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c146e70 0x7f959c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 -- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f959c15f4c0 msgr2=0x7f959c155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f959c15f4c0 0x7f959c155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 -- 172.21.15.82:0/1034710109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c147ba0 con 0x7f959c15f0e0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.848+0000 7f959a59c640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f958c00db10 tx=0x7f958c0020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c003bb0 con 0x7f959c15f0e0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c003d50 con 0x7f959c15f0e0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c144fa0 con 0x7f959c15f0e0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c006d80 con 0x7f959c15f0e0 2024-04-25T10:02:04.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c145560 con 0x7f959c15f0e0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c067c30 con 0x7f959c15f0e0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f958c01a020 con 0x7f959c15f0e0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 0x7f957c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f958c084840 con 0x7f959c15f0e0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9599d9b640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 0x7f957c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.852+0000 7f9599d9b640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 0x7f957c069450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9584000c10 tx=0x7f9584009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:04.856+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f958c055ba0 con 0x7f959c15f0e0 2024-04-25T10:02:05.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.012+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f959c06dab0 con 0x7f959c15f0e0 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.320+0000 7f9592ffd640 1 -- 172.21.15.82:0/1034710109 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f958c0554a0 con 0x7f959c15f0e0 2024-04-25T10:02:05.330 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 msgr2=0x7f957c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 0x7f957c069450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9584000c10 tx=0x7f9584009040 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 msgr2=0x7f959c154c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f958c00db10 tx=0x7f958c0020f0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 shutdown_connections 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f957c066f90 0x7f957c069450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f959c146e70 0x7f959c1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f959c15f4c0 0x7f959c155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 --2- 172.21.15.82:0/1034710109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f959c15f0e0 0x7f959c154c30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 >> 172.21.15.82:0/1034710109 conn(0x7f959c08b000 msgr2=0x7f959c0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 shutdown_connections 2024-04-25T10:02:05.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.324+0000 7f95a0978640 1 -- 172.21.15.82:0/1034710109 wait complete. 2024-04-25T10:02:05.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T10:02:05.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 Processor -- start 2024-04-25T10:02:05.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 -- start start 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff50815b6b0 0x7ff5080ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff5080cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080cbd20 con 0x7ff50815f130 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50815ba90 con 0x7ff50815b6b0 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff510528640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cff30 con 0x7ff5080cf590 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff50815f130 0x7ff50815f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57902/0 (socket says 172.21.15.82:57902) 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 -- 172.21.15.82:0/3779540508 learned_addr learned my addr 172.21.15.82:0/3779540508 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:05.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494668817 0 0) 0x7ff5080cbd20 con 0x7ff50815f130 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3779540508 >> v2:172.21.15.196:3300/0 conn(0x7ff50815b6b0 0x7ff5080ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f4003530 con 0x7ff50815f130 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff5080cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3559997738 0 0) 0x7ff4f4003530 con 0x7ff50815f130 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 msgr2=0x7ff5080cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff5080cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 >> v2:172.21.15.196:3300/0 conn(0x7ff50815b6b0 msgr2=0x7ff5080ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 --2- 172.21.15.82:0/3779540508 >> v2:172.21.15.196:3300/0 conn(0x7ff50815b6b0 0x7ff5080ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080cc000 con 0x7ff50815f130 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff500001a00 con 0x7ff50815f130 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff5080cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 >> v1:172.21.15.82:6789/0 conn(0x7ff50815f130 legacy=0x7ff50815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4003a30 0x7ff4f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff4f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f4004d50 0x7ff4f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50815ba90 con 0x7ff4f4004d50 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f400b9d0 con 0x7ff4f4003a30 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f400bb80 con 0x7ff5080cf590 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f4004d50 0x7ff4f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff4f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4003a30 0x7ff4f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 -- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 msgr2=0x7ff4f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cf590 0x7ff4f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 -- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4003a30 msgr2=0x7ff4f4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.472+0000 7ff50eac5640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4003a30 0x7ff4f4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50eac5640 1 -- 172.21.15.82:0/3779540508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f4003530 con 0x7ff4f4004d50 2024-04-25T10:02:05.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4003a30 0x7ff4f4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50eac5640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f4004d50 0x7ff4f40051d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff50000a8a0 tx=0x7ff50001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f744b8d02473ed8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000a4c0 con 0x7ff4f4004d50 2024-04-25T10:02:05.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5000a9020 con 0x7ff4f4004d50 2024-04-25T10:02:05.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50d2c2640 1 -- 172.21.15.82:0/3779540508 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000072d0 con 0x7ff4f4004d50 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 -- 172.21.15.82:0/3779540508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f4004d50 msgr2=0x7ff4f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 --2- 172.21.15.82:0/3779540508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff4f4004d50 0x7ff4f40051d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff50000a8a0 tx=0x7ff50001d3c0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff50eac5640 1 -- 172.21.15.82:0/3779540508 reap_dead start 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 -- 172.21.15.82:0/3779540508 shutdown_connections 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 -- 172.21.15.82:0/3779540508 >> 172.21.15.82:0/3779540508 conn(0x7ff50806ae60 msgr2=0x7ff50806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 -- 172.21.15.82:0/3779540508 shutdown_connections 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.476+0000 7ff510528640 1 -- 172.21.15.82:0/3779540508 wait complete. 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 Processor -- start 2024-04-25T10:02:05.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- start start 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4004d50 0x7ff50815a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cb200 0x7ff5080c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cc8a0 con 0x7ff5080ca840 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cc5c0 con 0x7ff4f4004d50 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cba30 con 0x7ff5080cb200 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46808/0 (socket says 172.21.15.82:46808) 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 -- 172.21.15.82:0/3299060916 learned_addr learned my addr 172.21.15.82:0/3299060916 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4004d50 0x7ff50815a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50eac5640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cb200 0x7ff5080c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 -- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cb200 msgr2=0x7ff5080c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cb200 0x7ff5080c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 -- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4004d50 msgr2=0x7ff50815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4004d50 0x7ff50815a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 -- 172.21.15.82:0/3299060916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080cb680 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff50dac3640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80108c0 tx=0x7ff4f8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f801a020 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f800cae0 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50815d1f0 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800e050 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5080cd210 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50804efa0 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.480+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4f801d020 con 0x7ff5080ca840 2024-04-25T10:02:05.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.484+0000 7ff4feffd640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 0x7ff4ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.484+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 0x7ff4ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.484+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff4f80854a0 con 0x7ff5080ca840 2024-04-25T10:02:05.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.484+0000 7ff50e2c4640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 0x7ff4ec069450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff504009b40 tx=0x7ff504008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.488+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4f8024020 con 0x7ff5080ca840 2024-04-25T10:02:05.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:05.644+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7ff50806b5f0 con 0x7ff5080ca840 2024-04-25T10:02:06.052 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:06.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:06.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.332+0000 7ff4feffd640 1 -- 172.21.15.82:0/3299060916 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7ff50815d1f0 con 0x7ff5080ca840 2024-04-25T10:02:06.340 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:02:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 msgr2=0x7ff4ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 0x7ff4ec069450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff504009b40 tx=0x7ff504008040 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 msgr2=0x7ff5080cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80108c0 tx=0x7ff4f8010d80 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 shutdown_connections 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f90 0x7ff4ec069450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080cb200 0x7ff5080c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080ca840 0x7ff5080cacc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 --2- 172.21.15.82:0/3299060916 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff4f4004d50 0x7ff50815a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 >> 172.21.15.82:0/3299060916 conn(0x7ff50806ae60 msgr2=0x7ff5081559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 shutdown_connections 2024-04-25T10:02:06.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.336+0000 7ff510528640 1 -- 172.21.15.82:0/3299060916 wait complete. 2024-04-25T10:02:06.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:02:06.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.496+0000 7f9d00336640 1 Processor -- start 2024-04-25T10:02:06.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.496+0000 7f9d00336640 1 -- start start 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9d00336640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf815b420 0x7f9cf80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9d00336640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9cf815b800 0x7f9cf80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9d00336640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf80cbd20 con 0x7f9cf80cfad0 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9d00336640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80cff20 con 0x7f9cf815b800 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9d00336640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cc930 con 0x7f9cf815b420 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9cf80cfad0 0x7f9cf815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57910/0 (socket says 172.21.15.82:57910) 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 -- 172.21.15.82:0/3035803515 learned_addr learned my addr 172.21.15.82:0/3035803515 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/3035803515 >> v2:172.21.15.196:3300/0 conn(0x7f9cf815b800 0x7f9cf80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe0d2640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf815b420 0x7f9cf80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124556170 0 0) 0x7f9cf80cbd20 con 0x7f9cf80cfad0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ce4003530 con 0x7f9cf80cfad0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 992167349 0 0) 0x7f9ce4003530 con 0x7f9cf80cfad0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf815b420 msgr2=0x7f9cf80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf815b420 0x7f9cf80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 >> v2:172.21.15.196:3300/0 conn(0x7f9cf815b800 msgr2=0x7f9cf80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 --2- 172.21.15.82:0/3035803515 >> v2:172.21.15.196:3300/0 conn(0x7f9cf815b800 0x7f9cf80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80cc000 con 0x7f9cf80cfad0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9cf0001a00 con 0x7f9cf80cfad0 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 >> v1:172.21.15.82:6789/0 conn(0x7f9cf80cfad0 legacy=0x7f9cf815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ce4003a30 0x7f9ce4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ce4004810 0x7f9ce4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9ce400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cff20 con 0x7f9ce40051d0 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cc930 con 0x7f9ce4004810 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce400bfc0 con 0x7f9ce4003a30 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe0d2640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf815b420 0x7f9cf80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9ce400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ce4004810 0x7f9ce4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 -- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ce4003a30 msgr2=0x7f9ce4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ce4003a30 0x7f9ce4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 -- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ce4004810 msgr2=0x7f9ce4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ce4004810 0x7f9ce4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfe8d3640 1 -- 172.21.15.82:0/3035803515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce4003530 con 0x7f9ce40051d0 2024-04-25T10:02:06.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.500+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ce4004810 0x7f9ce4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9ce400b9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000a850 tx=0x7f9cf001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf090748d77ba43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000a4a0 con 0x7f9ce40051d0 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf00ad650 con 0x7f9ce40051d0 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfd0d0640 1 -- 172.21.15.82:0/3035803515 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf00ada80 con 0x7f9ce40051d0 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/3035803515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 msgr2=0x7f9ce400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 --2- 172.21.15.82:0/3035803515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9ce400b9d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000a850 tx=0x7f9cf001d380 comp rx=0 tx=0).stop 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe8d3640 1 -- 172.21.15.82:0/3035803515 reap_dead start 2024-04-25T10:02:06.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/3035803515 shutdown_connections 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/3035803515 >> 172.21.15.82:0/3035803515 conn(0x7f9cf806ae60 msgr2=0x7f9cf806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/3035803515 shutdown_connections 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/3035803515 wait complete. 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 Processor -- start 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- start start 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf80af8d0 0x7f9cf80c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 0x7f9cf815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cb9f0 con 0x7f9ce40051d0 2024-04-25T10:02:06.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cc5c0 con 0x7f9cf80c30a0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf80cadb0 con 0x7f9cf80af8d0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46818/0 (socket says 172.21.15.82:46818) 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 -- 172.21.15.82:0/2415122669 learned_addr learned my addr 172.21.15.82:0/2415122669 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 0x7f9cf815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf80af8d0 0x7f9cf80c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 -- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf80af8d0 msgr2=0x7f9cf80c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf80af8d0 0x7f9cf80c2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 -- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 msgr2=0x7f9cf815c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 0x7f9cf815c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 -- 172.21.15.82:0/2415122669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf814c610 con 0x7f9ce40051d0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe8d3640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 0x7f9cf815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9cfe0d2640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d410 tx=0x7f9cf400d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40125f0 con 0x7f9ce40051d0 2024-04-25T10:02:06.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4018070 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf80cdae0 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.504+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40129d0 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80ce0a0 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf804efa0 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9cf4012b70 con 0x7f9ce40051d0 2024-04-25T10:02:06.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9ceeffd640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 0x7f9cdc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 0x7f9cdc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9cf4085120 con 0x7f9ce40051d0 2024-04-25T10:02:06.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.508+0000 7f9cfd8d1640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 0x7f9cdc069450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8068140 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:06.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.512+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9cf814c610 con 0x7f9ce40051d0 2024-04-25T10:02:06.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:06.712+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9cf80c9ac0 con 0x7f9ce40051d0 2024-04-25T10:02:07.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.336+0000 7f9ceeffd640 1 -- 172.21.15.82:0/2415122669 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9cf40560f0 con 0x7f9ce40051d0 2024-04-25T10:02:07.342 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:02:07.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 msgr2=0x7f9cdc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 0x7f9cdc069450 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8068140 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:07.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 msgr2=0x7f9cf80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400d410 tx=0x7f9cf400d8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 shutdown_connections 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9cdc066f90 0x7f9cdc069450 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9cf80af8d0 0x7f9cf80c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9cf80c30a0 0x7f9cf815c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 --2- 172.21.15.82:0/2415122669 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ce40051d0 0x7f9cf80ce8a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.340+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 >> 172.21.15.82:0/2415122669 conn(0x7f9cf806ae60 msgr2=0x7f9cf80afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.344+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 shutdown_connections 2024-04-25T10:02:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.348+0000 7f9d00336640 1 -- 172.21.15.82:0/2415122669 wait complete. 2024-04-25T10:02:07.398 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T10:02:07.398 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T10:02:07.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 Processor -- start 2024-04-25T10:02:07.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 -- start start 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 0x7ff06c0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0b3f70 con 0x7ff06c0b3b90 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0cef80 con 0x7ff06c0ce790 2024-04-25T10:02:07.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff071ecd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c06de70 con 0x7ff06c0cfc20 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 0x7ff06c0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06b7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff06c0b3b90 0x7ff06c0cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:54510/0 (socket says 172.21.15.82:54510) 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:44634/0 (socket says 172.21.15.82:44634) 2024-04-25T10:02:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06b7fe640 1 -- 172.21.15.82:0/3564392178 learned_addr learned my addr 172.21.15.82:0/3564392178 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.512+0000 7ff06bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 0x7ff06c0b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:47518/0 (socket says 172.21.15.82:47518) 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06a7fc640 1 -- 172.21.15.82:0/3564392178 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428548849 0 0) 0x7ff06c0b3f70 con 0x7ff06c0b3b90 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06a7fc640 1 -- 172.21.15.82:0/3564392178 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff054003530 con 0x7ff06c0b3b90 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06affd640 1 -- 172.21.15.82:0/3564392178 >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 msgr2=0x7ff06c0b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06affd640 1 --2- 172.21.15.82:0/3564392178 >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 0x7ff06c0b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06affd640 1 -- 172.21.15.82:0/3564392178 >> v1:172.21.15.196:6789/0 conn(0x7ff06c0b3b90 legacy=0x7ff06c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06affd640 1 -- 172.21.15.82:0/3564392178 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c06e150 con 0x7ff06c0cfc20 2024-04-25T10:02:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06affd640 1 --2- 172.21.15.82:0/3564392178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0600097c0 tx=0x7ff06001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=460ba580350fa572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06a7fc640 1 -- 172.21.15.82:0/3564392178 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600b1070 con 0x7ff06c0cfc20 2024-04-25T10:02:07.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06a7fc640 1 -- 172.21.15.82:0/3564392178 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0600ac440 con 0x7ff06c0cfc20 2024-04-25T10:02:07.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff06a7fc640 1 -- 172.21.15.82:0/3564392178 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0600ac7f0 con 0x7ff06c0cfc20 2024-04-25T10:02:07.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 -- 172.21.15.82:0/3564392178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 msgr2=0x7ff06c0ce250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 --2- 172.21.15.82:0/3564392178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff0600097c0 tx=0x7ff06001c410 comp rx=0 tx=0).stop 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 -- 172.21.15.82:0/3564392178 shutdown_connections 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 --2- 172.21.15.82:0/3564392178 >> v2:172.21.15.196:3300/0 conn(0x7ff06c0ce790 0x7ff06c0b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 --2- 172.21.15.82:0/3564392178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0cfc20 0x7ff06c0ce250 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 -- 172.21.15.82:0/3564392178 >> 172.21.15.82:0/3564392178 conn(0x7ff06c06ae60 msgr2=0x7ff06c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 -- 172.21.15.82:0/3564392178 shutdown_connections 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.516+0000 7ff071ecd640 1 -- 172.21.15.82:0/3564392178 wait complete. 2024-04-25T10:02:07.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 Processor -- start 2024-04-25T10:02:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- start start 2024-04-25T10:02:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0ce790 0x7ff06c15e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cfc20 0x7ff06c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c06edb0 con 0x7ff06c0b3b90 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c06ec30 con 0x7ff06c0cfc20 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c16a780 con 0x7ff06c0ce790 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46834/0 (socket says 172.21.15.82:46834) 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 -- 172.21.15.82:0/4007559523 learned_addr learned my addr 172.21.15.82:0/4007559523 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 -- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0ce790 msgr2=0x7ff06c15e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0ce790 0x7ff06c15e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 -- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cfc20 msgr2=0x7ff06c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cfc20 0x7ff06c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 -- 172.21.15.82:0/4007559523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c09a5c0 con 0x7ff06c0b3b90 2024-04-25T10:02:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff06b7fe640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c3f0 tx=0x7ff05c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c017070 con 0x7ff06c0b3b90 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c005cf0 con 0x7ff06c0b3b90 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c15b910 con 0x7ff06c0b3b90 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0072f0 con 0x7ff06c0b3b90 2024-04-25T10:02:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c15ef80 con 0x7ff06c0b3b90 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.520+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c067c30 con 0x7ff06c0b3b90 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.524+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff05c01a070 con 0x7ff06c0b3b90 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.524+0000 7ff053fff640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.524+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff05c084480 con 0x7ff06c0b3b90 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.524+0000 7ff06affd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.524+0000 7ff06affd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 0x7ff058069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0600ae000 tx=0x7ff060004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.544+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff05c055410 con 0x7ff06c0b3b90 2024-04-25T10:02:07.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:07.708+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7ff06c15e110 con 0x7ff06c0b3b90 2024-04-25T10:02:08.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.336+0000 7ff053fff640 1 -- 172.21.15.82:0/4007559523 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7ff05c054d10 con 0x7ff06c0b3b90 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 msgr2=0x7ff058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 0x7ff058069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff0600ae000 tx=0x7ff060004740 comp rx=0 tx=0).stop 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 msgr2=0x7ff06c15b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff05c00c3f0 tx=0x7ff05c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 shutdown_connections 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff058066f60 0x7ff058069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff06c0ce790 0x7ff06c15e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff06c0cfc20 0x7ff06c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 --2- 172.21.15.82:0/4007559523 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff06c0b3b90 0x7ff06c15b3d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 >> 172.21.15.82:0/4007559523 conn(0x7ff06c06ae60 msgr2=0x7ff06c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 shutdown_connections 2024-04-25T10:02:08.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.340+0000 7ff071ecd640 1 -- 172.21.15.82:0/4007559523 wait complete. 2024-04-25T10:02:08.368 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:02:08.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.492+0000 7ff97c7a8640 1 Processor -- start 2024-04-25T10:02:08.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.492+0000 7ff97c7a8640 1 -- start start 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.492+0000 7ff97c7a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff9740c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97c7a8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9740c9ef0 0x7ff9740ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97c7a8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740ca4c0 con 0x7ff9740cf590 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97c7a8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9740ca780 con 0x7ff9740c9ef0 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97c7a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff97415f0f0 con 0x7ff97415ecb0 2024-04-25T10:02:08.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9740c9ef0 0x7ff9740ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff979d43640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff9740c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9740c9ef0 0x7ff9740ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:47526/0 (socket says 172.21.15.82:47526) 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97a544640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff9740cf590 0x7ff9740cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:54516/0 (socket says 172.21.15.82:54516) 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 -- 172.21.15.82:0/1200630816 learned_addr learned my addr 172.21.15.82:0/1200630816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 -- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 msgr2=0x7ff9740c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:08.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff9740c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 -- 172.21.15.82:0/1200630816 >> v1:172.21.15.196:6789/0 conn(0x7ff9740cf590 legacy=0x7ff9740cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 -- 172.21.15.82:0/1200630816 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740cd690 con 0x7ff9740c9ef0 2024-04-25T10:02:08.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff979d43640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff9740c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff97ad45640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff9740ce170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9700098a0 tx=0x7ff97001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32e1a5ac2d019f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff979542640 1 -- 172.21.15.82:0/1200630816 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700b0070 con 0x7ff9740c9ef0 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff979542640 1 -- 172.21.15.82:0/1200630816 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9700ab600 con 0x7ff9740c9ef0 2024-04-25T10:02:08.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.496+0000 7ff979542640 1 -- 172.21.15.82:0/1200630816 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700ab9d0 con 0x7ff9740c9ef0 2024-04-25T10:02:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 msgr2=0x7ff9740ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff9740ce170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff9700098a0 tx=0x7ff97001c540 comp rx=0 tx=0).stop 2024-04-25T10:02:08.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- 172.21.15.82:0/1200630816 shutdown_connections 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff9740ce170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/1200630816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff9740c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- 172.21.15.82:0/1200630816 >> 172.21.15.82:0/1200630816 conn(0x7ff97406ae60 msgr2=0x7ff97406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- 172.21.15.82:0/1200630816 shutdown_connections 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- 172.21.15.82:0/1200630816 wait complete. 2024-04-25T10:02:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 Processor -- start 2024-04-25T10:02:08.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- start start 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff974159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff974153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974167870 con 0x7ff9740cf590 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741676f0 con 0x7ff9740c9ef0 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97c7a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741679f0 con 0x7ff97415ecb0 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46848/0 (socket says 172.21.15.82:46848) 2024-04-25T10:02:08.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 -- 172.21.15.82:0/3876981795 learned_addr learned my addr 172.21.15.82:0/3876981795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97a544640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff974159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97ad45640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff974153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 -- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 msgr2=0x7ff974153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff974153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 -- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 msgr2=0x7ff974159b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff974159b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 -- 172.21.15.82:0/3876981795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974155e60 con 0x7ff9740cf590 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff97a544640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff974159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff979d43640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff96400ec10 tx=0x7ff96400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.500+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964017070 con 0x7ff9740cf590 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400cc30 con 0x7ff9740cf590 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff974156140 con 0x7ff9740cf590 2024-04-25T10:02:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964013470 con 0x7ff9740cf590 2024-04-25T10:02:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff974156700 con 0x7ff9740cf590 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9740b4170 con 0x7ff9740cf590 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff96401a070 con 0x7ff9740cf590 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff96affd640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 0x7ff958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7ff9640854b0 con 0x7ff9740cf590 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff97a544640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 0x7ff958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.504+0000 7ff97a544640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 0x7ff958069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005fd0 tx=0x7ff96c005ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.508+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff964056390 con 0x7ff9740cf590 2024-04-25T10:02:08.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:08.668+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff97404ef80 con 0x7ff9740cf590 2024-04-25T10:02:10.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.384+0000 7ff96affd640 1 -- 172.21.15.82:0/3876981795 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff964055c90 con 0x7ff9740cf590 2024-04-25T10:02:10.391 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' created 2024-04-25T10:02:10.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 msgr2=0x7ff958069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 0x7ff958069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff96c005fd0 tx=0x7ff96c005ba0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 msgr2=0x7ff9741535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff96400ec10 tx=0x7ff96400c520 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 shutdown_connections 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff958066f60 0x7ff958069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff97415ecb0 0x7ff974153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9740cf590 0x7ff9741535e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 --2- 172.21.15.82:0/3876981795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9740c9ef0 0x7ff974159b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 >> 172.21.15.82:0/3876981795 conn(0x7ff97406ae60 msgr2=0x7ff9740ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 shutdown_connections 2024-04-25T10:02:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.388+0000 7ff97c7a8640 1 -- 172.21.15.82:0/3876981795 wait complete. 2024-04-25T10:02:10.422 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T10:02:10.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.536+0000 7f7139a80640 1 Processor -- start 2024-04-25T10:02:10.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.536+0000 7f7139a80640 1 -- start start 2024-04-25T10:02:10.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7139a80640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71340adc70 0x7f713406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7139a80640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340b28c0 con 0x7f71340cfc20 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7139a80640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713415bb20 con 0x7f713406e9f0 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7139a80640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340b2b60 con 0x7f71340adc70 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71340cfc20 0x7f71340ac170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57914/0 (socket says 172.21.15.82:57914) 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 -- 172.21.15.82:0/4251418887 learned_addr learned my addr 172.21.15.82:0/4251418887 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006821171 0 0) 0x7f71340b28c0 con 0x7f71340cfc20 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7120003530 con 0x7f71340cfc20 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4254343617 0 0) 0x7f7120003530 con 0x7f71340cfc20 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 >> v2:172.21.15.196:3300/0 conn(0x7f71340adc70 msgr2=0x7f713406e4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 --2- 172.21.15.82:0/4251418887 >> v2:172.21.15.196:3300/0 conn(0x7f71340adc70 0x7f713406e4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 >> v1:172.21.15.196:6789/0 conn(0x7f713406e9f0 legacy=0x7f71340cf510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:10.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340b2e40 con 0x7f71340cfc20 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f712c001a20 con 0x7f71340cfc20 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 >> v1:172.21.15.82:6789/0 conn(0x7f71340cfc20 legacy=0x7f71340ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71200039c0 0x7f7120003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7120004810 0x7f7120004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f712000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b2b60 con 0x7f71200052f0 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712000c1f0 con 0x7f71200039c0 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712000c3f0 con 0x7f7120004810 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f712000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f71337fe640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71200039c0 0x7f7120003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f712affd640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7120004810 0x7f7120004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 -- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7120004810 msgr2=0x7f7120004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7120004810 0x7f7120004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 -- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71200039c0 msgr2=0x7f7120003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71200039c0 0x7f7120003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 -- 172.21.15.82:0/4251418887 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120003530 con 0x7f71200052f0 2024-04-25T10:02:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 -- 172.21.15.82:0/4251418887 reap_dead start 2024-04-25T10:02:10.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7133fff640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f712000b9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71340b2a90 tx=0x7f712c01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c055cde1012afbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c00a4e0 con 0x7f71200052f0 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c0072f0 con 0x7f71200052f0 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.540+0000 7f7132ffd640 1 -- 172.21.15.82:0/4251418887 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0076e0 con 0x7f71200052f0 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 msgr2=0x7f712000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f712000b9d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f71340b2a90 tx=0x7f712c01d340 comp rx=0 tx=0).stop 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/4251418887 shutdown_connections 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 --2- 172.21.15.82:0/4251418887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f712000b9d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/4251418887 >> 172.21.15.82:0/4251418887 conn(0x7f713406ae60 msgr2=0x7f713406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/4251418887 shutdown_connections 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/4251418887 wait complete. 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 Processor -- start 2024-04-25T10:02:10.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- start start 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71340b1690 0x7f71340b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71340b0cf0 0x7f7134159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b36e0 con 0x7f71200052f0 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b3400 con 0x7f71340b0cf0 2024-04-25T10:02:10.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340b3c00 con 0x7f71340b1690 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46864/0 (socket says 172.21.15.82:46864) 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 -- 172.21.15.82:0/3405964938 learned_addr learned my addr 172.21.15.82:0/3405964938 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7133fff640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71340b0cf0 0x7f7134159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 -- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71340b1690 msgr2=0x7f71340b07b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71340b1690 0x7f71340b07b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 -- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71340b0cf0 msgr2=0x7f7134159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71340b0cf0 0x7f7134159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 -- 172.21.15.82:0/3405964938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71341f6ba0 con 0x7f71200052f0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f71337fe640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f712400d520 tx=0x7f712400d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124018020 con 0x7f71200052f0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124014620 con 0x7f71200052f0 2024-04-25T10:02:10.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71240149f0 con 0x7f71200052f0 2024-04-25T10:02:10.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71341f6e80 con 0x7f71200052f0 2024-04-25T10:02:10.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.544+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71341f7440 con 0x7f71200052f0 2024-04-25T10:02:10.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7134067c30 con 0x7f71200052f0 2024-04-25T10:02:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f712401b020 con 0x7f71200052f0 2024-04-25T10:02:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f7130ff9640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 0x7f7114069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f7124085160 con 0x7f71200052f0 2024-04-25T10:02:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f712affd640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 0x7f7114069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.548+0000 7f712affd640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 0x7f7114069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71340c5a60 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.552+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7124022070 con 0x7f71200052f0 2024-04-25T10:02:10.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:10.708+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f713406d760 con 0x7f71200052f0 2024-04-25T10:02:11.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.392+0000 7f7130ff9640 1 -- 172.21.15.82:0/3405964938 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f71240102e0 con 0x7f71200052f0 2024-04-25T10:02:11.399 INFO:teuthology.orchestra.run.smithi082.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.392+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 msgr2=0x7f7114069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.392+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 0x7f7114069450 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71340c5a60 tx=0x7f711c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.392+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 msgr2=0x7f71340aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.392+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f712400d520 tx=0x7f712400d9e0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 shutdown_connections 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7114066f90 0x7f7114069450 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71340b1690 0x7f71340b07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71340b0cf0 0x7f7134159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 --2- 172.21.15.82:0/3405964938 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71200052f0 0x7f71340aee00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 >> 172.21.15.82:0/3405964938 conn(0x7f713406ae60 msgr2=0x7f7134071380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 shutdown_connections 2024-04-25T10:02:11.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.396+0000 7f7139a80640 1 -- 172.21.15.82:0/3405964938 wait complete. 2024-04-25T10:02:11.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T10:02:11.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 Processor -- start 2024-04-25T10:02:11.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 -- start start 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba740cf590 0x7fba7415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740ce170 0x7fba740ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba740ca4c0 con 0x7fba740cf970 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740ca780 con 0x7fba740cf590 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba7a77a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740cd440 con 0x7fba740ce170 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba740cf590 0x7fba7415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba740cf590 0x7fba7415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46876/0 (socket says 172.21.15.82:46876) 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 -- 172.21.15.82:0/2180988374 learned_addr learned my addr 172.21.15.82:0/2180988374 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba78d17640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740ce170 0x7fba740ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 -- 172.21.15.82:0/2180988374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740ce170 msgr2=0x7fba740ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740ce170 0x7fba740ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 -- 172.21.15.82:0/2180988374 >> v1:172.21.15.196:6789/0 conn(0x7fba740cf970 legacy=0x7fba740c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.552+0000 7fba73fff640 1 -- 172.21.15.82:0/2180988374 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740cd690 con 0x7fba740cf590 2024-04-25T10:02:11.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740cf590 0x7fba7415ecb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d5bafa8e778a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba72ffd640 1 -- 172.21.15.82:0/2180988374 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640a7070 con 0x7fba740cf590 2024-04-25T10:02:11.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba72ffd640 1 -- 172.21.15.82:0/2180988374 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640b0070 con 0x7fba740cf590 2024-04-25T10:02:11.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba72ffd640 1 -- 172.21.15.82:0/2180988374 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640ab890 con 0x7fba740cf590 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/2180988374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740cf590 msgr2=0x7fba7415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740cf590 0x7fba7415ecb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6401c560 comp rx=0 tx=0).stop 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/2180988374 shutdown_connections 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740ce170 0x7fba740ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- 172.21.15.82:0/2180988374 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740cf590 0x7fba7415ecb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/2180988374 >> 172.21.15.82:0/2180988374 conn(0x7fba7406ae60 msgr2=0x7fba7406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/2180988374 shutdown_connections 2024-04-25T10:02:11.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/2180988374 wait complete. 2024-04-25T10:02:11.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 Processor -- start 2024-04-25T10:02:11.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- start start 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba740cf590 0x7fba7406de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740cf970 0x7fba74156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740cfed0 con 0x7fba740ce170 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba740cfd50 con 0x7fba740cf590 2024-04-25T10:02:11.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740ac170 con 0x7fba740cf970 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46882/0 (socket says 172.21.15.82:46882) 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 -- 172.21.15.82:0/1087184424 learned_addr learned my addr 172.21.15.82:0/1087184424 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 -- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740cf970 msgr2=0x7fba74156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740cf970 0x7fba74156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 -- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba740cf590 msgr2=0x7fba7406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba740cf590 0x7fba7406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 -- 172.21.15.82:0/1087184424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7409cfb0 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba73fff640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba640ad040 tx=0x7fba6401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640be5f0 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640be7b0 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7415ac70 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640beba0 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.556+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7415b230 con 0x7fba740ce170 2024-04-25T10:02:11.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7404efa0 con 0x7fba740ce170 2024-04-25T10:02:11.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba640b43f0 con 0x7fba740ce170 2024-04-25T10:02:11.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba70ff9640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 0x7fba60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fba640bee00 con 0x7fba740ce170 2024-04-25T10:02:11.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba737fe640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 0x7fba60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.560+0000 7fba737fe640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 0x7fba60069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba68000f70 tx=0x7fba680079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.564+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba640c4060 con 0x7fba740ce170 2024-04-25T10:02:11.631 INFO:tasks.ceph.mds.d.smithi196.stderr:2024-04-25T10:02:11.624+0000 7fe652c1c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:11.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:11.756+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fba740b4170 con 0x7fba740ce170 2024-04-25T10:02:12.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.392+0000 7fba70ff9640 1 -- 172.21.15.82:0/1087184424 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) v1 ==== 137+0+0 (secure 0 0 0) 0x7fba64016150 con 0x7fba740ce170 2024-04-25T10:02:12.401 INFO:teuthology.orchestra.run.smithi082.stderr:added data pool 4 to fsmap 2024-04-25T10:02:12.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 msgr2=0x7fba60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 0x7fba60069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fba68000f70 tx=0x7fba680079e0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 msgr2=0x7fba7406d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fba640ad040 tx=0x7fba6401ada0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 shutdown_connections 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba60066f60 0x7fba60069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba740cf970 0x7fba74156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba740cf590 0x7fba7406de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 --2- 172.21.15.82:0/1087184424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba740ce170 0x7fba7406d8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 >> 172.21.15.82:0/1087184424 conn(0x7fba7406ae60 msgr2=0x7fba7406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 shutdown_connections 2024-04-25T10:02:12.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.396+0000 7fba7a77a640 1 -- 172.21.15.82:0/1087184424 wait complete. 2024-04-25T10:02:12.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 Processor -- start 2024-04-25T10:02:12.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 -- start start 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 0x7fb1d806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1d806ec60 0x7fb1d8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d80ccc70 con 0x7fb1d80ce510 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d80cc2b0 con 0x7fb1d806ec60 2024-04-25T10:02:12.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dfecb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d806e290 con 0x7fb1d8144ca0 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1de468640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1d806ec60 0x7fb1d8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dd466640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 0x7fb1d806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1ddc67640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fb1d80ce510 0x7fb1d80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:54550/0 (socket says 172.21.15.82:54550) 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1ddc67640 1 -- 172.21.15.82:0/1145464955 learned_addr learned my addr 172.21.15.82:0/1145464955 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1de468640 1 -- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 msgr2=0x7fb1d806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1de468640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 0x7fb1d806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1de468640 1 -- 172.21.15.82:0/1145464955 >> v1:172.21.15.196:6789/0 conn(0x7fb1d80ce510 legacy=0x7fb1d80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1de468640 1 -- 172.21.15.82:0/1145464955 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d806dc40 con 0x7fb1d806ec60 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dcc65640 1 -- 172.21.15.82:0/1145464955 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455548873 0 0) 0x7fb1d80ccc70 con 0x7fb1d80ce510 2024-04-25T10:02:12.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.544+0000 7fb1dd466640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 0x7fb1d806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:12.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1de468640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d806ec60 0x7fb1d8142680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00097c0 tx=0x7fb1d001ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddcef573857081e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dcc65640 1 -- 172.21.15.82:0/1145464955 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b5070 con 0x7fb1d806ec60 2024-04-25T10:02:12.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dcc65640 1 -- 172.21.15.82:0/1145464955 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d00a8990 con 0x7fb1d806ec60 2024-04-25T10:02:12.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dcc65640 1 -- 172.21.15.82:0/1145464955 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00a8d60 con 0x7fb1d806ec60 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d806ec60 msgr2=0x7fb1d8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d806ec60 0x7fb1d8142680 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1d00097c0 tx=0x7fb1d001ec90 comp rx=0 tx=0).stop 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- 172.21.15.82:0/1145464955 shutdown_connections 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d806ec60 0x7fb1d8142680 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/1145464955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d8144ca0 0x7fb1d806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- 172.21.15.82:0/1145464955 >> 172.21.15.82:0/1145464955 conn(0x7fb1d80b11a0 msgr2=0x7fb1d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- 172.21.15.82:0/1145464955 shutdown_connections 2024-04-25T10:02:12.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- 172.21.15.82:0/1145464955 wait complete. 2024-04-25T10:02:12.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 Processor -- start 2024-04-25T10:02:12.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.548+0000 7fb1dfecb640 1 -- start start 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d806ec60 0x7fb1d81f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d80ce510 0x7fb1d80c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d81451f0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d80cc900 con 0x7fb1d80ce510 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d8145370 con 0x7fb1d806ec60 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46884/0 (socket says 172.21.15.82:46884) 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 -- 172.21.15.82:0/4025852361 learned_addr learned my addr 172.21.15.82:0/4025852361 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dd466640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d80ce510 0x7fb1d80c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ddc67640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d806ec60 0x7fb1d81f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 -- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d806ec60 msgr2=0x7fb1d81f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d806ec60 0x7fb1d81f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 -- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d80ce510 msgr2=0x7fb1d80c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d80ce510 0x7fb1d80c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 -- 172.21.15.82:0/4025852361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80c7d10 con 0x7fb1d8144ca0 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1de468640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001c8c0 tx=0x7fb1d001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00bf070 con 0x7fb1d8144ca0 2024-04-25T10:02:12.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d00b0640 con 0x7fb1d8144ca0 2024-04-25T10:02:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d80c7ff0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b5040 con 0x7fb1d8144ca0 2024-04-25T10:02:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d80c85b0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d804efa0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1d00c2070 con 0x7fb1d8144ca0 2024-04-25T10:02:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 0x7fb1bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ddc67640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 0x7fb1bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.552+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb1d0121c20 con 0x7fb1d8144ca0 2024-04-25T10:02:12.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.556+0000 7fb1ddc67640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 0x7fb1bc069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4000f70 tx=0x7fb1d4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.556+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1d00187f0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.752+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1d80cb4a0 con 0x7fb1d8144ca0 2024-04-25T10:02:12.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.752+0000 7fb1ca7fc640 1 -- 172.21.15.82:0/4025852361 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fb1d00f2960 con 0x7fb1d8144ca0 2024-04-25T10:02:12.761 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:12.762 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:11.763984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:12.762 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 6 2024-04-25T10:02:12.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 msgr2=0x7fb1bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 0x7fb1bc069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4000f70 tx=0x7fb1d4007b60 comp rx=0 tx=0).stop 2024-04-25T10:02:12.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 msgr2=0x7fb1d80c9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001c8c0 tx=0x7fb1d001ba30 comp rx=0 tx=0).stop 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 shutdown_connections 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1bc066f60 0x7fb1bc069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1d8144ca0 0x7fb1d80c9ea0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1d80ce510 0x7fb1d80c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 --2- 172.21.15.82:0/4025852361 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1d806ec60 0x7fb1d81f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 >> 172.21.15.82:0/4025852361 conn(0x7fb1d80b11a0 msgr2=0x7fb1d80c5490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 shutdown_connections 2024-04-25T10:02:12.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.756+0000 7fb1dfecb640 1 -- 172.21.15.82:0/4025852361 wait complete. 2024-04-25T10:02:12.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 Processor -- start 2024-04-25T10:02:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 -- start start 2024-04-25T10:02:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94fc144ca0 0x7f94fc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94fc06ec60 0x7f94fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc0ccc70 con 0x7f94fc0ce510 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc0cc2b0 con 0x7f94fc06ec60 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f950362d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc06e290 con 0x7f94fc144ca0 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94fc06ec60 0x7f94fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94fc06ec60 0x7f94fc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46900/0 (socket says 172.21.15.82:46900) 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 -- 172.21.15.82:0/894023918 learned_addr learned my addr 172.21.15.82:0/894023918 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9500bc8640 1 --2- 172.21.15.82:0/894023918 >> v2:172.21.15.196:3300/0 conn(0x7f94fc144ca0 0x7f94fc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 -- 172.21.15.82:0/894023918 >> v2:172.21.15.196:3300/0 conn(0x7f94fc144ca0 msgr2=0x7f94fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 --2- 172.21.15.82:0/894023918 >> v2:172.21.15.196:3300/0 conn(0x7f94fc144ca0 0x7f94fc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 -- 172.21.15.82:0/894023918 >> v1:172.21.15.196:6789/0 conn(0x7f94fc0ce510 legacy=0x7f94fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 -- 172.21.15.82:0/894023918 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc06dc40 con 0x7f94fc06ec60 2024-04-25T10:02:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.888+0000 7f9501bca640 1 --2- 172.21.15.82:0/894023918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc06ec60 0x7f94fc142680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94fc070fe0 tx=0x7f94f401f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab2c7c8b3ec0554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94fbfff640 1 -- 172.21.15.82:0/894023918 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40b4070 con 0x7f94fc06ec60 2024-04-25T10:02:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94fbfff640 1 -- 172.21.15.82:0/894023918 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f40af400 con 0x7f94fc06ec60 2024-04-25T10:02:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94fbfff640 1 -- 172.21.15.82:0/894023918 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f40af7d0 con 0x7f94fc06ec60 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/894023918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc06ec60 msgr2=0x7f94fc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- 172.21.15.82:0/894023918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc06ec60 0x7f94fc142680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94fc070fe0 tx=0x7f94f401f1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/894023918 shutdown_connections 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- 172.21.15.82:0/894023918 >> v2:172.21.15.196:3300/0 conn(0x7f94fc144ca0 0x7f94fc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- 172.21.15.82:0/894023918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc06ec60 0x7f94fc142680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/894023918 >> 172.21.15.82:0/894023918 conn(0x7f94fc0b11a0 msgr2=0x7f94fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/894023918 shutdown_connections 2024-04-25T10:02:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/894023918 wait complete. 2024-04-25T10:02:12.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 Processor -- start 2024-04-25T10:02:12.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- start start 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 0x7f94fc132640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 0x7f94fc130c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc1451f0 con 0x7f94fc0ce510 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0cc900 con 0x7f94fc06ec60 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc145370 con 0x7f94fc144ca0 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46914/0 (socket says 172.21.15.82:46914) 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 -- 172.21.15.82:0/383022124 learned_addr learned my addr 172.21.15.82:0/383022124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f95013c9640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 0x7f94fc132640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9501bca640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 0x7f94fc130c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 -- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 msgr2=0x7f94fc130c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 0x7f94fc130c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 -- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 msgr2=0x7f94fc132640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 0x7f94fc132640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 -- 172.21.15.82:0/383022124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b04b0 con 0x7f94fc0ce510 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f95013c9640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 0x7f94fc132640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9501bca640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 0x7f94fc130c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f9500bc8640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d5c0 tx=0x7f94f000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f00125f0 con 0x7f94fc0ce510 2024-04-25T10:02:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f0019070 con 0x7f94fc0ce510 2024-04-25T10:02:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc1315a0 con 0x7f94fc0ce510 2024-04-25T10:02:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f00129d0 con 0x7f94fc0ce510 2024-04-25T10:02:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.892+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc131b60 con 0x7f94fc0ce510 2024-04-25T10:02:12.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc04efa0 con 0x7f94fc0ce510 2024-04-25T10:02:12.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94f0012b70 con 0x7f94fc0ce510 2024-04-25T10:02:12.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f94f9ffb640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 0x7f94e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f94f00855b0 con 0x7f94fc0ce510 2024-04-25T10:02:12.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f95013c9640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 0x7f94e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.896+0000 7f95013c9640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 0x7f94e8069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003e40 tx=0x7f94ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:12.900+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94f0023020 con 0x7f94fc0ce510 2024-04-25T10:02:13.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.060+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94fc0b1cc0 con 0x7f94fc0ce510 2024-04-25T10:02:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f94f9ffb640 1 -- 172.21.15.82:0/383022124 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f94f0056280 con 0x7f94fc0ce510 2024-04-25T10:02:13.072 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:13.072 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 msgr2=0x7f94e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 0x7f94e8069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94ec003e40 tx=0x7f94ec009040 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 msgr2=0x7f94fc0aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f94f000d5c0 tx=0x7f94f000da80 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 shutdown_connections 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94e8066f60 0x7f94e8069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94fc144ca0 0x7f94fc130c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94fc0ce510 0x7f94fc0aff70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.064+0000 7f950362d640 1 --2- 172.21.15.82:0/383022124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94fc06ec60 0x7f94fc132640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.068+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 >> 172.21.15.82:0/383022124 conn(0x7f94fc0b11a0 msgr2=0x7f94fc0c6690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.068+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 shutdown_connections 2024-04-25T10:02:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.068+0000 7f950362d640 1 -- 172.21.15.82:0/383022124 wait complete. 2024-04-25T10:02:13.091 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:13.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 Processor -- start 2024-04-25T10:02:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 -- start start 2024-04-25T10:02:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2880af280 0x7fd28815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2880b2c30 0x7fd28806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd28806c1e0 0x7fd288161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28806daa0 con 0x7fd28806c1e0 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28806c640 con 0x7fd2880b2c30 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd288066c30 con 0x7fd2880af280 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd28806c1e0 0x7fd288161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd28806c1e0 0x7fd288161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46922/0 (socket says 172.21.15.82:46922) 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 -- 172.21.15.82:0/1925848015 learned_addr learned my addr 172.21.15.82:0/1925848015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd285d9b640 1 --2- 172.21.15.82:0/1925848015 >> v2:172.21.15.196:3300/0 conn(0x7fd2880b2c30 0x7fd28806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd28659c640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2880af280 0x7fd28815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 -- 172.21.15.82:0/1925848015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2880af280 msgr2=0x7fd28815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2880af280 0x7fd28815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 -- 172.21.15.82:0/1925848015 >> v2:172.21.15.196:3300/0 conn(0x7fd2880b2c30 msgr2=0x7fd28806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 --2- 172.21.15.82:0/1925848015 >> v2:172.21.15.196:3300/0 conn(0x7fd2880b2c30 0x7fd28806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.204+0000 7fd286d9d640 1 -- 172.21.15.82:0/1925848015 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880af660 con 0x7fd28806c1e0 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd28806c1e0 0x7fd288161500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd28806d7c0 tx=0x7fd27801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e571c3c48a13e1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28559a640 1 -- 172.21.15.82:0/1925848015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b0070 con 0x7fd28806c1e0 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28559a640 1 -- 172.21.15.82:0/1925848015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2780ab600 con 0x7fd28806c1e0 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28559a640 1 -- 172.21.15.82:0/1925848015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780ab9d0 con 0x7fd28806c1e0 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- 172.21.15.82:0/1925848015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd28806c1e0 msgr2=0x7fd288161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd28806c1e0 0x7fd288161500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd28806d7c0 tx=0x7fd27801c260 comp rx=0 tx=0).stop 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- 172.21.15.82:0/1925848015 shutdown_connections 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd28806c1e0 0x7fd288161500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- 172.21.15.82:0/1925848015 >> v2:172.21.15.196:3300/0 conn(0x7fd2880b2c30 0x7fd28806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- 172.21.15.82:0/1925848015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2880af280 0x7fd28815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- 172.21.15.82:0/1925848015 >> 172.21.15.82:0/1925848015 conn(0x7fd288071810 msgr2=0x7fd288071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- 172.21.15.82:0/1925848015 shutdown_connections 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- 172.21.15.82:0/1925848015 wait complete. 2024-04-25T10:02:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 Processor -- start 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- start start 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd28806c1e0 0x7fd28815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2880af280 0x7fd2880ae550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806e090 con 0x7fd2880b2c30 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806df10 con 0x7fd2880af280 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28d000640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28806e210 con 0x7fd28806c1e0 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46928/0 (socket says 172.21.15.82:46928) 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 -- 172.21.15.82:0/2209670743 learned_addr learned my addr 172.21.15.82:0/2209670743 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 -- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd28806c1e0 msgr2=0x7fd28815cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd28659c640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd28806c1e0 0x7fd28815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd285d9b640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2880af280 0x7fd2880ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd28806c1e0 0x7fd28815cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 -- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2880af280 msgr2=0x7fd2880ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2880af280 0x7fd2880ae550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.208+0000 7fd286d9d640 1 -- 172.21.15.82:0/2209670743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28814bd10 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd286d9d640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2780ae040 tx=0x7fd27801aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b0040 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2780bc070 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2881528c0 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2780b6660 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd288152e80 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28804efa0 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2780b6800 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd27811f670 con 0x7fd2880b2c30 2024-04-25T10:02:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd28659c640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.212+0000 7fd28659c640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 0x7fd268069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2880ae2d0 tx=0x7fd28000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.216+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd278016110 con 0x7fd2880b2c30 2024-04-25T10:02:13.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.372+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2880727c0 con 0x7fd2880b2c30 2024-04-25T10:02:13.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.372+0000 7fd276ffd640 1 -- 172.21.15.82:0/2209670743 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fd2780f0340 con 0x7fd2880b2c30 2024-04-25T10:02:13.381 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:13.381 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 msgr2=0x7fd268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 0x7fd268069420 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2880ae2d0 tx=0x7fd28000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 msgr2=0x7fd2880aea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd2780ae040 tx=0x7fd27801aaa0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 shutdown_connections 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd268066f60 0x7fd268069420 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2880b2c30 0x7fd2880aea90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2880af280 0x7fd2880ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 --2- 172.21.15.82:0/2209670743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd28806c1e0 0x7fd28815cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 >> 172.21.15.82:0/2209670743 conn(0x7fd288071810 msgr2=0x7fd2880b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 shutdown_connections 2024-04-25T10:02:13.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.376+0000 7fd28d000640 1 -- 172.21.15.82:0/2209670743 wait complete. 2024-04-25T10:02:13.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:13.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 Processor -- start 2024-04-25T10:02:13.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 -- start start 2024-04-25T10:02:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 0x7f2cc80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cc80d1cd0 0x7f2cc806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc806be60 con 0x7f2cc815da80 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc806c440 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2ccc9ae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc815de60 con 0x7f2cc80d07f0 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2cc815da80 0x7f2cc80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57922/0 (socket says 172.21.15.82:57922) 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2cc80d1cd0 0x7f2cc806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc659c640 1 -- 172.21.15.82:0/1411181736 learned_addr learned my addr 172.21.15.82:0/1411181736 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cbffff640 1 --2- 172.21.15.82:0/1411181736 >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 0x7f2cc80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 -- 172.21.15.82:0/1411181736 >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 msgr2=0x7f2cc80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 --2- 172.21.15.82:0/1411181736 >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 0x7f2cc80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc5d9b640 1 -- 172.21.15.82:0/1411181736 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3171846560 0 0) 0x7f2cc806be60 con 0x7f2cc815da80 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 -- 172.21.15.82:0/1411181736 >> v1:172.21.15.82:6789/0 conn(0x7f2cc815da80 legacy=0x7f2cc80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 -- 172.21.15.82:0/1411181736 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc80d2450 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cbffff640 1 --2- 172.21.15.82:0/1411181736 >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 0x7f2cc80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc6d9d640 1 --2- 172.21.15.82:0/1411181736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc806c8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cc808d880 tx=0x7f2cc001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c596603390b5848d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.516+0000 7f2cc5d9b640 1 -- 172.21.15.82:0/1411181736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00b2070 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cc5d9b640 1 -- 172.21.15.82:0/1411181736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc00a59e0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cc5d9b640 1 -- 172.21.15.82:0/1411181736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00a5db0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/1411181736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 msgr2=0x7f2cc806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/1411181736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc806c8d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2cc808d880 tx=0x7f2cc001c900 comp rx=0 tx=0).stop 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/1411181736 shutdown_connections 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/1411181736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc806c8d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/1411181736 >> v2:172.21.15.196:3300/0 conn(0x7f2cc80d07f0 0x7f2cc80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/1411181736 >> 172.21.15.82:0/1411181736 conn(0x7f2cc808b000 msgr2=0x7f2cc80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/1411181736 shutdown_connections 2024-04-25T10:02:13.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/1411181736 wait complete. 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 Processor -- start 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- start start 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 0x7f2cc8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cc815da80 0x7f2cc80b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc806d7a0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc806d620 con 0x7f2cc80d07f0 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc806d920 con 0x7f2cc815da80 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46952/0 (socket says 172.21.15.82:46952) 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 -- 172.21.15.82:0/3329503742 learned_addr learned my addr 172.21.15.82:0/3329503742 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 -- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cc815da80 msgr2=0x7f2cc80b8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cc815da80 0x7f2cc80b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 -- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 msgr2=0x7f2cc8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cc659c640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 0x7f2cc8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 0x7f2cc8159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 -- 172.21.15.82:0/3329503742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc80cbda0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cc659c640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 0x7f2cc8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbffff640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c7f0 tx=0x7f2cb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0005ba0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0005d40 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc80cc080 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0007a50 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.520+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc80c7720 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc804efa0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2cb0007bf0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2cbeffd640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2cb0085810 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2cc659c640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.524+0000 7f2cc659c640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 0x7f2ca8069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007fd0 tx=0x7f2cb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.528+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cb0015020 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cc80b1da0 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2cbeffd640 1 -- 172.21.15.82:0/3329503742 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f2cb0056530 con 0x7f2cc80d1cd0 2024-04-25T10:02:13.727 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:13.727 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:11.763984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:13.727 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 6 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 msgr2=0x7f2ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 0x7f2ca8069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007fd0 tx=0x7f2cb8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 msgr2=0x7f2cc80c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c7f0 tx=0x7f2cb000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 shutdown_connections 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ca8066f60 0x7f2ca8069420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2cc815da80 0x7f2cc80b8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cc80d1cd0 0x7f2cc80c71e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 --2- 172.21.15.82:0/3329503742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cc80d07f0 0x7f2cc8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 >> 172.21.15.82:0/3329503742 conn(0x7f2cc808b000 msgr2=0x7f2cc80b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 shutdown_connections 2024-04-25T10:02:13.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.720+0000 7f2ccc9ae640 1 -- 172.21.15.82:0/3329503742 wait complete. 2024-04-25T10:02:13.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:13.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.856+0000 7fe636494640 1 Processor -- start 2024-04-25T10:02:13.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 -- start start 2024-04-25T10:02:13.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6300ce510 0x7fe6300ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe630142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6300cee30 con 0x7fe630144ca0 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6300cc2b0 con 0x7fe6300ce510 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe636494640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63006e290 con 0x7fe6301451f0 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6300ce510 0x7fe6300ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6300ce510 0x7fe6300ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46966/0 (socket says 172.21.15.82:46966) 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 -- 172.21.15.82:0/928821881 learned_addr learned my addr 172.21.15.82:0/928821881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe634a31640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe630142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 -- 172.21.15.82:0/928821881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 msgr2=0x7fe630142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe630142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 -- 172.21.15.82:0/928821881 >> v1:172.21.15.196:6789/0 conn(0x7fe630144ca0 legacy=0x7fe63006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:13.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 -- 172.21.15.82:0/928821881 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6300ccc70 con 0x7fe6300ce510 2024-04-25T10:02:13.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.860+0000 7fe62ffff640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6300ce510 0x7fe6300ce8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe620009870 tx=0x7fe62001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18da7a64fc0738a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62effd640 1 -- 172.21.15.82:0/928821881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200b5070 con 0x7fe6300ce510 2024-04-25T10:02:13.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62effd640 1 -- 172.21.15.82:0/928821881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6200a8a50 con 0x7fe6300ce510 2024-04-25T10:02:13.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62effd640 1 -- 172.21.15.82:0/928821881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a8e20 con 0x7fe6300ce510 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/928821881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6300ce510 msgr2=0x7fe6300ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6300ce510 0x7fe6300ce8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe620009870 tx=0x7fe62001ed50 comp rx=0 tx=0).stop 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/928821881 shutdown_connections 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe630142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- 172.21.15.82:0/928821881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6300ce510 0x7fe6300ce8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/928821881 >> 172.21.15.82:0/928821881 conn(0x7fe6300b11a0 msgr2=0x7fe6300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/928821881 shutdown_connections 2024-04-25T10:02:13.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/928821881 wait complete. 2024-04-25T10:02:13.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 Processor -- start 2024-04-25T10:02:13.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- start start 2024-04-25T10:02:13.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6300ce510 0x7fe6301437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe6301e7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63006ee30 con 0x7fe630144ca0 2024-04-25T10:02:13.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63006df20 con 0x7fe6300ce510 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6300cc900 con 0x7fe6301451f0 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46972/0 (socket says 172.21.15.82:46972) 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 -- 172.21.15.82:0/1786561704 learned_addr learned my addr 172.21.15.82:0/1786561704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62ffff640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6300ce510 0x7fe6301437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe634a31640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe6301e7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 -- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 msgr2=0x7fe6301e7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe6301e7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 -- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6300ce510 msgr2=0x7fe6301437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:13.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6300ce510 0x7fe6301437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 -- 172.21.15.82:0/1786561704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6301e2c90 con 0x7fe630144ca0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62f7fe640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe62400c4d0 tx=0x7fe62400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6240052d0 con 0x7fe630144ca0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe624005470 con 0x7fe630144ca0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6301e2f70 con 0x7fe630144ca0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.864+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe624012830 con 0x7fe630144ca0 2024-04-25T10:02:13.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6301e3530 con 0x7fe630144ca0 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63004efa0 con 0x7fe630144ca0 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6240129d0 con 0x7fe630144ca0 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe62cff9640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 0x7fe61c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe624084e10 con 0x7fe630144ca0 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe62ffff640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 0x7fe61c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:13.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.868+0000 7fe62ffff640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 0x7fe61c069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b2000 tx=0x7fe620009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:13.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:13.872+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe624055490 con 0x7fe630144ca0 2024-04-25T10:02:14.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.028+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6300b0f80 con 0x7fe630144ca0 2024-04-25T10:02:14.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe62cff9640 1 -- 172.21.15.82:0/1786561704 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fe624059530 con 0x7fe630144ca0 2024-04-25T10:02:14.039 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:14.039 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 msgr2=0x7fe61c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 0x7fe61c069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b2000 tx=0x7fe620009040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 msgr2=0x7fe6301e4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe62400c4d0 tx=0x7fe62400c990 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 shutdown_connections 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe61c066f60 0x7fe61c069420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6301451f0 0x7fe6301e7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe630144ca0 0x7fe6301e4530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 --2- 172.21.15.82:0/1786561704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6300ce510 0x7fe6301437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 >> 172.21.15.82:0/1786561704 conn(0x7fe6300b11a0 msgr2=0x7fe6300c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 shutdown_connections 2024-04-25T10:02:14.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.032+0000 7fe636494640 1 -- 172.21.15.82:0/1786561704 wait complete. 2024-04-25T10:02:14.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:06.717111+0000', 'modified': '2024-04-25T10:02:11.763984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/3824448217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3824448217}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3824448217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T10:02:14.459 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:14.459 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.584+0000 7f3db5a0f640 1 Processor -- start 2024-04-25T10:02:14.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.584+0000 7f3db5a0f640 1 -- start start 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3db5a0f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3db015a700 0x7f3db015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3db5a0f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 0x7f3db00b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3db5a0f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db00b2090 con 0x7f3db00b3f50 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3db5a0f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db015f200 con 0x7f3db015a700 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3db5a0f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db00b43e0 con 0x7f3db00b24a0 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3db015a700 0x7f3db015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3db015a700 0x7f3db015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46982/0 (socket says 172.21.15.82:46982) 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 -- 172.21.15.82:0/2636318279 learned_addr learned my addr 172.21.15.82:0/2636318279 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/2636318279 >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 0x7f3db00b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 -- 172.21.15.82:0/2636318279 >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 msgr2=0x7f3db00b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 --2- 172.21.15.82:0/2636318279 >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 0x7f3db00b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 -- 172.21.15.82:0/2636318279 >> v1:172.21.15.196:6789/0 conn(0x7f3db00b3f50 legacy=0x7f3db00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 -- 172.21.15.82:0/2636318279 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db00b46c0 con 0x7f3db015a700 2024-04-25T10:02:14.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/2636318279 >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 0x7f3db00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:14.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.588+0000 7f3daeffd640 1 --2- 172.21.15.82:0/2636318279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db015a700 0x7f3db015ecc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3db006bd20 tx=0x7f3da001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4ffe03fc1ce185 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3dadffb640 1 -- 172.21.15.82:0/2636318279 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b4070 con 0x7f3db015a700 2024-04-25T10:02:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3dadffb640 1 -- 172.21.15.82:0/2636318279 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da00a7a30 con 0x7f3db015a700 2024-04-25T10:02:14.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3dadffb640 1 -- 172.21.15.82:0/2636318279 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00a7e00 con 0x7f3db015a700 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/2636318279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db015a700 msgr2=0x7f3db015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/2636318279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db015a700 0x7f3db015ecc0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3db006bd20 tx=0x7f3da001ea70 comp rx=0 tx=0).stop 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/2636318279 shutdown_connections 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/2636318279 >> v2:172.21.15.196:3300/0 conn(0x7f3db00b24a0 0x7f3db00b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/2636318279 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db015a700 0x7f3db015ecc0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/2636318279 >> 172.21.15.82:0/2636318279 conn(0x7f3db00b11a0 msgr2=0x7f3db00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/2636318279 shutdown_connections 2024-04-25T10:02:14.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/2636318279 wait complete. 2024-04-25T10:02:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 Processor -- start 2024-04-25T10:02:14.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- start start 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3db015a700 0x7f3db006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 0x7f3db00ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00cd310 con 0x7f3db00b24a0 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00ce600 con 0x7f3db006dde0 2024-04-25T10:02:14.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00cd490 con 0x7f3db015a700 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46992/0 (socket says 172.21.15.82:46992) 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 -- 172.21.15.82:0/597796462 learned_addr learned my addr 172.21.15.82:0/597796462 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 0x7f3db00ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3db015a700 0x7f3db006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 -- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3db015a700 msgr2=0x7f3db006d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3db015a700 0x7f3db006d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 -- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 msgr2=0x7f3db00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 0x7f3db00ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 -- 172.21.15.82:0/597796462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0160670 con 0x7f3db00b24a0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 0x7f3db00ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3daeffd640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3da001d120 tx=0x7f3da001d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3d97fff640 1 -- 172.21.15.82:0/597796462 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00bf070 con 0x7f3db00b24a0 2024-04-25T10:02:14.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3d97fff640 1 -- 172.21.15.82:0/597796462 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da00afc30 con 0x7f3db00b24a0 2024-04-25T10:02:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db015f5d0 con 0x7f3db00b24a0 2024-04-25T10:02:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3d97fff640 1 -- 172.21.15.82:0/597796462 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b4070 con 0x7f3db00b24a0 2024-04-25T10:02:14.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.592+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db015fb90 con 0x7f3db00b24a0 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3d97fff640 1 -- 172.21.15.82:0/597796462 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3da00c2070 con 0x7f3db00b24a0 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3d97fff640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 0x7f3d9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3d97fff640 1 -- 172.21.15.82:0/597796462 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3da011f7b0 con 0x7f3db00b24a0 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 0x7f3d9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 0x7f3d9c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3db0165440 tx=0x7f3da4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 Processor -- start 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 -- start start 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d168100d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168101240 0x55d1680b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167f9e5f0 con 0x55d1680b6f30 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167f05da0 con 0x55d168100920 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d167e8e150 con 0x55d168101240 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47000/0 (socket says 172.21.15.82:47000) 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 -- 172.21.15.82:0/1852804284 learned_addr learned my addr 172.21.15.82:0/1852804284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168101240 0x55d1680b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daeffd640 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d168100d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 -- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168101240 msgr2=0x55d1680b69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168101240 0x55d1680b69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 -- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 msgr2=0x55d168100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.596+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d168100d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 -- 172.21.15.82:0/1852804284 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d167ea7ca0 con 0x55d1680b6f30 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007fb0 tx=0x7f3da800d800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c219a6733bbb20e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d957fa640 1 -- 172.21.15.82:0/1852804284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800f920 con 0x55d1680b6f30 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d957fa640 1 -- 172.21.15.82:0/1852804284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800fac0 con 0x55d1680b6f30 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d957fa640 1 -- 172.21.15.82:0/1852804284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018480 con 0x55d1680b6f30 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/1852804284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 msgr2=0x55d1680b7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3da8007fb0 tx=0x7f3da800d800 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/1852804284 shutdown_connections 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168101240 0x55d1680b69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1680b6f30 0x55d1680b7310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/1852804284 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d168100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/1852804284 >> 172.21.15.82:0/1852804284 conn(0x55d1680bd2a0 msgr2=0x55d1680b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/1852804284 shutdown_connections 2024-04-25T10:02:14.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/1852804284 wait complete. 2024-04-25T10:02:14.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- start start 2024-04-25T10:02:14.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 Processor -- start 2024-04-25T10:02:14.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168107c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681081c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167ec0990 con 0x55d168101240 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167f01e40 con 0x55d1680b6f30 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d167e91900 con 0x55d168100920 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 10 client.0 _ll_register_callbacks cb 0x55d167fb91c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47016/0 (socket says 172.21.15.82:47016) 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 -- 172.21.15.82:0/3001502124 learned_addr learned my addr 172.21.15.82:0/3001502124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daeffd640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168107c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681081c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 -- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 msgr2=0x55d1681081c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681081c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 -- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 msgr2=0x55d168107c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168107c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 -- 172.21.15.82:0/3001502124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d167ebe630 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55d167ea9200 tx=0x7f3da8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aed2f0f67a64f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018890 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3da800fac0 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3da8018a30 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da8018c30 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 10 client.4395 fetch_fsmap learned FSMap version 6 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d167e8e3a0 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800fac0 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f3da800c040 con 0x55d168101240 2024-04-25T10:02:14.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 10 client.4395 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.600+0000 7f3db8b7c000 10 client.4395 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 20 client.4395 populate_metadata read hostname 'smithi082' 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d1680bb4b0 0x55d1680bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 4 client.4395 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 --> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d167ea1a50 con 0x55d1680bb4b0 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3daeffd640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d1680bb4b0 0x55d1680bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3d8e7fc640 10 client.4395 ms_handle_connect on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3daeffd640 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d1680bb4b0 0x55d1680bb890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e15e8b26868ca5f0 server_cookie=39764aafa46a651e in_seq=0 out_seq=0 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/3001502124 <== mds.0 v2:172.21.15.196:6834/3824448217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d167ea1a50 con 0x55d1680bb4b0 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3d8e7fc640 10 client.4395 handle_command_reply: tid=0 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 client.4395 shutdown 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 msgr2=0x55d1680baaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55d167ea9200 tx=0x7f3da8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 shutdown_connections 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d1680bb4b0 0x55d1680bb890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d1680baaa0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3d8e7fc640 0 client.4395 ms_handle_reset on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/3001502124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 >> 172.21.15.82:0/3001502124 conn(0x55d1680bd2a0 msgr2=0x55d1680b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 shutdown_connections 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 1 -- 172.21.15.82:0/3001502124 wait complete. 2024-04-25T10:02:14.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.604+0000 7f3db8b7c000 20 client.4395 trim_cache size 0 max 16384 2024-04-25T10:02:14.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 Processor -- start 2024-04-25T10:02:14.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 -- start start 2024-04-25T10:02:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1680b6f30 0x55d16815b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d16815bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0203030 con 0x55d168101240 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0163940 con 0x55d168100920 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0202a30 con 0x55d1680b6f30 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47024/0 (socket says 172.21.15.82:47024) 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 -- 172.21.15.82:0/825517244 learned_addr learned my addr 172.21.15.82:0/825517244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 -- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1680b6f30 msgr2=0x55d16815b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1680b6f30 0x55d16815b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 -- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 msgr2=0x55d16815bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d16815bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 -- 172.21.15.82:0/825517244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d168101070 con 0x55d168101240 2024-04-25T10:02:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3da80073c0 tx=0x7f3da801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8889a766e876705f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3d94ff9640 1 -- 172.21.15.82:0/825517244 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da800faa0 con 0x55d168101240 2024-04-25T10:02:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3d94ff9640 1 -- 172.21.15.82:0/825517244 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8027070 con 0x55d168101240 2024-04-25T10:02:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.620+0000 7f3d94ff9640 1 -- 172.21.15.82:0/825517244 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018690 con 0x55d168101240 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/825517244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 msgr2=0x55d16815c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3da80073c0 tx=0x7f3da801f000 comp rx=0 tx=0).stop 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/825517244 shutdown_connections 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d16815c210 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d168100920 0x55d16815bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/825517244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1680b6f30 0x55d16815b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/825517244 >> 172.21.15.82:0/825517244 conn(0x55d1680bd2a0 msgr2=0x55d1680ba470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/825517244 shutdown_connections 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/825517244 wait complete. 2024-04-25T10:02:14.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- start start 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 Processor -- start 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681158f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167f0a3d0 con 0x55d168101240 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d167fc1d50 con 0x55d1680b6f30 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d167ea0980 con 0x55d168100920 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 10 client.0 _ll_register_callbacks cb 0x7f3da8007750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47028/0 (socket says 172.21.15.82:47028) 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 -- 172.21.15.82:0/708085412 learned_addr learned my addr 172.21.15.82:0/708085412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daeffd640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3dae7fc640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681158f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 -- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 msgr2=0x55d1681158f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681158f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 -- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 msgr2=0x55d168144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168144230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 -- 172.21.15.82:0/708085412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d167f87600 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daf7fe640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3da8012040 tx=0x7f3da8007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947fe8a26106ca57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8041060 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3da8018690 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3da8027070 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da8047050 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 10 client.4401 fetch_fsmap learned FSMap version 6 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d167e9a9b0 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8027070 con 0x55d168101240 2024-04-25T10:02:14.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 6) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f3da804a020 con 0x55d168101240 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 10 client.4401 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 20 client.4401 populate_metadata read hostname 'smithi082' 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d168116370 0x55d168116750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 4 client.4401 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 --> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d167f132e0 con 0x55d168116370 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.624+0000 7f3daeffd640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d168116370 0x55d168116750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3d8e7fc640 10 client.4401 ms_handle_connect on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3daeffd640 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d168116370 0x55d168116750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3563d62d0575aa12 server_cookie=df6668be7b373835 in_seq=0 out_seq=0 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3d8e7fc640 1 -- 172.21.15.82:0/708085412 <== mds.0 v2:172.21.15.196:6834/3824448217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x55d167f132e0 con 0x55d168116370 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3d8e7fc640 10 client.4401 handle_command_reply: tid=0 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 client.4401 shutdown 2024-04-25T10:02:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 msgr2=0x55d168115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3da8012040 tx=0x7f3da8007880 comp rx=0 tx=0).stop 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 shutdown_connections 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55d168116370 0x55d168116750 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d168100920 0x55d1681158f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d168101240 0x55d168115e30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 --2- 172.21.15.82:0/708085412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1680b6f30 0x55d168144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3d8e7fc640 0 client.4401 ms_handle_reset on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 >> 172.21.15.82:0/708085412 conn(0x55d1680bd2a0 msgr2=0x55d168110e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 shutdown_connections 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 1 -- 172.21.15.82:0/708085412 wait complete. 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db8b7c000 20 client.4401 trim_cache size 0 max 16384 2024-04-25T10:02:14.640 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4282, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 7.2906113030000004, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 6, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 25, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 13.97063677 2024-04-25T10:02:14.641 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:02:14.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 msgr2=0x7f3d9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 0x7f3d9c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3db0165440 tx=0x7f3da4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 msgr2=0x7f3db0155b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3da001d120 tx=0x7f3da001d2b0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 shutdown_connections 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d9c066f60 0x7f3d9c069420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3db015a700 0x7f3db006d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3db006dde0 0x7f3db00ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 --2- 172.21.15.82:0/597796462 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3db00b24a0 0x7f3db0155b20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 >> 172.21.15.82:0/597796462 conn(0x7f3db00b11a0 msgr2=0x7f3db015cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 shutdown_connections 2024-04-25T10:02:14.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.628+0000 7f3db5a0f640 1 -- 172.21.15.82:0/597796462 wait complete. 2024-04-25T10:02:14.651 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:14.651 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:14.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 Processor -- start 2024-04-25T10:02:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 -- start start 2024-04-25T10:02:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 0x7efd880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd880cee30 con 0x7efd88144ca0 2024-04-25T10:02:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd880cc2b0 con 0x7efd880ce510 2024-04-25T10:02:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8d1ab640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806e290 con 0x7efd881451f0 2024-04-25T10:02:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd86d9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 0x7efd880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:36996/0 (socket says 172.21.15.82:36996) 2024-04-25T10:02:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd86d9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 0x7efd880ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:58130/0 (socket says 172.21.15.82:58130) 2024-04-25T10:02:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 -- 172.21.15.82:0/2302432193 learned_addr learned my addr 172.21.15.82:0/2302432193 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8659c640 1 --1- 172.21.15.82:0/2302432193 >> v1:172.21.15.196:6789/0 conn(0x7efd88144ca0 0x7efd8806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:40738/0 (socket says 172.21.15.82:40738) 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd85d9b640 1 -- 172.21.15.82:0/2302432193 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1072222765 0 0) 0x7efd880cee30 con 0x7efd88144ca0 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd85d9b640 1 -- 172.21.15.82:0/2302432193 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd70003530 con 0x7efd88144ca0 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 -- 172.21.15.82:0/2302432193 >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 msgr2=0x7efd880ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 --2- 172.21.15.82:0/2302432193 >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 0x7efd880ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 -- 172.21.15.82:0/2302432193 >> v1:172.21.15.196:6789/0 conn(0x7efd88144ca0 legacy=0x7efd8806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.756+0000 7efd8759e640 1 -- 172.21.15.82:0/2302432193 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880ccc70 con 0x7efd881451f0 2024-04-25T10:02:14.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8759e640 1 --2- 172.21.15.82:0/2302432193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd80009e50 tx=0x7efd8001f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51a81fd4a2e44b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd85d9b640 1 -- 172.21.15.82:0/2302432193 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800b4070 con 0x7efd881451f0 2024-04-25T10:02:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd85d9b640 1 -- 172.21.15.82:0/2302432193 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd800af400 con 0x7efd881451f0 2024-04-25T10:02:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd85d9b640 1 -- 172.21.15.82:0/2302432193 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800af7b0 con 0x7efd881451f0 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 -- 172.21.15.82:0/2302432193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 msgr2=0x7efd88142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/2302432193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd80009e50 tx=0x7efd8001f210 comp rx=0 tx=0).stop 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 -- 172.21.15.82:0/2302432193 shutdown_connections 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/2302432193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd881451f0 0x7efd88142680 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/2302432193 >> v2:172.21.15.196:3300/0 conn(0x7efd880ce510 0x7efd880ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 -- 172.21.15.82:0/2302432193 >> 172.21.15.82:0/2302432193 conn(0x7efd880b11a0 msgr2=0x7efd880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 -- 172.21.15.82:0/2302432193 shutdown_connections 2024-04-25T10:02:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.760+0000 7efd8d1ab640 1 -- 172.21.15.82:0/2302432193 wait complete. 2024-04-25T10:02:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 Processor -- start 2024-04-25T10:02:14.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- start start 2024-04-25T10:02:14.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd880ce510 0x7efd881e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd88144ca0 0x7efd88143390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806ee30 con 0x7efd881451f0 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8806df20 con 0x7efd880ce510 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd880cc900 con 0x7efd88144ca0 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59804/0 (socket says 172.21.15.82:59804) 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 -- 172.21.15.82:0/956719898 learned_addr learned my addr 172.21.15.82:0/956719898 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd86d9d640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd880ce510 0x7efd881e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8659c640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd88144ca0 0x7efd88143390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 -- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd88144ca0 msgr2=0x7efd88143390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd88144ca0 0x7efd88143390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 -- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd880ce510 msgr2=0x7efd881e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd880ce510 0x7efd881e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 -- 172.21.15.82:0/956719898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd880b27d0 con 0x7efd881451f0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8759e640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd80009040 tx=0x7efd8001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd800b4050 con 0x7efd881451f0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd800c0070 con 0x7efd881451f0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd881e67b0 con 0x7efd881451f0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd80009510 con 0x7efd881451f0 2024-04-25T10:02:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd881e6d70 con 0x7efd881451f0 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.764+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8804efa0 con 0x7efd881451f0 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.768+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd800a8d10 con 0x7efd881451f0 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.768+0000 7efd6f7fe640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 0x7efd74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.768+0000 7efd86d9d640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 0x7efd74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.768+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd80123630 con 0x7efd881451f0 2024-04-25T10:02:14.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.768+0000 7efd86d9d640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 0x7efd74069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd78009af0 tx=0x7efd78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.772+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd80018d70 con 0x7efd881451f0 2024-04-25T10:02:14.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.952+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efd880b1cc0 con 0x7efd881451f0 2024-04-25T10:02:14.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd6f7fe640 1 -- 172.21.15.82:0/956719898 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+88 (secure 0 0 0) 0x7efd800f8e20 con 0x7efd881451f0 2024-04-25T10:02:14.962 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 msgr2=0x7efd74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 0x7efd74069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efd78009af0 tx=0x7efd78008040 comp rx=0 tx=0).stop 2024-04-25T10:02:14.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 msgr2=0x7efd880a9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efd80009040 tx=0x7efd8001bfb0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 shutdown_connections 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd74066f90 0x7efd74069450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd88144ca0 0x7efd88143390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd881451f0 0x7efd880a9550 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 --2- 172.21.15.82:0/956719898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd880ce510 0x7efd881e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 >> 172.21.15.82:0/956719898 conn(0x7efd880b11a0 msgr2=0x7efd880ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 shutdown_connections 2024-04-25T10:02:14.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:14.956+0000 7efd8d1ab640 1 -- 172.21.15.82:0/956719898 wait complete. 2024-04-25T10:02:14.979 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:14.979 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:14.979 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:02:14.979 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T10:02:14.980 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:14.980 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:14.980 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:14.980 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:14.980 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:14.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:14.980 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:02:14.986 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42535sec preferred_lft 42535sec 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:14.987 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:14.988 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:14.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:14.988 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:15.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.067 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:15.117 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:02:15.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.119 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:15.190 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:15.191 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:15.191 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:15.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:15.191 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:15.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.265 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:02:15.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.311 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-25T10:02:15.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.359 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:15.359 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:15.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T10:02:15.359 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:15.428 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T10:02:15.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T10:02:15.429 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:15.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.534 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:15.534 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:15.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:02:15.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:15.535 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:15.553 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T10:02:15.554 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:15.554 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:15.601 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:15.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:15.602 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:15.649 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:15.703 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:15.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:15.751 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:15.751 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:15.751 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:15.751 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:15.752 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.754 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:15.801 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:15.801 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T10:02:15.847 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:15.859 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:15.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:15.907 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:15.908 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.910 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:15.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:2024-04-25T10:02:15.912+0000 7f101a560580 -1 init, newargv = 0x55d1e36289c0 newargc=15 2024-04-25T10:02:15.920 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[34452]: starting ceph client 2024-04-25T10:02:15.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[34452]: starting fuse 2024-04-25T10:02:17.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:17.044 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:17.092 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:17.093 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:17.094 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:17.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:17.095 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:17.138 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:02:17.139 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T10:02:17.139 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:17.140 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.admin")) 2024-04-25T10:02:17.141 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:02:17.256 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.admin.34452.asok 2024-04-25T10:02:17.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:17.269 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34452.asok 2024-04-25T10:02:17.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:17.269 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34452.asok status 2024-04-25T10:02:17.420 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "admin", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "34452", 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4410, 2024-04-25T10:02:17.421 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 4410 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 237272778 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:17.422 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 237272778 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.4410 192.168.0.1:0/237272778", 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/237272778", 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 6, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 25, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:02:17.423 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:02:17.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:17.435 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:02:17.435 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:17.440 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:02:17.440 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:17.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:17.440 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:17.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:17.495 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi082.stdout:# file: . 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi082.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T10:02:17.742 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:17.743 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:02:17.743 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:17.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:17.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.912+0000 7fa0502be640 1 Processor -- start 2024-04-25T10:02:17.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 -- start start 2024-04-25T10:02:17.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0480cf840 0x7fa0480cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa04815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa04806b710 con 0x7fa04806fc50 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa048161560 con 0x7fa0480cf840 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa0502be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481617b0 con 0x7fa04815d180 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04e05a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa04806fc50 0x7fa04806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57830/0 (socket says 172.21.15.82:57830) 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04e05a640 1 -- 172.21.15.82:0/1394598142 learned_addr learned my addr 172.21.15.82:0/1394598142 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 --2- 172.21.15.82:0/1394598142 >> v2:172.21.15.196:3300/0 conn(0x7fa0480cf840 0x7fa0480cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04e85b640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa04815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270667248 0 0) 0x7fa04806b710 con 0x7fa04806fc50 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa034003530 con 0x7fa04806fc50 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2338240195 0 0) 0x7fa034003530 con 0x7fa04806fc50 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 msgr2=0x7fa04815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa04815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 >> v2:172.21.15.196:3300/0 conn(0x7fa0480cf840 msgr2=0x7fa0480cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 --2- 172.21.15.82:0/1394598142 >> v2:172.21.15.196:3300/0 conn(0x7fa0480cf840 0x7fa0480cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048161a90 con 0x7fa04806fc50 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa044001680 con 0x7fa04806fc50 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 >> v1:172.21.15.82:6789/0 conn(0x7fa04806fc50 legacy=0x7fa04806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034003a30 0x7fa034003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa034004810 0x7fa034004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa0340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa048161560 con 0x7fa034004810 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0481617b0 con 0x7fa034003a30 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa03400b9d0 con 0x7fa04815d180 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa034004810 0x7fa034004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 -- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 msgr2=0x7fa0340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04e85b640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa0340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa04815d180 0x7fa0340051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 -- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034003a30 msgr2=0x7fa034003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa034003a30 0x7fa034003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.916+0000 7fa04d859640 1 -- 172.21.15.82:0/1394598142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa034003530 con 0x7fa034004810 2024-04-25T10:02:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa034004810 0x7fa034004c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b10 tx=0x7fa03801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662c9723da3c8bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380b2070 con 0x7fa034004810 2024-04-25T10:02:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0380a55c0 con 0x7fa034004810 2024-04-25T10:02:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d058640 1 -- 172.21.15.82:0/1394598142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380096b0 con 0x7fa034004810 2024-04-25T10:02:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- 172.21.15.82:0/1394598142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa034004810 msgr2=0x7fa034004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 --2- 172.21.15.82:0/1394598142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa034004810 0x7fa034004c90 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b10 tx=0x7fa03801cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04e85b640 1 -- 172.21.15.82:0/1394598142 reap_dead start 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- 172.21.15.82:0/1394598142 shutdown_connections 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- 172.21.15.82:0/1394598142 >> 172.21.15.82:0/1394598142 conn(0x7fa048099a70 msgr2=0x7fa0480d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- 172.21.15.82:0/1394598142 shutdown_connections 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- 172.21.15.82:0/1394598142 wait complete. 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 Processor -- start 2024-04-25T10:02:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- start start 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa034004810 0x7fa0480b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa04815df00 0x7fa04815e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04806b3a0 con 0x7fa04815d600 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d0210 con 0x7fa04815df00 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa0502be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0480d0e30 con 0x7fa034004810 2024-04-25T10:02:17.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59826/0 (socket says 172.21.15.82:59826) 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 -- 172.21.15.82:0/1803951106 learned_addr learned my addr 172.21.15.82:0/1803951106 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04e05a640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa034004810 0x7fa0480b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 -- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa034004810 msgr2=0x7fa0480b5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04e85b640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa04815df00 0x7fa04815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa034004810 0x7fa0480b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 -- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa04815df00 msgr2=0x7fa04815e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa04815df00 0x7fa04815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.920+0000 7fa04d859640 1 -- 172.21.15.82:0/1803951106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480c5c40 con 0x7fa04815d600 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa04d859640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b10 tx=0x7fa034005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380bc030 con 0x7fa04815d600 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa03800f000 con 0x7fa04815d600 2024-04-25T10:02:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0480c5ef0 con 0x7fa04815d600 2024-04-25T10:02:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0380b2040 con 0x7fa04815d600 2024-04-25T10:02:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0480c64b0 con 0x7fa04815d600 2024-04-25T10:02:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04804efa0 con 0x7fa04815d600 2024-04-25T10:02:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0380bf070 con 0x7fa04815d600 2024-04-25T10:02:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 0x7fa02c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa03811f310 con 0x7fa04815d600 2024-04-25T10:02:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa04e05a640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 0x7fa02c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.924+0000 7fa04e05a640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 0x7fa02c069450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa044001930 tx=0x7fa04400c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:17.928+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa038016800 con 0x7fa04815d600 2024-04-25T10:02:18.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.080+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0480b6480 con 0x7fa04815d600 2024-04-25T10:02:18.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.084+0000 7fa03effd640 1 -- 172.21.15.82:0/1803951106 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fa0380eff40 con 0x7fa04815d600 2024-04-25T10:02:18.092 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:18.092 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 msgr2=0x7fa02c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 0x7fa02c069450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa044001930 tx=0x7fa04400c040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 msgr2=0x7fa04815feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa038009b10 tx=0x7fa034005840 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 shutdown_connections 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa02c066f90 0x7fa02c069450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa04815df00 0x7fa04815e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa04815d600 0x7fa04815feb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 --2- 172.21.15.82:0/1803951106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa034004810 0x7fa0480b5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 >> 172.21.15.82:0/1803951106 conn(0x7fa048099a70 msgr2=0x7fa0480c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 shutdown_connections 2024-04-25T10:02:18.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.088+0000 7fa0502be640 1 -- 172.21.15.82:0/1803951106 wait complete. 2024-04-25T10:02:18.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:18.110 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:18.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 Processor -- start 2024-04-25T10:02:18.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 -- start start 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 0x7fe5b40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5b40cf590 0x7fe5b40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5b40cbd20 con 0x7fe5b415f130 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b415ba90 con 0x7fe5b40cf590 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5bbc98640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5b40cc000 con 0x7fe5b415b6b0 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5b40cf590 0x7fe5b40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5b40cf590 0x7fe5b40cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59836/0 (socket says 172.21.15.82:59836) 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 -- 172.21.15.82:0/1737492163 learned_addr learned my addr 172.21.15.82:0/1737492163 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9a34640 1 --2- 172.21.15.82:0/1737492163 >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 0x7fe5b40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 -- 172.21.15.82:0/1737492163 >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 msgr2=0x7fe5b40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 --2- 172.21.15.82:0/1737492163 >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 0x7fe5b40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 -- 172.21.15.82:0/1737492163 >> v1:172.21.15.82:6789/0 conn(0x7fe5b415f130 legacy=0x7fe5b415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9233640 1 -- 172.21.15.82:0/1737492163 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b40cc2e0 con 0x7fe5b40cf590 2024-04-25T10:02:18.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.224+0000 7fe5b9a34640 1 --2- 172.21.15.82:0/1737492163 >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 0x7fe5b40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- 172.21.15.82:0/1737492163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b40cf590 0x7fe5b40cfa20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40d0180 tx=0x7fe5a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27584bbf8e498f83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b8a32640 1 -- 172.21.15.82:0/1737492163 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40ab610 con 0x7fe5b40cf590 2024-04-25T10:02:18.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b8a32640 1 -- 172.21.15.82:0/1737492163 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40b0070 con 0x7fe5b40cf590 2024-04-25T10:02:18.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b8a32640 1 -- 172.21.15.82:0/1737492163 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40ab9d0 con 0x7fe5b40cf590 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/1737492163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b40cf590 msgr2=0x7fe5b40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/1737492163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b40cf590 0x7fe5b40cfa20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe5b40d0180 tx=0x7fe5a401c520 comp rx=0 tx=0).stop 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/1737492163 shutdown_connections 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/1737492163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b40cf590 0x7fe5b40cfa20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/1737492163 >> v2:172.21.15.196:3300/0 conn(0x7fe5b415b6b0 0x7fe5b40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/1737492163 >> 172.21.15.82:0/1737492163 conn(0x7fe5b406ae60 msgr2=0x7fe5b406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/1737492163 shutdown_connections 2024-04-25T10:02:18.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/1737492163 wait complete. 2024-04-25T10:02:18.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 Processor -- start 2024-04-25T10:02:18.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- start start 2024-04-25T10:02:18.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5b40cf590 0x7fe5b4098560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b415f130 0x7fe5b40b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40cadb0 con 0x7fe5b415b6b0 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40ca900 con 0x7fe5b415f130 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b40caf30 con 0x7fe5b40cf590 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59838/0 (socket says 172.21.15.82:59838) 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 -- 172.21.15.82:0/790583712 learned_addr learned my addr 172.21.15.82:0/790583712 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9a34640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5b40cf590 0x7fe5b4098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5ba235640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b415f130 0x7fe5b40b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 -- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5b40cf590 msgr2=0x7fe5b4098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5b40cf590 0x7fe5b4098560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 -- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b415f130 msgr2=0x7fe5b40b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b415f130 0x7fe5b40b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 -- 172.21.15.82:0/790583712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b41514b0 con 0x7fe5b415b6b0 2024-04-25T10:02:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5b9233640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4019130 tx=0x7fe5a401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40b4800 con 0x7fe5b415b6b0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5a40b49c0 con 0x7fe5b415b6b0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40b4db0 con 0x7fe5b415b6b0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b4151750 con 0x7fe5b415b6b0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.228+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b4151d10 con 0x7fe5b415b6b0 2024-04-25T10:02:18.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5b404efa0 con 0x7fe5b415b6b0 2024-04-25T10:02:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5a40b3380 con 0x7fe5b415b6b0 2024-04-25T10:02:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5aa7fc640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 0x7fe598069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe5a411f750 con 0x7fe5b415b6b0 2024-04-25T10:02:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5b9a34640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 0x7fe598069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.232+0000 7fe5b9a34640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 0x7fe598069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5b406b3c0 tx=0x7fe5b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.236+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5a40c4060 con 0x7fe5b415b6b0 2024-04-25T10:02:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.396+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5b4165b50 con 0x7fe5b415b6b0 2024-04-25T10:02:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.396+0000 7fe5aa7fc640 1 -- 172.21.15.82:0/790583712 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5a40f4e20 con 0x7fe5b415b6b0 2024-04-25T10:02:18.404 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 msgr2=0x7fe598069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 0x7fe598069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe5b406b3c0 tx=0x7fe5b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 msgr2=0x7fe5b40b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4019130 tx=0x7fe5a401ad60 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 shutdown_connections 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe598066f60 0x7fe598069420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5b415f130 0x7fe5b40b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5b415b6b0 0x7fe5b40b1380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 --2- 172.21.15.82:0/790583712 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5b40cf590 0x7fe5b4098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 >> 172.21.15.82:0/790583712 conn(0x7fe5b406ae60 msgr2=0x7fe5b40b3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 shutdown_connections 2024-04-25T10:02:18.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.400+0000 7fe5bbc98640 1 -- 172.21.15.82:0/790583712 wait complete. 2024-04-25T10:02:18.423 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 Processor -- start 2024-04-25T10:02:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 -- start start 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faa400cfca0 0x7faa40166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400cbd20 con 0x7faa4015b420 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa400cc000 con 0x7faa4015b800 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa48687640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa400cc930 con 0x7faa400cfca0 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46423640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faa4015b420 0x7faa400ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57852/0 (socket says 172.21.15.82:57852) 2024-04-25T10:02:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46423640 1 -- 172.21.15.82:0/526740469 learned_addr learned my addr 172.21.15.82:0/526740469 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 --2- 172.21.15.82:0/526740469 >> v2:172.21.15.196:3300/0 conn(0x7faa400cfca0 0x7faa40166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174335886 0 0) 0x7faa400cbd20 con 0x7faa4015b420 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa2c003530 con 0x7faa4015b420 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2794524506 0 0) 0x7faa2c003530 con 0x7faa4015b420 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 >> v2:172.21.15.196:3300/0 conn(0x7faa400cfca0 msgr2=0x7faa40166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 --2- 172.21.15.82:0/526740469 >> v2:172.21.15.196:3300/0 conn(0x7faa400cfca0 0x7faa40166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 >> v1:172.21.15.196:6789/0 conn(0x7faa4015b800 legacy=0x7faa400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400cc2e0 con 0x7faa4015b420 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7faa3c001680 con 0x7faa4015b420 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 >> v1:172.21.15.82:6789/0 conn(0x7faa4015b420 legacy=0x7faa400ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa2c003a30 0x7faa2c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa2c004810 0x7faa2c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa2c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cc930 con 0x7faa2c0051d0 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c00c100 con 0x7faa2c003a30 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c00c300 con 0x7faa2c004810 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa2c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 -- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa2c004810 msgr2=0x7faa2c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa2c004810 0x7faa2c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 -- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa2c003a30 msgr2=0x7faa2c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa2c003a30 0x7faa2c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.544+0000 7faa46c24640 1 -- 172.21.15.82:0/526740469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c003530 con 0x7faa2c0051d0 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa46c24640 1 -- 172.21.15.82:0/526740469 reap_dead start 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa46c24640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa2c00b9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f90 tx=0x7faa3801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a606f86f8f2f07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38009c10 con 0x7faa2c0051d0 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa380a5c90 con 0x7faa2c0051d0 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa45421640 1 -- 172.21.15.82:0/526740469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38003be0 con 0x7faa2c0051d0 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 msgr2=0x7faa2c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa2c00b9d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f90 tx=0x7faa3801cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- 172.21.15.82:0/526740469 shutdown_connections 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 --2- 172.21.15.82:0/526740469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa2c00b9d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- 172.21.15.82:0/526740469 >> 172.21.15.82:0/526740469 conn(0x7faa4006ae60 msgr2=0x7faa4006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- 172.21.15.82:0/526740469 shutdown_connections 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- 172.21.15.82:0/526740469 wait complete. 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 Processor -- start 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- start start 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa40158360 0x7faa401f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4014b1e0 0x7faa4014b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cbb70 con 0x7faa2c0051d0 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cb9f0 con 0x7faa40158360 2024-04-25T10:02:18.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa48687640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa400cadb0 con 0x7faa4014b1e0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa46423640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa46423640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59862/0 (socket says 172.21.15.82:59862) 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.548+0000 7faa46423640 1 -- 172.21.15.82:0/2225442866 learned_addr learned my addr 172.21.15.82:0/2225442866 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46c24640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4014b1e0 0x7faa4014b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa45c22640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa40158360 0x7faa401f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 -- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4014b1e0 msgr2=0x7faa4014b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4014b1e0 0x7faa4014b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 -- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa40158360 msgr2=0x7faa401f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa40158360 0x7faa401f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 -- 172.21.15.82:0/2225442866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa400cd2a0 con 0x7faa2c0051d0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa46423640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00c6c0 tx=0x7faa3c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c018070 con 0x7faa2c0051d0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c0072d0 con 0x7faa2c0051d0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa400cd580 con 0x7faa2c0051d0 2024-04-25T10:02:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0047f0 con 0x7faa2c0051d0 2024-04-25T10:02:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa400cdb40 con 0x7faa2c0051d0 2024-04-25T10:02:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4004efa0 con 0x7faa2c0051d0 2024-04-25T10:02:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa3c01b070 con 0x7faa2c0051d0 2024-04-25T10:02:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 0x7faa24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faa3c084790 con 0x7faa2c0051d0 2024-04-25T10:02:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa45c22640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 0x7faa24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.552+0000 7faa45c22640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 0x7faa24069450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa401f6780 tx=0x7faa30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.556+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa3c055460 con 0x7faa2c0051d0 2024-04-25T10:02:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa400ca010 con 0x7faa2c0051d0 2024-04-25T10:02:18.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa36ffd640 1 -- 172.21.15.82:0/2225442866 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa3c054d60 con 0x7faa2c0051d0 2024-04-25T10:02:18.722 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 msgr2=0x7faa24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 0x7faa24069450 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa401f6780 tx=0x7faa30009040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 msgr2=0x7faa400ce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00c6c0 tx=0x7faa3c00cb80 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 shutdown_connections 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa24066f90 0x7faa24069450 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa4014b1e0 0x7faa4014b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa40158360 0x7faa401f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 --2- 172.21.15.82:0/2225442866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa2c0051d0 0x7faa400ce340 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 >> 172.21.15.82:0/2225442866 conn(0x7faa4006ae60 msgr2=0x7faa400b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 shutdown_connections 2024-04-25T10:02:18.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.716+0000 7faa48687640 1 -- 172.21.15.82:0/2225442866 wait complete. 2024-04-25T10:02:18.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:18.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.856+0000 7fc63f60c640 1 Processor -- start 2024-04-25T10:02:18.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.856+0000 7fc63f60c640 1 -- start start 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63f60c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6380ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63f60c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc63815ff50 con 0x7fc638073010 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63f60c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6380cbd20 con 0x7fc6380cf590 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63f60c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63815ba90 con 0x7fc63815b6b0 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63dba9640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc638073010 0x7fc63815f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57868/0 (socket says 172.21.15.82:57868) 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63dba9640 1 -- 172.21.15.82:0/2577770895 learned_addr learned my addr 172.21.15.82:0/2577770895 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:18.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6380ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3125746415 0 0) 0x7fc63815ff50 con 0x7fc638073010 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc624003530 con 0x7fc638073010 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 473990209 0 0) 0x7fc624003530 con 0x7fc638073010 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 msgr2=0x7fc6380ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6380ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 >> v1:172.21.15.196:6789/0 conn(0x7fc6380cf590 legacy=0x7fc63815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380cc000 con 0x7fc638073010 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc630001a00 con 0x7fc638073010 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 >> v1:172.21.15.82:6789/0 conn(0x7fc638073010 legacy=0x7fc63815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc624003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc624004810 0x7fc624004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63815ba90 con 0x7fc624003a60 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62400bad0 con 0x7fc624004810 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62400bcd0 con 0x7fc63815b6b0 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc624003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63cba7640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc624004810 0x7fc624004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63dba9640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 msgr2=0x7fc6240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc624004810 msgr2=0x7fc624004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc624004810 0x7fc624004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2577770895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc624003530 con 0x7fc624003a60 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63dba9640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63815b6b0 0x7fc6240051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.860+0000 7fc63cba7640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc624004810 0x7fc624004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc624003e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ac0 tx=0x7fc62c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b23e4ab169cd58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0b2070 con 0x7fc624003a60 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c0a5510 con 0x7fc624003a60 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc62bfff640 1 -- 172.21.15.82:0/2577770895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c009690 con 0x7fc624003a60 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- 172.21.15.82:0/2577770895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 msgr2=0x7fc624003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2577770895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc624003e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009ac0 tx=0x7fc62c01ca20 comp rx=0 tx=0).stop 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63dba9640 1 -- 172.21.15.82:0/2577770895 reap_dead start 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- 172.21.15.82:0/2577770895 shutdown_connections 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- 172.21.15.82:0/2577770895 >> 172.21.15.82:0/2577770895 conn(0x7fc63806ae60 msgr2=0x7fc63806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- 172.21.15.82:0/2577770895 shutdown_connections 2024-04-25T10:02:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- 172.21.15.82:0/2577770895 wait complete. 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 Processor -- start 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- start start 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380aee00 0x7fc6380c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6380c5380 0x7fc6380c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380cfd80 con 0x7fc624003a60 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380cba30 con 0x7fc6380aee00 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63f60c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380cff00 con 0x7fc6380c5380 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63d3a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63d3a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59872/0 (socket says 172.21.15.82:59872) 2024-04-25T10:02:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.864+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2994962734 learned_addr learned my addr 172.21.15.82:0/2994962734 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63cba7640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380aee00 0x7fc6380c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6380c5380 msgr2=0x7fc6380c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63dba9640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6380c5380 0x7fc6380c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6380c5380 0x7fc6380c81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380aee00 msgr2=0x7fc6380c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380aee00 0x7fc6380c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 -- 172.21.15.82:0/2994962734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380c8fb0 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63d3a8640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc62c01cf00 tx=0x7fc624005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0c0070 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c0a5d20 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6380c9290 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0b2040 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63814a500 con 0x7fc624003a60 2024-04-25T10:02:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63804efa0 con 0x7fc624003a60 2024-04-25T10:02:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc62c0c3070 con 0x7fc624003a60 2024-04-25T10:02:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 0x7fc61c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc62c11ed40 con 0x7fc624003a60 2024-04-25T10:02:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63cba7640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 0x7fc61c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.868+0000 7fc63cba7640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 0x7fc61c069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc634000c10 tx=0x7fc634009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:18.872+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc62c016760 con 0x7fc624003a60 2024-04-25T10:02:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.028+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6380ca870 con 0x7fc624003a60 2024-04-25T10:02:19.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc629ffb640 1 -- 172.21.15.82:0/2994962734 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc62c0ef330 con 0x7fc624003a60 2024-04-25T10:02:19.039 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 msgr2=0x7fc61c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 0x7fc61c069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc634000c10 tx=0x7fc634009040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 msgr2=0x7fc638149fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc62c01cf00 tx=0x7fc624005840 comp rx=0 tx=0).stop 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 shutdown_connections 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc61c066f90 0x7fc61c069450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6380c5380 0x7fc6380c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6380aee00 0x7fc6380c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 --2- 172.21.15.82:0/2994962734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc624003a60 0x7fc638149fc0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 >> 172.21.15.82:0/2994962734 conn(0x7fc63806ae60 msgr2=0x7fc6380ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 shutdown_connections 2024-04-25T10:02:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.032+0000 7fc63f60c640 1 -- 172.21.15.82:0/2994962734 wait complete. 2024-04-25T10:02:19.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:19.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 Processor -- start 2024-04-25T10:02:19.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 -- start start 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6f0c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c0cbd20 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c0cc000 con 0x7f6f0c072000 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f11fe1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c15ba90 con 0x7f6f0c0cf590 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6f0c15b6b0 0x7f6f0c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57880/0 (socket says 172.21.15.82:57880) 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 -- 172.21.15.82:0/52656741 learned_addr learned my addr 172.21.15.82:0/52656741 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f02ffd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6f0c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359386941 0 0) 0x7f6f0c0cbd20 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef8003530 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1966604099 0 0) 0x7f6ef8003530 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 msgr2=0x7f6f0c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6f0c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 >> v1:172.21.15.196:6789/0 conn(0x7f6f0c072000 legacy=0x7f6f0c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0cc2e0 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6efc001680 con 0x7f6f0c15b6b0 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 >> v1:172.21.15.82:6789/0 conn(0x7f6f0c15b6b0 legacy=0x7f6f0c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef8003a60 0x7f6ef8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6ef8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8004d50 0x7f6ef80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c15ba90 con 0x7f6ef8003a60 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef800ba80 con 0x7f6ef8004d50 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef800bc30 con 0x7f6f0c0cf590 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef8003a60 0x7f6ef8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0bfff640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8004d50 0x7f6ef80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f02ffd640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6ef8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 -- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 msgr2=0x7f6ef8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c0cf590 0x7f6ef8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 -- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8004d50 msgr2=0x7f6ef80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8004d50 0x7f6ef80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.176+0000 7f6f0b7fe640 1 -- 172.21.15.82:0/52656741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8003530 con 0x7f6ef8003a60 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef8003a60 0x7f6ef8003e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c2c0 tx=0x7f6efc01cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ec3a072205cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc007d60 con 0x7f6ef8003a60 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6efc0a9070 con 0x7f6ef8003a60 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f0affd640 1 -- 172.21.15.82:0/52656741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6efc0a5dc0 con 0x7f6ef8003a60 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- 172.21.15.82:0/52656741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef8003a60 msgr2=0x7f6ef8003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/52656741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ef8003a60 0x7f6ef8003e40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c2c0 tx=0x7f6efc01cd10 comp rx=0 tx=0).stop 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f0bfff640 1 -- 172.21.15.82:0/52656741 reap_dead start 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- 172.21.15.82:0/52656741 shutdown_connections 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- 172.21.15.82:0/52656741 >> 172.21.15.82:0/52656741 conn(0x7f6f0c06ae60 msgr2=0x7f6f0c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- 172.21.15.82:0/52656741 shutdown_connections 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- 172.21.15.82:0/52656741 wait complete. 2024-04-25T10:02:19.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 Processor -- start 2024-04-25T10:02:19.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- start start 2024-04-25T10:02:19.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 0x7f6f0c0c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c15bc10 0x7f6f0c15a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0cff00 con 0x7f6f0c0aee00 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0cfd80 con 0x7f6ef8003a60 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.180+0000 7f6f11fe1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0cadb0 con 0x7f6f0c15bc10 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59892/0 (socket says 172.21.15.82:59892) 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 -- 172.21.15.82:0/1090679786 learned_addr learned my addr 172.21.15.82:0/1090679786 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 0x7f6f0c0c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f02ffd640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c15bc10 0x7f6f0c15a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 -- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c15bc10 msgr2=0x7f6f0c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c15bc10 0x7f6f0c15a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 -- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 msgr2=0x7f6f0c0c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 0x7f6f0c0c58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 -- 172.21.15.82:0/1090679786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c15c110 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 0x7f6f0c0c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0bfff640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ae740 tx=0x7f6f0400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04003980 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f04018070 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c15c3f0 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04003d60 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c15c9b0 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c04efa0 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f040050d0 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 0x7f6eec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6f040850d0 con 0x7f6f0c0aee00 2024-04-25T10:02:19.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 0x7f6eec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.184+0000 7f6f0b7fe640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 0x7f6eec069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ab000 tx=0x7f6efc0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.188+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f04055e30 con 0x7f6f0c0aee00 2024-04-25T10:02:19.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.348+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6f0c156990 con 0x7f6f0c0aee00 2024-04-25T10:02:19.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.348+0000 7f6f08ff9640 1 -- 172.21.15.82:0/1090679786 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6f04055730 con 0x7f6f0c0aee00 2024-04-25T10:02:19.356 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 msgr2=0x7f6eec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 0x7f6eec069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6efc0ab000 tx=0x7f6efc0b1040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 msgr2=0x7f6f0c0af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0ae740 tx=0x7f6f0400eb90 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 shutdown_connections 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6eec066f90 0x7f6eec069450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f0c15bc10 0x7f6f0c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f0c0aee00 0x7f6f0c0af290 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 --2- 172.21.15.82:0/1090679786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ef8003a60 0x7f6f0c0c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 >> 172.21.15.82:0/1090679786 conn(0x7f6f0c06ae60 msgr2=0x7f6f0c1617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 shutdown_connections 2024-04-25T10:02:19.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.352+0000 7f6f11fe1640 1 -- 172.21.15.82:0/1090679786 wait complete. 2024-04-25T10:02:19.375 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:19.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 Processor -- start 2024-04-25T10:02:19.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 -- start start 2024-04-25T10:02:19.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c380b24d0 0x7f5c3809aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b28b0 0x7f5c380b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c38099cc0 con 0x7f5c380b19f0 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c38099a70 con 0x7f5c380b24d0 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c3e4c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c38099fa0 con 0x7f5c380b28b0 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c380b24d0 0x7f5c3809aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c380b24d0 0x7f5c3809aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59904/0 (socket says 172.21.15.82:59904) 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 -- 172.21.15.82:0/2888126855 learned_addr learned my addr 172.21.15.82:0/2888126855 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b28b0 0x7f5c380b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 -- 172.21.15.82:0/2888126855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b28b0 msgr2=0x7f5c380b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b28b0 0x7f5c380b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 -- 172.21.15.82:0/2888126855 >> v1:172.21.15.196:6789/0 conn(0x7f5c380b19f0 legacy=0x7f5c380cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.480+0000 7f5c37fff640 1 -- 172.21.15.82:0/2888126855 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3809a280 con 0x7f5c380b24d0 2024-04-25T10:02:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c37fff640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c3809aa80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009870 tx=0x7f5c2801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e79e4810f3f680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c377fe640 1 -- 172.21.15.82:0/2888126855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280b0070 con 0x7f5c380b24d0 2024-04-25T10:02:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c377fe640 1 -- 172.21.15.82:0/2888126855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c280ab600 con 0x7f5c380b24d0 2024-04-25T10:02:19.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c377fe640 1 -- 172.21.15.82:0/2888126855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c280ab9d0 con 0x7f5c380b24d0 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2888126855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 msgr2=0x7f5c3809aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c3809aa80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5c28009870 tx=0x7f5c2801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2888126855 shutdown_connections 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b28b0 0x7f5c380b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2888126855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c3809aa80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2888126855 >> 172.21.15.82:0/2888126855 conn(0x7f5c380b3020 msgr2=0x7f5c380b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2888126855 shutdown_connections 2024-04-25T10:02:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2888126855 wait complete. 2024-04-25T10:02:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 Processor -- start 2024-04-25T10:02:19.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- start start 2024-04-25T10:02:19.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b19f0 0x7f5c380c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c380b28b0 0x7f5c3815f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815e020 con 0x7f5c380b24d0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3815dea0 con 0x7f5c380b28b0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3e4c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3816a770 con 0x7f5c380b19f0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59918/0 (socket says 172.21.15.82:59918) 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 -- 172.21.15.82:0/2697990639 learned_addr learned my addr 172.21.15.82:0/2697990639 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 -- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b19f0 msgr2=0x7f5c380c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c37fff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b19f0 0x7f5c380c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b19f0 0x7f5c380c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c3ca5f640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c380b28b0 0x7f5c3815f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 -- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c380b28b0 msgr2=0x7f5c3815f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c380b28b0 0x7f5c3815f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.484+0000 7f5c2ffff640 1 -- 172.21.15.82:0/2697990639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3815e340 con 0x7f5c380b24d0 2024-04-25T10:02:19.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c2ffff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000d5f0 tx=0x7f5c2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200125f0 con 0x7f5c380b24d0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c20019070 con 0x7f5c380b24d0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3815e620 con 0x7f5c380b24d0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200129b0 con 0x7f5c380b24d0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3815ebe0 con 0x7f5c380b24d0 2024-04-25T10:02:19.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3804efa0 con 0x7f5c380b24d0 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c20012b50 con 0x7f5c380b24d0 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 0x7f5c18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5c20085700 con 0x7f5c380b24d0 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c37fff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 0x7f5c18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.488+0000 7f5c37fff640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 0x7f5c18069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ad000 tx=0x7f5c28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.492+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c20056460 con 0x7f5c380b24d0 2024-04-25T10:02:19.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.648+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5c38154690 con 0x7f5c380b24d0 2024-04-25T10:02:19.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.648+0000 7f5c357fa640 1 -- 172.21.15.82:0/2697990639 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5c20055d60 con 0x7f5c380b24d0 2024-04-25T10:02:19.657 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 msgr2=0x7f5c18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 0x7f5c18069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5c280ad000 tx=0x7f5c28008040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 msgr2=0x7f5c381533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000d5f0 tx=0x7f5c2000dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 shutdown_connections 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c18066f60 0x7f5c18069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c380b28b0 0x7f5c3815f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c380b24d0 0x7f5c381533f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 --2- 172.21.15.82:0/2697990639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c380b19f0 0x7f5c380c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 >> 172.21.15.82:0/2697990639 conn(0x7f5c380b3020 msgr2=0x7f5c380b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 shutdown_connections 2024-04-25T10:02:19.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.652+0000 7f5c3e4c2640 1 -- 172.21.15.82:0/2697990639 wait complete. 2024-04-25T10:02:19.675 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:02:19.675 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:19.681 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.832+0000 7f5b7aa2d640 1 Processor -- start 2024-04-25T10:02:19.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.832+0000 7f5b7aa2d640 1 -- start start 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b7aa2d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b74144ca0 0x7f5b7406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b7aa2d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b7406ec60 0x7f5b74142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b7aa2d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b740ccc70 con 0x7f5b740ce510 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b7aa2d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b740cc2b0 con 0x7f5b7406ec60 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b7aa2d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7406e290 con 0x7f5b74144ca0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b7406ec60 0x7f5b74142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b7406ec60 0x7f5b74142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:58244/0 (socket says 172.21.15.82:58244) 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 -- 172.21.15.82:0/2609016965 learned_addr learned my addr 172.21.15.82:0/2609016965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:19.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b737fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b74144ca0 0x7f5b7406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 -- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b74144ca0 msgr2=0x7f5b7406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 --2- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b74144ca0 0x7f5b7406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 -- 172.21.15.82:0/2609016965 >> v1:172.21.15.196:6789/0 conn(0x7f5b740ce510 legacy=0x7f5b740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.836+0000 7f5b78fca640 1 -- 172.21.15.82:0/2609016965 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7406dc40 con 0x7f5b7406ec60 2024-04-25T10:02:19.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b78fca640 1 --2- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b7406ec60 0x7f5b74142680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009e50 tx=0x7f5b6c01f210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e611103f3c55e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b72ffd640 1 -- 172.21.15.82:0/2609016965 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0b4070 con 0x7f5b7406ec60 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b72ffd640 1 -- 172.21.15.82:0/2609016965 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0af400 con 0x7f5b7406ec60 2024-04-25T10:02:19.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b72ffd640 1 -- 172.21.15.82:0/2609016965 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0af7d0 con 0x7f5b7406ec60 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b7406ec60 msgr2=0x7f5b74142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b7406ec60 0x7f5b74142680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c009e50 tx=0x7f5b6c01f210 comp rx=0 tx=0).stop 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/2609016965 shutdown_connections 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b7406ec60 0x7f5b74142680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/2609016965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b74144ca0 0x7f5b7406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/2609016965 >> 172.21.15.82:0/2609016965 conn(0x7f5b740b11a0 msgr2=0x7f5b740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/2609016965 shutdown_connections 2024-04-25T10:02:19.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/2609016965 wait complete. 2024-04-25T10:02:19.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 Processor -- start 2024-04-25T10:02:19.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- start start 2024-04-25T10:02:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 0x7f5b7413f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b74144ca0 0x7f5b7413fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b741451f0 con 0x7f5b7406ec60 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b740cc900 con 0x7f5b74144ca0 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b7aa2d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b74145370 con 0x7f5b740ce510 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59930/0 (socket says 172.21.15.82:59930) 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 -- 172.21.15.82:0/347667891 learned_addr learned my addr 172.21.15.82:0/347667891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b78fca640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b74144ca0 0x7f5b7413fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b737fe640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 0x7f5b7413f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 -- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 msgr2=0x7f5b7413f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 0x7f5b7413f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 -- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b74144ca0 msgr2=0x7f5b7413fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b74144ca0 0x7f5b7413fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 -- 172.21.15.82:0/347667891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b740b3340 con 0x7f5b7406ec60 2024-04-25T10:02:19.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b737fe640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 0x7f5b7413f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.840+0000 7f5b73fff640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c420 tx=0x7f5b6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640052d0 con 0x7f5b7406ec60 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b64005470 con 0x7f5b7406ec60 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b740b3620 con 0x7f5b7406ec60 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64007260 con 0x7f5b7406ec60 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b740b3be0 con 0x7f5b7406ec60 2024-04-25T10:02:19.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7404efa0 con 0x7f5b7406ec60 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b64007400 con 0x7f5b7406ec60 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 0x7f5b60069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5b64084fa0 con 0x7f5b7406ec60 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b737fe640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 0x7f5b60069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.844+0000 7f5b737fe640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 0x7f5b60069450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b740b1b70 tx=0x7f5b68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:19.848+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b64055d50 con 0x7f5b7406ec60 2024-04-25T10:02:20.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b740b1cc0 con 0x7f5b7406ec60 2024-04-25T10:02:20.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b70ff9640 1 -- 172.21.15.82:0/347667891 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f5b64055650 con 0x7f5b7406ec60 2024-04-25T10:02:20.016 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:20.016 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 msgr2=0x7f5b60069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 0x7f5b60069450 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b740b1b70 tx=0x7f5b68008040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 msgr2=0x7f5b741424c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400c420 tx=0x7f5b6400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 shutdown_connections 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.008+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b60066f90 0x7f5b60069450 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b740ce510 0x7f5b7413f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b74144ca0 0x7f5b7413fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 --2- 172.21.15.82:0/347667891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b7406ec60 0x7f5b741424c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 >> 172.21.15.82:0/347667891 conn(0x7f5b740b11a0 msgr2=0x7f5b740c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 shutdown_connections 2024-04-25T10:02:20.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.012+0000 7f5b7aa2d640 1 -- 172.21.15.82:0/347667891 wait complete. 2024-04-25T10:02:20.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:20.035 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:20.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.144+0000 7fd7c60de640 1 Processor -- start 2024-04-25T10:02:20.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.144+0000 7fd7c60de640 1 -- start start 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7c60de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7c00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7c60de640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c015ff50 con 0x7fd7c0073010 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7c60de640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00cbd20 con 0x7fd7c00cf590 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7c60de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c015ba90 con 0x7fd7c015b6b0 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7bffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd7c0073010 0x7fd7c015f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57882/0 (socket says 172.21.15.82:57882) 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7bffff640 1 -- 172.21.15.82:0/467953364 learned_addr learned my addr 172.21.15.82:0/467953364 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7bf7fe640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7c00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692774223 0 0) 0x7fd7c015ff50 con 0x7fd7c0073010 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7ac003530 con 0x7fd7c0073010 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2032598445 0 0) 0x7fd7ac003530 con 0x7fd7c0073010 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 msgr2=0x7fd7c00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7c00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 >> v1:172.21.15.196:6789/0 conn(0x7fd7c00cf590 legacy=0x7fd7c015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00cc000 con 0x7fd7c0073010 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fd7b8001a00 con 0x7fd7c0073010 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 >> v1:172.21.15.82:6789/0 conn(0x7fd7c0073010 legacy=0x7fd7c015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7ac003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac003fe0 0x7fd7ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ac004d50 0x7fd7ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c015ba90 con 0x7fd7ac003fe0 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac00bb30 con 0x7fd7ac004d50 2024-04-25T10:02:20.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ac00bd30 con 0x7fd7c015b6b0 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac003fe0 0x7fd7ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7bf7fe640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7ac003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7bffff640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ac004d50 0x7fd7ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 msgr2=0x7fd7ac003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7c015b6b0 0x7fd7ac003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ac004d50 msgr2=0x7fd7ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7ac004d50 0x7fd7ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.148+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/467953364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac003530 con 0x7fd7ac003fe0 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac003fe0 0x7fd7ac004810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8001780 tx=0x7fd7a801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ff2ca27d9c6ebf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80013a0 con 0x7fd7ac003fe0 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a80052f0 con 0x7fd7ac003fe0 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7beffd640 1 -- 172.21.15.82:0/467953364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8003840 con 0x7fd7ac003fe0 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/467953364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac003fe0 msgr2=0x7fd7ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 --2- 172.21.15.82:0/467953364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7ac003fe0 0x7fd7ac004810 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8001780 tx=0x7fd7a801c360 comp rx=0 tx=0).stop 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bffff640 1 -- 172.21.15.82:0/467953364 reap_dead start 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/467953364 shutdown_connections 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/467953364 >> 172.21.15.82:0/467953364 conn(0x7fd7c006ae60 msgr2=0x7fd7c006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/467953364 shutdown_connections 2024-04-25T10:02:20.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/467953364 wait complete. 2024-04-25T10:02:20.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 Processor -- start 2024-04-25T10:02:20.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- start start 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7ac003fe0 0x7fd7c01547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c00ca840 0x7fd7c00cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cc8a0 con 0x7fd7c00ad290 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cc5c0 con 0x7fd7c00ca840 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00cba30 con 0x7fd7ac003fe0 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59950/0 (socket says 172.21.15.82:59950) 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/1726743763 learned_addr learned my addr 172.21.15.82:0/1726743763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bf7fe640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7ac003fe0 0x7fd7c01547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bffff640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c00ca840 0x7fd7c00cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7ac003fe0 msgr2=0x7fd7c01547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7ac003fe0 0x7fd7c01547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c00ca840 msgr2=0x7fd7c00cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c00ca840 0x7fd7c00cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 -- 172.21.15.82:0/1726743763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00ad790 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7b6ffd640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8005040 tx=0x7fd7ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80067e0 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a80bb070 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c015e090 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8006ba0 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.152+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c015e650 con 0x7fd7c00ad290 2024-04-25T10:02:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c004efa0 con 0x7fd7c00ad290 2024-04-25T10:02:20.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7a80b4040 con 0x7fd7c00ad290 2024-04-25T10:02:20.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7bcff9640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 0x7fd7a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd7a811f860 con 0x7fd7c00ad290 2024-04-25T10:02:20.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7bf7fe640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 0x7fd7a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.156+0000 7fd7bf7fe640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 0x7fd7a0069450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009a30 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.160+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7a80a4e20 con 0x7fd7c00ad290 2024-04-25T10:02:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.308+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7c00ac8d0 con 0x7fd7c00ad290 2024-04-25T10:02:20.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.308+0000 7fd7bcff9640 1 -- 172.21.15.82:0/1726743763 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7a8016070 con 0x7fd7c00ad290 2024-04-25T10:02:20.318 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 msgr2=0x7fd7a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 0x7fd7a0069450 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009a30 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 msgr2=0x7fd7c00c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7a8005040 tx=0x7fd7ac005840 comp rx=0 tx=0).stop 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 shutdown_connections 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd7a0066f90 0x7fd7a0069450 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd7c00ca840 0x7fd7c00cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd7c00ad290 0x7fd7c00c2e80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 --2- 172.21.15.82:0/1726743763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd7ac003fe0 0x7fd7c01547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 >> 172.21.15.82:0/1726743763 conn(0x7fd7c006ae60 msgr2=0x7fd7c015f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 shutdown_connections 2024-04-25T10:02:20.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.312+0000 7fd7c60de640 1 -- 172.21.15.82:0/1726743763 wait complete. 2024-04-25T10:02:20.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 Processor -- start 2024-04-25T10:02:20.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 -- start start 2024-04-25T10:02:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 0x7faee415c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faee40b2980 0x7faee40cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee40cfe60 con 0x7faee40cf9e0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faee40d0140 con 0x7faee40b2980 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeec678640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee415bf30 con 0x7faee40cc950 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faee40cf9e0 0x7faee406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57888/0 (socket says 172.21.15.82:57888) 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 -- 172.21.15.82:0/3170099317 learned_addr learned my addr 172.21.15.82:0/3170099317 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9c13640 1 --2- 172.21.15.82:0/3170099317 >> v2:172.21.15.196:3300/0 conn(0x7faee40b2980 0x7faee40cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeea414640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 0x7faee415c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234147478 0 0) 0x7faee40cfe60 con 0x7faee40cf9e0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faed0003530 con 0x7faee40cf9e0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053834619 0 0) 0x7faed0003530 con 0x7faee40cf9e0 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 msgr2=0x7faee415c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 0x7faee415c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 >> v2:172.21.15.196:3300/0 conn(0x7faee40b2980 msgr2=0x7faee40cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 --2- 172.21.15.82:0/3170099317 >> v2:172.21.15.196:3300/0 conn(0x7faee40b2980 0x7faee40cf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee415c210 con 0x7faee40cf9e0 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7faedc001a00 con 0x7faee40cf9e0 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 >> v1:172.21.15.82:6789/0 conn(0x7faee40cf9e0 legacy=0x7faee406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 0x7faed0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed0003fb0 0x7faed0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed0004d50 0x7faed00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee40d0140 con 0x7faed0004d50 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee415bf30 con 0x7faed0003fb0 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed000bb30 con 0x7faee40cc950 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed0004d50 0x7faed00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 -- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 msgr2=0x7faed0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40cc950 0x7faed0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 -- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed0003fb0 msgr2=0x7faed0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faed0003fb0 0x7faed0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.440+0000 7faeeac15640 1 -- 172.21.15.82:0/3170099317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed0003530 con 0x7faed0004d50 2024-04-25T10:02:20.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeeac15640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed0004d50 0x7faed00051d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faedc00a870 tx=0x7faedc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571ee62d7eb9319d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc00a4c0 con 0x7faed0004d50 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faedc0072d0 con 0x7faed0004d50 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faee9412640 1 -- 172.21.15.82:0/3170099317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faedc0076c0 con 0x7faed0004d50 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- 172.21.15.82:0/3170099317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed0004d50 msgr2=0x7faed00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 --2- 172.21.15.82:0/3170099317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faed0004d50 0x7faed00051d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7faedc00a870 tx=0x7faedc01d3a0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeeac15640 1 -- 172.21.15.82:0/3170099317 reap_dead start 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- 172.21.15.82:0/3170099317 shutdown_connections 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- 172.21.15.82:0/3170099317 >> 172.21.15.82:0/3170099317 conn(0x7faee4099a70 msgr2=0x7faee40b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- 172.21.15.82:0/3170099317 shutdown_connections 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- 172.21.15.82:0/3170099317 wait complete. 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 Processor -- start 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.444+0000 7faeec678640 1 -- start start 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 0x7faee40b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 0x7faee40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee415fb60 con 0x7faee41553e0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee415f9e0 con 0x7faee41e8bd0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee415fce0 con 0x7faee40b52a0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:59974/0 (socket says 172.21.15.82:59974) 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 -- 172.21.15.82:0/523322600 learned_addr learned my addr 172.21.15.82:0/523322600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faee9c13640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 0x7faee40b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeeac15640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 0x7faee40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 -- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 msgr2=0x7faee40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 0x7faee40c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 -- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 msgr2=0x7faee40b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 0x7faee40b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 -- 172.21.15.82:0/523322600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee41557c0 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faee9c13640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 0x7faee40b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeeac15640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 0x7faee40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeea414640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faee000e7c0 tx=0x7faee000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee0017070 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee000c8c0 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee41e75f0 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee000cc90 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee41e7bb0 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee404efa0 con 0x7faee41553e0 2024-04-25T10:02:20.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faee001a070 con 0x7faee41553e0 2024-04-25T10:02:20.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 0x7faec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faee00851d0 con 0x7faee41553e0 2024-04-25T10:02:20.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faee9c13640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 0x7faec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.448+0000 7faee9c13640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 0x7faec8069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faed40047c0 tx=0x7faed4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.452+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faee0055ea0 con 0x7faee41553e0 2024-04-25T10:02:20.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.604+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faee415e450 con 0x7faee41553e0 2024-04-25T10:02:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.604+0000 7faedaffd640 1 -- 172.21.15.82:0/523322600 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7faee00557a0 con 0x7faee41553e0 2024-04-25T10:02:20.613 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 msgr2=0x7faec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 0x7faec8069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7faed40047c0 tx=0x7faed4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 msgr2=0x7faee41e8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7faee000e7c0 tx=0x7faee000ec80 comp rx=0 tx=0).stop 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 shutdown_connections 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faec8066f60 0x7faec8069420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faee40b52a0 0x7faee40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faee41e8bd0 0x7faee40b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 --2- 172.21.15.82:0/523322600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faee41553e0 0x7faee41e8690 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 >> 172.21.15.82:0/523322600 conn(0x7faee4099a70 msgr2=0x7faee40b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 shutdown_connections 2024-04-25T10:02:20.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.608+0000 7faeec678640 1 -- 172.21.15.82:0/523322600 wait complete. 2024-04-25T10:02:20.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 Processor -- start 2024-04-25T10:02:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 -- start start 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ab80b3cf0 0x7f3ab80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80cbf10 con 0x7f3ab815e6c0 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab80cc1f0 con 0x7f3ab80cfad0 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abe34b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab80cff40 con 0x7f3ab80b3cf0 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abc8e8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3ab815e6c0 0x7f3ab80caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57904/0 (socket says 172.21.15.82:57904) 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abc8e8640 1 -- 172.21.15.82:0/863885164 learned_addr learned my addr 172.21.15.82:0/863885164 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/863885164 >> v2:172.21.15.196:3300/0 conn(0x7f3ab80b3cf0 0x7f3ab80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1922106796 0 0) 0x7f3ab80cbf10 con 0x7f3ab815e6c0 2024-04-25T10:02:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3aa0003530 con 0x7f3ab815e6c0 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 800318085 0 0) 0x7f3aa0003530 con 0x7f3ab815e6c0 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 >> v2:172.21.15.196:3300/0 conn(0x7f3ab80b3cf0 msgr2=0x7f3ab80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 --2- 172.21.15.82:0/863885164 >> v2:172.21.15.196:3300/0 conn(0x7f3ab80b3cf0 0x7f3ab80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 >> v1:172.21.15.196:6789/0 conn(0x7f3ab80cfad0 legacy=0x7f3ab815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80cc4d0 con 0x7f3ab815e6c0 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3ab0001680 con 0x7f3ab815e6c0 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 >> v1:172.21.15.82:6789/0 conn(0x7f3ab815e6c0 legacy=0x7f3ab80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 0x7f3aa0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa0004810 0x7f3aa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0005220 0x7f3aa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80cff40 con 0x7f3aa0003a80 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa000c150 con 0x7f3aa0004810 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa000c350 con 0x7f3aa0005220 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 0x7f3aa0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab77fe640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa0004810 0x7f3aa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abc8e8640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0005220 0x7f3aa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 -- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0005220 msgr2=0x7f3aa000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0005220 0x7f3aa000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 -- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa0004810 msgr2=0x7f3aa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa0004810 0x7f3aa0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab7fff640 1 -- 172.21.15.82:0/863885164 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0003530 con 0x7f3aa0003a80 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3abc8e8640 1 -- 172.21.15.82:0/863885164 reap_dead start 2024-04-25T10:02:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.744+0000 7f3ab77fe640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa0004810 0x7f3aa0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 0x7f3aa0003ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009fc0 tx=0x7f3aa801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa78ff2931b6fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa80b2070 con 0x7f3aa0003a80 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa80a5c10 con 0x7f3aa0003a80 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3ab6ffd640 1 -- 172.21.15.82:0/863885164 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8009c10 con 0x7f3aa0003a80 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 msgr2=0x7f3aa0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 0x7f3aa0003ea0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009fc0 tx=0x7f3aa801cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- 172.21.15.82:0/863885164 shutdown_connections 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 --2- 172.21.15.82:0/863885164 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa0003a80 0x7f3aa0003ea0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- 172.21.15.82:0/863885164 >> 172.21.15.82:0/863885164 conn(0x7f3ab806ae60 msgr2=0x7f3ab806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- 172.21.15.82:0/863885164 shutdown_connections 2024-04-25T10:02:20.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- 172.21.15.82:0/863885164 wait complete. 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 Processor -- start 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.748+0000 7f3abe34b640 1 -- start start 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 0x7f3ab80c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ab80c8bb0 0x7f3ab8070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab815eaa0 con 0x7f3ab80714e0 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80cccd0 con 0x7f3ab80c8bb0 2024-04-25T10:02:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab80b40d0 con 0x7f3aa0003a80 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60000/0 (socket says 172.21.15.82:60000) 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 -- 172.21.15.82:0/4161751097 learned_addr learned my addr 172.21.15.82:0/4161751097 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab77fe640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ab80c8bb0 0x7f3ab8070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 0x7f3ab80c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 -- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 msgr2=0x7f3ab80c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 0x7f3ab80c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 -- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ab80c8bb0 msgr2=0x7f3ab8070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ab80c8bb0 0x7f3ab8070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 -- 172.21.15.82:0/4161751097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80c4960 con 0x7f3ab80714e0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 0x7f3ab80c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abc8e8640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80d01d0 tx=0x7f3ab000d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab00074e0 con 0x7f3ab80714e0 2024-04-25T10:02:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ab00047f0 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab80c3380 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0007680 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab80c3940 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab804efa0 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ab001b020 con 0x7f3ab80714e0 2024-04-25T10:02:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 0x7f3aa4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3ab0084f50 con 0x7f3ab80714e0 2024-04-25T10:02:20.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 0x7f3aa4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.752+0000 7f3ab7fff640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 0x7f3aa4069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80af000 tx=0x7f3aa80a7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.756+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ab0055c10 con 0x7f3ab80714e0 2024-04-25T10:02:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.900+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3ab80c61f0 con 0x7f3ab80714e0 2024-04-25T10:02:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3ab4ff9640 1 -- 172.21.15.82:0/4161751097 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ab0055510 con 0x7f3ab80714e0 2024-04-25T10:02:20.911 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:20.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 msgr2=0x7f3aa4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 0x7f3aa4069450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80af000 tx=0x7f3aa80a7040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 msgr2=0x7f3ab80c4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80d01d0 tx=0x7f3ab000d8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 shutdown_connections 2024-04-25T10:02:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3aa4066f90 0x7f3aa4069450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ab80714e0 0x7f3ab80c4db0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ab80c8bb0 0x7f3ab8070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 --2- 172.21.15.82:0/4161751097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa0003a80 0x7f3ab80c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 >> 172.21.15.82:0/4161751097 conn(0x7f3ab806ae60 msgr2=0x7f3ab806d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 shutdown_connections 2024-04-25T10:02:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:20.904+0000 7f3abe34b640 1 -- 172.21.15.82:0/4161751097 wait complete. 2024-04-25T10:02:20.928 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:21.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.044+0000 7fe1571fa640 1 Processor -- start 2024-04-25T10:02:21.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.044+0000 7fe1571fa640 1 -- start start 2024-04-25T10:02:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe1571fa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 0x7fe15006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe1571fa640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe15015da40 0x7fe15006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe1571fa640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe150143b70 con 0x7fe15006c1e0 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe1571fa640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15006e170 con 0x7fe15015da40 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe1571fa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15006e450 con 0x7fe150143790 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe155797640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe15006c1e0 0x7fe15015a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57912/0 (socket says 172.21.15.82:57912) 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe155797640 1 -- 172.21.15.82:0/1635722888 learned_addr learned my addr 172.21.15.82:0/1635722888 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126962933 0 0) 0x7fe150143b70 con 0x7fe15006c1e0 2024-04-25T10:02:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe140004640 con 0x7fe15006c1e0 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 --2- 172.21.15.82:0/1635722888 >> v2:172.21.15.196:3300/0 conn(0x7fe15015da40 0x7fe15006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe154f96640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 0x7fe15006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3277367213 0 0) 0x7fe140004640 con 0x7fe15006c1e0 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 msgr2=0x7fe15006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 0x7fe15006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 >> v2:172.21.15.196:3300/0 conn(0x7fe15015da40 msgr2=0x7fe15006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 --2- 172.21.15.82:0/1635722888 >> v2:172.21.15.196:3300/0 conn(0x7fe15015da40 0x7fe15006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15006e730 con 0x7fe15006c1e0 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe148001a00 con 0x7fe15006c1e0 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 >> v1:172.21.15.82:6789/0 conn(0x7fe15006c1e0 legacy=0x7fe15015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe140004b40 0x7fe140004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe140005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 0x7fe1400062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15006e170 con 0x7fe140005920 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15006e450 con 0x7fe140004b40 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14000cb60 con 0x7fe150143790 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe140005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 -- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 msgr2=0x7fe1400062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe150143790 0x7fe1400062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe154f96640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe140004b40 0x7fe140004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 -- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe140004b40 msgr2=0x7fe140004f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe140004b40 0x7fe140004f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.048+0000 7fe14ffff640 1 -- 172.21.15.82:0/1635722888 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140004640 con 0x7fe140005920 2024-04-25T10:02:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14ffff640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe140005da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009ab0 tx=0x7fe13c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3429d62efcb2af64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b2070 con 0x7fe140005920 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0a55a0 con 0x7fe140005920 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14f7fe640 1 -- 172.21.15.82:0/1635722888 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0096b0 con 0x7fe140005920 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/1635722888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 msgr2=0x7fe140005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 --2- 172.21.15.82:0/1635722888 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe140005da0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009ab0 tx=0x7fe13c01ca90 comp rx=0 tx=0).stop 2024-04-25T10:02:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe155797640 1 -- 172.21.15.82:0/1635722888 reap_dead start 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/1635722888 shutdown_connections 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/1635722888 >> 172.21.15.82:0/1635722888 conn(0x7fe150071810 msgr2=0x7fe150071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/1635722888 shutdown_connections 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/1635722888 wait complete. 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 Processor -- start 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- start start 2024-04-25T10:02:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 0x7fe1500c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1500ce320 0x7fe1500c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b4380 con 0x7fe140005920 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b4200 con 0x7fe1500cdea0 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1500b4500 con 0x7fe1500ce320 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60010/0 (socket says 172.21.15.82:60010) 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 -- 172.21.15.82:0/2642598340 learned_addr learned my addr 172.21.15.82:0/2642598340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14ffff640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 0x7fe1500c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe155797640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1500ce320 0x7fe1500c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 -- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1500ce320 msgr2=0x7fe1500c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1500ce320 0x7fe1500c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 -- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 msgr2=0x7fe1500c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 0x7fe1500c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 -- 172.21.15.82:0/2642598340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500ca240 con 0x7fe140005920 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14ffff640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 0x7fe1500c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe154f96640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e810 tx=0x7fe14400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144017070 con 0x7fe140005920 2024-04-25T10:02:21.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400c6d0 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400f040 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1501322e0 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.052+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1500ca7c0 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15004efa0 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe14401a070 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe14d7fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe144084f00 con 0x7fe140005920 2024-04-25T10:02:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe14ffff640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.056+0000 7fe14ffff640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 0x7fe134069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009ab0 tx=0x7fe13c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.060+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe144055b30 con 0x7fe140005920 2024-04-25T10:02:21.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.208+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe150072670 con 0x7fe140005920 2024-04-25T10:02:21.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.208+0000 7fe14d7fa640 1 -- 172.21.15.82:0/2642598340 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe144055430 con 0x7fe140005920 2024-04-25T10:02:21.217 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 msgr2=0x7fe134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 0x7fe134069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe13c009ab0 tx=0x7fe13c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 msgr2=0x7fe15012f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe14400e810 tx=0x7fe14400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 shutdown_connections 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe134066f60 0x7fe134069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1500ce320 0x7fe1500c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1500cdea0 0x7fe1500c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 --2- 172.21.15.82:0/2642598340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe140005920 0x7fe15012f5b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 >> 172.21.15.82:0/2642598340 conn(0x7fe150071810 msgr2=0x7fe150073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 shutdown_connections 2024-04-25T10:02:21.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.212+0000 7fe1571fa640 1 -- 172.21.15.82:0/2642598340 wait complete. 2024-04-25T10:02:21.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 Processor -- start 2024-04-25T10:02:21.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 -- start start 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fde10144ca0 0x7fde1006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 0x7fde10142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde100ccc70 con 0x7fde100ce510 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde100cc2b0 con 0x7fde10144ca0 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde18314640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1006e290 con 0x7fde1006ec60 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde158af640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fde10144ca0 0x7fde1006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde160b0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fde100ce510 0x7fde100ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57920/0 (socket says 172.21.15.82:57920) 2024-04-25T10:02:21.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde160b0640 1 -- 172.21.15.82:0/3753246800 learned_addr learned my addr 172.21.15.82:0/3753246800 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde168b1640 1 --2- 172.21.15.82:0/3753246800 >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 0x7fde10142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde158af640 1 -- 172.21.15.82:0/3753246800 >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 msgr2=0x7fde10142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde158af640 1 --2- 172.21.15.82:0/3753246800 >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 0x7fde10142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde150ae640 1 -- 172.21.15.82:0/3753246800 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23600203 0 0) 0x7fde100ccc70 con 0x7fde100ce510 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde158af640 1 -- 172.21.15.82:0/3753246800 >> v1:172.21.15.82:6789/0 conn(0x7fde100ce510 legacy=0x7fde100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde158af640 1 -- 172.21.15.82:0/3753246800 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1006dc40 con 0x7fde10144ca0 2024-04-25T10:02:21.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.348+0000 7fde168b1640 1 --2- 172.21.15.82:0/3753246800 >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 0x7fde10142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde158af640 1 --2- 172.21.15.82:0/3753246800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde1006e720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde100b48f0 tx=0x7fde0001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fea100c5fdf3628e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde150ae640 1 -- 172.21.15.82:0/3753246800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000b5070 con 0x7fde10144ca0 2024-04-25T10:02:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde150ae640 1 -- 172.21.15.82:0/3753246800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde000a8970 con 0x7fde10144ca0 2024-04-25T10:02:21.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde150ae640 1 -- 172.21.15.82:0/3753246800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde000a8d40 con 0x7fde10144ca0 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- 172.21.15.82:0/3753246800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 msgr2=0x7fde1006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 --2- 172.21.15.82:0/3753246800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde1006e720 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde100b48f0 tx=0x7fde0001ec70 comp rx=0 tx=0).stop 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- 172.21.15.82:0/3753246800 shutdown_connections 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 --2- 172.21.15.82:0/3753246800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde1006e720 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 --2- 172.21.15.82:0/3753246800 >> v2:172.21.15.196:3300/0 conn(0x7fde1006ec60 0x7fde10142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- 172.21.15.82:0/3753246800 >> 172.21.15.82:0/3753246800 conn(0x7fde100b11a0 msgr2=0x7fde100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- 172.21.15.82:0/3753246800 shutdown_connections 2024-04-25T10:02:21.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- 172.21.15.82:0/3753246800 wait complete. 2024-04-25T10:02:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 Processor -- start 2024-04-25T10:02:21.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 -- start start 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.352+0000 7fde18314640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1006ec60 0x7fde100c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 0x7fde100c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde101451f0 con 0x7fde10144ca0 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100cc900 con 0x7fde1006ec60 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde10145370 con 0x7fde100ce510 2024-04-25T10:02:21.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60036/0 (socket says 172.21.15.82:60036) 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 -- 172.21.15.82:0/1173889433 learned_addr learned my addr 172.21.15.82:0/1173889433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde158af640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 0x7fde100c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde160b0640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1006ec60 0x7fde100c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 -- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 msgr2=0x7fde100c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 0x7fde100c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 -- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1006ec60 msgr2=0x7fde100c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1006ec60 0x7fde100c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 -- 172.21.15.82:0/1173889433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde100c76c0 con 0x7fde10144ca0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde158af640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 0x7fde100c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde168b1640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde0800cef0 tx=0x7fde0800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0801a070 con 0x7fde10144ca0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0800d860 con 0x7fde10144ca0 2024-04-25T10:02:21.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde100c79a0 con 0x7fde10144ca0 2024-04-25T10:02:21.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde0800dc50 con 0x7fde10144ca0 2024-04-25T10:02:21.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde100c7f60 con 0x7fde10144ca0 2024-04-25T10:02:21.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1004efa0 con 0x7fde10144ca0 2024-04-25T10:02:21.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde0801d070 con 0x7fde10144ca0 2024-04-25T10:02:21.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 0x7fddf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fde08085b90 con 0x7fde10144ca0 2024-04-25T10:02:21.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde160b0640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 0x7fddf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.356+0000 7fde160b0640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 0x7fddf4069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000f70 tx=0x7fde0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.360+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde080569b0 con 0x7fde10144ca0 2024-04-25T10:02:21.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde100b1cc0 con 0x7fde10144ca0 2024-04-25T10:02:21.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde06ffd640 1 -- 172.21.15.82:0/1173889433 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde080562b0 con 0x7fde10144ca0 2024-04-25T10:02:21.535 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 msgr2=0x7fddf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 0x7fddf4069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fde0c000f70 tx=0x7fde0c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 msgr2=0x7fde100c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fde0800cef0 tx=0x7fde0800a710 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 shutdown_connections 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fddf4066f60 0x7fddf4069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fde100ce510 0x7fde100c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fde10144ca0 0x7fde100c6f50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 --2- 172.21.15.82:0/1173889433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fde1006ec60 0x7fde100c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.528+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 >> 172.21.15.82:0/1173889433 conn(0x7fde100b11a0 msgr2=0x7fde100a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.532+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 shutdown_connections 2024-04-25T10:02:21.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:21.532+0000 7fde18314640 1 -- 172.21.15.82:0/1173889433 wait complete. 2024-04-25T10:02:21.553 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:02:21.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.554 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:21.574 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi082.stderr:ceph-fuse[34452]: fuse finished with error 0 and tester_r 0 2024-04-25T10:02:21.575 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:27.677 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:02:27.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:27.677 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T10:02:27.701 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 Processor -- start 2024-04-25T10:02:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 -- start start 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcfd80d1cd0 0x7fcfd80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfd80d0620 0x7fcfd80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd806c440 con 0x7fcfd80d0620 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd806be60 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfdcc09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd815de60 con 0x7fcfd815da80 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfd80d0620 0x7fcfd80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfd80d0620 0x7fcfd80ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48670/0 (socket says 172.21.15.82:48670) 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 -- 172.21.15.82:0/4276246563 learned_addr learned my addr 172.21.15.82:0/4276246563 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd5d9b640 1 --2- 172.21.15.82:0/4276246563 >> v2:172.21.15.196:3300/0 conn(0x7fcfd80d1cd0 0x7fcfd80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd659c640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 -- 172.21.15.82:0/4276246563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 msgr2=0x7fcfd80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 -- 172.21.15.82:0/4276246563 >> v2:172.21.15.196:3300/0 conn(0x7fcfd80d1cd0 msgr2=0x7fcfd80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 --2- 172.21.15.82:0/4276246563 >> v2:172.21.15.196:3300/0 conn(0x7fcfd80d1cd0 0x7fcfd80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd6d9d640 1 -- 172.21.15.82:0/4276246563 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd80d2450 con 0x7fcfd80d0620 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.864+0000 7fcfd659c640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfd6d9d640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d0620 0x7fcfd80ce3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfd806bd60 tx=0x7fcfd001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a89218ef7fbe5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfd559a640 1 -- 172.21.15.82:0/4276246563 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a5690 con 0x7fcfd80d0620 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfd559a640 1 -- 172.21.15.82:0/4276246563 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd00a5830 con 0x7fcfd80d0620 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfd559a640 1 -- 172.21.15.82:0/4276246563 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd00a5c00 con 0x7fcfd80d0620 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4276246563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d0620 msgr2=0x7fcfd80ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d0620 0x7fcfd80ce3b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcfd806bd60 tx=0x7fcfd001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4276246563 shutdown_connections 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d0620 0x7fcfd80ce3b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4276246563 >> v2:172.21.15.196:3300/0 conn(0x7fcfd80d1cd0 0x7fcfd80d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4276246563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4276246563 >> 172.21.15.82:0/4276246563 conn(0x7fcfd808b000 msgr2=0x7fcfd80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4276246563 shutdown_connections 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.868+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4276246563 wait complete. 2024-04-25T10:02:27.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 Processor -- start 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- start start 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd80d0620 0x7fcfd8152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806ca30 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806c8b0 con 0x7fcfd80d0620 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806cbb0 con 0x7fcfd815da80 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48684/0 (socket says 172.21.15.82:48684) 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 -- 172.21.15.82:0/4147506678 learned_addr learned my addr 172.21.15.82:0/4147506678 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd659c640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd80d0620 0x7fcfd8152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd6d9d640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 -- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 msgr2=0x7fcfd81584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd81584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 -- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd80d0620 msgr2=0x7fcfd8152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd80d0620 0x7fcfd8152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 -- 172.21.15.82:0/4147506678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd8154430 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfd5d9b640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d800 tx=0x7fcfcc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc018020 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc0096a0 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd8154710 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc009a50 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd8156440 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.872+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd804efa0 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.876+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfcc01b020 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.876+0000 7fcfc6ffd640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 0x7fcfb8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.876+0000 7fcfd659c640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 0x7fcfb8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.876+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcfcc0858c0 con 0x7fcfd80d1cd0 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.876+0000 7fcfd659c640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 0x7fcfb8069450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:27.880+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfcc022020 con 0x7fcfd80d1cd0 2024-04-25T10:02:28.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.040+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd80b3770 con 0x7fcfd80d1cd0 2024-04-25T10:02:28.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.040+0000 7fcfc6ffd640 1 -- 172.21.15.82:0/4147506678 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fcfcc056620 con 0x7fcfd80d1cd0 2024-04-25T10:02:28.050 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:28.050 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 msgr2=0x7fcfb8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 0x7fcfb8069450 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcfc8006fd0 tx=0x7fcfc8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 msgr2=0x7fcfd8157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00d800 tx=0x7fcfcc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 shutdown_connections 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfb8066f90 0x7fcfb8069450 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfd815da80 0x7fcfd81584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfd80d1cd0 0x7fcfd8157f60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 --2- 172.21.15.82:0/4147506678 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfd80d0620 0x7fcfd8152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 >> 172.21.15.82:0/4147506678 conn(0x7fcfd808b000 msgr2=0x7fcfd80b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 shutdown_connections 2024-04-25T10:02:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.044+0000 7fcfdcc09640 1 -- 172.21.15.82:0/4147506678 wait complete. 2024-04-25T10:02:28.068 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.180+0000 7f3046d70640 1 Processor -- start 2024-04-25T10:02:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.180+0000 7f3046d70640 1 -- start start 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3046d70640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f304015d970 0x7f30400c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3046d70640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30400cec00 con 0x7f30400cb1e0 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3046d70640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30400ceee0 con 0x7f30400cf9c0 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3046d70640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30400c73a0 con 0x7f304015d970 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f304530d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f30400cb1e0 0x7f3040161490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57872/0 (socket says 172.21.15.82:57872) 2024-04-25T10:02:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f304530d640 1 -- 172.21.15.82:0/473057962 learned_addr learned my addr 172.21.15.82:0/473057962 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 --2- 172.21.15.82:0/473057962 >> v2:172.21.15.196:3300/0 conn(0x7f304015d970 0x7f30400c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510649519 0 0) 0x7f30400cec00 con 0x7f30400cb1e0 2024-04-25T10:02:28.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3028003530 con 0x7f30400cb1e0 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3124620551 0 0) 0x7f3028003530 con 0x7f30400cb1e0 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 >> v2:172.21.15.196:3300/0 conn(0x7f304015d970 msgr2=0x7f30400c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 --2- 172.21.15.82:0/473057962 >> v2:172.21.15.196:3300/0 conn(0x7f304015d970 0x7f30400c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 >> v1:172.21.15.196:6789/0 conn(0x7f30400cf9c0 legacy=0x7f30400caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400cf1c0 con 0x7f30400cb1e0 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3038001a00 con 0x7f30400cb1e0 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 >> v1:172.21.15.82:6789/0 conn(0x7f30400cb1e0 legacy=0x7f3040161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 0x7f3028003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028004810 0x7f3028004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280051d0 0x7f302800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302800c0d0 con 0x7f3028003a30 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302800c280 con 0x7f30280051d0 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302800c480 con 0x7f3028004810 2024-04-25T10:02:28.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 0x7f3028003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 -- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028004810 msgr2=0x7f3028004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f303ffff640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028004810 0x7f3028004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f304530d640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280051d0 0x7f302800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028004810 0x7f3028004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 -- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280051d0 msgr2=0x7f302800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280051d0 0x7f302800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 -- 172.21.15.82:0/473057962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3028003530 con 0x7f3028003a30 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f304530d640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30280051d0 0x7f302800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f304530d640 1 -- 172.21.15.82:0/473057962 reap_dead start 2024-04-25T10:02:28.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.184+0000 7f3044b0c640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 0x7f3028003e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3030009ae0 tx=0x7f303001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d1356c70d364ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300b2070 con 0x7f3028003a30 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300a55e0 con 0x7f3028003a30 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303f7fe640 1 -- 172.21.15.82:0/473057962 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300096b0 con 0x7f3028003a30 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 msgr2=0x7f3028003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 0x7f3028003e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3030009ae0 tx=0x7f303001ca30 comp rx=0 tx=0).stop 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/473057962 shutdown_connections 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 --2- 172.21.15.82:0/473057962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3028003a30 0x7f3028003e50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/473057962 >> 172.21.15.82:0/473057962 conn(0x7f3040099a70 msgr2=0x7f30400d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/473057962 shutdown_connections 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/473057962 wait complete. 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 Processor -- start 2024-04-25T10:02:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- start start 2024-04-25T10:02:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028003a30 0x7f3040159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304015bfd0 0x7f304015c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30400cae60 con 0x7f30400c85a0 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30400cace0 con 0x7f304015bfd0 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30400cafe0 con 0x7f3028003a30 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48690/0 (socket says 172.21.15.82:48690) 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 -- 172.21.15.82:0/2181648353 learned_addr learned my addr 172.21.15.82:0/2181648353 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f304530d640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304015bfd0 0x7f304015c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3044b0c640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028003a30 0x7f3040159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 -- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028003a30 msgr2=0x7f3040159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028003a30 0x7f3040159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 -- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304015bfd0 msgr2=0x7f304015c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304015bfd0 0x7f304015c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 -- 172.21.15.82:0/2181648353 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30400c91b0 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303ffff640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f303400d570 tx=0x7f303400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034005480 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3034017070 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30400c9490 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034003820 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.188+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30400c9a50 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304004efa0 con 0x7f30400c85a0 2024-04-25T10:02:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30340039c0 con 0x7f30400c85a0 2024-04-25T10:02:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f303d7fa640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3034085240 con 0x7f30400c85a0 2024-04-25T10:02:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f3044b0c640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.192+0000 7f3044b0c640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 0x7f302c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30300af000 tx=0x7f3030008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.196+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3034019310 con 0x7f30400c85a0 2024-04-25T10:02:28.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.352+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3040099fc0 con 0x7f30400c85a0 2024-04-25T10:02:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f303d7fa640 1 -- 172.21.15.82:0/2181648353 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f3034055f00 con 0x7f30400c85a0 2024-04-25T10:02:28.363 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:28.363 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 msgr2=0x7f302c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 0x7f302c069450 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f30300af000 tx=0x7f3030008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 msgr2=0x7f30400c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f303400d570 tx=0x7f303400da30 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 shutdown_connections 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f304015bfd0 0x7f304015c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f30400c85a0 0x7f30400c8a20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 --2- 172.21.15.82:0/2181648353 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3028003a30 0x7f3040159e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 >> 172.21.15.82:0/2181648353 conn(0x7f3040099a70 msgr2=0x7f30400cfda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 shutdown_connections 2024-04-25T10:02:28.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.356+0000 7f3046d70640 1 -- 172.21.15.82:0/2181648353 wait complete. 2024-04-25T10:02:28.380 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:28.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 Processor -- start 2024-04-25T10:02:28.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 -- start start 2024-04-25T10:02:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 0x7ffa700cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa7015c260 con 0x7ffa7006b1d0 2024-04-25T10:02:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa7006b640 con 0x7ffa7015ba20 2024-04-25T10:02:28.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa75f90640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d1430 con 0x7ffa7015fd90 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa6f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 0x7ffa700cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56626/0 (socket says 172.21.15.82:56626) 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa6f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 0x7ffa700cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43416/0 (socket says 172.21.15.82:43416) 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 -- 172.21.15.82:0/1980047147 learned_addr learned my addr 172.21.15.82:0/1980047147 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 -- 172.21.15.82:0/1980047147 >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 msgr2=0x7ffa700cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 --2- 172.21.15.82:0/1980047147 >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 0x7ffa700cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 -- 172.21.15.82:0/1980047147 >> v1:172.21.15.196:6789/0 conn(0x7ffa7006b1d0 legacy=0x7ffa700cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.488+0000 7ffa67fff640 1 -- 172.21.15.82:0/1980047147 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa7015c540 con 0x7ffa7015fd90 2024-04-25T10:02:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa67fff640 1 --2- 172.21.15.82:0/1980047147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009870 tx=0x7ffa5801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=417546bbef19ae70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa6effd640 1 -- 172.21.15.82:0/1980047147 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580b2070 con 0x7ffa7015fd90 2024-04-25T10:02:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa6effd640 1 -- 172.21.15.82:0/1980047147 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa580a9070 con 0x7ffa7015fd90 2024-04-25T10:02:28.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa6effd640 1 -- 172.21.15.82:0/1980047147 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa580ad880 con 0x7ffa7015fd90 2024-04-25T10:02:28.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- 172.21.15.82:0/1980047147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 msgr2=0x7ffa7006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 --2- 172.21.15.82:0/1980047147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009870 tx=0x7ffa5801c560 comp rx=0 tx=0).stop 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- 172.21.15.82:0/1980047147 shutdown_connections 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 --2- 172.21.15.82:0/1980047147 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa7006ac90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 --2- 172.21.15.82:0/1980047147 >> v2:172.21.15.196:3300/0 conn(0x7ffa7015ba20 0x7ffa700cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- 172.21.15.82:0/1980047147 >> 172.21.15.82:0/1980047147 conn(0x7ffa70099a70 msgr2=0x7ffa700d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- 172.21.15.82:0/1980047147 shutdown_connections 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- 172.21.15.82:0/1980047147 wait complete. 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 Processor -- start 2024-04-25T10:02:28.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.492+0000 7ffa75f90640 1 -- start start 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa7015ba20 0x7ffa70152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa70156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d01f0 con 0x7ffa7006b1d0 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7015be00 con 0x7ffa7015ba20 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b4740 con 0x7ffa7015fd90 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48702/0 (socket says 172.21.15.82:48702) 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 -- 172.21.15.82:0/436959998 learned_addr learned my addr 172.21.15.82:0/436959998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6ffff640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa70156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 -- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 msgr2=0x7ffa70156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa70156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 -- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa7015ba20 msgr2=0x7ffa70152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa7015ba20 0x7ffa70152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 -- 172.21.15.82:0/436959998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700c4410 con 0x7ffa7006b1d0 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6ffff640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa70156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6f7fe640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e960 tx=0x7ffa6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60017070 con 0x7ffa7006b1d0 2024-04-25T10:02:28.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000ca00 con 0x7ffa7006b1d0 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700c46f0 con 0x7ffa7006b1d0 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000cdb0 con 0x7ffa7006b1d0 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700c4cb0 con 0x7ffa7006b1d0 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa7004efa0 con 0x7ffa7006b1d0 2024-04-25T10:02:28.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa6001a070 con 0x7ffa7006b1d0 2024-04-25T10:02:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 0x7ffa50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa67fff640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 0x7ffa50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ffa60085450 con 0x7ffa7006b1d0 2024-04-25T10:02:28.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.496+0000 7ffa67fff640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 0x7ffa50069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa580af000 tx=0x7ffa58004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.504+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa60056080 con 0x7ffa7006b1d0 2024-04-25T10:02:28.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:28.680+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffa7015f5c0 con 0x7ffa7006b1d0 2024-04-25T10:02:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.464+0000 7ffa6cff9640 1 -- 172.21.15.82:0/436959998 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffa60055980 con 0x7ffa7006b1d0 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 msgr2=0x7ffa50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 0x7ffa50069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ffa580af000 tx=0x7ffa58004740 comp rx=0 tx=0).stop 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 msgr2=0x7ffa700b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000e960 tx=0x7ffa6000ee20 comp rx=0 tx=0).stop 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 shutdown_connections 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa50066f60 0x7ffa50069420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa7015fd90 0x7ffa70156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa7015ba20 0x7ffa70152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 --2- 172.21.15.82:0/436959998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa7006b1d0 0x7ffa700b4200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 >> 172.21.15.82:0/436959998 conn(0x7ffa70099a70 msgr2=0x7ffa70073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 shutdown_connections 2024-04-25T10:02:29.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.468+0000 7ffa75f90640 1 -- 172.21.15.82:0/436959998 wait complete. 2024-04-25T10:02:29.498 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T10:02:29.498 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:29.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 Processor -- start 2024-04-25T10:02:29.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 -- start start 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25f415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f25f40b2c30 0x7f25f406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25f4066c30 con 0x7f25f406c1e0 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25f406daa0 con 0x7f25f40b2c30 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25facd4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f406c640 con 0x7f25f40af280 2024-04-25T10:02:29.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f9271640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f25f406c1e0 0x7f25f4161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57884/0 (socket says 172.21.15.82:57884) 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f9271640 1 -- 172.21.15.82:0/4098401951 learned_addr learned my addr 172.21.15.82:0/4098401951 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f3fff640 1 --2- 172.21.15.82:0/4098401951 >> v2:172.21.15.196:3300/0 conn(0x7f25f40b2c30 0x7f25f406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25f415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76997338 0 0) 0x7f25f4066c30 con 0x7f25f406c1e0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25dc003530 con 0x7f25f406c1e0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3938529958 0 0) 0x7f25dc003530 con 0x7f25f406c1e0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 msgr2=0x7f25f415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25f415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 >> v2:172.21.15.196:3300/0 conn(0x7f25f40b2c30 msgr2=0x7f25f406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 --2- 172.21.15.82:0/4098401951 >> v2:172.21.15.196:3300/0 conn(0x7f25f40b2c30 0x7f25f406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f40af660 con 0x7f25f406c1e0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f25ec001a00 con 0x7f25f406c1e0 2024-04-25T10:02:29.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 >> v1:172.21.15.82:6789/0 conn(0x7f25f406c1e0 legacy=0x7f25f4161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25dc003a30 0x7f25dc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25dc004810 0x7f25dc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25dc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc00b9d0 con 0x7f25dc003a30 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc00bb50 con 0x7f25dc004810 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc00bcd0 con 0x7f25f40af280 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25dc003a30 0x7f25dc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f3fff640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25dc004810 0x7f25dc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f9271640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25dc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 -- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 msgr2=0x7f25dc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25dc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 -- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25dc004810 msgr2=0x7f25dc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25dc004810 0x7f25dc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f8a70640 1 -- 172.21.15.82:0/4098401951 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc003530 con 0x7f25dc003a30 2024-04-25T10:02:29.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.620+0000 7f25f9271640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f40af280 0x7f25dc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25f8a70640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25dc003a30 0x7f25dc003e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009b10 tx=0x7f25e801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cab0967cf4255b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e80096b0 con 0x7f25dc003a30 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e80a6460 con 0x7f25dc003a30 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25f37fe640 1 -- 172.21.15.82:0/4098401951 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e80a6850 con 0x7f25dc003a30 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- 172.21.15.82:0/4098401951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25dc003a30 msgr2=0x7f25dc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 --2- 172.21.15.82:0/4098401951 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25dc003a30 0x7f25dc003e50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009b10 tx=0x7f25e801ca70 comp rx=0 tx=0).stop 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25f9271640 1 -- 172.21.15.82:0/4098401951 reap_dead start 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- 172.21.15.82:0/4098401951 shutdown_connections 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- 172.21.15.82:0/4098401951 >> 172.21.15.82:0/4098401951 conn(0x7f25f4071810 msgr2=0x7f25f4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- 172.21.15.82:0/4098401951 shutdown_connections 2024-04-25T10:02:29.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- 172.21.15.82:0/4098401951 wait complete. 2024-04-25T10:02:29.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 Processor -- start 2024-04-25T10:02:29.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 -- start start 2024-04-25T10:02:29.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f413e790 0x7f25f41651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.624+0000 7f25facd4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f4166de0 0x7f25f4167260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4168890 con 0x7f25f414f2a0 2024-04-25T10:02:29.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f415e320 con 0x7f25f4166de0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25f4168a10 con 0x7f25f413e790 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48720/0 (socket says 172.21.15.82:48720) 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 -- 172.21.15.82:0/2488712660 learned_addr learned my addr 172.21.15.82:0/2488712660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f8a70640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f413e790 0x7f25f41651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f3fff640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f4166de0 0x7f25f4167260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 -- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f413e790 msgr2=0x7f25f41651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f413e790 0x7f25f41651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 -- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f4166de0 msgr2=0x7f25f4167260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f4166de0 0x7f25f4167260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 -- 172.21.15.82:0/2488712660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f413eb70 con 0x7f25f414f2a0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f9271640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25f4067980 tx=0x7f25ec00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec019070 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ec00bda0 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f415f2f0 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ec012610 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f415f8b0 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f404efa0 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25ec01c070 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 0x7f25e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f25ec0851c0 con 0x7f25f414f2a0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f8a70640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 0x7f25e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.628+0000 7f25f8a70640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 0x7f25e0069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25e80ad000 tx=0x7f25e80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.632+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f25f413eb70 con 0x7f25f414f2a0 2024-04-25T10:02:29.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:29.816+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f25f40727c0 con 0x7f25f414f2a0 2024-04-25T10:02:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25f17fa640 1 -- 172.21.15.82:0/2488712660 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f25ec055e90 con 0x7f25f414f2a0 2024-04-25T10:02:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 msgr2=0x7f25e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 0x7f25e0069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f25e80ad000 tx=0x7f25e80046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 msgr2=0x7f25f414f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25f4067980 tx=0x7f25ec00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 shutdown_connections 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f25e0066f60 0x7f25e0069420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25f414f2a0 0x7f25f414f720 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f25f4166de0 0x7f25f4167260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 --2- 172.21.15.82:0/2488712660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25f413e790 0x7f25f41651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.468+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 >> 172.21.15.82:0/2488712660 conn(0x7f25f4071810 msgr2=0x7f25f40ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.472+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 shutdown_connections 2024-04-25T10:02:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.472+0000 7f25facd4640 1 -- 172.21.15.82:0/2488712660 wait complete. 2024-04-25T10:02:30.497 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:30.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:30.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 Processor -- start 2024-04-25T10:02:30.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 -- start start 2024-04-25T10:02:30.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbde406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde40ccc70 con 0x7fbde406ec60 2024-04-25T10:02:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde406dc40 con 0x7fbde40ce510 2024-04-25T10:02:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbdea143640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40cc2b0 con 0x7fbde4144ca0 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbde406ec60 0x7fbde4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57886/0 (socket says 172.21.15.82:57886) 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 -- 172.21.15.82:0/3937699852 learned_addr learned my addr 172.21.15.82:0/3937699852 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbde406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575588608 0 0) 0x7fbde40ccc70 con 0x7fbde406ec60 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd0003530 con 0x7fbde406ec60 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 443254962 0 0) 0x7fbdd0003530 con 0x7fbde406ec60 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 msgr2=0x7fbde406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbde406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 >> v1:172.21.15.196:6789/0 conn(0x7fbde40ce510 legacy=0x7fbde40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde406df20 con 0x7fbde406ec60 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fbddc001a00 con 0x7fbde406ec60 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 >> v1:172.21.15.82:6789/0 conn(0x7fbde406ec60 legacy=0x7fbde4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbdd0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdd0003fe0 0x7fbdd0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbdd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40cc2b0 con 0x7fbdd0004d50 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd000bbc0 con 0x7fbdd0003fe0 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd000bdc0 con 0x7fbde4144ca0 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbdd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde37fe640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbdd0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdd0003fe0 0x7fbdd0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 -- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 msgr2=0x7fbdd0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde4144ca0 0x7fbdd0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 -- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdd0003fe0 msgr2=0x7fbdd0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdd0003fe0 0x7fbdd0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde3fff640 1 -- 172.21.15.82:0/3937699852 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd0003530 con 0x7fbdd0004d50 2024-04-25T10:02:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.612+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbdd0003fe0 0x7fbdd0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde3fff640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbdd00051d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbde404f950 tx=0x7fbddc01d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afdddf015f54ae5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00a4a0 con 0x7fbdd0004d50 2024-04-25T10:02:30.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc0ad650 con 0x7fbdd0004d50 2024-04-25T10:02:30.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde27fc640 1 -- 172.21.15.82:0/3937699852 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc0ada80 con 0x7fbdd0004d50 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- 172.21.15.82:0/3937699852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 msgr2=0x7fbdd00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 --2- 172.21.15.82:0/3937699852 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbdd00051d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbde404f950 tx=0x7fbddc01d340 comp rx=0 tx=0).stop 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde3fff640 1 -- 172.21.15.82:0/3937699852 reap_dead start 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- 172.21.15.82:0/3937699852 shutdown_connections 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- 172.21.15.82:0/3937699852 >> 172.21.15.82:0/3937699852 conn(0x7fbde40b11a0 msgr2=0x7fbde40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- 172.21.15.82:0/3937699852 shutdown_connections 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- 172.21.15.82:0/3937699852 wait complete. 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 Processor -- start 2024-04-25T10:02:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- start start 2024-04-25T10:02:30.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 0x7fbde41565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4156af0 0x7fbde40ad380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4142d90 con 0x7fbdd0004d50 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde406e290 con 0x7fbde4156af0 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbdea143640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde4142f10 con 0x7fbde41f33e0 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48742/0 (socket says 172.21.15.82:48742) 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.616+0000 7fbde37fe640 1 -- 172.21.15.82:0/20989672 learned_addr learned my addr 172.21.15.82:0/20989672 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 0x7fbde41565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde3fff640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4156af0 0x7fbde40ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 -- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 msgr2=0x7fbde41565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 0x7fbde41565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 -- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4156af0 msgr2=0x7fbde40ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4156af0 0x7fbde40ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 -- 172.21.15.82:0/20989672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40ac8e0 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 0x7fbde41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde37fe640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e550 tx=0x7fbdd400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd40052e0 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd4018070 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4003820 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde40c4410 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde40c49d0 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4156f00 con 0x7fbdd0004d50 2024-04-25T10:02:30.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbdd40039c0 con 0x7fbdd0004d50 2024-04-25T10:02:30.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 0x7fbdc4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbdd4084c10 con 0x7fbdd0004d50 2024-04-25T10:02:30.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 0x7fbdc4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.620+0000 7fbde2ffd640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 0x7fbdc4069450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbde4156560 tx=0x7fbdcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.624+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbde4156f00 con 0x7fbdd0004d50 2024-04-25T10:02:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.816+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbde404efa0 con 0x7fbdd0004d50 2024-04-25T10:02:30.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.816+0000 7fbddbfff640 1 -- 172.21.15.82:0/20989672 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fbdd40558e0 con 0x7fbdd0004d50 2024-04-25T10:02:30.825 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:30.825 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:30.825 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 msgr2=0x7fbdc4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 0x7fbdc4069450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbde4156560 tx=0x7fbdcc008040 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 msgr2=0x7fbde413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdd400e550 tx=0x7fbdd400ea10 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 shutdown_connections 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbdc4066f90 0x7fbdc4069450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbde41f33e0 0x7fbde41565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbde4156af0 0x7fbde40ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 --2- 172.21.15.82:0/20989672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbdd0004d50 0x7fbde413c010 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 >> 172.21.15.82:0/20989672 conn(0x7fbde40b11a0 msgr2=0x7fbde4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 shutdown_connections 2024-04-25T10:02:30.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.820+0000 7fbdea143640 1 -- 172.21.15.82:0/20989672 wait complete. 2024-04-25T10:02:30.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:30.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 Processor -- start 2024-04-25T10:02:30.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 -- start start 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f366015ecc0 0x7f366006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b2090 con 0x7f366015a700 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36600b43e0 con 0x7f36600b3f50 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3666287640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36600b46c0 con 0x7f366015ecc0 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3664824640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f366015ecc0 0x7f366006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f365ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f366015a700 0x7f36600cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57890/0 (socket says 172.21.15.82:57890) 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f365ffff640 1 -- 172.21.15.82:0/438159427 learned_addr learned my addr 172.21.15.82:0/438159427 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:30.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3664824640 1 -- 172.21.15.82:0/438159427 >> v1:172.21.15.196:6789/0 conn(0x7f36600b3f50 legacy=0x7f36600b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f365effd640 1 -- 172.21.15.82:0/438159427 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476627475 0 0) 0x7f36600b2090 con 0x7f366015a700 2024-04-25T10:02:30.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3664824640 1 -- 172.21.15.82:0/438159427 >> v1:172.21.15.82:6789/0 conn(0x7f366015a700 legacy=0x7f36600cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.956+0000 7f3664824640 1 -- 172.21.15.82:0/438159427 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36600cd6f0 con 0x7f366015ecc0 2024-04-25T10:02:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 --2- 172.21.15.82:0/438159427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366015ecc0 0x7f366006e1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36580097c0 tx=0x7f365801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b057e8e6759a0834 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365effd640 1 -- 172.21.15.82:0/438159427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580b0070 con 0x7f366015ecc0 2024-04-25T10:02:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365effd640 1 -- 172.21.15.82:0/438159427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36580ab600 con 0x7f366015ecc0 2024-04-25T10:02:30.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365effd640 1 -- 172.21.15.82:0/438159427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580ab9d0 con 0x7f366015ecc0 2024-04-25T10:02:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- 172.21.15.82:0/438159427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366015ecc0 msgr2=0x7f366006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 --2- 172.21.15.82:0/438159427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366015ecc0 0x7f366006e1a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f36580097c0 tx=0x7f365801c350 comp rx=0 tx=0).stop 2024-04-25T10:02:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- 172.21.15.82:0/438159427 shutdown_connections 2024-04-25T10:02:30.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 --2- 172.21.15.82:0/438159427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366015ecc0 0x7f366006e1a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- 172.21.15.82:0/438159427 >> 172.21.15.82:0/438159427 conn(0x7f36600b11a0 msgr2=0x7f36600b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- 172.21.15.82:0/438159427 shutdown_connections 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- 172.21.15.82:0/438159427 wait complete. 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 Processor -- start 2024-04-25T10:02:30.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- start start 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 0x7f3660155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 0x7f3660155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600cd310 con 0x7f366014de80 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36600ce600 con 0x7f366015a700 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3666287640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36600cd490 con 0x7f366015ecc0 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48760/0 (socket says 172.21.15.82:48760) 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 -- 172.21.15.82:0/3124745305 learned_addr learned my addr 172.21.15.82:0/3124745305 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365ffff640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 0x7f3660155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365f7fe640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 0x7f3660155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 -- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 msgr2=0x7f3660155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 0x7f3660155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 -- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 msgr2=0x7f3660155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 0x7f3660155fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f3664824640 1 -- 172.21.15.82:0/3124745305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36601565a0 con 0x7f366014de80 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365f7fe640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 0x7f3660155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.960+0000 7f365ffff640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 0x7f3660155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f3664824640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3658018f60 tx=0x7f365801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580b0050 con 0x7f366014de80 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36580bc070 con 0x7f366014de80 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3660150b20 con 0x7f366014de80 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580b7670 con 0x7f366014de80 2024-04-25T10:02:30.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36601510e0 con 0x7f366014de80 2024-04-25T10:02:30.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366004efa0 con 0x7f366014de80 2024-04-25T10:02:30.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36580b7810 con 0x7f366014de80 2024-04-25T10:02:30.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 0x7f3640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f365811f950 con 0x7f366014de80 2024-04-25T10:02:30.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365ffff640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 0x7f3640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.964+0000 7f365ffff640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 0x7f3640069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36500012f0 tx=0x7f3650009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:30.968+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3658016090 con 0x7f366014de80 2024-04-25T10:02:31.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.124+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f366006f260 con 0x7f366014de80 2024-04-25T10:02:31.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.124+0000 7f365cff9640 1 -- 172.21.15.82:0/3124745305 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f36580f06b0 con 0x7f366014de80 2024-04-25T10:02:31.133 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:31.133 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 msgr2=0x7f3640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 0x7f3640069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f36500012f0 tx=0x7f3650009040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 msgr2=0x7f3660153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3658018f60 tx=0x7f365801ab90 comp rx=0 tx=0).stop 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 shutdown_connections 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3640066f60 0x7f3640069420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366015ecc0 0x7f3660155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f366014de80 0x7f3660153660 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 --2- 172.21.15.82:0/3124745305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f366015a700 0x7f3660155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 >> 172.21.15.82:0/3124745305 conn(0x7f36600b11a0 msgr2=0x7f36600af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 shutdown_connections 2024-04-25T10:02:31.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.128+0000 7f3666287640 1 -- 172.21.15.82:0/3124745305 wait complete. 2024-04-25T10:02:31.151 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:31.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:31.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 Processor -- start 2024-04-25T10:02:31.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 -- start start 2024-04-25T10:02:31.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ad815a700 0x7f8ad815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ad80b24a0 0x7f8ad80b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad80b2090 con 0x7f8ad80b3f50 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad815f200 con 0x7f8ad80b24a0 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8add32e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad80b43e0 con 0x7f8ad815a700 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ad80b24a0 0x7f8ad80b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ad80b24a0 0x7f8ad80b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48766/0 (socket says 172.21.15.82:48766) 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 -- 172.21.15.82:0/3077117593 learned_addr learned my addr 172.21.15.82:0/3077117593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad6ffd640 1 --2- 172.21.15.82:0/3077117593 >> v2:172.21.15.196:3300/0 conn(0x7f8ad815a700 0x7f8ad815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 -- 172.21.15.82:0/3077117593 >> v2:172.21.15.196:3300/0 conn(0x7f8ad815a700 msgr2=0x7f8ad815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 --2- 172.21.15.82:0/3077117593 >> v2:172.21.15.196:3300/0 conn(0x7f8ad815a700 0x7f8ad815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 -- 172.21.15.82:0/3077117593 >> v1:172.21.15.196:6789/0 conn(0x7f8ad80b3f50 legacy=0x7f8ad80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.252+0000 7f8ad77fe640 1 -- 172.21.15.82:0/3077117593 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad80b46c0 con 0x7f8ad80b24a0 2024-04-25T10:02:31.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- 172.21.15.82:0/3077117593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad80b24a0 0x7f8ad80b28f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ad806c980 tx=0x7f8ad001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612ee6ce17abcd7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad5ffb640 1 -- 172.21.15.82:0/3077117593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0070 con 0x7f8ad80b24a0 2024-04-25T10:02:31.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad5ffb640 1 -- 172.21.15.82:0/3077117593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00ab600 con 0x7f8ad80b24a0 2024-04-25T10:02:31.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad5ffb640 1 -- 172.21.15.82:0/3077117593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00ab9d0 con 0x7f8ad80b24a0 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- 172.21.15.82:0/3077117593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad80b24a0 msgr2=0x7f8ad80b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- 172.21.15.82:0/3077117593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad80b24a0 0x7f8ad80b28f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8ad806c980 tx=0x7f8ad001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- 172.21.15.82:0/3077117593 shutdown_connections 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- 172.21.15.82:0/3077117593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad80b24a0 0x7f8ad80b28f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- 172.21.15.82:0/3077117593 >> v2:172.21.15.196:3300/0 conn(0x7f8ad815a700 0x7f8ad815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- 172.21.15.82:0/3077117593 >> 172.21.15.82:0/3077117593 conn(0x7f8ad80b11a0 msgr2=0x7f8ad80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- 172.21.15.82:0/3077117593 shutdown_connections 2024-04-25T10:02:31.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- 172.21.15.82:0/3077117593 wait complete. 2024-04-25T10:02:31.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 Processor -- start 2024-04-25T10:02:31.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- start start 2024-04-25T10:02:31.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ad80b24a0 0x7f8ad8159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad815a700 0x7f8ad8168090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80cd310 con 0x7f8ad8145e20 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80ce600 con 0x7f8ad815a700 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8add32e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad80cd490 con 0x7f8ad80b24a0 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48774/0 (socket says 172.21.15.82:48774) 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 -- 172.21.15.82:0/1486139305 learned_addr learned my addr 172.21.15.82:0/1486139305 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad67fc640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad815a700 0x7f8ad8168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad6ffd640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ad80b24a0 0x7f8ad8159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 -- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ad80b24a0 msgr2=0x7f8ad8159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ad80b24a0 0x7f8ad8159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 -- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad815a700 msgr2=0x7f8ad8168090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad815a700 0x7f8ad8168090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 -- 172.21.15.82:0/1486139305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad8166c30 con 0x7f8ad8145e20 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8ad77fe640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00ae040 tx=0x7f8ad0019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b0040 con 0x7f8ad8145e20 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00b9070 con 0x7f8ad8145e20 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.256+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b4640 con 0x7f8ad8145e20 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad81666b0 con 0x7f8ad8145e20 2024-04-25T10:02:31.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad81588f0 con 0x7f8ad8145e20 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad8158720 con 0x7f8ad8145e20 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ad00b47e0 con 0x7f8ad8145e20 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8acf7fe640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 0x7f8ab8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8ad6ffd640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 0x7f8ab8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8ad011ee20 con 0x7f8ad8145e20 2024-04-25T10:02:31.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.260+0000 7f8ad6ffd640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 0x7f8ab8069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0007a60 tx=0x7f8ac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.264+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ad8166c30 con 0x7f8ad8145e20 2024-04-25T10:02:31.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.412+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ad804efa0 con 0x7f8ab8066f60 2024-04-25T10:02:31.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.420+0000 7f8acf7fe640 1 -- 172.21.15.82:0/1486139305 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f8ad804efa0 con 0x7f8ab8066f60 2024-04-25T10:02:31.431 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 msgr2=0x7f8ab8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 0x7f8ab8069420 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0007a60 tx=0x7f8ac0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 msgr2=0x7f8ad806da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00ae040 tx=0x7f8ad0019140 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 shutdown_connections 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab8066f60 0x7f8ab8069420 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ad8145e20 0x7f8ad806da10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ad815a700 0x7f8ad8168090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 --2- 172.21.15.82:0/1486139305 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ad80b24a0 0x7f8ad8159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 >> 172.21.15.82:0/1486139305 conn(0x7f8ad80b11a0 msgr2=0x7f8ad80af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 shutdown_connections 2024-04-25T10:02:31.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.424+0000 7f8add32e640 1 -- 172.21.15.82:0/1486139305 wait complete. 2024-04-25T10:02:31.447 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T10:02:31.266708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223632,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751088,"statfs":{"total":773094113280,"available":772865114112,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003933"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462685+0000","last_change":"2024-04-25T10:02:06.377944+0000","last_active":"2024-04-25T10:02:13.462685+0000","last_peered":"2024-04-25T10:02:13.462685+0000","last_clean":"2024-04-25T10:02:13.462685+0000","last_became_active":"2024-04-25T10:02:06.377162+0000","last_became_peered":"2024-04-25T10:02:06.377162+0000","last_unstale":"2024-04-25T10:02:13.462685+0000","last_undegraded":"2024-04-25T10:02:13.462685+0000","last_fullsized":"2024-04-25T10:02:13.462685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406752+0000","last_change":"2024-04-25T10:02:07.375159+0000","last_active":"2024-04-25T10:02:12.406752+0000","last_peered":"2024-04-25T10:02:12.406752+0000","last_clean":"2024-04-25T10:02:12.406752+0000","last_became_active":"2024-04-25T10:02:07.374938+0000","last_became_peered":"2024-04-25T10:02:07.374938+0000","last_unstale":"2024-04-25T10:02:12.406752+0000","last_undegraded":"2024-04-25T10:02:12.406752+0000","last_fullsized":"2024-04-25T10:02:12.406752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404043+0000","last_change":"2024-04-25T10:02:11.459079+0000","last_active":"2024-04-25T10:02:12.404043+0000","last_peered":"2024-04-25T10:02:12.404043+0000","last_clean":"2024-04-25T10:02:12.404043+0000","last_became_active":"2024-04-25T10:02:11.458605+0000","last_became_peered":"2024-04-25T10:02:11.458605+0000","last_unstale":"2024-04-25T10:02:12.404043+0000","last_undegraded":"2024-04-25T10:02:12.404043+0000","last_fullsized":"2024-04-25T10:02:12.404043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407158+0000","last_change":"2024-04-25T10:02:06.377863+0000","last_active":"2024-04-25T10:02:12.407158+0000","last_peered":"2024-04-25T10:02:12.407158+0000","last_clean":"2024-04-25T10:02:12.407158+0000","last_became_active":"2024-04-25T10:02:06.377260+0000","last_became_peered":"2024-04-25T10:02:06.377260+0000","last_unstale":"2024-04-25T10:02:12.407158+0000","last_undegraded":"2024-04-25T10:02:12.407158+0000","last_fullsized":"2024-04-25T10:02:12.407158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462420+0000","last_change":"2024-04-25T10:02:07.371882+0000","last_active":"2024-04-25T10:02:13.462420+0000","last_peered":"2024-04-25T10:02:13.462420+0000","last_clean":"2024-04-25T10:02:13.462420+0000","last_became_active":"2024-04-25T10:02:07.371244+0000","last_became_peered":"2024-04-25T10:02:07.371244+0000","last_unstale":"2024-04-25T10:02:13.462420+0000","last_undegraded":"2024-04-25T10:02:13.462420+0000","last_fullsized":"2024-04-25T10:02:13.462420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405866+0000","last_change":"2024-04-25T10:02:11.435335+0000","last_active":"2024-04-25T10:02:12.405866+0000","last_peered":"2024-04-25T10:02:12.405866+0000","last_clean":"2024-04-25T10:02:12.405866+0000","last_became_active":"2024-04-25T10:02:11.434197+0000","last_became_peered":"2024-04-25T10:02:11.434197+0000","last_unstale":"2024-04-25T10:02:12.405866+0000","last_undegraded":"2024-04-25T10:02:12.405866+0000","last_fullsized":"2024-04-25T10:02:12.405866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:43.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465851+0000","last_change":"2024-04-25T10:02:06.374878+0000","last_active":"2024-04-25T10:02:13.465851+0000","last_peered":"2024-04-25T10:02:13.465851+0000","last_clean":"2024-04-25T10:02:13.465851+0000","last_became_active":"2024-04-25T10:02:06.374210+0000","last_became_peered":"2024-04-25T10:02:06.374210+0000","last_unstale":"2024-04-25T10:02:13.465851+0000","last_undegraded":"2024-04-25T10:02:13.465851+0000","last_fullsized":"2024-04-25T10:02:13.465851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465104+0000","last_change":"2024-04-25T10:02:07.382381+0000","last_active":"2024-04-25T10:02:13.465104+0000","last_peered":"2024-04-25T10:02:13.465104+0000","last_clean":"2024-04-25T10:02:13.465104+0000","last_became_active":"2024-04-25T10:02:07.381718+0000","last_became_peered":"2024-04-25T10:02:07.381718+0000","last_unstale":"2024-04-25T10:02:13.465104+0000","last_undegraded":"2024-04-25T10:02:13.465104+0000","last_fullsized":"2024-04-25T10:02:13.465104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407612+0000","last_change":"2024-04-25T10:02:11.442974+0000","last_active":"2024-04-25T10:02:12.407612+0000","last_peered":"2024-04-25T10:02:12.407612+0000","last_clean":"2024-04-25T10:02:12.407612+0000","last_became_active":"2024-04-25T10:02:11.441674+0000","last_became_peered":"2024-04-25T10:02:11.441674+0000","last_unstale":"2024-04-25T10:02:12.407612+0000","last_undegraded":"2024-04-25T10:02:12.407612+0000","last_fullsized":"2024-04-25T10:02:12.407612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464230+0000","last_change":"2024-04-25T10:02:06.388913+0000","last_active":"2024-04-25T10:02:13.464230+0000","last_peered":"2024-04-25T10:02:13.464230+0000","last_clean":"2024-04-25T10:02:13.464230+0000","last_became_active":"2024-04-25T10:02:06.388536+0000","last_became_peered":"2024-04-25T10:02:06.388536+0000","last_unstale":"2024-04-25T10:02:13.464230+0000","last_undegraded":"2024-04-25T10:02:13.464230+0000","last_fullsized":"2024-04-25T10:02:13.464230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:41.104882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465595+0000","last_change":"2024-04-25T10:02:07.382060+0000","last_active":"2024-04-25T10:02:13.465595+0000","last_peered":"2024-04-25T10:02:13.465595+0000","last_clean":"2024-04-25T10:02:13.465595+0000","last_became_active":"2024-04-25T10:02:07.381779+0000","last_became_peered":"2024-04-25T10:02:07.381779+0000","last_unstale":"2024-04-25T10:02:13.465595+0000","last_undegraded":"2024-04-25T10:02:13.465595+0000","last_fullsized":"2024-04-25T10:02:13.465595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406133+0000","last_change":"2024-04-25T10:02:11.430607+0000","last_active":"2024-04-25T10:02:12.406133+0000","last_peered":"2024-04-25T10:02:12.406133+0000","last_clean":"2024-04-25T10:02:12.406133+0000","last_became_active":"2024-04-25T10:02:11.429281+0000","last_became_peered":"2024-04-25T10:02:11.429281+0000","last_unstale":"2024-04-25T10:02:12.406133+0000","last_undegraded":"2024-04-25T10:02:12.406133+0000","last_fullsized":"2024-04-25T10:02:12.406133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:06.213948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405179+0000","last_change":"2024-04-25T10:02:07.365923+0000","last_active":"2024-04-25T10:02:12.405179+0000","last_peered":"2024-04-25T10:02:12.405179+0000","last_clean":"2024-04-25T10:02:12.405179+0000","last_became_active":"2024-04-25T10:02:07.365345+0000","last_became_peered":"2024-04-25T10:02:07.365345+0000","last_unstale":"2024-04-25T10:02:12.405179+0000","last_undegraded":"2024-04-25T10:02:12.405179+0000","last_fullsized":"2024-04-25T10:02:12.405179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465328+0000","last_change":"2024-04-25T10:02:06.373663+0000","last_active":"2024-04-25T10:02:13.465328+0000","last_peered":"2024-04-25T10:02:13.465328+0000","last_clean":"2024-04-25T10:02:13.465328+0000","last_became_active":"2024-04-25T10:02:06.372972+0000","last_became_peered":"2024-04-25T10:02:06.372972+0000","last_unstale":"2024-04-25T10:02:13.465328+0000","last_undegraded":"2024-04-25T10:02:13.465328+0000","last_fullsized":"2024-04-25T10:02:13.465328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408366+0000","last_change":"2024-04-25T10:02:11.432916+0000","last_active":"2024-04-25T10:02:12.408366+0000","last_peered":"2024-04-25T10:02:12.408366+0000","last_clean":"2024-04-25T10:02:12.408366+0000","last_became_active":"2024-04-25T10:02:11.431758+0000","last_became_peered":"2024-04-25T10:02:11.431758+0000","last_unstale":"2024-04-25T10:02:12.408366+0000","last_undegraded":"2024-04-25T10:02:12.408366+0000","last_fullsized":"2024-04-25T10:02:12.408366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403824+0000","last_change":"2024-04-25T10:02:07.371074+0000","last_active":"2024-04-25T10:02:12.403824+0000","last_peered":"2024-04-25T10:02:12.403824+0000","last_clean":"2024-04-25T10:02:12.403824+0000","last_became_active":"2024-04-25T10:02:07.370770+0000","last_became_peered":"2024-04-25T10:02:07.370770+0000","last_unstale":"2024-04-25T10:02:12.403824+0000","last_undegraded":"2024-04-25T10:02:12.403824+0000","last_fullsized":"2024-04-25T10:02:12.403824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462663+0000","last_change":"2024-04-25T10:02:06.369892+0000","last_active":"2024-04-25T10:02:13.462663+0000","last_peered":"2024-04-25T10:02:13.462663+0000","last_clean":"2024-04-25T10:02:13.462663+0000","last_became_active":"2024-04-25T10:02:06.369183+0000","last_became_peered":"2024-04-25T10:02:06.369183+0000","last_unstale":"2024-04-25T10:02:13.462663+0000","last_undegraded":"2024-04-25T10:02:13.462663+0000","last_fullsized":"2024-04-25T10:02:13.462663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406907+0000","last_change":"2024-04-25T10:02:11.442628+0000","last_active":"2024-04-25T10:02:12.406907+0000","last_peered":"2024-04-25T10:02:12.406907+0000","last_clean":"2024-04-25T10:02:12.406907+0000","last_became_active":"2024-04-25T10:02:11.441578+0000","last_became_peered":"2024-04-25T10:02:11.441578+0000","last_unstale":"2024-04-25T10:02:12.406907+0000","last_undegraded":"2024-04-25T10:02:12.406907+0000","last_fullsized":"2024-04-25T10:02:12.406907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:00.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403861+0000","last_change":"2024-04-25T10:02:07.361735+0000","last_active":"2024-04-25T10:02:12.403861+0000","last_peered":"2024-04-25T10:02:12.403861+0000","last_clean":"2024-04-25T10:02:12.403861+0000","last_became_active":"2024-04-25T10:02:07.361435+0000","last_became_peered":"2024-04-25T10:02:07.361435+0000","last_unstale":"2024-04-25T10:02:12.403861+0000","last_undegraded":"2024-04-25T10:02:12.403861+0000","last_fullsized":"2024-04-25T10:02:12.403861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461678+0000","last_change":"2024-04-25T10:02:06.369621+0000","last_active":"2024-04-25T10:02:13.461678+0000","last_peered":"2024-04-25T10:02:13.461678+0000","last_clean":"2024-04-25T10:02:13.461678+0000","last_became_active":"2024-04-25T10:02:06.368883+0000","last_became_peered":"2024-04-25T10:02:06.368883+0000","last_unstale":"2024-04-25T10:02:13.461678+0000","last_undegraded":"2024-04-25T10:02:13.461678+0000","last_fullsized":"2024-04-25T10:02:13.461678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462909+0000","last_change":"2024-04-25T10:02:11.462237+0000","last_active":"2024-04-25T10:02:13.462909+0000","last_peered":"2024-04-25T10:02:13.462909+0000","last_clean":"2024-04-25T10:02:13.462909+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.462909+0000","last_undegraded":"2024-04-25T10:02:13.462909+0000","last_fullsized":"2024-04-25T10:02:13.462909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406991+0000","last_change":"2024-04-25T10:02:07.357629+0000","last_active":"2024-04-25T10:02:12.406991+0000","last_peered":"2024-04-25T10:02:12.406991+0000","last_clean":"2024-04-25T10:02:12.406991+0000","last_became_active":"2024-04-25T10:02:07.357353+0000","last_became_peered":"2024-04-25T10:02:07.357353+0000","last_unstale":"2024-04-25T10:02:12.406991+0000","last_undegraded":"2024-04-25T10:02:12.406991+0000","last_fullsized":"2024-04-25T10:02:12.406991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:17.663068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464640+0000","last_change":"2024-04-25T10:02:06.384551+0000","last_active":"2024-04-25T10:02:13.464640+0000","last_peered":"2024-04-25T10:02:13.464640+0000","last_clean":"2024-04-25T10:02:13.464640+0000","last_became_active":"2024-04-25T10:02:06.384131+0000","last_became_peered":"2024-04-25T10:02:06.384131+0000","last_unstale":"2024-04-25T10:02:13.464640+0000","last_undegraded":"2024-04-25T10:02:13.464640+0000","last_fullsized":"2024-04-25T10:02:13.464640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408508+0000","last_change":"2024-04-25T10:02:11.443507+0000","last_active":"2024-04-25T10:02:12.408508+0000","last_peered":"2024-04-25T10:02:12.408508+0000","last_clean":"2024-04-25T10:02:12.408508+0000","last_became_active":"2024-04-25T10:02:11.440496+0000","last_became_peered":"2024-04-25T10:02:11.440496+0000","last_unstale":"2024-04-25T10:02:12.408508+0000","last_undegraded":"2024-04-25T10:02:12.408508+0000","last_fullsized":"2024-04-25T10:02:12.408508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462520+0000","last_change":"2024-04-25T10:02:07.367892+0000","last_active":"2024-04-25T10:02:13.462520+0000","last_peered":"2024-04-25T10:02:13.462520+0000","last_clean":"2024-04-25T10:02:13.462520+0000","last_became_active":"2024-04-25T10:02:07.367404+0000","last_became_peered":"2024-04-25T10:02:07.367404+0000","last_unstale":"2024-04-25T10:02:13.462520+0000","last_undegraded":"2024-04-25T10:02:13.462520+0000","last_fullsized":"2024-04-25T10:02:13.462520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406984+0000","last_change":"2024-04-25T10:02:06.371009+0000","last_active":"2024-04-25T10:02:12.406984+0000","last_peered":"2024-04-25T10:02:12.406984+0000","last_clean":"2024-04-25T10:02:12.406984+0000","last_became_active":"2024-04-25T10:02:06.369415+0000","last_became_peered":"2024-04-25T10:02:06.369415+0000","last_unstale":"2024-04-25T10:02:12.406984+0000","last_undegraded":"2024-04-25T10:02:12.406984+0000","last_fullsized":"2024-04-25T10:02:12.406984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406594+0000","last_change":"2024-04-25T10:02:11.460234+0000","last_active":"2024-04-25T10:02:12.406594+0000","last_peered":"2024-04-25T10:02:12.406594+0000","last_clean":"2024-04-25T10:02:12.406594+0000","last_became_active":"2024-04-25T10:02:11.459268+0000","last_became_peered":"2024-04-25T10:02:11.459268+0000","last_unstale":"2024-04-25T10:02:12.406594+0000","last_undegraded":"2024-04-25T10:02:12.406594+0000","last_fullsized":"2024-04-25T10:02:12.406594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:35.022355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406871+0000","last_change":"2024-04-25T10:02:07.385204+0000","last_active":"2024-04-25T10:02:12.406871+0000","last_peered":"2024-04-25T10:02:12.406871+0000","last_clean":"2024-04-25T10:02:12.406871+0000","last_became_active":"2024-04-25T10:02:07.384798+0000","last_became_peered":"2024-04-25T10:02:07.384798+0000","last_unstale":"2024-04-25T10:02:12.406871+0000","last_undegraded":"2024-04-25T10:02:12.406871+0000","last_fullsized":"2024-04-25T10:02:12.406871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462229+0000","last_change":"2024-04-25T10:02:06.363593+0000","last_active":"2024-04-25T10:02:13.462229+0000","last_peered":"2024-04-25T10:02:13.462229+0000","last_clean":"2024-04-25T10:02:13.462229+0000","last_became_active":"2024-04-25T10:02:06.363127+0000","last_became_peered":"2024-04-25T10:02:06.363127+0000","last_unstale":"2024-04-25T10:02:13.462229+0000","last_undegraded":"2024-04-25T10:02:13.462229+0000","last_fullsized":"2024-04-25T10:02:13.462229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403605+0000","last_change":"2024-04-25T10:02:11.453752+0000","last_active":"2024-04-25T10:02:12.403605+0000","last_peered":"2024-04-25T10:02:12.403605+0000","last_clean":"2024-04-25T10:02:12.403605+0000","last_became_active":"2024-04-25T10:02:11.453254+0000","last_became_peered":"2024-04-25T10:02:11.453254+0000","last_unstale":"2024-04-25T10:02:12.403605+0000","last_undegraded":"2024-04-25T10:02:12.403605+0000","last_fullsized":"2024-04-25T10:02:12.403605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:04.566083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462068+0000","last_change":"2024-04-25T10:02:07.356096+0000","last_active":"2024-04-25T10:02:13.462068+0000","last_peered":"2024-04-25T10:02:13.462068+0000","last_clean":"2024-04-25T10:02:13.462068+0000","last_became_active":"2024-04-25T10:02:07.355433+0000","last_became_peered":"2024-04-25T10:02:07.355433+0000","last_unstale":"2024-04-25T10:02:13.462068+0000","last_undegraded":"2024-04-25T10:02:13.462068+0000","last_fullsized":"2024-04-25T10:02:13.462068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466154+0000","last_change":"2024-04-25T10:02:06.385234+0000","last_active":"2024-04-25T10:02:13.466154+0000","last_peered":"2024-04-25T10:02:13.466154+0000","last_clean":"2024-04-25T10:02:13.466154+0000","last_became_active":"2024-04-25T10:02:06.384494+0000","last_became_peered":"2024-04-25T10:02:06.384494+0000","last_unstale":"2024-04-25T10:02:13.466154+0000","last_undegraded":"2024-04-25T10:02:13.466154+0000","last_fullsized":"2024-04-25T10:02:13.466154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462761+0000","last_change":"2024-04-25T10:02:11.459695+0000","last_active":"2024-04-25T10:02:13.462761+0000","last_peered":"2024-04-25T10:02:13.462761+0000","last_clean":"2024-04-25T10:02:13.462761+0000","last_became_active":"2024-04-25T10:02:11.459247+0000","last_became_peered":"2024-04-25T10:02:11.459247+0000","last_unstale":"2024-04-25T10:02:13.462761+0000","last_undegraded":"2024-04-25T10:02:13.462761+0000","last_fullsized":"2024-04-25T10:02:13.462761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465853+0000","last_change":"2024-04-25T10:02:07.374402+0000","last_active":"2024-04-25T10:02:13.465853+0000","last_peered":"2024-04-25T10:02:13.465853+0000","last_clean":"2024-04-25T10:02:13.465853+0000","last_became_active":"2024-04-25T10:02:07.374110+0000","last_became_peered":"2024-04-25T10:02:07.374110+0000","last_unstale":"2024-04-25T10:02:13.465853+0000","last_undegraded":"2024-04-25T10:02:13.465853+0000","last_fullsized":"2024-04-25T10:02:13.465853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462182+0000","last_change":"2024-04-25T10:02:06.368581+0000","last_active":"2024-04-25T10:02:13.462182+0000","last_peered":"2024-04-25T10:02:13.462182+0000","last_clean":"2024-04-25T10:02:13.462182+0000","last_became_active":"2024-04-25T10:02:06.368085+0000","last_became_peered":"2024-04-25T10:02:06.368085+0000","last_unstale":"2024-04-25T10:02:13.462182+0000","last_undegraded":"2024-04-25T10:02:13.462182+0000","last_fullsized":"2024-04-25T10:02:13.462182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461807+0000","last_change":"2024-04-25T10:02:11.435403+0000","last_active":"2024-04-25T10:02:13.461807+0000","last_peered":"2024-04-25T10:02:13.461807+0000","last_clean":"2024-04-25T10:02:13.461807+0000","last_became_active":"2024-04-25T10:02:11.435010+0000","last_became_peered":"2024-04-25T10:02:11.435010+0000","last_unstale":"2024-04-25T10:02:13.461807+0000","last_undegraded":"2024-04-25T10:02:13.461807+0000","last_fullsized":"2024-04-25T10:02:13.461807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407792+0000","last_change":"2024-04-25T10:02:07.378715+0000","last_active":"2024-04-25T10:02:12.407792+0000","last_peered":"2024-04-25T10:02:12.407792+0000","last_clean":"2024-04-25T10:02:12.407792+0000","last_became_active":"2024-04-25T10:02:07.378297+0000","last_became_peered":"2024-04-25T10:02:07.378297+0000","last_unstale":"2024-04-25T10:02:12.407792+0000","last_undegraded":"2024-04-25T10:02:12.407792+0000","last_fullsized":"2024-04-25T10:02:12.407792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465696+0000","last_change":"2024-04-25T10:02:06.381259+0000","last_active":"2024-04-25T10:02:13.465696+0000","last_peered":"2024-04-25T10:02:13.465696+0000","last_clean":"2024-04-25T10:02:13.465696+0000","last_became_active":"2024-04-25T10:02:06.377864+0000","last_became_peered":"2024-04-25T10:02:06.377864+0000","last_unstale":"2024-04-25T10:02:13.465696+0000","last_undegraded":"2024-04-25T10:02:13.465696+0000","last_fullsized":"2024-04-25T10:02:13.465696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462019+0000","last_change":"2024-04-25T10:02:11.435443+0000","last_active":"2024-04-25T10:02:13.462019+0000","last_peered":"2024-04-25T10:02:13.462019+0000","last_clean":"2024-04-25T10:02:13.462019+0000","last_became_active":"2024-04-25T10:02:11.434691+0000","last_became_peered":"2024-04-25T10:02:11.434691+0000","last_unstale":"2024-04-25T10:02:13.462019+0000","last_undegraded":"2024-04-25T10:02:13.462019+0000","last_fullsized":"2024-04-25T10:02:13.462019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462938+0000","last_change":"2024-04-25T10:02:07.360180+0000","last_active":"2024-04-25T10:02:13.462938+0000","last_peered":"2024-04-25T10:02:13.462938+0000","last_clean":"2024-04-25T10:02:13.462938+0000","last_became_active":"2024-04-25T10:02:07.358876+0000","last_became_peered":"2024-04-25T10:02:07.358876+0000","last_unstale":"2024-04-25T10:02:13.462938+0000","last_undegraded":"2024-04-25T10:02:13.462938+0000","last_fullsized":"2024-04-25T10:02:13.462938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462463+0000","last_change":"2024-04-25T10:02:06.376414+0000","last_active":"2024-04-25T10:02:13.462463+0000","last_peered":"2024-04-25T10:02:13.462463+0000","last_clean":"2024-04-25T10:02:13.462463+0000","last_became_active":"2024-04-25T10:02:06.375943+0000","last_became_peered":"2024-04-25T10:02:06.375943+0000","last_unstale":"2024-04-25T10:02:13.462463+0000","last_undegraded":"2024-04-25T10:02:13.462463+0000","last_fullsized":"2024-04-25T10:02:13.462463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463033+0000","last_change":"2024-04-25T10:02:11.438382+0000","last_active":"2024-04-25T10:02:13.463033+0000","last_peered":"2024-04-25T10:02:13.463033+0000","last_clean":"2024-04-25T10:02:13.463033+0000","last_became_active":"2024-04-25T10:02:11.436135+0000","last_became_peered":"2024-04-25T10:02:11.436135+0000","last_unstale":"2024-04-25T10:02:13.463033+0000","last_undegraded":"2024-04-25T10:02:13.463033+0000","last_fullsized":"2024-04-25T10:02:13.463033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.460832+0000","last_change":"2024-04-25T10:02:07.366816+0000","last_active":"2024-04-25T10:02:13.460832+0000","last_peered":"2024-04-25T10:02:13.460832+0000","last_clean":"2024-04-25T10:02:13.460832+0000","last_became_active":"2024-04-25T10:02:07.366193+0000","last_became_peered":"2024-04-25T10:02:07.366193+0000","last_unstale":"2024-04-25T10:02:13.460832+0000","last_undegraded":"2024-04-25T10:02:13.460832+0000","last_fullsized":"2024-04-25T10:02:13.460832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463419+0000","last_change":"2024-04-25T10:02:06.376427+0000","last_active":"2024-04-25T10:02:13.463419+0000","last_peered":"2024-04-25T10:02:13.463419+0000","last_clean":"2024-04-25T10:02:13.463419+0000","last_became_active":"2024-04-25T10:02:06.375955+0000","last_became_peered":"2024-04-25T10:02:06.375955+0000","last_unstale":"2024-04-25T10:02:13.463419+0000","last_undegraded":"2024-04-25T10:02:13.463419+0000","last_fullsized":"2024-04-25T10:02:13.463419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462868+0000","last_change":"2024-04-25T10:02:11.458524+0000","last_active":"2024-04-25T10:02:13.462868+0000","last_peered":"2024-04-25T10:02:13.462868+0000","last_clean":"2024-04-25T10:02:13.462868+0000","last_became_active":"2024-04-25T10:02:11.458105+0000","last_became_peered":"2024-04-25T10:02:11.458105+0000","last_unstale":"2024-04-25T10:02:13.462868+0000","last_undegraded":"2024-04-25T10:02:13.462868+0000","last_fullsized":"2024-04-25T10:02:13.462868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403806+0000","last_change":"2024-04-25T10:02:07.369247+0000","last_active":"2024-04-25T10:02:12.403806+0000","last_peered":"2024-04-25T10:02:12.403806+0000","last_clean":"2024-04-25T10:02:12.403806+0000","last_became_active":"2024-04-25T10:02:07.368616+0000","last_became_peered":"2024-04-25T10:02:07.368616+0000","last_unstale":"2024-04-25T10:02:12.403806+0000","last_undegraded":"2024-04-25T10:02:12.403806+0000","last_fullsized":"2024-04-25T10:02:12.403806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466462+0000","last_change":"2024-04-25T10:02:06.385925+0000","last_active":"2024-04-25T10:02:13.466462+0000","last_peered":"2024-04-25T10:02:13.466462+0000","last_clean":"2024-04-25T10:02:13.466462+0000","last_became_active":"2024-04-25T10:02:06.382216+0000","last_became_peered":"2024-04-25T10:02:06.382216+0000","last_unstale":"2024-04-25T10:02:13.466462+0000","last_undegraded":"2024-04-25T10:02:13.466462+0000","last_fullsized":"2024-04-25T10:02:13.466462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463713+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:02:13.463713+0000","last_peered":"2024-04-25T10:02:13.463713+0000","last_clean":"2024-04-25T10:02:13.463713+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:02:13.463713+0000","last_undegraded":"2024-04-25T10:02:13.463713+0000","last_fullsized":"2024-04-25T10:02:13.463713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406156+0000","last_change":"2024-04-25T10:02:11.460389+0000","last_active":"2024-04-25T10:02:12.406156+0000","last_peered":"2024-04-25T10:02:12.406156+0000","last_clean":"2024-04-25T10:02:12.406156+0000","last_became_active":"2024-04-25T10:02:11.459440+0000","last_became_peered":"2024-04-25T10:02:11.459440+0000","last_unstale":"2024-04-25T10:02:12.406156+0000","last_undegraded":"2024-04-25T10:02:12.406156+0000","last_fullsized":"2024-04-25T10:02:12.406156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465341+0000","last_change":"2024-04-25T10:02:07.388181+0000","last_active":"2024-04-25T10:02:13.465341+0000","last_peered":"2024-04-25T10:02:13.465341+0000","last_clean":"2024-04-25T10:02:13.465341+0000","last_became_active":"2024-04-25T10:02:07.385106+0000","last_became_peered":"2024-04-25T10:02:07.385106+0000","last_unstale":"2024-04-25T10:02:13.465341+0000","last_undegraded":"2024-04-25T10:02:13.465341+0000","last_fullsized":"2024-04-25T10:02:13.465341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463240+0000","last_change":"2024-04-25T10:02:06.386894+0000","last_active":"2024-04-25T10:02:13.463240+0000","last_peered":"2024-04-25T10:02:13.463240+0000","last_clean":"2024-04-25T10:02:13.463240+0000","last_became_active":"2024-04-25T10:02:06.386546+0000","last_became_peered":"2024-04-25T10:02:06.386546+0000","last_unstale":"2024-04-25T10:02:13.463240+0000","last_undegraded":"2024-04-25T10:02:13.463240+0000","last_fullsized":"2024-04-25T10:02:13.463240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405476+0000","last_change":"2024-04-25T10:02:11.455613+0000","last_active":"2024-04-25T10:02:12.405476+0000","last_peered":"2024-04-25T10:02:12.405476+0000","last_clean":"2024-04-25T10:02:12.405476+0000","last_became_active":"2024-04-25T10:02:11.454401+0000","last_became_peered":"2024-04-25T10:02:11.454401+0000","last_unstale":"2024-04-25T10:02:12.405476+0000","last_undegraded":"2024-04-25T10:02:12.405476+0000","last_fullsized":"2024-04-25T10:02:12.405476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465364+0000","last_change":"2024-04-25T10:02:07.382626+0000","last_active":"2024-04-25T10:02:13.465364+0000","last_peered":"2024-04-25T10:02:13.465364+0000","last_clean":"2024-04-25T10:02:13.465364+0000","last_became_active":"2024-04-25T10:02:07.382110+0000","last_became_peered":"2024-04-25T10:02:07.382110+0000","last_unstale":"2024-04-25T10:02:13.465364+0000","last_undegraded":"2024-04-25T10:02:13.465364+0000","last_fullsized":"2024-04-25T10:02:13.465364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461856+0000","last_change":"2024-04-25T10:02:06.361762+0000","last_active":"2024-04-25T10:02:13.461856+0000","last_peered":"2024-04-25T10:02:13.461856+0000","last_clean":"2024-04-25T10:02:13.461856+0000","last_became_active":"2024-04-25T10:02:06.360531+0000","last_became_peered":"2024-04-25T10:02:06.360531+0000","last_unstale":"2024-04-25T10:02:13.461856+0000","last_undegraded":"2024-04-25T10:02:13.461856+0000","last_fullsized":"2024-04-25T10:02:13.461856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463028+0000","last_change":"2024-04-25T10:02:11.435880+0000","last_active":"2024-04-25T10:02:13.463028+0000","last_peered":"2024-04-25T10:02:13.463028+0000","last_clean":"2024-04-25T10:02:13.463028+0000","last_became_active":"2024-04-25T10:02:11.435244+0000","last_became_peered":"2024-04-25T10:02:11.435244+0000","last_unstale":"2024-04-25T10:02:13.463028+0000","last_undegraded":"2024-04-25T10:02:13.463028+0000","last_fullsized":"2024-04-25T10:02:13.463028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466834+0000","last_change":"2024-04-25T10:02:07.385486+0000","last_active":"2024-04-25T10:02:13.466834+0000","last_peered":"2024-04-25T10:02:13.466834+0000","last_clean":"2024-04-25T10:02:13.466834+0000","last_became_active":"2024-04-25T10:02:07.385202+0000","last_became_peered":"2024-04-25T10:02:07.385202+0000","last_unstale":"2024-04-25T10:02:13.466834+0000","last_undegraded":"2024-04-25T10:02:13.466834+0000","last_fullsized":"2024-04-25T10:02:13.466834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:16.103362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462441+0000","last_change":"2024-04-25T10:02:06.379591+0000","last_active":"2024-04-25T10:02:13.462441+0000","last_peered":"2024-04-25T10:02:13.462441+0000","last_clean":"2024-04-25T10:02:13.462441+0000","last_became_active":"2024-04-25T10:02:06.379178+0000","last_became_peered":"2024-04-25T10:02:06.379178+0000","last_unstale":"2024-04-25T10:02:13.462441+0000","last_undegraded":"2024-04-25T10:02:13.462441+0000","last_fullsized":"2024-04-25T10:02:13.462441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463716+0000","last_change":"2024-04-25T10:02:11.459834+0000","last_active":"2024-04-25T10:02:13.463716+0000","last_peered":"2024-04-25T10:02:13.463716+0000","last_clean":"2024-04-25T10:02:13.463716+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.463716+0000","last_undegraded":"2024-04-25T10:02:13.463716+0000","last_fullsized":"2024-04-25T10:02:13.463716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466278+0000","last_change":"2024-04-25T10:02:07.387540+0000","last_active":"2024-04-25T10:02:13.466278+0000","last_peered":"2024-04-25T10:02:13.466278+0000","last_clean":"2024-04-25T10:02:13.466278+0000","last_became_active":"2024-04-25T10:02:07.385395+0000","last_became_peered":"2024-04-25T10:02:07.385395+0000","last_unstale":"2024-04-25T10:02:13.466278+0000","last_undegraded":"2024-04-25T10:02:13.466278+0000","last_fullsized":"2024-04-25T10:02:13.466278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408935+0000","last_change":"2024-04-25T10:02:06.379749+0000","last_active":"2024-04-25T10:02:12.408935+0000","last_peered":"2024-04-25T10:02:12.408935+0000","last_clean":"2024-04-25T10:02:12.408935+0000","last_became_active":"2024-04-25T10:02:06.378947+0000","last_became_peered":"2024-04-25T10:02:06.378947+0000","last_unstale":"2024-04-25T10:02:12.408935+0000","last_undegraded":"2024-04-25T10:02:12.408935+0000","last_fullsized":"2024-04-25T10:02:12.408935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:03.055592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405719+0000","last_change":"2024-04-25T10:02:11.437602+0000","last_active":"2024-04-25T10:02:12.405719+0000","last_peered":"2024-04-25T10:02:12.405719+0000","last_clean":"2024-04-25T10:02:12.405719+0000","last_became_active":"2024-04-25T10:02:11.436553+0000","last_became_peered":"2024-04-25T10:02:11.436553+0000","last_unstale":"2024-04-25T10:02:12.405719+0000","last_undegraded":"2024-04-25T10:02:12.405719+0000","last_fullsized":"2024-04-25T10:02:12.405719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:53.618014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464059+0000","last_change":"2024-04-25T10:02:07.384411+0000","last_active":"2024-04-25T10:02:13.464059+0000","last_peered":"2024-04-25T10:02:13.464059+0000","last_clean":"2024-04-25T10:02:13.464059+0000","last_became_active":"2024-04-25T10:02:07.377814+0000","last_became_peered":"2024-04-25T10:02:07.377814+0000","last_unstale":"2024-04-25T10:02:13.464059+0000","last_undegraded":"2024-04-25T10:02:13.464059+0000","last_fullsized":"2024-04-25T10:02:13.464059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461988+0000","last_change":"2024-04-25T10:02:06.360702+0000","last_active":"2024-04-25T10:02:13.461988+0000","last_peered":"2024-04-25T10:02:13.461988+0000","last_clean":"2024-04-25T10:02:13.461988+0000","last_became_active":"2024-04-25T10:02:06.360297+0000","last_became_peered":"2024-04-25T10:02:06.360297+0000","last_unstale":"2024-04-25T10:02:13.461988+0000","last_undegraded":"2024-04-25T10:02:13.461988+0000","last_fullsized":"2024-04-25T10:02:13.461988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462271+0000","last_change":"2024-04-25T10:02:11.427201+0000","last_active":"2024-04-25T10:02:13.462271+0000","last_peered":"2024-04-25T10:02:13.462271+0000","last_clean":"2024-04-25T10:02:13.462271+0000","last_became_active":"2024-04-25T10:02:11.426734+0000","last_became_peered":"2024-04-25T10:02:11.426734+0000","last_unstale":"2024-04-25T10:02:13.462271+0000","last_undegraded":"2024-04-25T10:02:13.462271+0000","last_fullsized":"2024-04-25T10:02:13.462271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404061+0000","last_change":"2024-04-25T10:02:07.371003+0000","last_active":"2024-04-25T10:02:12.404061+0000","last_peered":"2024-04-25T10:02:12.404061+0000","last_clean":"2024-04-25T10:02:12.404061+0000","last_became_active":"2024-04-25T10:02:07.370695+0000","last_became_peered":"2024-04-25T10:02:07.370695+0000","last_unstale":"2024-04-25T10:02:12.404061+0000","last_undegraded":"2024-04-25T10:02:12.404061+0000","last_fullsized":"2024-04-25T10:02:12.404061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405106+0000","last_change":"2024-04-25T10:02:06.362100+0000","last_active":"2024-04-25T10:02:12.405106+0000","last_peered":"2024-04-25T10:02:12.405106+0000","last_clean":"2024-04-25T10:02:12.405106+0000","last_became_active":"2024-04-25T10:02:06.361358+0000","last_became_peered":"2024-04-25T10:02:06.361358+0000","last_unstale":"2024-04-25T10:02:12.405106+0000","last_undegraded":"2024-04-25T10:02:12.405106+0000","last_fullsized":"2024-04-25T10:02:12.405106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406644+0000","last_change":"2024-04-25T10:02:11.460013+0000","last_active":"2024-04-25T10:02:12.406644+0000","last_peered":"2024-04-25T10:02:12.406644+0000","last_clean":"2024-04-25T10:02:12.406644+0000","last_became_active":"2024-04-25T10:02:11.459491+0000","last_became_peered":"2024-04-25T10:02:11.459491+0000","last_unstale":"2024-04-25T10:02:12.406644+0000","last_undegraded":"2024-04-25T10:02:12.406644+0000","last_fullsized":"2024-04-25T10:02:12.406644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.241421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465085+0000","last_change":"2024-04-25T10:02:07.373359+0000","last_active":"2024-04-25T10:02:13.465085+0000","last_peered":"2024-04-25T10:02:13.465085+0000","last_clean":"2024-04-25T10:02:13.465085+0000","last_became_active":"2024-04-25T10:02:07.373100+0000","last_became_peered":"2024-04-25T10:02:07.373100+0000","last_unstale":"2024-04-25T10:02:13.465085+0000","last_undegraded":"2024-04-25T10:02:13.465085+0000","last_fullsized":"2024-04-25T10:02:13.465085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.957032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465425+0000","last_change":"2024-04-25T10:02:06.382032+0000","last_active":"2024-04-25T10:02:13.465425+0000","last_peered":"2024-04-25T10:02:13.465425+0000","last_clean":"2024-04-25T10:02:13.465425+0000","last_became_active":"2024-04-25T10:02:06.381711+0000","last_became_peered":"2024-04-25T10:02:06.381711+0000","last_unstale":"2024-04-25T10:02:13.465425+0000","last_undegraded":"2024-04-25T10:02:13.465425+0000","last_fullsized":"2024-04-25T10:02:13.465425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404352+0000","last_change":"2024-04-25T10:02:11.458829+0000","last_active":"2024-04-25T10:02:12.404352+0000","last_peered":"2024-04-25T10:02:12.404352+0000","last_clean":"2024-04-25T10:02:12.404352+0000","last_became_active":"2024-04-25T10:02:11.458386+0000","last_became_peered":"2024-04-25T10:02:11.458386+0000","last_unstale":"2024-04-25T10:02:12.404352+0000","last_undegraded":"2024-04-25T10:02:12.404352+0000","last_fullsized":"2024-04-25T10:02:12.404352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:15.902040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462813+0000","last_change":"2024-04-25T10:02:07.372352+0000","last_active":"2024-04-25T10:02:13.462813+0000","last_peered":"2024-04-25T10:02:13.462813+0000","last_clean":"2024-04-25T10:02:13.462813+0000","last_became_active":"2024-04-25T10:02:07.372124+0000","last_became_peered":"2024-04-25T10:02:07.372124+0000","last_unstale":"2024-04-25T10:02:13.462813+0000","last_undegraded":"2024-04-25T10:02:13.462813+0000","last_fullsized":"2024-04-25T10:02:13.462813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463253+0000","last_change":"2024-04-25T10:02:06.361119+0000","last_active":"2024-04-25T10:02:13.463253+0000","last_peered":"2024-04-25T10:02:13.463253+0000","last_clean":"2024-04-25T10:02:13.463253+0000","last_became_active":"2024-04-25T10:02:06.360459+0000","last_became_peered":"2024-04-25T10:02:06.360459+0000","last_unstale":"2024-04-25T10:02:13.463253+0000","last_undegraded":"2024-04-25T10:02:13.463253+0000","last_fullsized":"2024-04-25T10:02:13.463253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465848+0000","last_change":"2024-04-25T10:02:11.460123+0000","last_active":"2024-04-25T10:02:13.465848+0000","last_peered":"2024-04-25T10:02:13.465848+0000","last_clean":"2024-04-25T10:02:13.465848+0000","last_became_active":"2024-04-25T10:02:11.459152+0000","last_became_peered":"2024-04-25T10:02:11.459152+0000","last_unstale":"2024-04-25T10:02:13.465848+0000","last_undegraded":"2024-04-25T10:02:13.465848+0000","last_fullsized":"2024-04-25T10:02:13.465848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465062+0000","last_change":"2024-04-25T10:02:07.388512+0000","last_active":"2024-04-25T10:02:13.465062+0000","last_peered":"2024-04-25T10:02:13.465062+0000","last_clean":"2024-04-25T10:02:13.465062+0000","last_became_active":"2024-04-25T10:02:07.387797+0000","last_became_peered":"2024-04-25T10:02:07.387797+0000","last_unstale":"2024-04-25T10:02:13.465062+0000","last_undegraded":"2024-04-25T10:02:13.465062+0000","last_fullsized":"2024-04-25T10:02:13.465062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:17.622024+0000","last_change":"2024-04-25T10:02:06.370256+0000","last_active":"2024-04-25T10:02:17.622024+0000","last_peered":"2024-04-25T10:02:17.622024+0000","last_clean":"2024-04-25T10:02:17.622024+0000","last_became_active":"2024-04-25T10:02:06.368370+0000","last_became_peered":"2024-04-25T10:02:06.368370+0000","last_unstale":"2024-04-25T10:02:17.622024+0000","last_undegraded":"2024-04-25T10:02:17.622024+0000","last_fullsized":"2024-04-25T10:02:17.622024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462482+0000","last_change":"2024-04-25T10:02:11.438518+0000","last_active":"2024-04-25T10:02:13.462482+0000","last_peered":"2024-04-25T10:02:13.462482+0000","last_clean":"2024-04-25T10:02:13.462482+0000","last_became_active":"2024-04-25T10:02:11.436290+0000","last_became_peered":"2024-04-25T10:02:11.436290+0000","last_unstale":"2024-04-25T10:02:13.462482+0000","last_undegraded":"2024-04-25T10:02:13.462482+0000","last_fullsized":"2024-04-25T10:02:13.462482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407354+0000","last_change":"2024-04-25T10:02:07.358234+0000","last_active":"2024-04-25T10:02:12.407354+0000","last_peered":"2024-04-25T10:02:12.407354+0000","last_clean":"2024-04-25T10:02:12.407354+0000","last_became_active":"2024-04-25T10:02:07.357961+0000","last_became_peered":"2024-04-25T10:02:07.357961+0000","last_unstale":"2024-04-25T10:02:12.407354+0000","last_undegraded":"2024-04-25T10:02:12.407354+0000","last_fullsized":"2024-04-25T10:02:12.407354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461751+0000","last_change":"2024-04-25T10:02:06.386529+0000","last_active":"2024-04-25T10:02:13.461751+0000","last_peered":"2024-04-25T10:02:13.461751+0000","last_clean":"2024-04-25T10:02:13.461751+0000","last_became_active":"2024-04-25T10:02:06.386097+0000","last_became_peered":"2024-04-25T10:02:06.386097+0000","last_unstale":"2024-04-25T10:02:13.461751+0000","last_undegraded":"2024-04-25T10:02:13.461751+0000","last_fullsized":"2024-04-25T10:02:13.461751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407672+0000","last_change":"2024-04-25T10:02:11.437594+0000","last_active":"2024-04-25T10:02:12.407672+0000","last_peered":"2024-04-25T10:02:12.407672+0000","last_clean":"2024-04-25T10:02:12.407672+0000","last_became_active":"2024-04-25T10:02:11.436114+0000","last_became_peered":"2024-04-25T10:02:11.436114+0000","last_unstale":"2024-04-25T10:02:12.407672+0000","last_undegraded":"2024-04-25T10:02:12.407672+0000","last_fullsized":"2024-04-25T10:02:12.407672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462591+0000","last_change":"2024-04-25T10:02:07.371078+0000","last_active":"2024-04-25T10:02:13.462591+0000","last_peered":"2024-04-25T10:02:13.462591+0000","last_clean":"2024-04-25T10:02:13.462591+0000","last_became_active":"2024-04-25T10:02:07.370749+0000","last_became_peered":"2024-04-25T10:02:07.370749+0000","last_unstale":"2024-04-25T10:02:13.462591+0000","last_undegraded":"2024-04-25T10:02:13.462591+0000","last_fullsized":"2024-04-25T10:02:13.462591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466420+0000","last_change":"2024-04-25T10:02:06.377512+0000","last_active":"2024-04-25T10:02:13.466420+0000","last_peered":"2024-04-25T10:02:13.466420+0000","last_clean":"2024-04-25T10:02:13.466420+0000","last_became_active":"2024-04-25T10:02:06.376824+0000","last_became_peered":"2024-04-25T10:02:06.376824+0000","last_unstale":"2024-04-25T10:02:13.466420+0000","last_undegraded":"2024-04-25T10:02:13.466420+0000","last_fullsized":"2024-04-25T10:02:13.466420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403770+0000","last_change":"2024-04-25T10:02:11.442056+0000","last_active":"2024-04-25T10:02:12.403770+0000","last_peered":"2024-04-25T10:02:12.403770+0000","last_clean":"2024-04-25T10:02:12.403770+0000","last_became_active":"2024-04-25T10:02:11.441158+0000","last_became_peered":"2024-04-25T10:02:11.441158+0000","last_unstale":"2024-04-25T10:02:12.403770+0000","last_undegraded":"2024-04-25T10:02:12.403770+0000","last_fullsized":"2024-04-25T10:02:12.403770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466185+0000","last_change":"2024-04-25T10:02:07.389094+0000","last_active":"2024-04-25T10:02:13.466185+0000","last_peered":"2024-04-25T10:02:13.466185+0000","last_clean":"2024-04-25T10:02:13.466185+0000","last_became_active":"2024-04-25T10:02:07.388187+0000","last_became_peered":"2024-04-25T10:02:07.388187+0000","last_unstale":"2024-04-25T10:02:13.466185+0000","last_undegraded":"2024-04-25T10:02:13.466185+0000","last_fullsized":"2024-04-25T10:02:13.466185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406550+0000","last_change":"2024-04-25T10:02:06.383181+0000","last_active":"2024-04-25T10:02:12.406550+0000","last_peered":"2024-04-25T10:02:12.406550+0000","last_clean":"2024-04-25T10:02:12.406550+0000","last_became_active":"2024-04-25T10:02:06.382921+0000","last_became_peered":"2024-04-25T10:02:06.382921+0000","last_unstale":"2024-04-25T10:02:12.406550+0000","last_undegraded":"2024-04-25T10:02:12.406550+0000","last_fullsized":"2024-04-25T10:02:12.406550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404862+0000","last_change":"2024-04-25T10:02:11.454031+0000","last_active":"2024-04-25T10:02:12.404862+0000","last_peered":"2024-04-25T10:02:12.404862+0000","last_clean":"2024-04-25T10:02:12.404862+0000","last_became_active":"2024-04-25T10:02:11.453256+0000","last_became_peered":"2024-04-25T10:02:11.453256+0000","last_unstale":"2024-04-25T10:02:12.404862+0000","last_undegraded":"2024-04-25T10:02:12.404862+0000","last_fullsized":"2024-04-25T10:02:12.404862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:26.875452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462061+0000","last_change":"2024-04-25T10:02:11.460406+0000","last_active":"2024-04-25T10:02:13.462061+0000","last_peered":"2024-04-25T10:02:13.462061+0000","last_clean":"2024-04-25T10:02:13.462061+0000","last_became_active":"2024-04-25T10:02:11.459947+0000","last_became_peered":"2024-04-25T10:02:11.459947+0000","last_unstale":"2024-04-25T10:02:13.462061+0000","last_undegraded":"2024-04-25T10:02:13.462061+0000","last_fullsized":"2024-04-25T10:02:13.462061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461463+0000","last_change":"2024-04-25T10:02:07.361531+0000","last_active":"2024-04-25T10:02:13.461463+0000","last_peered":"2024-04-25T10:02:13.461463+0000","last_clean":"2024-04-25T10:02:13.461463+0000","last_became_active":"2024-04-25T10:02:07.361213+0000","last_became_peered":"2024-04-25T10:02:07.361213+0000","last_unstale":"2024-04-25T10:02:13.461463+0000","last_undegraded":"2024-04-25T10:02:13.461463+0000","last_fullsized":"2024-04-25T10:02:13.461463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464490+0000","last_change":"2024-04-25T10:02:06.386304+0000","last_active":"2024-04-25T10:02:13.464490+0000","last_peered":"2024-04-25T10:02:13.464490+0000","last_clean":"2024-04-25T10:02:13.464490+0000","last_became_active":"2024-04-25T10:02:06.382588+0000","last_became_peered":"2024-04-25T10:02:06.382588+0000","last_unstale":"2024-04-25T10:02:13.464490+0000","last_undegraded":"2024-04-25T10:02:13.464490+0000","last_fullsized":"2024-04-25T10:02:13.464490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406241+0000","last_change":"2024-04-25T10:02:11.433821+0000","last_active":"2024-04-25T10:02:12.406241+0000","last_peered":"2024-04-25T10:02:12.406241+0000","last_clean":"2024-04-25T10:02:12.406241+0000","last_became_active":"2024-04-25T10:02:11.432467+0000","last_became_peered":"2024-04-25T10:02:11.432467+0000","last_unstale":"2024-04-25T10:02:12.406241+0000","last_undegraded":"2024-04-25T10:02:12.406241+0000","last_fullsized":"2024-04-25T10:02:12.406241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403810+0000","last_change":"2024-04-25T10:02:07.376574+0000","last_active":"2024-04-25T10:02:12.403810+0000","last_peered":"2024-04-25T10:02:12.403810+0000","last_clean":"2024-04-25T10:02:12.403810+0000","last_became_active":"2024-04-25T10:02:07.376275+0000","last_became_peered":"2024-04-25T10:02:07.376275+0000","last_unstale":"2024-04-25T10:02:12.403810+0000","last_undegraded":"2024-04-25T10:02:12.403810+0000","last_fullsized":"2024-04-25T10:02:12.403810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405129+0000","last_change":"2024-04-25T10:02:06.363949+0000","last_active":"2024-04-25T10:02:12.405129+0000","last_peered":"2024-04-25T10:02:12.405129+0000","last_clean":"2024-04-25T10:02:12.405129+0000","last_became_active":"2024-04-25T10:02:06.363379+0000","last_became_peered":"2024-04-25T10:02:06.363379+0000","last_unstale":"2024-04-25T10:02:12.405129+0000","last_undegraded":"2024-04-25T10:02:12.405129+0000","last_fullsized":"2024-04-25T10:02:12.405129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405641+0000","last_change":"2024-04-25T10:02:06.361583+0000","last_active":"2024-04-25T10:02:12.405641+0000","last_peered":"2024-04-25T10:02:12.405641+0000","last_clean":"2024-04-25T10:02:12.405641+0000","last_became_active":"2024-04-25T10:02:06.360807+0000","last_became_peered":"2024-04-25T10:02:06.360807+0000","last_unstale":"2024-04-25T10:02:12.405641+0000","last_undegraded":"2024-04-25T10:02:12.405641+0000","last_fullsized":"2024-04-25T10:02:12.405641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463206+0000","last_change":"2024-04-25T10:02:07.367030+0000","last_active":"2024-04-25T10:02:13.463206+0000","last_peered":"2024-04-25T10:02:13.463206+0000","last_clean":"2024-04-25T10:02:13.463206+0000","last_became_active":"2024-04-25T10:02:07.366570+0000","last_became_peered":"2024-04-25T10:02:07.366570+0000","last_unstale":"2024-04-25T10:02:13.463206+0000","last_undegraded":"2024-04-25T10:02:13.463206+0000","last_fullsized":"2024-04-25T10:02:13.463206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463018+0000","last_change":"2024-04-25T10:02:11.434159+0000","last_active":"2024-04-25T10:02:13.463018+0000","last_peered":"2024-04-25T10:02:13.463018+0000","last_clean":"2024-04-25T10:02:13.463018+0000","last_became_active":"2024-04-25T10:02:11.433378+0000","last_became_peered":"2024-04-25T10:02:11.433378+0000","last_unstale":"2024-04-25T10:02:13.463018+0000","last_undegraded":"2024-04-25T10:02:13.463018+0000","last_fullsized":"2024-04-25T10:02:13.463018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:25.406090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403964+0000","last_change":"2024-04-25T10:02:06.375949+0000","last_active":"2024-04-25T10:02:12.403964+0000","last_peered":"2024-04-25T10:02:12.403964+0000","last_clean":"2024-04-25T10:02:12.403964+0000","last_became_active":"2024-04-25T10:02:06.375558+0000","last_became_peered":"2024-04-25T10:02:06.375558+0000","last_unstale":"2024-04-25T10:02:12.403964+0000","last_undegraded":"2024-04-25T10:02:12.403964+0000","last_fullsized":"2024-04-25T10:02:12.403964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406442+0000","last_change":"2024-04-25T10:02:07.352019+0000","last_active":"2024-04-25T10:02:12.406442+0000","last_peered":"2024-04-25T10:02:12.406442+0000","last_clean":"2024-04-25T10:02:12.406442+0000","last_became_active":"2024-04-25T10:02:07.351303+0000","last_became_peered":"2024-04-25T10:02:07.351303+0000","last_unstale":"2024-04-25T10:02:12.406442+0000","last_undegraded":"2024-04-25T10:02:12.406442+0000","last_fullsized":"2024-04-25T10:02:12.406442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:08.301904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406592+0000","last_change":"2024-04-25T10:02:11.441304+0000","last_active":"2024-04-25T10:02:12.406592+0000","last_peered":"2024-04-25T10:02:12.406592+0000","last_clean":"2024-04-25T10:02:12.406592+0000","last_became_active":"2024-04-25T10:02:11.440847+0000","last_became_peered":"2024-04-25T10:02:11.440847+0000","last_unstale":"2024-04-25T10:02:12.406592+0000","last_undegraded":"2024-04-25T10:02:12.406592+0000","last_fullsized":"2024-04-25T10:02:12.406592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:07.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403875+0000","last_change":"2024-04-25T10:02:07.372731+0000","last_active":"2024-04-25T10:02:12.403875+0000","last_peered":"2024-04-25T10:02:12.403875+0000","last_clean":"2024-04-25T10:02:12.403875+0000","last_became_active":"2024-04-25T10:02:07.372438+0000","last_became_peered":"2024-04-25T10:02:07.372438+0000","last_unstale":"2024-04-25T10:02:12.403875+0000","last_undegraded":"2024-04-25T10:02:12.403875+0000","last_fullsized":"2024-04-25T10:02:12.403875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462340+0000","last_change":"2024-04-25T10:02:07.364477+0000","last_active":"2024-04-25T10:02:13.462340+0000","last_peered":"2024-04-25T10:02:13.462340+0000","last_clean":"2024-04-25T10:02:13.462340+0000","last_became_active":"2024-04-25T10:02:07.363263+0000","last_became_peered":"2024-04-25T10:02:07.363263+0000","last_unstale":"2024-04-25T10:02:13.462340+0000","last_undegraded":"2024-04-25T10:02:13.462340+0000","last_fullsized":"2024-04-25T10:02:13.462340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:42.118057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465162+0000","last_change":"2024-04-25T10:02:07.366856+0000","last_active":"2024-04-25T10:02:13.465162+0000","last_peered":"2024-04-25T10:02:13.465162+0000","last_clean":"2024-04-25T10:02:13.465162+0000","last_became_active":"2024-04-25T10:02:07.366608+0000","last_became_peered":"2024-04-25T10:02:07.366608+0000","last_unstale":"2024-04-25T10:02:13.465162+0000","last_undegraded":"2024-04-25T10:02:13.465162+0000","last_fullsized":"2024-04-25T10:02:13.465162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463076+0000","last_change":"2024-04-25T10:02:07.374664+0000","last_active":"2024-04-25T10:02:13.463076+0000","last_peered":"2024-04-25T10:02:13.463076+0000","last_clean":"2024-04-25T10:02:13.463076+0000","last_became_active":"2024-04-25T10:02:07.374088+0000","last_became_peered":"2024-04-25T10:02:07.374088+0000","last_unstale":"2024-04-25T10:02:13.463076+0000","last_undegraded":"2024-04-25T10:02:13.463076+0000","last_fullsized":"2024-04-25T10:02:13.463076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404526+0000","last_change":"2024-04-25T10:02:07.365480+0000","last_active":"2024-04-25T10:02:12.404526+0000","last_peered":"2024-04-25T10:02:12.404526+0000","last_clean":"2024-04-25T10:02:12.404526+0000","last_became_active":"2024-04-25T10:02:07.365034+0000","last_became_peered":"2024-04-25T10:02:07.365034+0000","last_unstale":"2024-04-25T10:02:12.404526+0000","last_undegraded":"2024-04-25T10:02:12.404526+0000","last_fullsized":"2024-04-25T10:02:12.404526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461634+0000","last_change":"2024-04-25T10:02:07.362200+0000","last_active":"2024-04-25T10:02:13.461634+0000","last_peered":"2024-04-25T10:02:13.461634+0000","last_clean":"2024-04-25T10:02:13.461634+0000","last_became_active":"2024-04-25T10:02:07.361929+0000","last_became_peered":"2024-04-25T10:02:07.361929+0000","last_unstale":"2024-04-25T10:02:13.461634+0000","last_undegraded":"2024-04-25T10:02:13.461634+0000","last_fullsized":"2024-04-25T10:02:13.461634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407180+0000","last_change":"2024-04-25T10:02:07.375688+0000","last_active":"2024-04-25T10:02:12.407180+0000","last_peered":"2024-04-25T10:02:12.407180+0000","last_clean":"2024-04-25T10:02:12.407180+0000","last_became_active":"2024-04-25T10:02:07.375046+0000","last_became_peered":"2024-04-25T10:02:07.375046+0000","last_unstale":"2024-04-25T10:02:12.407180+0000","last_undegraded":"2024-04-25T10:02:12.407180+0000","last_fullsized":"2024-04-25T10:02:12.407180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406734+0000","last_change":"2024-04-25T10:02:07.365322+0000","last_active":"2024-04-25T10:02:12.406734+0000","last_peered":"2024-04-25T10:02:12.406734+0000","last_clean":"2024-04-25T10:02:12.406734+0000","last_became_active":"2024-04-25T10:02:07.365003+0000","last_became_peered":"2024-04-25T10:02:07.365003+0000","last_unstale":"2024-04-25T10:02:12.406734+0000","last_undegraded":"2024-04-25T10:02:12.406734+0000","last_fullsized":"2024-04-25T10:02:12.406734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464320+0000","last_change":"2024-04-25T10:02:07.374731+0000","last_active":"2024-04-25T10:02:13.464320+0000","last_peered":"2024-04-25T10:02:13.464320+0000","last_clean":"2024-04-25T10:02:13.464320+0000","last_became_active":"2024-04-25T10:02:07.374113+0000","last_became_peered":"2024-04-25T10:02:07.374113+0000","last_unstale":"2024-04-25T10:02:13.464320+0000","last_undegraded":"2024-04-25T10:02:13.464320+0000","last_fullsized":"2024-04-25T10:02:13.464320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404344+0000","last_change":"2024-04-25T10:02:07.365139+0000","last_active":"2024-04-25T10:02:12.404344+0000","last_peered":"2024-04-25T10:02:12.404344+0000","last_clean":"2024-04-25T10:02:12.404344+0000","last_became_active":"2024-04-25T10:02:07.364860+0000","last_became_peered":"2024-04-25T10:02:07.364860+0000","last_unstale":"2024-04-25T10:02:12.404344+0000","last_undegraded":"2024-04-25T10:02:12.404344+0000","last_fullsized":"2024-04-25T10:02:12.404344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464902+0000","last_change":"2024-04-25T10:02:07.379737+0000","last_active":"2024-04-25T10:02:13.464902+0000","last_peered":"2024-04-25T10:02:13.464902+0000","last_clean":"2024-04-25T10:02:13.464902+0000","last_became_active":"2024-04-25T10:02:07.379465+0000","last_became_peered":"2024-04-25T10:02:07.379465+0000","last_unstale":"2024-04-25T10:02:13.464902+0000","last_undegraded":"2024-04-25T10:02:13.464902+0000","last_fullsized":"2024-04-25T10:02:13.464902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462820+0000","last_change":"2024-04-25T10:02:07.379170+0000","last_active":"2024-04-25T10:02:13.462820+0000","last_peered":"2024-04-25T10:02:13.462820+0000","last_clean":"2024-04-25T10:02:13.462820+0000","last_became_active":"2024-04-25T10:02:07.378877+0000","last_became_peered":"2024-04-25T10:02:07.378877+0000","last_unstale":"2024-04-25T10:02:13.462820+0000","last_undegraded":"2024-04-25T10:02:13.462820+0000","last_fullsized":"2024-04-25T10:02:13.462820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406740+0000","last_change":"2024-04-25T10:02:07.367522+0000","last_active":"2024-04-25T10:02:12.406740+0000","last_peered":"2024-04-25T10:02:12.406740+0000","last_clean":"2024-04-25T10:02:12.406740+0000","last_became_active":"2024-04-25T10:02:07.367226+0000","last_became_peered":"2024-04-25T10:02:07.367226+0000","last_unstale":"2024-04-25T10:02:12.406740+0000","last_undegraded":"2024-04-25T10:02:12.406740+0000","last_fullsized":"2024-04-25T10:02:12.406740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465868+0000","last_change":"2024-04-25T10:02:07.378749+0000","last_active":"2024-04-25T10:02:13.465868+0000","last_peered":"2024-04-25T10:02:13.465868+0000","last_clean":"2024-04-25T10:02:13.465868+0000","last_became_active":"2024-04-25T10:02:07.378388+0000","last_became_peered":"2024-04-25T10:02:07.378388+0000","last_unstale":"2024-04-25T10:02:13.465868+0000","last_undegraded":"2024-04-25T10:02:13.465868+0000","last_fullsized":"2024-04-25T10:02:13.465868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:00.508687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462346+0000","last_change":"2024-04-25T10:02:07.377744+0000","last_active":"2024-04-25T10:02:13.462346+0000","last_peered":"2024-04-25T10:02:13.462346+0000","last_clean":"2024-04-25T10:02:13.462346+0000","last_became_active":"2024-04-25T10:02:07.376844+0000","last_became_peered":"2024-04-25T10:02:07.376844+0000","last_unstale":"2024-04-25T10:02:13.462346+0000","last_undegraded":"2024-04-25T10:02:13.462346+0000","last_fullsized":"2024-04-25T10:02:13.462346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465699+0000","last_change":"2024-04-25T10:02:07.388777+0000","last_active":"2024-04-25T10:02:13.465699+0000","last_peered":"2024-04-25T10:02:13.465699+0000","last_clean":"2024-04-25T10:02:13.465699+0000","last_became_active":"2024-04-25T10:02:07.388058+0000","last_became_peered":"2024-04-25T10:02:07.388058+0000","last_unstale":"2024-04-25T10:02:13.465699+0000","last_undegraded":"2024-04-25T10:02:13.465699+0000","last_fullsized":"2024-04-25T10:02:13.465699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464055+0000","last_change":"2024-04-25T10:02:07.377996+0000","last_active":"2024-04-25T10:02:13.464055+0000","last_peered":"2024-04-25T10:02:13.464055+0000","last_clean":"2024-04-25T10:02:13.464055+0000","last_became_active":"2024-04-25T10:02:07.376664+0000","last_became_peered":"2024-04-25T10:02:07.376664+0000","last_unstale":"2024-04-25T10:02:13.464055+0000","last_undegraded":"2024-04-25T10:02:13.464055+0000","last_fullsized":"2024-04-25T10:02:13.464055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461571+0000","last_change":"2024-04-25T10:02:07.364951+0000","last_active":"2024-04-25T10:02:13.461571+0000","last_peered":"2024-04-25T10:02:13.461571+0000","last_clean":"2024-04-25T10:02:13.461571+0000","last_became_active":"2024-04-25T10:02:07.364138+0000","last_became_peered":"2024-04-25T10:02:07.364138+0000","last_unstale":"2024-04-25T10:02:13.461571+0000","last_undegraded":"2024-04-25T10:02:13.461571+0000","last_fullsized":"2024-04-25T10:02:13.461571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462765+0000","last_change":"2024-04-25T10:02:07.370504+0000","last_active":"2024-04-25T10:02:13.462765+0000","last_peered":"2024-04-25T10:02:13.462765+0000","last_clean":"2024-04-25T10:02:13.462765+0000","last_became_active":"2024-04-25T10:02:07.370157+0000","last_became_peered":"2024-04-25T10:02:07.370157+0000","last_unstale":"2024-04-25T10:02:13.462765+0000","last_undegraded":"2024-04-25T10:02:13.462765+0000","last_fullsized":"2024-04-25T10:02:13.462765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403675+0000","last_change":"2024-04-25T10:02:07.360200+0000","last_active":"2024-04-25T10:02:12.403675+0000","last_peered":"2024-04-25T10:02:12.403675+0000","last_clean":"2024-04-25T10:02:12.403675+0000","last_became_active":"2024-04-25T10:02:07.358078+0000","last_became_peered":"2024-04-25T10:02:07.358078+0000","last_unstale":"2024-04-25T10:02:12.403675+0000","last_undegraded":"2024-04-25T10:02:12.403675+0000","last_fullsized":"2024-04-25T10:02:12.403675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407201+0000","last_change":"2024-04-25T10:02:07.376059+0000","last_active":"2024-04-25T10:02:12.407201+0000","last_peered":"2024-04-25T10:02:12.407201+0000","last_clean":"2024-04-25T10:02:12.407201+0000","last_became_active":"2024-04-25T10:02:07.374735+0000","last_became_peered":"2024-04-25T10:02:07.374735+0000","last_unstale":"2024-04-25T10:02:12.407201+0000","last_undegraded":"2024-04-25T10:02:12.407201+0000","last_fullsized":"2024-04-25T10:02:12.407201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:54.666473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406312+0000","last_change":"2024-04-25T10:02:07.367747+0000","last_active":"2024-04-25T10:02:12.406312+0000","last_peered":"2024-04-25T10:02:12.406312+0000","last_clean":"2024-04-25T10:02:12.406312+0000","last_became_active":"2024-04-25T10:02:07.367055+0000","last_became_peered":"2024-04-25T10:02:07.367055+0000","last_unstale":"2024-04-25T10:02:12.406312+0000","last_undegraded":"2024-04-25T10:02:12.406312+0000","last_fullsized":"2024-04-25T10:02:12.406312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406946+0000","last_change":"2024-04-25T10:02:07.366278+0000","last_active":"2024-04-25T10:02:12.406946+0000","last_peered":"2024-04-25T10:02:12.406946+0000","last_clean":"2024-04-25T10:02:12.406946+0000","last_became_active":"2024-04-25T10:02:07.365591+0000","last_became_peered":"2024-04-25T10:02:07.365591+0000","last_unstale":"2024-04-25T10:02:12.406946+0000","last_undegraded":"2024-04-25T10:02:12.406946+0000","last_fullsized":"2024-04-25T10:02:12.406946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466682+0000","last_change":"2024-04-25T10:02:07.389531+0000","last_active":"2024-04-25T10:02:13.466682+0000","last_peered":"2024-04-25T10:02:13.466682+0000","last_clean":"2024-04-25T10:02:13.466682+0000","last_became_active":"2024-04-25T10:02:07.389091+0000","last_became_peered":"2024-04-25T10:02:07.389091+0000","last_unstale":"2024-04-25T10:02:13.466682+0000","last_undegraded":"2024-04-25T10:02:13.466682+0000","last_fullsized":"2024-04-25T10:02:13.466682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403425+0000","last_change":"2024-04-25T10:02:07.368434+0000","last_active":"2024-04-25T10:02:12.403425+0000","last_peered":"2024-04-25T10:02:12.403425+0000","last_clean":"2024-04-25T10:02:12.403425+0000","last_became_active":"2024-04-25T10:02:07.366907+0000","last_became_peered":"2024-04-25T10:02:07.366907+0000","last_unstale":"2024-04-25T10:02:12.403425+0000","last_undegraded":"2024-04-25T10:02:12.403425+0000","last_fullsized":"2024-04-25T10:02:12.403425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463789+0000","last_change":"2024-04-25T10:02:07.377948+0000","last_active":"2024-04-25T10:02:13.463789+0000","last_peered":"2024-04-25T10:02:13.463789+0000","last_clean":"2024-04-25T10:02:13.463789+0000","last_became_active":"2024-04-25T10:02:07.377170+0000","last_became_peered":"2024-04-25T10:02:07.377170+0000","last_unstale":"2024-04-25T10:02:13.463789+0000","last_undegraded":"2024-04-25T10:02:13.463789+0000","last_fullsized":"2024-04-25T10:02:13.463789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464659+0000","last_change":"2024-04-25T10:02:07.376985+0000","last_active":"2024-04-25T10:02:13.464659+0000","last_peered":"2024-04-25T10:02:13.464659+0000","last_clean":"2024-04-25T10:02:13.464659+0000","last_became_active":"2024-04-25T10:02:07.376669+0000","last_became_peered":"2024-04-25T10:02:07.376669+0000","last_unstale":"2024-04-25T10:02:13.464659+0000","last_undegraded":"2024-04-25T10:02:13.464659+0000","last_fullsized":"2024-04-25T10:02:13.464659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464634+0000","last_change":"2024-04-25T10:02:07.377373+0000","last_active":"2024-04-25T10:02:13.464634+0000","last_peered":"2024-04-25T10:02:13.464634+0000","last_clean":"2024-04-25T10:02:13.464634+0000","last_became_active":"2024-04-25T10:02:07.373588+0000","last_became_peered":"2024-04-25T10:02:07.373588+0000","last_unstale":"2024-04-25T10:02:13.464634+0000","last_undegraded":"2024-04-25T10:02:13.464634+0000","last_fullsized":"2024-04-25T10:02:13.464634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464847+0000","last_change":"2024-04-25T10:02:07.382540+0000","last_active":"2024-04-25T10:02:13.464847+0000","last_peered":"2024-04-25T10:02:13.464847+0000","last_clean":"2024-04-25T10:02:13.464847+0000","last_became_active":"2024-04-25T10:02:07.381996+0000","last_became_peered":"2024-04-25T10:02:07.381996+0000","last_unstale":"2024-04-25T10:02:13.464847+0000","last_undegraded":"2024-04-25T10:02:13.464847+0000","last_fullsized":"2024-04-25T10:02:13.464847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462277+0000","last_change":"2024-04-25T10:02:07.376120+0000","last_active":"2024-04-25T10:02:13.462277+0000","last_peered":"2024-04-25T10:02:13.462277+0000","last_clean":"2024-04-25T10:02:13.462277+0000","last_became_active":"2024-04-25T10:02:07.375589+0000","last_became_peered":"2024-04-25T10:02:07.375589+0000","last_unstale":"2024-04-25T10:02:13.462277+0000","last_undegraded":"2024-04-25T10:02:13.462277+0000","last_fullsized":"2024-04-25T10:02:13.462277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:35.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462177+0000","last_change":"2024-04-25T10:02:07.369060+0000","last_active":"2024-04-25T10:02:13.462177+0000","last_peered":"2024-04-25T10:02:13.462177+0000","last_clean":"2024-04-25T10:02:13.462177+0000","last_became_active":"2024-04-25T10:02:07.368434+0000","last_became_peered":"2024-04-25T10:02:07.368434+0000","last_unstale":"2024-04-25T10:02:13.462177+0000","last_undegraded":"2024-04-25T10:02:13.462177+0000","last_fullsized":"2024-04-25T10:02:13.462177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466451+0000","last_change":"2024-04-25T10:02:07.385523+0000","last_active":"2024-04-25T10:02:13.466451+0000","last_peered":"2024-04-25T10:02:13.466451+0000","last_clean":"2024-04-25T10:02:13.466451+0000","last_became_active":"2024-04-25T10:02:07.385218+0000","last_became_peered":"2024-04-25T10:02:07.385218+0000","last_unstale":"2024-04-25T10:02:13.466451+0000","last_undegraded":"2024-04-25T10:02:13.466451+0000","last_fullsized":"2024-04-25T10:02:13.466451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:31.453 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:31.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.600+0000 7f74409bc640 1 Processor -- start 2024-04-25T10:02:31.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 -- start start 2024-04-25T10:02:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f743c0b2e20 0x7f743c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c15ae30 con 0x7f743c159ea0 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c06db10 con 0x7f743c0b2e20 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f74409bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0b2990 con 0x7f743c0b3200 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f743c0b2e20 0x7f743c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743ad9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f743c159ea0 0x7f743c15a330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57906/0 (socket says 172.21.15.82:57906) 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743ad9d640 1 -- 172.21.15.82:0/3792177212 learned_addr learned my addr 172.21.15.82:0/3792177212 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7439d9b640 1 -- 172.21.15.82:0/3792177212 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223837706 0 0) 0x7f743c15ae30 con 0x7f743c159ea0 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7439d9b640 1 -- 172.21.15.82:0/3792177212 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7428003530 con 0x7f743c159ea0 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 -- 172.21.15.82:0/3792177212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 msgr2=0x7f743c06d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7433fff640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c06d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 -- 172.21.15.82:0/3792177212 >> v1:172.21.15.82:6789/0 conn(0x7f743c159ea0 legacy=0x7f743c15a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 -- 172.21.15.82:0/3792177212 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c15b110 con 0x7f743c0b2e20 2024-04-25T10:02:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7433fff640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f743a59c640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c0b2e20 0x7f743c06d090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f742c0097c0 tx=0x7f742c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f676f22e1c535a3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7439d9b640 1 -- 172.21.15.82:0/3792177212 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0b1070 con 0x7f743c0b2e20 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7439d9b640 1 -- 172.21.15.82:0/3792177212 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c0ac410 con 0x7f743c0b2e20 2024-04-25T10:02:31.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.604+0000 7f7439d9b640 1 -- 172.21.15.82:0/3792177212 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c0ac7c0 con 0x7f743c0b2e20 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3792177212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c0b2e20 msgr2=0x7f743c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c0b2e20 0x7f743c06d090 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f742c0097c0 tx=0x7f742c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3792177212 shutdown_connections 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c06d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- 172.21.15.82:0/3792177212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c0b2e20 0x7f743c06d090 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3792177212 >> 172.21.15.82:0/3792177212 conn(0x7f743c0b11a0 msgr2=0x7f743c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3792177212 shutdown_connections 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3792177212 wait complete. 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 Processor -- start 2024-04-25T10:02:31.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- start start 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 0x7f743c157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0721c0 con 0x7f743c159ea0 2024-04-25T10:02:31.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c072040 con 0x7f743c0b2e20 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c1652b0 con 0x7f743c0b3200 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48796/0 (socket says 172.21.15.82:48796) 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 -- 172.21.15.82:0/3041524599 learned_addr learned my addr 172.21.15.82:0/3041524599 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743a59c640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 0x7f743c157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 -- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 msgr2=0x7f743c14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f7433fff640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c14f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 -- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 msgr2=0x7f743c157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 0x7f743c157190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 -- 172.21.15.82:0/3041524599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c14ba50 con 0x7f743c159ea0 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743a59c640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 0x7f743c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f743ad9d640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f743400cad0 tx=0x7f743400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434017070 con 0x7f743c159ea0 2024-04-25T10:02:31.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743400ac30 con 0x7f743c159ea0 2024-04-25T10:02:31.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c14bd30 con 0x7f743c159ea0 2024-04-25T10:02:31.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7434012400 con 0x7f743c159ea0 2024-04-25T10:02:31.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.608+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c14c010 con 0x7f743c159ea0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c04efa0 con 0x7f743c159ea0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f743401a070 con 0x7f743c159ea0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f7432ffd640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f743a59c640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7434085040 con 0x7f743c159ea0 2024-04-25T10:02:31.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.612+0000 7f743a59c640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 0x7f741c069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f742c0ae000 tx=0x7f742c0a6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.616+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7434055da0 con 0x7f743c159ea0 2024-04-25T10:02:31.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.760+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f743c06e6b0 con 0x7f741c066f60 2024-04-25T10:02:31.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.784+0000 7f7432ffd640 1 -- 172.21.15.82:0/3041524599 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f743c06e6b0 con 0x7f741c066f60 2024-04-25T10:02:31.794 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:31.796 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 msgr2=0x7f741c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 0x7f741c069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f742c0ae000 tx=0x7f742c0a6040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 msgr2=0x7f743c1529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f743400cad0 tx=0x7f743400cf90 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 shutdown_connections 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f741c066f60 0x7f741c069420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f743c0b3200 0x7f743c14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f743c159ea0 0x7f743c1529f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 --2- 172.21.15.82:0/3041524599 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f743c0b2e20 0x7f743c157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 >> 172.21.15.82:0/3041524599 conn(0x7f743c0b11a0 msgr2=0x7f743c0cb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 shutdown_connections 2024-04-25T10:02:31.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.788+0000 7f74409bc640 1 -- 172.21.15.82:0/3041524599 wait complete. 2024-04-25T10:02:31.810 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T10:02:31.266708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223632,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751088,"statfs":{"total":773094113280,"available":772865114112,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003933"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462685+0000","last_change":"2024-04-25T10:02:06.377944+0000","last_active":"2024-04-25T10:02:13.462685+0000","last_peered":"2024-04-25T10:02:13.462685+0000","last_clean":"2024-04-25T10:02:13.462685+0000","last_became_active":"2024-04-25T10:02:06.377162+0000","last_became_peered":"2024-04-25T10:02:06.377162+0000","last_unstale":"2024-04-25T10:02:13.462685+0000","last_undegraded":"2024-04-25T10:02:13.462685+0000","last_fullsized":"2024-04-25T10:02:13.462685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406752+0000","last_change":"2024-04-25T10:02:07.375159+0000","last_active":"2024-04-25T10:02:12.406752+0000","last_peered":"2024-04-25T10:02:12.406752+0000","last_clean":"2024-04-25T10:02:12.406752+0000","last_became_active":"2024-04-25T10:02:07.374938+0000","last_became_peered":"2024-04-25T10:02:07.374938+0000","last_unstale":"2024-04-25T10:02:12.406752+0000","last_undegraded":"2024-04-25T10:02:12.406752+0000","last_fullsized":"2024-04-25T10:02:12.406752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404043+0000","last_change":"2024-04-25T10:02:11.459079+0000","last_active":"2024-04-25T10:02:12.404043+0000","last_peered":"2024-04-25T10:02:12.404043+0000","last_clean":"2024-04-25T10:02:12.404043+0000","last_became_active":"2024-04-25T10:02:11.458605+0000","last_became_peered":"2024-04-25T10:02:11.458605+0000","last_unstale":"2024-04-25T10:02:12.404043+0000","last_undegraded":"2024-04-25T10:02:12.404043+0000","last_fullsized":"2024-04-25T10:02:12.404043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407158+0000","last_change":"2024-04-25T10:02:06.377863+0000","last_active":"2024-04-25T10:02:12.407158+0000","last_peered":"2024-04-25T10:02:12.407158+0000","last_clean":"2024-04-25T10:02:12.407158+0000","last_became_active":"2024-04-25T10:02:06.377260+0000","last_became_peered":"2024-04-25T10:02:06.377260+0000","last_unstale":"2024-04-25T10:02:12.407158+0000","last_undegraded":"2024-04-25T10:02:12.407158+0000","last_fullsized":"2024-04-25T10:02:12.407158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462420+0000","last_change":"2024-04-25T10:02:07.371882+0000","last_active":"2024-04-25T10:02:13.462420+0000","last_peered":"2024-04-25T10:02:13.462420+0000","last_clean":"2024-04-25T10:02:13.462420+0000","last_became_active":"2024-04-25T10:02:07.371244+0000","last_became_peered":"2024-04-25T10:02:07.371244+0000","last_unstale":"2024-04-25T10:02:13.462420+0000","last_undegraded":"2024-04-25T10:02:13.462420+0000","last_fullsized":"2024-04-25T10:02:13.462420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405866+0000","last_change":"2024-04-25T10:02:11.435335+0000","last_active":"2024-04-25T10:02:12.405866+0000","last_peered":"2024-04-25T10:02:12.405866+0000","last_clean":"2024-04-25T10:02:12.405866+0000","last_became_active":"2024-04-25T10:02:11.434197+0000","last_became_peered":"2024-04-25T10:02:11.434197+0000","last_unstale":"2024-04-25T10:02:12.405866+0000","last_undegraded":"2024-04-25T10:02:12.405866+0000","last_fullsized":"2024-04-25T10:02:12.405866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:43.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465851+0000","last_change":"2024-04-25T10:02:06.374878+0000","last_active":"2024-04-25T10:02:13.465851+0000","last_peered":"2024-04-25T10:02:13.465851+0000","last_clean":"2024-04-25T10:02:13.465851+0000","last_became_active":"2024-04-25T10:02:06.374210+0000","last_became_peered":"2024-04-25T10:02:06.374210+0000","last_unstale":"2024-04-25T10:02:13.465851+0000","last_undegraded":"2024-04-25T10:02:13.465851+0000","last_fullsized":"2024-04-25T10:02:13.465851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465104+0000","last_change":"2024-04-25T10:02:07.382381+0000","last_active":"2024-04-25T10:02:13.465104+0000","last_peered":"2024-04-25T10:02:13.465104+0000","last_clean":"2024-04-25T10:02:13.465104+0000","last_became_active":"2024-04-25T10:02:07.381718+0000","last_became_peered":"2024-04-25T10:02:07.381718+0000","last_unstale":"2024-04-25T10:02:13.465104+0000","last_undegraded":"2024-04-25T10:02:13.465104+0000","last_fullsized":"2024-04-25T10:02:13.465104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407612+0000","last_change":"2024-04-25T10:02:11.442974+0000","last_active":"2024-04-25T10:02:12.407612+0000","last_peered":"2024-04-25T10:02:12.407612+0000","last_clean":"2024-04-25T10:02:12.407612+0000","last_became_active":"2024-04-25T10:02:11.441674+0000","last_became_peered":"2024-04-25T10:02:11.441674+0000","last_unstale":"2024-04-25T10:02:12.407612+0000","last_undegraded":"2024-04-25T10:02:12.407612+0000","last_fullsized":"2024-04-25T10:02:12.407612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464230+0000","last_change":"2024-04-25T10:02:06.388913+0000","last_active":"2024-04-25T10:02:13.464230+0000","last_peered":"2024-04-25T10:02:13.464230+0000","last_clean":"2024-04-25T10:02:13.464230+0000","last_became_active":"2024-04-25T10:02:06.388536+0000","last_became_peered":"2024-04-25T10:02:06.388536+0000","last_unstale":"2024-04-25T10:02:13.464230+0000","last_undegraded":"2024-04-25T10:02:13.464230+0000","last_fullsized":"2024-04-25T10:02:13.464230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:41.104882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465595+0000","last_change":"2024-04-25T10:02:07.382060+0000","last_active":"2024-04-25T10:02:13.465595+0000","last_peered":"2024-04-25T10:02:13.465595+0000","last_clean":"2024-04-25T10:02:13.465595+0000","last_became_active":"2024-04-25T10:02:07.381779+0000","last_became_peered":"2024-04-25T10:02:07.381779+0000","last_unstale":"2024-04-25T10:02:13.465595+0000","last_undegraded":"2024-04-25T10:02:13.465595+0000","last_fullsized":"2024-04-25T10:02:13.465595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406133+0000","last_change":"2024-04-25T10:02:11.430607+0000","last_active":"2024-04-25T10:02:12.406133+0000","last_peered":"2024-04-25T10:02:12.406133+0000","last_clean":"2024-04-25T10:02:12.406133+0000","last_became_active":"2024-04-25T10:02:11.429281+0000","last_became_peered":"2024-04-25T10:02:11.429281+0000","last_unstale":"2024-04-25T10:02:12.406133+0000","last_undegraded":"2024-04-25T10:02:12.406133+0000","last_fullsized":"2024-04-25T10:02:12.406133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:06.213948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405179+0000","last_change":"2024-04-25T10:02:07.365923+0000","last_active":"2024-04-25T10:02:12.405179+0000","last_peered":"2024-04-25T10:02:12.405179+0000","last_clean":"2024-04-25T10:02:12.405179+0000","last_became_active":"2024-04-25T10:02:07.365345+0000","last_became_peered":"2024-04-25T10:02:07.365345+0000","last_unstale":"2024-04-25T10:02:12.405179+0000","last_undegraded":"2024-04-25T10:02:12.405179+0000","last_fullsized":"2024-04-25T10:02:12.405179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465328+0000","last_change":"2024-04-25T10:02:06.373663+0000","last_active":"2024-04-25T10:02:13.465328+0000","last_peered":"2024-04-25T10:02:13.465328+0000","last_clean":"2024-04-25T10:02:13.465328+0000","last_became_active":"2024-04-25T10:02:06.372972+0000","last_became_peered":"2024-04-25T10:02:06.372972+0000","last_unstale":"2024-04-25T10:02:13.465328+0000","last_undegraded":"2024-04-25T10:02:13.465328+0000","last_fullsized":"2024-04-25T10:02:13.465328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408366+0000","last_change":"2024-04-25T10:02:11.432916+0000","last_active":"2024-04-25T10:02:12.408366+0000","last_peered":"2024-04-25T10:02:12.408366+0000","last_clean":"2024-04-25T10:02:12.408366+0000","last_became_active":"2024-04-25T10:02:11.431758+0000","last_became_peered":"2024-04-25T10:02:11.431758+0000","last_unstale":"2024-04-25T10:02:12.408366+0000","last_undegraded":"2024-04-25T10:02:12.408366+0000","last_fullsized":"2024-04-25T10:02:12.408366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403824+0000","last_change":"2024-04-25T10:02:07.371074+0000","last_active":"2024-04-25T10:02:12.403824+0000","last_peered":"2024-04-25T10:02:12.403824+0000","last_clean":"2024-04-25T10:02:12.403824+0000","last_became_active":"2024-04-25T10:02:07.370770+0000","last_became_peered":"2024-04-25T10:02:07.370770+0000","last_unstale":"2024-04-25T10:02:12.403824+0000","last_undegraded":"2024-04-25T10:02:12.403824+0000","last_fullsized":"2024-04-25T10:02:12.403824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462663+0000","last_change":"2024-04-25T10:02:06.369892+0000","last_active":"2024-04-25T10:02:13.462663+0000","last_peered":"2024-04-25T10:02:13.462663+0000","last_clean":"2024-04-25T10:02:13.462663+0000","last_became_active":"2024-04-25T10:02:06.369183+0000","last_became_peered":"2024-04-25T10:02:06.369183+0000","last_unstale":"2024-04-25T10:02:13.462663+0000","last_undegraded":"2024-04-25T10:02:13.462663+0000","last_fullsized":"2024-04-25T10:02:13.462663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406907+0000","last_change":"2024-04-25T10:02:11.442628+0000","last_active":"2024-04-25T10:02:12.406907+0000","last_peered":"2024-04-25T10:02:12.406907+0000","last_clean":"2024-04-25T10:02:12.406907+0000","last_became_active":"2024-04-25T10:02:11.441578+0000","last_became_peered":"2024-04-25T10:02:11.441578+0000","last_unstale":"2024-04-25T10:02:12.406907+0000","last_undegraded":"2024-04-25T10:02:12.406907+0000","last_fullsized":"2024-04-25T10:02:12.406907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:00.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403861+0000","last_change":"2024-04-25T10:02:07.361735+0000","last_active":"2024-04-25T10:02:12.403861+0000","last_peered":"2024-04-25T10:02:12.403861+0000","last_clean":"2024-04-25T10:02:12.403861+0000","last_became_active":"2024-04-25T10:02:07.361435+0000","last_became_peered":"2024-04-25T10:02:07.361435+0000","last_unstale":"2024-04-25T10:02:12.403861+0000","last_undegraded":"2024-04-25T10:02:12.403861+0000","last_fullsized":"2024-04-25T10:02:12.403861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461678+0000","last_change":"2024-04-25T10:02:06.369621+0000","last_active":"2024-04-25T10:02:13.461678+0000","last_peered":"2024-04-25T10:02:13.461678+0000","last_clean":"2024-04-25T10:02:13.461678+0000","last_became_active":"2024-04-25T10:02:06.368883+0000","last_became_peered":"2024-04-25T10:02:06.368883+0000","last_unstale":"2024-04-25T10:02:13.461678+0000","last_undegraded":"2024-04-25T10:02:13.461678+0000","last_fullsized":"2024-04-25T10:02:13.461678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462909+0000","last_change":"2024-04-25T10:02:11.462237+0000","last_active":"2024-04-25T10:02:13.462909+0000","last_peered":"2024-04-25T10:02:13.462909+0000","last_clean":"2024-04-25T10:02:13.462909+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.462909+0000","last_undegraded":"2024-04-25T10:02:13.462909+0000","last_fullsized":"2024-04-25T10:02:13.462909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406991+0000","last_change":"2024-04-25T10:02:07.357629+0000","last_active":"2024-04-25T10:02:12.406991+0000","last_peered":"2024-04-25T10:02:12.406991+0000","last_clean":"2024-04-25T10:02:12.406991+0000","last_became_active":"2024-04-25T10:02:07.357353+0000","last_became_peered":"2024-04-25T10:02:07.357353+0000","last_unstale":"2024-04-25T10:02:12.406991+0000","last_undegraded":"2024-04-25T10:02:12.406991+0000","last_fullsized":"2024-04-25T10:02:12.406991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:17.663068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464640+0000","last_change":"2024-04-25T10:02:06.384551+0000","last_active":"2024-04-25T10:02:13.464640+0000","last_peered":"2024-04-25T10:02:13.464640+0000","last_clean":"2024-04-25T10:02:13.464640+0000","last_became_active":"2024-04-25T10:02:06.384131+0000","last_became_peered":"2024-04-25T10:02:06.384131+0000","last_unstale":"2024-04-25T10:02:13.464640+0000","last_undegraded":"2024-04-25T10:02:13.464640+0000","last_fullsized":"2024-04-25T10:02:13.464640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408508+0000","last_change":"2024-04-25T10:02:11.443507+0000","last_active":"2024-04-25T10:02:12.408508+0000","last_peered":"2024-04-25T10:02:12.408508+0000","last_clean":"2024-04-25T10:02:12.408508+0000","last_became_active":"2024-04-25T10:02:11.440496+0000","last_became_peered":"2024-04-25T10:02:11.440496+0000","last_unstale":"2024-04-25T10:02:12.408508+0000","last_undegraded":"2024-04-25T10:02:12.408508+0000","last_fullsized":"2024-04-25T10:02:12.408508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462520+0000","last_change":"2024-04-25T10:02:07.367892+0000","last_active":"2024-04-25T10:02:13.462520+0000","last_peered":"2024-04-25T10:02:13.462520+0000","last_clean":"2024-04-25T10:02:13.462520+0000","last_became_active":"2024-04-25T10:02:07.367404+0000","last_became_peered":"2024-04-25T10:02:07.367404+0000","last_unstale":"2024-04-25T10:02:13.462520+0000","last_undegraded":"2024-04-25T10:02:13.462520+0000","last_fullsized":"2024-04-25T10:02:13.462520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406984+0000","last_change":"2024-04-25T10:02:06.371009+0000","last_active":"2024-04-25T10:02:12.406984+0000","last_peered":"2024-04-25T10:02:12.406984+0000","last_clean":"2024-04-25T10:02:12.406984+0000","last_became_active":"2024-04-25T10:02:06.369415+0000","last_became_peered":"2024-04-25T10:02:06.369415+0000","last_unstale":"2024-04-25T10:02:12.406984+0000","last_undegraded":"2024-04-25T10:02:12.406984+0000","last_fullsized":"2024-04-25T10:02:12.406984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406594+0000","last_change":"2024-04-25T10:02:11.460234+0000","last_active":"2024-04-25T10:02:12.406594+0000","last_peered":"2024-04-25T10:02:12.406594+0000","last_clean":"2024-04-25T10:02:12.406594+0000","last_became_active":"2024-04-25T10:02:11.459268+0000","last_became_peered":"2024-04-25T10:02:11.459268+0000","last_unstale":"2024-04-25T10:02:12.406594+0000","last_undegraded":"2024-04-25T10:02:12.406594+0000","last_fullsized":"2024-04-25T10:02:12.406594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:35.022355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406871+0000","last_change":"2024-04-25T10:02:07.385204+0000","last_active":"2024-04-25T10:02:12.406871+0000","last_peered":"2024-04-25T10:02:12.406871+0000","last_clean":"2024-04-25T10:02:12.406871+0000","last_became_active":"2024-04-25T10:02:07.384798+0000","last_became_peered":"2024-04-25T10:02:07.384798+0000","last_unstale":"2024-04-25T10:02:12.406871+0000","last_undegraded":"2024-04-25T10:02:12.406871+0000","last_fullsized":"2024-04-25T10:02:12.406871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462229+0000","last_change":"2024-04-25T10:02:06.363593+0000","last_active":"2024-04-25T10:02:13.462229+0000","last_peered":"2024-04-25T10:02:13.462229+0000","last_clean":"2024-04-25T10:02:13.462229+0000","last_became_active":"2024-04-25T10:02:06.363127+0000","last_became_peered":"2024-04-25T10:02:06.363127+0000","last_unstale":"2024-04-25T10:02:13.462229+0000","last_undegraded":"2024-04-25T10:02:13.462229+0000","last_fullsized":"2024-04-25T10:02:13.462229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403605+0000","last_change":"2024-04-25T10:02:11.453752+0000","last_active":"2024-04-25T10:02:12.403605+0000","last_peered":"2024-04-25T10:02:12.403605+0000","last_clean":"2024-04-25T10:02:12.403605+0000","last_became_active":"2024-04-25T10:02:11.453254+0000","last_became_peered":"2024-04-25T10:02:11.453254+0000","last_unstale":"2024-04-25T10:02:12.403605+0000","last_undegraded":"2024-04-25T10:02:12.403605+0000","last_fullsized":"2024-04-25T10:02:12.403605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:04.566083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462068+0000","last_change":"2024-04-25T10:02:07.356096+0000","last_active":"2024-04-25T10:02:13.462068+0000","last_peered":"2024-04-25T10:02:13.462068+0000","last_clean":"2024-04-25T10:02:13.462068+0000","last_became_active":"2024-04-25T10:02:07.355433+0000","last_became_peered":"2024-04-25T10:02:07.355433+0000","last_unstale":"2024-04-25T10:02:13.462068+0000","last_undegraded":"2024-04-25T10:02:13.462068+0000","last_fullsized":"2024-04-25T10:02:13.462068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466154+0000","last_change":"2024-04-25T10:02:06.385234+0000","last_active":"2024-04-25T10:02:13.466154+0000","last_peered":"2024-04-25T10:02:13.466154+0000","last_clean":"2024-04-25T10:02:13.466154+0000","last_became_active":"2024-04-25T10:02:06.384494+0000","last_became_peered":"2024-04-25T10:02:06.384494+0000","last_unstale":"2024-04-25T10:02:13.466154+0000","last_undegraded":"2024-04-25T10:02:13.466154+0000","last_fullsized":"2024-04-25T10:02:13.466154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462761+0000","last_change":"2024-04-25T10:02:11.459695+0000","last_active":"2024-04-25T10:02:13.462761+0000","last_peered":"2024-04-25T10:02:13.462761+0000","last_clean":"2024-04-25T10:02:13.462761+0000","last_became_active":"2024-04-25T10:02:11.459247+0000","last_became_peered":"2024-04-25T10:02:11.459247+0000","last_unstale":"2024-04-25T10:02:13.462761+0000","last_undegraded":"2024-04-25T10:02:13.462761+0000","last_fullsized":"2024-04-25T10:02:13.462761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465853+0000","last_change":"2024-04-25T10:02:07.374402+0000","last_active":"2024-04-25T10:02:13.465853+0000","last_peered":"2024-04-25T10:02:13.465853+0000","last_clean":"2024-04-25T10:02:13.465853+0000","last_became_active":"2024-04-25T10:02:07.374110+0000","last_became_peered":"2024-04-25T10:02:07.374110+0000","last_unstale":"2024-04-25T10:02:13.465853+0000","last_undegraded":"2024-04-25T10:02:13.465853+0000","last_fullsized":"2024-04-25T10:02:13.465853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462182+0000","last_change":"2024-04-25T10:02:06.368581+0000","last_active":"2024-04-25T10:02:13.462182+0000","last_peered":"2024-04-25T10:02:13.462182+0000","last_clean":"2024-04-25T10:02:13.462182+0000","last_became_active":"2024-04-25T10:02:06.368085+0000","last_became_peered":"2024-04-25T10:02:06.368085+0000","last_unstale":"2024-04-25T10:02:13.462182+0000","last_undegraded":"2024-04-25T10:02:13.462182+0000","last_fullsized":"2024-04-25T10:02:13.462182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461807+0000","last_change":"2024-04-25T10:02:11.435403+0000","last_active":"2024-04-25T10:02:13.461807+0000","last_peered":"2024-04-25T10:02:13.461807+0000","last_clean":"2024-04-25T10:02:13.461807+0000","last_became_active":"2024-04-25T10:02:11.435010+0000","last_became_peered":"2024-04-25T10:02:11.435010+0000","last_unstale":"2024-04-25T10:02:13.461807+0000","last_undegraded":"2024-04-25T10:02:13.461807+0000","last_fullsized":"2024-04-25T10:02:13.461807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407792+0000","last_change":"2024-04-25T10:02:07.378715+0000","last_active":"2024-04-25T10:02:12.407792+0000","last_peered":"2024-04-25T10:02:12.407792+0000","last_clean":"2024-04-25T10:02:12.407792+0000","last_became_active":"2024-04-25T10:02:07.378297+0000","last_became_peered":"2024-04-25T10:02:07.378297+0000","last_unstale":"2024-04-25T10:02:12.407792+0000","last_undegraded":"2024-04-25T10:02:12.407792+0000","last_fullsized":"2024-04-25T10:02:12.407792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465696+0000","last_change":"2024-04-25T10:02:06.381259+0000","last_active":"2024-04-25T10:02:13.465696+0000","last_peered":"2024-04-25T10:02:13.465696+0000","last_clean":"2024-04-25T10:02:13.465696+0000","last_became_active":"2024-04-25T10:02:06.377864+0000","last_became_peered":"2024-04-25T10:02:06.377864+0000","last_unstale":"2024-04-25T10:02:13.465696+0000","last_undegraded":"2024-04-25T10:02:13.465696+0000","last_fullsized":"2024-04-25T10:02:13.465696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462019+0000","last_change":"2024-04-25T10:02:11.435443+0000","last_active":"2024-04-25T10:02:13.462019+0000","last_peered":"2024-04-25T10:02:13.462019+0000","last_clean":"2024-04-25T10:02:13.462019+0000","last_became_active":"2024-04-25T10:02:11.434691+0000","last_became_peered":"2024-04-25T10:02:11.434691+0000","last_unstale":"2024-04-25T10:02:13.462019+0000","last_undegraded":"2024-04-25T10:02:13.462019+0000","last_fullsized":"2024-04-25T10:02:13.462019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462938+0000","last_change":"2024-04-25T10:02:07.360180+0000","last_active":"2024-04-25T10:02:13.462938+0000","last_peered":"2024-04-25T10:02:13.462938+0000","last_clean":"2024-04-25T10:02:13.462938+0000","last_became_active":"2024-04-25T10:02:07.358876+0000","last_became_peered":"2024-04-25T10:02:07.358876+0000","last_unstale":"2024-04-25T10:02:13.462938+0000","last_undegraded":"2024-04-25T10:02:13.462938+0000","last_fullsized":"2024-04-25T10:02:13.462938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462463+0000","last_change":"2024-04-25T10:02:06.376414+0000","last_active":"2024-04-25T10:02:13.462463+0000","last_peered":"2024-04-25T10:02:13.462463+0000","last_clean":"2024-04-25T10:02:13.462463+0000","last_became_active":"2024-04-25T10:02:06.375943+0000","last_became_peered":"2024-04-25T10:02:06.375943+0000","last_unstale":"2024-04-25T10:02:13.462463+0000","last_undegraded":"2024-04-25T10:02:13.462463+0000","last_fullsized":"2024-04-25T10:02:13.462463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463033+0000","last_change":"2024-04-25T10:02:11.438382+0000","last_active":"2024-04-25T10:02:13.463033+0000","last_peered":"2024-04-25T10:02:13.463033+0000","last_clean":"2024-04-25T10:02:13.463033+0000","last_became_active":"2024-04-25T10:02:11.436135+0000","last_became_peered":"2024-04-25T10:02:11.436135+0000","last_unstale":"2024-04-25T10:02:13.463033+0000","last_undegraded":"2024-04-25T10:02:13.463033+0000","last_fullsized":"2024-04-25T10:02:13.463033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.460832+0000","last_change":"2024-04-25T10:02:07.366816+0000","last_active":"2024-04-25T10:02:13.460832+0000","last_peered":"2024-04-25T10:02:13.460832+0000","last_clean":"2024-04-25T10:02:13.460832+0000","last_became_active":"2024-04-25T10:02:07.366193+0000","last_became_peered":"2024-04-25T10:02:07.366193+0000","last_unstale":"2024-04-25T10:02:13.460832+0000","last_undegraded":"2024-04-25T10:02:13.460832+0000","last_fullsized":"2024-04-25T10:02:13.460832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463419+0000","last_change":"2024-04-25T10:02:06.376427+0000","last_active":"2024-04-25T10:02:13.463419+0000","last_peered":"2024-04-25T10:02:13.463419+0000","last_clean":"2024-04-25T10:02:13.463419+0000","last_became_active":"2024-04-25T10:02:06.375955+0000","last_became_peered":"2024-04-25T10:02:06.375955+0000","last_unstale":"2024-04-25T10:02:13.463419+0000","last_undegraded":"2024-04-25T10:02:13.463419+0000","last_fullsized":"2024-04-25T10:02:13.463419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462868+0000","last_change":"2024-04-25T10:02:11.458524+0000","last_active":"2024-04-25T10:02:13.462868+0000","last_peered":"2024-04-25T10:02:13.462868+0000","last_clean":"2024-04-25T10:02:13.462868+0000","last_became_active":"2024-04-25T10:02:11.458105+0000","last_became_peered":"2024-04-25T10:02:11.458105+0000","last_unstale":"2024-04-25T10:02:13.462868+0000","last_undegraded":"2024-04-25T10:02:13.462868+0000","last_fullsized":"2024-04-25T10:02:13.462868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403806+0000","last_change":"2024-04-25T10:02:07.369247+0000","last_active":"2024-04-25T10:02:12.403806+0000","last_peered":"2024-04-25T10:02:12.403806+0000","last_clean":"2024-04-25T10:02:12.403806+0000","last_became_active":"2024-04-25T10:02:07.368616+0000","last_became_peered":"2024-04-25T10:02:07.368616+0000","last_unstale":"2024-04-25T10:02:12.403806+0000","last_undegraded":"2024-04-25T10:02:12.403806+0000","last_fullsized":"2024-04-25T10:02:12.403806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466462+0000","last_change":"2024-04-25T10:02:06.385925+0000","last_active":"2024-04-25T10:02:13.466462+0000","last_peered":"2024-04-25T10:02:13.466462+0000","last_clean":"2024-04-25T10:02:13.466462+0000","last_became_active":"2024-04-25T10:02:06.382216+0000","last_became_peered":"2024-04-25T10:02:06.382216+0000","last_unstale":"2024-04-25T10:02:13.466462+0000","last_undegraded":"2024-04-25T10:02:13.466462+0000","last_fullsized":"2024-04-25T10:02:13.466462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463713+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:02:13.463713+0000","last_peered":"2024-04-25T10:02:13.463713+0000","last_clean":"2024-04-25T10:02:13.463713+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:02:13.463713+0000","last_undegraded":"2024-04-25T10:02:13.463713+0000","last_fullsized":"2024-04-25T10:02:13.463713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406156+0000","last_change":"2024-04-25T10:02:11.460389+0000","last_active":"2024-04-25T10:02:12.406156+0000","last_peered":"2024-04-25T10:02:12.406156+0000","last_clean":"2024-04-25T10:02:12.406156+0000","last_became_active":"2024-04-25T10:02:11.459440+0000","last_became_peered":"2024-04-25T10:02:11.459440+0000","last_unstale":"2024-04-25T10:02:12.406156+0000","last_undegraded":"2024-04-25T10:02:12.406156+0000","last_fullsized":"2024-04-25T10:02:12.406156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465341+0000","last_change":"2024-04-25T10:02:07.388181+0000","last_active":"2024-04-25T10:02:13.465341+0000","last_peered":"2024-04-25T10:02:13.465341+0000","last_clean":"2024-04-25T10:02:13.465341+0000","last_became_active":"2024-04-25T10:02:07.385106+0000","last_became_peered":"2024-04-25T10:02:07.385106+0000","last_unstale":"2024-04-25T10:02:13.465341+0000","last_undegraded":"2024-04-25T10:02:13.465341+0000","last_fullsized":"2024-04-25T10:02:13.465341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463240+0000","last_change":"2024-04-25T10:02:06.386894+0000","last_active":"2024-04-25T10:02:13.463240+0000","last_peered":"2024-04-25T10:02:13.463240+0000","last_clean":"2024-04-25T10:02:13.463240+0000","last_became_active":"2024-04-25T10:02:06.386546+0000","last_became_peered":"2024-04-25T10:02:06.386546+0000","last_unstale":"2024-04-25T10:02:13.463240+0000","last_undegraded":"2024-04-25T10:02:13.463240+0000","last_fullsized":"2024-04-25T10:02:13.463240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405476+0000","last_change":"2024-04-25T10:02:11.455613+0000","last_active":"2024-04-25T10:02:12.405476+0000","last_peered":"2024-04-25T10:02:12.405476+0000","last_clean":"2024-04-25T10:02:12.405476+0000","last_became_active":"2024-04-25T10:02:11.454401+0000","last_became_peered":"2024-04-25T10:02:11.454401+0000","last_unstale":"2024-04-25T10:02:12.405476+0000","last_undegraded":"2024-04-25T10:02:12.405476+0000","last_fullsized":"2024-04-25T10:02:12.405476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465364+0000","last_change":"2024-04-25T10:02:07.382626+0000","last_active":"2024-04-25T10:02:13.465364+0000","last_peered":"2024-04-25T10:02:13.465364+0000","last_clean":"2024-04-25T10:02:13.465364+0000","last_became_active":"2024-04-25T10:02:07.382110+0000","last_became_peered":"2024-04-25T10:02:07.382110+0000","last_unstale":"2024-04-25T10:02:13.465364+0000","last_undegraded":"2024-04-25T10:02:13.465364+0000","last_fullsized":"2024-04-25T10:02:13.465364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461856+0000","last_change":"2024-04-25T10:02:06.361762+0000","last_active":"2024-04-25T10:02:13.461856+0000","last_peered":"2024-04-25T10:02:13.461856+0000","last_clean":"2024-04-25T10:02:13.461856+0000","last_became_active":"2024-04-25T10:02:06.360531+0000","last_became_peered":"2024-04-25T10:02:06.360531+0000","last_unstale":"2024-04-25T10:02:13.461856+0000","last_undegraded":"2024-04-25T10:02:13.461856+0000","last_fullsized":"2024-04-25T10:02:13.461856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463028+0000","last_change":"2024-04-25T10:02:11.435880+0000","last_active":"2024-04-25T10:02:13.463028+0000","last_peered":"2024-04-25T10:02:13.463028+0000","last_clean":"2024-04-25T10:02:13.463028+0000","last_became_active":"2024-04-25T10:02:11.435244+0000","last_became_peered":"2024-04-25T10:02:11.435244+0000","last_unstale":"2024-04-25T10:02:13.463028+0000","last_undegraded":"2024-04-25T10:02:13.463028+0000","last_fullsized":"2024-04-25T10:02:13.463028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466834+0000","last_change":"2024-04-25T10:02:07.385486+0000","last_active":"2024-04-25T10:02:13.466834+0000","last_peered":"2024-04-25T10:02:13.466834+0000","last_clean":"2024-04-25T10:02:13.466834+0000","last_became_active":"2024-04-25T10:02:07.385202+0000","last_became_peered":"2024-04-25T10:02:07.385202+0000","last_unstale":"2024-04-25T10:02:13.466834+0000","last_undegraded":"2024-04-25T10:02:13.466834+0000","last_fullsized":"2024-04-25T10:02:13.466834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:16.103362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462441+0000","last_change":"2024-04-25T10:02:06.379591+0000","last_active":"2024-04-25T10:02:13.462441+0000","last_peered":"2024-04-25T10:02:13.462441+0000","last_clean":"2024-04-25T10:02:13.462441+0000","last_became_active":"2024-04-25T10:02:06.379178+0000","last_became_peered":"2024-04-25T10:02:06.379178+0000","last_unstale":"2024-04-25T10:02:13.462441+0000","last_undegraded":"2024-04-25T10:02:13.462441+0000","last_fullsized":"2024-04-25T10:02:13.462441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463716+0000","last_change":"2024-04-25T10:02:11.459834+0000","last_active":"2024-04-25T10:02:13.463716+0000","last_peered":"2024-04-25T10:02:13.463716+0000","last_clean":"2024-04-25T10:02:13.463716+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.463716+0000","last_undegraded":"2024-04-25T10:02:13.463716+0000","last_fullsized":"2024-04-25T10:02:13.463716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466278+0000","last_change":"2024-04-25T10:02:07.387540+0000","last_active":"2024-04-25T10:02:13.466278+0000","last_peered":"2024-04-25T10:02:13.466278+0000","last_clean":"2024-04-25T10:02:13.466278+0000","last_became_active":"2024-04-25T10:02:07.385395+0000","last_became_peered":"2024-04-25T10:02:07.385395+0000","last_unstale":"2024-04-25T10:02:13.466278+0000","last_undegraded":"2024-04-25T10:02:13.466278+0000","last_fullsized":"2024-04-25T10:02:13.466278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408935+0000","last_change":"2024-04-25T10:02:06.379749+0000","last_active":"2024-04-25T10:02:12.408935+0000","last_peered":"2024-04-25T10:02:12.408935+0000","last_clean":"2024-04-25T10:02:12.408935+0000","last_became_active":"2024-04-25T10:02:06.378947+0000","last_became_peered":"2024-04-25T10:02:06.378947+0000","last_unstale":"2024-04-25T10:02:12.408935+0000","last_undegraded":"2024-04-25T10:02:12.408935+0000","last_fullsized":"2024-04-25T10:02:12.408935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:03.055592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405719+0000","last_change":"2024-04-25T10:02:11.437602+0000","last_active":"2024-04-25T10:02:12.405719+0000","last_peered":"2024-04-25T10:02:12.405719+0000","last_clean":"2024-04-25T10:02:12.405719+0000","last_became_active":"2024-04-25T10:02:11.436553+0000","last_became_peered":"2024-04-25T10:02:11.436553+0000","last_unstale":"2024-04-25T10:02:12.405719+0000","last_undegraded":"2024-04-25T10:02:12.405719+0000","last_fullsized":"2024-04-25T10:02:12.405719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:53.618014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464059+0000","last_change":"2024-04-25T10:02:07.384411+0000","last_active":"2024-04-25T10:02:13.464059+0000","last_peered":"2024-04-25T10:02:13.464059+0000","last_clean":"2024-04-25T10:02:13.464059+0000","last_became_active":"2024-04-25T10:02:07.377814+0000","last_became_peered":"2024-04-25T10:02:07.377814+0000","last_unstale":"2024-04-25T10:02:13.464059+0000","last_undegraded":"2024-04-25T10:02:13.464059+0000","last_fullsized":"2024-04-25T10:02:13.464059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461988+0000","last_change":"2024-04-25T10:02:06.360702+0000","last_active":"2024-04-25T10:02:13.461988+0000","last_peered":"2024-04-25T10:02:13.461988+0000","last_clean":"2024-04-25T10:02:13.461988+0000","last_became_active":"2024-04-25T10:02:06.360297+0000","last_became_peered":"2024-04-25T10:02:06.360297+0000","last_unstale":"2024-04-25T10:02:13.461988+0000","last_undegraded":"2024-04-25T10:02:13.461988+0000","last_fullsized":"2024-04-25T10:02:13.461988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462271+0000","last_change":"2024-04-25T10:02:11.427201+0000","last_active":"2024-04-25T10:02:13.462271+0000","last_peered":"2024-04-25T10:02:13.462271+0000","last_clean":"2024-04-25T10:02:13.462271+0000","last_became_active":"2024-04-25T10:02:11.426734+0000","last_became_peered":"2024-04-25T10:02:11.426734+0000","last_unstale":"2024-04-25T10:02:13.462271+0000","last_undegraded":"2024-04-25T10:02:13.462271+0000","last_fullsized":"2024-04-25T10:02:13.462271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404061+0000","last_change":"2024-04-25T10:02:07.371003+0000","last_active":"2024-04-25T10:02:12.404061+0000","last_peered":"2024-04-25T10:02:12.404061+0000","last_clean":"2024-04-25T10:02:12.404061+0000","last_became_active":"2024-04-25T10:02:07.370695+0000","last_became_peered":"2024-04-25T10:02:07.370695+0000","last_unstale":"2024-04-25T10:02:12.404061+0000","last_undegraded":"2024-04-25T10:02:12.404061+0000","last_fullsized":"2024-04-25T10:02:12.404061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405106+0000","last_change":"2024-04-25T10:02:06.362100+0000","last_active":"2024-04-25T10:02:12.405106+0000","last_peered":"2024-04-25T10:02:12.405106+0000","last_clean":"2024-04-25T10:02:12.405106+0000","last_became_active":"2024-04-25T10:02:06.361358+0000","last_became_peered":"2024-04-25T10:02:06.361358+0000","last_unstale":"2024-04-25T10:02:12.405106+0000","last_undegraded":"2024-04-25T10:02:12.405106+0000","last_fullsized":"2024-04-25T10:02:12.405106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406644+0000","last_change":"2024-04-25T10:02:11.460013+0000","last_active":"2024-04-25T10:02:12.406644+0000","last_peered":"2024-04-25T10:02:12.406644+0000","last_clean":"2024-04-25T10:02:12.406644+0000","last_became_active":"2024-04-25T10:02:11.459491+0000","last_became_peered":"2024-04-25T10:02:11.459491+0000","last_unstale":"2024-04-25T10:02:12.406644+0000","last_undegraded":"2024-04-25T10:02:12.406644+0000","last_fullsized":"2024-04-25T10:02:12.406644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.241421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465085+0000","last_change":"2024-04-25T10:02:07.373359+0000","last_active":"2024-04-25T10:02:13.465085+0000","last_peered":"2024-04-25T10:02:13.465085+0000","last_clean":"2024-04-25T10:02:13.465085+0000","last_became_active":"2024-04-25T10:02:07.373100+0000","last_became_peered":"2024-04-25T10:02:07.373100+0000","last_unstale":"2024-04-25T10:02:13.465085+0000","last_undegraded":"2024-04-25T10:02:13.465085+0000","last_fullsized":"2024-04-25T10:02:13.465085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.957032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465425+0000","last_change":"2024-04-25T10:02:06.382032+0000","last_active":"2024-04-25T10:02:13.465425+0000","last_peered":"2024-04-25T10:02:13.465425+0000","last_clean":"2024-04-25T10:02:13.465425+0000","last_became_active":"2024-04-25T10:02:06.381711+0000","last_became_peered":"2024-04-25T10:02:06.381711+0000","last_unstale":"2024-04-25T10:02:13.465425+0000","last_undegraded":"2024-04-25T10:02:13.465425+0000","last_fullsized":"2024-04-25T10:02:13.465425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404352+0000","last_change":"2024-04-25T10:02:11.458829+0000","last_active":"2024-04-25T10:02:12.404352+0000","last_peered":"2024-04-25T10:02:12.404352+0000","last_clean":"2024-04-25T10:02:12.404352+0000","last_became_active":"2024-04-25T10:02:11.458386+0000","last_became_peered":"2024-04-25T10:02:11.458386+0000","last_unstale":"2024-04-25T10:02:12.404352+0000","last_undegraded":"2024-04-25T10:02:12.404352+0000","last_fullsized":"2024-04-25T10:02:12.404352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:15.902040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462813+0000","last_change":"2024-04-25T10:02:07.372352+0000","last_active":"2024-04-25T10:02:13.462813+0000","last_peered":"2024-04-25T10:02:13.462813+0000","last_clean":"2024-04-25T10:02:13.462813+0000","last_became_active":"2024-04-25T10:02:07.372124+0000","last_became_peered":"2024-04-25T10:02:07.372124+0000","last_unstale":"2024-04-25T10:02:13.462813+0000","last_undegraded":"2024-04-25T10:02:13.462813+0000","last_fullsized":"2024-04-25T10:02:13.462813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463253+0000","last_change":"2024-04-25T10:02:06.361119+0000","last_active":"2024-04-25T10:02:13.463253+0000","last_peered":"2024-04-25T10:02:13.463253+0000","last_clean":"2024-04-25T10:02:13.463253+0000","last_became_active":"2024-04-25T10:02:06.360459+0000","last_became_peered":"2024-04-25T10:02:06.360459+0000","last_unstale":"2024-04-25T10:02:13.463253+0000","last_undegraded":"2024-04-25T10:02:13.463253+0000","last_fullsized":"2024-04-25T10:02:13.463253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465848+0000","last_change":"2024-04-25T10:02:11.460123+0000","last_active":"2024-04-25T10:02:13.465848+0000","last_peered":"2024-04-25T10:02:13.465848+0000","last_clean":"2024-04-25T10:02:13.465848+0000","last_became_active":"2024-04-25T10:02:11.459152+0000","last_became_peered":"2024-04-25T10:02:11.459152+0000","last_unstale":"2024-04-25T10:02:13.465848+0000","last_undegraded":"2024-04-25T10:02:13.465848+0000","last_fullsized":"2024-04-25T10:02:13.465848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465062+0000","last_change":"2024-04-25T10:02:07.388512+0000","last_active":"2024-04-25T10:02:13.465062+0000","last_peered":"2024-04-25T10:02:13.465062+0000","last_clean":"2024-04-25T10:02:13.465062+0000","last_became_active":"2024-04-25T10:02:07.387797+0000","last_became_peered":"2024-04-25T10:02:07.387797+0000","last_unstale":"2024-04-25T10:02:13.465062+0000","last_undegraded":"2024-04-25T10:02:13.465062+0000","last_fullsized":"2024-04-25T10:02:13.465062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:17.622024+0000","last_change":"2024-04-25T10:02:06.370256+0000","last_active":"2024-04-25T10:02:17.622024+0000","last_peered":"2024-04-25T10:02:17.622024+0000","last_clean":"2024-04-25T10:02:17.622024+0000","last_became_active":"2024-04-25T10:02:06.368370+0000","last_became_peered":"2024-04-25T10:02:06.368370+0000","last_unstale":"2024-04-25T10:02:17.622024+0000","last_undegraded":"2024-04-25T10:02:17.622024+0000","last_fullsized":"2024-04-25T10:02:17.622024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462482+0000","last_change":"2024-04-25T10:02:11.438518+0000","last_active":"2024-04-25T10:02:13.462482+0000","last_peered":"2024-04-25T10:02:13.462482+0000","last_clean":"2024-04-25T10:02:13.462482+0000","last_became_active":"2024-04-25T10:02:11.436290+0000","last_became_peered":"2024-04-25T10:02:11.436290+0000","last_unstale":"2024-04-25T10:02:13.462482+0000","last_undegraded":"2024-04-25T10:02:13.462482+0000","last_fullsized":"2024-04-25T10:02:13.462482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407354+0000","last_change":"2024-04-25T10:02:07.358234+0000","last_active":"2024-04-25T10:02:12.407354+0000","last_peered":"2024-04-25T10:02:12.407354+0000","last_clean":"2024-04-25T10:02:12.407354+0000","last_became_active":"2024-04-25T10:02:07.357961+0000","last_became_peered":"2024-04-25T10:02:07.357961+0000","last_unstale":"2024-04-25T10:02:12.407354+0000","last_undegraded":"2024-04-25T10:02:12.407354+0000","last_fullsized":"2024-04-25T10:02:12.407354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461751+0000","last_change":"2024-04-25T10:02:06.386529+0000","last_active":"2024-04-25T10:02:13.461751+0000","last_peered":"2024-04-25T10:02:13.461751+0000","last_clean":"2024-04-25T10:02:13.461751+0000","last_became_active":"2024-04-25T10:02:06.386097+0000","last_became_peered":"2024-04-25T10:02:06.386097+0000","last_unstale":"2024-04-25T10:02:13.461751+0000","last_undegraded":"2024-04-25T10:02:13.461751+0000","last_fullsized":"2024-04-25T10:02:13.461751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407672+0000","last_change":"2024-04-25T10:02:11.437594+0000","last_active":"2024-04-25T10:02:12.407672+0000","last_peered":"2024-04-25T10:02:12.407672+0000","last_clean":"2024-04-25T10:02:12.407672+0000","last_became_active":"2024-04-25T10:02:11.436114+0000","last_became_peered":"2024-04-25T10:02:11.436114+0000","last_unstale":"2024-04-25T10:02:12.407672+0000","last_undegraded":"2024-04-25T10:02:12.407672+0000","last_fullsized":"2024-04-25T10:02:12.407672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462591+0000","last_change":"2024-04-25T10:02:07.371078+0000","last_active":"2024-04-25T10:02:13.462591+0000","last_peered":"2024-04-25T10:02:13.462591+0000","last_clean":"2024-04-25T10:02:13.462591+0000","last_became_active":"2024-04-25T10:02:07.370749+0000","last_became_peered":"2024-04-25T10:02:07.370749+0000","last_unstale":"2024-04-25T10:02:13.462591+0000","last_undegraded":"2024-04-25T10:02:13.462591+0000","last_fullsized":"2024-04-25T10:02:13.462591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466420+0000","last_change":"2024-04-25T10:02:06.377512+0000","last_active":"2024-04-25T10:02:13.466420+0000","last_peered":"2024-04-25T10:02:13.466420+0000","last_clean":"2024-04-25T10:02:13.466420+0000","last_became_active":"2024-04-25T10:02:06.376824+0000","last_became_peered":"2024-04-25T10:02:06.376824+0000","last_unstale":"2024-04-25T10:02:13.466420+0000","last_undegraded":"2024-04-25T10:02:13.466420+0000","last_fullsized":"2024-04-25T10:02:13.466420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403770+0000","last_change":"2024-04-25T10:02:11.442056+0000","last_active":"2024-04-25T10:02:12.403770+0000","last_peered":"2024-04-25T10:02:12.403770+0000","last_clean":"2024-04-25T10:02:12.403770+0000","last_became_active":"2024-04-25T10:02:11.441158+0000","last_became_peered":"2024-04-25T10:02:11.441158+0000","last_unstale":"2024-04-25T10:02:12.403770+0000","last_undegraded":"2024-04-25T10:02:12.403770+0000","last_fullsized":"2024-04-25T10:02:12.403770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466185+0000","last_change":"2024-04-25T10:02:07.389094+0000","last_active":"2024-04-25T10:02:13.466185+0000","last_peered":"2024-04-25T10:02:13.466185+0000","last_clean":"2024-04-25T10:02:13.466185+0000","last_became_active":"2024-04-25T10:02:07.388187+0000","last_became_peered":"2024-04-25T10:02:07.388187+0000","last_unstale":"2024-04-25T10:02:13.466185+0000","last_undegraded":"2024-04-25T10:02:13.466185+0000","last_fullsized":"2024-04-25T10:02:13.466185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406550+0000","last_change":"2024-04-25T10:02:06.383181+0000","last_active":"2024-04-25T10:02:12.406550+0000","last_peered":"2024-04-25T10:02:12.406550+0000","last_clean":"2024-04-25T10:02:12.406550+0000","last_became_active":"2024-04-25T10:02:06.382921+0000","last_became_peered":"2024-04-25T10:02:06.382921+0000","last_unstale":"2024-04-25T10:02:12.406550+0000","last_undegraded":"2024-04-25T10:02:12.406550+0000","last_fullsized":"2024-04-25T10:02:12.406550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404862+0000","last_change":"2024-04-25T10:02:11.454031+0000","last_active":"2024-04-25T10:02:12.404862+0000","last_peered":"2024-04-25T10:02:12.404862+0000","last_clean":"2024-04-25T10:02:12.404862+0000","last_became_active":"2024-04-25T10:02:11.453256+0000","last_became_peered":"2024-04-25T10:02:11.453256+0000","last_unstale":"2024-04-25T10:02:12.404862+0000","last_undegraded":"2024-04-25T10:02:12.404862+0000","last_fullsized":"2024-04-25T10:02:12.404862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:26.875452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462061+0000","last_change":"2024-04-25T10:02:11.460406+0000","last_active":"2024-04-25T10:02:13.462061+0000","last_peered":"2024-04-25T10:02:13.462061+0000","last_clean":"2024-04-25T10:02:13.462061+0000","last_became_active":"2024-04-25T10:02:11.459947+0000","last_became_peered":"2024-04-25T10:02:11.459947+0000","last_unstale":"2024-04-25T10:02:13.462061+0000","last_undegraded":"2024-04-25T10:02:13.462061+0000","last_fullsized":"2024-04-25T10:02:13.462061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461463+0000","last_change":"2024-04-25T10:02:07.361531+0000","last_active":"2024-04-25T10:02:13.461463+0000","last_peered":"2024-04-25T10:02:13.461463+0000","last_clean":"2024-04-25T10:02:13.461463+0000","last_became_active":"2024-04-25T10:02:07.361213+0000","last_became_peered":"2024-04-25T10:02:07.361213+0000","last_unstale":"2024-04-25T10:02:13.461463+0000","last_undegraded":"2024-04-25T10:02:13.461463+0000","last_fullsized":"2024-04-25T10:02:13.461463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464490+0000","last_change":"2024-04-25T10:02:06.386304+0000","last_active":"2024-04-25T10:02:13.464490+0000","last_peered":"2024-04-25T10:02:13.464490+0000","last_clean":"2024-04-25T10:02:13.464490+0000","last_became_active":"2024-04-25T10:02:06.382588+0000","last_became_peered":"2024-04-25T10:02:06.382588+0000","last_unstale":"2024-04-25T10:02:13.464490+0000","last_undegraded":"2024-04-25T10:02:13.464490+0000","last_fullsized":"2024-04-25T10:02:13.464490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406241+0000","last_change":"2024-04-25T10:02:11.433821+0000","last_active":"2024-04-25T10:02:12.406241+0000","last_peered":"2024-04-25T10:02:12.406241+0000","last_clean":"2024-04-25T10:02:12.406241+0000","last_became_active":"2024-04-25T10:02:11.432467+0000","last_became_peered":"2024-04-25T10:02:11.432467+0000","last_unstale":"2024-04-25T10:02:12.406241+0000","last_undegraded":"2024-04-25T10:02:12.406241+0000","last_fullsized":"2024-04-25T10:02:12.406241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403810+0000","last_change":"2024-04-25T10:02:07.376574+0000","last_active":"2024-04-25T10:02:12.403810+0000","last_peered":"2024-04-25T10:02:12.403810+0000","last_clean":"2024-04-25T10:02:12.403810+0000","last_became_active":"2024-04-25T10:02:07.376275+0000","last_became_peered":"2024-04-25T10:02:07.376275+0000","last_unstale":"2024-04-25T10:02:12.403810+0000","last_undegraded":"2024-04-25T10:02:12.403810+0000","last_fullsized":"2024-04-25T10:02:12.403810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405129+0000","last_change":"2024-04-25T10:02:06.363949+0000","last_active":"2024-04-25T10:02:12.405129+0000","last_peered":"2024-04-25T10:02:12.405129+0000","last_clean":"2024-04-25T10:02:12.405129+0000","last_became_active":"2024-04-25T10:02:06.363379+0000","last_became_peered":"2024-04-25T10:02:06.363379+0000","last_unstale":"2024-04-25T10:02:12.405129+0000","last_undegraded":"2024-04-25T10:02:12.405129+0000","last_fullsized":"2024-04-25T10:02:12.405129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405641+0000","last_change":"2024-04-25T10:02:06.361583+0000","last_active":"2024-04-25T10:02:12.405641+0000","last_peered":"2024-04-25T10:02:12.405641+0000","last_clean":"2024-04-25T10:02:12.405641+0000","last_became_active":"2024-04-25T10:02:06.360807+0000","last_became_peered":"2024-04-25T10:02:06.360807+0000","last_unstale":"2024-04-25T10:02:12.405641+0000","last_undegraded":"2024-04-25T10:02:12.405641+0000","last_fullsized":"2024-04-25T10:02:12.405641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463206+0000","last_change":"2024-04-25T10:02:07.367030+0000","last_active":"2024-04-25T10:02:13.463206+0000","last_peered":"2024-04-25T10:02:13.463206+0000","last_clean":"2024-04-25T10:02:13.463206+0000","last_became_active":"2024-04-25T10:02:07.366570+0000","last_became_peered":"2024-04-25T10:02:07.366570+0000","last_unstale":"2024-04-25T10:02:13.463206+0000","last_undegraded":"2024-04-25T10:02:13.463206+0000","last_fullsized":"2024-04-25T10:02:13.463206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463018+0000","last_change":"2024-04-25T10:02:11.434159+0000","last_active":"2024-04-25T10:02:13.463018+0000","last_peered":"2024-04-25T10:02:13.463018+0000","last_clean":"2024-04-25T10:02:13.463018+0000","last_became_active":"2024-04-25T10:02:11.433378+0000","last_became_peered":"2024-04-25T10:02:11.433378+0000","last_unstale":"2024-04-25T10:02:13.463018+0000","last_undegraded":"2024-04-25T10:02:13.463018+0000","last_fullsized":"2024-04-25T10:02:13.463018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:25.406090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403964+0000","last_change":"2024-04-25T10:02:06.375949+0000","last_active":"2024-04-25T10:02:12.403964+0000","last_peered":"2024-04-25T10:02:12.403964+0000","last_clean":"2024-04-25T10:02:12.403964+0000","last_became_active":"2024-04-25T10:02:06.375558+0000","last_became_peered":"2024-04-25T10:02:06.375558+0000","last_unstale":"2024-04-25T10:02:12.403964+0000","last_undegraded":"2024-04-25T10:02:12.403964+0000","last_fullsized":"2024-04-25T10:02:12.403964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406442+0000","last_change":"2024-04-25T10:02:07.352019+0000","last_active":"2024-04-25T10:02:12.406442+0000","last_peered":"2024-04-25T10:02:12.406442+0000","last_clean":"2024-04-25T10:02:12.406442+0000","last_became_active":"2024-04-25T10:02:07.351303+0000","last_became_peered":"2024-04-25T10:02:07.351303+0000","last_unstale":"2024-04-25T10:02:12.406442+0000","last_undegraded":"2024-04-25T10:02:12.406442+0000","last_fullsized":"2024-04-25T10:02:12.406442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:08.301904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406592+0000","last_change":"2024-04-25T10:02:11.441304+0000","last_active":"2024-04-25T10:02:12.406592+0000","last_peered":"2024-04-25T10:02:12.406592+0000","last_clean":"2024-04-25T10:02:12.406592+0000","last_became_active":"2024-04-25T10:02:11.440847+0000","last_became_peered":"2024-04-25T10:02:11.440847+0000","last_unstale":"2024-04-25T10:02:12.406592+0000","last_undegraded":"2024-04-25T10:02:12.406592+0000","last_fullsized":"2024-04-25T10:02:12.406592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:07.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403875+0000","last_change":"2024-04-25T10:02:07.372731+0000","last_active":"2024-04-25T10:02:12.403875+0000","last_peered":"2024-04-25T10:02:12.403875+0000","last_clean":"2024-04-25T10:02:12.403875+0000","last_became_active":"2024-04-25T10:02:07.372438+0000","last_became_peered":"2024-04-25T10:02:07.372438+0000","last_unstale":"2024-04-25T10:02:12.403875+0000","last_undegraded":"2024-04-25T10:02:12.403875+0000","last_fullsized":"2024-04-25T10:02:12.403875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462340+0000","last_change":"2024-04-25T10:02:07.364477+0000","last_active":"2024-04-25T10:02:13.462340+0000","last_peered":"2024-04-25T10:02:13.462340+0000","last_clean":"2024-04-25T10:02:13.462340+0000","last_became_active":"2024-04-25T10:02:07.363263+0000","last_became_peered":"2024-04-25T10:02:07.363263+0000","last_unstale":"2024-04-25T10:02:13.462340+0000","last_undegraded":"2024-04-25T10:02:13.462340+0000","last_fullsized":"2024-04-25T10:02:13.462340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:42.118057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465162+0000","last_change":"2024-04-25T10:02:07.366856+0000","last_active":"2024-04-25T10:02:13.465162+0000","last_peered":"2024-04-25T10:02:13.465162+0000","last_clean":"2024-04-25T10:02:13.465162+0000","last_became_active":"2024-04-25T10:02:07.366608+0000","last_became_peered":"2024-04-25T10:02:07.366608+0000","last_unstale":"2024-04-25T10:02:13.465162+0000","last_undegraded":"2024-04-25T10:02:13.465162+0000","last_fullsized":"2024-04-25T10:02:13.465162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463076+0000","last_change":"2024-04-25T10:02:07.374664+0000","last_active":"2024-04-25T10:02:13.463076+0000","last_peered":"2024-04-25T10:02:13.463076+0000","last_clean":"2024-04-25T10:02:13.463076+0000","last_became_active":"2024-04-25T10:02:07.374088+0000","last_became_peered":"2024-04-25T10:02:07.374088+0000","last_unstale":"2024-04-25T10:02:13.463076+0000","last_undegraded":"2024-04-25T10:02:13.463076+0000","last_fullsized":"2024-04-25T10:02:13.463076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404526+0000","last_change":"2024-04-25T10:02:07.365480+0000","last_active":"2024-04-25T10:02:12.404526+0000","last_peered":"2024-04-25T10:02:12.404526+0000","last_clean":"2024-04-25T10:02:12.404526+0000","last_became_active":"2024-04-25T10:02:07.365034+0000","last_became_peered":"2024-04-25T10:02:07.365034+0000","last_unstale":"2024-04-25T10:02:12.404526+0000","last_undegraded":"2024-04-25T10:02:12.404526+0000","last_fullsized":"2024-04-25T10:02:12.404526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461634+0000","last_change":"2024-04-25T10:02:07.362200+0000","last_active":"2024-04-25T10:02:13.461634+0000","last_peered":"2024-04-25T10:02:13.461634+0000","last_clean":"2024-04-25T10:02:13.461634+0000","last_became_active":"2024-04-25T10:02:07.361929+0000","last_became_peered":"2024-04-25T10:02:07.361929+0000","last_unstale":"2024-04-25T10:02:13.461634+0000","last_undegraded":"2024-04-25T10:02:13.461634+0000","last_fullsized":"2024-04-25T10:02:13.461634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407180+0000","last_change":"2024-04-25T10:02:07.375688+0000","last_active":"2024-04-25T10:02:12.407180+0000","last_peered":"2024-04-25T10:02:12.407180+0000","last_clean":"2024-04-25T10:02:12.407180+0000","last_became_active":"2024-04-25T10:02:07.375046+0000","last_became_peered":"2024-04-25T10:02:07.375046+0000","last_unstale":"2024-04-25T10:02:12.407180+0000","last_undegraded":"2024-04-25T10:02:12.407180+0000","last_fullsized":"2024-04-25T10:02:12.407180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406734+0000","last_change":"2024-04-25T10:02:07.365322+0000","last_active":"2024-04-25T10:02:12.406734+0000","last_peered":"2024-04-25T10:02:12.406734+0000","last_clean":"2024-04-25T10:02:12.406734+0000","last_became_active":"2024-04-25T10:02:07.365003+0000","last_became_peered":"2024-04-25T10:02:07.365003+0000","last_unstale":"2024-04-25T10:02:12.406734+0000","last_undegraded":"2024-04-25T10:02:12.406734+0000","last_fullsized":"2024-04-25T10:02:12.406734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464320+0000","last_change":"2024-04-25T10:02:07.374731+0000","last_active":"2024-04-25T10:02:13.464320+0000","last_peered":"2024-04-25T10:02:13.464320+0000","last_clean":"2024-04-25T10:02:13.464320+0000","last_became_active":"2024-04-25T10:02:07.374113+0000","last_became_peered":"2024-04-25T10:02:07.374113+0000","last_unstale":"2024-04-25T10:02:13.464320+0000","last_undegraded":"2024-04-25T10:02:13.464320+0000","last_fullsized":"2024-04-25T10:02:13.464320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404344+0000","last_change":"2024-04-25T10:02:07.365139+0000","last_active":"2024-04-25T10:02:12.404344+0000","last_peered":"2024-04-25T10:02:12.404344+0000","last_clean":"2024-04-25T10:02:12.404344+0000","last_became_active":"2024-04-25T10:02:07.364860+0000","last_became_peered":"2024-04-25T10:02:07.364860+0000","last_unstale":"2024-04-25T10:02:12.404344+0000","last_undegraded":"2024-04-25T10:02:12.404344+0000","last_fullsized":"2024-04-25T10:02:12.404344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464902+0000","last_change":"2024-04-25T10:02:07.379737+0000","last_active":"2024-04-25T10:02:13.464902+0000","last_peered":"2024-04-25T10:02:13.464902+0000","last_clean":"2024-04-25T10:02:13.464902+0000","last_became_active":"2024-04-25T10:02:07.379465+0000","last_became_peered":"2024-04-25T10:02:07.379465+0000","last_unstale":"2024-04-25T10:02:13.464902+0000","last_undegraded":"2024-04-25T10:02:13.464902+0000","last_fullsized":"2024-04-25T10:02:13.464902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462820+0000","last_change":"2024-04-25T10:02:07.379170+0000","last_active":"2024-04-25T10:02:13.462820+0000","last_peered":"2024-04-25T10:02:13.462820+0000","last_clean":"2024-04-25T10:02:13.462820+0000","last_became_active":"2024-04-25T10:02:07.378877+0000","last_became_peered":"2024-04-25T10:02:07.378877+0000","last_unstale":"2024-04-25T10:02:13.462820+0000","last_undegraded":"2024-04-25T10:02:13.462820+0000","last_fullsized":"2024-04-25T10:02:13.462820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406740+0000","last_change":"2024-04-25T10:02:07.367522+0000","last_active":"2024-04-25T10:02:12.406740+0000","last_peered":"2024-04-25T10:02:12.406740+0000","last_clean":"2024-04-25T10:02:12.406740+0000","last_became_active":"2024-04-25T10:02:07.367226+0000","last_became_peered":"2024-04-25T10:02:07.367226+0000","last_unstale":"2024-04-25T10:02:12.406740+0000","last_undegraded":"2024-04-25T10:02:12.406740+0000","last_fullsized":"2024-04-25T10:02:12.406740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465868+0000","last_change":"2024-04-25T10:02:07.378749+0000","last_active":"2024-04-25T10:02:13.465868+0000","last_peered":"2024-04-25T10:02:13.465868+0000","last_clean":"2024-04-25T10:02:13.465868+0000","last_became_active":"2024-04-25T10:02:07.378388+0000","last_became_peered":"2024-04-25T10:02:07.378388+0000","last_unstale":"2024-04-25T10:02:13.465868+0000","last_undegraded":"2024-04-25T10:02:13.465868+0000","last_fullsized":"2024-04-25T10:02:13.465868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:00.508687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462346+0000","last_change":"2024-04-25T10:02:07.377744+0000","last_active":"2024-04-25T10:02:13.462346+0000","last_peered":"2024-04-25T10:02:13.462346+0000","last_clean":"2024-04-25T10:02:13.462346+0000","last_became_active":"2024-04-25T10:02:07.376844+0000","last_became_peered":"2024-04-25T10:02:07.376844+0000","last_unstale":"2024-04-25T10:02:13.462346+0000","last_undegraded":"2024-04-25T10:02:13.462346+0000","last_fullsized":"2024-04-25T10:02:13.462346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465699+0000","last_change":"2024-04-25T10:02:07.388777+0000","last_active":"2024-04-25T10:02:13.465699+0000","last_peered":"2024-04-25T10:02:13.465699+0000","last_clean":"2024-04-25T10:02:13.465699+0000","last_became_active":"2024-04-25T10:02:07.388058+0000","last_became_peered":"2024-04-25T10:02:07.388058+0000","last_unstale":"2024-04-25T10:02:13.465699+0000","last_undegraded":"2024-04-25T10:02:13.465699+0000","last_fullsized":"2024-04-25T10:02:13.465699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464055+0000","last_change":"2024-04-25T10:02:07.377996+0000","last_active":"2024-04-25T10:02:13.464055+0000","last_peered":"2024-04-25T10:02:13.464055+0000","last_clean":"2024-04-25T10:02:13.464055+0000","last_became_active":"2024-04-25T10:02:07.376664+0000","last_became_peered":"2024-04-25T10:02:07.376664+0000","last_unstale":"2024-04-25T10:02:13.464055+0000","last_undegraded":"2024-04-25T10:02:13.464055+0000","last_fullsized":"2024-04-25T10:02:13.464055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461571+0000","last_change":"2024-04-25T10:02:07.364951+0000","last_active":"2024-04-25T10:02:13.461571+0000","last_peered":"2024-04-25T10:02:13.461571+0000","last_clean":"2024-04-25T10:02:13.461571+0000","last_became_active":"2024-04-25T10:02:07.364138+0000","last_became_peered":"2024-04-25T10:02:07.364138+0000","last_unstale":"2024-04-25T10:02:13.461571+0000","last_undegraded":"2024-04-25T10:02:13.461571+0000","last_fullsized":"2024-04-25T10:02:13.461571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462765+0000","last_change":"2024-04-25T10:02:07.370504+0000","last_active":"2024-04-25T10:02:13.462765+0000","last_peered":"2024-04-25T10:02:13.462765+0000","last_clean":"2024-04-25T10:02:13.462765+0000","last_became_active":"2024-04-25T10:02:07.370157+0000","last_became_peered":"2024-04-25T10:02:07.370157+0000","last_unstale":"2024-04-25T10:02:13.462765+0000","last_undegraded":"2024-04-25T10:02:13.462765+0000","last_fullsized":"2024-04-25T10:02:13.462765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403675+0000","last_change":"2024-04-25T10:02:07.360200+0000","last_active":"2024-04-25T10:02:12.403675+0000","last_peered":"2024-04-25T10:02:12.403675+0000","last_clean":"2024-04-25T10:02:12.403675+0000","last_became_active":"2024-04-25T10:02:07.358078+0000","last_became_peered":"2024-04-25T10:02:07.358078+0000","last_unstale":"2024-04-25T10:02:12.403675+0000","last_undegraded":"2024-04-25T10:02:12.403675+0000","last_fullsized":"2024-04-25T10:02:12.403675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407201+0000","last_change":"2024-04-25T10:02:07.376059+0000","last_active":"2024-04-25T10:02:12.407201+0000","last_peered":"2024-04-25T10:02:12.407201+0000","last_clean":"2024-04-25T10:02:12.407201+0000","last_became_active":"2024-04-25T10:02:07.374735+0000","last_became_peered":"2024-04-25T10:02:07.374735+0000","last_unstale":"2024-04-25T10:02:12.407201+0000","last_undegraded":"2024-04-25T10:02:12.407201+0000","last_fullsized":"2024-04-25T10:02:12.407201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:54.666473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406312+0000","last_change":"2024-04-25T10:02:07.367747+0000","last_active":"2024-04-25T10:02:12.406312+0000","last_peered":"2024-04-25T10:02:12.406312+0000","last_clean":"2024-04-25T10:02:12.406312+0000","last_became_active":"2024-04-25T10:02:07.367055+0000","last_became_peered":"2024-04-25T10:02:07.367055+0000","last_unstale":"2024-04-25T10:02:12.406312+0000","last_undegraded":"2024-04-25T10:02:12.406312+0000","last_fullsized":"2024-04-25T10:02:12.406312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406946+0000","last_change":"2024-04-25T10:02:07.366278+0000","last_active":"2024-04-25T10:02:12.406946+0000","last_peered":"2024-04-25T10:02:12.406946+0000","last_clean":"2024-04-25T10:02:12.406946+0000","last_became_active":"2024-04-25T10:02:07.365591+0000","last_became_peered":"2024-04-25T10:02:07.365591+0000","last_unstale":"2024-04-25T10:02:12.406946+0000","last_undegraded":"2024-04-25T10:02:12.406946+0000","last_fullsized":"2024-04-25T10:02:12.406946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466682+0000","last_change":"2024-04-25T10:02:07.389531+0000","last_active":"2024-04-25T10:02:13.466682+0000","last_peered":"2024-04-25T10:02:13.466682+0000","last_clean":"2024-04-25T10:02:13.466682+0000","last_became_active":"2024-04-25T10:02:07.389091+0000","last_became_peered":"2024-04-25T10:02:07.389091+0000","last_unstale":"2024-04-25T10:02:13.466682+0000","last_undegraded":"2024-04-25T10:02:13.466682+0000","last_fullsized":"2024-04-25T10:02:13.466682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403425+0000","last_change":"2024-04-25T10:02:07.368434+0000","last_active":"2024-04-25T10:02:12.403425+0000","last_peered":"2024-04-25T10:02:12.403425+0000","last_clean":"2024-04-25T10:02:12.403425+0000","last_became_active":"2024-04-25T10:02:07.366907+0000","last_became_peered":"2024-04-25T10:02:07.366907+0000","last_unstale":"2024-04-25T10:02:12.403425+0000","last_undegraded":"2024-04-25T10:02:12.403425+0000","last_fullsized":"2024-04-25T10:02:12.403425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463789+0000","last_change":"2024-04-25T10:02:07.377948+0000","last_active":"2024-04-25T10:02:13.463789+0000","last_peered":"2024-04-25T10:02:13.463789+0000","last_clean":"2024-04-25T10:02:13.463789+0000","last_became_active":"2024-04-25T10:02:07.377170+0000","last_became_peered":"2024-04-25T10:02:07.377170+0000","last_unstale":"2024-04-25T10:02:13.463789+0000","last_undegraded":"2024-04-25T10:02:13.463789+0000","last_fullsized":"2024-04-25T10:02:13.463789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464659+0000","last_change":"2024-04-25T10:02:07.376985+0000","last_active":"2024-04-25T10:02:13.464659+0000","last_peered":"2024-04-25T10:02:13.464659+0000","last_clean":"2024-04-25T10:02:13.464659+0000","last_became_active":"2024-04-25T10:02:07.376669+0000","last_became_peered":"2024-04-25T10:02:07.376669+0000","last_unstale":"2024-04-25T10:02:13.464659+0000","last_undegraded":"2024-04-25T10:02:13.464659+0000","last_fullsized":"2024-04-25T10:02:13.464659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464634+0000","last_change":"2024-04-25T10:02:07.377373+0000","last_active":"2024-04-25T10:02:13.464634+0000","last_peered":"2024-04-25T10:02:13.464634+0000","last_clean":"2024-04-25T10:02:13.464634+0000","last_became_active":"2024-04-25T10:02:07.373588+0000","last_became_peered":"2024-04-25T10:02:07.373588+0000","last_unstale":"2024-04-25T10:02:13.464634+0000","last_undegraded":"2024-04-25T10:02:13.464634+0000","last_fullsized":"2024-04-25T10:02:13.464634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464847+0000","last_change":"2024-04-25T10:02:07.382540+0000","last_active":"2024-04-25T10:02:13.464847+0000","last_peered":"2024-04-25T10:02:13.464847+0000","last_clean":"2024-04-25T10:02:13.464847+0000","last_became_active":"2024-04-25T10:02:07.381996+0000","last_became_peered":"2024-04-25T10:02:07.381996+0000","last_unstale":"2024-04-25T10:02:13.464847+0000","last_undegraded":"2024-04-25T10:02:13.464847+0000","last_fullsized":"2024-04-25T10:02:13.464847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462277+0000","last_change":"2024-04-25T10:02:07.376120+0000","last_active":"2024-04-25T10:02:13.462277+0000","last_peered":"2024-04-25T10:02:13.462277+0000","last_clean":"2024-04-25T10:02:13.462277+0000","last_became_active":"2024-04-25T10:02:07.375589+0000","last_became_peered":"2024-04-25T10:02:07.375589+0000","last_unstale":"2024-04-25T10:02:13.462277+0000","last_undegraded":"2024-04-25T10:02:13.462277+0000","last_fullsized":"2024-04-25T10:02:13.462277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:35.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462177+0000","last_change":"2024-04-25T10:02:07.369060+0000","last_active":"2024-04-25T10:02:13.462177+0000","last_peered":"2024-04-25T10:02:13.462177+0000","last_clean":"2024-04-25T10:02:13.462177+0000","last_became_active":"2024-04-25T10:02:07.368434+0000","last_became_peered":"2024-04-25T10:02:07.368434+0000","last_unstale":"2024-04-25T10:02:13.462177+0000","last_undegraded":"2024-04-25T10:02:13.462177+0000","last_fullsized":"2024-04-25T10:02:13.462177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466451+0000","last_change":"2024-04-25T10:02:07.385523+0000","last_active":"2024-04-25T10:02:13.466451+0000","last_peered":"2024-04-25T10:02:13.466451+0000","last_clean":"2024-04-25T10:02:13.466451+0000","last_became_active":"2024-04-25T10:02:07.385218+0000","last_became_peered":"2024-04-25T10:02:07.385218+0000","last_unstale":"2024-04-25T10:02:13.466451+0000","last_undegraded":"2024-04-25T10:02:13.466451+0000","last_fullsized":"2024-04-25T10:02:13.466451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:31.815 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:31.815 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:02:31.815 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:02:31.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:02:31.816 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:02:31.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.964+0000 7f0a25bf3640 1 Processor -- start 2024-04-25T10:02:31.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.964+0000 7f0a25bf3640 1 -- start start 2024-04-25T10:02:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a25bf3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a200ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a25bf3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0a201451f0 0x7f0a20142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a25bf3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a200cee30 con 0x7f0a20144ca0 2024-04-25T10:02:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a25bf3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a200cc2b0 con 0x7f0a201451f0 2024-04-25T10:02:31.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a25bf3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2006e290 con 0x7f0a200ce510 2024-04-25T10:02:31.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0a20144ca0 0x7f0a2006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57908/0 (socket says 172.21.15.82:57908) 2024-04-25T10:02:31.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1effd640 1 -- 172.21.15.82:0/1618393953 learned_addr learned my addr 172.21.15.82:0/1618393953 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:31.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1ffff640 1 --2- 172.21.15.82:0/1618393953 >> v2:172.21.15.196:3300/0 conn(0x7f0a201451f0 0x7f0a20142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a200ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081118264 0 0) 0x7f0a200cee30 con 0x7f0a20144ca0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a08003530 con 0x7f0a20144ca0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2253166820 0 0) 0x7f0a08003530 con 0x7f0a20144ca0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 msgr2=0x7f0a200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a200ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 >> v2:172.21.15.196:3300/0 conn(0x7f0a201451f0 msgr2=0x7f0a20142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 --2- 172.21.15.82:0/1618393953 >> v2:172.21.15.196:3300/0 conn(0x7f0a201451f0 0x7f0a20142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a200ccc70 con 0x7f0a20144ca0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0a14001680 con 0x7f0a20144ca0 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a200ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 >> v1:172.21.15.82:6789/0 conn(0x7f0a20144ca0 legacy=0x7f0a2006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a08003a30 0x7f0a08003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a08004810 0x7f0a08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200cc2b0 con 0x7f0a08003a30 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2006e290 con 0x7f0a08004810 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0800ba00 con 0x7f0a200ce510 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a08003a30 0x7f0a08003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 -- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 msgr2=0x7f0a080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a200ce510 0x7f0a080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1effd640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a08004810 0x7f0a08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 -- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a08004810 msgr2=0x7f0a08004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a08004810 0x7f0a08004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1f7fe640 1 -- 172.21.15.82:0/1618393953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08003530 con 0x7f0a08003a30 2024-04-25T10:02:31.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.968+0000 7f0a1effd640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a08004810 0x7f0a08004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a08003a30 0x7f0a08003e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009cb0 tx=0x7f0a1001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b142b5c86640e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100b2070 con 0x7f0a08003a30 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a100a5730 con 0x7f0a08003a30 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1e7fc640 1 -- 172.21.15.82:0/1618393953 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10009880 con 0x7f0a08003a30 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1618393953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a08003a30 msgr2=0x7f0a08003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1618393953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a08003a30 0x7f0a08003e50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009cb0 tx=0x7f0a1001cc50 comp rx=0 tx=0).stop 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1ffff640 1 -- 172.21.15.82:0/1618393953 reap_dead start 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1618393953 shutdown_connections 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1618393953 >> 172.21.15.82:0/1618393953 conn(0x7f0a200b11a0 msgr2=0x7f0a200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1618393953 shutdown_connections 2024-04-25T10:02:31.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1618393953 wait complete. 2024-04-25T10:02:31.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 Processor -- start 2024-04-25T10:02:31.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- start start 2024-04-25T10:02:31.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 0x7f0a201e4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 0x7f0a201e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2006ee30 con 0x7f0a200caae0 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2006df20 con 0x7f0a200ca700 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a200cc900 con 0x7f0a201e0c70 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48808/0 (socket says 172.21.15.82:48808) 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 -- 172.21.15.82:0/1683300375 learned_addr learned my addr 172.21.15.82:0/1683300375 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 0x7f0a201e4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 -- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 msgr2=0x7f0a201e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1ffff640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 0x7f0a201e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 0x7f0a201e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 -- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 msgr2=0x7f0a201e4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 0x7f0a201e4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 -- 172.21.15.82:0/1683300375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a201e19a0 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 0x7f0a201e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1ffff640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 0x7f0a201e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a1effd640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a14004850 tx=0x7f0a14005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1401b070 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14011040 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2012dee0 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140143a0 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.972+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2012e4a0 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2004efa0 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a1401e070 con 0x7f0a200caae0 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a07fff640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 0x7f0a0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 0x7f0a0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0a14087e90 con 0x7f0a200caae0 2024-04-25T10:02:31.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.976+0000 7f0a1f7fe640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 0x7f0a0c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a100af000 tx=0x7f0a10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:31.980+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a14058b60 con 0x7f0a200caae0 2024-04-25T10:02:32.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.176+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0a200b1cc0 con 0x7f0a200caae0 2024-04-25T10:02:32.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.180+0000 7f0a07fff640 1 -- 172.21.15.82:0/1683300375 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165730 (secure 0 0 0) 0x7f0a14058460 con 0x7f0a200caae0 2024-04-25T10:02:32.191 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:32.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 msgr2=0x7f0a0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 0x7f0a0c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a100af000 tx=0x7f0a10008040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 msgr2=0x7f0a201e5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0a14004850 tx=0x7f0a14005850 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 shutdown_connections 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a0c066f60 0x7f0a0c069420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a201e0c70 0x7f0a201e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a200caae0 0x7f0a201e5300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 --2- 172.21.15.82:0/1683300375 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a200ca700 0x7f0a201e4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 >> 172.21.15.82:0/1683300375 conn(0x7f0a200b11a0 msgr2=0x7f0a200c5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 shutdown_connections 2024-04-25T10:02:32.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.184+0000 7f0a25bf3640 1 -- 172.21.15.82:0/1683300375 wait complete. 2024-04-25T10:02:32.208 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6832","nonce":1711591888},{"type":"v1","addr":"172.21.15.82:6833","nonce":1711591888}]},"active_addr":"172.21.15.82:6833/1711591888","active_change":"2024-04-25T10:01:47.248980+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":955366075}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":58935575}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2853616994}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.82:0","nonce":2530438792}]}]} 2024-04-25T10:02:32.211 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:02:32.212 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:02:32.212 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:32.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 Processor -- start 2024-04-25T10:02:32.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 -- start start 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f14340ce510 0x7f14340ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1434144ca0 0x7f143406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14340cee30 con 0x7f143406ec60 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340cc2b0 con 0x7f1434144ca0 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f143961f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143406e290 con 0x7f14340ce510 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1434144ca0 0x7f143406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1434144ca0 0x7f143406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48820/0 (socket says 172.21.15.82:48820) 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 -- 172.21.15.82:0/182651230 learned_addr learned my addr 172.21.15.82:0/182651230 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f1432ffd640 1 --2- 172.21.15.82:0/182651230 >> v2:172.21.15.196:3300/0 conn(0x7f14340ce510 0x7f14340ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 -- 172.21.15.82:0/182651230 >> v2:172.21.15.196:3300/0 conn(0x7f14340ce510 msgr2=0x7f14340ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 --2- 172.21.15.82:0/182651230 >> v2:172.21.15.196:3300/0 conn(0x7f14340ce510 0x7f14340ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 -- 172.21.15.82:0/182651230 >> v1:172.21.15.196:6789/0 conn(0x7f143406ec60 legacy=0x7f1434142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.320+0000 7f14327fc640 1 -- 172.21.15.82:0/182651230 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340ccc70 con 0x7f1434144ca0 2024-04-25T10:02:32.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14327fc640 1 --2- 172.21.15.82:0/182651230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1434144ca0 0x7f143406e720 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1434144c50 tx=0x7f141c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293a6d16d0f174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f1431ffb640 1 -- 172.21.15.82:0/182651230 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0b5070 con 0x7f1434144ca0 2024-04-25T10:02:32.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f1431ffb640 1 -- 172.21.15.82:0/182651230 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c0a89d0 con 0x7f1434144ca0 2024-04-25T10:02:32.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f1431ffb640 1 -- 172.21.15.82:0/182651230 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0a8da0 con 0x7f1434144ca0 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- 172.21.15.82:0/182651230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1434144ca0 msgr2=0x7f143406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- 172.21.15.82:0/182651230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1434144ca0 0x7f143406e720 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1434144c50 tx=0x7f141c01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- 172.21.15.82:0/182651230 shutdown_connections 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- 172.21.15.82:0/182651230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1434144ca0 0x7f143406e720 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- 172.21.15.82:0/182651230 >> v2:172.21.15.196:3300/0 conn(0x7f14340ce510 0x7f14340ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- 172.21.15.82:0/182651230 >> 172.21.15.82:0/182651230 conn(0x7f14340b11a0 msgr2=0x7f14340b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- 172.21.15.82:0/182651230 shutdown_connections 2024-04-25T10:02:32.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- 172.21.15.82:0/182651230 wait complete. 2024-04-25T10:02:32.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 Processor -- start 2024-04-25T10:02:32.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- start start 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 0x7f14340ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1434144ca0 0x7f14340a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340cc900 con 0x7f14340ce510 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143406df20 con 0x7f1434144ca0 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f143961f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1434142d90 con 0x7f143406ec60 2024-04-25T10:02:32.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14327fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14327fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48822/0 (socket says 172.21.15.82:48822) 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14327fc640 1 -- 172.21.15.82:0/4160568972 learned_addr learned my addr 172.21.15.82:0/4160568972 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14337fe640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1434144ca0 0x7f14340a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f1432ffd640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 0x7f14340ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.324+0000 7f14327fc640 1 -- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 msgr2=0x7f14340ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f14327fc640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 0x7f14340ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f14327fc640 1 -- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1434144ca0 msgr2=0x7f14340a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f14327fc640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1434144ca0 0x7f14340a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f14327fc640 1 -- 172.21.15.82:0/4160568972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340aa2c0 con 0x7f14340ce510 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f1432ffd640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 0x7f14340ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f14327fc640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b3040 tx=0x7f141c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0c0070 con 0x7f14340ce510 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c0b0a50 con 0x7f14340ce510 2024-04-25T10:02:32.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0b5070 con 0x7f14340ce510 2024-04-25T10:02:32.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340aa5a0 con 0x7f14340ce510 2024-04-25T10:02:32.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14340aab60 con 0x7f14340ce510 2024-04-25T10:02:32.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143404efa0 con 0x7f14340ce510 2024-04-25T10:02:32.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f141c0ba3e0 con 0x7f14340ce510 2024-04-25T10:02:32.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 0x7f1414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f1432ffd640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 0x7f1414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f141c122530 con 0x7f14340ce510 2024-04-25T10:02:32.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.328+0000 7f1432ffd640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 0x7f1414069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1424007330 tx=0x7f142400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.332+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f141c018830 con 0x7f14340ce510 2024-04-25T10:02:32.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.492+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14340b1cc0 con 0x7f14340ce510 2024-04-25T10:02:32.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f142b7fe640 1 -- 172.21.15.82:0/4160568972 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f141c0f3200 con 0x7f14340ce510 2024-04-25T10:02:32.502 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:32.503 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 msgr2=0x7f1414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 0x7f1414069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1424007330 tx=0x7f142400b040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 msgr2=0x7f14340af960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f141c0b3040 tx=0x7f141c01ba70 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 shutdown_connections 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1414066f60 0x7f1414069420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1434144ca0 0x7f14340a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f14340ce510 0x7f14340af960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 --2- 172.21.15.82:0/4160568972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f143406ec60 0x7f14340ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 >> 172.21.15.82:0/4160568972 conn(0x7f14340b11a0 msgr2=0x7f14340ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 shutdown_connections 2024-04-25T10:02:32.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.496+0000 7f143961f640 1 -- 172.21.15.82:0/4160568972 wait complete. 2024-04-25T10:02:32.521 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:02:32.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:32.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.620+0000 7f8b9786e640 1 Processor -- start 2024-04-25T10:02:32.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.620+0000 7f8b9786e640 1 -- start start 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b900ce510 0x7f8b900ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b901451f0 0x7f8b90142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b900cee30 con 0x7f8b90144ca0 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b900cc2b0 con 0x7f8b900ce510 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9006e290 con 0x7f8b901451f0 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b900ce510 0x7f8b900ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b900ce510 0x7f8b900ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48828/0 (socket says 172.21.15.82:48828) 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 -- 172.21.15.82:0/1770987860 learned_addr learned my addr 172.21.15.82:0/1770987860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 -- 172.21.15.82:0/1770987860 >> v2:172.21.15.196:3300/0 conn(0x7f8b901451f0 msgr2=0x7f8b90142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 --2- 172.21.15.82:0/1770987860 >> v2:172.21.15.196:3300/0 conn(0x7f8b901451f0 0x7f8b90142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 -- 172.21.15.82:0/1770987860 >> v1:172.21.15.196:6789/0 conn(0x7f8b90144ca0 legacy=0x7f8b9006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 -- 172.21.15.82:0/1770987860 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b900ccc70 con 0x7f8b900ce510 2024-04-25T10:02:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9560a640 1 --2- 172.21.15.82:0/1770987860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b900ce510 0x7f8b900ce8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b900b48f0 tx=0x7f8b8c01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df2908f48189ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b87fff640 1 -- 172.21.15.82:0/1770987860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0b4070 con 0x7f8b900ce510 2024-04-25T10:02:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b87fff640 1 -- 172.21.15.82:0/1770987860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8c0af400 con 0x7f8b900ce510 2024-04-25T10:02:32.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b87fff640 1 -- 172.21.15.82:0/1770987860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c0af7d0 con 0x7f8b900ce510 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- 172.21.15.82:0/1770987860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b900ce510 msgr2=0x7f8b900ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 --2- 172.21.15.82:0/1770987860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b900ce510 0x7f8b900ce8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b900b48f0 tx=0x7f8b8c01f1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- 172.21.15.82:0/1770987860 shutdown_connections 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 --2- 172.21.15.82:0/1770987860 >> v2:172.21.15.196:3300/0 conn(0x7f8b901451f0 0x7f8b90142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 --2- 172.21.15.82:0/1770987860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b900ce510 0x7f8b900ce8f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- 172.21.15.82:0/1770987860 >> 172.21.15.82:0/1770987860 conn(0x7f8b900b11a0 msgr2=0x7f8b900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- 172.21.15.82:0/1770987860 shutdown_connections 2024-04-25T10:02:32.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.624+0000 7f8b9786e640 1 -- 172.21.15.82:0/1770987860 wait complete. 2024-04-25T10:02:32.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 Processor -- start 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- start start 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 0x7f8b901e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 0x7f8b90143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9006ee30 con 0x7f8b901451f0 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9006df20 con 0x7f8b900ce510 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b900cc900 con 0x7f8b90144ca0 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48838/0 (socket says 172.21.15.82:48838) 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 -- 172.21.15.82:0/578865269 learned_addr learned my addr 172.21.15.82:0/578865269 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9560a640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 0x7f8b901e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b94e09640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 0x7f8b90143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 -- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 msgr2=0x7f8b90143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 0x7f8b90143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 -- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 msgr2=0x7f8b901e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 0x7f8b901e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 -- 172.21.15.82:0/578865269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b901e4bb0 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9560a640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 0x7f8b901e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b94e09640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 0x7f8b90143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b95e0b640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800b410 tx=0x7f8b8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88015020 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b88011850 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b901e4e90 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88011c40 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901e5450 con 0x7f8b901451f0 2024-04-25T10:02:32.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9004efa0 con 0x7f8b901451f0 2024-04-25T10:02:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.628+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b88018020 con 0x7f8b901451f0 2024-04-25T10:02:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.632+0000 7f8b85ffb640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 0x7f8b74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.632+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b880852c0 con 0x7f8b901451f0 2024-04-25T10:02:32.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.632+0000 7f8b9560a640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 0x7f8b74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.632+0000 7f8b9560a640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 0x7f8b74069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0b1000 tx=0x7f8b8c0aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.636+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b8801f020 con 0x7f8b901451f0 2024-04-25T10:02:32.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.780+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b9015bac0 con 0x7f8b901451f0 2024-04-25T10:02:32.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b85ffb640 1 -- 172.21.15.82:0/578865269 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f8b880172c0 con 0x7f8b901451f0 2024-04-25T10:02:32.791 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:32.791 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:32.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 msgr2=0x7f8b74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 0x7f8b74069420 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0b1000 tx=0x7f8b8c0aa000 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 msgr2=0x7f8b901e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800b410 tx=0x7f8b8800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 shutdown_connections 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b74066f60 0x7f8b74069420 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b90144ca0 0x7f8b90143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b901451f0 0x7f8b901e4240 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 --2- 172.21.15.82:0/578865269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b900ce510 0x7f8b901e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 >> 172.21.15.82:0/578865269 conn(0x7f8b900b11a0 msgr2=0x7f8b900c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 shutdown_connections 2024-04-25T10:02:32.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.784+0000 7f8b9786e640 1 -- 172.21.15.82:0/578865269 wait complete. 2024-04-25T10:02:32.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:02:32.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:02:32.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:02:32.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:02:32.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:02:32.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:02:32.811 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:02:32.811 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:02:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.976+0000 7fa7d1636640 1 Processor -- start 2024-04-25T10:02:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.976+0000 7fa7d1636640 1 -- start start 2024-04-25T10:02:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 Processor -- start 2024-04-25T10:02:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 -- start start 2024-04-25T10:02:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7d1636640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7cc15da80 0x7fa7cc06e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7d1636640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 0x7fa7cc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7d1636640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc071b60 con 0x7fa7cc0ae7a0 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7d1636640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc072770 con 0x7fa7cc15da80 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7d1636640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc071e40 con 0x7fa7cc0ce880 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 Processor -- start 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7caffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa7cc15da80 0x7fa7cc06e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7cb7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 0x7fa7cc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7ca7fc640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fa7cc0ae7a0 0x7fa7cc0ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48846/0 (socket says 172.21.15.82:48846) 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7ca7fc640 1 -- 172.21.15.82:0/3196010656 learned_addr learned my addr 172.21.15.82:0/3196010656 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 -- start start 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61f4052a60 0x7f61f4052ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40e4660 con 0x7f61f40e3470 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f40e4920 con 0x7f61f40399c0 2024-04-25T10:02:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6201b8c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f40391c0 con 0x7f61f4052a60 2024-04-25T10:02:32.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7cb7fe640 1 -- 172.21.15.82:0/3196010656 >> v2:172.21.15.196:3300/0 conn(0x7fa7cc15da80 msgr2=0x7fa7cc06e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7cb7fe640 1 --2- 172.21.15.82:0/3196010656 >> v2:172.21.15.196:3300/0 conn(0x7fa7cc15da80 0x7fa7cc06e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7c9ffb640 1 -- 172.21.15.82:0/3196010656 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677516374 0 0) 0x7fa7cc071b60 con 0x7fa7cc0ae7a0 2024-04-25T10:02:32.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7cb7fe640 1 -- 172.21.15.82:0/3196010656 >> v1:172.21.15.196:6789/0 conn(0x7fa7cc0ae7a0 legacy=0x7fa7cc0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7fa7cb7fe640 1 -- 172.21.15.82:0/3196010656 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc072120 con 0x7fa7cc0ce880 2024-04-25T10:02:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6200b8a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f61f40e3470 0x7f61f40e3c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57924/0 (socket says 172.21.15.82:57924) 2024-04-25T10:02:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61f4052a60 0x7f61f4052ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61f4052a60 0x7f61f4052ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48844/0 (socket says 172.21.15.82:48844) 2024-04-25T10:02:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f6200b8a640 1 -- 172.21.15.82:0/3773466232 learned_addr learned my addr 172.21.15.82:0/3773466232 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f20b7b51640 1 Processor -- start 2024-04-25T10:02:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7c9c161b80 0x7f7c9c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 0x7f7c9c0c9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0c9d50 con 0x7f7c9c06e180 2024-04-25T10:02:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9c0cd600 con 0x7f7c9c161b80 2024-04-25T10:02:32.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9bfff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c165cd0 con 0x7f7c9c165810 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f20b7b51640 1 -- start start 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059432428 0 0) 0x7f61f40e4920 con 0x7f61f40399c0 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61ec003530 con 0x7f61f40399c0 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792103016 0 0) 0x7f61f40e4660 con 0x7f61f40e3470 2024-04-25T10:02:32.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61f40e4920 con 0x7f61f40e3470 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fbfff640 1 -- 172.21.15.82:0/3773466232 >> v1:172.21.15.196:6789/0 conn(0x7f61f40399c0 legacy=0x7f61f4050700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fbfff640 1 -- 172.21.15.82:0/3773466232 >> v1:172.21.15.82:6789/0 conn(0x7f61f40e3470 legacy=0x7f61f40e3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f61fbfff640 1 -- 172.21.15.82:0/3773466232 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f40394a0 con 0x7f61f4052a60 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9affd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7c9c161b80 0x7f7c9c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9a7fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 0x7f7c9c0c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9b7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7c9c06e180 0x7f7c9c06d1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48860/0 (socket says 172.21.15.82:48860) 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9a7fc640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 0x7f7c9c0c9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56768/0 (socket says 172.21.15.82:56768) 2024-04-25T10:02:32.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9b7fe640 1 -- 172.21.15.82:0/267105262 learned_addr learned my addr 172.21.15.82:0/267105262 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9affd640 1 -- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 msgr2=0x7f7c9c0c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9affd640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 0x7f7c9c0c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9affd640 1 -- 172.21.15.82:0/267105262 >> v1:172.21.15.196:6789/0 conn(0x7f7c9c06e180 legacy=0x7f7c9c06d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.980+0000 7f7c9affd640 1 -- 172.21.15.82:0/267105262 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c165fb0 con 0x7f7c9c161b80 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b7b51640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b7b51640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20b00cbff0 0x7f20b00c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 Processor -- start 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 -- start start 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b58ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b58ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43614/0 (socket says 172.21.15.82:43614) 2024-04-25T10:02:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b58ed640 1 -- 172.21.15.82:0/1038264008 learned_addr learned my addr 172.21.15.82:0/1038264008 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b015d4d0 con 0x7f20b00c9f90 2024-04-25T10:02:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b015d7b0 con 0x7f20b00cbff0 2024-04-25T10:02:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7cb7fe640 1 --2- 172.21.15.82:0/3196010656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 0x7fa7cc0ca1a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009dc0 tx=0x7fa7c401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de392ba87de14ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b015da90 con 0x7f20b015d050 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b50ec640 1 --2- 172.21.15.82:0/1038264008 >> v2:172.21.15.82:3300/0 conn(0x7f20b00cbff0 0x7f20b00c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa600033e40 0x7fa6000eb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa600005ea0 0x7fa6000062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa600034220 con 0x7fa600047c20 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa600050210 con 0x7fa600033e40 2024-04-25T10:02:32.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa60d717640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa600037bc0 con 0x7fa600005ea0 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7c9ffb640 1 -- 172.21.15.82:0/3196010656 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40b2070 con 0x7fa7cc0ce880 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa600033e40 0x7fa6000eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7c9ffb640 1 -- 172.21.15.82:0/3196010656 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c40a57d0 con 0x7fa7cc0ce880 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7c9ffb640 1 -- 172.21.15.82:0/3196010656 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40a5ba0 con 0x7fa7cc0ce880 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b48eb640 1 -- 172.21.15.82:0/1038264008 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943347569 0 0) 0x7f20b015d4d0 con 0x7f20b00c9f90 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b50ec640 1 -- 172.21.15.82:0/1038264008 >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 msgr2=0x7f20b00cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b50ec640 1 --2- 172.21.15.82:0/1038264008 >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa600033e40 0x7fa6000eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48870/0 (socket says 172.21.15.82:48870) 2024-04-25T10:02:33.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b50ec640 1 -- 172.21.15.82:0/1038264008 >> v1:172.21.15.196:6789/0 conn(0x7f20b00c9f90 legacy=0x7f20b00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 -- 172.21.15.82:0/2277017617 learned_addr learned my addr 172.21.15.82:0/2277017617 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b50ec640 1 -- 172.21.15.82:0/1038264008 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b015dd70 con 0x7f20b00cbff0 2024-04-25T10:02:33.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 Processor -- start 2024-04-25T10:02:33.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f20b58ed640 1 --2- 172.21.15.82:0/1038264008 >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa6077fe640 1 --2- 172.21.15.82:0/2277017617 >> v2:172.21.15.196:3300/0 conn(0x7fa600005ea0 0x7fa6000062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 -- 172.21.15.82:0/2277017617 >> v2:172.21.15.196:3300/0 conn(0x7fa600005ea0 msgr2=0x7fa6000062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 --2- 172.21.15.82:0/2277017617 >> v2:172.21.15.196:3300/0 conn(0x7fa600005ea0 0x7fa6000062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 -- 172.21.15.82:0/2277017617 >> v1:172.21.15.196:6789/0 conn(0x7fa600047c20 legacy=0x7fa6000eb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa606ffd640 1 -- 172.21.15.82:0/2277017617 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60004fa30 con 0x7fa600033e40 2024-04-25T10:02:33.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f61fbfff640 1 --2- 172.21.15.82:0/3773466232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f4052ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009a20 tx=0x7f61e801c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4929590c4e86331c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 -- start start 2024-04-25T10:02:33.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 -- 172.21.15.82:0/3196010656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 msgr2=0x7fa7cc0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 --2- 172.21.15.82:0/3196010656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 0x7fa7cc0ca1a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009dc0 tx=0x7fa7c401c950 comp rx=0 tx=0).stop 2024-04-25T10:02:33.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 -- 172.21.15.82:0/3196010656 shutdown_connections 2024-04-25T10:02:33.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 --2- 172.21.15.82:0/3196010656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ce880 0x7fa7cc0ca1a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 --2- 172.21.15.82:0/3196010656 >> v2:172.21.15.196:3300/0 conn(0x7fa7cc15da80 0x7fa7cc06e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 -- 172.21.15.82:0/3196010656 >> 172.21.15.82:0/3196010656 conn(0x7fa7cc0b0cf0 msgr2=0x7fa7cc0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f7c9affd640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0cd0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009d90 tx=0x7f7c9401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a940c9e6940e0aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 -- 172.21.15.82:0/3196010656 shutdown_connections 2024-04-25T10:02:33.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80a66e0 con 0x7f61f4052a60 2024-04-25T10:02:33.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa7d1636640 1 -- 172.21.15.82:0/3196010656 wait complete. 2024-04-25T10:02:33.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa976302640 1 Processor -- start 2024-04-25T10:02:33.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7fa976302640 1 -- start start 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 0x7efd7c15e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efd7c15d1d0 0x7efd7c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c072a30 con 0x7efd7c15d1d0 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0ae7d0 con 0x7efd7c15cdf0 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd842cf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0aea30 con 0x7efd7c0b1810 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f7c99ffb640 1 -- 172.21.15.82:0/267105262 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b2070 con 0x7f7c9c161b80 2024-04-25T10:02:33.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f7c99ffb640 1 -- 172.21.15.82:0/267105262 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940a5920 con 0x7f7c9c161b80 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f7c99ffb640 1 -- 172.21.15.82:0/267105262 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940a5cf0 con 0x7f7c9c161b80 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80ae070 con 0x7f61f4052a60 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd8206b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 0x7efd7c15e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd8286c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd8206b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 0x7efd7c15e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43632/0 (socket says 172.21.15.82:43632) 2024-04-25T10:02:33.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.984+0000 7efd8206b640 1 -- 172.21.15.82:0/3829181003 learned_addr learned my addr 172.21.15.82:0/3829181003 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8186a640 1 --2- 172.21.15.82:0/3829181003 >> v2:172.21.15.82:3300/0 conn(0x7efd7c15d1d0 0x7efd7c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 -- 172.21.15.82:0/3773466232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 msgr2=0x7f61f4052ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 msgr2=0x7f7c9c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0cd0c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009d90 tx=0x7f7c9401c970 comp rx=0 tx=0).stop 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- 172.21.15.82:0/267105262 shutdown_connections 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c165810 0x7f7c9c0c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 --2- 172.21.15.82:0/267105262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0cd0c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- 172.21.15.82:0/267105262 >> 172.21.15.82:0/267105262 conn(0x7f7c9c06ac90 msgr2=0x7f7c9c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 Processor -- start 2024-04-25T10:02:33.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- 172.21.15.82:0/267105262 shutdown_connections 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa976302640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa97015e170 0x7fa9700ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa976302640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97006e240 con 0x7fa9700cbf40 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa976302640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9700aea30 con 0x7fa9700b1810 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa976302640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa970072a30 con 0x7fa97015e170 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 -- start start 2024-04-25T10:02:33.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa97015e170 0x7fa9700ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa97015e170 0x7fa9700ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43640/0 (socket says 172.21.15.82:43640) 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 -- 172.21.15.82:0/1654390139 learned_addr learned my addr 172.21.15.82:0/1654390139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 -- 172.21.15.82:0/3829181003 >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 msgr2=0x7efd7c15e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 --2- 172.21.15.82:0/3829181003 >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 0x7efd7c15e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 -- 172.21.15.82:0/3829181003 >> v2:172.21.15.82:3300/0 conn(0x7efd7c15d1d0 msgr2=0x7efd7c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 --2- 172.21.15.82:0/3829181003 >> v2:172.21.15.82:3300/0 conn(0x7efd7c15d1d0 0x7efd7c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 -- 172.21.15.82:0/3829181003 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0c9850 con 0x7efd7c0b1810 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f61fb7fe640 1 -- 172.21.15.82:0/3773466232 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80a66e0 con 0x7f61f4052a60 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 --2- 172.21.15.82:0/3773466232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f4052ed0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f61e8009a20 tx=0x7f61e801c720 comp rx=0 tx=0).stop 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638798717 0 0) 0x7fa9700aea30 con 0x7fa9700b1810 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa958003530 con 0x7fa9700b1810 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa606ffd640 1 --2- 172.21.15.82:0/2277017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600033e40 0x7fa6000eb180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa60003ac20 tx=0x7fa5fc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7afb5c4e2d219bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975776043 0 0) 0x7fa97006e240 con 0x7fa9700cbf40 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9700aea30 con 0x7fa9700cbf40 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa605ffb640 1 -- 172.21.15.82:0/2277017617 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0b5070 con 0x7fa600033e40 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.1 v1:172.21.15.196:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1641230439 0 0) 0x7fa958003530 con 0x7fa9700b1810 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 >> v2:172.21.15.196:3300/0 conn(0x7fa97015e170 msgr2=0x7fa9700ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 --2- 172.21.15.82:0/1654390139 >> v2:172.21.15.196:3300/0 conn(0x7fa97015e170 0x7fa9700ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 >> v1:172.21.15.82:6789/0 conn(0x7fa9700cbf40 legacy=0x7fa9700cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 0x7fa7cc0ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> v1:172.21.15.196:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9700c9850 con 0x7fa9700b1810 2024-04-25T10:02:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.1 v1:172.21.15.196:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa968001a00 con 0x7fa9700b1810 2024-04-25T10:02:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 >> v1:172.21.15.196:6789/0 conn(0x7fa9700b1810 legacy=0x7fa97015cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958003ba0 0x7fa958003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 0x7fa958004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9580051d0 0x7fa95800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970072a30 con 0x7fa958004810 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95800c0a0 con 0x7fa9580051d0 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95800c2a0 con 0x7fa958003ba0 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 -- 172.21.15.82:0/3773466232 shutdown_connections 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 --2- 172.21.15.82:0/3773466232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f4052ed0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 -- 172.21.15.82:0/3773466232 >> 172.21.15.82:0/3773466232 conn(0x7f61f400a410 msgr2=0x7f61f400a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc0ce880 0x7fa7cc167980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7cc15da80 0x7fa7cc0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 0x7fa7cc0ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 0x7fa7cc0ad020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56774/0 (socket says 172.21.15.82:56774) 2024-04-25T10:02:33.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96ffff640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958003ba0 0x7fa958003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 -- 172.21.15.82:0/4100805317 learned_addr learned my addr 172.21.15.82:0/4100805317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc072d60 con 0x7fa7cc15da80 2024-04-25T10:02:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc072be0 con 0x7fa7cc0ce880 2024-04-25T10:02:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0caef0 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 0x7fa958004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7cb7fe640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7cc15da80 0x7fa7cc0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7ca7fc640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc0ce880 0x7fa7cc167980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- 172.21.15.82:0/267105262 wait complete. 2024-04-25T10:02:33.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 Processor -- start 2024-04-25T10:02:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f6201b8c640 1 -- 172.21.15.82:0/3773466232 shutdown_connections 2024-04-25T10:02:33.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa605ffb640 1 -- 172.21.15.82:0/2277017617 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5fc0a8a10 con 0x7fa600033e40 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 -- 172.21.15.82:0/2277017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600033e40 msgr2=0x7fa6000eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa605ffb640 1 -- 172.21.15.82:0/2277017617 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5fc0b5070 con 0x7fa600033e40 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 --2- 172.21.15.82:0/2277017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600033e40 0x7fa6000eb180 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa60003ac20 tx=0x7fa5fc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 -- 172.21.15.82:0/2277017617 shutdown_connections 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 --2- 172.21.15.82:0/2277017617 >> v2:172.21.15.196:3300/0 conn(0x7fa600005ea0 0x7fa6000062b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 --2- 172.21.15.82:0/2277017617 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600033e40 0x7fa6000eb180 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 -- 172.21.15.82:0/2277017617 >> 172.21.15.82:0/2277017617 conn(0x7fa600009fc0 msgr2=0x7fa60000a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 -- 172.21.15.82:0/2277017617 shutdown_connections 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 -- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc0ce880 msgr2=0x7fa7cc167980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7cc0ce880 0x7fa7cc167980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 -- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7cc15da80 msgr2=0x7fa7cc0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7cc15da80 0x7fa7cc0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa7caffd640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc166920 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7efd8286c640 1 --2- 172.21.15.82:0/3829181003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c0cbf40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0cea00 tx=0x7efd7801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acd7fea8fdc2d656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 -- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958003ba0 msgr2=0x7fa958003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958003ba0 0x7fa958003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 -- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9580051d0 msgr2=0x7fa95800b9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9580051d0 0x7fa95800b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96f7fe640 1 -- 172.21.15.82:0/1654390139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa958003530 con 0x7fa958004810 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa97489f640 1 -- 172.21.15.82:0/1654390139 reap_dead start 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa96ffff640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958003ba0 0x7fa958003fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7caffd640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 0x7fa7cc0ad020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e870 tx=0x7fa7c000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7fa60d717640 1 -- 172.21.15.82:0/2277017617 wait complete. 2024-04-25T10:02:33.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd81069640 1 -- 172.21.15.82:0/3829181003 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780b0070 con 0x7efd7c0b1810 2024-04-25T10:02:33.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd81069640 1 -- 172.21.15.82:0/3829181003 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd780ab600 con 0x7efd7c0b1810 2024-04-25T10:02:33.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd81069640 1 -- 172.21.15.82:0/3829181003 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd780ab9d0 con 0x7efd7c0b1810 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.988+0000 7f7c9bfff640 1 -- start start 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 0x7f7c9c206470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c165810 0x7f7c9c0b3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c06e970 con 0x7f7c9c165810 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c06dcd0 con 0x7f7c9c161b80 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c06eaf0 con 0x7f7c9c06e180 2024-04-25T10:02:33.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b50ec640 1 --2- 172.21.15.82:0/1038264008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b00cbff0 0x7f20b00c9a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ce2b0 tx=0x7f20a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130e3b464a728b14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 0x7f7c9c206470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 0x7f7c9c206470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56794/0 (socket says 172.21.15.82:56794) 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 -- 172.21.15.82:0/567497281 learned_addr learned my addr 172.21.15.82:0/567497281 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9a7fc640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 -- 172.21.15.82:0/3829181003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 msgr2=0x7efd7c0cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 --2- 172.21.15.82:0/3829181003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c0cbf40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0cea00 tx=0x7efd7801c430 comp rx=0 tx=0).stop 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 -- 172.21.15.82:0/3829181003 shutdown_connections 2024-04-25T10:02:33.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 --2- 172.21.15.82:0/3829181003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c0cbf40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 --2- 172.21.15.82:0/3829181003 >> v2:172.21.15.82:3300/0 conn(0x7efd7c15d1d0 0x7efd7c0ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 --2- 172.21.15.82:0/3829181003 >> v2:172.21.15.196:3300/0 conn(0x7efd7c15cdf0 0x7efd7c15e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 -- 172.21.15.82:0/3829181003 >> 172.21.15.82:0/3829181003 conn(0x7efd7c0b0cf0 msgr2=0x7efd7c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 Processor -- start 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0017070 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c000c940 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000ccf0 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b48eb640 1 -- 172.21.15.82:0/1038264008 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b0070 con 0x7f20b00cbff0 2024-04-25T10:02:33.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b48eb640 1 -- 172.21.15.82:0/1038264008 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a0004230 con 0x7f20b00cbff0 2024-04-25T10:02:33.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b48eb640 1 -- 172.21.15.82:0/1038264008 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a0003620 con 0x7f20b00cbff0 2024-04-25T10:02:33.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 -- 172.21.15.82:0/3829181003 shutdown_connections 2024-04-25T10:02:33.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9b7fe640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c165810 0x7f7c9c0b3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7efd842cf640 1 -- 172.21.15.82:0/3829181003 wait complete. 2024-04-25T10:02:33.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f6201b8c640 1 -- 172.21.15.82:0/3773466232 wait complete. 2024-04-25T10:02:33.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 -- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 msgr2=0x7f7c9c0b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c161b80 0x7f7c9c0b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 -- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c165810 msgr2=0x7f7c9c0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c165810 0x7f7c9c0b3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0728d0 con 0x7f7c9c06e180 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 0x7fa958004c90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa960009b90 tx=0x7fa96001cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c914719f99954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa960009760 con 0x7fa958004810 2024-04-25T10:02:33.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9600a8410 con 0x7fa958004810 2024-04-25T10:02:33.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa96effd640 1 -- 172.21.15.82:0/1654390139 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9600a87e0 con 0x7fa958004810 2024-04-25T10:02:33.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b00cbff0 msgr2=0x7f20b00c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f20b7b51640 1 --2- 172.21.15.82:0/1038264008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b00cbff0 0x7f20b00c9a50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f20b00ce2b0 tx=0x7f20a001c480 comp rx=0 tx=0).stop 2024-04-25T10:02:33.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9affd640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 0x7f7c9c206470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401a5a0 tx=0x7f7c94019710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc166bc0 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b2070 con 0x7f7c9c06e180 2024-04-25T10:02:33.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c940bf070 con 0x7f7c9c06e180 2024-04-25T10:02:33.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940b9a90 con 0x7f7c9c06e180 2024-04-25T10:02:33.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f6201b8c640 1 Processor -- start 2024-04-25T10:02:33.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc167180 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa7cc04ef30 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 -- start start 2024-04-25T10:02:33.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa600005ea0 0x7fa6000e2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa600033e40 0x7fa6000e0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 0x7fa6000db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60004fd80 con 0x7fa600047c20 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60004f090 con 0x7fa600033e40 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa60d717640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa60004f5c0 con 0x7fa600005ea0 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c0cb4e0 con 0x7f7c9c06e180 2024-04-25T10:02:33.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 0x7fa6000db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa606ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa600005ea0 0x7fa6000e2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa6077fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 0x7fa6000db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48914/0 (socket says 172.21.15.82:48914) 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa6077fe640 1 -- 172.21.15.82:0/3968151457 learned_addr learned my addr 172.21.15.82:0/3968151457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa6067fc640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa600033e40 0x7fa6000e0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f7c9bfff640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c1596a0 con 0x7f7c9c06e180 2024-04-25T10:02:33.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 shutdown_connections 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 --2- 172.21.15.82:0/1038264008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b00cbff0 0x7f20b00c9a50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 --2- 172.21.15.82:0/1038264008 >> v2:172.21.15.196:3300/0 conn(0x7f20b015d050 0x7f20b00cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 >> 172.21.15.82:0/1038264008 conn(0x7f20b00b0cf0 msgr2=0x7f20b00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 Processor -- start 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa976302640 1 -- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 msgr2=0x7fa958004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7fa976302640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 0x7fa958004c90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa960009b90 tx=0x7fa96001cb70 comp rx=0 tx=0).stop 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 -- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa600005ea0 msgr2=0x7fa6000e2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa600005ea0 0x7fa6000e2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 -- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa600033e40 msgr2=0x7fa6000e0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa600033e40 0x7fa6000e0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6000e1a20 con 0x7fa600047c20 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa606ffd640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa600005ea0 0x7fa6000e2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6067fc640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa600033e40 0x7fa6000e0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:33.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 -- start start 2024-04-25T10:02:33.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa6077fe640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 0x7fa6000db6f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c7f0 tx=0x7fa5f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c9bfff640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7c9c15a7f0 con 0x7f7c9c06e180 2024-04-25T10:02:33.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8015070 con 0x7fa600047c20 2024-04-25T10:02:33.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f8004220 con 0x7fa600047c20 2024-04-25T10:02:33.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8003620 con 0x7fa600047c20 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 shutdown_connections 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.992+0000 7f6201b8c640 1 -- start start 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f40399c0 0x7f61f404ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f404d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61f40e3470 0x7f61f4044480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40e39d0 con 0x7f61f4052a60 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40e3850 con 0x7f61f40399c0 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6201b8c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4050e10 con 0x7f61f40e3470 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa60d717640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6000e1d00 con 0x7fa600047c20 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa60d717640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6000e1020 con 0x7fa600047c20 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f6200b8a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f40399c0 0x7f61f404ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f404d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f404d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48926/0 (socket says 172.21.15.82:48926) 2024-04-25T10:02:33.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 -- 172.21.15.82:0/2147519546 learned_addr learned my addr 172.21.15.82:0/2147519546 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 -- 172.21.15.82:0/1654390139 shutdown_connections 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 --2- 172.21.15.82:0/1654390139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa958004810 0x7fa958004c90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 -- 172.21.15.82:0/1654390139 >> 172.21.15.82:0/1654390139 conn(0x7fa9700b0cf0 msgr2=0x7fa9700b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 -- 172.21.15.82:0/1654390139 shutdown_connections 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 -- 172.21.15.82:0/2147519546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61f40e3470 msgr2=0x7f61f4044480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61f40e3470 0x7f61f4044480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 -- 172.21.15.82:0/2147519546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f40399c0 msgr2=0x7f61f404ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61f40399c0 0x7f61f404ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 -- 172.21.15.82:0/2147519546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f4047ac0 con 0x7f61f4052a60 2024-04-25T10:02:33.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 -- 172.21.15.82:0/1654390139 wait complete. 2024-04-25T10:02:33.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa60d717640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6000053a0 con 0x7fa600047c20 2024-04-25T10:02:33.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f61fbfff640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f404d980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c10 tx=0x7f61e80194c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa7b8066f60 0x7fa7b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7c15cdf0 0x7efd7c1f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd7c15d1d0 0x7efd7c1fa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0ca610 con 0x7efd7c15d1d0 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd8186a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7c15cdf0 0x7efd7c1f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0ca0f0 con 0x7efd7c15cdf0 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd8206b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd8186a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7c15cdf0 0x7efd7c1f9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43694/0 (socket says 172.21.15.82:43694) 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd8186a640 1 -- 172.21.15.82:0/913175215 learned_addr learned my addr 172.21.15.82:0/913175215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7efd842cf640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0ca790 con 0x7efd7c0b1810 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f330d3a5640 1 Processor -- start 2024-04-25T10:02:33.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f330d3a5640 1 -- start start 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 Processor -- start 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa976302640 1 -- start start 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5f80037c0 con 0x7fa600047c20 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa5f4067030 0x7fa5f40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa5f8085280 con 0x7fa600047c20 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7fa5f4070730 0x7fa5f4072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5f801ce50 con 0x7fa5f4070730 2024-04-25T10:02:33.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5f808c050 con 0x7fa600047c20 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa7c00859b0 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7ca7fc640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa7b8066f60 0x7fa7b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7fa7b8070660 0x7fa7b8072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa7c0001d40 con 0x7fa7b8070660 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c0085d60 con 0x7fa7cc0ae7a0 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa7cb7fe640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7fa7b8070660 0x7fa7b8072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa7cb7fe640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7fa7b8070660 0x7fa7b8072aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 -- 172.21.15.82:0/913175215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7c15cdf0 msgr2=0x7efd7c1f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd7c15cdf0 0x7efd7c1f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 -- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd7c15d1d0 msgr2=0x7efd7c1fa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd7c15d1d0 0x7efd7c1fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c1548b0 con 0x7efd7c0b1810 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd8206b640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c150b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e840 tx=0x7efd6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f20b7b51640 1 -- 172.21.15.82:0/1038264008 wait complete. 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 Processor -- start 2024-04-25T10:02:33.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f330d3a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f330815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f330d3a5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330815a1b0 con 0x7f3308159ac0 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f330d3a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330815a490 con 0x7f33080cd220 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f3306ffd640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f33080cd220 0x7f33080cd600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48892/0 (socket says 172.21.15.82:48892) 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f3306ffd640 1 -- 172.21.15.82:0/3570177106 learned_addr learned my addr 172.21.15.82:0/3570177106 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f330d3a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33080c5570 con 0x7f330806b0d0 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f33077fe640 1 --2- 172.21.15.82:0/3570177106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f330815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f3305ffb640 1 -- 172.21.15.82:0/3570177106 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039436661 0 0) 0x7f330815a490 con 0x7f33080cd220 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f3305ffb640 1 -- 172.21.15.82:0/3570177106 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32f0005740 con 0x7f33080cd220 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c940b9c30 con 0x7f7c9c06e180 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c88066f90 0x7f7c88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7c94002030 con 0x7f7c9c06e180 2024-04-25T10:02:33.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7f7c88070680 0x7f7c88072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c940b5e50 con 0x7f7c88070680 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:32.996+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c94120f70 con 0x7f7c9c06e180 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f7c9b7fe640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7f7c88070680 0x7f7c88072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f7c9a7fc640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c88066f90 0x7f7c88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f7c9b7fe640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7f7c88070680 0x7f7c88072ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958004810 0x7fa97015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9700b3980 0x7fa9700b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9700ad020 0x7fa9700ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700ca0f0 con 0x7fa9700b3980 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700c9e10 con 0x7fa9700ad020 2024-04-25T10:02:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9700ca610 con 0x7fa958004810 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958004810 0x7fa97015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958004810 0x7fa97015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56836/0 (socket says 172.21.15.82:56836) 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96ffff640 1 -- 172.21.15.82:0/3780346924 learned_addr learned my addr 172.21.15.82:0/3780346924 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9700ad020 0x7fa9700ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 -- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958004810 msgr2=0x7fa97015c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa958004810 0x7fa97015c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 -- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9700b3980 msgr2=0x7fa9700b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa9700b3980 0x7fa9700b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9700b3e00 con 0x7fa9700ad020 2024-04-25T10:02:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c017070 con 0x7efd7c0b1810 2024-04-25T10:02:33.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c00c8e0 con 0x7efd7c0b1810 2024-04-25T10:02:33.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00cc90 con 0x7efd7c0b1810 2024-04-25T10:02:33.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa97489f640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9700ad020 0x7fa9700ad4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa97006e410 tx=0x7fa96400fb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9640079e0 con 0x7fa9700ad020 2024-04-25T10:02:33.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa7ca7fc640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa7b8066f60 0x7fa7b8069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc1fd9b0 tx=0x7fa7bc007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f7c9a7fc640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c88066f90 0x7f7c88069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b1dd0 tx=0x7f7c90009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9700b1e80 con 0x7fa9700ad020 2024-04-25T10:02:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9700b2440 con 0x7fa9700ad020 2024-04-25T10:02:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa606ffd640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa5f4067030 0x7fa5f40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa6067fc640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7fa5f4070730 0x7fa5f4072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa6067fc640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7fa5f4070730 0x7fa5f4072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa606ffd640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa5f4067030 0x7fa5f40694f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0b2000 tx=0x7fa5fc0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa5f801ce50 con 0x7fa5f4070730 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 -- start start 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b00c9f90 0x7f20b00c3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00cbff0 0x7f20b00c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 0x7f20b00ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0166810 con 0x7f20b015d050 2024-04-25T10:02:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b00ca5a0 con 0x7f20b00c9f90 2024-04-25T10:02:33.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b7b51640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b0166990 con 0x7f20b00cbff0 2024-04-25T10:02:33.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 0x7f20b00ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 0x7f20b00ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48960/0 (socket says 172.21.15.82:48960) 2024-04-25T10:02:33.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa7c0001d40 con 0x7fa7b8070660 2024-04-25T10:02:33.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 -- 172.21.15.82:0/3768220966 learned_addr learned my addr 172.21.15.82:0/3768220966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8003720 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e80038c0 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8003c70 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f6201b8c640 1 -- 172.21.15.82:0/2147519546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f4047da0 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f6201b8c640 1 -- 172.21.15.82:0/2147519546 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f404e6d0 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61e80a64e0 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f61f97fa640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61e0067030 0x7f61e00694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f61e811e2f0 con 0x7f61f4052a60 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f6200b8a640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61e0067030 0x7f61e00694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b50ec640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00cbff0 0x7f20b00c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b58ed640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b00c9f90 0x7f20b00c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 -- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00cbff0 msgr2=0x7f20b00c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20b00cbff0 0x7f20b00c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 -- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b00c9f90 msgr2=0x7f20b00c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20b00c9f90 0x7f20b00c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f20b60ee640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00c4fc0 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20b60ee640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 0x7f20b00ad530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c7b0 tx=0x7f20a800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8003db0 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a8005070 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8005460 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b00c52a0 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b00c2f70 con 0x7f20b015d050 2024-04-25T10:02:33.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f20b004ef30 con 0x7f20b015d050 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20a8005600 con 0x7f20b015d050 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2094066fe0 0x7f20940694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f33077fe640 1 -- 172.21.15.82:0/3570177106 >> v1:172.21.15.196:6789/0 conn(0x7f33080cd220 legacy=0x7f33080cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f33077fe640 1 -- 172.21.15.82:0/3570177106 >> v1:172.21.15.82:6789/0 conn(0x7f3308159ac0 legacy=0x7f33080c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f33077fe640 1 -- 172.21.15.82:0/3570177106 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330815a770 con 0x7f330806b0d0 2024-04-25T10:02:33.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7c940b5e50 con 0x7f7c88070680 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964007b80 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa96400a030 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.000+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa950001040 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa964014070 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa95c066f60 0x7fa95c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa964087bf0 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7fa95c0705e0 0x7fa95c072a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa95c0730f0 con 0x7fa95c0705e0 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa964088080 con 0x7fa9700ad020 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7fa95c0705e0 0x7fa95c072a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.004+0000 7fa96f7fe640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7fa95c0705e0 0x7fa95c072a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7fa96ffff640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa95c066f60 0x7fa95c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f20a80851b0 con 0x7f20b015d050 2024-04-25T10:02:33.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa95c0730f0 con 0x7fa95c0705e0 2024-04-25T10:02:33.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f2094070760 0x7f2094072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2094073270 con 0x7f2094070760 2024-04-25T10:02:33.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a808c050 con 0x7f20b015d050 2024-04-25T10:02:33.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f6200b8a640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61e0067030 0x7f61e00694f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61f0007fd0 tx=0x7f61f0007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20b50ec640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f2094070760 0x7f2094072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20b58ed640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2094066fe0 0x7f20940694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20b50ec640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f2094070760 0x7f2094072ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20b58ed640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2094066fe0 0x7f20940694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0093b0 tx=0x7f20ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f33077fe640 1 --2- 172.21.15.82:0/3570177106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f330815d700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f330000af00 tx=0x7f330001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=925dadbd3efe92f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f3305ffb640 1 -- 172.21.15.82:0/3570177106 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000a5610 con 0x7f330806b0d0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f3305ffb640 1 -- 172.21.15.82:0/3570177106 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000a57b0 con 0x7f330806b0d0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f3305ffb640 1 -- 172.21.15.82:0/3570177106 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000a5b60 con 0x7f330806b0d0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f330d3a5640 1 -- 172.21.15.82:0/3570177106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 msgr2=0x7f330815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f330d3a5640 1 --2- 172.21.15.82:0/3570177106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f330815d700 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f330000af00 tx=0x7f330001c950 comp rx=0 tx=0).stop 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7fa96ffff640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa95c066f60 0x7fa95c069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa9700ae830 tx=0x7fa968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2094073270 con 0x7f2094070760 2024-04-25T10:02:33.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f330d3a5640 1 -- 172.21.15.82:0/3570177106 shutdown_connections 2024-04-25T10:02:33.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f330d3a5640 1 --2- 172.21.15.82:0/3570177106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f330815d700 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7f330d3a5640 1 -- 172.21.15.82:0/3570177106 >> 172.21.15.82:0/3570177106 conn(0x7f33080cf820 msgr2=0x7f33080cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd842cf640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c154b90 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd842cf640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c155150 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efd58001040 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd6c01a070 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd60066fb0 0x7efd60069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efd6c0857b0 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7efd60070730 0x7efd60072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efd6c001d40 con 0x7efd60070730 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.008+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== mon.2 v2:172.21.15.196:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd6c085c90 con 0x7efd7c0b1810 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7efd8286c640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7efd60070730 0x7efd60072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7efd8186a640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd60066fb0 0x7efd60069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f6201b8c640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f61f40ccc60 0x7f61f40d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f6201b8c640 1 -- 172.21.15.82:0/2147519546 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61f4008e30 con 0x7f61f40ccc60 2024-04-25T10:02:33.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f620138b640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f61f40ccc60 0x7f61f40d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f620138b640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f61f40ccc60 0x7f61f40d0370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7efd8286c640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7efd60070730 0x7efd60072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f330d3a5640 1 -- 172.21.15.82:0/3570177106 shutdown_connections 2024-04-25T10:02:33.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f330d3a5640 1 -- 172.21.15.82:0/3570177106 wait complete. 2024-04-25T10:02:33.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efd6c001d40 con 0x7efd60070730 2024-04-25T10:02:33.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7efd8186a640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd60066fb0 0x7efd60069470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd7c06d2a0 tx=0x7efd74009970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f61f4008e30 con 0x7f61f40ccc60 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.012+0000 7f330d3a5640 1 Processor -- start 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- start start 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f33081570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33080cd220 0x7f3308151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308159ac0 0x7f33080c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308067c30 con 0x7f3308159ac0 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330806b7b0 con 0x7f33080cd220 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3308067db0 con 0x7f330806b0d0 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33080cd220 0x7f3308151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f3306ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f33081570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33067fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33080cd220 0x7f3308151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43722/0 (socket says 172.21.15.82:43722) 2024-04-25T10:02:33.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33067fc640 1 -- 172.21.15.82:0/297171937 learned_addr learned my addr 172.21.15.82:0/297171937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308159ac0 0x7f33080c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 -- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 msgr2=0x7f33081570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f330806b0d0 0x7f33081570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 -- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33080cd220 msgr2=0x7f3308151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33080cd220 0x7f3308151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33080c8d50 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f33077fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308159ac0 0x7f33080c8400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33000ab000 tx=0x7f33000196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000a5960 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000a5b00 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300004040 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33080c9030 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.016+0000 7f330d3a5640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33080b32f0 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32e0001040 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33000be020 con 0x7f3308159ac0 2024-04-25T10:02:33.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f32f4066f90 0x7f32f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f330011ef40 con 0x7f3308159ac0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7f32f4070690 0x7f32f4072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33000a57b0 con 0x7f32f4070690 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3300124060 con 0x7f3308159ac0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f3306ffd640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f32f4066f90 0x7f32f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f33067fc640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7f32f4070690 0x7f32f4072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f3306ffd640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f32f4066f90 0x7f32f4069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32f8000f70 tx=0x7f32f8004e80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.020+0000 7f33067fc640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7f32f4070690 0x7f32f4072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.024+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f33000a57b0 con 0x7f32f4070690 2024-04-25T10:02:33.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa950002de0 con 0x7fa95c0705e0 2024-04-25T10:02:33.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa96cff9640 1 -- 172.21.15.82:0/3780346924 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa950002de0 con 0x7fa95c0705e0 2024-04-25T10:02:33.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7fa95c0705e0 msgr2=0x7fa95c072a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7fa95c0705e0 0x7fa95c072a20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa95c066f60 msgr2=0x7fa95c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa95c066f60 0x7fa95c069420 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa9700ae830 tx=0x7fa968009040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9700ad020 msgr2=0x7fa9700ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 --2- 172.21.15.82:0/3780346924 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa9700ad020 0x7fa9700ad4b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa97006e410 tx=0x7fa96400fb20 comp rx=0 tx=0).stop 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa97489f640 1 -- 172.21.15.82:0/3780346924 reap_dead start 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 shutdown_connections 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 >> 172.21.15.82:0/3780346924 conn(0x7fa9700b0cf0 msgr2=0x7fa9700aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa7cc0ca220 con 0x7fa7b8070660 2024-04-25T10:02:33.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 shutdown_connections 2024-04-25T10:02:33.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.028+0000 7fa976302640 1 -- 172.21.15.82:0/3780346924 wait complete. 2024-04-25T10:02:33.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7b37fe640 1 -- 172.21.15.82:0/4100805317 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa7cc0ca220 con 0x7fa7b8070660 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7fa7b8070660 msgr2=0x7fa7b8072aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7fa7b8070660 0x7fa7b8072aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa7b8066f60 msgr2=0x7fa7b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa7b8066f60 0x7fa7b8069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc1fd9b0 tx=0x7fa7bc007240 comp rx=0 tx=0).stop 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 msgr2=0x7fa7cc0ad020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7d1636640 1 --2- 172.21.15.82:0/4100805317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7cc0ae7a0 0x7fa7cc0ad020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000e870 tx=0x7fa7c000ec70 comp rx=0 tx=0).stop 2024-04-25T10:02:33.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7fa7cb7fe640 1 -- 172.21.15.82:0/4100805317 reap_dead start 2024-04-25T10:02:33.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f20b006d010 con 0x7f2094070760 2024-04-25T10:02:33.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.032+0000 7f20a67fc640 1 -- 172.21.15.82:0/3768220966 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f20b006d010 con 0x7f2094070760 2024-04-25T10:02:33.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f2094070760 msgr2=0x7f2094072ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f2094070760 0x7f2094072ba0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 shutdown_connections 2024-04-25T10:02:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 >> 172.21.15.82:0/4100805317 conn(0x7fa7cc0b0cf0 msgr2=0x7fa7cc1537e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2094066fe0 msgr2=0x7f20940694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2094066fe0 0x7f20940694a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0093b0 tx=0x7f20ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 msgr2=0x7f20b00ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 --2- 172.21.15.82:0/3768220966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20b015d050 0x7f20b00ad530 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f20a800c7b0 tx=0x7f20a800cc70 comp rx=0 tx=0).stop 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b60ee640 1 -- 172.21.15.82:0/3768220966 reap_dead start 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 shutdown_connections 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 shutdown_connections 2024-04-25T10:02:33.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 >> 172.21.15.82:0/3768220966 conn(0x7f20b00b0cf0 msgr2=0x7f20b00cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7fa7d1636640 1 -- 172.21.15.82:0/4100805317 wait complete. 2024-04-25T10:02:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 shutdown_connections 2024-04-25T10:02:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f20b7b51640 1 -- 172.21.15.82:0/3768220966 wait complete. 2024-04-25T10:02:33.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f7c9bfff640 1 -- 172.21.15.82:0/567497281 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7c9c04ef30 con 0x7f7c88070680 2024-04-25T10:02:33.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f7c83fff640 1 -- 172.21.15.82:0/567497281 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7c9c04ef30 con 0x7f7c88070680 2024-04-25T10:02:33.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f6201b8c640 1 -- 172.21.15.82:0/2147519546 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61f40057f0 con 0x7f61f40ccc60 2024-04-25T10:02:33.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f61f97fa640 1 -- 172.21.15.82:0/2147519546 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f61f40057f0 con 0x7f61f40ccc60 2024-04-25T10:02:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7f7c88070680 msgr2=0x7f7c88072ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.036+0000 7f7c81ffb640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7f7c88070680 0x7f7c88072ac0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c88066f90 msgr2=0x7f7c88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c88066f90 0x7f7c88069450 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0b1dd0 tx=0x7f7c90009190 comp rx=0 tx=0).stop 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 msgr2=0x7f7c9c206470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 --2- 172.21.15.82:0/567497281 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c9c06e180 0x7f7c9c206470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7c9401a5a0 tx=0x7f7c94019710 comp rx=0 tx=0).stop 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c9b7fe640 1 -- 172.21.15.82:0/567497281 reap_dead start 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 shutdown_connections 2024-04-25T10:02:33.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 >> 172.21.15.82:0/567497281 conn(0x7f7c9c06ac90 msgr2=0x7f7c9c0cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efd58002de0 con 0x7efd60070730 2024-04-25T10:02:33.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd72ffd640 1 -- 172.21.15.82:0/913175215 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efd58002de0 con 0x7efd60070730 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7efd60070730 msgr2=0x7efd60072b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7efd60070730 0x7efd60072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd60066fb0 msgr2=0x7efd60069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd60066fb0 0x7efd60069470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efd7c06d2a0 tx=0x7efd74009970 comp rx=0 tx=0).stop 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 msgr2=0x7efd7c150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd70ff9640 1 --2- 172.21.15.82:0/913175215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd7c0b1810 0x7efd7c150b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00e840 tx=0x7efd6c00ed00 comp rx=0 tx=0).stop 2024-04-25T10:02:33.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.040+0000 7efd8286c640 1 -- 172.21.15.82:0/913175215 reap_dead start 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 shutdown_connections 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 >> 172.21.15.82:0/913175215 conn(0x7efd7c0b0cf0 msgr2=0x7efd7c0cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 shutdown_connections 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 shutdown_connections 2024-04-25T10:02:33.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7efd70ff9640 1 -- 172.21.15.82:0/913175215 wait complete. 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.044+0000 7f7c81ffb640 1 -- 172.21.15.82:0/567497281 wait complete. 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32e0002de0 con 0x7f32f4070690 2024-04-25T10:02:33.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ef7fe640 1 -- 172.21.15.82:0/297171937 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f32e0002de0 con 0x7f32f4070690 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7f32f4070690 msgr2=0x7f32f4072ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7f32f4070690 0x7f32f4072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f32f4066f90 msgr2=0x7f32f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f32f4066f90 0x7f32f4069450 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f32f8000f70 tx=0x7f32f8004e80 comp rx=0 tx=0).stop 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308159ac0 msgr2=0x7f33080c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 --2- 172.21.15.82:0/297171937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3308159ac0 0x7f33080c8400 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f33000ab000 tx=0x7f33000196f0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f33077fe640 1 -- 172.21.15.82:0/297171937 reap_dead start 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 shutdown_connections 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 >> 172.21.15.82:0/297171937 conn(0x7f33080cf820 msgr2=0x7f330815d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 shutdown_connections 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.048+0000 7f32ed7fa640 1 -- 172.21.15.82:0/297171937 wait complete. 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa60d717640 1 -- 172.21.15.82:0/3968151457 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa600037420 con 0x7fa5f4070730 2024-04-25T10:02:33.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ef7fe640 1 -- 172.21.15.82:0/3968151457 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa600037420 con 0x7fa5f4070730 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f61f40ccc60 msgr2=0x7f61f40d0370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f61f40ccc60 0x7f61f40d0370 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61e0067030 msgr2=0x7f61e00694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61e0067030 0x7f61e00694f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f61f0007fd0 tx=0x7f61f0007d60 comp rx=0 tx=0).stop 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 msgr2=0x7f61f404d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f61e6ffd640 1 --2- 172.21.15.82:0/2147519546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61f4052a60 0x7f61f404d980 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f61e8000c10 tx=0x7f61e80194c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7f620138b640 1 -- 172.21.15.82:0/2147519546 reap_dead start 2024-04-25T10:02:33.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7fa5f4070730 msgr2=0x7fa5f4072b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7fa5f4070730 0x7fa5f4072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa5f4067030 msgr2=0x7fa5f40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa5f4067030 0x7fa5f40694f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc0b2000 tx=0x7fa5fc0aa070 comp rx=0 tx=0).stop 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 msgr2=0x7fa6000db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 --2- 172.21.15.82:0/3968151457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa600047c20 0x7fa6000db6f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800c7f0 tx=0x7fa5f800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa6077fe640 1 -- 172.21.15.82:0/3968151457 reap_dead start 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 shutdown_connections 2024-04-25T10:02:33.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 >> 172.21.15.82:0/3968151457 conn(0x7fa600009fc0 msgr2=0x7fa600052f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.089 INFO:teuthology.orchestra.run.smithi082.stdout:55834574858 2024-04-25T10:02:33.090 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:33.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 shutdown_connections 2024-04-25T10:02:33.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.052+0000 7fa5ed7fa640 1 -- 172.21.15.82:0/3968151457 wait complete. 2024-04-25T10:02:33.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.068+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 shutdown_connections 2024-04-25T10:02:33.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.068+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 >> 172.21.15.82:0/2147519546 conn(0x7f61f400a410 msgr2=0x7f61f40532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.092 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.093 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.068+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 shutdown_connections 2024-04-25T10:02:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.068+0000 7f61e6ffd640 1 -- 172.21.15.82:0/2147519546 wait complete. 2024-04-25T10:02:33.094 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.094 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:33.095 INFO:teuthology.orchestra.run.smithi082.stdout:55834574858 2024-04-25T10:02:33.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:33.096 INFO:teuthology.orchestra.run.smithi082.stdout:55834574860 2024-04-25T10:02:33.096 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:33.097 INFO:teuthology.orchestra.run.smithi082.stdout:55834574860 2024-04-25T10:02:33.097 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:33.098 INFO:teuthology.orchestra.run.smithi082.stdout:55834574857 2024-04-25T10:02:33.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:33.099 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:33.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 Processor -- start 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 -- start start 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc815d1d0 0x7f0fc815e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc80aea30 con 0x7f0fc815cdf0 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc80c9850 con 0x7f0fc80cc650 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcfcda640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc8072a30 con 0x7f0fc815d1d0 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc815d1d0 0x7f0fc815e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0fc815d1d0 0x7f0fc815e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48978/0 (socket says 172.21.15.82:48978) 2024-04-25T10:02:33.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 -- 172.21.15.82:0/2389218204 learned_addr learned my addr 172.21.15.82:0/2389218204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 -- 172.21.15.82:0/2389218204 >> v1:172.21.15.196:6789/0 conn(0x7f0fc80cc650 legacy=0x7f0fc806d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 -- 172.21.15.82:0/2389218204 >> v1:172.21.15.82:6789/0 conn(0x7f0fc815cdf0 legacy=0x7f0fc80cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.296+0000 7f0fcd275640 1 -- 172.21.15.82:0/2389218204 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc80c9b30 con 0x7f0fc815d1d0 2024-04-25T10:02:33.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 Processor -- start 2024-04-25T10:02:33.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f0fcd275640 1 --2- 172.21.15.82:0/2389218204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc815e170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d1a0 tx=0x7f0fb801ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c3db8151f273d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f0fcca74640 1 -- 172.21.15.82:0/2389218204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80ad070 con 0x7f0fc815d1d0 2024-04-25T10:02:33.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f0fcca74640 1 -- 172.21.15.82:0/2389218204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb80a7d50 con 0x7f0fc815d1d0 2024-04-25T10:02:33.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f0fcca74640 1 -- 172.21.15.82:0/2389218204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb80b1620 con 0x7f0fc815d1d0 2024-04-25T10:02:33.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 -- start start 2024-04-25T10:02:33.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f3757e20640 1 Processor -- start 2024-04-25T10:02:33.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7f3757e20640 1 -- start start 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec78159a80 0x7fec780aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780c5950 0x7fec780cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec780acb10 con 0x7fec780cd600 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec780ace40 con 0x7fec78159a80 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7dcd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c5110 con 0x7fec780c5950 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7ccd6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec78159a80 0x7fec780aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7ccd6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec78159a80 0x7fec780aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43726/0 (socket says 172.21.15.82:43726) 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7ccd6640 1 -- 172.21.15.82:0/98689728 learned_addr learned my addr 172.21.15.82:0/98689728 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec7d4d7640 1 --1- 172.21.15.82:0/98689728 >> v1:172.21.15.196:6789/0 conn(0x7fec780cd600 0x7fec780ad2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48922/0 (socket says 172.21.15.82:48922) 2024-04-25T10:02:33.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.300+0000 7fec77fff640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780c5950 0x7fec780cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- 172.21.15.82:0/2389218204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 msgr2=0x7f0fc815e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 --2- 172.21.15.82:0/2389218204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc815e170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0fb800d1a0 tx=0x7f0fb801ec80 comp rx=0 tx=0).stop 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- 172.21.15.82:0/2389218204 shutdown_connections 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 --2- 172.21.15.82:0/2389218204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc815e170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- 172.21.15.82:0/2389218204 >> 172.21.15.82:0/2389218204 conn(0x7f0fc80b0cf0 msgr2=0x7f0fc80b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- 172.21.15.82:0/2389218204 shutdown_connections 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7fec7ccd6640 1 -- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780c5950 msgr2=0x7fec780cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7fec7ccd6640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780c5950 0x7fec780cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7fec7ccd6640 1 -- 172.21.15.82:0/98689728 >> v1:172.21.15.196:6789/0 conn(0x7fec780cd600 legacy=0x7fec780ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3757e20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f375015b2f0 0x7f375015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3757e20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37500ce880 0x7f37500cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3757e20640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37500cd100 con 0x7f37500cddd0 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3757e20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375015e6e0 con 0x7f375015b2f0 2024-04-25T10:02:33.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3757e20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375015e9c0 con 0x7f37500ce880 2024-04-25T10:02:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7fec7ccd6640 1 -- 172.21.15.82:0/98689728 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780c53f0 con 0x7fec78159a80 2024-04-25T10:02:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f375015b2f0 0x7f375015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f375015b2f0 0x7f375015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48980/0 (socket says 172.21.15.82:48980) 2024-04-25T10:02:33.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 -- 172.21.15.82:0/1102040457 learned_addr learned my addr 172.21.15.82:0/1102040457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f37553bb640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37500ce880 0x7f37500cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 -- 172.21.15.82:0/1102040457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37500ce880 msgr2=0x7f37500cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37500ce880 0x7f37500cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 -- 172.21.15.82:0/1102040457 >> v1:172.21.15.196:6789/0 conn(0x7f37500cddd0 legacy=0x7f37500cbe10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f3755bbc640 1 -- 172.21.15.82:0/1102040457 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37500cd370 con 0x7f375015b2f0 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 Processor -- start 2024-04-25T10:02:33.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 -- start start 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- 172.21.15.82:0/2389218204 wait complete. 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f4290163e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4290164380 0x7f4290164810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42900ce400 con 0x7f4290097e80 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900cf010 con 0x7f4290164380 2024-04-25T10:02:33.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 Processor -- start 2024-04-25T10:02:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428ffff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900ce6e0 con 0x7f42900b1910 2024-04-25T10:02:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 Processor -- start 2024-04-25T10:02:33.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 -- start start 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4290097e80 0x7f4290098260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57966/0 (socket says 172.21.15.82:57966) 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428effd640 1 -- 172.21.15.82:0/1968958072 learned_addr learned my addr 172.21.15.82:0/1968958072 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428f7fe640 1 --2- 172.21.15.82:0/1968958072 >> v2:172.21.15.196:3300/0 conn(0x7f4290164380 0x7f4290164810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428e7fc640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f4290163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- start start 2024-04-25T10:02:33.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106452047 0 0) 0x7f42900ce400 con 0x7f4290097e80 2024-04-25T10:02:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4278003530 con 0x7f4290097e80 2024-04-25T10:02:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7fec7ccd6640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78159a80 0x7fec780aec30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec7000d370 tx=0x7fec700215d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e07a1d6762c948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc80cc650 0x7f0fc8207610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc815cdf0 0x7f0fc80b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80ca610 con 0x7f0fc815d1d0 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc806e150 con 0x7f0fc80cc650 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcfcda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc80ca790 con 0x7f0fc815cdf0 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 0x7f06340b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 0x7f06340b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063415b990 con 0x7f063415b5b0 2024-04-25T10:02:33.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063406da40 con 0x7f06340ce2f0 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f063b412640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0634070d00 con 0x7f06340cd720 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fce277640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fcda76640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc80cc650 0x7f0fc8207610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f0fce277640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48986/0 (socket says 172.21.15.82:48986) 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257077600 0 0) 0x7f4278003530 con 0x7f4290097e80 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428f7fe640 1 -- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 msgr2=0x7f4290163e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428f7fe640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f4290163e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428f7fe640 1 -- 172.21.15.82:0/1968958072 >> v1:172.21.15.82:6789/0 conn(0x7f4290097e80 legacy=0x7f4290098260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428f7fe640 1 -- 172.21.15.82:0/1968958072 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900ce9c0 con 0x7f4290164380 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.304+0000 7f428e7fc640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f4290163e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 -- 172.21.15.82:0/631417526 learned_addr learned my addr 172.21.15.82:0/631417526 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fcd275640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc815cdf0 0x7f0fc80b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063a410640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f063415b5b0 0x7f06340cdbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48938/0 (socket says 172.21.15.82:48938) 2024-04-25T10:02:33.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063a410640 1 -- 172.21.15.82:0/1122677693 learned_addr learned my addr 172.21.15.82:0/1122677693 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063ac11640 1 --2- 172.21.15.82:0/1122677693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 0x7f06340b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 Processor -- start 2024-04-25T10:02:33.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 -- start start 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 -- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc815cdf0 msgr2=0x7f0fc80b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc815cdf0 0x7f0fc80b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 -- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc80cc650 msgr2=0x7f0fc8207610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc80cc650 0x7f0fc8207610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 -- 172.21.15.82:0/631417526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc8207b50 con 0x7f0fc815d1d0 2024-04-25T10:02:33.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3755bbc640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f375015ddb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0ea0e24290838f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fce277640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c710 tx=0x7f0fc000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0639c0f640 1 --2- 172.21.15.82:0/1122677693 >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 0x7f06340b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0016020 con 0x7f0fc815d1d0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fcfcda640 1 -- 172.21.15.82:0/631417526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc82039b0 con 0x7f0fc815d1d0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fcfcda640 1 -- 172.21.15.82:0/631417526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc8203f70 con 0x7f0fc815d1d0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc0006d60 con 0x7f0fc815d1d0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0003bc0 con 0x7f0fc815d1d0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3754bba640 1 -- 172.21.15.82:0/1102040457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400ab610 con 0x7f375015b2f0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3754bba640 1 -- 172.21.15.82:0/1102040457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37400b0070 con 0x7f375015b2f0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3754bba640 1 -- 172.21.15.82:0/1102040457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37400ab9d0 con 0x7f375015b2f0 2024-04-25T10:02:33.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 Processor -- start 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 -- start start 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 -- 172.21.15.82:0/1102040457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 msgr2=0x7f375015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f375015ddb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3740009870 tx=0x7f374001c520 comp rx=0 tx=0).stop 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fec777fe640 1 -- 172.21.15.82:0/98689728 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec700b8070 con 0x7fec78159a80 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fec777fe640 1 -- 172.21.15.82:0/98689728 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700b33f0 con 0x7fec78159a80 2024-04-25T10:02:33.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fec777fe640 1 -- 172.21.15.82:0/98689728 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7000be60 con 0x7fec78159a80 2024-04-25T10:02:33.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f428f7fe640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290164380 0x7f4290164810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4288009d90 tx=0x7f428801c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e521f3fa1bad316f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 -- 172.21.15.82:0/1102040457 shutdown_connections 2024-04-25T10:02:33.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37500ce880 0x7f37500cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 --2- 172.21.15.82:0/1102040457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f375015ddb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 -- 172.21.15.82:0/1102040457 >> 172.21.15.82:0/1102040457 conn(0x7f37500b0cf0 msgr2=0x7f37500b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fc000a850 con 0x7f0fc815d1d0 2024-04-25T10:02:33.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fec7dcd8640 1 -- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78159a80 msgr2=0x7fec780aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78159a80 0x7fec780aec30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fec7000d370 tx=0x7fec700215d0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 -- 172.21.15.82:0/1102040457 shutdown_connections 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f0fbe7fc640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 0x7f0fac0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0fc0084e80 con 0x7f0fc815d1d0 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f0fcfcda640 1 -- 172.21.15.82:0/631417526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb00052a0 con 0x7f0fc815d1d0 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f3757e20640 1 -- 172.21.15.82:0/1102040457 wait complete. 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 -- 172.21.15.82:0/98689728 shutdown_connections 2024-04-25T10:02:33.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780c5950 0x7fec780cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/98689728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec78159a80 0x7fec780aec30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 -- 172.21.15.82:0/98689728 >> 172.21.15.82:0/98689728 conn(0x7fec780b0df0 msgr2=0x7fec780b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063ac11640 1 -- 172.21.15.82:0/1122677693 >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 msgr2=0x7f06340b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063ac11640 1 --2- 172.21.15.82:0/1122677693 >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 0x7f06340b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063ac11640 1 -- 172.21.15.82:0/1122677693 >> v1:172.21.15.196:6789/0 conn(0x7f063415b5b0 legacy=0x7f06340cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f063ac11640 1 -- 172.21.15.82:0/1122677693 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063406dc70 con 0x7f06340cd720 2024-04-25T10:02:33.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f0639c0f640 1 --2- 172.21.15.82:0/1122677693 >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 0x7f06340b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f957c0540b0 0x7f957c006f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c057df0 con 0x7f957c0e32e0 2024-04-25T10:02:33.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c0580d0 con 0x7f957c053c20 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958a39e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c03bfd0 con 0x7f957c0540b0 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f95837fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f957c053c20 0x7f957c006800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48948/0 (socket says 172.21.15.82:48948) 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f95837fe640 1 -- 172.21.15.82:0/3770187102 learned_addr learned my addr 172.21.15.82:0/3770187102 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958893b640 1 --2- 172.21.15.82:0/3770187102 >> v2:172.21.15.82:3300/0 conn(0x7f957c0540b0 0x7f957c006f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958893b640 1 -- 172.21.15.82:0/3770187102 >> v1:172.21.15.196:6789/0 conn(0x7f957c053c20 legacy=0x7f957c006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958893b640 1 -- 172.21.15.82:0/3770187102 >> v1:172.21.15.82:6789/0 conn(0x7f957c0e32e0 legacy=0x7f957c03e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f958893b640 1 -- 172.21.15.82:0/3770187102 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0583b0 con 0x7f957c0540b0 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00ae7a0 0x7fb0a015da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb0a006e3a0 0x7fb0a00cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a01611c0 con 0x7fb0a00cbf70 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a0072770 con 0x7fb0a006e3a0 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a8227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0071b60 con 0x7fb0a00ae7a0 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a67c4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb0a00cbf70 0x7fb0a00ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57972/0 (socket says 172.21.15.82:57972) 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a67c4640 1 -- 172.21.15.82:0/766077611 learned_addr learned my addr 172.21.15.82:0/766077611 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a57c2640 1 --2- 172.21.15.82:0/766077611 >> v2:172.21.15.82:3300/0 conn(0x7fb0a006e3a0 0x7fb0a00cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a57c2640 1 -- 172.21.15.82:0/766077611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00ae7a0 msgr2=0x7fb0a015da80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a57c2640 1 --2- 172.21.15.82:0/766077611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00ae7a0 0x7fb0a015da80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a57c2640 1 -- 172.21.15.82:0/766077611 >> v1:172.21.15.82:6789/0 conn(0x7fb0a00cbf70 legacy=0x7fb0a00ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a57c2640 1 -- 172.21.15.82:0/766077611 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a0071e40 con 0x7fb0a006e3a0 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7fb0a4fc1640 1 -- 172.21.15.82:0/766077611 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247568560 0 0) 0x7fb0a01611c0 con 0x7fb0a00cbf70 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880ac070 con 0x7f4290164380 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880a6410 con 0x7f4290164380 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.308+0000 7f428dffb640 1 -- 172.21.15.82:0/1968958072 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280001680 con 0x7f4290164380 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290164380 msgr2=0x7f4290164810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290164380 0x7f4290164810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4288009d90 tx=0x7f428801c800 comp rx=0 tx=0).stop 2024-04-25T10:02:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f0fcda76640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 0x7f0fac0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063ac11640 1 --2- 172.21.15.82:0/1122677693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 0x7f06340b3490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d90 tx=0x7f062c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f6c8a776078767 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 -- 172.21.15.82:0/98689728 shutdown_connections 2024-04-25T10:02:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- 172.21.15.82:0/1968958072 shutdown_connections 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f4290163e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- 172.21.15.82:0/1968958072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290164380 0x7f4290164810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- 172.21.15.82:0/1968958072 >> 172.21.15.82:0/1968958072 conn(0x7f42900b0df0 msgr2=0x7f42900b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f0fcda76640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 0x7f0fac0694f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8165950 tx=0x7f0fc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- 172.21.15.82:0/1968958072 shutdown_connections 2024-04-25T10:02:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f3757e20640 1 Processor -- start 2024-04-25T10:02:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- 172.21.15.82:0/1968958072 wait complete. 2024-04-25T10:02:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 Processor -- start 2024-04-25T10:02:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fb0a57c2640 1 --2- 172.21.15.82:0/766077611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a006e3a0 0x7fb0a00cba30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097c0 tx=0x7fb09c02d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd4743bf6e4ffa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063940e640 1 -- 172.21.15.82:0/1122677693 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0b2070 con 0x7f06340cd720 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063940e640 1 -- 172.21.15.82:0/1122677693 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0a5740 con 0x7f06340cd720 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063940e640 1 -- 172.21.15.82:0/1122677693 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c0a5b10 con 0x7f06340cd720 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f958893b640 1 --2- 172.21.15.82:0/3770187102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c006f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95840751c0 tx=0x7f9584096da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d049538dbd4277 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063b412640 1 -- 172.21.15.82:0/1122677693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 msgr2=0x7f06340b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f063b412640 1 --2- 172.21.15.82:0/1122677693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 0x7f06340b3490 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f062c009d90 tx=0x7f062c01c8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 -- 172.21.15.82:0/98689728 wait complete. 2024-04-25T10:02:33.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 Processor -- start 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- start start 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f429014c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290164380 0x7f429014c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429016ecf0 con 0x7f4290164380 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429016eb70 con 0x7f4290097e80 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428ffff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429016ee70 con 0x7f42900b1910 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428effd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43760/0 (socket says 172.21.15.82:43760) 2024-04-25T10:02:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f428effd640 1 -- 172.21.15.82:0/1822009954 learned_addr learned my addr 172.21.15.82:0/1822009954 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- 172.21.15.82:0/1122677693 shutdown_connections 2024-04-25T10:02:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 --2- 172.21.15.82:0/1122677693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340cd720 0x7f06340b3490 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 --2- 172.21.15.82:0/1122677693 >> v2:172.21.15.82:3300/0 conn(0x7f06340ce2f0 0x7f06340b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- 172.21.15.82:0/1122677693 >> 172.21.15.82:0/1122677693 conn(0x7f063406ac90 msgr2=0x7f063406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428e7fc640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f429014c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7f3757e20640 1 -- start start 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 0x7f3750158780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37501609b0 0x7f37500ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375006d6f0 con 0x7f375015b2f0 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375006d570 con 0x7f37500ce880 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3757e20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37500cc520 con 0x7f37501609b0 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.312+0000 7fec7dcd8640 1 -- start start 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec780c5950 0x7fec78154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780cd600 0x7fec780b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec78159e60 con 0x7fec78159a80 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec780c56d0 con 0x7fec780c5950 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 -- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 msgr2=0x7f429014c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f429014c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 -- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290164380 msgr2=0x7f429014c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290164380 0x7f429014c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 -- 172.21.15.82:0/1822009954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900b3a10 con 0x7f4290097e80 2024-04-25T10:02:33.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7dcd8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec78159fe0 con 0x7fec780cd600 2024-04-25T10:02:33.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- 172.21.15.82:0/1122677693 shutdown_connections 2024-04-25T10:02:33.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49030/0 (socket says 172.21.15.82:49030) 2024-04-25T10:02:33.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 -- 172.21.15.82:0/3708274756 learned_addr learned my addr 172.21.15.82:0/3708274756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7ccd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec780c5950 0x7fec78154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49020/0 (socket says 172.21.15.82:49020) 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 -- 172.21.15.82:0/4128177166 learned_addr learned my addr 172.21.15.82:0/4128177166 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec77fff640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780cd600 0x7fec780b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428effd640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f428000c700 tx=0x7f428000cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37563bd640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37501609b0 0x7f37500ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3755bbc640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 0x7f3750158780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 -- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37501609b0 msgr2=0x7f37500ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37501609b0 0x7f37500ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 -- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 msgr2=0x7f3750158780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 0x7f3750158780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 -- 172.21.15.82:0/3708274756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37500c3de0 con 0x7f375015b2f0 2024-04-25T10:02:33.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f3755bbc640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 0x7f3750158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 -- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780cd600 msgr2=0x7fec780b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780cd600 0x7fec780b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 -- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec780c5950 msgr2=0x7fec78154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec780c5950 0x7fec78154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 -- 172.21.15.82:0/4128177166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780cdb30 con 0x7fec78159a80 2024-04-25T10:02:33.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f37553bb640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e930 tx=0x7f374c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 Processor -- start 2024-04-25T10:02:33.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 -- start start 2024-04-25T10:02:33.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7fec7d4d7640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c3c0 tx=0x7fec6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- 172.21.15.82:0/1122677693 wait complete. 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 Processor -- start 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280005ae0 con 0x7f4290097e80 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280005c80 con 0x7f4290097e80 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428ffff640 1 -- 172.21.15.82:0/1822009954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42901fb190 con 0x7f4290097e80 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280007290 con 0x7f4290097e80 2024-04-25T10:02:33.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f428ffff640 1 -- 172.21.15.82:0/1822009954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42901fb750 con 0x7f4290097e80 2024-04-25T10:02:33.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f59d8052ca0 0x7f59d80355b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d8055640 con 0x7f59d8055260 2024-04-25T10:02:33.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d8037450 con 0x7f59d8036fa0 2024-04-25T10:02:33.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59e4a9a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d803b670 con 0x7f59d8052ca0 2024-04-25T10:02:33.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59deffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f59d8052ca0 0x7f59d80355b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59dffff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f59d8036fa0 0x7f59d80df8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:48952/0 (socket says 172.21.15.82:48952) 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59dffff640 1 -- 172.21.15.82:0/217079725 learned_addr learned my addr 172.21.15.82:0/217079725 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59de7fc640 1 -- 172.21.15.82:0/217079725 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404154413 0 0) 0x7f59d8037450 con 0x7f59d8036fa0 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f59de7fc640 1 -- 172.21.15.82:0/217079725 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d0003530 con 0x7f59d8036fa0 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- start start 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340ce2f0 0x7f0634153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 0x7f0634154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06340ce9a0 con 0x7f06340cd720 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063406e7d0 con 0x7f063415b5b0 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.316+0000 7f063b412640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06341664a0 con 0x7f06340ce2f0 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49046/0 (socket says 172.21.15.82:49046) 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 -- 172.21.15.82:0/1350301330 learned_addr learned my addr 172.21.15.82:0/1350301330 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f428ffff640 1 -- 172.21.15.82:0/1822009954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429004ef30 con 0x7f4290097e80 2024-04-25T10:02:33.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fc0020020 con 0x7f0fc815d1d0 2024-04-25T10:02:33.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f9582ffd640 1 -- 172.21.15.82:0/3770187102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958411fa00 con 0x7f957c0540b0 2024-04-25T10:02:33.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680074e0 con 0x7fec78159a80 2024-04-25T10:02:33.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c017070 con 0x7f375015b2f0 2024-04-25T10:02:33.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37500c40c0 con 0x7f375015b2f0 2024-04-25T10:02:33.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37500c4680 con 0x7f375015b2f0 2024-04-25T10:02:33.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4280007430 con 0x7f4290097e80 2024-04-25T10:02:33.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f0639c0f640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340ce2f0 0x7f0634153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 -- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340ce2f0 msgr2=0x7f0634153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340ce2f0 0x7f0634153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 -- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 msgr2=0x7f0634154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063ac11640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 0x7f0634154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 0x7f0634154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 -- 172.21.15.82:0/1350301330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340c94d0 con 0x7f06340cd720 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063ac11640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 0x7f0634154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063a410640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f062400b460 tx=0x7f062400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a4fc1640 1 -- 172.21.15.82:0/766077611 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0c1070 con 0x7fb0a006e3a0 2024-04-25T10:02:33.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a4fc1640 1 -- 172.21.15.82:0/766077611 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c0b5e50 con 0x7fb0a006e3a0 2024-04-25T10:02:33.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a4fc1640 1 -- 172.21.15.82:0/766077611 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0bc820 con 0x7fb0a006e3a0 2024-04-25T10:02:33.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f9582ffd640 1 -- 172.21.15.82:0/3770187102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958411fba0 con 0x7f957c0540b0 2024-04-25T10:02:33.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f9582ffd640 1 -- 172.21.15.82:0/3770187102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584127640 con 0x7f957c0540b0 2024-04-25T10:02:33.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624015020 con 0x7f06340cd720 2024-04-25T10:02:33.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec780b3870 con 0x7fec78159a80 2024-04-25T10:02:33.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec780b3e30 con 0x7fec78159a80 2024-04-25T10:02:33.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f42777fe640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4280085ac0 con 0x7f4290097e80 2024-04-25T10:02:33.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f428e7fc640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 -- 172.21.15.82:0/766077611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a006e3a0 msgr2=0x7fb0a00cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 --2- 172.21.15.82:0/766077611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a006e3a0 0x7fb0a00cba30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0097c0 tx=0x7fb09c02d4d0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 -- 172.21.15.82:0/766077611 shutdown_connections 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 --2- 172.21.15.82:0/766077611 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a006e3a0 0x7fb0a00cba30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 --2- 172.21.15.82:0/766077611 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00ae7a0 0x7fb0a015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 -- 172.21.15.82:0/766077611 >> 172.21.15.82:0/766077611 conn(0x7fb0a00b0cf0 msgr2=0x7fb0a00b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063b412640 1 -- 172.21.15.82:0/1350301330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634153d90 con 0x7f06340cd720 2024-04-25T10:02:33.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f063b412640 1 -- 172.21.15.82:0/1350301330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0634154350 con 0x7f06340cd720 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00ca00 con 0x7f375015b2f0 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c021060 con 0x7f375015b2f0 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec68007680 con 0x7fec78159a80 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec680049f0 con 0x7fec78159a80 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06240118b0 con 0x7f06340cd720 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0624011ca0 con 0x7f06340cd720 2024-04-25T10:02:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f958a39e640 1 -- 172.21.15.82:0/3770187102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 msgr2=0x7f957c006f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f958a39e640 1 --2- 172.21.15.82:0/3770187102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c006f10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f95840751c0 tx=0x7f9584096da0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7fb0a8227640 1 -- 172.21.15.82:0/766077611 shutdown_connections 2024-04-25T10:02:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f428e7fc640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 0x7f427c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4284009580 tx=0x7f4284009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f063b412640 1 -- 172.21.15.82:0/1350301330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c0052a0 con 0x7f06340cd720 2024-04-25T10:02:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f59deffd640 1 -- 172.21.15.82:0/217079725 >> v1:172.21.15.196:6789/0 conn(0x7f59d8036fa0 legacy=0x7f59d80df8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f59deffd640 1 -- 172.21.15.82:0/217079725 >> v1:172.21.15.82:6789/0 conn(0x7f59d8055260 legacy=0x7f59d8052590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.320+0000 7f59deffd640 1 -- 172.21.15.82:0/217079725 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d803aa60 con 0x7f59d8052ca0 2024-04-25T10:02:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f59deffd640 1 --2- 172.21.15.82:0/217079725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8052ca0 0x7f59d80355b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0097f0 tx=0x7f59cc01e6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa892b90fcf48708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f958a39e640 1 -- 172.21.15.82:0/3770187102 shutdown_connections 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f958a39e640 1 --2- 172.21.15.82:0/3770187102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c006f10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f958a39e640 1 -- 172.21.15.82:0/3770187102 >> 172.21.15.82:0/3770187102 conn(0x7f957c0559f0 msgr2=0x7f957c055e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec580052a0 con 0x7fec78159a80 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37380052a0 con 0x7f375015b2f0 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f374c01a070 con 0x7f375015b2f0 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f37467fc640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 0x7f3734069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0624018020 con 0x7f06340cd720 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f374c0539e0 con 0x7f375015b2f0 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f3755bbc640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 0x7f3734069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f062affd640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 0x7f0618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0624085920 con 0x7f06340cd720 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f3755bbc640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 0x7f3734069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37400af000 tx=0x7f3740008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f0639c0f640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 0x7f0618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec6801a020 con 0x7fec78159a80 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fec757fa640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 0x7fec64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fec68084a50 con 0x7fec78159a80 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f0639c0f640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 0x7f0618069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06341533f0 tx=0x7f0630008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fec7ccd6640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 0x7fec64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f958a39e640 1 -- 172.21.15.82:0/3770187102 shutdown_connections 2024-04-25T10:02:33.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7fb0a8227640 1 -- 172.21.15.82:0/766077611 wait complete. 2024-04-25T10:02:33.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.324+0000 7f958a39e640 1 -- 172.21.15.82:0/3770187102 wait complete. 2024-04-25T10:02:33.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 Processor -- start 2024-04-25T10:02:33.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 Processor -- start 2024-04-25T10:02:33.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 -- start start 2024-04-25T10:02:33.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 -- start start 2024-04-25T10:02:33.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 0x7f957c0e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a00ae7a0 0x7fb0a00ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00cbf70 0x7fb0a00adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a0072c00 con 0x7fb0a00ae7a0 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a006e800 con 0x7fb0a006e3a0 2024-04-25T10:02:33.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a8227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a015bf90 con 0x7fb0a00cbf70 2024-04-25T10:02:33.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c00d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f957c1845b0 0x7f957c184a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0f0e90 con 0x7f957c0540b0 2024-04-25T10:02:33.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fec7ccd6640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 0x7fec64069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec700b5000 tx=0x7fec7000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f9583fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 0x7f957c0e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f9583fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 0x7f957c0e1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:56946/0 (socket says 172.21.15.82:56946) 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f9583fff640 1 -- 172.21.15.82:0/4122840721 learned_addr learned my addr 172.21.15.82:0/4122840721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0f0d10 con 0x7f957c1845b0 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0f1010 con 0x7f957c053c20 2024-04-25T10:02:33.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c00d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a67c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00cbf70 0x7fb0a00adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43794/0 (socket says 172.21.15.82:43794) 2024-04-25T10:02:33.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 -- 172.21.15.82:0/1138173011 learned_addr learned my addr 172.21.15.82:0/1138173011 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f958893b640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f957c1845b0 0x7f957c184a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 -- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 msgr2=0x7f957c0e1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 0x7f957c0e1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 -- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f957c1845b0 msgr2=0x7f957c184a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f957c1845b0 0x7f957c184a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7f95837fe640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0e0730 con 0x7f957c0540b0 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 -- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00cbf70 msgr2=0x7fb0a00adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00cbf70 0x7fb0a00adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 -- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a00ae7a0 msgr2=0x7fb0a00ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a00ae7a0 0x7fb0a00ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.328+0000 7fb0a5fc3640 1 -- 172.21.15.82:0/1138173011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a015b380 con 0x7fb0a006e3a0 2024-04-25T10:02:33.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f95837fe640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c00d580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f957000c520 tx=0x7f957000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7fb0a5fc3640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09800ba80 tx=0x7fb09800bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f062401f020 con 0x7f06340cd720 2024-04-25T10:02:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098019070 con 0x7fb0a006e3a0 2024-04-25T10:02:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f59de7fc640 1 -- 172.21.15.82:0/217079725 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0ac650 con 0x7f59d8052ca0 2024-04-25T10:02:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f59de7fc640 1 -- 172.21.15.82:0/217079725 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc0b1070 con 0x7f59d8052ca0 2024-04-25T10:02:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f59de7fc640 1 -- 172.21.15.82:0/217079725 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0aca30 con 0x7f59d8052ca0 2024-04-25T10:02:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4280007720 con 0x7f4290097e80 2024-04-25T10:02:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 -- 172.21.15.82:0/217079725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8052ca0 msgr2=0x7f59d80355b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a015b660 con 0x7fb0a006e3a0 2024-04-25T10:02:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.332+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a015bc20 con 0x7fb0a006e3a0 2024-04-25T10:02:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/217079725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8052ca0 0x7f59d80355b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0097f0 tx=0x7f59cc01e6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800cc00 con 0x7fb0a006e3a0 2024-04-25T10:02:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098014440 con 0x7fb0a006e3a0 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 -- 172.21.15.82:0/217079725 shutdown_connections 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/217079725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8052ca0 0x7f59d80355b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 -- 172.21.15.82:0/217079725 >> 172.21.15.82:0/217079725 conn(0x7f59d801ef50 msgr2=0x7f59d8055c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 -- 172.21.15.82:0/217079725 shutdown_connections 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 -- 172.21.15.82:0/217079725 wait complete. 2024-04-25T10:02:33.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0a015c400 con 0x7fb0a006e3a0 2024-04-25T10:02:33.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb09801c070 con 0x7fb0a006e3a0 2024-04-25T10:02:33.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0967fc640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 0x7fb084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570018070 con 0x7f957c0540b0 2024-04-25T10:02:33.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f374c056040 con 0x7f375015b2f0 2024-04-25T10:02:33.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f59e4a9a640 1 Processor -- start 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c0e2250 con 0x7f957c0540b0 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c0e2af0 con 0x7f957c0540b0 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c0057f0 con 0x7f957c0540b0 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0a57c2640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 0x7fb084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb098085070 con 0x7fb0a006e3a0 2024-04-25T10:02:33.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.336+0000 7fb0a57c2640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 0x7fb084069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0be000 tx=0x7fb09c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 -- start start 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d8036fa0 0x7f59d80d43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59d8052ca0 0x7f59d804a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d80072d0 con 0x7f59d8052ca0 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d80e03f0 con 0x7f59d8055260 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59e4a9a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d8007450 con 0x7f59d8036fa0 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:43824/0 (socket says 172.21.15.82:43824) 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 -- 172.21.15.82:0/1562863488 learned_addr learned my addr 172.21.15.82:0/1562863488 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59deffd640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59d8052ca0 0x7f59d804a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 -- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d8036fa0 msgr2=0x7f59d80d43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d8036fa0 0x7f59d80d43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 -- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59d8052ca0 msgr2=0x7f59d804a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59d8052ca0 0x7f59d804a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 -- 172.21.15.82:0/1562863488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d804ad40 con 0x7f59d8055260 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.340+0000 7f59dffff640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59e0069050 tx=0x7f59e0074660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700074e0 con 0x7f957c0540b0 2024-04-25T10:02:33.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570022060 con 0x7f957c0540b0 2024-04-25T10:02:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec68054fb0 con 0x7fec78159a80 2024-04-25T10:02:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f957001b070 con 0x7f957c0540b0 2024-04-25T10:02:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7f9580ff9640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 0x7f9568069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.344+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9570084c00 con 0x7f957c0540b0 2024-04-25T10:02:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f9583fff640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 0x7f9568069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f9583fff640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 0x7f9568069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f957c039e00 tx=0x7f9578009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e00685f0 con 0x7f59d8055260 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e0094020 con 0x7f59d8055260 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e0073d80 con 0x7f59d8055260 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d804b020 con 0x7f59d8055260 2024-04-25T10:02:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d804b5e0 con 0x7f59d8055260 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.348+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0a015c400 con 0x7fb0a006e3a0 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.352+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59e006c110 con 0x7f59d8055260 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.352+0000 7f59cbfff640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 0x7f59c40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.352+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f59e0103c50 con 0x7f59d8055260 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.352+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d80057f0 con 0x7f59d8055260 2024-04-25T10:02:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.356+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9570055280 con 0x7f957c0540b0 2024-04-25T10:02:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.360+0000 7f59df7fe640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 0x7f59c40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.360+0000 7f59df7fe640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 0x7f59c40694f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59d4000f70 tx=0x7f59d40079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.364+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59e009e030 con 0x7f59d8055260 2024-04-25T10:02:33.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.640+0000 7f0fcfcda640 1 -- 172.21.15.82:0/631417526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0fb0005590 con 0x7f0fc815d1d0 2024-04-25T10:02:33.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.640+0000 7f0fbe7fc640 1 -- 172.21.15.82:0/631417526 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fc8207b50 con 0x7f0fc815d1d0 2024-04-25T10:02:33.649 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 msgr2=0x7f0fac0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 0x7f0fac0694f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8165950 tx=0x7f0fc4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 msgr2=0x7f0fc80c8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c710 tx=0x7f0fc000cbd0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 shutdown_connections 2024-04-25T10:02:33.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0fac067030 0x7f0fac0694f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fc815cdf0 0x7f0fc80b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fc815d1d0 0x7f0fc80c8400 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.644+0000 7f0fabfff640 1 --2- 172.21.15.82:0/631417526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fc80cc650 0x7f0fc8207610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 >> 172.21.15.82:0/631417526 conn(0x7f0fc80b0cf0 msgr2=0x7f0fc80adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 shutdown_connections 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb0a004ef30 con 0x7fb0a006e3a0 2024-04-25T10:02:33.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7f0fabfff640 1 -- 172.21.15.82:0/631417526 wait complete. 2024-04-25T10:02:33.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7fb0967fc640 1 -- 172.21.15.82:0/1138173011 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb098055740 con 0x7fb0a006e3a0 2024-04-25T10:02:33.655 INFO:teuthology.orchestra.run.smithi082.stdout:55834574857 2024-04-25T10:02:33.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7f063b412640 1 -- 172.21.15.82:0/1350301330 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f061c005860 con 0x7f06340cd720 2024-04-25T10:02:33.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.648+0000 7f062affd640 1 -- 172.21.15.82:0/1350301330 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f062400d040 con 0x7f06340cd720 2024-04-25T10:02:33.660 INFO:teuthology.orchestra.run.smithi082.stdout:55834574860 2024-04-25T10:02:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 msgr2=0x7f0618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 0x7f0618069420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f06341533f0 tx=0x7f0630008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 msgr2=0x7f0634152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f062400b460 tx=0x7f062400b920 comp rx=0 tx=0).stop 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 shutdown_connections 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0618066f60 0x7f0618069420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f06340ce2f0 0x7f0634153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f063415b5b0 0x7f0634154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f0628ff9640 1 --2- 172.21.15.82:0/1350301330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f06340cd720 0x7f0634152b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.652+0000 7f428ffff640 1 -- 172.21.15.82:0/1822009954 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4290072f90 con 0x7f4290097e80 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 >> 172.21.15.82:0/1350301330 conn(0x7f063406ac90 msgr2=0x7f063406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 msgr2=0x7fb084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 0x7fb084069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb09c0be000 tx=0x7fb09c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 msgr2=0x7fb0a014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb09800ba80 tx=0x7fb09800bf40 comp rx=0 tx=0).stop 2024-04-25T10:02:33.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 shutdown_connections 2024-04-25T10:02:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 shutdown_connections 2024-04-25T10:02:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb084066f60 0x7fb084069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0a00cbf70 0x7fb0a00adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0a00ae7a0 0x7fb0a00ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 --2- 172.21.15.82:0/1138173011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0a006e3a0 0x7fb0a014df60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 >> 172.21.15.82:0/1138173011 conn(0x7fb0a00b0cf0 msgr2=0x7fb0a00cd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7f0628ff9640 1 -- 172.21.15.82:0/1350301330 wait complete. 2024-04-25T10:02:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7f42777fe640 1 -- 172.21.15.82:0/1822009954 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4280056870 con 0x7f4290097e80 2024-04-25T10:02:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 shutdown_connections 2024-04-25T10:02:33.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.656+0000 7fb0a8227640 1 -- 172.21.15.82:0/1138173011 wait complete. 2024-04-25T10:02:33.666 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 msgr2=0x7f427c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 0x7f427c069420 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4284009580 tx=0x7f4284009190 comp rx=0 tx=0).stop 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 msgr2=0x7f42900c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f428000c700 tx=0x7f428000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 shutdown_connections 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f427c066f60 0x7f427c069420 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fec58005590 con 0x7fec78159a80 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42900b1910 0x7f429014c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4290164380 0x7f429014c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 --2- 172.21.15.82:0/1822009954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4290097e80 0x7f42900c80e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 >> 172.21.15.82:0/1822009954 conn(0x7f42900b0df0 msgr2=0x7f42900b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3738005590 con 0x7f375015b2f0 2024-04-25T10:02:33.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7fec757fa640 1 -- 172.21.15.82:0/4128177166 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fec68059050 con 0x7fec78159a80 2024-04-25T10:02:33.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 shutdown_connections 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi082.stdout:55834574859 2024-04-25T10:02:33.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f37467fc640 1 -- 172.21.15.82:0/3708274756 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f374c055940 con 0x7f375015b2f0 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi082.stdout:55834574858 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.664+0000 7f42757fa640 1 -- 172.21.15.82:0/1822009954 wait complete. 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 msgr2=0x7f3734069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 0x7f3734069450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f37400af000 tx=0x7f3740008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 msgr2=0x7f37500c3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f374c00e930 tx=0x7f374c00edf0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 msgr2=0x7fec64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 0x7fec64069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec700b5000 tx=0x7fec7000b040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 msgr2=0x7fec78158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fec6800c3c0 tx=0x7fec6800c880 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 shutdown_connections 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec64066f60 0x7fec64069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec780cd600 0x7fec780b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec78159a80 0x7fec78158470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 --2- 172.21.15.82:0/4128177166 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec780c5950 0x7fec78154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 >> 172.21.15.82:0/4128177166 conn(0x7fec780b0df0 msgr2=0x7fec780b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 shutdown_connections 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3734066f90 0x7f3734069450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37501609b0 0x7f37500ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f375015b2f0 0x7f37500c3670 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 --2- 172.21.15.82:0/3708274756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37500ce880 0x7f3750158780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 >> 172.21.15.82:0/3708274756 conn(0x7f37500b0cf0 msgr2=0x7f37500cbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 shutdown_connections 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7fec7dcd8640 1 -- 172.21.15.82:0/4128177166 wait complete. 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 shutdown_connections 2024-04-25T10:02:33.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.668+0000 7f3757e20640 1 -- 172.21.15.82:0/3708274756 wait complete. 2024-04-25T10:02:33.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.2 2024-04-25T10:02:33.679 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.676+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f957c038e70 con 0x7f957c0540b0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.676+0000 7f9580ff9640 1 -- 172.21.15.82:0/4122840721 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9570059320 con 0x7f957c0540b0 2024-04-25T10:02:33.683 INFO:teuthology.orchestra.run.smithi082.stdout:55834574860 2024-04-25T10:02:33.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 msgr2=0x7f9568069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 0x7f9568069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f957c039e00 tx=0x7f9578009040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 msgr2=0x7f957c00d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c00d580 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f957000c520 tx=0x7f957000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 shutdown_connections 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9568066f60 0x7f9568069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f957c1845b0 0x7f957c184a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f957c0540b0 0x7f957c00d580 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 --2- 172.21.15.82:0/4122840721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f957c053c20 0x7f957c0e1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 >> 172.21.15.82:0/4122840721 conn(0x7f957c0559f0 msgr2=0x7f957c03cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 shutdown_connections 2024-04-25T10:02:33.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.680+0000 7f958a39e640 1 -- 172.21.15.82:0/4122840721 wait complete. 2024-04-25T10:02:33.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.684+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f59d800f3d0 con 0x7f59d8055260 2024-04-25T10:02:33.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.684+0000 7f59cbfff640 1 -- 172.21.15.82:0/1562863488 <== mon.1 v2:172.21.15.196:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59e00d4900 con 0x7f59d8055260 2024-04-25T10:02:33.691 INFO:teuthology.orchestra.run.smithi082.stdout:55834574858 2024-04-25T10:02:33.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.5 2024-04-25T10:02:33.694 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-04-25T10:02:33.696 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 msgr2=0x7f59c40694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 0x7f59c40694f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59d4000f70 tx=0x7f59d40079c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 msgr2=0x7f59d804a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59e0069050 tx=0x7f59e0074660 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 shutdown_connections 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f59c4067030 0x7f59c40694f0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f59d4000f70 tx=0x7f59d40079c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59d8055260 0x7f59d804a700 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f59d8052ca0 0x7f59d804a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 --2- 172.21.15.82:0/1562863488 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f59d8036fa0 0x7f59d80d43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 >> 172.21.15.82:0/1562863488 conn(0x7f59d801ef50 msgr2=0x7f59d800b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 shutdown_connections 2024-04-25T10:02:33.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.688+0000 7f59e4a9a640 1 -- 172.21.15.82:0/1562863488 wait complete. 2024-04-25T10:02:33.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T10:02:33.701 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T10:02:33.708 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-25T10:02:33.710 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.712 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.3 2024-04-25T10:02:33.712 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T10:02:33.719 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:33.719 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:02:33.719 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.856+0000 7f10ff1d2640 1 Processor -- start 2024-04-25T10:02:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.856+0000 7f10ff1d2640 1 -- start start 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10ff1d2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10f80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10ff1d2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f10f8144ca0 0x7f10f806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10ff1d2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f80cee30 con 0x7f10f806ec60 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10ff1d2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f80cc2b0 con 0x7f10f8144ca0 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10ff1d2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f806e290 con 0x7f10f80ce510 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10fd76f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f10f806ec60 0x7f10f8142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58006/0 (socket says 172.21.15.82:58006) 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10fd76f640 1 -- 172.21.15.82:0/2977379424 learned_addr learned my addr 172.21.15.82:0/2977379424 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 --2- 172.21.15.82:0/2977379424 >> v2:172.21.15.196:3300/0 conn(0x7f10f8144ca0 0x7f10f806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828525292 0 0) 0x7f10f80cee30 con 0x7f10f806ec60 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10fcf6e640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10f80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10e4003530 con 0x7f10f806ec60 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2542610501 0 0) 0x7f10e4003530 con 0x7f10f806ec60 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 msgr2=0x7f10f80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10f80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 >> v2:172.21.15.196:3300/0 conn(0x7f10f8144ca0 msgr2=0x7f10f806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 --2- 172.21.15.82:0/2977379424 >> v2:172.21.15.196:3300/0 conn(0x7f10f8144ca0 0x7f10f806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80ccc70 con 0x7f10f806ec60 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f10f0001680 con 0x7f10f806ec60 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 >> v1:172.21.15.82:6789/0 conn(0x7f10f806ec60 legacy=0x7f10f8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10e4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e4004d50 0x7f10e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80cc2b0 con 0x7f10e4003fb0 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f806e290 con 0x7f10e4004d50 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e400ba80 con 0x7f10f80ce510 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10fcf6e640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10e4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 -- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 msgr2=0x7f10e4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f80ce510 0x7f10e4003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 -- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e4004d50 msgr2=0x7f10e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10e4004d50 0x7f10e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 -- 172.21.15.82:0/2977379424 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e4003530 con 0x7f10e4003fb0 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f7fff640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10e4004810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009ab0 tx=0x7f10e001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa461b1c1825e5f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00b2070 con 0x7f10e4003fb0 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e00a5680 con 0x7f10e4003fb0 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.860+0000 7f10f77fe640 1 -- 172.21.15.82:0/2977379424 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00096b0 con 0x7f10e4003fb0 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/2977379424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 msgr2=0x7f10e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/2977379424 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10e4004810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f10e0009ab0 tx=0x7f10e001ca90 comp rx=0 tx=0).stop 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fd76f640 1 -- 172.21.15.82:0/2977379424 reap_dead start 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/2977379424 shutdown_connections 2024-04-25T10:02:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/2977379424 >> 172.21.15.82:0/2977379424 conn(0x7f10f80b11a0 msgr2=0x7f10f80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/2977379424 shutdown_connections 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/2977379424 wait complete. 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 Processor -- start 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- start start 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f81e2da0 0x7f10f80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81e3520 0x7f10f81e39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80cc900 con 0x7f10e4003fb0 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f806df20 con 0x7f10f81e3520 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8142d90 con 0x7f10f81e2da0 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49082/0 (socket says 172.21.15.82:49082) 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 -- 172.21.15.82:0/3418527763 learned_addr learned my addr 172.21.15.82:0/3418527763 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10f7fff640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f81e2da0 0x7f10f80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fd76f640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81e3520 0x7f10f81e39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 -- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f81e2da0 msgr2=0x7f10f80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f81e2da0 0x7f10f80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 -- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81e3520 msgr2=0x7f10f81e39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81e3520 0x7f10f81e39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 -- 172.21.15.82:0/3418527763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f81e3200 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10fcf6e640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10e800f380 tx=0x7f10e800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e801c070 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8017620 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80c93c0 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8017a10 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.864+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80c9970 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f804efa0 con 0x7f10e4003fb0 2024-04-25T10:02:33.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10e801f070 con 0x7f10e4003fb0 2024-04-25T10:02:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10f57fa640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 0x7f10d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f10e80859d0 con 0x7f10e4003fb0 2024-04-25T10:02:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10f7fff640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 0x7f10d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.868+0000 7f10f7fff640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 0x7f10d8069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10e00af000 tx=0x7f10e00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:33.872+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10e8012070 con 0x7f10e4003fb0 2024-04-25T10:02:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.012+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10f8144a00 con 0x7f10d8066f60 2024-04-25T10:02:34.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.020+0000 7f10f57fa640 1 -- 172.21.15.82:0/3418527763 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f10e010e310 con 0x7f10d8066f60 2024-04-25T10:02:34.032 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:34.035 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:02:34.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 msgr2=0x7f10d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 0x7f10d8069420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f10e00af000 tx=0x7f10e00046d0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 msgr2=0x7f10f81f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10e800f380 tx=0x7f10e800f840 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 shutdown_connections 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10d8066f60 0x7f10d8069420 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10f81e2da0 0x7f10f80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10f81e3520 0x7f10f81e39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 --2- 172.21.15.82:0/3418527763 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10e4003fb0 0x7f10f81f56c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 >> 172.21.15.82:0/3418527763 conn(0x7f10f80b11a0 msgr2=0x7f10f80ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 shutdown_connections 2024-04-25T10:02:34.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.028+0000 7f10ff1d2640 1 -- 172.21.15.82:0/3418527763 wait complete. 2024-04-25T10:02:34.051 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:02:33.267200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223632,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751088,"statfs":{"total":773094113280,"available":772865114112,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004073"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462685+0000","last_change":"2024-04-25T10:02:06.377944+0000","last_active":"2024-04-25T10:02:13.462685+0000","last_peered":"2024-04-25T10:02:13.462685+0000","last_clean":"2024-04-25T10:02:13.462685+0000","last_became_active":"2024-04-25T10:02:06.377162+0000","last_became_peered":"2024-04-25T10:02:06.377162+0000","last_unstale":"2024-04-25T10:02:13.462685+0000","last_undegraded":"2024-04-25T10:02:13.462685+0000","last_fullsized":"2024-04-25T10:02:13.462685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406752+0000","last_change":"2024-04-25T10:02:07.375159+0000","last_active":"2024-04-25T10:02:12.406752+0000","last_peered":"2024-04-25T10:02:12.406752+0000","last_clean":"2024-04-25T10:02:12.406752+0000","last_became_active":"2024-04-25T10:02:07.374938+0000","last_became_peered":"2024-04-25T10:02:07.374938+0000","last_unstale":"2024-04-25T10:02:12.406752+0000","last_undegraded":"2024-04-25T10:02:12.406752+0000","last_fullsized":"2024-04-25T10:02:12.406752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404043+0000","last_change":"2024-04-25T10:02:11.459079+0000","last_active":"2024-04-25T10:02:12.404043+0000","last_peered":"2024-04-25T10:02:12.404043+0000","last_clean":"2024-04-25T10:02:12.404043+0000","last_became_active":"2024-04-25T10:02:11.458605+0000","last_became_peered":"2024-04-25T10:02:11.458605+0000","last_unstale":"2024-04-25T10:02:12.404043+0000","last_undegraded":"2024-04-25T10:02:12.404043+0000","last_fullsized":"2024-04-25T10:02:12.404043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407158+0000","last_change":"2024-04-25T10:02:06.377863+0000","last_active":"2024-04-25T10:02:12.407158+0000","last_peered":"2024-04-25T10:02:12.407158+0000","last_clean":"2024-04-25T10:02:12.407158+0000","last_became_active":"2024-04-25T10:02:06.377260+0000","last_became_peered":"2024-04-25T10:02:06.377260+0000","last_unstale":"2024-04-25T10:02:12.407158+0000","last_undegraded":"2024-04-25T10:02:12.407158+0000","last_fullsized":"2024-04-25T10:02:12.407158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462420+0000","last_change":"2024-04-25T10:02:07.371882+0000","last_active":"2024-04-25T10:02:13.462420+0000","last_peered":"2024-04-25T10:02:13.462420+0000","last_clean":"2024-04-25T10:02:13.462420+0000","last_became_active":"2024-04-25T10:02:07.371244+0000","last_became_peered":"2024-04-25T10:02:07.371244+0000","last_unstale":"2024-04-25T10:02:13.462420+0000","last_undegraded":"2024-04-25T10:02:13.462420+0000","last_fullsized":"2024-04-25T10:02:13.462420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405866+0000","last_change":"2024-04-25T10:02:11.435335+0000","last_active":"2024-04-25T10:02:12.405866+0000","last_peered":"2024-04-25T10:02:12.405866+0000","last_clean":"2024-04-25T10:02:12.405866+0000","last_became_active":"2024-04-25T10:02:11.434197+0000","last_became_peered":"2024-04-25T10:02:11.434197+0000","last_unstale":"2024-04-25T10:02:12.405866+0000","last_undegraded":"2024-04-25T10:02:12.405866+0000","last_fullsized":"2024-04-25T10:02:12.405866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:43.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465851+0000","last_change":"2024-04-25T10:02:06.374878+0000","last_active":"2024-04-25T10:02:13.465851+0000","last_peered":"2024-04-25T10:02:13.465851+0000","last_clean":"2024-04-25T10:02:13.465851+0000","last_became_active":"2024-04-25T10:02:06.374210+0000","last_became_peered":"2024-04-25T10:02:06.374210+0000","last_unstale":"2024-04-25T10:02:13.465851+0000","last_undegraded":"2024-04-25T10:02:13.465851+0000","last_fullsized":"2024-04-25T10:02:13.465851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465104+0000","last_change":"2024-04-25T10:02:07.382381+0000","last_active":"2024-04-25T10:02:13.465104+0000","last_peered":"2024-04-25T10:02:13.465104+0000","last_clean":"2024-04-25T10:02:13.465104+0000","last_became_active":"2024-04-25T10:02:07.381718+0000","last_became_peered":"2024-04-25T10:02:07.381718+0000","last_unstale":"2024-04-25T10:02:13.465104+0000","last_undegraded":"2024-04-25T10:02:13.465104+0000","last_fullsized":"2024-04-25T10:02:13.465104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407612+0000","last_change":"2024-04-25T10:02:11.442974+0000","last_active":"2024-04-25T10:02:12.407612+0000","last_peered":"2024-04-25T10:02:12.407612+0000","last_clean":"2024-04-25T10:02:12.407612+0000","last_became_active":"2024-04-25T10:02:11.441674+0000","last_became_peered":"2024-04-25T10:02:11.441674+0000","last_unstale":"2024-04-25T10:02:12.407612+0000","last_undegraded":"2024-04-25T10:02:12.407612+0000","last_fullsized":"2024-04-25T10:02:12.407612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464230+0000","last_change":"2024-04-25T10:02:06.388913+0000","last_active":"2024-04-25T10:02:13.464230+0000","last_peered":"2024-04-25T10:02:13.464230+0000","last_clean":"2024-04-25T10:02:13.464230+0000","last_became_active":"2024-04-25T10:02:06.388536+0000","last_became_peered":"2024-04-25T10:02:06.388536+0000","last_unstale":"2024-04-25T10:02:13.464230+0000","last_undegraded":"2024-04-25T10:02:13.464230+0000","last_fullsized":"2024-04-25T10:02:13.464230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:41.104882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465595+0000","last_change":"2024-04-25T10:02:07.382060+0000","last_active":"2024-04-25T10:02:13.465595+0000","last_peered":"2024-04-25T10:02:13.465595+0000","last_clean":"2024-04-25T10:02:13.465595+0000","last_became_active":"2024-04-25T10:02:07.381779+0000","last_became_peered":"2024-04-25T10:02:07.381779+0000","last_unstale":"2024-04-25T10:02:13.465595+0000","last_undegraded":"2024-04-25T10:02:13.465595+0000","last_fullsized":"2024-04-25T10:02:13.465595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406133+0000","last_change":"2024-04-25T10:02:11.430607+0000","last_active":"2024-04-25T10:02:12.406133+0000","last_peered":"2024-04-25T10:02:12.406133+0000","last_clean":"2024-04-25T10:02:12.406133+0000","last_became_active":"2024-04-25T10:02:11.429281+0000","last_became_peered":"2024-04-25T10:02:11.429281+0000","last_unstale":"2024-04-25T10:02:12.406133+0000","last_undegraded":"2024-04-25T10:02:12.406133+0000","last_fullsized":"2024-04-25T10:02:12.406133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:06.213948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405179+0000","last_change":"2024-04-25T10:02:07.365923+0000","last_active":"2024-04-25T10:02:12.405179+0000","last_peered":"2024-04-25T10:02:12.405179+0000","last_clean":"2024-04-25T10:02:12.405179+0000","last_became_active":"2024-04-25T10:02:07.365345+0000","last_became_peered":"2024-04-25T10:02:07.365345+0000","last_unstale":"2024-04-25T10:02:12.405179+0000","last_undegraded":"2024-04-25T10:02:12.405179+0000","last_fullsized":"2024-04-25T10:02:12.405179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465328+0000","last_change":"2024-04-25T10:02:06.373663+0000","last_active":"2024-04-25T10:02:13.465328+0000","last_peered":"2024-04-25T10:02:13.465328+0000","last_clean":"2024-04-25T10:02:13.465328+0000","last_became_active":"2024-04-25T10:02:06.372972+0000","last_became_peered":"2024-04-25T10:02:06.372972+0000","last_unstale":"2024-04-25T10:02:13.465328+0000","last_undegraded":"2024-04-25T10:02:13.465328+0000","last_fullsized":"2024-04-25T10:02:13.465328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408366+0000","last_change":"2024-04-25T10:02:11.432916+0000","last_active":"2024-04-25T10:02:12.408366+0000","last_peered":"2024-04-25T10:02:12.408366+0000","last_clean":"2024-04-25T10:02:12.408366+0000","last_became_active":"2024-04-25T10:02:11.431758+0000","last_became_peered":"2024-04-25T10:02:11.431758+0000","last_unstale":"2024-04-25T10:02:12.408366+0000","last_undegraded":"2024-04-25T10:02:12.408366+0000","last_fullsized":"2024-04-25T10:02:12.408366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403824+0000","last_change":"2024-04-25T10:02:07.371074+0000","last_active":"2024-04-25T10:02:12.403824+0000","last_peered":"2024-04-25T10:02:12.403824+0000","last_clean":"2024-04-25T10:02:12.403824+0000","last_became_active":"2024-04-25T10:02:07.370770+0000","last_became_peered":"2024-04-25T10:02:07.370770+0000","last_unstale":"2024-04-25T10:02:12.403824+0000","last_undegraded":"2024-04-25T10:02:12.403824+0000","last_fullsized":"2024-04-25T10:02:12.403824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462663+0000","last_change":"2024-04-25T10:02:06.369892+0000","last_active":"2024-04-25T10:02:13.462663+0000","last_peered":"2024-04-25T10:02:13.462663+0000","last_clean":"2024-04-25T10:02:13.462663+0000","last_became_active":"2024-04-25T10:02:06.369183+0000","last_became_peered":"2024-04-25T10:02:06.369183+0000","last_unstale":"2024-04-25T10:02:13.462663+0000","last_undegraded":"2024-04-25T10:02:13.462663+0000","last_fullsized":"2024-04-25T10:02:13.462663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406907+0000","last_change":"2024-04-25T10:02:11.442628+0000","last_active":"2024-04-25T10:02:12.406907+0000","last_peered":"2024-04-25T10:02:12.406907+0000","last_clean":"2024-04-25T10:02:12.406907+0000","last_became_active":"2024-04-25T10:02:11.441578+0000","last_became_peered":"2024-04-25T10:02:11.441578+0000","last_unstale":"2024-04-25T10:02:12.406907+0000","last_undegraded":"2024-04-25T10:02:12.406907+0000","last_fullsized":"2024-04-25T10:02:12.406907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:00.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403861+0000","last_change":"2024-04-25T10:02:07.361735+0000","last_active":"2024-04-25T10:02:12.403861+0000","last_peered":"2024-04-25T10:02:12.403861+0000","last_clean":"2024-04-25T10:02:12.403861+0000","last_became_active":"2024-04-25T10:02:07.361435+0000","last_became_peered":"2024-04-25T10:02:07.361435+0000","last_unstale":"2024-04-25T10:02:12.403861+0000","last_undegraded":"2024-04-25T10:02:12.403861+0000","last_fullsized":"2024-04-25T10:02:12.403861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461678+0000","last_change":"2024-04-25T10:02:06.369621+0000","last_active":"2024-04-25T10:02:13.461678+0000","last_peered":"2024-04-25T10:02:13.461678+0000","last_clean":"2024-04-25T10:02:13.461678+0000","last_became_active":"2024-04-25T10:02:06.368883+0000","last_became_peered":"2024-04-25T10:02:06.368883+0000","last_unstale":"2024-04-25T10:02:13.461678+0000","last_undegraded":"2024-04-25T10:02:13.461678+0000","last_fullsized":"2024-04-25T10:02:13.461678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462909+0000","last_change":"2024-04-25T10:02:11.462237+0000","last_active":"2024-04-25T10:02:13.462909+0000","last_peered":"2024-04-25T10:02:13.462909+0000","last_clean":"2024-04-25T10:02:13.462909+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.462909+0000","last_undegraded":"2024-04-25T10:02:13.462909+0000","last_fullsized":"2024-04-25T10:02:13.462909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406991+0000","last_change":"2024-04-25T10:02:07.357629+0000","last_active":"2024-04-25T10:02:12.406991+0000","last_peered":"2024-04-25T10:02:12.406991+0000","last_clean":"2024-04-25T10:02:12.406991+0000","last_became_active":"2024-04-25T10:02:07.357353+0000","last_became_peered":"2024-04-25T10:02:07.357353+0000","last_unstale":"2024-04-25T10:02:12.406991+0000","last_undegraded":"2024-04-25T10:02:12.406991+0000","last_fullsized":"2024-04-25T10:02:12.406991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:17.663068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464640+0000","last_change":"2024-04-25T10:02:06.384551+0000","last_active":"2024-04-25T10:02:13.464640+0000","last_peered":"2024-04-25T10:02:13.464640+0000","last_clean":"2024-04-25T10:02:13.464640+0000","last_became_active":"2024-04-25T10:02:06.384131+0000","last_became_peered":"2024-04-25T10:02:06.384131+0000","last_unstale":"2024-04-25T10:02:13.464640+0000","last_undegraded":"2024-04-25T10:02:13.464640+0000","last_fullsized":"2024-04-25T10:02:13.464640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408508+0000","last_change":"2024-04-25T10:02:11.443507+0000","last_active":"2024-04-25T10:02:12.408508+0000","last_peered":"2024-04-25T10:02:12.408508+0000","last_clean":"2024-04-25T10:02:12.408508+0000","last_became_active":"2024-04-25T10:02:11.440496+0000","last_became_peered":"2024-04-25T10:02:11.440496+0000","last_unstale":"2024-04-25T10:02:12.408508+0000","last_undegraded":"2024-04-25T10:02:12.408508+0000","last_fullsized":"2024-04-25T10:02:12.408508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462520+0000","last_change":"2024-04-25T10:02:07.367892+0000","last_active":"2024-04-25T10:02:13.462520+0000","last_peered":"2024-04-25T10:02:13.462520+0000","last_clean":"2024-04-25T10:02:13.462520+0000","last_became_active":"2024-04-25T10:02:07.367404+0000","last_became_peered":"2024-04-25T10:02:07.367404+0000","last_unstale":"2024-04-25T10:02:13.462520+0000","last_undegraded":"2024-04-25T10:02:13.462520+0000","last_fullsized":"2024-04-25T10:02:13.462520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406984+0000","last_change":"2024-04-25T10:02:06.371009+0000","last_active":"2024-04-25T10:02:12.406984+0000","last_peered":"2024-04-25T10:02:12.406984+0000","last_clean":"2024-04-25T10:02:12.406984+0000","last_became_active":"2024-04-25T10:02:06.369415+0000","last_became_peered":"2024-04-25T10:02:06.369415+0000","last_unstale":"2024-04-25T10:02:12.406984+0000","last_undegraded":"2024-04-25T10:02:12.406984+0000","last_fullsized":"2024-04-25T10:02:12.406984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406594+0000","last_change":"2024-04-25T10:02:11.460234+0000","last_active":"2024-04-25T10:02:12.406594+0000","last_peered":"2024-04-25T10:02:12.406594+0000","last_clean":"2024-04-25T10:02:12.406594+0000","last_became_active":"2024-04-25T10:02:11.459268+0000","last_became_peered":"2024-04-25T10:02:11.459268+0000","last_unstale":"2024-04-25T10:02:12.406594+0000","last_undegraded":"2024-04-25T10:02:12.406594+0000","last_fullsized":"2024-04-25T10:02:12.406594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:35.022355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406871+0000","last_change":"2024-04-25T10:02:07.385204+0000","last_active":"2024-04-25T10:02:12.406871+0000","last_peered":"2024-04-25T10:02:12.406871+0000","last_clean":"2024-04-25T10:02:12.406871+0000","last_became_active":"2024-04-25T10:02:07.384798+0000","last_became_peered":"2024-04-25T10:02:07.384798+0000","last_unstale":"2024-04-25T10:02:12.406871+0000","last_undegraded":"2024-04-25T10:02:12.406871+0000","last_fullsized":"2024-04-25T10:02:12.406871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462229+0000","last_change":"2024-04-25T10:02:06.363593+0000","last_active":"2024-04-25T10:02:13.462229+0000","last_peered":"2024-04-25T10:02:13.462229+0000","last_clean":"2024-04-25T10:02:13.462229+0000","last_became_active":"2024-04-25T10:02:06.363127+0000","last_became_peered":"2024-04-25T10:02:06.363127+0000","last_unstale":"2024-04-25T10:02:13.462229+0000","last_undegraded":"2024-04-25T10:02:13.462229+0000","last_fullsized":"2024-04-25T10:02:13.462229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403605+0000","last_change":"2024-04-25T10:02:11.453752+0000","last_active":"2024-04-25T10:02:12.403605+0000","last_peered":"2024-04-25T10:02:12.403605+0000","last_clean":"2024-04-25T10:02:12.403605+0000","last_became_active":"2024-04-25T10:02:11.453254+0000","last_became_peered":"2024-04-25T10:02:11.453254+0000","last_unstale":"2024-04-25T10:02:12.403605+0000","last_undegraded":"2024-04-25T10:02:12.403605+0000","last_fullsized":"2024-04-25T10:02:12.403605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:04.566083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462068+0000","last_change":"2024-04-25T10:02:07.356096+0000","last_active":"2024-04-25T10:02:13.462068+0000","last_peered":"2024-04-25T10:02:13.462068+0000","last_clean":"2024-04-25T10:02:13.462068+0000","last_became_active":"2024-04-25T10:02:07.355433+0000","last_became_peered":"2024-04-25T10:02:07.355433+0000","last_unstale":"2024-04-25T10:02:13.462068+0000","last_undegraded":"2024-04-25T10:02:13.462068+0000","last_fullsized":"2024-04-25T10:02:13.462068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466154+0000","last_change":"2024-04-25T10:02:06.385234+0000","last_active":"2024-04-25T10:02:13.466154+0000","last_peered":"2024-04-25T10:02:13.466154+0000","last_clean":"2024-04-25T10:02:13.466154+0000","last_became_active":"2024-04-25T10:02:06.384494+0000","last_became_peered":"2024-04-25T10:02:06.384494+0000","last_unstale":"2024-04-25T10:02:13.466154+0000","last_undegraded":"2024-04-25T10:02:13.466154+0000","last_fullsized":"2024-04-25T10:02:13.466154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462761+0000","last_change":"2024-04-25T10:02:11.459695+0000","last_active":"2024-04-25T10:02:13.462761+0000","last_peered":"2024-04-25T10:02:13.462761+0000","last_clean":"2024-04-25T10:02:13.462761+0000","last_became_active":"2024-04-25T10:02:11.459247+0000","last_became_peered":"2024-04-25T10:02:11.459247+0000","last_unstale":"2024-04-25T10:02:13.462761+0000","last_undegraded":"2024-04-25T10:02:13.462761+0000","last_fullsized":"2024-04-25T10:02:13.462761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465853+0000","last_change":"2024-04-25T10:02:07.374402+0000","last_active":"2024-04-25T10:02:13.465853+0000","last_peered":"2024-04-25T10:02:13.465853+0000","last_clean":"2024-04-25T10:02:13.465853+0000","last_became_active":"2024-04-25T10:02:07.374110+0000","last_became_peered":"2024-04-25T10:02:07.374110+0000","last_unstale":"2024-04-25T10:02:13.465853+0000","last_undegraded":"2024-04-25T10:02:13.465853+0000","last_fullsized":"2024-04-25T10:02:13.465853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462182+0000","last_change":"2024-04-25T10:02:06.368581+0000","last_active":"2024-04-25T10:02:13.462182+0000","last_peered":"2024-04-25T10:02:13.462182+0000","last_clean":"2024-04-25T10:02:13.462182+0000","last_became_active":"2024-04-25T10:02:06.368085+0000","last_became_peered":"2024-04-25T10:02:06.368085+0000","last_unstale":"2024-04-25T10:02:13.462182+0000","last_undegraded":"2024-04-25T10:02:13.462182+0000","last_fullsized":"2024-04-25T10:02:13.462182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461807+0000","last_change":"2024-04-25T10:02:11.435403+0000","last_active":"2024-04-25T10:02:13.461807+0000","last_peered":"2024-04-25T10:02:13.461807+0000","last_clean":"2024-04-25T10:02:13.461807+0000","last_became_active":"2024-04-25T10:02:11.435010+0000","last_became_peered":"2024-04-25T10:02:11.435010+0000","last_unstale":"2024-04-25T10:02:13.461807+0000","last_undegraded":"2024-04-25T10:02:13.461807+0000","last_fullsized":"2024-04-25T10:02:13.461807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407792+0000","last_change":"2024-04-25T10:02:07.378715+0000","last_active":"2024-04-25T10:02:12.407792+0000","last_peered":"2024-04-25T10:02:12.407792+0000","last_clean":"2024-04-25T10:02:12.407792+0000","last_became_active":"2024-04-25T10:02:07.378297+0000","last_became_peered":"2024-04-25T10:02:07.378297+0000","last_unstale":"2024-04-25T10:02:12.407792+0000","last_undegraded":"2024-04-25T10:02:12.407792+0000","last_fullsized":"2024-04-25T10:02:12.407792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465696+0000","last_change":"2024-04-25T10:02:06.381259+0000","last_active":"2024-04-25T10:02:13.465696+0000","last_peered":"2024-04-25T10:02:13.465696+0000","last_clean":"2024-04-25T10:02:13.465696+0000","last_became_active":"2024-04-25T10:02:06.377864+0000","last_became_peered":"2024-04-25T10:02:06.377864+0000","last_unstale":"2024-04-25T10:02:13.465696+0000","last_undegraded":"2024-04-25T10:02:13.465696+0000","last_fullsized":"2024-04-25T10:02:13.465696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462019+0000","last_change":"2024-04-25T10:02:11.435443+0000","last_active":"2024-04-25T10:02:13.462019+0000","last_peered":"2024-04-25T10:02:13.462019+0000","last_clean":"2024-04-25T10:02:13.462019+0000","last_became_active":"2024-04-25T10:02:11.434691+0000","last_became_peered":"2024-04-25T10:02:11.434691+0000","last_unstale":"2024-04-25T10:02:13.462019+0000","last_undegraded":"2024-04-25T10:02:13.462019+0000","last_fullsized":"2024-04-25T10:02:13.462019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462938+0000","last_change":"2024-04-25T10:02:07.360180+0000","last_active":"2024-04-25T10:02:13.462938+0000","last_peered":"2024-04-25T10:02:13.462938+0000","last_clean":"2024-04-25T10:02:13.462938+0000","last_became_active":"2024-04-25T10:02:07.358876+0000","last_became_peered":"2024-04-25T10:02:07.358876+0000","last_unstale":"2024-04-25T10:02:13.462938+0000","last_undegraded":"2024-04-25T10:02:13.462938+0000","last_fullsized":"2024-04-25T10:02:13.462938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462463+0000","last_change":"2024-04-25T10:02:06.376414+0000","last_active":"2024-04-25T10:02:13.462463+0000","last_peered":"2024-04-25T10:02:13.462463+0000","last_clean":"2024-04-25T10:02:13.462463+0000","last_became_active":"2024-04-25T10:02:06.375943+0000","last_became_peered":"2024-04-25T10:02:06.375943+0000","last_unstale":"2024-04-25T10:02:13.462463+0000","last_undegraded":"2024-04-25T10:02:13.462463+0000","last_fullsized":"2024-04-25T10:02:13.462463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463033+0000","last_change":"2024-04-25T10:02:11.438382+0000","last_active":"2024-04-25T10:02:13.463033+0000","last_peered":"2024-04-25T10:02:13.463033+0000","last_clean":"2024-04-25T10:02:13.463033+0000","last_became_active":"2024-04-25T10:02:11.436135+0000","last_became_peered":"2024-04-25T10:02:11.436135+0000","last_unstale":"2024-04-25T10:02:13.463033+0000","last_undegraded":"2024-04-25T10:02:13.463033+0000","last_fullsized":"2024-04-25T10:02:13.463033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.460832+0000","last_change":"2024-04-25T10:02:07.366816+0000","last_active":"2024-04-25T10:02:13.460832+0000","last_peered":"2024-04-25T10:02:13.460832+0000","last_clean":"2024-04-25T10:02:13.460832+0000","last_became_active":"2024-04-25T10:02:07.366193+0000","last_became_peered":"2024-04-25T10:02:07.366193+0000","last_unstale":"2024-04-25T10:02:13.460832+0000","last_undegraded":"2024-04-25T10:02:13.460832+0000","last_fullsized":"2024-04-25T10:02:13.460832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463419+0000","last_change":"2024-04-25T10:02:06.376427+0000","last_active":"2024-04-25T10:02:13.463419+0000","last_peered":"2024-04-25T10:02:13.463419+0000","last_clean":"2024-04-25T10:02:13.463419+0000","last_became_active":"2024-04-25T10:02:06.375955+0000","last_became_peered":"2024-04-25T10:02:06.375955+0000","last_unstale":"2024-04-25T10:02:13.463419+0000","last_undegraded":"2024-04-25T10:02:13.463419+0000","last_fullsized":"2024-04-25T10:02:13.463419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462868+0000","last_change":"2024-04-25T10:02:11.458524+0000","last_active":"2024-04-25T10:02:13.462868+0000","last_peered":"2024-04-25T10:02:13.462868+0000","last_clean":"2024-04-25T10:02:13.462868+0000","last_became_active":"2024-04-25T10:02:11.458105+0000","last_became_peered":"2024-04-25T10:02:11.458105+0000","last_unstale":"2024-04-25T10:02:13.462868+0000","last_undegraded":"2024-04-25T10:02:13.462868+0000","last_fullsized":"2024-04-25T10:02:13.462868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403806+0000","last_change":"2024-04-25T10:02:07.369247+0000","last_active":"2024-04-25T10:02:12.403806+0000","last_peered":"2024-04-25T10:02:12.403806+0000","last_clean":"2024-04-25T10:02:12.403806+0000","last_became_active":"2024-04-25T10:02:07.368616+0000","last_became_peered":"2024-04-25T10:02:07.368616+0000","last_unstale":"2024-04-25T10:02:12.403806+0000","last_undegraded":"2024-04-25T10:02:12.403806+0000","last_fullsized":"2024-04-25T10:02:12.403806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466462+0000","last_change":"2024-04-25T10:02:06.385925+0000","last_active":"2024-04-25T10:02:13.466462+0000","last_peered":"2024-04-25T10:02:13.466462+0000","last_clean":"2024-04-25T10:02:13.466462+0000","last_became_active":"2024-04-25T10:02:06.382216+0000","last_became_peered":"2024-04-25T10:02:06.382216+0000","last_unstale":"2024-04-25T10:02:13.466462+0000","last_undegraded":"2024-04-25T10:02:13.466462+0000","last_fullsized":"2024-04-25T10:02:13.466462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463713+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:02:13.463713+0000","last_peered":"2024-04-25T10:02:13.463713+0000","last_clean":"2024-04-25T10:02:13.463713+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:02:13.463713+0000","last_undegraded":"2024-04-25T10:02:13.463713+0000","last_fullsized":"2024-04-25T10:02:13.463713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406156+0000","last_change":"2024-04-25T10:02:11.460389+0000","last_active":"2024-04-25T10:02:12.406156+0000","last_peered":"2024-04-25T10:02:12.406156+0000","last_clean":"2024-04-25T10:02:12.406156+0000","last_became_active":"2024-04-25T10:02:11.459440+0000","last_became_peered":"2024-04-25T10:02:11.459440+0000","last_unstale":"2024-04-25T10:02:12.406156+0000","last_undegraded":"2024-04-25T10:02:12.406156+0000","last_fullsized":"2024-04-25T10:02:12.406156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465341+0000","last_change":"2024-04-25T10:02:07.388181+0000","last_active":"2024-04-25T10:02:13.465341+0000","last_peered":"2024-04-25T10:02:13.465341+0000","last_clean":"2024-04-25T10:02:13.465341+0000","last_became_active":"2024-04-25T10:02:07.385106+0000","last_became_peered":"2024-04-25T10:02:07.385106+0000","last_unstale":"2024-04-25T10:02:13.465341+0000","last_undegraded":"2024-04-25T10:02:13.465341+0000","last_fullsized":"2024-04-25T10:02:13.465341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463240+0000","last_change":"2024-04-25T10:02:06.386894+0000","last_active":"2024-04-25T10:02:13.463240+0000","last_peered":"2024-04-25T10:02:13.463240+0000","last_clean":"2024-04-25T10:02:13.463240+0000","last_became_active":"2024-04-25T10:02:06.386546+0000","last_became_peered":"2024-04-25T10:02:06.386546+0000","last_unstale":"2024-04-25T10:02:13.463240+0000","last_undegraded":"2024-04-25T10:02:13.463240+0000","last_fullsized":"2024-04-25T10:02:13.463240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405476+0000","last_change":"2024-04-25T10:02:11.455613+0000","last_active":"2024-04-25T10:02:12.405476+0000","last_peered":"2024-04-25T10:02:12.405476+0000","last_clean":"2024-04-25T10:02:12.405476+0000","last_became_active":"2024-04-25T10:02:11.454401+0000","last_became_peered":"2024-04-25T10:02:11.454401+0000","last_unstale":"2024-04-25T10:02:12.405476+0000","last_undegraded":"2024-04-25T10:02:12.405476+0000","last_fullsized":"2024-04-25T10:02:12.405476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465364+0000","last_change":"2024-04-25T10:02:07.382626+0000","last_active":"2024-04-25T10:02:13.465364+0000","last_peered":"2024-04-25T10:02:13.465364+0000","last_clean":"2024-04-25T10:02:13.465364+0000","last_became_active":"2024-04-25T10:02:07.382110+0000","last_became_peered":"2024-04-25T10:02:07.382110+0000","last_unstale":"2024-04-25T10:02:13.465364+0000","last_undegraded":"2024-04-25T10:02:13.465364+0000","last_fullsized":"2024-04-25T10:02:13.465364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461856+0000","last_change":"2024-04-25T10:02:06.361762+0000","last_active":"2024-04-25T10:02:13.461856+0000","last_peered":"2024-04-25T10:02:13.461856+0000","last_clean":"2024-04-25T10:02:13.461856+0000","last_became_active":"2024-04-25T10:02:06.360531+0000","last_became_peered":"2024-04-25T10:02:06.360531+0000","last_unstale":"2024-04-25T10:02:13.461856+0000","last_undegraded":"2024-04-25T10:02:13.461856+0000","last_fullsized":"2024-04-25T10:02:13.461856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463028+0000","last_change":"2024-04-25T10:02:11.435880+0000","last_active":"2024-04-25T10:02:13.463028+0000","last_peered":"2024-04-25T10:02:13.463028+0000","last_clean":"2024-04-25T10:02:13.463028+0000","last_became_active":"2024-04-25T10:02:11.435244+0000","last_became_peered":"2024-04-25T10:02:11.435244+0000","last_unstale":"2024-04-25T10:02:13.463028+0000","last_undegraded":"2024-04-25T10:02:13.463028+0000","last_fullsized":"2024-04-25T10:02:13.463028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466834+0000","last_change":"2024-04-25T10:02:07.385486+0000","last_active":"2024-04-25T10:02:13.466834+0000","last_peered":"2024-04-25T10:02:13.466834+0000","last_clean":"2024-04-25T10:02:13.466834+0000","last_became_active":"2024-04-25T10:02:07.385202+0000","last_became_peered":"2024-04-25T10:02:07.385202+0000","last_unstale":"2024-04-25T10:02:13.466834+0000","last_undegraded":"2024-04-25T10:02:13.466834+0000","last_fullsized":"2024-04-25T10:02:13.466834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:16.103362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462441+0000","last_change":"2024-04-25T10:02:06.379591+0000","last_active":"2024-04-25T10:02:13.462441+0000","last_peered":"2024-04-25T10:02:13.462441+0000","last_clean":"2024-04-25T10:02:13.462441+0000","last_became_active":"2024-04-25T10:02:06.379178+0000","last_became_peered":"2024-04-25T10:02:06.379178+0000","last_unstale":"2024-04-25T10:02:13.462441+0000","last_undegraded":"2024-04-25T10:02:13.462441+0000","last_fullsized":"2024-04-25T10:02:13.462441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463716+0000","last_change":"2024-04-25T10:02:11.459834+0000","last_active":"2024-04-25T10:02:13.463716+0000","last_peered":"2024-04-25T10:02:13.463716+0000","last_clean":"2024-04-25T10:02:13.463716+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.463716+0000","last_undegraded":"2024-04-25T10:02:13.463716+0000","last_fullsized":"2024-04-25T10:02:13.463716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466278+0000","last_change":"2024-04-25T10:02:07.387540+0000","last_active":"2024-04-25T10:02:13.466278+0000","last_peered":"2024-04-25T10:02:13.466278+0000","last_clean":"2024-04-25T10:02:13.466278+0000","last_became_active":"2024-04-25T10:02:07.385395+0000","last_became_peered":"2024-04-25T10:02:07.385395+0000","last_unstale":"2024-04-25T10:02:13.466278+0000","last_undegraded":"2024-04-25T10:02:13.466278+0000","last_fullsized":"2024-04-25T10:02:13.466278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408935+0000","last_change":"2024-04-25T10:02:06.379749+0000","last_active":"2024-04-25T10:02:12.408935+0000","last_peered":"2024-04-25T10:02:12.408935+0000","last_clean":"2024-04-25T10:02:12.408935+0000","last_became_active":"2024-04-25T10:02:06.378947+0000","last_became_peered":"2024-04-25T10:02:06.378947+0000","last_unstale":"2024-04-25T10:02:12.408935+0000","last_undegraded":"2024-04-25T10:02:12.408935+0000","last_fullsized":"2024-04-25T10:02:12.408935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:03.055592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405719+0000","last_change":"2024-04-25T10:02:11.437602+0000","last_active":"2024-04-25T10:02:12.405719+0000","last_peered":"2024-04-25T10:02:12.405719+0000","last_clean":"2024-04-25T10:02:12.405719+0000","last_became_active":"2024-04-25T10:02:11.436553+0000","last_became_peered":"2024-04-25T10:02:11.436553+0000","last_unstale":"2024-04-25T10:02:12.405719+0000","last_undegraded":"2024-04-25T10:02:12.405719+0000","last_fullsized":"2024-04-25T10:02:12.405719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:53.618014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464059+0000","last_change":"2024-04-25T10:02:07.384411+0000","last_active":"2024-04-25T10:02:13.464059+0000","last_peered":"2024-04-25T10:02:13.464059+0000","last_clean":"2024-04-25T10:02:13.464059+0000","last_became_active":"2024-04-25T10:02:07.377814+0000","last_became_peered":"2024-04-25T10:02:07.377814+0000","last_unstale":"2024-04-25T10:02:13.464059+0000","last_undegraded":"2024-04-25T10:02:13.464059+0000","last_fullsized":"2024-04-25T10:02:13.464059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461988+0000","last_change":"2024-04-25T10:02:06.360702+0000","last_active":"2024-04-25T10:02:13.461988+0000","last_peered":"2024-04-25T10:02:13.461988+0000","last_clean":"2024-04-25T10:02:13.461988+0000","last_became_active":"2024-04-25T10:02:06.360297+0000","last_became_peered":"2024-04-25T10:02:06.360297+0000","last_unstale":"2024-04-25T10:02:13.461988+0000","last_undegraded":"2024-04-25T10:02:13.461988+0000","last_fullsized":"2024-04-25T10:02:13.461988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462271+0000","last_change":"2024-04-25T10:02:11.427201+0000","last_active":"2024-04-25T10:02:13.462271+0000","last_peered":"2024-04-25T10:02:13.462271+0000","last_clean":"2024-04-25T10:02:13.462271+0000","last_became_active":"2024-04-25T10:02:11.426734+0000","last_became_peered":"2024-04-25T10:02:11.426734+0000","last_unstale":"2024-04-25T10:02:13.462271+0000","last_undegraded":"2024-04-25T10:02:13.462271+0000","last_fullsized":"2024-04-25T10:02:13.462271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404061+0000","last_change":"2024-04-25T10:02:07.371003+0000","last_active":"2024-04-25T10:02:12.404061+0000","last_peered":"2024-04-25T10:02:12.404061+0000","last_clean":"2024-04-25T10:02:12.404061+0000","last_became_active":"2024-04-25T10:02:07.370695+0000","last_became_peered":"2024-04-25T10:02:07.370695+0000","last_unstale":"2024-04-25T10:02:12.404061+0000","last_undegraded":"2024-04-25T10:02:12.404061+0000","last_fullsized":"2024-04-25T10:02:12.404061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405106+0000","last_change":"2024-04-25T10:02:06.362100+0000","last_active":"2024-04-25T10:02:12.405106+0000","last_peered":"2024-04-25T10:02:12.405106+0000","last_clean":"2024-04-25T10:02:12.405106+0000","last_became_active":"2024-04-25T10:02:06.361358+0000","last_became_peered":"2024-04-25T10:02:06.361358+0000","last_unstale":"2024-04-25T10:02:12.405106+0000","last_undegraded":"2024-04-25T10:02:12.405106+0000","last_fullsized":"2024-04-25T10:02:12.405106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406644+0000","last_change":"2024-04-25T10:02:11.460013+0000","last_active":"2024-04-25T10:02:12.406644+0000","last_peered":"2024-04-25T10:02:12.406644+0000","last_clean":"2024-04-25T10:02:12.406644+0000","last_became_active":"2024-04-25T10:02:11.459491+0000","last_became_peered":"2024-04-25T10:02:11.459491+0000","last_unstale":"2024-04-25T10:02:12.406644+0000","last_undegraded":"2024-04-25T10:02:12.406644+0000","last_fullsized":"2024-04-25T10:02:12.406644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.241421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465085+0000","last_change":"2024-04-25T10:02:07.373359+0000","last_active":"2024-04-25T10:02:13.465085+0000","last_peered":"2024-04-25T10:02:13.465085+0000","last_clean":"2024-04-25T10:02:13.465085+0000","last_became_active":"2024-04-25T10:02:07.373100+0000","last_became_peered":"2024-04-25T10:02:07.373100+0000","last_unstale":"2024-04-25T10:02:13.465085+0000","last_undegraded":"2024-04-25T10:02:13.465085+0000","last_fullsized":"2024-04-25T10:02:13.465085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.957032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465425+0000","last_change":"2024-04-25T10:02:06.382032+0000","last_active":"2024-04-25T10:02:13.465425+0000","last_peered":"2024-04-25T10:02:13.465425+0000","last_clean":"2024-04-25T10:02:13.465425+0000","last_became_active":"2024-04-25T10:02:06.381711+0000","last_became_peered":"2024-04-25T10:02:06.381711+0000","last_unstale":"2024-04-25T10:02:13.465425+0000","last_undegraded":"2024-04-25T10:02:13.465425+0000","last_fullsized":"2024-04-25T10:02:13.465425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404352+0000","last_change":"2024-04-25T10:02:11.458829+0000","last_active":"2024-04-25T10:02:12.404352+0000","last_peered":"2024-04-25T10:02:12.404352+0000","last_clean":"2024-04-25T10:02:12.404352+0000","last_became_active":"2024-04-25T10:02:11.458386+0000","last_became_peered":"2024-04-25T10:02:11.458386+0000","last_unstale":"2024-04-25T10:02:12.404352+0000","last_undegraded":"2024-04-25T10:02:12.404352+0000","last_fullsized":"2024-04-25T10:02:12.404352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:15.902040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462813+0000","last_change":"2024-04-25T10:02:07.372352+0000","last_active":"2024-04-25T10:02:13.462813+0000","last_peered":"2024-04-25T10:02:13.462813+0000","last_clean":"2024-04-25T10:02:13.462813+0000","last_became_active":"2024-04-25T10:02:07.372124+0000","last_became_peered":"2024-04-25T10:02:07.372124+0000","last_unstale":"2024-04-25T10:02:13.462813+0000","last_undegraded":"2024-04-25T10:02:13.462813+0000","last_fullsized":"2024-04-25T10:02:13.462813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463253+0000","last_change":"2024-04-25T10:02:06.361119+0000","last_active":"2024-04-25T10:02:13.463253+0000","last_peered":"2024-04-25T10:02:13.463253+0000","last_clean":"2024-04-25T10:02:13.463253+0000","last_became_active":"2024-04-25T10:02:06.360459+0000","last_became_peered":"2024-04-25T10:02:06.360459+0000","last_unstale":"2024-04-25T10:02:13.463253+0000","last_undegraded":"2024-04-25T10:02:13.463253+0000","last_fullsized":"2024-04-25T10:02:13.463253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465848+0000","last_change":"2024-04-25T10:02:11.460123+0000","last_active":"2024-04-25T10:02:13.465848+0000","last_peered":"2024-04-25T10:02:13.465848+0000","last_clean":"2024-04-25T10:02:13.465848+0000","last_became_active":"2024-04-25T10:02:11.459152+0000","last_became_peered":"2024-04-25T10:02:11.459152+0000","last_unstale":"2024-04-25T10:02:13.465848+0000","last_undegraded":"2024-04-25T10:02:13.465848+0000","last_fullsized":"2024-04-25T10:02:13.465848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465062+0000","last_change":"2024-04-25T10:02:07.388512+0000","last_active":"2024-04-25T10:02:13.465062+0000","last_peered":"2024-04-25T10:02:13.465062+0000","last_clean":"2024-04-25T10:02:13.465062+0000","last_became_active":"2024-04-25T10:02:07.387797+0000","last_became_peered":"2024-04-25T10:02:07.387797+0000","last_unstale":"2024-04-25T10:02:13.465062+0000","last_undegraded":"2024-04-25T10:02:13.465062+0000","last_fullsized":"2024-04-25T10:02:13.465062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:17.622024+0000","last_change":"2024-04-25T10:02:06.370256+0000","last_active":"2024-04-25T10:02:17.622024+0000","last_peered":"2024-04-25T10:02:17.622024+0000","last_clean":"2024-04-25T10:02:17.622024+0000","last_became_active":"2024-04-25T10:02:06.368370+0000","last_became_peered":"2024-04-25T10:02:06.368370+0000","last_unstale":"2024-04-25T10:02:17.622024+0000","last_undegraded":"2024-04-25T10:02:17.622024+0000","last_fullsized":"2024-04-25T10:02:17.622024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462482+0000","last_change":"2024-04-25T10:02:11.438518+0000","last_active":"2024-04-25T10:02:13.462482+0000","last_peered":"2024-04-25T10:02:13.462482+0000","last_clean":"2024-04-25T10:02:13.462482+0000","last_became_active":"2024-04-25T10:02:11.436290+0000","last_became_peered":"2024-04-25T10:02:11.436290+0000","last_unstale":"2024-04-25T10:02:13.462482+0000","last_undegraded":"2024-04-25T10:02:13.462482+0000","last_fullsized":"2024-04-25T10:02:13.462482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407354+0000","last_change":"2024-04-25T10:02:07.358234+0000","last_active":"2024-04-25T10:02:12.407354+0000","last_peered":"2024-04-25T10:02:12.407354+0000","last_clean":"2024-04-25T10:02:12.407354+0000","last_became_active":"2024-04-25T10:02:07.357961+0000","last_became_peered":"2024-04-25T10:02:07.357961+0000","last_unstale":"2024-04-25T10:02:12.407354+0000","last_undegraded":"2024-04-25T10:02:12.407354+0000","last_fullsized":"2024-04-25T10:02:12.407354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461751+0000","last_change":"2024-04-25T10:02:06.386529+0000","last_active":"2024-04-25T10:02:13.461751+0000","last_peered":"2024-04-25T10:02:13.461751+0000","last_clean":"2024-04-25T10:02:13.461751+0000","last_became_active":"2024-04-25T10:02:06.386097+0000","last_became_peered":"2024-04-25T10:02:06.386097+0000","last_unstale":"2024-04-25T10:02:13.461751+0000","last_undegraded":"2024-04-25T10:02:13.461751+0000","last_fullsized":"2024-04-25T10:02:13.461751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407672+0000","last_change":"2024-04-25T10:02:11.437594+0000","last_active":"2024-04-25T10:02:12.407672+0000","last_peered":"2024-04-25T10:02:12.407672+0000","last_clean":"2024-04-25T10:02:12.407672+0000","last_became_active":"2024-04-25T10:02:11.436114+0000","last_became_peered":"2024-04-25T10:02:11.436114+0000","last_unstale":"2024-04-25T10:02:12.407672+0000","last_undegraded":"2024-04-25T10:02:12.407672+0000","last_fullsized":"2024-04-25T10:02:12.407672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462591+0000","last_change":"2024-04-25T10:02:07.371078+0000","last_active":"2024-04-25T10:02:13.462591+0000","last_peered":"2024-04-25T10:02:13.462591+0000","last_clean":"2024-04-25T10:02:13.462591+0000","last_became_active":"2024-04-25T10:02:07.370749+0000","last_became_peered":"2024-04-25T10:02:07.370749+0000","last_unstale":"2024-04-25T10:02:13.462591+0000","last_undegraded":"2024-04-25T10:02:13.462591+0000","last_fullsized":"2024-04-25T10:02:13.462591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466420+0000","last_change":"2024-04-25T10:02:06.377512+0000","last_active":"2024-04-25T10:02:13.466420+0000","last_peered":"2024-04-25T10:02:13.466420+0000","last_clean":"2024-04-25T10:02:13.466420+0000","last_became_active":"2024-04-25T10:02:06.376824+0000","last_became_peered":"2024-04-25T10:02:06.376824+0000","last_unstale":"2024-04-25T10:02:13.466420+0000","last_undegraded":"2024-04-25T10:02:13.466420+0000","last_fullsized":"2024-04-25T10:02:13.466420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403770+0000","last_change":"2024-04-25T10:02:11.442056+0000","last_active":"2024-04-25T10:02:12.403770+0000","last_peered":"2024-04-25T10:02:12.403770+0000","last_clean":"2024-04-25T10:02:12.403770+0000","last_became_active":"2024-04-25T10:02:11.441158+0000","last_became_peered":"2024-04-25T10:02:11.441158+0000","last_unstale":"2024-04-25T10:02:12.403770+0000","last_undegraded":"2024-04-25T10:02:12.403770+0000","last_fullsized":"2024-04-25T10:02:12.403770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466185+0000","last_change":"2024-04-25T10:02:07.389094+0000","last_active":"2024-04-25T10:02:13.466185+0000","last_peered":"2024-04-25T10:02:13.466185+0000","last_clean":"2024-04-25T10:02:13.466185+0000","last_became_active":"2024-04-25T10:02:07.388187+0000","last_became_peered":"2024-04-25T10:02:07.388187+0000","last_unstale":"2024-04-25T10:02:13.466185+0000","last_undegraded":"2024-04-25T10:02:13.466185+0000","last_fullsized":"2024-04-25T10:02:13.466185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406550+0000","last_change":"2024-04-25T10:02:06.383181+0000","last_active":"2024-04-25T10:02:12.406550+0000","last_peered":"2024-04-25T10:02:12.406550+0000","last_clean":"2024-04-25T10:02:12.406550+0000","last_became_active":"2024-04-25T10:02:06.382921+0000","last_became_peered":"2024-04-25T10:02:06.382921+0000","last_unstale":"2024-04-25T10:02:12.406550+0000","last_undegraded":"2024-04-25T10:02:12.406550+0000","last_fullsized":"2024-04-25T10:02:12.406550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404862+0000","last_change":"2024-04-25T10:02:11.454031+0000","last_active":"2024-04-25T10:02:12.404862+0000","last_peered":"2024-04-25T10:02:12.404862+0000","last_clean":"2024-04-25T10:02:12.404862+0000","last_became_active":"2024-04-25T10:02:11.453256+0000","last_became_peered":"2024-04-25T10:02:11.453256+0000","last_unstale":"2024-04-25T10:02:12.404862+0000","last_undegraded":"2024-04-25T10:02:12.404862+0000","last_fullsized":"2024-04-25T10:02:12.404862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:26.875452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462061+0000","last_change":"2024-04-25T10:02:11.460406+0000","last_active":"2024-04-25T10:02:13.462061+0000","last_peered":"2024-04-25T10:02:13.462061+0000","last_clean":"2024-04-25T10:02:13.462061+0000","last_became_active":"2024-04-25T10:02:11.459947+0000","last_became_peered":"2024-04-25T10:02:11.459947+0000","last_unstale":"2024-04-25T10:02:13.462061+0000","last_undegraded":"2024-04-25T10:02:13.462061+0000","last_fullsized":"2024-04-25T10:02:13.462061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461463+0000","last_change":"2024-04-25T10:02:07.361531+0000","last_active":"2024-04-25T10:02:13.461463+0000","last_peered":"2024-04-25T10:02:13.461463+0000","last_clean":"2024-04-25T10:02:13.461463+0000","last_became_active":"2024-04-25T10:02:07.361213+0000","last_became_peered":"2024-04-25T10:02:07.361213+0000","last_unstale":"2024-04-25T10:02:13.461463+0000","last_undegraded":"2024-04-25T10:02:13.461463+0000","last_fullsized":"2024-04-25T10:02:13.461463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464490+0000","last_change":"2024-04-25T10:02:06.386304+0000","last_active":"2024-04-25T10:02:13.464490+0000","last_peered":"2024-04-25T10:02:13.464490+0000","last_clean":"2024-04-25T10:02:13.464490+0000","last_became_active":"2024-04-25T10:02:06.382588+0000","last_became_peered":"2024-04-25T10:02:06.382588+0000","last_unstale":"2024-04-25T10:02:13.464490+0000","last_undegraded":"2024-04-25T10:02:13.464490+0000","last_fullsized":"2024-04-25T10:02:13.464490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406241+0000","last_change":"2024-04-25T10:02:11.433821+0000","last_active":"2024-04-25T10:02:12.406241+0000","last_peered":"2024-04-25T10:02:12.406241+0000","last_clean":"2024-04-25T10:02:12.406241+0000","last_became_active":"2024-04-25T10:02:11.432467+0000","last_became_peered":"2024-04-25T10:02:11.432467+0000","last_unstale":"2024-04-25T10:02:12.406241+0000","last_undegraded":"2024-04-25T10:02:12.406241+0000","last_fullsized":"2024-04-25T10:02:12.406241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403810+0000","last_change":"2024-04-25T10:02:07.376574+0000","last_active":"2024-04-25T10:02:12.403810+0000","last_peered":"2024-04-25T10:02:12.403810+0000","last_clean":"2024-04-25T10:02:12.403810+0000","last_became_active":"2024-04-25T10:02:07.376275+0000","last_became_peered":"2024-04-25T10:02:07.376275+0000","last_unstale":"2024-04-25T10:02:12.403810+0000","last_undegraded":"2024-04-25T10:02:12.403810+0000","last_fullsized":"2024-04-25T10:02:12.403810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405129+0000","last_change":"2024-04-25T10:02:06.363949+0000","last_active":"2024-04-25T10:02:12.405129+0000","last_peered":"2024-04-25T10:02:12.405129+0000","last_clean":"2024-04-25T10:02:12.405129+0000","last_became_active":"2024-04-25T10:02:06.363379+0000","last_became_peered":"2024-04-25T10:02:06.363379+0000","last_unstale":"2024-04-25T10:02:12.405129+0000","last_undegraded":"2024-04-25T10:02:12.405129+0000","last_fullsized":"2024-04-25T10:02:12.405129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405641+0000","last_change":"2024-04-25T10:02:06.361583+0000","last_active":"2024-04-25T10:02:12.405641+0000","last_peered":"2024-04-25T10:02:12.405641+0000","last_clean":"2024-04-25T10:02:12.405641+0000","last_became_active":"2024-04-25T10:02:06.360807+0000","last_became_peered":"2024-04-25T10:02:06.360807+0000","last_unstale":"2024-04-25T10:02:12.405641+0000","last_undegraded":"2024-04-25T10:02:12.405641+0000","last_fullsized":"2024-04-25T10:02:12.405641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463206+0000","last_change":"2024-04-25T10:02:07.367030+0000","last_active":"2024-04-25T10:02:13.463206+0000","last_peered":"2024-04-25T10:02:13.463206+0000","last_clean":"2024-04-25T10:02:13.463206+0000","last_became_active":"2024-04-25T10:02:07.366570+0000","last_became_peered":"2024-04-25T10:02:07.366570+0000","last_unstale":"2024-04-25T10:02:13.463206+0000","last_undegraded":"2024-04-25T10:02:13.463206+0000","last_fullsized":"2024-04-25T10:02:13.463206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463018+0000","last_change":"2024-04-25T10:02:11.434159+0000","last_active":"2024-04-25T10:02:13.463018+0000","last_peered":"2024-04-25T10:02:13.463018+0000","last_clean":"2024-04-25T10:02:13.463018+0000","last_became_active":"2024-04-25T10:02:11.433378+0000","last_became_peered":"2024-04-25T10:02:11.433378+0000","last_unstale":"2024-04-25T10:02:13.463018+0000","last_undegraded":"2024-04-25T10:02:13.463018+0000","last_fullsized":"2024-04-25T10:02:13.463018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:25.406090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403964+0000","last_change":"2024-04-25T10:02:06.375949+0000","last_active":"2024-04-25T10:02:12.403964+0000","last_peered":"2024-04-25T10:02:12.403964+0000","last_clean":"2024-04-25T10:02:12.403964+0000","last_became_active":"2024-04-25T10:02:06.375558+0000","last_became_peered":"2024-04-25T10:02:06.375558+0000","last_unstale":"2024-04-25T10:02:12.403964+0000","last_undegraded":"2024-04-25T10:02:12.403964+0000","last_fullsized":"2024-04-25T10:02:12.403964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406442+0000","last_change":"2024-04-25T10:02:07.352019+0000","last_active":"2024-04-25T10:02:12.406442+0000","last_peered":"2024-04-25T10:02:12.406442+0000","last_clean":"2024-04-25T10:02:12.406442+0000","last_became_active":"2024-04-25T10:02:07.351303+0000","last_became_peered":"2024-04-25T10:02:07.351303+0000","last_unstale":"2024-04-25T10:02:12.406442+0000","last_undegraded":"2024-04-25T10:02:12.406442+0000","last_fullsized":"2024-04-25T10:02:12.406442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:08.301904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406592+0000","last_change":"2024-04-25T10:02:11.441304+0000","last_active":"2024-04-25T10:02:12.406592+0000","last_peered":"2024-04-25T10:02:12.406592+0000","last_clean":"2024-04-25T10:02:12.406592+0000","last_became_active":"2024-04-25T10:02:11.440847+0000","last_became_peered":"2024-04-25T10:02:11.440847+0000","last_unstale":"2024-04-25T10:02:12.406592+0000","last_undegraded":"2024-04-25T10:02:12.406592+0000","last_fullsized":"2024-04-25T10:02:12.406592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:07.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403875+0000","last_change":"2024-04-25T10:02:07.372731+0000","last_active":"2024-04-25T10:02:12.403875+0000","last_peered":"2024-04-25T10:02:12.403875+0000","last_clean":"2024-04-25T10:02:12.403875+0000","last_became_active":"2024-04-25T10:02:07.372438+0000","last_became_peered":"2024-04-25T10:02:07.372438+0000","last_unstale":"2024-04-25T10:02:12.403875+0000","last_undegraded":"2024-04-25T10:02:12.403875+0000","last_fullsized":"2024-04-25T10:02:12.403875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462340+0000","last_change":"2024-04-25T10:02:07.364477+0000","last_active":"2024-04-25T10:02:13.462340+0000","last_peered":"2024-04-25T10:02:13.462340+0000","last_clean":"2024-04-25T10:02:13.462340+0000","last_became_active":"2024-04-25T10:02:07.363263+0000","last_became_peered":"2024-04-25T10:02:07.363263+0000","last_unstale":"2024-04-25T10:02:13.462340+0000","last_undegraded":"2024-04-25T10:02:13.462340+0000","last_fullsized":"2024-04-25T10:02:13.462340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:42.118057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465162+0000","last_change":"2024-04-25T10:02:07.366856+0000","last_active":"2024-04-25T10:02:13.465162+0000","last_peered":"2024-04-25T10:02:13.465162+0000","last_clean":"2024-04-25T10:02:13.465162+0000","last_became_active":"2024-04-25T10:02:07.366608+0000","last_became_peered":"2024-04-25T10:02:07.366608+0000","last_unstale":"2024-04-25T10:02:13.465162+0000","last_undegraded":"2024-04-25T10:02:13.465162+0000","last_fullsized":"2024-04-25T10:02:13.465162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463076+0000","last_change":"2024-04-25T10:02:07.374664+0000","last_active":"2024-04-25T10:02:13.463076+0000","last_peered":"2024-04-25T10:02:13.463076+0000","last_clean":"2024-04-25T10:02:13.463076+0000","last_became_active":"2024-04-25T10:02:07.374088+0000","last_became_peered":"2024-04-25T10:02:07.374088+0000","last_unstale":"2024-04-25T10:02:13.463076+0000","last_undegraded":"2024-04-25T10:02:13.463076+0000","last_fullsized":"2024-04-25T10:02:13.463076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404526+0000","last_change":"2024-04-25T10:02:07.365480+0000","last_active":"2024-04-25T10:02:12.404526+0000","last_peered":"2024-04-25T10:02:12.404526+0000","last_clean":"2024-04-25T10:02:12.404526+0000","last_became_active":"2024-04-25T10:02:07.365034+0000","last_became_peered":"2024-04-25T10:02:07.365034+0000","last_unstale":"2024-04-25T10:02:12.404526+0000","last_undegraded":"2024-04-25T10:02:12.404526+0000","last_fullsized":"2024-04-25T10:02:12.404526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461634+0000","last_change":"2024-04-25T10:02:07.362200+0000","last_active":"2024-04-25T10:02:13.461634+0000","last_peered":"2024-04-25T10:02:13.461634+0000","last_clean":"2024-04-25T10:02:13.461634+0000","last_became_active":"2024-04-25T10:02:07.361929+0000","last_became_peered":"2024-04-25T10:02:07.361929+0000","last_unstale":"2024-04-25T10:02:13.461634+0000","last_undegraded":"2024-04-25T10:02:13.461634+0000","last_fullsized":"2024-04-25T10:02:13.461634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407180+0000","last_change":"2024-04-25T10:02:07.375688+0000","last_active":"2024-04-25T10:02:12.407180+0000","last_peered":"2024-04-25T10:02:12.407180+0000","last_clean":"2024-04-25T10:02:12.407180+0000","last_became_active":"2024-04-25T10:02:07.375046+0000","last_became_peered":"2024-04-25T10:02:07.375046+0000","last_unstale":"2024-04-25T10:02:12.407180+0000","last_undegraded":"2024-04-25T10:02:12.407180+0000","last_fullsized":"2024-04-25T10:02:12.407180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406734+0000","last_change":"2024-04-25T10:02:07.365322+0000","last_active":"2024-04-25T10:02:12.406734+0000","last_peered":"2024-04-25T10:02:12.406734+0000","last_clean":"2024-04-25T10:02:12.406734+0000","last_became_active":"2024-04-25T10:02:07.365003+0000","last_became_peered":"2024-04-25T10:02:07.365003+0000","last_unstale":"2024-04-25T10:02:12.406734+0000","last_undegraded":"2024-04-25T10:02:12.406734+0000","last_fullsized":"2024-04-25T10:02:12.406734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464320+0000","last_change":"2024-04-25T10:02:07.374731+0000","last_active":"2024-04-25T10:02:13.464320+0000","last_peered":"2024-04-25T10:02:13.464320+0000","last_clean":"2024-04-25T10:02:13.464320+0000","last_became_active":"2024-04-25T10:02:07.374113+0000","last_became_peered":"2024-04-25T10:02:07.374113+0000","last_unstale":"2024-04-25T10:02:13.464320+0000","last_undegraded":"2024-04-25T10:02:13.464320+0000","last_fullsized":"2024-04-25T10:02:13.464320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404344+0000","last_change":"2024-04-25T10:02:07.365139+0000","last_active":"2024-04-25T10:02:12.404344+0000","last_peered":"2024-04-25T10:02:12.404344+0000","last_clean":"2024-04-25T10:02:12.404344+0000","last_became_active":"2024-04-25T10:02:07.364860+0000","last_became_peered":"2024-04-25T10:02:07.364860+0000","last_unstale":"2024-04-25T10:02:12.404344+0000","last_undegraded":"2024-04-25T10:02:12.404344+0000","last_fullsized":"2024-04-25T10:02:12.404344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464902+0000","last_change":"2024-04-25T10:02:07.379737+0000","last_active":"2024-04-25T10:02:13.464902+0000","last_peered":"2024-04-25T10:02:13.464902+0000","last_clean":"2024-04-25T10:02:13.464902+0000","last_became_active":"2024-04-25T10:02:07.379465+0000","last_became_peered":"2024-04-25T10:02:07.379465+0000","last_unstale":"2024-04-25T10:02:13.464902+0000","last_undegraded":"2024-04-25T10:02:13.464902+0000","last_fullsized":"2024-04-25T10:02:13.464902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462820+0000","last_change":"2024-04-25T10:02:07.379170+0000","last_active":"2024-04-25T10:02:13.462820+0000","last_peered":"2024-04-25T10:02:13.462820+0000","last_clean":"2024-04-25T10:02:13.462820+0000","last_became_active":"2024-04-25T10:02:07.378877+0000","last_became_peered":"2024-04-25T10:02:07.378877+0000","last_unstale":"2024-04-25T10:02:13.462820+0000","last_undegraded":"2024-04-25T10:02:13.462820+0000","last_fullsized":"2024-04-25T10:02:13.462820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406740+0000","last_change":"2024-04-25T10:02:07.367522+0000","last_active":"2024-04-25T10:02:12.406740+0000","last_peered":"2024-04-25T10:02:12.406740+0000","last_clean":"2024-04-25T10:02:12.406740+0000","last_became_active":"2024-04-25T10:02:07.367226+0000","last_became_peered":"2024-04-25T10:02:07.367226+0000","last_unstale":"2024-04-25T10:02:12.406740+0000","last_undegraded":"2024-04-25T10:02:12.406740+0000","last_fullsized":"2024-04-25T10:02:12.406740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465868+0000","last_change":"2024-04-25T10:02:07.378749+0000","last_active":"2024-04-25T10:02:13.465868+0000","last_peered":"2024-04-25T10:02:13.465868+0000","last_clean":"2024-04-25T10:02:13.465868+0000","last_became_active":"2024-04-25T10:02:07.378388+0000","last_became_peered":"2024-04-25T10:02:07.378388+0000","last_unstale":"2024-04-25T10:02:13.465868+0000","last_undegraded":"2024-04-25T10:02:13.465868+0000","last_fullsized":"2024-04-25T10:02:13.465868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:00.508687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462346+0000","last_change":"2024-04-25T10:02:07.377744+0000","last_active":"2024-04-25T10:02:13.462346+0000","last_peered":"2024-04-25T10:02:13.462346+0000","last_clean":"2024-04-25T10:02:13.462346+0000","last_became_active":"2024-04-25T10:02:07.376844+0000","last_became_peered":"2024-04-25T10:02:07.376844+0000","last_unstale":"2024-04-25T10:02:13.462346+0000","last_undegraded":"2024-04-25T10:02:13.462346+0000","last_fullsized":"2024-04-25T10:02:13.462346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465699+0000","last_change":"2024-04-25T10:02:07.388777+0000","last_active":"2024-04-25T10:02:13.465699+0000","last_peered":"2024-04-25T10:02:13.465699+0000","last_clean":"2024-04-25T10:02:13.465699+0000","last_became_active":"2024-04-25T10:02:07.388058+0000","last_became_peered":"2024-04-25T10:02:07.388058+0000","last_unstale":"2024-04-25T10:02:13.465699+0000","last_undegraded":"2024-04-25T10:02:13.465699+0000","last_fullsized":"2024-04-25T10:02:13.465699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464055+0000","last_change":"2024-04-25T10:02:07.377996+0000","last_active":"2024-04-25T10:02:13.464055+0000","last_peered":"2024-04-25T10:02:13.464055+0000","last_clean":"2024-04-25T10:02:13.464055+0000","last_became_active":"2024-04-25T10:02:07.376664+0000","last_became_peered":"2024-04-25T10:02:07.376664+0000","last_unstale":"2024-04-25T10:02:13.464055+0000","last_undegraded":"2024-04-25T10:02:13.464055+0000","last_fullsized":"2024-04-25T10:02:13.464055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461571+0000","last_change":"2024-04-25T10:02:07.364951+0000","last_active":"2024-04-25T10:02:13.461571+0000","last_peered":"2024-04-25T10:02:13.461571+0000","last_clean":"2024-04-25T10:02:13.461571+0000","last_became_active":"2024-04-25T10:02:07.364138+0000","last_became_peered":"2024-04-25T10:02:07.364138+0000","last_unstale":"2024-04-25T10:02:13.461571+0000","last_undegraded":"2024-04-25T10:02:13.461571+0000","last_fullsized":"2024-04-25T10:02:13.461571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462765+0000","last_change":"2024-04-25T10:02:07.370504+0000","last_active":"2024-04-25T10:02:13.462765+0000","last_peered":"2024-04-25T10:02:13.462765+0000","last_clean":"2024-04-25T10:02:13.462765+0000","last_became_active":"2024-04-25T10:02:07.370157+0000","last_became_peered":"2024-04-25T10:02:07.370157+0000","last_unstale":"2024-04-25T10:02:13.462765+0000","last_undegraded":"2024-04-25T10:02:13.462765+0000","last_fullsized":"2024-04-25T10:02:13.462765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403675+0000","last_change":"2024-04-25T10:02:07.360200+0000","last_active":"2024-04-25T10:02:12.403675+0000","last_peered":"2024-04-25T10:02:12.403675+0000","last_clean":"2024-04-25T10:02:12.403675+0000","last_became_active":"2024-04-25T10:02:07.358078+0000","last_became_peered":"2024-04-25T10:02:07.358078+0000","last_unstale":"2024-04-25T10:02:12.403675+0000","last_undegraded":"2024-04-25T10:02:12.403675+0000","last_fullsized":"2024-04-25T10:02:12.403675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407201+0000","last_change":"2024-04-25T10:02:07.376059+0000","last_active":"2024-04-25T10:02:12.407201+0000","last_peered":"2024-04-25T10:02:12.407201+0000","last_clean":"2024-04-25T10:02:12.407201+0000","last_became_active":"2024-04-25T10:02:07.374735+0000","last_became_peered":"2024-04-25T10:02:07.374735+0000","last_unstale":"2024-04-25T10:02:12.407201+0000","last_undegraded":"2024-04-25T10:02:12.407201+0000","last_fullsized":"2024-04-25T10:02:12.407201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:54.666473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406312+0000","last_change":"2024-04-25T10:02:07.367747+0000","last_active":"2024-04-25T10:02:12.406312+0000","last_peered":"2024-04-25T10:02:12.406312+0000","last_clean":"2024-04-25T10:02:12.406312+0000","last_became_active":"2024-04-25T10:02:07.367055+0000","last_became_peered":"2024-04-25T10:02:07.367055+0000","last_unstale":"2024-04-25T10:02:12.406312+0000","last_undegraded":"2024-04-25T10:02:12.406312+0000","last_fullsized":"2024-04-25T10:02:12.406312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406946+0000","last_change":"2024-04-25T10:02:07.366278+0000","last_active":"2024-04-25T10:02:12.406946+0000","last_peered":"2024-04-25T10:02:12.406946+0000","last_clean":"2024-04-25T10:02:12.406946+0000","last_became_active":"2024-04-25T10:02:07.365591+0000","last_became_peered":"2024-04-25T10:02:07.365591+0000","last_unstale":"2024-04-25T10:02:12.406946+0000","last_undegraded":"2024-04-25T10:02:12.406946+0000","last_fullsized":"2024-04-25T10:02:12.406946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466682+0000","last_change":"2024-04-25T10:02:07.389531+0000","last_active":"2024-04-25T10:02:13.466682+0000","last_peered":"2024-04-25T10:02:13.466682+0000","last_clean":"2024-04-25T10:02:13.466682+0000","last_became_active":"2024-04-25T10:02:07.389091+0000","last_became_peered":"2024-04-25T10:02:07.389091+0000","last_unstale":"2024-04-25T10:02:13.466682+0000","last_undegraded":"2024-04-25T10:02:13.466682+0000","last_fullsized":"2024-04-25T10:02:13.466682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403425+0000","last_change":"2024-04-25T10:02:07.368434+0000","last_active":"2024-04-25T10:02:12.403425+0000","last_peered":"2024-04-25T10:02:12.403425+0000","last_clean":"2024-04-25T10:02:12.403425+0000","last_became_active":"2024-04-25T10:02:07.366907+0000","last_became_peered":"2024-04-25T10:02:07.366907+0000","last_unstale":"2024-04-25T10:02:12.403425+0000","last_undegraded":"2024-04-25T10:02:12.403425+0000","last_fullsized":"2024-04-25T10:02:12.403425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463789+0000","last_change":"2024-04-25T10:02:07.377948+0000","last_active":"2024-04-25T10:02:13.463789+0000","last_peered":"2024-04-25T10:02:13.463789+0000","last_clean":"2024-04-25T10:02:13.463789+0000","last_became_active":"2024-04-25T10:02:07.377170+0000","last_became_peered":"2024-04-25T10:02:07.377170+0000","last_unstale":"2024-04-25T10:02:13.463789+0000","last_undegraded":"2024-04-25T10:02:13.463789+0000","last_fullsized":"2024-04-25T10:02:13.463789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464659+0000","last_change":"2024-04-25T10:02:07.376985+0000","last_active":"2024-04-25T10:02:13.464659+0000","last_peered":"2024-04-25T10:02:13.464659+0000","last_clean":"2024-04-25T10:02:13.464659+0000","last_became_active":"2024-04-25T10:02:07.376669+0000","last_became_peered":"2024-04-25T10:02:07.376669+0000","last_unstale":"2024-04-25T10:02:13.464659+0000","last_undegraded":"2024-04-25T10:02:13.464659+0000","last_fullsized":"2024-04-25T10:02:13.464659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464634+0000","last_change":"2024-04-25T10:02:07.377373+0000","last_active":"2024-04-25T10:02:13.464634+0000","last_peered":"2024-04-25T10:02:13.464634+0000","last_clean":"2024-04-25T10:02:13.464634+0000","last_became_active":"2024-04-25T10:02:07.373588+0000","last_became_peered":"2024-04-25T10:02:07.373588+0000","last_unstale":"2024-04-25T10:02:13.464634+0000","last_undegraded":"2024-04-25T10:02:13.464634+0000","last_fullsized":"2024-04-25T10:02:13.464634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464847+0000","last_change":"2024-04-25T10:02:07.382540+0000","last_active":"2024-04-25T10:02:13.464847+0000","last_peered":"2024-04-25T10:02:13.464847+0000","last_clean":"2024-04-25T10:02:13.464847+0000","last_became_active":"2024-04-25T10:02:07.381996+0000","last_became_peered":"2024-04-25T10:02:07.381996+0000","last_unstale":"2024-04-25T10:02:13.464847+0000","last_undegraded":"2024-04-25T10:02:13.464847+0000","last_fullsized":"2024-04-25T10:02:13.464847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462277+0000","last_change":"2024-04-25T10:02:07.376120+0000","last_active":"2024-04-25T10:02:13.462277+0000","last_peered":"2024-04-25T10:02:13.462277+0000","last_clean":"2024-04-25T10:02:13.462277+0000","last_became_active":"2024-04-25T10:02:07.375589+0000","last_became_peered":"2024-04-25T10:02:07.375589+0000","last_unstale":"2024-04-25T10:02:13.462277+0000","last_undegraded":"2024-04-25T10:02:13.462277+0000","last_fullsized":"2024-04-25T10:02:13.462277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:35.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462177+0000","last_change":"2024-04-25T10:02:07.369060+0000","last_active":"2024-04-25T10:02:13.462177+0000","last_peered":"2024-04-25T10:02:13.462177+0000","last_clean":"2024-04-25T10:02:13.462177+0000","last_became_active":"2024-04-25T10:02:07.368434+0000","last_became_peered":"2024-04-25T10:02:07.368434+0000","last_unstale":"2024-04-25T10:02:13.462177+0000","last_undegraded":"2024-04-25T10:02:13.462177+0000","last_fullsized":"2024-04-25T10:02:13.462177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466451+0000","last_change":"2024-04-25T10:02:07.385523+0000","last_active":"2024-04-25T10:02:13.466451+0000","last_peered":"2024-04-25T10:02:13.466451+0000","last_clean":"2024-04-25T10:02:13.466451+0000","last_became_active":"2024-04-25T10:02:07.385218+0000","last_became_peered":"2024-04-25T10:02:07.385218+0000","last_unstale":"2024-04-25T10:02:13.466451+0000","last_undegraded":"2024-04-25T10:02:13.466451+0000","last_fullsized":"2024-04-25T10:02:13.466451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:34.056 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:34.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 Processor -- start 2024-04-25T10:02:34.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 -- start start 2024-04-25T10:02:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f652015da80 0x7f65200d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 0x7f65200d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652006be60 con 0x7f65200d0620 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652006c440 con 0x7f652015da80 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6527e7d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f652015de60 con 0x7f65200d1cd0 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f652641a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f65200d0620 0x7f65200ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58020/0 (socket says 172.21.15.82:58020) 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f652641a640 1 -- 172.21.15.82:0/1888718745 learned_addr learned my addr 172.21.15.82:0/1888718745 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525418640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 0x7f65200d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 --2- 172.21.15.82:0/1888718745 >> v2:172.21.15.196:3300/0 conn(0x7f652015da80 0x7f65200d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376189665 0 0) 0x7f652006be60 con 0x7f65200d0620 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f650c003530 con 0x7f65200d0620 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613746303 0 0) 0x7f650c003530 con 0x7f65200d0620 2024-04-25T10:02:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 msgr2=0x7f65200d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 0x7f65200d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 >> v2:172.21.15.196:3300/0 conn(0x7f652015da80 msgr2=0x7f65200d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 --2- 172.21.15.82:0/1888718745 >> v2:172.21.15.196:3300/0 conn(0x7f652015da80 0x7f65200d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200d2450 con 0x7f65200d0620 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6518001a00 con 0x7f65200d0620 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 >> v1:172.21.15.82:6789/0 conn(0x7f65200d0620 legacy=0x7f65200ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650c003a30 0x7f650c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 0x7f650c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c004d50 0x7f650c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652006c440 con 0x7f650c003a30 2024-04-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652015de60 con 0x7f650c004d50 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650c00b9d0 con 0x7f65200d1cd0 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650c003a30 0x7f650c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f652641a640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c004d50 0x7f650c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 -- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 msgr2=0x7f650c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65200d1cd0 0x7f650c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 -- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c004d50 msgr2=0x7f650c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650c004d50 0x7f650c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.196+0000 7f6525c19640 1 -- 172.21.15.82:0/1888718745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f650c003530 con 0x7f650c003a30 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6525c19640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650c003a30 0x7f650c003e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f651c009ae0 tx=0x7f651c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd5f685f30399ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0096b0 con 0x7f650c003a30 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c0a64b0 con 0x7f650c003a30 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6524c17640 1 -- 172.21.15.82:0/1888718745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0a68a0 con 0x7f650c003a30 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- 172.21.15.82:0/1888718745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650c003a30 msgr2=0x7f650c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 --2- 172.21.15.82:0/1888718745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f650c003a30 0x7f650c003e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f651c009ae0 tx=0x7f651c01cab0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 -- 172.21.15.82:0/1888718745 reap_dead start 2024-04-25T10:02:34.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- 172.21.15.82:0/1888718745 shutdown_connections 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- 172.21.15.82:0/1888718745 >> 172.21.15.82:0/1888718745 conn(0x7f652008b000 msgr2=0x7f65200d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- 172.21.15.82:0/1888718745 shutdown_connections 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- 172.21.15.82:0/1888718745 wait complete. 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 Processor -- start 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- start start 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6520153500 0x7f65200cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65201538e0 0x7f6520158dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201685a0 con 0x7f65201604d0 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cf2d0 con 0x7f6520153500 2024-04-25T10:02:34.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6527e7d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6520168720 con 0x7f65201538e0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49090/0 (socket says 172.21.15.82:49090) 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 -- 172.21.15.82:0/634518249 learned_addr learned my addr 172.21.15.82:0/634518249 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6525418640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65201538e0 0x7f6520158dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f6525c19640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6520153500 0x7f65200cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 -- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65201538e0 msgr2=0x7f6520158dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65201538e0 0x7f6520158dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 -- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6520153500 msgr2=0x7f65200cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6520153500 0x7f65200cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 -- 172.21.15.82:0/634518249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200c6f60 con 0x7f65201604d0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f652641a640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6518000f90 tx=0x7f651800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518019070 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651800bd80 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.200+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518012610 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65200c71d0 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65200cc8b0 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652004efa0 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65180127b0 con 0x7f65201604d0 2024-04-25T10:02:34.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f65167fc640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 0x7f6504069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f6525c19640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 0x7f6504069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6518085040 con 0x7f65201604d0 2024-04-25T10:02:34.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.204+0000 7f6525c19640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 0x7f6504069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f651c0af000 tx=0x7f651c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.208+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6518055d10 con 0x7f65201604d0 2024-04-25T10:02:34.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.352+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f652006ad40 con 0x7f6504066f90 2024-04-25T10:02:34.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.360+0000 7f65167fc640 1 -- 172.21.15.82:0/634518249 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f652006ad40 con 0x7f6504066f90 2024-04-25T10:02:34.370 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:34.372 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 msgr2=0x7f6504069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 0x7f6504069450 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f651c0af000 tx=0x7f651c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 msgr2=0x7f6520159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6518000f90 tx=0x7f651800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 shutdown_connections 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6504066f90 0x7f6504069450 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65201538e0 0x7f6520158dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65201604d0 0x7f6520159300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 --2- 172.21.15.82:0/634518249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6520153500 0x7f65200cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 >> 172.21.15.82:0/634518249 conn(0x7f652008b000 msgr2=0x7f6520167b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 shutdown_connections 2024-04-25T10:02:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.364+0000 7f6527e7d640 1 -- 172.21.15.82:0/634518249 wait complete. 2024-04-25T10:02:34.386 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-25T10:02:33.267200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223632,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751088,"statfs":{"total":773094113280,"available":772865114112,"internally_reserved":0,"allocated":3383296,"data_stored":2029432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004073"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462685+0000","last_change":"2024-04-25T10:02:06.377944+0000","last_active":"2024-04-25T10:02:13.462685+0000","last_peered":"2024-04-25T10:02:13.462685+0000","last_clean":"2024-04-25T10:02:13.462685+0000","last_became_active":"2024-04-25T10:02:06.377162+0000","last_became_peered":"2024-04-25T10:02:06.377162+0000","last_unstale":"2024-04-25T10:02:13.462685+0000","last_undegraded":"2024-04-25T10:02:13.462685+0000","last_fullsized":"2024-04-25T10:02:13.462685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406752+0000","last_change":"2024-04-25T10:02:07.375159+0000","last_active":"2024-04-25T10:02:12.406752+0000","last_peered":"2024-04-25T10:02:12.406752+0000","last_clean":"2024-04-25T10:02:12.406752+0000","last_became_active":"2024-04-25T10:02:07.374938+0000","last_became_peered":"2024-04-25T10:02:07.374938+0000","last_unstale":"2024-04-25T10:02:12.406752+0000","last_undegraded":"2024-04-25T10:02:12.406752+0000","last_fullsized":"2024-04-25T10:02:12.406752+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404043+0000","last_change":"2024-04-25T10:02:11.459079+0000","last_active":"2024-04-25T10:02:12.404043+0000","last_peered":"2024-04-25T10:02:12.404043+0000","last_clean":"2024-04-25T10:02:12.404043+0000","last_became_active":"2024-04-25T10:02:11.458605+0000","last_became_peered":"2024-04-25T10:02:11.458605+0000","last_unstale":"2024-04-25T10:02:12.404043+0000","last_undegraded":"2024-04-25T10:02:12.404043+0000","last_fullsized":"2024-04-25T10:02:12.404043+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407158+0000","last_change":"2024-04-25T10:02:06.377863+0000","last_active":"2024-04-25T10:02:12.407158+0000","last_peered":"2024-04-25T10:02:12.407158+0000","last_clean":"2024-04-25T10:02:12.407158+0000","last_became_active":"2024-04-25T10:02:06.377260+0000","last_became_peered":"2024-04-25T10:02:06.377260+0000","last_unstale":"2024-04-25T10:02:12.407158+0000","last_undegraded":"2024-04-25T10:02:12.407158+0000","last_fullsized":"2024-04-25T10:02:12.407158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462420+0000","last_change":"2024-04-25T10:02:07.371882+0000","last_active":"2024-04-25T10:02:13.462420+0000","last_peered":"2024-04-25T10:02:13.462420+0000","last_clean":"2024-04-25T10:02:13.462420+0000","last_became_active":"2024-04-25T10:02:07.371244+0000","last_became_peered":"2024-04-25T10:02:07.371244+0000","last_unstale":"2024-04-25T10:02:13.462420+0000","last_undegraded":"2024-04-25T10:02:13.462420+0000","last_fullsized":"2024-04-25T10:02:13.462420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405866+0000","last_change":"2024-04-25T10:02:11.435335+0000","last_active":"2024-04-25T10:02:12.405866+0000","last_peered":"2024-04-25T10:02:12.405866+0000","last_clean":"2024-04-25T10:02:12.405866+0000","last_became_active":"2024-04-25T10:02:11.434197+0000","last_became_peered":"2024-04-25T10:02:11.434197+0000","last_unstale":"2024-04-25T10:02:12.405866+0000","last_undegraded":"2024-04-25T10:02:12.405866+0000","last_fullsized":"2024-04-25T10:02:12.405866+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:43.303051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465851+0000","last_change":"2024-04-25T10:02:06.374878+0000","last_active":"2024-04-25T10:02:13.465851+0000","last_peered":"2024-04-25T10:02:13.465851+0000","last_clean":"2024-04-25T10:02:13.465851+0000","last_became_active":"2024-04-25T10:02:06.374210+0000","last_became_peered":"2024-04-25T10:02:06.374210+0000","last_unstale":"2024-04-25T10:02:13.465851+0000","last_undegraded":"2024-04-25T10:02:13.465851+0000","last_fullsized":"2024-04-25T10:02:13.465851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465104+0000","last_change":"2024-04-25T10:02:07.382381+0000","last_active":"2024-04-25T10:02:13.465104+0000","last_peered":"2024-04-25T10:02:13.465104+0000","last_clean":"2024-04-25T10:02:13.465104+0000","last_became_active":"2024-04-25T10:02:07.381718+0000","last_became_peered":"2024-04-25T10:02:07.381718+0000","last_unstale":"2024-04-25T10:02:13.465104+0000","last_undegraded":"2024-04-25T10:02:13.465104+0000","last_fullsized":"2024-04-25T10:02:13.465104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407612+0000","last_change":"2024-04-25T10:02:11.442974+0000","last_active":"2024-04-25T10:02:12.407612+0000","last_peered":"2024-04-25T10:02:12.407612+0000","last_clean":"2024-04-25T10:02:12.407612+0000","last_became_active":"2024-04-25T10:02:11.441674+0000","last_became_peered":"2024-04-25T10:02:11.441674+0000","last_unstale":"2024-04-25T10:02:12.407612+0000","last_undegraded":"2024-04-25T10:02:12.407612+0000","last_fullsized":"2024-04-25T10:02:12.407612+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464230+0000","last_change":"2024-04-25T10:02:06.388913+0000","last_active":"2024-04-25T10:02:13.464230+0000","last_peered":"2024-04-25T10:02:13.464230+0000","last_clean":"2024-04-25T10:02:13.464230+0000","last_became_active":"2024-04-25T10:02:06.388536+0000","last_became_peered":"2024-04-25T10:02:06.388536+0000","last_unstale":"2024-04-25T10:02:13.464230+0000","last_undegraded":"2024-04-25T10:02:13.464230+0000","last_fullsized":"2024-04-25T10:02:13.464230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:41.104882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465595+0000","last_change":"2024-04-25T10:02:07.382060+0000","last_active":"2024-04-25T10:02:13.465595+0000","last_peered":"2024-04-25T10:02:13.465595+0000","last_clean":"2024-04-25T10:02:13.465595+0000","last_became_active":"2024-04-25T10:02:07.381779+0000","last_became_peered":"2024-04-25T10:02:07.381779+0000","last_unstale":"2024-04-25T10:02:13.465595+0000","last_undegraded":"2024-04-25T10:02:13.465595+0000","last_fullsized":"2024-04-25T10:02:13.465595+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406133+0000","last_change":"2024-04-25T10:02:11.430607+0000","last_active":"2024-04-25T10:02:12.406133+0000","last_peered":"2024-04-25T10:02:12.406133+0000","last_clean":"2024-04-25T10:02:12.406133+0000","last_became_active":"2024-04-25T10:02:11.429281+0000","last_became_peered":"2024-04-25T10:02:11.429281+0000","last_unstale":"2024-04-25T10:02:12.406133+0000","last_undegraded":"2024-04-25T10:02:12.406133+0000","last_fullsized":"2024-04-25T10:02:12.406133+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:06.213948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405179+0000","last_change":"2024-04-25T10:02:07.365923+0000","last_active":"2024-04-25T10:02:12.405179+0000","last_peered":"2024-04-25T10:02:12.405179+0000","last_clean":"2024-04-25T10:02:12.405179+0000","last_became_active":"2024-04-25T10:02:07.365345+0000","last_became_peered":"2024-04-25T10:02:07.365345+0000","last_unstale":"2024-04-25T10:02:12.405179+0000","last_undegraded":"2024-04-25T10:02:12.405179+0000","last_fullsized":"2024-04-25T10:02:12.405179+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465328+0000","last_change":"2024-04-25T10:02:06.373663+0000","last_active":"2024-04-25T10:02:13.465328+0000","last_peered":"2024-04-25T10:02:13.465328+0000","last_clean":"2024-04-25T10:02:13.465328+0000","last_became_active":"2024-04-25T10:02:06.372972+0000","last_became_peered":"2024-04-25T10:02:06.372972+0000","last_unstale":"2024-04-25T10:02:13.465328+0000","last_undegraded":"2024-04-25T10:02:13.465328+0000","last_fullsized":"2024-04-25T10:02:13.465328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408366+0000","last_change":"2024-04-25T10:02:11.432916+0000","last_active":"2024-04-25T10:02:12.408366+0000","last_peered":"2024-04-25T10:02:12.408366+0000","last_clean":"2024-04-25T10:02:12.408366+0000","last_became_active":"2024-04-25T10:02:11.431758+0000","last_became_peered":"2024-04-25T10:02:11.431758+0000","last_unstale":"2024-04-25T10:02:12.408366+0000","last_undegraded":"2024-04-25T10:02:12.408366+0000","last_fullsized":"2024-04-25T10:02:12.408366+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403824+0000","last_change":"2024-04-25T10:02:07.371074+0000","last_active":"2024-04-25T10:02:12.403824+0000","last_peered":"2024-04-25T10:02:12.403824+0000","last_clean":"2024-04-25T10:02:12.403824+0000","last_became_active":"2024-04-25T10:02:07.370770+0000","last_became_peered":"2024-04-25T10:02:07.370770+0000","last_unstale":"2024-04-25T10:02:12.403824+0000","last_undegraded":"2024-04-25T10:02:12.403824+0000","last_fullsized":"2024-04-25T10:02:12.403824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462663+0000","last_change":"2024-04-25T10:02:06.369892+0000","last_active":"2024-04-25T10:02:13.462663+0000","last_peered":"2024-04-25T10:02:13.462663+0000","last_clean":"2024-04-25T10:02:13.462663+0000","last_became_active":"2024-04-25T10:02:06.369183+0000","last_became_peered":"2024-04-25T10:02:06.369183+0000","last_unstale":"2024-04-25T10:02:13.462663+0000","last_undegraded":"2024-04-25T10:02:13.462663+0000","last_fullsized":"2024-04-25T10:02:13.462663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:53.653298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406907+0000","last_change":"2024-04-25T10:02:11.442628+0000","last_active":"2024-04-25T10:02:12.406907+0000","last_peered":"2024-04-25T10:02:12.406907+0000","last_clean":"2024-04-25T10:02:12.406907+0000","last_became_active":"2024-04-25T10:02:11.441578+0000","last_became_peered":"2024-04-25T10:02:11.441578+0000","last_unstale":"2024-04-25T10:02:12.406907+0000","last_undegraded":"2024-04-25T10:02:12.406907+0000","last_fullsized":"2024-04-25T10:02:12.406907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:00.571221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403861+0000","last_change":"2024-04-25T10:02:07.361735+0000","last_active":"2024-04-25T10:02:12.403861+0000","last_peered":"2024-04-25T10:02:12.403861+0000","last_clean":"2024-04-25T10:02:12.403861+0000","last_became_active":"2024-04-25T10:02:07.361435+0000","last_became_peered":"2024-04-25T10:02:07.361435+0000","last_unstale":"2024-04-25T10:02:12.403861+0000","last_undegraded":"2024-04-25T10:02:12.403861+0000","last_fullsized":"2024-04-25T10:02:12.403861+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461678+0000","last_change":"2024-04-25T10:02:06.369621+0000","last_active":"2024-04-25T10:02:13.461678+0000","last_peered":"2024-04-25T10:02:13.461678+0000","last_clean":"2024-04-25T10:02:13.461678+0000","last_became_active":"2024-04-25T10:02:06.368883+0000","last_became_peered":"2024-04-25T10:02:06.368883+0000","last_unstale":"2024-04-25T10:02:13.461678+0000","last_undegraded":"2024-04-25T10:02:13.461678+0000","last_fullsized":"2024-04-25T10:02:13.461678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:18.817322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462909+0000","last_change":"2024-04-25T10:02:11.462237+0000","last_active":"2024-04-25T10:02:13.462909+0000","last_peered":"2024-04-25T10:02:13.462909+0000","last_clean":"2024-04-25T10:02:13.462909+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.462909+0000","last_undegraded":"2024-04-25T10:02:13.462909+0000","last_fullsized":"2024-04-25T10:02:13.462909+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406991+0000","last_change":"2024-04-25T10:02:07.357629+0000","last_active":"2024-04-25T10:02:12.406991+0000","last_peered":"2024-04-25T10:02:12.406991+0000","last_clean":"2024-04-25T10:02:12.406991+0000","last_became_active":"2024-04-25T10:02:07.357353+0000","last_became_peered":"2024-04-25T10:02:07.357353+0000","last_unstale":"2024-04-25T10:02:12.406991+0000","last_undegraded":"2024-04-25T10:02:12.406991+0000","last_fullsized":"2024-04-25T10:02:12.406991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:17.663068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464640+0000","last_change":"2024-04-25T10:02:06.384551+0000","last_active":"2024-04-25T10:02:13.464640+0000","last_peered":"2024-04-25T10:02:13.464640+0000","last_clean":"2024-04-25T10:02:13.464640+0000","last_became_active":"2024-04-25T10:02:06.384131+0000","last_became_peered":"2024-04-25T10:02:06.384131+0000","last_unstale":"2024-04-25T10:02:13.464640+0000","last_undegraded":"2024-04-25T10:02:13.464640+0000","last_fullsized":"2024-04-25T10:02:13.464640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408508+0000","last_change":"2024-04-25T10:02:11.443507+0000","last_active":"2024-04-25T10:02:12.408508+0000","last_peered":"2024-04-25T10:02:12.408508+0000","last_clean":"2024-04-25T10:02:12.408508+0000","last_became_active":"2024-04-25T10:02:11.440496+0000","last_became_peered":"2024-04-25T10:02:11.440496+0000","last_unstale":"2024-04-25T10:02:12.408508+0000","last_undegraded":"2024-04-25T10:02:12.408508+0000","last_fullsized":"2024-04-25T10:02:12.408508+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462520+0000","last_change":"2024-04-25T10:02:07.367892+0000","last_active":"2024-04-25T10:02:13.462520+0000","last_peered":"2024-04-25T10:02:13.462520+0000","last_clean":"2024-04-25T10:02:13.462520+0000","last_became_active":"2024-04-25T10:02:07.367404+0000","last_became_peered":"2024-04-25T10:02:07.367404+0000","last_unstale":"2024-04-25T10:02:13.462520+0000","last_undegraded":"2024-04-25T10:02:13.462520+0000","last_fullsized":"2024-04-25T10:02:13.462520+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406984+0000","last_change":"2024-04-25T10:02:06.371009+0000","last_active":"2024-04-25T10:02:12.406984+0000","last_peered":"2024-04-25T10:02:12.406984+0000","last_clean":"2024-04-25T10:02:12.406984+0000","last_became_active":"2024-04-25T10:02:06.369415+0000","last_became_peered":"2024-04-25T10:02:06.369415+0000","last_unstale":"2024-04-25T10:02:12.406984+0000","last_undegraded":"2024-04-25T10:02:12.406984+0000","last_fullsized":"2024-04-25T10:02:12.406984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406594+0000","last_change":"2024-04-25T10:02:11.460234+0000","last_active":"2024-04-25T10:02:12.406594+0000","last_peered":"2024-04-25T10:02:12.406594+0000","last_clean":"2024-04-25T10:02:12.406594+0000","last_became_active":"2024-04-25T10:02:11.459268+0000","last_became_peered":"2024-04-25T10:02:11.459268+0000","last_unstale":"2024-04-25T10:02:12.406594+0000","last_undegraded":"2024-04-25T10:02:12.406594+0000","last_fullsized":"2024-04-25T10:02:12.406594+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:35.022355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406871+0000","last_change":"2024-04-25T10:02:07.385204+0000","last_active":"2024-04-25T10:02:12.406871+0000","last_peered":"2024-04-25T10:02:12.406871+0000","last_clean":"2024-04-25T10:02:12.406871+0000","last_became_active":"2024-04-25T10:02:07.384798+0000","last_became_peered":"2024-04-25T10:02:07.384798+0000","last_unstale":"2024-04-25T10:02:12.406871+0000","last_undegraded":"2024-04-25T10:02:12.406871+0000","last_fullsized":"2024-04-25T10:02:12.406871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462229+0000","last_change":"2024-04-25T10:02:06.363593+0000","last_active":"2024-04-25T10:02:13.462229+0000","last_peered":"2024-04-25T10:02:13.462229+0000","last_clean":"2024-04-25T10:02:13.462229+0000","last_became_active":"2024-04-25T10:02:06.363127+0000","last_became_peered":"2024-04-25T10:02:06.363127+0000","last_unstale":"2024-04-25T10:02:13.462229+0000","last_undegraded":"2024-04-25T10:02:13.462229+0000","last_fullsized":"2024-04-25T10:02:13.462229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403605+0000","last_change":"2024-04-25T10:02:11.453752+0000","last_active":"2024-04-25T10:02:12.403605+0000","last_peered":"2024-04-25T10:02:12.403605+0000","last_clean":"2024-04-25T10:02:12.403605+0000","last_became_active":"2024-04-25T10:02:11.453254+0000","last_became_peered":"2024-04-25T10:02:11.453254+0000","last_unstale":"2024-04-25T10:02:12.403605+0000","last_undegraded":"2024-04-25T10:02:12.403605+0000","last_fullsized":"2024-04-25T10:02:12.403605+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:04.566083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462068+0000","last_change":"2024-04-25T10:02:07.356096+0000","last_active":"2024-04-25T10:02:13.462068+0000","last_peered":"2024-04-25T10:02:13.462068+0000","last_clean":"2024-04-25T10:02:13.462068+0000","last_became_active":"2024-04-25T10:02:07.355433+0000","last_became_peered":"2024-04-25T10:02:07.355433+0000","last_unstale":"2024-04-25T10:02:13.462068+0000","last_undegraded":"2024-04-25T10:02:13.462068+0000","last_fullsized":"2024-04-25T10:02:13.462068+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466154+0000","last_change":"2024-04-25T10:02:06.385234+0000","last_active":"2024-04-25T10:02:13.466154+0000","last_peered":"2024-04-25T10:02:13.466154+0000","last_clean":"2024-04-25T10:02:13.466154+0000","last_became_active":"2024-04-25T10:02:06.384494+0000","last_became_peered":"2024-04-25T10:02:06.384494+0000","last_unstale":"2024-04-25T10:02:13.466154+0000","last_undegraded":"2024-04-25T10:02:13.466154+0000","last_fullsized":"2024-04-25T10:02:13.466154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:34.191177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462761+0000","last_change":"2024-04-25T10:02:11.459695+0000","last_active":"2024-04-25T10:02:13.462761+0000","last_peered":"2024-04-25T10:02:13.462761+0000","last_clean":"2024-04-25T10:02:13.462761+0000","last_became_active":"2024-04-25T10:02:11.459247+0000","last_became_peered":"2024-04-25T10:02:11.459247+0000","last_unstale":"2024-04-25T10:02:13.462761+0000","last_undegraded":"2024-04-25T10:02:13.462761+0000","last_fullsized":"2024-04-25T10:02:13.462761+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465853+0000","last_change":"2024-04-25T10:02:07.374402+0000","last_active":"2024-04-25T10:02:13.465853+0000","last_peered":"2024-04-25T10:02:13.465853+0000","last_clean":"2024-04-25T10:02:13.465853+0000","last_became_active":"2024-04-25T10:02:07.374110+0000","last_became_peered":"2024-04-25T10:02:07.374110+0000","last_unstale":"2024-04-25T10:02:13.465853+0000","last_undegraded":"2024-04-25T10:02:13.465853+0000","last_fullsized":"2024-04-25T10:02:13.465853+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462182+0000","last_change":"2024-04-25T10:02:06.368581+0000","last_active":"2024-04-25T10:02:13.462182+0000","last_peered":"2024-04-25T10:02:13.462182+0000","last_clean":"2024-04-25T10:02:13.462182+0000","last_became_active":"2024-04-25T10:02:06.368085+0000","last_became_peered":"2024-04-25T10:02:06.368085+0000","last_unstale":"2024-04-25T10:02:13.462182+0000","last_undegraded":"2024-04-25T10:02:13.462182+0000","last_fullsized":"2024-04-25T10:02:13.462182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461807+0000","last_change":"2024-04-25T10:02:11.435403+0000","last_active":"2024-04-25T10:02:13.461807+0000","last_peered":"2024-04-25T10:02:13.461807+0000","last_clean":"2024-04-25T10:02:13.461807+0000","last_became_active":"2024-04-25T10:02:11.435010+0000","last_became_peered":"2024-04-25T10:02:11.435010+0000","last_unstale":"2024-04-25T10:02:13.461807+0000","last_undegraded":"2024-04-25T10:02:13.461807+0000","last_fullsized":"2024-04-25T10:02:13.461807+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407792+0000","last_change":"2024-04-25T10:02:07.378715+0000","last_active":"2024-04-25T10:02:12.407792+0000","last_peered":"2024-04-25T10:02:12.407792+0000","last_clean":"2024-04-25T10:02:12.407792+0000","last_became_active":"2024-04-25T10:02:07.378297+0000","last_became_peered":"2024-04-25T10:02:07.378297+0000","last_unstale":"2024-04-25T10:02:12.407792+0000","last_undegraded":"2024-04-25T10:02:12.407792+0000","last_fullsized":"2024-04-25T10:02:12.407792+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:19.830497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465696+0000","last_change":"2024-04-25T10:02:06.381259+0000","last_active":"2024-04-25T10:02:13.465696+0000","last_peered":"2024-04-25T10:02:13.465696+0000","last_clean":"2024-04-25T10:02:13.465696+0000","last_became_active":"2024-04-25T10:02:06.377864+0000","last_became_peered":"2024-04-25T10:02:06.377864+0000","last_unstale":"2024-04-25T10:02:13.465696+0000","last_undegraded":"2024-04-25T10:02:13.465696+0000","last_fullsized":"2024-04-25T10:02:13.465696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462019+0000","last_change":"2024-04-25T10:02:11.435443+0000","last_active":"2024-04-25T10:02:13.462019+0000","last_peered":"2024-04-25T10:02:13.462019+0000","last_clean":"2024-04-25T10:02:13.462019+0000","last_became_active":"2024-04-25T10:02:11.434691+0000","last_became_peered":"2024-04-25T10:02:11.434691+0000","last_unstale":"2024-04-25T10:02:13.462019+0000","last_undegraded":"2024-04-25T10:02:13.462019+0000","last_fullsized":"2024-04-25T10:02:13.462019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462938+0000","last_change":"2024-04-25T10:02:07.360180+0000","last_active":"2024-04-25T10:02:13.462938+0000","last_peered":"2024-04-25T10:02:13.462938+0000","last_clean":"2024-04-25T10:02:13.462938+0000","last_became_active":"2024-04-25T10:02:07.358876+0000","last_became_peered":"2024-04-25T10:02:07.358876+0000","last_unstale":"2024-04-25T10:02:13.462938+0000","last_undegraded":"2024-04-25T10:02:13.462938+0000","last_fullsized":"2024-04-25T10:02:13.462938+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462463+0000","last_change":"2024-04-25T10:02:06.376414+0000","last_active":"2024-04-25T10:02:13.462463+0000","last_peered":"2024-04-25T10:02:13.462463+0000","last_clean":"2024-04-25T10:02:13.462463+0000","last_became_active":"2024-04-25T10:02:06.375943+0000","last_became_peered":"2024-04-25T10:02:06.375943+0000","last_unstale":"2024-04-25T10:02:13.462463+0000","last_undegraded":"2024-04-25T10:02:13.462463+0000","last_fullsized":"2024-04-25T10:02:13.462463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463033+0000","last_change":"2024-04-25T10:02:11.438382+0000","last_active":"2024-04-25T10:02:13.463033+0000","last_peered":"2024-04-25T10:02:13.463033+0000","last_clean":"2024-04-25T10:02:13.463033+0000","last_became_active":"2024-04-25T10:02:11.436135+0000","last_became_peered":"2024-04-25T10:02:11.436135+0000","last_unstale":"2024-04-25T10:02:13.463033+0000","last_undegraded":"2024-04-25T10:02:13.463033+0000","last_fullsized":"2024-04-25T10:02:13.463033+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.460832+0000","last_change":"2024-04-25T10:02:07.366816+0000","last_active":"2024-04-25T10:02:13.460832+0000","last_peered":"2024-04-25T10:02:13.460832+0000","last_clean":"2024-04-25T10:02:13.460832+0000","last_became_active":"2024-04-25T10:02:07.366193+0000","last_became_peered":"2024-04-25T10:02:07.366193+0000","last_unstale":"2024-04-25T10:02:13.460832+0000","last_undegraded":"2024-04-25T10:02:13.460832+0000","last_fullsized":"2024-04-25T10:02:13.460832+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463419+0000","last_change":"2024-04-25T10:02:06.376427+0000","last_active":"2024-04-25T10:02:13.463419+0000","last_peered":"2024-04-25T10:02:13.463419+0000","last_clean":"2024-04-25T10:02:13.463419+0000","last_became_active":"2024-04-25T10:02:06.375955+0000","last_became_peered":"2024-04-25T10:02:06.375955+0000","last_unstale":"2024-04-25T10:02:13.463419+0000","last_undegraded":"2024-04-25T10:02:13.463419+0000","last_fullsized":"2024-04-25T10:02:13.463419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462868+0000","last_change":"2024-04-25T10:02:11.458524+0000","last_active":"2024-04-25T10:02:13.462868+0000","last_peered":"2024-04-25T10:02:13.462868+0000","last_clean":"2024-04-25T10:02:13.462868+0000","last_became_active":"2024-04-25T10:02:11.458105+0000","last_became_peered":"2024-04-25T10:02:11.458105+0000","last_unstale":"2024-04-25T10:02:13.462868+0000","last_undegraded":"2024-04-25T10:02:13.462868+0000","last_fullsized":"2024-04-25T10:02:13.462868+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:35.014428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403806+0000","last_change":"2024-04-25T10:02:07.369247+0000","last_active":"2024-04-25T10:02:12.403806+0000","last_peered":"2024-04-25T10:02:12.403806+0000","last_clean":"2024-04-25T10:02:12.403806+0000","last_became_active":"2024-04-25T10:02:07.368616+0000","last_became_peered":"2024-04-25T10:02:07.368616+0000","last_unstale":"2024-04-25T10:02:12.403806+0000","last_undegraded":"2024-04-25T10:02:12.403806+0000","last_fullsized":"2024-04-25T10:02:12.403806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466462+0000","last_change":"2024-04-25T10:02:06.385925+0000","last_active":"2024-04-25T10:02:13.466462+0000","last_peered":"2024-04-25T10:02:13.466462+0000","last_clean":"2024-04-25T10:02:13.466462+0000","last_became_active":"2024-04-25T10:02:06.382216+0000","last_became_peered":"2024-04-25T10:02:06.382216+0000","last_unstale":"2024-04-25T10:02:13.466462+0000","last_undegraded":"2024-04-25T10:02:13.466462+0000","last_fullsized":"2024-04-25T10:02:13.466462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463713+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:02:13.463713+0000","last_peered":"2024-04-25T10:02:13.463713+0000","last_clean":"2024-04-25T10:02:13.463713+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:02:13.463713+0000","last_undegraded":"2024-04-25T10:02:13.463713+0000","last_fullsized":"2024-04-25T10:02:13.463713+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406156+0000","last_change":"2024-04-25T10:02:11.460389+0000","last_active":"2024-04-25T10:02:12.406156+0000","last_peered":"2024-04-25T10:02:12.406156+0000","last_clean":"2024-04-25T10:02:12.406156+0000","last_became_active":"2024-04-25T10:02:11.459440+0000","last_became_peered":"2024-04-25T10:02:11.459440+0000","last_unstale":"2024-04-25T10:02:12.406156+0000","last_undegraded":"2024-04-25T10:02:12.406156+0000","last_fullsized":"2024-04-25T10:02:12.406156+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465341+0000","last_change":"2024-04-25T10:02:07.388181+0000","last_active":"2024-04-25T10:02:13.465341+0000","last_peered":"2024-04-25T10:02:13.465341+0000","last_clean":"2024-04-25T10:02:13.465341+0000","last_became_active":"2024-04-25T10:02:07.385106+0000","last_became_peered":"2024-04-25T10:02:07.385106+0000","last_unstale":"2024-04-25T10:02:13.465341+0000","last_undegraded":"2024-04-25T10:02:13.465341+0000","last_fullsized":"2024-04-25T10:02:13.465341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463240+0000","last_change":"2024-04-25T10:02:06.386894+0000","last_active":"2024-04-25T10:02:13.463240+0000","last_peered":"2024-04-25T10:02:13.463240+0000","last_clean":"2024-04-25T10:02:13.463240+0000","last_became_active":"2024-04-25T10:02:06.386546+0000","last_became_peered":"2024-04-25T10:02:06.386546+0000","last_unstale":"2024-04-25T10:02:13.463240+0000","last_undegraded":"2024-04-25T10:02:13.463240+0000","last_fullsized":"2024-04-25T10:02:13.463240+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:39.307540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405476+0000","last_change":"2024-04-25T10:02:11.455613+0000","last_active":"2024-04-25T10:02:12.405476+0000","last_peered":"2024-04-25T10:02:12.405476+0000","last_clean":"2024-04-25T10:02:12.405476+0000","last_became_active":"2024-04-25T10:02:11.454401+0000","last_became_peered":"2024-04-25T10:02:11.454401+0000","last_unstale":"2024-04-25T10:02:12.405476+0000","last_undegraded":"2024-04-25T10:02:12.405476+0000","last_fullsized":"2024-04-25T10:02:12.405476+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:04.688836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465364+0000","last_change":"2024-04-25T10:02:07.382626+0000","last_active":"2024-04-25T10:02:13.465364+0000","last_peered":"2024-04-25T10:02:13.465364+0000","last_clean":"2024-04-25T10:02:13.465364+0000","last_became_active":"2024-04-25T10:02:07.382110+0000","last_became_peered":"2024-04-25T10:02:07.382110+0000","last_unstale":"2024-04-25T10:02:13.465364+0000","last_undegraded":"2024-04-25T10:02:13.465364+0000","last_fullsized":"2024-04-25T10:02:13.465364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461856+0000","last_change":"2024-04-25T10:02:06.361762+0000","last_active":"2024-04-25T10:02:13.461856+0000","last_peered":"2024-04-25T10:02:13.461856+0000","last_clean":"2024-04-25T10:02:13.461856+0000","last_became_active":"2024-04-25T10:02:06.360531+0000","last_became_peered":"2024-04-25T10:02:06.360531+0000","last_unstale":"2024-04-25T10:02:13.461856+0000","last_undegraded":"2024-04-25T10:02:13.461856+0000","last_fullsized":"2024-04-25T10:02:13.461856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463028+0000","last_change":"2024-04-25T10:02:11.435880+0000","last_active":"2024-04-25T10:02:13.463028+0000","last_peered":"2024-04-25T10:02:13.463028+0000","last_clean":"2024-04-25T10:02:13.463028+0000","last_became_active":"2024-04-25T10:02:11.435244+0000","last_became_peered":"2024-04-25T10:02:11.435244+0000","last_unstale":"2024-04-25T10:02:13.463028+0000","last_undegraded":"2024-04-25T10:02:13.463028+0000","last_fullsized":"2024-04-25T10:02:13.463028+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:20.160758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466834+0000","last_change":"2024-04-25T10:02:07.385486+0000","last_active":"2024-04-25T10:02:13.466834+0000","last_peered":"2024-04-25T10:02:13.466834+0000","last_clean":"2024-04-25T10:02:13.466834+0000","last_became_active":"2024-04-25T10:02:07.385202+0000","last_became_peered":"2024-04-25T10:02:07.385202+0000","last_unstale":"2024-04-25T10:02:13.466834+0000","last_undegraded":"2024-04-25T10:02:13.466834+0000","last_fullsized":"2024-04-25T10:02:13.466834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:16.103362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462441+0000","last_change":"2024-04-25T10:02:06.379591+0000","last_active":"2024-04-25T10:02:13.462441+0000","last_peered":"2024-04-25T10:02:13.462441+0000","last_clean":"2024-04-25T10:02:13.462441+0000","last_became_active":"2024-04-25T10:02:06.379178+0000","last_became_peered":"2024-04-25T10:02:06.379178+0000","last_unstale":"2024-04-25T10:02:13.462441+0000","last_undegraded":"2024-04-25T10:02:13.462441+0000","last_fullsized":"2024-04-25T10:02:13.462441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463716+0000","last_change":"2024-04-25T10:02:11.459834+0000","last_active":"2024-04-25T10:02:13.463716+0000","last_peered":"2024-04-25T10:02:13.463716+0000","last_clean":"2024-04-25T10:02:13.463716+0000","last_became_active":"2024-04-25T10:02:11.459000+0000","last_became_peered":"2024-04-25T10:02:11.459000+0000","last_unstale":"2024-04-25T10:02:13.463716+0000","last_undegraded":"2024-04-25T10:02:13.463716+0000","last_fullsized":"2024-04-25T10:02:13.463716+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466278+0000","last_change":"2024-04-25T10:02:07.387540+0000","last_active":"2024-04-25T10:02:13.466278+0000","last_peered":"2024-04-25T10:02:13.466278+0000","last_clean":"2024-04-25T10:02:13.466278+0000","last_became_active":"2024-04-25T10:02:07.385395+0000","last_became_peered":"2024-04-25T10:02:07.385395+0000","last_unstale":"2024-04-25T10:02:13.466278+0000","last_undegraded":"2024-04-25T10:02:13.466278+0000","last_fullsized":"2024-04-25T10:02:13.466278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:00.631440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.408935+0000","last_change":"2024-04-25T10:02:06.379749+0000","last_active":"2024-04-25T10:02:12.408935+0000","last_peered":"2024-04-25T10:02:12.408935+0000","last_clean":"2024-04-25T10:02:12.408935+0000","last_became_active":"2024-04-25T10:02:06.378947+0000","last_became_peered":"2024-04-25T10:02:06.378947+0000","last_unstale":"2024-04-25T10:02:12.408935+0000","last_undegraded":"2024-04-25T10:02:12.408935+0000","last_fullsized":"2024-04-25T10:02:12.408935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:03.055592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405719+0000","last_change":"2024-04-25T10:02:11.437602+0000","last_active":"2024-04-25T10:02:12.405719+0000","last_peered":"2024-04-25T10:02:12.405719+0000","last_clean":"2024-04-25T10:02:12.405719+0000","last_became_active":"2024-04-25T10:02:11.436553+0000","last_became_peered":"2024-04-25T10:02:11.436553+0000","last_unstale":"2024-04-25T10:02:12.405719+0000","last_undegraded":"2024-04-25T10:02:12.405719+0000","last_fullsized":"2024-04-25T10:02:12.405719+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:53.618014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464059+0000","last_change":"2024-04-25T10:02:07.384411+0000","last_active":"2024-04-25T10:02:13.464059+0000","last_peered":"2024-04-25T10:02:13.464059+0000","last_clean":"2024-04-25T10:02:13.464059+0000","last_became_active":"2024-04-25T10:02:07.377814+0000","last_became_peered":"2024-04-25T10:02:07.377814+0000","last_unstale":"2024-04-25T10:02:13.464059+0000","last_undegraded":"2024-04-25T10:02:13.464059+0000","last_fullsized":"2024-04-25T10:02:13.464059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461988+0000","last_change":"2024-04-25T10:02:06.360702+0000","last_active":"2024-04-25T10:02:13.461988+0000","last_peered":"2024-04-25T10:02:13.461988+0000","last_clean":"2024-04-25T10:02:13.461988+0000","last_became_active":"2024-04-25T10:02:06.360297+0000","last_became_peered":"2024-04-25T10:02:06.360297+0000","last_unstale":"2024-04-25T10:02:13.461988+0000","last_undegraded":"2024-04-25T10:02:13.461988+0000","last_fullsized":"2024-04-25T10:02:13.461988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462271+0000","last_change":"2024-04-25T10:02:11.427201+0000","last_active":"2024-04-25T10:02:13.462271+0000","last_peered":"2024-04-25T10:02:13.462271+0000","last_clean":"2024-04-25T10:02:13.462271+0000","last_became_active":"2024-04-25T10:02:11.426734+0000","last_became_peered":"2024-04-25T10:02:11.426734+0000","last_unstale":"2024-04-25T10:02:13.462271+0000","last_undegraded":"2024-04-25T10:02:13.462271+0000","last_fullsized":"2024-04-25T10:02:13.462271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:33.474168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404061+0000","last_change":"2024-04-25T10:02:07.371003+0000","last_active":"2024-04-25T10:02:12.404061+0000","last_peered":"2024-04-25T10:02:12.404061+0000","last_clean":"2024-04-25T10:02:12.404061+0000","last_became_active":"2024-04-25T10:02:07.370695+0000","last_became_peered":"2024-04-25T10:02:07.370695+0000","last_unstale":"2024-04-25T10:02:12.404061+0000","last_undegraded":"2024-04-25T10:02:12.404061+0000","last_fullsized":"2024-04-25T10:02:12.404061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405106+0000","last_change":"2024-04-25T10:02:06.362100+0000","last_active":"2024-04-25T10:02:12.405106+0000","last_peered":"2024-04-25T10:02:12.405106+0000","last_clean":"2024-04-25T10:02:12.405106+0000","last_became_active":"2024-04-25T10:02:06.361358+0000","last_became_peered":"2024-04-25T10:02:06.361358+0000","last_unstale":"2024-04-25T10:02:12.405106+0000","last_undegraded":"2024-04-25T10:02:12.405106+0000","last_fullsized":"2024-04-25T10:02:12.405106+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406644+0000","last_change":"2024-04-25T10:02:11.460013+0000","last_active":"2024-04-25T10:02:12.406644+0000","last_peered":"2024-04-25T10:02:12.406644+0000","last_clean":"2024-04-25T10:02:12.406644+0000","last_became_active":"2024-04-25T10:02:11.459491+0000","last_became_peered":"2024-04-25T10:02:11.459491+0000","last_unstale":"2024-04-25T10:02:12.406644+0000","last_undegraded":"2024-04-25T10:02:12.406644+0000","last_fullsized":"2024-04-25T10:02:12.406644+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.241421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465085+0000","last_change":"2024-04-25T10:02:07.373359+0000","last_active":"2024-04-25T10:02:13.465085+0000","last_peered":"2024-04-25T10:02:13.465085+0000","last_clean":"2024-04-25T10:02:13.465085+0000","last_became_active":"2024-04-25T10:02:07.373100+0000","last_became_peered":"2024-04-25T10:02:07.373100+0000","last_unstale":"2024-04-25T10:02:13.465085+0000","last_undegraded":"2024-04-25T10:02:13.465085+0000","last_fullsized":"2024-04-25T10:02:13.465085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:30.957032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465425+0000","last_change":"2024-04-25T10:02:06.382032+0000","last_active":"2024-04-25T10:02:13.465425+0000","last_peered":"2024-04-25T10:02:13.465425+0000","last_clean":"2024-04-25T10:02:13.465425+0000","last_became_active":"2024-04-25T10:02:06.381711+0000","last_became_peered":"2024-04-25T10:02:06.381711+0000","last_unstale":"2024-04-25T10:02:13.465425+0000","last_undegraded":"2024-04-25T10:02:13.465425+0000","last_fullsized":"2024-04-25T10:02:13.465425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404352+0000","last_change":"2024-04-25T10:02:11.458829+0000","last_active":"2024-04-25T10:02:12.404352+0000","last_peered":"2024-04-25T10:02:12.404352+0000","last_clean":"2024-04-25T10:02:12.404352+0000","last_became_active":"2024-04-25T10:02:11.458386+0000","last_became_peered":"2024-04-25T10:02:11.458386+0000","last_unstale":"2024-04-25T10:02:12.404352+0000","last_undegraded":"2024-04-25T10:02:12.404352+0000","last_fullsized":"2024-04-25T10:02:12.404352+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:15.902040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462813+0000","last_change":"2024-04-25T10:02:07.372352+0000","last_active":"2024-04-25T10:02:13.462813+0000","last_peered":"2024-04-25T10:02:13.462813+0000","last_clean":"2024-04-25T10:02:13.462813+0000","last_became_active":"2024-04-25T10:02:07.372124+0000","last_became_peered":"2024-04-25T10:02:07.372124+0000","last_unstale":"2024-04-25T10:02:13.462813+0000","last_undegraded":"2024-04-25T10:02:13.462813+0000","last_fullsized":"2024-04-25T10:02:13.462813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463253+0000","last_change":"2024-04-25T10:02:06.361119+0000","last_active":"2024-04-25T10:02:13.463253+0000","last_peered":"2024-04-25T10:02:13.463253+0000","last_clean":"2024-04-25T10:02:13.463253+0000","last_became_active":"2024-04-25T10:02:06.360459+0000","last_became_peered":"2024-04-25T10:02:06.360459+0000","last_unstale":"2024-04-25T10:02:13.463253+0000","last_undegraded":"2024-04-25T10:02:13.463253+0000","last_fullsized":"2024-04-25T10:02:13.463253+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465848+0000","last_change":"2024-04-25T10:02:11.460123+0000","last_active":"2024-04-25T10:02:13.465848+0000","last_peered":"2024-04-25T10:02:13.465848+0000","last_clean":"2024-04-25T10:02:13.465848+0000","last_became_active":"2024-04-25T10:02:11.459152+0000","last_became_peered":"2024-04-25T10:02:11.459152+0000","last_unstale":"2024-04-25T10:02:13.465848+0000","last_undegraded":"2024-04-25T10:02:13.465848+0000","last_fullsized":"2024-04-25T10:02:13.465848+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:54.154362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465062+0000","last_change":"2024-04-25T10:02:07.388512+0000","last_active":"2024-04-25T10:02:13.465062+0000","last_peered":"2024-04-25T10:02:13.465062+0000","last_clean":"2024-04-25T10:02:13.465062+0000","last_became_active":"2024-04-25T10:02:07.387797+0000","last_became_peered":"2024-04-25T10:02:07.387797+0000","last_unstale":"2024-04-25T10:02:13.465062+0000","last_undegraded":"2024-04-25T10:02:13.465062+0000","last_fullsized":"2024-04-25T10:02:13.465062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:17.622024+0000","last_change":"2024-04-25T10:02:06.370256+0000","last_active":"2024-04-25T10:02:17.622024+0000","last_peered":"2024-04-25T10:02:17.622024+0000","last_clean":"2024-04-25T10:02:17.622024+0000","last_became_active":"2024-04-25T10:02:06.368370+0000","last_became_peered":"2024-04-25T10:02:06.368370+0000","last_unstale":"2024-04-25T10:02:17.622024+0000","last_undegraded":"2024-04-25T10:02:17.622024+0000","last_fullsized":"2024-04-25T10:02:17.622024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:52.000506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462482+0000","last_change":"2024-04-25T10:02:11.438518+0000","last_active":"2024-04-25T10:02:13.462482+0000","last_peered":"2024-04-25T10:02:13.462482+0000","last_clean":"2024-04-25T10:02:13.462482+0000","last_became_active":"2024-04-25T10:02:11.436290+0000","last_became_peered":"2024-04-25T10:02:11.436290+0000","last_unstale":"2024-04-25T10:02:13.462482+0000","last_undegraded":"2024-04-25T10:02:13.462482+0000","last_fullsized":"2024-04-25T10:02:13.462482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:47.057347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407354+0000","last_change":"2024-04-25T10:02:07.358234+0000","last_active":"2024-04-25T10:02:12.407354+0000","last_peered":"2024-04-25T10:02:12.407354+0000","last_clean":"2024-04-25T10:02:12.407354+0000","last_became_active":"2024-04-25T10:02:07.357961+0000","last_became_peered":"2024-04-25T10:02:07.357961+0000","last_unstale":"2024-04-25T10:02:12.407354+0000","last_undegraded":"2024-04-25T10:02:12.407354+0000","last_fullsized":"2024-04-25T10:02:12.407354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461751+0000","last_change":"2024-04-25T10:02:06.386529+0000","last_active":"2024-04-25T10:02:13.461751+0000","last_peered":"2024-04-25T10:02:13.461751+0000","last_clean":"2024-04-25T10:02:13.461751+0000","last_became_active":"2024-04-25T10:02:06.386097+0000","last_became_peered":"2024-04-25T10:02:06.386097+0000","last_unstale":"2024-04-25T10:02:13.461751+0000","last_undegraded":"2024-04-25T10:02:13.461751+0000","last_fullsized":"2024-04-25T10:02:13.461751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407672+0000","last_change":"2024-04-25T10:02:11.437594+0000","last_active":"2024-04-25T10:02:12.407672+0000","last_peered":"2024-04-25T10:02:12.407672+0000","last_clean":"2024-04-25T10:02:12.407672+0000","last_became_active":"2024-04-25T10:02:11.436114+0000","last_became_peered":"2024-04-25T10:02:11.436114+0000","last_unstale":"2024-04-25T10:02:12.407672+0000","last_undegraded":"2024-04-25T10:02:12.407672+0000","last_fullsized":"2024-04-25T10:02:12.407672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:58.568076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462591+0000","last_change":"2024-04-25T10:02:07.371078+0000","last_active":"2024-04-25T10:02:13.462591+0000","last_peered":"2024-04-25T10:02:13.462591+0000","last_clean":"2024-04-25T10:02:13.462591+0000","last_became_active":"2024-04-25T10:02:07.370749+0000","last_became_peered":"2024-04-25T10:02:07.370749+0000","last_unstale":"2024-04-25T10:02:13.462591+0000","last_undegraded":"2024-04-25T10:02:13.462591+0000","last_fullsized":"2024-04-25T10:02:13.462591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466420+0000","last_change":"2024-04-25T10:02:06.377512+0000","last_active":"2024-04-25T10:02:13.466420+0000","last_peered":"2024-04-25T10:02:13.466420+0000","last_clean":"2024-04-25T10:02:13.466420+0000","last_became_active":"2024-04-25T10:02:06.376824+0000","last_became_peered":"2024-04-25T10:02:06.376824+0000","last_unstale":"2024-04-25T10:02:13.466420+0000","last_undegraded":"2024-04-25T10:02:13.466420+0000","last_fullsized":"2024-04-25T10:02:13.466420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403770+0000","last_change":"2024-04-25T10:02:11.442056+0000","last_active":"2024-04-25T10:02:12.403770+0000","last_peered":"2024-04-25T10:02:12.403770+0000","last_clean":"2024-04-25T10:02:12.403770+0000","last_became_active":"2024-04-25T10:02:11.441158+0000","last_became_peered":"2024-04-25T10:02:11.441158+0000","last_unstale":"2024-04-25T10:02:12.403770+0000","last_undegraded":"2024-04-25T10:02:12.403770+0000","last_fullsized":"2024-04-25T10:02:12.403770+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:24.900639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466185+0000","last_change":"2024-04-25T10:02:07.389094+0000","last_active":"2024-04-25T10:02:13.466185+0000","last_peered":"2024-04-25T10:02:13.466185+0000","last_clean":"2024-04-25T10:02:13.466185+0000","last_became_active":"2024-04-25T10:02:07.388187+0000","last_became_peered":"2024-04-25T10:02:07.388187+0000","last_unstale":"2024-04-25T10:02:13.466185+0000","last_undegraded":"2024-04-25T10:02:13.466185+0000","last_fullsized":"2024-04-25T10:02:13.466185+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:45.258295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406550+0000","last_change":"2024-04-25T10:02:06.383181+0000","last_active":"2024-04-25T10:02:12.406550+0000","last_peered":"2024-04-25T10:02:12.406550+0000","last_clean":"2024-04-25T10:02:12.406550+0000","last_became_active":"2024-04-25T10:02:06.382921+0000","last_became_peered":"2024-04-25T10:02:06.382921+0000","last_unstale":"2024-04-25T10:02:12.406550+0000","last_undegraded":"2024-04-25T10:02:12.406550+0000","last_fullsized":"2024-04-25T10:02:12.406550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:16.649893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404862+0000","last_change":"2024-04-25T10:02:11.454031+0000","last_active":"2024-04-25T10:02:12.404862+0000","last_peered":"2024-04-25T10:02:12.404862+0000","last_clean":"2024-04-25T10:02:12.404862+0000","last_became_active":"2024-04-25T10:02:11.453256+0000","last_became_peered":"2024-04-25T10:02:11.453256+0000","last_unstale":"2024-04-25T10:02:12.404862+0000","last_undegraded":"2024-04-25T10:02:12.404862+0000","last_fullsized":"2024-04-25T10:02:12.404862+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:26.875452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462061+0000","last_change":"2024-04-25T10:02:11.460406+0000","last_active":"2024-04-25T10:02:13.462061+0000","last_peered":"2024-04-25T10:02:13.462061+0000","last_clean":"2024-04-25T10:02:13.462061+0000","last_became_active":"2024-04-25T10:02:11.459947+0000","last_became_peered":"2024-04-25T10:02:11.459947+0000","last_unstale":"2024-04-25T10:02:13.462061+0000","last_undegraded":"2024-04-25T10:02:13.462061+0000","last_fullsized":"2024-04-25T10:02:13.462061+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:44.187322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461463+0000","last_change":"2024-04-25T10:02:07.361531+0000","last_active":"2024-04-25T10:02:13.461463+0000","last_peered":"2024-04-25T10:02:13.461463+0000","last_clean":"2024-04-25T10:02:13.461463+0000","last_became_active":"2024-04-25T10:02:07.361213+0000","last_became_peered":"2024-04-25T10:02:07.361213+0000","last_unstale":"2024-04-25T10:02:13.461463+0000","last_undegraded":"2024-04-25T10:02:13.461463+0000","last_fullsized":"2024-04-25T10:02:13.461463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464490+0000","last_change":"2024-04-25T10:02:06.386304+0000","last_active":"2024-04-25T10:02:13.464490+0000","last_peered":"2024-04-25T10:02:13.464490+0000","last_clean":"2024-04-25T10:02:13.464490+0000","last_became_active":"2024-04-25T10:02:06.382588+0000","last_became_peered":"2024-04-25T10:02:06.382588+0000","last_unstale":"2024-04-25T10:02:13.464490+0000","last_undegraded":"2024-04-25T10:02:13.464490+0000","last_fullsized":"2024-04-25T10:02:13.464490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:07.288729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406241+0000","last_change":"2024-04-25T10:02:11.433821+0000","last_active":"2024-04-25T10:02:12.406241+0000","last_peered":"2024-04-25T10:02:12.406241+0000","last_clean":"2024-04-25T10:02:12.406241+0000","last_became_active":"2024-04-25T10:02:11.432467+0000","last_became_peered":"2024-04-25T10:02:11.432467+0000","last_unstale":"2024-04-25T10:02:12.406241+0000","last_undegraded":"2024-04-25T10:02:12.406241+0000","last_fullsized":"2024-04-25T10:02:12.406241+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:49.315692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403810+0000","last_change":"2024-04-25T10:02:07.376574+0000","last_active":"2024-04-25T10:02:12.403810+0000","last_peered":"2024-04-25T10:02:12.403810+0000","last_clean":"2024-04-25T10:02:12.403810+0000","last_became_active":"2024-04-25T10:02:07.376275+0000","last_became_peered":"2024-04-25T10:02:07.376275+0000","last_unstale":"2024-04-25T10:02:12.403810+0000","last_undegraded":"2024-04-25T10:02:12.403810+0000","last_fullsized":"2024-04-25T10:02:12.403810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:52.149321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405129+0000","last_change":"2024-04-25T10:02:06.363949+0000","last_active":"2024-04-25T10:02:12.405129+0000","last_peered":"2024-04-25T10:02:12.405129+0000","last_clean":"2024-04-25T10:02:12.405129+0000","last_became_active":"2024-04-25T10:02:06.363379+0000","last_became_peered":"2024-04-25T10:02:06.363379+0000","last_unstale":"2024-04-25T10:02:12.405129+0000","last_undegraded":"2024-04-25T10:02:12.405129+0000","last_fullsized":"2024-04-25T10:02:12.405129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:22.822632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.405641+0000","last_change":"2024-04-25T10:02:06.361583+0000","last_active":"2024-04-25T10:02:12.405641+0000","last_peered":"2024-04-25T10:02:12.405641+0000","last_clean":"2024-04-25T10:02:12.405641+0000","last_became_active":"2024-04-25T10:02:06.360807+0000","last_became_peered":"2024-04-25T10:02:06.360807+0000","last_unstale":"2024-04-25T10:02:12.405641+0000","last_undegraded":"2024-04-25T10:02:12.405641+0000","last_fullsized":"2024-04-25T10:02:12.405641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:55.200127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463206+0000","last_change":"2024-04-25T10:02:07.367030+0000","last_active":"2024-04-25T10:02:13.463206+0000","last_peered":"2024-04-25T10:02:13.463206+0000","last_clean":"2024-04-25T10:02:13.463206+0000","last_became_active":"2024-04-25T10:02:07.366570+0000","last_became_peered":"2024-04-25T10:02:07.366570+0000","last_unstale":"2024-04-25T10:02:13.463206+0000","last_undegraded":"2024-04-25T10:02:13.463206+0000","last_fullsized":"2024-04-25T10:02:13.463206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463018+0000","last_change":"2024-04-25T10:02:11.434159+0000","last_active":"2024-04-25T10:02:13.463018+0000","last_peered":"2024-04-25T10:02:13.463018+0000","last_clean":"2024-04-25T10:02:13.463018+0000","last_became_active":"2024-04-25T10:02:11.433378+0000","last_became_peered":"2024-04-25T10:02:11.433378+0000","last_unstale":"2024-04-25T10:02:13.463018+0000","last_undegraded":"2024-04-25T10:02:13.463018+0000","last_fullsized":"2024-04-25T10:02:13.463018+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:25.406090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403964+0000","last_change":"2024-04-25T10:02:06.375949+0000","last_active":"2024-04-25T10:02:12.403964+0000","last_peered":"2024-04-25T10:02:12.403964+0000","last_clean":"2024-04-25T10:02:12.403964+0000","last_became_active":"2024-04-25T10:02:06.375558+0000","last_became_peered":"2024-04-25T10:02:06.375558+0000","last_unstale":"2024-04-25T10:02:12.403964+0000","last_undegraded":"2024-04-25T10:02:12.403964+0000","last_fullsized":"2024-04-25T10:02:12.403964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:05.313916+0000","last_clean_scrub_stamp":"2024-04-25T10:02:05.313916+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.141023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406442+0000","last_change":"2024-04-25T10:02:07.352019+0000","last_active":"2024-04-25T10:02:12.406442+0000","last_peered":"2024-04-25T10:02:12.406442+0000","last_clean":"2024-04-25T10:02:12.406442+0000","last_became_active":"2024-04-25T10:02:07.351303+0000","last_became_peered":"2024-04-25T10:02:07.351303+0000","last_unstale":"2024-04-25T10:02:12.406442+0000","last_undegraded":"2024-04-25T10:02:12.406442+0000","last_fullsized":"2024-04-25T10:02:12.406442+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:08.301904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406592+0000","last_change":"2024-04-25T10:02:11.441304+0000","last_active":"2024-04-25T10:02:12.406592+0000","last_peered":"2024-04-25T10:02:12.406592+0000","last_clean":"2024-04-25T10:02:12.406592+0000","last_became_active":"2024-04-25T10:02:11.440847+0000","last_became_peered":"2024-04-25T10:02:11.440847+0000","last_unstale":"2024-04-25T10:02:12.406592+0000","last_undegraded":"2024-04-25T10:02:12.406592+0000","last_fullsized":"2024-04-25T10:02:12.406592+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:10.384488+0000","last_clean_scrub_stamp":"2024-04-25T10:02:10.384488+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:07.657768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403875+0000","last_change":"2024-04-25T10:02:07.372731+0000","last_active":"2024-04-25T10:02:12.403875+0000","last_peered":"2024-04-25T10:02:12.403875+0000","last_clean":"2024-04-25T10:02:12.403875+0000","last_became_active":"2024-04-25T10:02:07.372438+0000","last_became_peered":"2024-04-25T10:02:07.372438+0000","last_unstale":"2024-04-25T10:02:12.403875+0000","last_undegraded":"2024-04-25T10:02:12.403875+0000","last_fullsized":"2024-04-25T10:02:12.403875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462340+0000","last_change":"2024-04-25T10:02:07.364477+0000","last_active":"2024-04-25T10:02:13.462340+0000","last_peered":"2024-04-25T10:02:13.462340+0000","last_clean":"2024-04-25T10:02:13.462340+0000","last_became_active":"2024-04-25T10:02:07.363263+0000","last_became_peered":"2024-04-25T10:02:07.363263+0000","last_unstale":"2024-04-25T10:02:13.462340+0000","last_undegraded":"2024-04-25T10:02:13.462340+0000","last_fullsized":"2024-04-25T10:02:13.462340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:42.118057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465162+0000","last_change":"2024-04-25T10:02:07.366856+0000","last_active":"2024-04-25T10:02:13.465162+0000","last_peered":"2024-04-25T10:02:13.465162+0000","last_clean":"2024-04-25T10:02:13.465162+0000","last_became_active":"2024-04-25T10:02:07.366608+0000","last_became_peered":"2024-04-25T10:02:07.366608+0000","last_unstale":"2024-04-25T10:02:13.465162+0000","last_undegraded":"2024-04-25T10:02:13.465162+0000","last_fullsized":"2024-04-25T10:02:13.465162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463076+0000","last_change":"2024-04-25T10:02:07.374664+0000","last_active":"2024-04-25T10:02:13.463076+0000","last_peered":"2024-04-25T10:02:13.463076+0000","last_clean":"2024-04-25T10:02:13.463076+0000","last_became_active":"2024-04-25T10:02:07.374088+0000","last_became_peered":"2024-04-25T10:02:07.374088+0000","last_unstale":"2024-04-25T10:02:13.463076+0000","last_undegraded":"2024-04-25T10:02:13.463076+0000","last_fullsized":"2024-04-25T10:02:13.463076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404526+0000","last_change":"2024-04-25T10:02:07.365480+0000","last_active":"2024-04-25T10:02:12.404526+0000","last_peered":"2024-04-25T10:02:12.404526+0000","last_clean":"2024-04-25T10:02:12.404526+0000","last_became_active":"2024-04-25T10:02:07.365034+0000","last_became_peered":"2024-04-25T10:02:07.365034+0000","last_unstale":"2024-04-25T10:02:12.404526+0000","last_undegraded":"2024-04-25T10:02:12.404526+0000","last_fullsized":"2024-04-25T10:02:12.404526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461634+0000","last_change":"2024-04-25T10:02:07.362200+0000","last_active":"2024-04-25T10:02:13.461634+0000","last_peered":"2024-04-25T10:02:13.461634+0000","last_clean":"2024-04-25T10:02:13.461634+0000","last_became_active":"2024-04-25T10:02:07.361929+0000","last_became_peered":"2024-04-25T10:02:07.361929+0000","last_unstale":"2024-04-25T10:02:13.461634+0000","last_undegraded":"2024-04-25T10:02:13.461634+0000","last_fullsized":"2024-04-25T10:02:13.461634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407180+0000","last_change":"2024-04-25T10:02:07.375688+0000","last_active":"2024-04-25T10:02:12.407180+0000","last_peered":"2024-04-25T10:02:12.407180+0000","last_clean":"2024-04-25T10:02:12.407180+0000","last_became_active":"2024-04-25T10:02:07.375046+0000","last_became_peered":"2024-04-25T10:02:07.375046+0000","last_unstale":"2024-04-25T10:02:12.407180+0000","last_undegraded":"2024-04-25T10:02:12.407180+0000","last_fullsized":"2024-04-25T10:02:12.407180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:22.412065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406734+0000","last_change":"2024-04-25T10:02:07.365322+0000","last_active":"2024-04-25T10:02:12.406734+0000","last_peered":"2024-04-25T10:02:12.406734+0000","last_clean":"2024-04-25T10:02:12.406734+0000","last_became_active":"2024-04-25T10:02:07.365003+0000","last_became_peered":"2024-04-25T10:02:07.365003+0000","last_unstale":"2024-04-25T10:02:12.406734+0000","last_undegraded":"2024-04-25T10:02:12.406734+0000","last_fullsized":"2024-04-25T10:02:12.406734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464320+0000","last_change":"2024-04-25T10:02:07.374731+0000","last_active":"2024-04-25T10:02:13.464320+0000","last_peered":"2024-04-25T10:02:13.464320+0000","last_clean":"2024-04-25T10:02:13.464320+0000","last_became_active":"2024-04-25T10:02:07.374113+0000","last_became_peered":"2024-04-25T10:02:07.374113+0000","last_unstale":"2024-04-25T10:02:13.464320+0000","last_undegraded":"2024-04-25T10:02:13.464320+0000","last_fullsized":"2024-04-25T10:02:13.464320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.404344+0000","last_change":"2024-04-25T10:02:07.365139+0000","last_active":"2024-04-25T10:02:12.404344+0000","last_peered":"2024-04-25T10:02:12.404344+0000","last_clean":"2024-04-25T10:02:12.404344+0000","last_became_active":"2024-04-25T10:02:07.364860+0000","last_became_peered":"2024-04-25T10:02:07.364860+0000","last_unstale":"2024-04-25T10:02:12.404344+0000","last_undegraded":"2024-04-25T10:02:12.404344+0000","last_fullsized":"2024-04-25T10:02:12.404344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:59.738240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464902+0000","last_change":"2024-04-25T10:02:07.379737+0000","last_active":"2024-04-25T10:02:13.464902+0000","last_peered":"2024-04-25T10:02:13.464902+0000","last_clean":"2024-04-25T10:02:13.464902+0000","last_became_active":"2024-04-25T10:02:07.379465+0000","last_became_peered":"2024-04-25T10:02:07.379465+0000","last_unstale":"2024-04-25T10:02:13.464902+0000","last_undegraded":"2024-04-25T10:02:13.464902+0000","last_fullsized":"2024-04-25T10:02:13.464902+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:49.560618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462820+0000","last_change":"2024-04-25T10:02:07.379170+0000","last_active":"2024-04-25T10:02:13.462820+0000","last_peered":"2024-04-25T10:02:13.462820+0000","last_clean":"2024-04-25T10:02:13.462820+0000","last_became_active":"2024-04-25T10:02:07.378877+0000","last_became_peered":"2024-04-25T10:02:07.378877+0000","last_unstale":"2024-04-25T10:02:13.462820+0000","last_undegraded":"2024-04-25T10:02:13.462820+0000","last_fullsized":"2024-04-25T10:02:13.462820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406740+0000","last_change":"2024-04-25T10:02:07.367522+0000","last_active":"2024-04-25T10:02:12.406740+0000","last_peered":"2024-04-25T10:02:12.406740+0000","last_clean":"2024-04-25T10:02:12.406740+0000","last_became_active":"2024-04-25T10:02:07.367226+0000","last_became_peered":"2024-04-25T10:02:07.367226+0000","last_unstale":"2024-04-25T10:02:12.406740+0000","last_undegraded":"2024-04-25T10:02:12.406740+0000","last_fullsized":"2024-04-25T10:02:12.406740+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465868+0000","last_change":"2024-04-25T10:02:07.378749+0000","last_active":"2024-04-25T10:02:13.465868+0000","last_peered":"2024-04-25T10:02:13.465868+0000","last_clean":"2024-04-25T10:02:13.465868+0000","last_became_active":"2024-04-25T10:02:07.378388+0000","last_became_peered":"2024-04-25T10:02:07.378388+0000","last_unstale":"2024-04-25T10:02:13.465868+0000","last_undegraded":"2024-04-25T10:02:13.465868+0000","last_fullsized":"2024-04-25T10:02:13.465868+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:00.508687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462346+0000","last_change":"2024-04-25T10:02:07.377744+0000","last_active":"2024-04-25T10:02:13.462346+0000","last_peered":"2024-04-25T10:02:13.462346+0000","last_clean":"2024-04-25T10:02:13.462346+0000","last_became_active":"2024-04-25T10:02:07.376844+0000","last_became_peered":"2024-04-25T10:02:07.376844+0000","last_unstale":"2024-04-25T10:02:13.462346+0000","last_undegraded":"2024-04-25T10:02:13.462346+0000","last_fullsized":"2024-04-25T10:02:13.462346+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:35.204352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.465699+0000","last_change":"2024-04-25T10:02:07.388777+0000","last_active":"2024-04-25T10:02:13.465699+0000","last_peered":"2024-04-25T10:02:13.465699+0000","last_clean":"2024-04-25T10:02:13.465699+0000","last_became_active":"2024-04-25T10:02:07.388058+0000","last_became_peered":"2024-04-25T10:02:07.388058+0000","last_unstale":"2024-04-25T10:02:13.465699+0000","last_undegraded":"2024-04-25T10:02:13.465699+0000","last_fullsized":"2024-04-25T10:02:13.465699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:42.999951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464055+0000","last_change":"2024-04-25T10:02:07.377996+0000","last_active":"2024-04-25T10:02:13.464055+0000","last_peered":"2024-04-25T10:02:13.464055+0000","last_clean":"2024-04-25T10:02:13.464055+0000","last_became_active":"2024-04-25T10:02:07.376664+0000","last_became_peered":"2024-04-25T10:02:07.376664+0000","last_unstale":"2024-04-25T10:02:13.464055+0000","last_undegraded":"2024-04-25T10:02:13.464055+0000","last_fullsized":"2024-04-25T10:02:13.464055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:02.156552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.461571+0000","last_change":"2024-04-25T10:02:07.364951+0000","last_active":"2024-04-25T10:02:13.461571+0000","last_peered":"2024-04-25T10:02:13.461571+0000","last_clean":"2024-04-25T10:02:13.461571+0000","last_became_active":"2024-04-25T10:02:07.364138+0000","last_became_peered":"2024-04-25T10:02:07.364138+0000","last_unstale":"2024-04-25T10:02:13.461571+0000","last_undegraded":"2024-04-25T10:02:13.461571+0000","last_fullsized":"2024-04-25T10:02:13.461571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462765+0000","last_change":"2024-04-25T10:02:07.370504+0000","last_active":"2024-04-25T10:02:13.462765+0000","last_peered":"2024-04-25T10:02:13.462765+0000","last_clean":"2024-04-25T10:02:13.462765+0000","last_became_active":"2024-04-25T10:02:07.370157+0000","last_became_peered":"2024-04-25T10:02:07.370157+0000","last_unstale":"2024-04-25T10:02:13.462765+0000","last_undegraded":"2024-04-25T10:02:13.462765+0000","last_fullsized":"2024-04-25T10:02:13.462765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:21.348693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403675+0000","last_change":"2024-04-25T10:02:07.360200+0000","last_active":"2024-04-25T10:02:12.403675+0000","last_peered":"2024-04-25T10:02:12.403675+0000","last_clean":"2024-04-25T10:02:12.403675+0000","last_became_active":"2024-04-25T10:02:07.358078+0000","last_became_peered":"2024-04-25T10:02:07.358078+0000","last_unstale":"2024-04-25T10:02:12.403675+0000","last_undegraded":"2024-04-25T10:02:12.403675+0000","last_fullsized":"2024-04-25T10:02:12.403675+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:02.518203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.407201+0000","last_change":"2024-04-25T10:02:07.376059+0000","last_active":"2024-04-25T10:02:12.407201+0000","last_peered":"2024-04-25T10:02:12.407201+0000","last_clean":"2024-04-25T10:02:12.407201+0000","last_became_active":"2024-04-25T10:02:07.374735+0000","last_became_peered":"2024-04-25T10:02:07.374735+0000","last_unstale":"2024-04-25T10:02:12.407201+0000","last_undegraded":"2024-04-25T10:02:12.407201+0000","last_fullsized":"2024-04-25T10:02:12.407201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:54.666473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406312+0000","last_change":"2024-04-25T10:02:07.367747+0000","last_active":"2024-04-25T10:02:12.406312+0000","last_peered":"2024-04-25T10:02:12.406312+0000","last_clean":"2024-04-25T10:02:12.406312+0000","last_became_active":"2024-04-25T10:02:07.367055+0000","last_became_peered":"2024-04-25T10:02:07.367055+0000","last_unstale":"2024-04-25T10:02:12.406312+0000","last_undegraded":"2024-04-25T10:02:12.406312+0000","last_fullsized":"2024-04-25T10:02:12.406312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.406946+0000","last_change":"2024-04-25T10:02:07.366278+0000","last_active":"2024-04-25T10:02:12.406946+0000","last_peered":"2024-04-25T10:02:12.406946+0000","last_clean":"2024-04-25T10:02:12.406946+0000","last_became_active":"2024-04-25T10:02:07.365591+0000","last_became_peered":"2024-04-25T10:02:07.365591+0000","last_unstale":"2024-04-25T10:02:12.406946+0000","last_undegraded":"2024-04-25T10:02:12.406946+0000","last_fullsized":"2024-04-25T10:02:12.406946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466682+0000","last_change":"2024-04-25T10:02:07.389531+0000","last_active":"2024-04-25T10:02:13.466682+0000","last_peered":"2024-04-25T10:02:13.466682+0000","last_clean":"2024-04-25T10:02:13.466682+0000","last_became_active":"2024-04-25T10:02:07.389091+0000","last_became_peered":"2024-04-25T10:02:07.389091+0000","last_unstale":"2024-04-25T10:02:13.466682+0000","last_undegraded":"2024-04-25T10:02:13.466682+0000","last_fullsized":"2024-04-25T10:02:13.466682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:29.416771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:12.403425+0000","last_change":"2024-04-25T10:02:07.368434+0000","last_active":"2024-04-25T10:02:12.403425+0000","last_peered":"2024-04-25T10:02:12.403425+0000","last_clean":"2024-04-25T10:02:12.403425+0000","last_became_active":"2024-04-25T10:02:07.366907+0000","last_became_peered":"2024-04-25T10:02:07.366907+0000","last_unstale":"2024-04-25T10:02:12.403425+0000","last_undegraded":"2024-04-25T10:02:12.403425+0000","last_fullsized":"2024-04-25T10:02:12.403425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:24.934864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.463789+0000","last_change":"2024-04-25T10:02:07.377948+0000","last_active":"2024-04-25T10:02:13.463789+0000","last_peered":"2024-04-25T10:02:13.463789+0000","last_clean":"2024-04-25T10:02:13.463789+0000","last_became_active":"2024-04-25T10:02:07.377170+0000","last_became_peered":"2024-04-25T10:02:07.377170+0000","last_unstale":"2024-04-25T10:02:13.463789+0000","last_undegraded":"2024-04-25T10:02:13.463789+0000","last_fullsized":"2024-04-25T10:02:13.463789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:30.964959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464659+0000","last_change":"2024-04-25T10:02:07.376985+0000","last_active":"2024-04-25T10:02:13.464659+0000","last_peered":"2024-04-25T10:02:13.464659+0000","last_clean":"2024-04-25T10:02:13.464659+0000","last_became_active":"2024-04-25T10:02:07.376669+0000","last_became_peered":"2024-04-25T10:02:07.376669+0000","last_unstale":"2024-04-25T10:02:13.464659+0000","last_undegraded":"2024-04-25T10:02:13.464659+0000","last_fullsized":"2024-04-25T10:02:13.464659+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:30.790328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464634+0000","last_change":"2024-04-25T10:02:07.377373+0000","last_active":"2024-04-25T10:02:13.464634+0000","last_peered":"2024-04-25T10:02:13.464634+0000","last_clean":"2024-04-25T10:02:13.464634+0000","last_became_active":"2024-04-25T10:02:07.373588+0000","last_became_peered":"2024-04-25T10:02:07.373588+0000","last_unstale":"2024-04-25T10:02:13.464634+0000","last_undegraded":"2024-04-25T10:02:13.464634+0000","last_fullsized":"2024-04-25T10:02:13.464634+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:40.129925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.464847+0000","last_change":"2024-04-25T10:02:07.382540+0000","last_active":"2024-04-25T10:02:13.464847+0000","last_peered":"2024-04-25T10:02:13.464847+0000","last_clean":"2024-04-25T10:02:13.464847+0000","last_became_active":"2024-04-25T10:02:07.381996+0000","last_became_peered":"2024-04-25T10:02:07.381996+0000","last_unstale":"2024-04-25T10:02:13.464847+0000","last_undegraded":"2024-04-25T10:02:13.464847+0000","last_fullsized":"2024-04-25T10:02:13.464847+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:56.513825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462277+0000","last_change":"2024-04-25T10:02:07.376120+0000","last_active":"2024-04-25T10:02:13.462277+0000","last_peered":"2024-04-25T10:02:13.462277+0000","last_clean":"2024-04-25T10:02:13.462277+0000","last_became_active":"2024-04-25T10:02:07.375589+0000","last_became_peered":"2024-04-25T10:02:07.375589+0000","last_unstale":"2024-04-25T10:02:13.462277+0000","last_undegraded":"2024-04-25T10:02:13.462277+0000","last_fullsized":"2024-04-25T10:02:13.462277+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:35.267443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.462177+0000","last_change":"2024-04-25T10:02:07.369060+0000","last_active":"2024-04-25T10:02:13.462177+0000","last_peered":"2024-04-25T10:02:13.462177+0000","last_clean":"2024-04-25T10:02:13.462177+0000","last_became_active":"2024-04-25T10:02:07.368434+0000","last_became_peered":"2024-04-25T10:02:07.368434+0000","last_unstale":"2024-04-25T10:02:13.462177+0000","last_undegraded":"2024-04-25T10:02:13.462177+0000","last_fullsized":"2024-04-25T10:02:13.462177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:39.245655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-25T10:02:13.466451+0000","last_change":"2024-04-25T10:02:07.385523+0000","last_active":"2024-04-25T10:02:13.466451+0000","last_peered":"2024-04-25T10:02:13.466451+0000","last_clean":"2024-04-25T10:02:13.466451+0000","last_became_active":"2024-04-25T10:02:07.385218+0000","last_became_peered":"2024-04-25T10:02:07.385218+0000","last_unstale":"2024-04-25T10:02:13.466451+0000","last_undegraded":"2024-04-25T10:02:13.466451+0000","last_fullsized":"2024-04-25T10:02:13.466451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:06.327091+0000","last_clean_scrub_stamp":"2024-04-25T10:02:06.327091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:54.510679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":864256,"data_stored":695877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":105017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":105053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":108769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":104927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":105087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":872448,"data_stored":695933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:34.392 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:02:34.392 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:02:34.392 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:02:34.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:02:34.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 Processor -- start 2024-04-25T10:02:34.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 -- start start 2024-04-25T10:02:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c0b24c0 0x7fa11c0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa11c0cc030 0x7fa11c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0cff70 con 0x7fa11c15ee10 2024-04-25T10:02:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0cfd40 con 0x7fa11c0cc030 2024-04-25T10:02:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa121583640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c09a480 con 0x7fa11c0b24c0 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa11c0cc030 0x7fa11c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa11c15ee10 0x7fa11c0cf0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58030/0 (socket says 172.21.15.82:58030) 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11affd640 1 -- 172.21.15.82:0/2927798883 learned_addr learned my addr 172.21.15.82:0/2927798883 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11a7fc640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c0b24c0 0x7fa11c0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11b7fe640 1 -- 172.21.15.82:0/2927798883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c0b24c0 msgr2=0x7fa11c0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11b7fe640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c0b24c0 0x7fa11c0cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11b7fe640 1 -- 172.21.15.82:0/2927798883 >> v1:172.21.15.82:6789/0 conn(0x7fa11c15ee10 legacy=0x7fa11c0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.532+0000 7fa11b7fe640 1 -- 172.21.15.82:0/2927798883 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0d0250 con 0x7fa11c0cc030 2024-04-25T10:02:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa11b7fe640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c0cc460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa114009d90 tx=0x7fa11401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e377b5a604195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa119ffb640 1 -- 172.21.15.82:0/2927798883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140b2070 con 0x7fa11c0cc030 2024-04-25T10:02:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa119ffb640 1 -- 172.21.15.82:0/2927798883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1140a5940 con 0x7fa11c0cc030 2024-04-25T10:02:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa119ffb640 1 -- 172.21.15.82:0/2927798883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140a5d10 con 0x7fa11c0cc030 2024-04-25T10:02:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- 172.21.15.82:0/2927798883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 msgr2=0x7fa11c0cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c0cc460 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa114009d90 tx=0x7fa11401c970 comp rx=0 tx=0).stop 2024-04-25T10:02:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- 172.21.15.82:0/2927798883 shutdown_connections 2024-04-25T10:02:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c0cc460 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 --2- 172.21.15.82:0/2927798883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c0b24c0 0x7fa11c0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- 172.21.15.82:0/2927798883 >> 172.21.15.82:0/2927798883 conn(0x7fa11c0b3010 msgr2=0x7fa11c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- 172.21.15.82:0/2927798883 shutdown_connections 2024-04-25T10:02:34.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- 172.21.15.82:0/2927798883 wait complete. 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 Processor -- start 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.536+0000 7fa121583640 1 -- start start 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa11c0b24c0 0x7fa11c15ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c15ee10 0x7fa11c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c6200 con 0x7fa11c0cc030 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0c6080 con 0x7fa11c0b24c0 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c0cd1b0 con 0x7fa11c15ee10 2024-04-25T10:02:34.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49106/0 (socket says 172.21.15.82:49106) 2024-04-25T10:02:34.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 -- 172.21.15.82:0/2243384746 learned_addr learned my addr 172.21.15.82:0/2243384746 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 -- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c15ee10 msgr2=0x7fa11c151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c15ee10 0x7fa11c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 -- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa11c0b24c0 msgr2=0x7fa11c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11affd640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa11c0b24c0 0x7fa11c15ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa11c0b24c0 0x7fa11c15ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 -- 172.21.15.82:0/2243384746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c152870 con 0x7fa11c0cc030 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11a7fc640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7f0 tx=0x7fa11000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110015070 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa110004240 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110003620 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c152b50 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c149420 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c0c85d0 con 0x7fa11c0cc030 2024-04-25T10:02:34.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1100039d0 con 0x7fa11c0cc030 2024-04-25T10:02:34.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 0x7fa108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa110085450 con 0x7fa11c0cc030 2024-04-25T10:02:34.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11affd640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 0x7fa108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.540+0000 7fa11affd640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 0x7fa108069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa11c1665b0 tx=0x7fa10c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.544+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa11c0c85d0 con 0x7fa11c0cc030 2024-04-25T10:02:34.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.720+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa11c04efa0 con 0x7fa11c0cc030 2024-04-25T10:02:34.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa1037fe640 1 -- 172.21.15.82:0/2243384746 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa110056120 con 0x7fa11c0cc030 2024-04-25T10:02:34.730 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:34.730 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:02:34.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 msgr2=0x7fa108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 0x7fa108069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa11c1665b0 tx=0x7fa10c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 msgr2=0x7fa11c148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa11000c7f0 tx=0x7fa11000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 shutdown_connections 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa108066f60 0x7fa108069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa11c15ee10 0x7fa11c151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa11c0cc030 0x7fa11c148ee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 --2- 172.21.15.82:0/2243384746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa11c0b24c0 0x7fa11c15ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 >> 172.21.15.82:0/2243384746 conn(0x7fa11c0b3010 msgr2=0x7fa11c0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 shutdown_connections 2024-04-25T10:02:34.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.724+0000 7fa121583640 1 -- 172.21.15.82:0/2243384746 wait complete. 2024-04-25T10:02:34.747 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:02:34.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 Processor -- start 2024-04-25T10:02:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 -- start start 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ff8159ea0 0x7f2ff80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ff815a280 0x7f2ff815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff815ae30 con 0x7f2ff80718c0 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80b2990 con 0x7f2ff8159ea0 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.856+0000 7f2fff0f5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff815b110 con 0x7f2ff815a280 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ff8159ea0 0x7f2ff80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ff8159ea0 0x7f2ff80b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41868/0 (socket says 172.21.15.82:41868) 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 -- 172.21.15.82:0/1489685419 learned_addr learned my addr 172.21.15.82:0/1489685419 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ff7fff640 1 --2- 172.21.15.82:0/1489685419 >> v2:172.21.15.196:3300/0 conn(0x7f2ff815a280 0x7f2ff815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 -- 172.21.15.82:0/1489685419 >> v2:172.21.15.196:3300/0 conn(0x7f2ff815a280 msgr2=0x7f2ff815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 --2- 172.21.15.82:0/1489685419 >> v2:172.21.15.196:3300/0 conn(0x7f2ff815a280 0x7f2ff815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 -- 172.21.15.82:0/1489685419 >> v1:172.21.15.196:6789/0 conn(0x7f2ff80718c0 legacy=0x7f2ff806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 -- 172.21.15.82:0/1489685419 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80b2620 con 0x7f2ff8159ea0 2024-04-25T10:02:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ffce91640 1 --2- 172.21.15.82:0/1489685419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff8159ea0 0x7f2ff80b2e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80728e0 tx=0x7f2fe801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d669a7960da107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ff77fe640 1 -- 172.21.15.82:0/1489685419 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80a56c0 con 0x7f2ff8159ea0 2024-04-25T10:02:34.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ff77fe640 1 -- 172.21.15.82:0/1489685419 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80a5860 con 0x7f2ff8159ea0 2024-04-25T10:02:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2ff77fe640 1 -- 172.21.15.82:0/1489685419 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe80a5c30 con 0x7f2ff8159ea0 2024-04-25T10:02:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 -- 172.21.15.82:0/1489685419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff8159ea0 msgr2=0x7f2ff80b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/1489685419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff8159ea0 0x7f2ff80b2e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80728e0 tx=0x7f2fe801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 -- 172.21.15.82:0/1489685419 shutdown_connections 2024-04-25T10:02:34.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/1489685419 >> v2:172.21.15.196:3300/0 conn(0x7f2ff815a280 0x7f2ff815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/1489685419 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff8159ea0 0x7f2ff80b2e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 -- 172.21.15.82:0/1489685419 >> 172.21.15.82:0/1489685419 conn(0x7f2ff80b11a0 msgr2=0x7f2ff80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 -- 172.21.15.82:0/1489685419 shutdown_connections 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.860+0000 7f2fff0f5640 1 -- 172.21.15.82:0/1489685419 wait complete. 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 Processor -- start 2024-04-25T10:02:34.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- start start 2024-04-25T10:02:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80718c0 0x7f2ff81487c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff8159ea0 0x7f2ff8147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8168e50 con 0x7f2ff815a280 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8168cd0 con 0x7f2ff8159ea0 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8168fd0 con 0x7f2ff80718c0 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41876/0 (socket says 172.21.15.82:41876) 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 -- 172.21.15.82:0/3133406577 learned_addr learned my addr 172.21.15.82:0/3133406577 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ff7fff640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff8159ea0 0x7f2ff8147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffce91640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80718c0 0x7f2ff81487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 -- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80718c0 msgr2=0x7f2ff81487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80718c0 0x7f2ff81487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 -- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff8159ea0 msgr2=0x7f2ff8147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff8159ea0 0x7f2ff8147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 -- 172.21.15.82:0/3133406577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff8147670 con 0x7f2ff815a280 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ffd692640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c340 tx=0x7f2ff000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0005220 con 0x7f2ff815a280 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff00053c0 con 0x7f2ff815a280 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff0007270 con 0x7f2ff815a280 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff81490c0 con 0x7f2ff815a280 2024-04-25T10:02:34.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff8147a00 con 0x7f2ff815a280 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.864+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff804efa0 con 0x7f2ff815a280 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.868+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ff0019020 con 0x7f2ff815a280 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.868+0000 7f2ff57fa640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 0x7f2fe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.868+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2ff0084520 con 0x7f2ff815a280 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.868+0000 7f2ffce91640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 0x7f2fe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.868+0000 7f2ffce91640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 0x7f2fe4069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80ae000 tx=0x7f2fe80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:34.872+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ff0054ba0 con 0x7f2ff815a280 2024-04-25T10:02:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.016+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ff815bc30 con 0x7f2ff815a280 2024-04-25T10:02:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.020+0000 7f2ff57fa640 1 -- 172.21.15.82:0/3133406577 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f2ff0058c40 con 0x7f2ff815a280 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 msgr2=0x7f2fe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 0x7f2fe4069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80ae000 tx=0x7f2fe80a7070 comp rx=0 tx=0).stop 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 msgr2=0x7f2ff8158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000c340 tx=0x7f2ff000c800 comp rx=0 tx=0).stop 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 shutdown_connections 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2fe4066f60 0x7f2fe4069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ff815a280 0x7f2ff8158540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ff8159ea0 0x7f2ff8147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 --2- 172.21.15.82:0/3133406577 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ff80718c0 0x7f2ff81487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 >> 172.21.15.82:0/3133406577 conn(0x7f2ff80b11a0 msgr2=0x7f2ff815beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 shutdown_connections 2024-04-25T10:02:35.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.024+0000 7f2fff0f5640 1 -- 172.21.15.82:0/3133406577 wait complete. 2024-04-25T10:02:35.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:35.046 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:35.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 Processor -- start 2024-04-25T10:02:35.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 -- start start 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d6415ecb0 0x7f3d640c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640ca4c0 con 0x7f3d640cf590 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d640cd690 con 0x7f3d640c9ef0 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6c306640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d640ca780 con 0x7f3d6415ecb0 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6a0a2640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3d640cf590 0x7f3d640cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46754/0 (socket says 172.21.15.82:46754) 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6a0a2640 1 -- 172.21.15.82:0/632989972 learned_addr learned my addr 172.21.15.82:0/632989972 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:35.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684287870 0 0) 0x7f3d640ca4c0 con 0x7f3d640cf590 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> v2:172.21.15.196:3300/0 conn(0x7f3d6415ecb0 0x7f3d640c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d4c005740 con 0x7f3d640cf590 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2052943381 0 0) 0x7f3d4c005740 con 0x7f3d640cf590 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 >> v2:172.21.15.196:3300/0 conn(0x7f3d6415ecb0 msgr2=0x7f3d640c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 --2- 172.21.15.82:0/632989972 >> v2:172.21.15.196:3300/0 conn(0x7f3d6415ecb0 0x7f3d640c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 >> v1:172.21.15.196:6789/0 conn(0x7f3d640c9ef0 legacy=0x7f3d640ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d640cd970 con 0x7f3d640cf590 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3d54001680 con 0x7f3d640cf590 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 >> v1:172.21.15.82:6789/0 conn(0x7f3d640cf590 legacy=0x7f3d640cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d4c005c40 0x7f3d4c006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 0x7f3d4c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c007450 0x7f3d4c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c00e380 con 0x7f3d4c006a20 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c00e530 con 0x7f3d4c007450 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d4c00e730 con 0x7f3d4c005c40 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> v2:172.21.15.196:3300/0 conn(0x7f3d6415ecb0 0x7f3d640c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 0x7f3d4c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6a0a2640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d4c005c40 0x7f3d4c006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6a8a3640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c007450 0x7f3d4c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 -- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d4c005c40 msgr2=0x7f3d4c006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d4c005c40 0x7f3d4c006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 -- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c007450 msgr2=0x7f3d4c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c007450 0x7f3d4c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d6a8a3640 1 -- 172.21.15.82:0/632989972 reap_dead start 2024-04-25T10:02:35.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.156+0000 7f3d698a1640 1 -- 172.21.15.82:0/632989972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c005740 con 0x7f3d4c006a20 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d698a1640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 0x7f3d4c006ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d580073e0 tx=0x7f3d5801cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830a7fbcc068b3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d580b2070 con 0x7f3d4c006a20 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d580a5cb0 con 0x7f3d4c006a20 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d690a0640 1 -- 172.21.15.82:0/632989972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d58009c70 con 0x7f3d4c006a20 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 msgr2=0x7f3d4c006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 0x7f3d4c006ea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3d580073e0 tx=0x7f3d5801cec0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- 172.21.15.82:0/632989972 shutdown_connections 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 --2- 172.21.15.82:0/632989972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d4c006a20 0x7f3d4c006ea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- 172.21.15.82:0/632989972 >> 172.21.15.82:0/632989972 conn(0x7f3d6406ae60 msgr2=0x7f3d6406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- 172.21.15.82:0/632989972 shutdown_connections 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- 172.21.15.82:0/632989972 wait complete. 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 Processor -- start 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- start start 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c006a20 0x7f3d640718d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d64155de0 0x7f3d641555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cd440 con 0x7f3d641e47e0 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6415f0f0 con 0x7f3d4c006a20 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6c306640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d640cdce0 con 0x7f3d64155de0 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6a8a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6a8a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41890/0 (socket says 172.21.15.82:41890) 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.160+0000 7f3d6a8a3640 1 -- 172.21.15.82:0/2489532937 learned_addr learned my addr 172.21.15.82:0/2489532937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d698a1640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d64155de0 0x7f3d641555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 -- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d64155de0 msgr2=0x7f3d641555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d64155de0 0x7f3d641555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 -- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c006a20 msgr2=0x7f3d640718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c006a20 0x7f3d640718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 -- 172.21.15.82:0/2489532937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64071e10 con 0x7f3d641e47e0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a8a3640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d640c2de0 tx=0x7f3d5c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c019070 con 0x7f3d641e47e0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c0050d0 con 0x7f3d641e47e0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d640720e0 con 0x7f3d641e47e0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c0054a0 con 0x7f3d641e47e0 2024-04-25T10:02:35.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d640726a0 con 0x7f3d641e47e0 2024-04-25T10:02:35.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6404efa0 con 0x7f3d641e47e0 2024-04-25T10:02:35.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d5c01c070 con 0x7f3d641e47e0 2024-04-25T10:02:35.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 0x7f3d50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d5c084990 con 0x7f3d641e47e0 2024-04-25T10:02:35.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a0a2640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 0x7f3d50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.164+0000 7f3d6a0a2640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 0x7f3d50069450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001060 tx=0x7f3d54000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.168+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d5c023020 con 0x7f3d641e47e0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d640732c0 con 0x7f3d641e47e0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d62ffd640 1 -- 172.21.15.82:0/2489532937 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d5c0556d0 con 0x7f3d641e47e0 2024-04-25T10:02:35.322 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 msgr2=0x7f3d50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 0x7f3d50069450 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3d54001060 tx=0x7f3d54000ff0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 msgr2=0x7f3d641e4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d640c2de0 tx=0x7f3d5c007f90 comp rx=0 tx=0).stop 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 shutdown_connections 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d50066f90 0x7f3d50069450 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d64155de0 0x7f3d641555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d641e47e0 0x7f3d641e4c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 --2- 172.21.15.82:0/2489532937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d4c006a20 0x7f3d640718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 >> 172.21.15.82:0/2489532937 conn(0x7f3d6406ae60 msgr2=0x7f3d640cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 shutdown_connections 2024-04-25T10:02:35.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.316+0000 7f3d6c306640 1 -- 172.21.15.82:0/2489532937 wait complete. 2024-04-25T10:02:35.339 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:35.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 Processor -- start 2024-04-25T10:02:35.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 -- start start 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81a015a750 0x7f81a006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a015ab30 con 0x7f81a0066bb0 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a00cf560 con 0x7f81a006c3b0 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a7839640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a0067100 con 0x7f81a015a750 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a55d5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f81a015a750 0x7f81a006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a4dd4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f81a0066bb0 0x7f81a006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46756/0 (socket says 172.21.15.82:46756) 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a4dd4640 1 -- 172.21.15.82:0/491264400 learned_addr learned my addr 172.21.15.82:0/491264400 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a55d5640 1 -- 172.21.15.82:0/491264400 >> v1:172.21.15.196:6789/0 conn(0x7f81a006c3b0 legacy=0x7f81a00b2c30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:35.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a55d5640 1 -- 172.21.15.82:0/491264400 >> v1:172.21.15.82:6789/0 conn(0x7f81a0066bb0 legacy=0x7f81a006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.452+0000 7f81a55d5640 1 -- 172.21.15.82:0/491264400 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00cf840 con 0x7f81a015a750 2024-04-25T10:02:35.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a55d5640 1 --2- 172.21.15.82:0/491264400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a015a750 0x7f81a006ee60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f819c0097c0 tx=0x7f819c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87b383b41672c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f8197fff640 1 -- 172.21.15.82:0/491264400 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0b0070 con 0x7f81a015a750 2024-04-25T10:02:35.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f8197fff640 1 -- 172.21.15.82:0/491264400 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c0ab400 con 0x7f81a015a750 2024-04-25T10:02:35.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f8197fff640 1 -- 172.21.15.82:0/491264400 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0ab7d0 con 0x7f81a015a750 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 -- 172.21.15.82:0/491264400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a015a750 msgr2=0x7f81a006ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 --2- 172.21.15.82:0/491264400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a015a750 0x7f81a006ee60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f819c0097c0 tx=0x7f819c01c340 comp rx=0 tx=0).stop 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 -- 172.21.15.82:0/491264400 shutdown_connections 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 --2- 172.21.15.82:0/491264400 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a015a750 0x7f81a006ee60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 -- 172.21.15.82:0/491264400 >> 172.21.15.82:0/491264400 conn(0x7f81a0071810 msgr2=0x7f81a0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 -- 172.21.15.82:0/491264400 shutdown_connections 2024-04-25T10:02:35.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.456+0000 7f81a7839640 1 -- 172.21.15.82:0/491264400 wait complete. 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 Processor -- start 2024-04-25T10:02:35.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- start start 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0066bb0 0x7f81a00b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81a015a750 0x7f81a0148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a01687c0 con 0x7f81a006c3b0 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00ceda0 con 0x7f81a0066bb0 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a0168940 con 0x7f81a015a750 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41894/0 (socket says 172.21.15.82:41894) 2024-04-25T10:02:35.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 -- 172.21.15.82:0/3626877531 learned_addr learned my addr 172.21.15.82:0/3626877531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a55d5640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0066bb0 0x7f81a00b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a5dd6640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81a015a750 0x7f81a0148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 -- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81a015a750 msgr2=0x7f81a0148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81a015a750 0x7f81a0148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 -- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0066bb0 msgr2=0x7f81a00b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0066bb0 0x7f81a00b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 -- 172.21.15.82:0/3626877531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a014ed20 con 0x7f81a006c3b0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a4dd4640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f819000c340 tx=0x7f819000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190017070 con 0x7f81a006c3b0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8190005d10 con 0x7f81a006c3b0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a014efc0 con 0x7f81a006c3b0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190007310 con 0x7f81a006c3b0 2024-04-25T10:02:35.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a014f580 con 0x7f81a006c3b0 2024-04-25T10:02:35.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a004efa0 con 0x7f81a006c3b0 2024-04-25T10:02:35.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f819001a070 con 0x7f81a006c3b0 2024-04-25T10:02:35.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.460+0000 7f8195ffb640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 0x7f8184069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.464+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8190084600 con 0x7f81a006c3b0 2024-04-25T10:02:35.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.464+0000 7f81a55d5640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 0x7f8184069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.464+0000 7f81a55d5640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 0x7f8184069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f819c0ad000 tx=0x7f819c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.468+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8190055360 con 0x7f81a006c3b0 2024-04-25T10:02:35.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.620+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81a00b0840 con 0x7f81a006c3b0 2024-04-25T10:02:35.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.620+0000 7f8195ffb640 1 -- 172.21.15.82:0/3626877531 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8190054c60 con 0x7f81a006c3b0 2024-04-25T10:02:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 msgr2=0x7f8184069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 0x7f8184069450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f819c0ad000 tx=0x7f819c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 msgr2=0x7f81a0148310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f819000c340 tx=0x7f819000c800 comp rx=0 tx=0).stop 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 shutdown_connections 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8184066f90 0x7f8184069450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81a015a750 0x7f81a0148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81a006c3b0 0x7f81a0148310 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 --2- 172.21.15.82:0/3626877531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f81a0066bb0 0x7f81a00b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 >> 172.21.15.82:0/3626877531 conn(0x7f81a0071810 msgr2=0x7f81a00b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 shutdown_connections 2024-04-25T10:02:35.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.624+0000 7f81a7839640 1 -- 172.21.15.82:0/3626877531 wait complete. 2024-04-25T10:02:35.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:35.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 Processor -- start 2024-04-25T10:02:35.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 -- start start 2024-04-25T10:02:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33fc0cf970 0x7f33fc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 0x7f33fc0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc0ca780 con 0x7f33fc0cf970 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33fc0cd440 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3403764640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0cdce0 con 0x7f33fc0cf590 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33fc0cf970 0x7f33fc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33fc0cf970 0x7f33fc0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41904/0 (socket says 172.21.15.82:41904) 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 -- 172.21.15.82:0/1406916514 learned_addr learned my addr 172.21.15.82:0/1406916514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 -- 172.21.15.82:0/1406916514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 msgr2=0x7f33fc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3401d01640 1 --2- 172.21.15.82:0/1406916514 >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 0x7f33fc0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 --2- 172.21.15.82:0/1406916514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 -- 172.21.15.82:0/1406916514 >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 msgr2=0x7f33fc0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 --2- 172.21.15.82:0/1406916514 >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 0x7f33fc0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3400cff640 1 -- 172.21.15.82:0/1406916514 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc0ca4c0 con 0x7f33fc0cf970 2024-04-25T10:02:35.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.752+0000 7f3401d01640 1 --2- 172.21.15.82:0/1406916514 >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 0x7f33fc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3400cff640 1 --2- 172.21.15.82:0/1406916514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0cf970 0x7f33fc0c99b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0cf310 tx=0x7f33f801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294159b4ddcc3af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f33f3fff640 1 -- 172.21.15.82:0/1406916514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80b0070 con 0x7f33fc0cf970 2024-04-25T10:02:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f33f3fff640 1 -- 172.21.15.82:0/1406916514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f80ab600 con 0x7f33fc0cf970 2024-04-25T10:02:35.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f33f3fff640 1 -- 172.21.15.82:0/1406916514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f80ab9d0 con 0x7f33fc0cf970 2024-04-25T10:02:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- 172.21.15.82:0/1406916514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0cf970 msgr2=0x7f33fc0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 --2- 172.21.15.82:0/1406916514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0cf970 0x7f33fc0c99b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33fc0cf310 tx=0x7f33f801c230 comp rx=0 tx=0).stop 2024-04-25T10:02:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- 172.21.15.82:0/1406916514 shutdown_connections 2024-04-25T10:02:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 --2- 172.21.15.82:0/1406916514 >> v2:172.21.15.196:3300/0 conn(0x7f33fc0c9ef0 0x7f33fc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 --2- 172.21.15.82:0/1406916514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0cf970 0x7f33fc0c99b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 --2- 172.21.15.82:0/1406916514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- 172.21.15.82:0/1406916514 >> 172.21.15.82:0/1406916514 conn(0x7f33fc06ae60 msgr2=0x7f33fc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- 172.21.15.82:0/1406916514 shutdown_connections 2024-04-25T10:02:35.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- 172.21.15.82:0/1406916514 wait complete. 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 Processor -- start 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.756+0000 7f3403764640 1 -- start start 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc0c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33fc0cf970 0x7f33fc15c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0ac2f0 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0ac170 con 0x7f33fc0cf970 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc0ce170 con 0x7f33fc0cf590 2024-04-25T10:02:35.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41912/0 (socket says 172.21.15.82:41912) 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 -- 172.21.15.82:0/2853685900 learned_addr learned my addr 172.21.15.82:0/2853685900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401d01640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33fc0cf970 0x7f33fc15c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3400cff640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc0c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 -- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 msgr2=0x7f33fc0c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc0c3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 -- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33fc0cf970 msgr2=0x7f33fc15c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33fc0cf970 0x7f33fc15c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 -- 172.21.15.82:0/2853685900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc1f5880 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3401500640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b550 tx=0x7f33ec00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec014070 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec010890 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc1f5b60 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec010c80 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc1f6120 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc04efa0 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33ec017070 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 0x7f33e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f33ec0850b0 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3400cff640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 0x7f33e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.760+0000 7f3400cff640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 0x7f33e0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33f80ad000 tx=0x7f33f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.764+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33ec055e10 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.912+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f33fc0b4170 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f33f1ffb640 1 -- 172.21.15.82:0/2853685900 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f33ec010e20 con 0x7f33fc0c9ef0 2024-04-25T10:02:35.922 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 msgr2=0x7f33e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 0x7f33e0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f33f80ad000 tx=0x7f33f8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 msgr2=0x7f33fc098ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f33ec00b550 tx=0x7f33ec00ba10 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 shutdown_connections 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33e0066f60 0x7f33e0069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33fc0cf590 0x7f33fc0c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33fc0cf970 0x7f33fc15c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 --2- 172.21.15.82:0/2853685900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33fc0c9ef0 0x7f33fc098ab0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 >> 172.21.15.82:0/2853685900 conn(0x7f33fc06ae60 msgr2=0x7f33fc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 shutdown_connections 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:35.916+0000 7f3403764640 1 -- 172.21.15.82:0/2853685900 wait complete. 2024-04-25T10:02:35.940 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:36.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 Processor -- start 2024-04-25T10:02:36.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 -- start start 2024-04-25T10:02:36.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5968159bd0 0x7f59680c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f59680b28c0 0x7f5968163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59680c9ef0 con 0x7f5968159fb0 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5968072780 con 0x7f5968159bd0 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f596db72640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5968071b70 con 0x7f59680b28c0 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5968159bd0 0x7f59680c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5968159bd0 0x7f59680c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41926/0 (socket says 172.21.15.82:41926) 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 -- 172.21.15.82:0/2283849671 learned_addr learned my addr 172.21.15.82:0/2283849671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 -- 172.21.15.82:0/2283849671 >> v2:172.21.15.196:3300/0 conn(0x7f59680b28c0 msgr2=0x7f5968163960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 --2- 172.21.15.82:0/2283849671 >> v2:172.21.15.196:3300/0 conn(0x7f59680b28c0 0x7f5968163960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 -- 172.21.15.82:0/2283849671 >> v1:172.21.15.196:6789/0 conn(0x7f5968159fb0 legacy=0x7f59680b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 -- 172.21.15.82:0/2283849671 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5968071e50 con 0x7f5968159bd0 2024-04-25T10:02:36.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.044+0000 7f59677fe640 1 --2- 172.21.15.82:0/2283849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159bd0 0x7f59680c99b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5968072bb0 tx=0x7f595c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7a972aaacb24c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f59667fc640 1 -- 172.21.15.82:0/2283849671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0ab650 con 0x7f5968159bd0 2024-04-25T10:02:36.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f59667fc640 1 -- 172.21.15.82:0/2283849671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c0b0070 con 0x7f5968159bd0 2024-04-25T10:02:36.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f59667fc640 1 -- 172.21.15.82:0/2283849671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0aba10 con 0x7f5968159bd0 2024-04-25T10:02:36.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- 172.21.15.82:0/2283849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159bd0 msgr2=0x7f59680c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- 172.21.15.82:0/2283849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159bd0 0x7f59680c99b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5968072bb0 tx=0x7f595c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- 172.21.15.82:0/2283849671 shutdown_connections 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- 172.21.15.82:0/2283849671 >> v2:172.21.15.196:3300/0 conn(0x7f59680b28c0 0x7f5968163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- 172.21.15.82:0/2283849671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159bd0 0x7f59680c99b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- 172.21.15.82:0/2283849671 >> 172.21.15.82:0/2283849671 conn(0x7f596806ae60 msgr2=0x7f596806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- 172.21.15.82:0/2283849671 shutdown_connections 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- 172.21.15.82:0/2283849671 wait complete. 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 Processor -- start 2024-04-25T10:02:36.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- start start 2024-04-25T10:02:36.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59680b28c0 0x7f59680c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5968159bd0 0x7f5968155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968164020 con 0x7f5968159fb0 2024-04-25T10:02:36.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5968163ea0 con 0x7f59680b28c0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f596db72640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59681641a0 con 0x7f5968159bd0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41932/0 (socket says 172.21.15.82:41932) 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 -- 172.21.15.82:0/358406009 learned_addr learned my addr 172.21.15.82:0/358406009 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 -- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5968159bd0 msgr2=0x7f5968155b90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5968159bd0 0x7f5968155b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f59677fe640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59680b28c0 0x7f59680c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 -- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59680b28c0 msgr2=0x7f59680c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59680b28c0 0x7f59680c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 -- 172.21.15.82:0/358406009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680c84f0 con 0x7f5968159fb0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f5967fff640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f596000b9c0 tx=0x7f596000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960019070 con 0x7f5968159fb0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596000cb40 con 0x7f5968159fb0 2024-04-25T10:02:36.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.048+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5960014440 con 0x7f5968159fb0 2024-04-25T10:02:36.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59680ad770 con 0x7f5968159fb0 2024-04-25T10:02:36.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59680c7c50 con 0x7f5968159fb0 2024-04-25T10:02:36.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596804efa0 con 0x7f5968159fb0 2024-04-25T10:02:36.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f596001c070 con 0x7f5968159fb0 2024-04-25T10:02:36.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f594ffff640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f59677fe640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5960084fd0 con 0x7f5968159fb0 2024-04-25T10:02:36.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.052+0000 7f59677fe640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 0x7f5954069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f595c0af000 tx=0x7f595c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.056+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5960055d30 con 0x7f5968159fb0 2024-04-25T10:02:36.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:36.102 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f59680b24c0 con 0x7f5968159fb0 2024-04-25T10:02:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f594ffff640 1 -- 172.21.15.82:0/358406009 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5960055630 con 0x7f5968159fb0 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 msgr2=0x7f5954069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 0x7f5954069420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f595c0af000 tx=0x7f595c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 msgr2=0x7f59680cdae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f596000b9c0 tx=0x7f596000be80 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 shutdown_connections 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5954066f60 0x7f5954069420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5968159bd0 0x7f5968155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5968159fb0 0x7f59680cdae0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 --2- 172.21.15.82:0/358406009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f59680b28c0 0x7f59680c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 >> 172.21.15.82:0/358406009 conn(0x7f596806ae60 msgr2=0x7f59680af570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 shutdown_connections 2024-04-25T10:02:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.204+0000 7f596db72640 1 -- 172.21.15.82:0/358406009 wait complete. 2024-04-25T10:02:36.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 Processor -- start 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 -- start start 2024-04-25T10:02:36.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46201451f0 0x7f4620142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46200ccc70 con 0x7f4620144ca0 2024-04-25T10:02:36.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f462006dc40 con 0x7f46200ce510 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4624f12640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46200cc2b0 con 0x7f46201451f0 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46201451f0 0x7f4620142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4615d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4620144ca0 0x7f462006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46768/0 (socket says 172.21.15.82:46768) 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46201451f0 0x7f4620142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41938/0 (socket says 172.21.15.82:41938) 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f4615d9b640 1 -- 172.21.15.82:0/2503237878 learned_addr learned my addr 172.21.15.82:0/2503237878 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461ed9d640 1 -- 172.21.15.82:0/2503237878 >> v1:172.21.15.196:6789/0 conn(0x7f46200ce510 legacy=0x7f46200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461ed9d640 1 -- 172.21.15.82:0/2503237878 >> v1:172.21.15.82:6789/0 conn(0x7f4620144ca0 legacy=0x7f462006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461ed9d640 1 -- 172.21.15.82:0/2503237878 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462006df20 con 0x7f46201451f0 2024-04-25T10:02:36.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.340+0000 7f461dd9b640 1 -- 172.21.15.82:0/2503237878 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459306890 0 0) 0x7f46200ccc70 con 0x7f4620144ca0 2024-04-25T10:02:36.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f461ed9d640 1 --2- 172.21.15.82:0/2503237878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46201451f0 0x7f4620142680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4618009dc0 tx=0x7f461801f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70567b283a06cd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f461dd9b640 1 -- 172.21.15.82:0/2503237878 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180b4070 con 0x7f46201451f0 2024-04-25T10:02:36.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f461dd9b640 1 -- 172.21.15.82:0/2503237878 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46180af600 con 0x7f46201451f0 2024-04-25T10:02:36.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f461dd9b640 1 -- 172.21.15.82:0/2503237878 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46180af9d0 con 0x7f46201451f0 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 -- 172.21.15.82:0/2503237878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46201451f0 msgr2=0x7f4620142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 --2- 172.21.15.82:0/2503237878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46201451f0 0x7f4620142680 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4618009dc0 tx=0x7f461801f100 comp rx=0 tx=0).stop 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 -- 172.21.15.82:0/2503237878 shutdown_connections 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 --2- 172.21.15.82:0/2503237878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46201451f0 0x7f4620142680 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 -- 172.21.15.82:0/2503237878 >> 172.21.15.82:0/2503237878 conn(0x7f46200b11a0 msgr2=0x7f46200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 -- 172.21.15.82:0/2503237878 shutdown_connections 2024-04-25T10:02:36.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.344+0000 7f4624f12640 1 -- 172.21.15.82:0/2503237878 wait complete. 2024-04-25T10:02:36.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 Processor -- start 2024-04-25T10:02:36.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- start start 2024-04-25T10:02:36.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46200ce510 0x7f46201f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46201451f0 0x7f462013f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46200cc900 con 0x7f4620144ca0 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462006e290 con 0x7f46200ce510 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4620142bc0 con 0x7f46201451f0 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41944/0 (socket says 172.21.15.82:41944) 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 -- 172.21.15.82:0/3985841605 learned_addr learned my addr 172.21.15.82:0/3985841605 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 -- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46201451f0 msgr2=0x7f462013f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f461e59c640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46200ce510 0x7f46201f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46201451f0 0x7f462013f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 -- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46200ce510 msgr2=0x7f46201f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46200ce510 0x7f46201f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 -- 172.21.15.82:0/3985841605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46201e1690 con 0x7f4620144ca0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4615d9b640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f460800c480 tx=0x7f460800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608018070 con 0x7f4620144ca0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46080074e0 con 0x7f4620144ca0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46201e1970 con 0x7f4620144ca0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608004a30 con 0x7f4620144ca0 2024-04-25T10:02:36.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4620140280 con 0x7f4620144ca0 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462004efa0 con 0x7f4620144ca0 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f460801b070 con 0x7f4620144ca0 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 0x7f4600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46080846e0 con 0x7f4620144ca0 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.348+0000 7f461e59c640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 0x7f4600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.352+0000 7f461e59c640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 0x7f4600069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4610003d60 tx=0x7f4610009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.352+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4608055440 con 0x7f4620144ca0 2024-04-25T10:02:36.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.544+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46201424d0 con 0x7f4620144ca0 2024-04-25T10:02:36.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f46177fe640 1 -- 172.21.15.82:0/3985841605 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f4608054d40 con 0x7f4620144ca0 2024-04-25T10:02:36.554 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:36.554 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:36.555 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 msgr2=0x7f4600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 0x7f4600069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4610003d60 tx=0x7f4610009040 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 msgr2=0x7f46200c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f460800c480 tx=0x7f460800c940 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 shutdown_connections 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4600066f60 0x7f4600069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46201451f0 0x7f462013f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4620144ca0 0x7f46200c8640 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 --2- 172.21.15.82:0/3985841605 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46200ce510 0x7f46201f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 >> 172.21.15.82:0/3985841605 conn(0x7f46200b11a0 msgr2=0x7f4620071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 shutdown_connections 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.548+0000 7f4624f12640 1 -- 172.21.15.82:0/3985841605 wait complete. 2024-04-25T10:02:36.573 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.684+0000 7f62181a0640 1 Processor -- start 2024-04-25T10:02:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.684+0000 7f62181a0640 1 -- start start 2024-04-25T10:02:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f62181a0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f621015da10 0x7f6210071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f62181a0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f62181a0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62100adbd0 con 0x7f6210071e90 2024-04-25T10:02:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f62181a0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62100ae860 con 0x7f621015da10 2024-04-25T10:02:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f62181a0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621015e210 con 0x7f62100b24a0 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:50794/0 (socket says 172.21.15.82:50794) 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 -- 172.21.15.82:0/3109002823 learned_addr learned my addr 172.21.15.82:0/3109002823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f6215f3c640 1 --2- 172.21.15.82:0/3109002823 >> v2:172.21.15.196:3300/0 conn(0x7f621015da10 0x7f6210071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 -- 172.21.15.82:0/3109002823 >> v2:172.21.15.196:3300/0 conn(0x7f621015da10 msgr2=0x7f6210071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 --2- 172.21.15.82:0/3109002823 >> v2:172.21.15.196:3300/0 conn(0x7f621015da10 0x7f6210071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 -- 172.21.15.82:0/3109002823 >> v1:172.21.15.196:6789/0 conn(0x7f6210071e90 legacy=0x7f62101614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.688+0000 7f621573b640 1 -- 172.21.15.82:0/3109002823 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621015e4f0 con 0x7f62100b24a0 2024-04-25T10:02:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f621573b640 1 --2- 172.21.15.82:0/3109002823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f620c009870 tx=0x7f620c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5417cacc7ef4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6214f3a640 1 -- 172.21.15.82:0/3109002823 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0b0070 con 0x7f62100b24a0 2024-04-25T10:02:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6214f3a640 1 -- 172.21.15.82:0/3109002823 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620c0ab600 con 0x7f62100b24a0 2024-04-25T10:02:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6214f3a640 1 -- 172.21.15.82:0/3109002823 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620c0ab9d0 con 0x7f62100b24a0 2024-04-25T10:02:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- 172.21.15.82:0/3109002823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 msgr2=0x7f6210071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- 172.21.15.82:0/3109002823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f620c009870 tx=0x7f620c01c440 comp rx=0 tx=0).stop 2024-04-25T10:02:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- 172.21.15.82:0/3109002823 shutdown_connections 2024-04-25T10:02:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- 172.21.15.82:0/3109002823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62100b24a0 0x7f6210071570 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- 172.21.15.82:0/3109002823 >> v2:172.21.15.196:3300/0 conn(0x7f621015da10 0x7f6210071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- 172.21.15.82:0/3109002823 >> 172.21.15.82:0/3109002823 conn(0x7f62100b11a0 msgr2=0x7f62100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- 172.21.15.82:0/3109002823 shutdown_connections 2024-04-25T10:02:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- 172.21.15.82:0/3109002823 wait complete. 2024-04-25T10:02:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 Processor -- start 2024-04-25T10:02:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- start start 2024-04-25T10:02:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62100b24a0 0x7f621014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 0x7f621014eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621016aa10 con 0x7f6210071e90 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621016a890 con 0x7f62100b24a0 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f62181a0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621016ab90 con 0x7f621015da10 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6215f3c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6215f3c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41948/0 (socket says 172.21.15.82:41948) 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f6215f3c640 1 -- 172.21.15.82:0/3101187639 learned_addr learned my addr 172.21.15.82:0/3101187639 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.692+0000 7f621573b640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62100b24a0 0x7f621014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f621673d640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 0x7f621014eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 -- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 msgr2=0x7f621014eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 0x7f621014eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 -- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62100b24a0 msgr2=0x7f621014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62100b24a0 0x7f621014e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 -- 172.21.15.82:0/3101187639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621015bcb0 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f621673d640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 0x7f621014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f6215f3c640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f620000e870 tx=0x7f620000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6200017070 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f620000c940 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621015bf90 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f620000ccf0 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621015c550 con 0x7f6210071e90 2024-04-25T10:02:36.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621004efa0 con 0x7f6210071e90 2024-04-25T10:02:36.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f620001a070 con 0x7f6210071e90 2024-04-25T10:02:36.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 0x7f61f4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6200085940 con 0x7f6210071e90 2024-04-25T10:02:36.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f621573b640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 0x7f61f4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.696+0000 7f621573b640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 0x7f61f4069450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f620c0ad000 tx=0x7f620c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.700+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62000566f0 con 0x7f6210071e90 2024-04-25T10:02:36.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.860+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62101619c0 con 0x7f6210071e90 2024-04-25T10:02:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.860+0000 7f62067fc640 1 -- 172.21.15.82:0/3101187639 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f6200055ff0 con 0x7f6210071e90 2024-04-25T10:02:36.869 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:36.869 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 msgr2=0x7f61f4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 0x7f61f4069450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f620c0ad000 tx=0x7f620c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 msgr2=0x7f621014dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f620000e870 tx=0x7f620000ec70 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 shutdown_connections 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61f4066f90 0x7f61f4069450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f621015da10 0x7f621014eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62100b24a0 0x7f621014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 --2- 172.21.15.82:0/3101187639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6210071e90 0x7f621014dde0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 >> 172.21.15.82:0/3101187639 conn(0x7f62100b11a0 msgr2=0x7f62100b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 shutdown_connections 2024-04-25T10:02:36.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:36.864+0000 7f62181a0640 1 -- 172.21.15.82:0/3101187639 wait complete. 2024-04-25T10:02:36.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:36.889 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 Processor -- start 2024-04-25T10:02:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 -- start start 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8df406b440 0x7f8df40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df40b1ac0 0x7f8df40cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df40afb20 con 0x7f8df406b820 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df40b06c0 con 0x7f8df406b440 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8dfa933640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40afd90 con 0x7f8df40b1ac0 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8df406b820 0x7f8df40b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46774/0 (socket says 172.21.15.82:46774) 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df37fe640 1 -- 172.21.15.82:0/3562973958 learned_addr learned my addr 172.21.15.82:0/3562973958 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df3fff640 1 --2- 172.21.15.82:0/3562973958 >> v2:172.21.15.82:3300/0 conn(0x7f8df406b440 0x7f8df40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df8ed0640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df40b1ac0 0x7f8df40cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df2ffd640 1 -- 172.21.15.82:0/3562973958 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980285798 0 0) 0x7f8df40afb20 con 0x7f8df406b820 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df2ffd640 1 -- 172.21.15.82:0/3562973958 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ddc003530 con 0x7f8df406b820 2024-04-25T10:02:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df3fff640 1 -- 172.21.15.82:0/3562973958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df40b1ac0 msgr2=0x7f8df40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df3fff640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df40b1ac0 0x7f8df40cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df3fff640 1 -- 172.21.15.82:0/3562973958 >> v1:172.21.15.82:6789/0 conn(0x7f8df406b820 legacy=0x7f8df40b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.008+0000 7f8df3fff640 1 -- 172.21.15.82:0/3562973958 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40b0070 con 0x7f8df406b440 2024-04-25T10:02:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df3fff640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df406b440 0x7f8df40b0b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8de40097c0 tx=0x7f8de401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4698b01b0a4207d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df2ffd640 1 -- 172.21.15.82:0/3562973958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40b1070 con 0x7f8df406b440 2024-04-25T10:02:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df2ffd640 1 -- 172.21.15.82:0/3562973958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de40ac410 con 0x7f8df406b440 2024-04-25T10:02:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df2ffd640 1 -- 172.21.15.82:0/3562973958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de40ac7c0 con 0x7f8df406b440 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- 172.21.15.82:0/3562973958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df406b440 msgr2=0x7f8df40b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df406b440 0x7f8df40b0b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8de40097c0 tx=0x7f8de401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- 172.21.15.82:0/3562973958 shutdown_connections 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df40b1ac0 0x7f8df40cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- 172.21.15.82:0/3562973958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df406b440 0x7f8df40b0b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- 172.21.15.82:0/3562973958 >> 172.21.15.82:0/3562973958 conn(0x7f8df40b1f10 msgr2=0x7f8df40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- 172.21.15.82:0/3562973958 shutdown_connections 2024-04-25T10:02:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- 172.21.15.82:0/3562973958 wait complete. 2024-04-25T10:02:37.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 Processor -- start 2024-04-25T10:02:37.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- start start 2024-04-25T10:02:37.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df406b440 0x7f8df40b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 0x7f8df415b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40cd130 con 0x7f8df40b1ac0 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df40ccfb0 con 0x7f8df406b820 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8dfa933640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df415d080 con 0x7f8df406b440 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41970/0 (socket says 172.21.15.82:41970) 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 -- 172.21.15.82:0/2615346316 learned_addr learned my addr 172.21.15.82:0/2615346316 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df3fff640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df406b440 0x7f8df40b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df37fe640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 0x7f8df415b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 -- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df406b440 msgr2=0x7f8df40b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df406b440 0x7f8df40b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 -- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 msgr2=0x7f8df415b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 0x7f8df415b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df8ed0640 1 -- 172.21.15.82:0/2615346316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df40b4980 con 0x7f8df40b1ac0 2024-04-25T10:02:37.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.012+0000 7f8df37fe640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 0x7f8df415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df8ed0640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dec007d80 tx=0x7f8dec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec014420 con 0x7f8df40b1ac0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dec019070 con 0x7f8df40b1ac0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df4157360 con 0x7f8df40b1ac0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dec014800 con 0x7f8df40b1ac0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df40b4c00 con 0x7f8df40b1ac0 2024-04-25T10:02:37.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df404efa0 con 0x7f8df40b1ac0 2024-04-25T10:02:37.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dec0149a0 con 0x7f8df40b1ac0 2024-04-25T10:02:37.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 0x7f8de0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8dec085be0 con 0x7f8df40b1ac0 2024-04-25T10:02:37.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df3fff640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 0x7f8de0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.016+0000 7f8df3fff640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 0x7f8de0069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8de40ae000 tx=0x7f8de40a6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.020+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8df40b4980 con 0x7f8df40b1ac0 2024-04-25T10:02:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.180+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8df40af890 con 0x7f8df40b1ac0 2024-04-25T10:02:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.180+0000 7f8df0ff9640 1 -- 172.21.15.82:0/2615346316 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dec011260 con 0x7f8df40b1ac0 2024-04-25T10:02:37.189 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:37.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 msgr2=0x7f8de0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 0x7f8de0069420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8de40ae000 tx=0x7f8de40a6040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 msgr2=0x7f8df41502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8dec007d80 tx=0x7f8dec00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 shutdown_connections 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8de0066f60 0x7f8de0069420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8df40b1ac0 0x7f8df41502b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8df406b820 0x7f8df415b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 --2- 172.21.15.82:0/2615346316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8df406b440 0x7f8df40b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 >> 172.21.15.82:0/2615346316 conn(0x7f8df40b1f10 msgr2=0x7f8df4161560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 shutdown_connections 2024-04-25T10:02:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.184+0000 7f8dfa933640 1 -- 172.21.15.82:0/2615346316 wait complete. 2024-04-25T10:02:37.207 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:37.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 Processor -- start 2024-04-25T10:02:37.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 -- start start 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8f40b5270 0x7fa8f40b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 0x7fa8f406d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f406bff0 con 0x7fa8f40b5650 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f406cc00 con 0x7fa8f40b5270 2024-04-25T10:02:37.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f9105640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f406c2d0 con 0x7fa8f406d090 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8f40b5270 0x7fa8f40b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa8f40b5270 0x7fa8f40b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41974/0 (socket says 172.21.15.82:41974) 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/182188027 learned_addr learned my addr 172.21.15.82:0/182188027 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f359e640 1 --2- 172.21.15.82:0/182188027 >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 0x7fa8f406d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/182188027 >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 msgr2=0x7fa8f406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 --2- 172.21.15.82:0/182188027 >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 0x7fa8f406d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/182188027 >> v1:172.21.15.196:6789/0 conn(0x7fa8f40b5650 legacy=0x7fa8f40d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/182188027 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f406c5b0 con 0x7fa8f40b5270 2024-04-25T10:02:37.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.320+0000 7fa8f359e640 1 --2- 172.21.15.82:0/182188027 >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 0x7fa8f406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f2d9d640 1 --2- 172.21.15.82:0/182188027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40b5270 0x7fa8f40b0380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40d0240 tx=0x7fa8e401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374c302e78ae74f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f1d9b640 1 -- 172.21.15.82:0/182188027 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40ab5f0 con 0x7fa8f40b5270 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f1d9b640 1 -- 172.21.15.82:0/182188027 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e40b0070 con 0x7fa8f40b5270 2024-04-25T10:02:37.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f1d9b640 1 -- 172.21.15.82:0/182188027 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40ab9b0 con 0x7fa8f40b5270 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- 172.21.15.82:0/182188027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40b5270 msgr2=0x7fa8f40b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 --2- 172.21.15.82:0/182188027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40b5270 0x7fa8f40b0380 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40d0240 tx=0x7fa8e401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- 172.21.15.82:0/182188027 shutdown_connections 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 --2- 172.21.15.82:0/182188027 >> v2:172.21.15.196:3300/0 conn(0x7fa8f406d090 0x7fa8f406d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 --2- 172.21.15.82:0/182188027 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f40b5270 0x7fa8f40b0380 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- 172.21.15.82:0/182188027 >> 172.21.15.82:0/182188027 conn(0x7fa8f408b000 msgr2=0x7fa8f40d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- 172.21.15.82:0/182188027 shutdown_connections 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- 172.21.15.82:0/182188027 wait complete. 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 Processor -- start 2024-04-25T10:02:37.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.324+0000 7fa8f9105640 1 -- start start 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 0x7fa8f40b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 0x7fa8f40b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f415de20 con 0x7fa8f406d090 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f40b5ac0 con 0x7fa8f40b5650 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f415dfa0 con 0x7fa8f40b5270 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41980/0 (socket says 172.21.15.82:41980) 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/1137637772 learned_addr learned my addr 172.21.15.82:0/1137637772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f359e640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 0x7fa8f40b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f259c640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 0x7fa8f40b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 msgr2=0x7fa8f40b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 0x7fa8f40b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 msgr2=0x7fa8f40b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 0x7fa8f40b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 -- 172.21.15.82:0/1137637772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f414db50 con 0x7fa8f406d090 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f259c640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 0x7fa8f40b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f359e640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 0x7fa8f40b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f2d9d640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa8e401a110 tx=0x7fa8e4019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4003cf0 con 0x7fa8f406d090 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e40bd400 con 0x7fa8f406d090 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f414de30 con 0x7fa8f406d090 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e40bd7f0 con 0x7fa8f406d090 2024-04-25T10:02:37.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f414e3f0 con 0x7fa8f406d090 2024-04-25T10:02:37.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f404efa0 con 0x7fa8f406d090 2024-04-25T10:02:37.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8e40b5400 con 0x7fa8f406d090 2024-04-25T10:02:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 0x7fa8e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa8e411f990 con 0x7fa8f406d090 2024-04-25T10:02:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f259c640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 0x7fa8e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.328+0000 7fa8f259c640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 0x7fa8e0069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40ce570 tx=0x7fa8e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.332+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8e40b9030 con 0x7fa8f406d090 2024-04-25T10:02:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.492+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8f40d1b20 con 0x7fa8f406d090 2024-04-25T10:02:37.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.492+0000 7fa8db7fe640 1 -- 172.21.15.82:0/1137637772 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8e40160d0 con 0x7fa8f406d090 2024-04-25T10:02:37.500 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 msgr2=0x7fa8e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 0x7fa8e0069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40ce570 tx=0x7fa8e8009040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 msgr2=0x7fa8f415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa8e401a110 tx=0x7fa8e4019280 comp rx=0 tx=0).stop 2024-04-25T10:02:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 shutdown_connections 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8e0066f60 0x7fa8e0069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8f40b5270 0x7fa8f40b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8f40b5650 0x7fa8f40b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 --2- 172.21.15.82:0/1137637772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8f406d090 0x7fa8f415ce50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 >> 172.21.15.82:0/1137637772 conn(0x7fa8f408b000 msgr2=0x7fa8f40b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 shutdown_connections 2024-04-25T10:02:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.496+0000 7fa8f9105640 1 -- 172.21.15.82:0/1137637772 wait complete. 2024-04-25T10:02:37.519 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:37.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 Processor -- start 2024-04-25T10:02:37.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 -- start start 2024-04-25T10:02:37.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f27e415d970 0x7f27e40d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27e40d3de0 0x7f27e406d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e40cf460 con 0x7f27e406d1f0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e406c710 con 0x7f27e415d970 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e8be1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e406c9f0 con 0x7f27e40d3de0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e1d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f27e406d1f0 0x7f27e40d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46786/0 (socket says 172.21.15.82:46786) 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e1d9b640 1 -- 172.21.15.82:0/1595145538 learned_addr learned my addr 172.21.15.82:0/1595145538 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e259c640 1 --2- 172.21.15.82:0/1595145538 >> v2:172.21.15.82:3300/0 conn(0x7f27e415d970 0x7f27e40d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e159a640 1 -- 172.21.15.82:0/1595145538 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955994669 0 0) 0x7f27e40cf460 con 0x7f27e406d1f0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e2d9d640 1 --2- 172.21.15.82:0/1595145538 >> v2:172.21.15.196:3300/0 conn(0x7f27e40d3de0 0x7f27e406d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e159a640 1 -- 172.21.15.82:0/1595145538 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27cc003530 con 0x7f27e406d1f0 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e259c640 1 -- 172.21.15.82:0/1595145538 >> v2:172.21.15.196:3300/0 conn(0x7f27e40d3de0 msgr2=0x7f27e406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e259c640 1 --2- 172.21.15.82:0/1595145538 >> v2:172.21.15.196:3300/0 conn(0x7f27e40d3de0 0x7f27e406d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e259c640 1 -- 172.21.15.82:0/1595145538 >> v1:172.21.15.82:6789/0 conn(0x7f27e406d1f0 legacy=0x7f27e40d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.636+0000 7f27e259c640 1 -- 172.21.15.82:0/1595145538 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e40cf740 con 0x7f27e415d970 2024-04-25T10:02:37.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e259c640 1 --2- 172.21.15.82:0/1595145538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40d1340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27e40c7270 tx=0x7f27d401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0cbf278b83369d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e159a640 1 -- 172.21.15.82:0/1595145538 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40b0070 con 0x7f27e415d970 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e159a640 1 -- 172.21.15.82:0/1595145538 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d40ab600 con 0x7f27e415d970 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e159a640 1 -- 172.21.15.82:0/1595145538 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40ab9d0 con 0x7f27e415d970 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- 172.21.15.82:0/1595145538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 msgr2=0x7f27e40d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1595145538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40d1340 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f27e40c7270 tx=0x7f27d401c360 comp rx=0 tx=0).stop 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- 172.21.15.82:0/1595145538 shutdown_connections 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1595145538 >> v2:172.21.15.196:3300/0 conn(0x7f27e40d3de0 0x7f27e406d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1595145538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40d1340 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- 172.21.15.82:0/1595145538 >> 172.21.15.82:0/1595145538 conn(0x7f27e409cae0 msgr2=0x7f27e408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- 172.21.15.82:0/1595145538 shutdown_connections 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- 172.21.15.82:0/1595145538 wait complete. 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 Processor -- start 2024-04-25T10:02:37.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.640+0000 7f27e8be1640 1 -- start start 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27e406d1f0 0x7f27e4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40d3de0 0x7f27e40cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40cf110 con 0x7f27e415d970 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e415df60 con 0x7f27e40d3de0 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40d1a90 con 0x7f27e406d1f0 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41994/0 (socket says 172.21.15.82:41994) 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 -- 172.21.15.82:0/1979214685 learned_addr learned my addr 172.21.15.82:0/1979214685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e259c640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27e406d1f0 0x7f27e4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 -- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27e406d1f0 msgr2=0x7f27e4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27e406d1f0 0x7f27e4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 -- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40d3de0 msgr2=0x7f27e40cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40d3de0 0x7f27e40cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 -- 172.21.15.82:0/1979214685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e41f69f0 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e2d9d640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007e90 tx=0x7f27dc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc014420 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc019070 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41f6cd0 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc014800 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41f7290 con 0x7f27e415d970 2024-04-25T10:02:37.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e404efa0 con 0x7f27e415d970 2024-04-25T10:02:37.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27dc0149a0 con 0x7f27e415d970 2024-04-25T10:02:37.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 0x7f27c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27dc085b50 con 0x7f27e415d970 2024-04-25T10:02:37.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e259c640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 0x7f27c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.644+0000 7f27e259c640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 0x7f27c4069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27d40ad000 tx=0x7f27d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.648+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27dc011260 con 0x7f27e415d970 2024-04-25T10:02:37.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27e40cfd30 con 0x7f27e415d970 2024-04-25T10:02:37.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27d2ffd640 1 -- 172.21.15.82:0/1979214685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27dc0568b0 con 0x7f27e415d970 2024-04-25T10:02:37.815 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 msgr2=0x7f27c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 0x7f27c4069420 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f27d40ad000 tx=0x7f27d4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 msgr2=0x7f27e40b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007e90 tx=0x7f27dc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 shutdown_connections 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f27c4066f60 0x7f27c4069420 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27e415d970 0x7f27e40b7a90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27e40d3de0 0x7f27e40cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 --2- 172.21.15.82:0/1979214685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27e406d1f0 0x7f27e4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 >> 172.21.15.82:0/1979214685 conn(0x7f27e409cae0 msgr2=0x7f27e41609b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 shutdown_connections 2024-04-25T10:02:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.808+0000 7f27e8be1640 1 -- 172.21.15.82:0/1979214685 wait complete. 2024-04-25T10:02:37.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 Processor -- start 2024-04-25T10:02:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 -- start start 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c0cf590 0x7fc74c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 0x7fc74c15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c15ba90 con 0x7fc74c15b6b0 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74c0cff30 con 0x7fc74c15f130 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc754579640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0cbd20 con 0x7fc74c0cf590 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41996/0 (socket says 172.21.15.82:41996) 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 -- 172.21.15.82:0/2420449869 learned_addr learned my addr 172.21.15.82:0/2420449869 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752b16640 1 --2- 172.21.15.82:0/2420449869 >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 0x7fc74c15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc751b14640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c0cf590 0x7fc74c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 -- 172.21.15.82:0/2420449869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c0cf590 msgr2=0x7fc74c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c0cf590 0x7fc74c0cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 -- 172.21.15.82:0/2420449869 >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 msgr2=0x7fc74c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 --2- 172.21.15.82:0/2420449869 >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 0x7fc74c15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 -- 172.21.15.82:0/2420449869 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c0cc000 con 0x7fc74c15b6b0 2024-04-25T10:02:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752b16640 1 --2- 172.21.15.82:0/2420449869 >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 0x7fc74c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.948+0000 7fc752315640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0c2de0 tx=0x7fc74801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136bbeeb3fd80826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc751313640 1 -- 172.21.15.82:0/2420449869 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480aa610 con 0x7fc74c15b6b0 2024-04-25T10:02:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc751313640 1 -- 172.21.15.82:0/2420449869 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7480af070 con 0x7fc74c15b6b0 2024-04-25T10:02:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc751313640 1 -- 172.21.15.82:0/2420449869 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480aa9d0 con 0x7fc74c15b6b0 2024-04-25T10:02:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/2420449869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c15b6b0 msgr2=0x7fc74c0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0c2de0 tx=0x7fc74801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/2420449869 shutdown_connections 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- 172.21.15.82:0/2420449869 >> v2:172.21.15.196:3300/0 conn(0x7fc74c15f130 0x7fc74c15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c0cf590 0x7fc74c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- 172.21.15.82:0/2420449869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0ca3c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/2420449869 >> 172.21.15.82:0/2420449869 conn(0x7fc74c06ae60 msgr2=0x7fc74c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/2420449869 shutdown_connections 2024-04-25T10:02:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/2420449869 wait complete. 2024-04-25T10:02:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 Processor -- start 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- start start 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c15f130 0x7fc74c0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c15fef0 con 0x7fc74c0cf590 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0cba30 con 0x7fc74c15b6b0 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74c0ca900 con 0x7fc74c15f130 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42006/0 (socket says 172.21.15.82:42006) 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 -- 172.21.15.82:0/1729096862 learned_addr learned my addr 172.21.15.82:0/1729096862 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc751b14640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 -- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c15f130 msgr2=0x7fc74c0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c15f130 0x7fc74c0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 -- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c15b6b0 msgr2=0x7fc74c0afd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0afd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 -- 172.21.15.82:0/1729096862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74c15c620 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc752315640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc74801aba0 tx=0x7fc74801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480b3600 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7480b37a0 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74c15c900 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7480b3b90 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.952+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc74c0c2dc0 con 0x7fc74c0cf590 2024-04-25T10:02:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74c04efa0 con 0x7fc74c0cf590 2024-04-25T10:02:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7480c6390 con 0x7fc74c0cf590 2024-04-25T10:02:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc742ffd640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc751b14640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc74811fb20 con 0x7fc74c0cf590 2024-04-25T10:02:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.956+0000 7fc751b14640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 0x7fc730069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0b3520 tx=0x7fc73c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:37.960+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7480c3060 con 0x7fc74c0cf590 2024-04-25T10:02:38.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.104+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc74c0c3000 con 0x7fc74c0cf590 2024-04-25T10:02:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.104+0000 7fc742ffd640 1 -- 172.21.15.82:0/1729096862 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc7480a7070 con 0x7fc74c0cf590 2024-04-25T10:02:38.113 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 msgr2=0x7fc730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 0x7fc730069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0b3520 tx=0x7fc73c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 msgr2=0x7fc74c1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc74801aba0 tx=0x7fc74801ad30 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 shutdown_connections 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f60 0x7fc730069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc74c15f130 0x7fc74c0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc74c15b6b0 0x7fc74c0afd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 --2- 172.21.15.82:0/1729096862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc74c0cf590 0x7fc74c1564f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 >> 172.21.15.82:0/1729096862 conn(0x7fc74c06ae60 msgr2=0x7fc74c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 shutdown_connections 2024-04-25T10:02:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.108+0000 7fc754579640 1 -- 172.21.15.82:0/1729096862 wait complete. 2024-04-25T10:02:38.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 Processor -- start 2024-04-25T10:02:38.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 -- start start 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb90809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb90815bd30 con 0x7fb9080cf420 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9080cc0d0 con 0x7fb9080cfb80 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb90dec0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90809ab10 con 0x7fb90815b950 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb8fffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb9080cf420 0x7fb9080cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46798/0 (socket says 172.21.15.82:46798) 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb8fffff640 1 -- 172.21.15.82:0/3479211425 learned_addr learned my addr 172.21.15.82:0/3479211425 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb90809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424964559 0 0) 0x7fb90815bd30 con 0x7fb9080cf420 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8e8003530 con 0x7fb9080cf420 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3111797999 0 0) 0x7fb8e8003530 con 0x7fb9080cf420 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 msgr2=0x7fb90809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb90809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 >> v1:172.21.15.196:6789/0 conn(0x7fb9080cfb80 legacy=0x7fb90806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080cc370 con 0x7fb9080cf420 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb8f0001680 con 0x7fb9080cf420 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 >> v1:172.21.15.82:6789/0 conn(0x7fb9080cf420 legacy=0x7fb9080cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8003a60 0x7fb8e8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e8004810 0x7fb8e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb8e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90809ab10 con 0x7fb8e8003a60 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800bad0 con 0x7fb8e8004810 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e800bcd0 con 0x7fb90815b950 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8003a60 0x7fb8e8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb8fffff640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e8004810 0x7fb8e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb907fff640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb8e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 -- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 msgr2=0x7fb8e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb90815b950 0x7fb8e80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 -- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e8004810 msgr2=0x7fb8e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb8e8004810 0x7fb8e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.232+0000 7fb9077fe640 1 -- 172.21.15.82:0/3479211425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8003530 con 0x7fb8e8003a60 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb9077fe640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8003a60 0x7fb8e8003e80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009ae0 tx=0x7fb8f801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955dc778aa30b716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80096b0 con 0x7fb8e8003a60 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f80a9470 con 0x7fb8e8003a60 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb906ffd640 1 -- 172.21.15.82:0/3479211425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f80a9860 con 0x7fb8e8003a60 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- 172.21.15.82:0/3479211425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8003a60 msgr2=0x7fb8e8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 --2- 172.21.15.82:0/3479211425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8e8003a60 0x7fb8e8003e80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8009ae0 tx=0x7fb8f801ca70 comp rx=0 tx=0).stop 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb907fff640 1 -- 172.21.15.82:0/3479211425 reap_dead start 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- 172.21.15.82:0/3479211425 shutdown_connections 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- 172.21.15.82:0/3479211425 >> 172.21.15.82:0/3479211425 conn(0x7fb9080b30b0 msgr2=0x7fb9080b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- 172.21.15.82:0/3479211425 shutdown_connections 2024-04-25T10:02:38.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- 172.21.15.82:0/3479211425 wait complete. 2024-04-25T10:02:38.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 Processor -- start 2024-04-25T10:02:38.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.236+0000 7fb90dec0640 1 -- start start 2024-04-25T10:02:38.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb908155a00 0x7fb9080c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 0x7fb9080ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080cff90 con 0x7fb908155de0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb90809a890 con 0x7fb908155a00 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080d0110 con 0x7fb908156310 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42030/0 (socket says 172.21.15.82:42030) 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 -- 172.21.15.82:0/879753138 learned_addr learned my addr 172.21.15.82:0/879753138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb8fffff640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 0x7fb9080ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb9077fe640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb908155a00 0x7fb9080c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 -- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 msgr2=0x7fb9080ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 0x7fb9080ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 -- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb908155a00 msgr2=0x7fb9080c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb908155a00 0x7fb9080c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 -- 172.21.15.82:0/879753138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080c7590 con 0x7fb908155de0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb8fffff640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 0x7fb9080ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb907fff640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb908072850 tx=0x7fb90000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90000ee90 con 0x7fb908155de0 2024-04-25T10:02:38.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb900018070 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9080c5d50 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb900003bc0 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9080c6310 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9080c6510 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9000050d0 con 0x7fb908155de0 2024-04-25T10:02:38.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 0x7fb8ec0695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb9077fe640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 0x7fb8ec0695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb900084fe0 con 0x7fb908155de0 2024-04-25T10:02:38.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.240+0000 7fb9077fe640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 0x7fb8ec0695f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80ae000 tx=0x7fb8f80b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.244+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9080c6510 con 0x7fb908155de0 2024-04-25T10:02:38.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.436+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9080d13a0 con 0x7fb908155de0 2024-04-25T10:02:38.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.436+0000 7fb904ff9640 1 -- 172.21.15.82:0/879753138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fb900055c30 con 0x7fb908155de0 2024-04-25T10:02:38.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:38.445 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:38.445 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 msgr2=0x7fb8ec0695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 0x7fb8ec0695f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80ae000 tx=0x7fb8f80b3040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 msgr2=0x7fb9080ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb908072850 tx=0x7fb90000eb50 comp rx=0 tx=0).stop 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 shutdown_connections 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb8ec067130 0x7fb8ec0695f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb908156310 0x7fb9080ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb908155de0 0x7fb9080ca630 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 --2- 172.21.15.82:0/879753138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb908155a00 0x7fb9080c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 >> 172.21.15.82:0/879753138 conn(0x7fb9080b30b0 msgr2=0x7fb908073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 shutdown_connections 2024-04-25T10:02:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.440+0000 7fb90dec0640 1 -- 172.21.15.82:0/879753138 wait complete. 2024-04-25T10:02:38.462 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:38.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.564+0000 7f4ef58d4640 1 Processor -- start 2024-04-25T10:02:38.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.564+0000 7f4ef58d4640 1 -- start start 2024-04-25T10:02:38.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4ef00d07f0 0x7f4ef00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef006be60 con 0x7f4ef00d1cd0 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef015de60 con 0x7f4ef015da80 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef006c440 con 0x7f4ef00d07f0 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eee7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4ef00d07f0 0x7f4ef00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eef7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4ef00d1cd0 0x7f4ef006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46802/0 (socket says 172.21.15.82:46802) 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eef7fe640 1 -- 172.21.15.82:0/527938817 learned_addr learned my addr 172.21.15.82:0/527938817 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eee7fc640 1 -- 172.21.15.82:0/527938817 >> v1:172.21.15.196:6789/0 conn(0x7f4ef015da80 legacy=0x7f4ef00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eee7fc640 1 -- 172.21.15.82:0/527938817 >> v1:172.21.15.82:6789/0 conn(0x7f4ef00d1cd0 legacy=0x7f4ef006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eee7fc640 1 -- 172.21.15.82:0/527938817 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00d2450 con 0x7f4ef00d07f0 2024-04-25T10:02:38.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eedffb640 1 -- 172.21.15.82:0/527938817 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 631385259 0 0) 0x7f4ef006be60 con 0x7f4ef00d1cd0 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eee7fc640 1 --2- 172.21.15.82:0/527938817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00d4230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40097a0 tx=0x7f4ee401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532cd0037967476d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eedffb640 1 -- 172.21.15.82:0/527938817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40b0070 con 0x7f4ef00d07f0 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eedffb640 1 -- 172.21.15.82:0/527938817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee40ab600 con 0x7f4ef00d07f0 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4eedffb640 1 -- 172.21.15.82:0/527938817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40ab9d0 con 0x7f4ef00d07f0 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- 172.21.15.82:0/527938817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 msgr2=0x7f4ef00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/527938817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00d4230 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40097a0 tx=0x7f4ee401c410 comp rx=0 tx=0).stop 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- 172.21.15.82:0/527938817 shutdown_connections 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/527938817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00d4230 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- 172.21.15.82:0/527938817 >> 172.21.15.82:0/527938817 conn(0x7f4ef008b000 msgr2=0x7f4ef00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- 172.21.15.82:0/527938817 shutdown_connections 2024-04-25T10:02:38.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.568+0000 7f4ef58d4640 1 -- 172.21.15.82:0/527938817 wait complete. 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 Processor -- start 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- start start 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 0x7f4ef0158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef015da80 0x7f4ef00b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006d970 con 0x7f4ef00d07f0 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006d7f0 con 0x7f4ef00d1cd0 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef006daf0 con 0x7f4ef015da80 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42052/0 (socket says 172.21.15.82:42052) 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 -- 172.21.15.82:0/301771017 learned_addr learned my addr 172.21.15.82:0/301771017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eee7fc640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 0x7f4ef0158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 -- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef015da80 msgr2=0x7f4ef00b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef015da80 0x7f4ef00b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 -- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 msgr2=0x7f4ef0158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 0x7f4ef0158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 -- 172.21.15.82:0/301771017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef00b8680 con 0x7f4ef00d07f0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eee7fc640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 0x7f4ef0158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4eeeffd640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c450 tx=0x7f4ee000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0012830 con 0x7f4ef00d07f0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee0017070 con 0x7f4ef00d07f0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef01499a0 con 0x7f4ef00d07f0 2024-04-25T10:02:38.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0012c30 con 0x7f4ef00d07f0 2024-04-25T10:02:38.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef0149f60 con 0x7f4ef00d07f0 2024-04-25T10:02:38.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.572+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef004efa0 con 0x7f4ef00d07f0 2024-04-25T10:02:38.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.576+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ee001c460 con 0x7f4ef00d07f0 2024-04-25T10:02:38.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.576+0000 7f4ed7fff640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 0x7f4edc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.576+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4ee0085120 con 0x7f4ef00d07f0 2024-04-25T10:02:38.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.576+0000 7f4eee7fc640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 0x7f4edc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.576+0000 7f4eee7fc640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 0x7f4edc069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ef006de30 tx=0x7f4ee40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.580+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ee0055e80 con 0x7f4ef00d07f0 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.724+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ef00b9570 con 0x7f4ef00d07f0 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.728+0000 7f4ed7fff640 1 -- 172.21.15.82:0/301771017 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18526 (secure 0 0 0) 0x7f4ee0055780 con 0x7f4ef00d07f0 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":25,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:11.763825+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:38.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 msgr2=0x7f4edc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 0x7f4edc069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f4ef006de30 tx=0x7f4ee40a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 msgr2=0x7f4ef00b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000c450 tx=0x7f4ee000c910 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 shutdown_connections 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4edc066f60 0x7f4edc069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef015da80 0x7f4ef00b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00d1cd0 0x7f4ef0158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 --2- 172.21.15.82:0/301771017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef00d07f0 0x7f4ef00b8050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 >> 172.21.15.82:0/301771017 conn(0x7f4ef008b000 msgr2=0x7f4ef00b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 shutdown_connections 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.732+0000 7f4ef58d4640 1 -- 172.21.15.82:0/301771017 wait complete. 2024-04-25T10:02:38.754 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.864+0000 7fea2437e640 1 Processor -- start 2024-04-25T10:02:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.864+0000 7fea2437e640 1 -- start start 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2437e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea1c0b3f50 0x7fea1c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2437e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fea1c0b24a0 0x7fea1c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2437e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c0b2090 con 0x7fea1c15a700 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2437e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c15f200 con 0x7fea1c0b3f50 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2437e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c0b43e0 con 0x7fea1c0b24a0 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea1c0b3f50 0x7fea1c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea1c0b3f50 0x7fea1c15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42068/0 (socket says 172.21.15.82:42068) 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 -- 172.21.15.82:0/691232591 learned_addr learned my addr 172.21.15.82:0/691232591 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea2291b640 1 --2- 172.21.15.82:0/691232591 >> v2:172.21.15.196:3300/0 conn(0x7fea1c0b24a0 0x7fea1c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 -- 172.21.15.82:0/691232591 >> v2:172.21.15.196:3300/0 conn(0x7fea1c0b24a0 msgr2=0x7fea1c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 --2- 172.21.15.82:0/691232591 >> v2:172.21.15.196:3300/0 conn(0x7fea1c0b24a0 0x7fea1c0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 -- 172.21.15.82:0/691232591 >> v1:172.21.15.196:6789/0 conn(0x7fea1c15a700 legacy=0x7fea1c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 -- 172.21.15.82:0/691232591 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0b46c0 con 0x7fea1c0b3f50 2024-04-25T10:02:38.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21919640 1 --2- 172.21.15.82:0/691232591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c0b3f50 0x7fea1c15ecc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b3e20 tx=0x7fea0c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f05593554440c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21118640 1 -- 172.21.15.82:0/691232591 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0b0070 con 0x7fea1c0b3f50 2024-04-25T10:02:38.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21118640 1 -- 172.21.15.82:0/691232591 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c0ab600 con 0x7fea1c0b3f50 2024-04-25T10:02:38.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.868+0000 7fea21118640 1 -- 172.21.15.82:0/691232591 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c0ab9d0 con 0x7fea1c0b3f50 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/691232591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c0b3f50 msgr2=0x7fea1c15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- 172.21.15.82:0/691232591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c0b3f50 0x7fea1c15ecc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0b3e20 tx=0x7fea0c01c3a0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/691232591 shutdown_connections 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- 172.21.15.82:0/691232591 >> v2:172.21.15.196:3300/0 conn(0x7fea1c0b24a0 0x7fea1c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- 172.21.15.82:0/691232591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c0b3f50 0x7fea1c15ecc0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/691232591 >> 172.21.15.82:0/691232591 conn(0x7fea1c0b11a0 msgr2=0x7fea1c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/691232591 shutdown_connections 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/691232591 wait complete. 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 Processor -- start 2024-04-25T10:02:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- start start 2024-04-25T10:02:38.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea1c0b24a0 0x7fea1c153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea1c15a700 0x7fea1c144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0cd310 con 0x7fea1c145030 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0ce600 con 0x7fea1c0b24a0 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0cd490 con 0x7fea1c15a700 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42082/0 (socket says 172.21.15.82:42082) 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 -- 172.21.15.82:0/1742454151 learned_addr learned my addr 172.21.15.82:0/1742454151 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2211a640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea1c0b24a0 0x7fea1c153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea21919640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea1c15a700 0x7fea1c144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 -- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea1c15a700 msgr2=0x7fea1c144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea1c15a700 0x7fea1c144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 -- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea1c0b24a0 msgr2=0x7fea1c153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea1c0b24a0 0x7fea1c153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 -- 172.21.15.82:0/1742454151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c15f420 con 0x7fea1c145030 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2291b640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea14007e90 tx=0x7fea1400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14014420 con 0x7fea1c145030 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea14019070 con 0x7fea1c145030 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c15f6c0 con 0x7fea1c145030 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14014800 con 0x7fea1c145030 2024-04-25T10:02:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.872+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c15fc80 con 0x7fea1c145030 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c04efa0 con 0x7fea1c145030 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea140149a0 con 0x7fea1c145030 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea12ffd640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 0x7fea00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea2211a640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 0x7fea00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fea14085c70 con 0x7fea1c145030 2024-04-25T10:02:38.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.876+0000 7fea2211a640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 0x7fea00069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea18004e30 tx=0x7fea18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:38.880+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea140568b0 con 0x7fea1c145030 2024-04-25T10:02:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.056+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea1c06f260 con 0x7fea1c145030 2024-04-25T10:02:39.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea12ffd640 1 -- 172.21.15.82:0/1742454151 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fea140561b0 con 0x7fea1c145030 2024-04-25T10:02:39.066 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:39.066 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:39.067 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 msgr2=0x7fea00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 0x7fea00069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fea18004e30 tx=0x7fea18009040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 msgr2=0x7fea1c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fea14007e90 tx=0x7fea1400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 shutdown_connections 2024-04-25T10:02:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea00066f60 0x7fea00069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea1c15a700 0x7fea1c144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea1c145030 0x7fea1c0cf7b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 --2- 172.21.15.82:0/1742454151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea1c0b24a0 0x7fea1c153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 >> 172.21.15.82:0/1742454151 conn(0x7fea1c0b11a0 msgr2=0x7fea1c0cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 shutdown_connections 2024-04-25T10:02:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.060+0000 7fea2437e640 1 -- 172.21.15.82:0/1742454151 wait complete. 2024-04-25T10:02:39.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:06.717111+0000', 'modified': '2024-04-25T10:02:29.823823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.196:6835/3824448217', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.196:6834', 'nonce': 3824448217}, {'type': 'v1', 'addr': '172.21.15.196:6835', 'nonce': 3824448217}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:39.486 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:39.486 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 Processor -- start 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 -- start start 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b1815a700 0x7f8b1815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b180b24a0 0x7f8b180b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b180b2090 con 0x7f8b180b3f50 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1815f200 con 0x7f8b1815a700 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1fc05640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b180b43e0 con 0x7f8b180b24a0 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d9a1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b1815a700 0x7f8b1815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d1a0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8b180b3f50 0x7f8b180cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46806/0 (socket says 172.21.15.82:46806) 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/693604347 learned_addr learned my addr 172.21.15.82:0/693604347 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/693604347 >> v2:172.21.15.196:3300/0 conn(0x7f8b180b24a0 msgr2=0x7f8b180b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/693604347 >> v2:172.21.15.196:3300/0 conn(0x7f8b180b24a0 0x7f8b180b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/693604347 >> v1:172.21.15.82:6789/0 conn(0x7f8b180b3f50 legacy=0x7f8b180cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.612+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/693604347 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b180b46c0 con 0x7f8b1815a700 2024-04-25T10:02:39.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/693604347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815a700 0x7f8b1815ecc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b1806c980 tx=0x7f8b0801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c114e4df812eafd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1c99f640 1 -- 172.21.15.82:0/693604347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080b0070 con 0x7f8b1815a700 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1c99f640 1 -- 172.21.15.82:0/693604347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080ab600 con 0x7f8b1815a700 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1c99f640 1 -- 172.21.15.82:0/693604347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080ab9d0 con 0x7f8b1815a700 2024-04-25T10:02:39.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 -- 172.21.15.82:0/693604347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815a700 msgr2=0x7f8b1815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/693604347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815a700 0x7f8b1815ecc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8b1806c980 tx=0x7f8b0801c360 comp rx=0 tx=0).stop 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 -- 172.21.15.82:0/693604347 shutdown_connections 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/693604347 >> v2:172.21.15.196:3300/0 conn(0x7f8b180b24a0 0x7f8b180b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/693604347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815a700 0x7f8b1815ecc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 -- 172.21.15.82:0/693604347 >> 172.21.15.82:0/693604347 conn(0x7f8b180b11a0 msgr2=0x7f8b180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 -- 172.21.15.82:0/693604347 shutdown_connections 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.616+0000 7f8b1fc05640 1 -- 172.21.15.82:0/693604347 wait complete. 2024-04-25T10:02:39.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 Processor -- start 2024-04-25T10:02:39.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- start start 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 0x7f8b181f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b1815cd90 0x7f8b180af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180cd310 con 0x7f8b1815d210 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180ce600 con 0x7f8b180b24a0 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b180cd490 con 0x7f8b1815cd90 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42102/0 (socket says 172.21.15.82:42102) 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 -- 172.21.15.82:0/1032057510 learned_addr learned my addr 172.21.15.82:0/1032057510 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 0x7f8b181f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b1815cd90 0x7f8b180af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 -- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b1815cd90 msgr2=0x7f8b180af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b1815cd90 0x7f8b180af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 -- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 msgr2=0x7f8b181f7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 0x7f8b181f7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 -- 172.21.15.82:0/1032057510 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b181f7ad0 con 0x7f8b1815d210 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 0x7f8b181f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1e1a2640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000b970 tx=0x7f8b1000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b0e7fc640 1 -- 172.21.15.82:0/1032057510 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10019070 con 0x7f8b1815d210 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b0e7fc640 1 -- 172.21.15.82:0/1032057510 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1000cb60 con 0x7f8b1815d210 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1815b540 con 0x7f8b1815d210 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b0e7fc640 1 -- 172.21.15.82:0/1032057510 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10014440 con 0x7f8b1815d210 2024-04-25T10:02:39.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1815bb00 con 0x7f8b1815d210 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.620+0000 7f8b0e7fc640 1 -- 172.21.15.82:0/1032057510 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b1001c070 con 0x7f8b1815d210 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b0e7fc640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 0x7f8afc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b0e7fc640 1 -- 172.21.15.82:0/1032057510 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b10084fd0 con 0x7f8b1815d210 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 0x7f8afc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 0x7f8afc069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b080ad000 tx=0x7f8b08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 Processor -- start 2024-04-25T10:02:39.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- start start 2024-04-25T10:02:39.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6649f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f665310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f54c5f0 con 0x55bf6f6ae920 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f4b3da0 con 0x55bf6f664f30 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f43c150 con 0x55bf6f6af240 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42108/0 (socket says 172.21.15.82:42108) 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/383985094 learned_addr learned my addr 172.21.15.82:0/383985094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 msgr2=0x55bf6f6649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 msgr2=0x55bf6f665310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1e1a2640 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f665310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f665310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/383985094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf6f455ca0 con 0x55bf6f6ae920 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b08008010 tx=0x7f8b080a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f332ce1f4c2d7f34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8afb7fe640 1 -- 172.21.15.82:0/383985094 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080c7070 con 0x55bf6f6ae920 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8afb7fe640 1 -- 172.21.15.82:0/383985094 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080b6670 con 0x55bf6f6ae920 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8afb7fe640 1 -- 172.21.15.82:0/383985094 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080b0040 con 0x55bf6f6ae920 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- 172.21.15.82:0/383985094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 msgr2=0x55bf6f6aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8b08008010 tx=0x7f8b080a6000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- 172.21.15.82:0/383985094 shutdown_connections 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f665310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 --2- 172.21.15.82:0/383985094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6aed00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- 172.21.15.82:0/383985094 >> 172.21.15.82:0/383985094 conn(0x55bf6f66b2a0 msgr2=0x55bf6f6632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- 172.21.15.82:0/383985094 shutdown_connections 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- 172.21.15.82:0/383985094 wait complete. 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 -- start start 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.624+0000 7f8b22d72000 1 Processor -- start 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6b5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f668aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f46e990 con 0x55bf6f6ae920 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f4afe40 con 0x55bf6f664f30 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f43f900 con 0x55bf6f6af240 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 10 client.0 _ll_register_callbacks cb 0x55bf6f5671c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42112/0 (socket says 172.21.15.82:42112) 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/1963561987 learned_addr learned my addr 172.21.15.82:0/1963561987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6b5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 msgr2=0x55bf6f668aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f668aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 msgr2=0x55bf6f6b5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6b5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/1963561987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf6f46c630 con 0x55bf6f6ae920 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001d10 tx=0x7f8b1400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df67d376b53c7b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14018060 con 0x55bf6f6ae920 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b14005a60 con 0x55bf6f6ae920 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b080b6670 con 0x55bf6f6ae920 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b1401e050 con 0x55bf6f6ae920 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 10 client.4767 fetch_fsmap learned FSMap version 8 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bf6f43c3a0 con 0x55bf6f6ae920 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b080b6670 con 0x55bf6f6ae920 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8b14021020 con 0x55bf6f6ae920 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 10 client.4767 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 10 client.4767 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 20 client.4767 populate_metadata read hostname 'smithi082' 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f66a160 0x55bf6f6bfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 4 client.4767 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 --> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf6f44fa50 con 0x55bf6f66a160 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f66a160 0x55bf6f6bfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8af8ff9640 10 client.4767 ms_handle_connect on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:39.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.628+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f66a160 0x55bf6f6bfe30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bcf956643770e7f server_cookie=44b7b1495cd2e72f in_seq=0 out_seq=0 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8af8ff9640 1 -- 172.21.15.82:0/1963561987 <== mds.0 v2:172.21.15.196:6834/3824448217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf6f44fa50 con 0x55bf6f66a160 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8af8ff9640 10 client.4767 handle_command_reply: tid=0 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 client.4767 shutdown 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 msgr2=0x55bf6f6b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8b14001d10 tx=0x7f8b1400d520 comp rx=0 tx=0).stop 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 shutdown_connections 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f66a160 0x55bf6f6bfe30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f668aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f6b6310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8af8ff9640 0 client.4767 ms_handle_reset on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 --2- 172.21.15.82:0/1963561987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6b5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 >> 172.21.15.82:0/1963561987 conn(0x55bf6f66b2a0 msgr2=0x55bf6f6639e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 shutdown_connections 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 1 -- 172.21.15.82:0/1963561987 wait complete. 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.632+0000 7f8b22d72000 20 client.4767 trim_cache size 0 max 16384 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.644+0000 7f8b22d72000 1 Processor -- start 2024-04-25T10:02:39.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.644+0000 7f8b22d72000 1 -- start start 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b182014a0 con 0x55bf6f6ae920 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b181520d0 con 0x55bf6f664f30 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b18200fd0 con 0x55bf6f6af240 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42118/0 (socket says 172.21.15.82:42118) 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/3275763189 learned_addr learned my addr 172.21.15.82:0/3275763189 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 msgr2=0x55bf6f6eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 msgr2=0x55bf6f6682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6682c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 -- 172.21.15.82:0/3275763189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf6f6af070 con 0x55bf6f6ae920 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b14000eb0 tx=0x7f8b1400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd08b44998c9d114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8afaffd640 1 -- 172.21.15.82:0/3275763189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14005a60 con 0x55bf6f6ae920 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8afaffd640 1 -- 172.21.15.82:0/3275763189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14007260 con 0x55bf6f6ae920 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8afaffd640 1 -- 172.21.15.82:0/3275763189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b14007610 con 0x55bf6f6ae920 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- 172.21.15.82:0/3275763189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 msgr2=0x55bf6f668800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b14000eb0 tx=0x7f8b1400b000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- 172.21.15.82:0/3275763189 shutdown_connections 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f6ae920 0x55bf6f668800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3275763189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f664f30 0x55bf6f6682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- 172.21.15.82:0/3275763189 >> 172.21.15.82:0/3275763189 conn(0x55bf6f66b2a0 msgr2=0x55bf6f667bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- 172.21.15.82:0/3275763189 shutdown_connections 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- 172.21.15.82:0/3275763189 wait complete. 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- start start 2024-04-25T10:02:39.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 Processor -- start 2024-04-25T10:02:39.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f6ae920 0x55bf6f6f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f4b83d0 con 0x55bf6f664f30 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f56fd50 con 0x55bf6f6ae920 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf6f44e980 con 0x55bf6f6af240 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b22d72000 10 client.0 _ll_register_callbacks cb 0x7f8b080ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42126/0 (socket says 172.21.15.82:42126) 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/3370985121 learned_addr learned my addr 172.21.15.82:0/3370985121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f6ae920 0x55bf6f6f26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 msgr2=0x55bf6f6f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f6ae920 msgr2=0x55bf6f6f26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f6ae920 0x55bf6f6f26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 -- 172.21.15.82:0/3370985121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf6f535600 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8b1d9a1640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c10 tx=0x7f8b0801aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a02bf87accfbab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.648+0000 7f8af8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b080c7070 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8b08004230 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b14007260 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b080dc050 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 10 client.4773 fetch_fsmap learned FSMap version 8 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55bf6f4489b0 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b14007260 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 8) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f8b080df020 con 0x55bf6f664f30 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 10 client.4773 fetch_fsmap finished waiting for FSMap version 8 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 10 client.4773 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 20 client.4773 populate_metadata read hostname 'smithi082' 2024-04-25T10:02:39.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f6f3510 0x55bf6f6f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 4 client.4773 mds_command: new command op to 4282 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 --> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bf6f4c12e0 con 0x55bf6f6f3510 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f6f3510 0x55bf6f6f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 10 client.4773 ms_handle_connect on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1d1a0640 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f6f3510 0x55bf6f6f38f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c60c29817cb2b server_cookie=b4fbec16baa4ca7c in_seq=0 out_seq=0 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 1 -- 172.21.15.82:0/3370985121 <== mds.0 v2:172.21.15.196:6834/3824448217 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55bf6f4c12e0 con 0x55bf6f6f3510 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 10 client.4773 handle_command_reply: tid=0 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 client.4773 shutdown 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 msgr2=0x55bf6f6bb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b08000c10 tx=0x7f8b0801aba0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 shutdown_connections 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:6834/3824448217,v1:172.21.15.196:6835/3824448217] conn(0x55bf6f6f3510 0x55bf6f6f38f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bf6f6af240 0x55bf6f6f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bf6f6ae920 0x55bf6f6f26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 --2- 172.21.15.82:0/3370985121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bf6f664f30 0x55bf6f6bb420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8af8ff9640 0 client.4773 ms_handle_reset on v2:172.21.15.196:6834/3824448217 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 >> 172.21.15.82:0/3370985121 conn(0x55bf6f66b2a0 msgr2=0x55bf6f6682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 shutdown_connections 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 1 -- 172.21.15.82:0/3370985121 wait complete. 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b22d72000 20 client.4773 trim_cache size 0 max 16384 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:02:39.665 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4282, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 32.314999180999997, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 8, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 25, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 38.995026592000002 2024-04-25T10:02:39.666 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 msgr2=0x7f8afc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 0x7f8afc069420 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8b080ad000 tx=0x7f8b08008040 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 msgr2=0x7f8b180afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000b970 tx=0x7f8b1000be30 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 shutdown_connections 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8afc066f60 0x7f8afc069420 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b1815cd90 0x7f8b180af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b1815d210 0x7f8b180afc30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 --2- 172.21.15.82:0/1032057510 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b180b24a0 0x7f8b181f7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 >> 172.21.15.82:0/1032057510 conn(0x7f8b180b11a0 msgr2=0x7f8b180b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 shutdown_connections 2024-04-25T10:02:39.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:39.652+0000 7f8b1fc05640 1 -- 172.21.15.82:0/1032057510 wait complete. 2024-04-25T10:02:39.676 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T10:02:39.686 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T10:02:39.687 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:02:39.687 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:39.687 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:39.687 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:39.687 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:39.687 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T10:02:39.688 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:39.688 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:39.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.688 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:02:39.694 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T10:02:39.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.695 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:39.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.751 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T10:02:39.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.932 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T10:02:39.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.939 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-25T10:02:39.997 INFO:teuthology.orchestra.run.smithi196.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:39.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:39.999 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T10:02:39.999 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:40.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.120+0000 7fa8b1bf6640 1 Processor -- start 2024-04-25T10:02:40.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8b1bf6640 1 -- start start 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8b1bf6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa8ac0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8b1bf6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0b0e10 con 0x7fa8ac09a180 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8b1bf6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8ac0d0180 con 0x7fa8ac06f4a0 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8b1bf6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0d0d90 con 0x7fa8ac0b0a30 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8abfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa8ac09a180 0x7fa8ac15d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46812/0 (socket says 172.21.15.82:46812) 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8abfff640 1 -- 172.21.15.82:0/3280581972 learned_addr learned my addr 172.21.15.82:0/3280581972 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8ab7fe640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa8ac0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773379957 0 0) 0x7fa8ac0b0e10 con 0x7fa8ac09a180 2024-04-25T10:02:40.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa898003530 con 0x7fa8ac09a180 2024-04-25T10:02:40.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 385081619 0 0) 0x7fa898003530 con 0x7fa8ac09a180 2024-04-25T10:02:40.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 msgr2=0x7fa8ac0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa8ac0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 >> v1:172.21.15.196:6789/0 conn(0x7fa8ac06f4a0 legacy=0x7fa8ac099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac0d0460 con 0x7fa8ac09a180 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa8a4001a00 con 0x7fa8ac09a180 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 >> v1:172.21.15.82:6789/0 conn(0x7fa8ac09a180 legacy=0x7fa8ac15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa898003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa898003fa0 0x7fa898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898004d50 0x7fa8980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0d0d90 con 0x7fa898003fa0 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89800bab0 con 0x7fa898004d50 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89800bcb0 con 0x7fa8ac0b0a30 2024-04-25T10:02:40.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa898003fa0 0x7fa898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8abfff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898004d50 0x7fa8980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8ab7fe640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa898003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 -- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 msgr2=0x7fa898003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac0b0a30 0x7fa898003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 -- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898004d50 msgr2=0x7fa8980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898004d50 0x7fa8980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8a3fff640 1 -- 172.21.15.82:0/3280581972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa898003530 con 0x7fa898003fa0 2024-04-25T10:02:40.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.124+0000 7fa8abfff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa898004d50 0x7fa8980051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:40.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa898003fa0 0x7fa898004810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa894001910 tx=0x7fa89401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86c0b2e18f79640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894001510 con 0x7fa898003fa0 2024-04-25T10:02:40.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8940b0070 con 0x7fa898003fa0 2024-04-25T10:02:40.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8aaffd640 1 -- 172.21.15.82:0/3280581972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8940ab870 con 0x7fa898003fa0 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/3280581972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa898003fa0 msgr2=0x7fa898004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/3280581972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa898003fa0 0x7fa898004810 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa894001910 tx=0x7fa89401c530 comp rx=0 tx=0).stop 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8abfff640 1 -- 172.21.15.82:0/3280581972 reap_dead start 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/3280581972 shutdown_connections 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/3280581972 >> 172.21.15.82:0/3280581972 conn(0x7fa8ac0b30b0 msgr2=0x7fa8ac0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/3280581972 shutdown_connections 2024-04-25T10:02:40.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/3280581972 wait complete. 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 Processor -- start 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.128+0000 7fa8b1bf6640 1 -- start start 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac1597d0 0x7fa8ac0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 0x7fa8ac1eaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac15dd50 con 0x7fa8ac159bb0 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac0cce30 con 0x7fa8ac1597d0 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac15ded0 con 0x7fa8ac1ea640 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42150/0 (socket says 172.21.15.82:42150) 2024-04-25T10:02:40.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 -- 172.21.15.82:0/873264653 learned_addr learned my addr 172.21.15.82:0/873264653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8ab7fe640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac1597d0 0x7fa8ac0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 0x7fa8ac1eaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 -- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 msgr2=0x7fa8ac1eaac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 0x7fa8ac1eaac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 -- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac1597d0 msgr2=0x7fa8ac0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac1597d0 0x7fa8ac0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 -- 172.21.15.82:0/873264653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac15a9a0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a3fff640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 0x7fa8ac1eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8abfff640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001360 tx=0x7fa8a400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4013420 con 0x7fa8ac159bb0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a4018070 con 0x7fa8ac159bb0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac15ac80 con 0x7fa8ac159bb0 2024-04-25T10:02:40.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a4013820 con 0x7fa8ac159bb0 2024-04-25T10:02:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac0b5030 con 0x7fa8ac159bb0 2024-04-25T10:02:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac0b4e20 con 0x7fa8ac159bb0 2024-04-25T10:02:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8a40139c0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 0x7fa88c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa8a40859d0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8ab7fe640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 0x7fa88c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.132+0000 7fa8ab7fe640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 0x7fa88c069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009a30 tx=0x7fa89c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.140+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8a40566a0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.332+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa8ac04efa0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.332+0000 7fa8a8ff9640 1 -- 172.21.15.82:0/873264653 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7fa8a4055fa0 con 0x7fa8ac159bb0 2024-04-25T10:02:40.340 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 msgr2=0x7fa88c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 0x7fa88c069450 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009a30 tx=0x7fa89c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 msgr2=0x7fa8ac15a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001360 tx=0x7fa8a400e530 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 shutdown_connections 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa88c066f90 0x7fa88c069450 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8ac1ea640 0x7fa8ac1eaac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8ac159bb0 0x7fa8ac15a010 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 --2- 172.21.15.82:0/873264653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8ac1597d0 0x7fa8ac0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 >> 172.21.15.82:0/873264653 conn(0x7fa8ac0b30b0 msgr2=0x7fa8ac15bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 shutdown_connections 2024-04-25T10:02:40.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:40.336+0000 7fa8b1bf6640 1 -- 172.21.15.82:0/873264653 wait complete. 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:40.359 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:40.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.360 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:40.365 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42509sec preferred_lft 42509sec 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:02:40.366 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.367 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:40.367 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.367 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:40.367 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.367 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:40.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.367 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:40.367 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:40.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:40.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:40.368 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set ceph-brx up 2024-04-25T10:02:40.368 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:40.368 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:40.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.448 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:40.498 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:02:40.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.500 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:02:40.554 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:40.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:40.556 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.631 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:02:40.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.678 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-25T10:02:40.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.727 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:40.727 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:40.727 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:40.728 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:02:40.728 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:40.795 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:02:40.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.796 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:40.796 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:40.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:02:40.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:40.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:02:40.797 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:02:40.797 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:02:40.797 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:40.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.894 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:40.895 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:40.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:02:40.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:40.895 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:40.918 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:02:40.918 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:40.919 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.970 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:40.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:40.971 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:41.018 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:41.073 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:41.122 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:41.123 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:41.124 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:41.125 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:41.126 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:41.127 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.127 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.127 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:41.170 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:41.171 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:02:41.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:41.235 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:41.283 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:41.284 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.285 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:41.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:02:41.292+0000 7f5f26df8580 -1 init, newargv = 0x561852a43f30 newargc=15 2024-04-25T10:02:41.299 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[36157]: starting ceph client 2024-04-25T10:02:41.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[36157]: starting fuse 2024-04-25T10:02:42.404 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:42.420 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:42.467 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:42.467 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:42.467 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:42.467 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:42.467 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:42.468 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:42.469 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.470 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:42.518 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:02:42.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:42.518 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:42.518 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:02:42.519 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:02:42.520 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:02:42.649 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.36157.asok 2024-04-25T10:02:42.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:42.660 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36157.asok 2024-04-25T10:02:42.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36157.asok status 2024-04-25T10:02:42.815 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:02:42.815 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:02:42.815 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "36157", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4785, 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 4785 2024-04-25T10:02:42.816 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2908681058 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2908681058 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.4785 192.168.0.1:0/2908681058", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2908681058", 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:02:42.817 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 8, 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 25, 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:02:42.818 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:02:42.829 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 Processor -- start 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 -- start start 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40f015da80 0x7f40f00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f006be60 con 0x7f40f00d1cd0 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40f006c440 con 0x7f40f015da80 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f831c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f015de60 con 0x7f40f00d07f0 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f60b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40f015da80 0x7f40f00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f58b7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f40f00d1cd0 0x7f40f00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:46824/0 (socket says 172.21.15.82:46824) 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f60b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40f015da80 0x7f40f00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42166/0 (socket says 172.21.15.82:42166) 2024-04-25T10:02:42.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.948+0000 7f40f58b7640 1 -- 172.21.15.82:0/2566413928 learned_addr learned my addr 172.21.15.82:0/2566413928 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f68b9640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f60b8640 1 -- 172.21.15.82:0/2566413928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 msgr2=0x7f40f006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f60b8640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f60b8640 1 -- 172.21.15.82:0/2566413928 >> v1:172.21.15.82:6789/0 conn(0x7f40f00d1cd0 legacy=0x7f40f00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f60b8640 1 -- 172.21.15.82:0/2566413928 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f00d2450 con 0x7f40f015da80 2024-04-25T10:02:42.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f50b6640 1 -- 172.21.15.82:0/2566413928 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661169869 0 0) 0x7f40f006be60 con 0x7f40f00d1cd0 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f60b8640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f015da80 0x7f40f00d4230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0097f0 tx=0x7f40ec01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d3aba7f4096050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f50b6640 1 -- 172.21.15.82:0/2566413928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0b1070 con 0x7f40f015da80 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f50b6640 1 -- 172.21.15.82:0/2566413928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec0ac630 con 0x7f40f015da80 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f50b6640 1 -- 172.21.15.82:0/2566413928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0a8050 con 0x7f40f015da80 2024-04-25T10:02:42.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 -- 172.21.15.82:0/2566413928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f015da80 msgr2=0x7f40f00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f015da80 0x7f40f00d4230 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0097f0 tx=0x7f40ec01c470 comp rx=0 tx=0).stop 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 -- 172.21.15.82:0/2566413928 shutdown_connections 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 --2- 172.21.15.82:0/2566413928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f015da80 0x7f40f00d4230 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 -- 172.21.15.82:0/2566413928 >> 172.21.15.82:0/2566413928 conn(0x7f40f008b000 msgr2=0x7f40f00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 -- 172.21.15.82:0/2566413928 shutdown_connections 2024-04-25T10:02:42.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.952+0000 7f40f831c640 1 -- 172.21.15.82:0/2566413928 wait complete. 2024-04-25T10:02:42.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 Processor -- start 2024-04-25T10:02:42.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- start start 2024-04-25T10:02:42.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f0155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f015da80 0x7f40f01607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f006d7a0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f006d620 con 0x7f40f015da80 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f006d920 con 0x7f40f00d07f0 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42180/0 (socket says 172.21.15.82:42180) 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 -- 172.21.15.82:0/4048000119 learned_addr learned my addr 172.21.15.82:0/4048000119 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f60b8640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f0155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f68b9640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f015da80 0x7f40f01607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 -- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 msgr2=0x7f40f0155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f0155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 -- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f015da80 msgr2=0x7f40f01607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f015da80 0x7f40f01607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 -- 172.21.15.82:0/4048000119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f0154650 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f58b7640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c700 tx=0x7f40e000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0007480 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e00057b0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f0154930 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e0005ba0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f0154ef0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f004efa0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.956+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40e0025430 con 0x7f40f00d1cd0 2024-04-25T10:02:42.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.960+0000 7f40e6ffd640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 0x7f40d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:42.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.960+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f40e00859f0 con 0x7f40f00d1cd0 2024-04-25T10:02:42.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.960+0000 7f40f60b8640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 0x7f40d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:42.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.960+0000 7f40f60b8640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 0x7f40d4069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0ae000 tx=0x7f40ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:42.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:42.964+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40e00567a0 con 0x7f40f00d1cd0 2024-04-25T10:02:43.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.156+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f40f01550c0 con 0x7f40f00d1cd0 2024-04-25T10:02:43.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.156+0000 7f40e6ffd640 1 -- 172.21.15.82:0/4048000119 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f40e00560a0 con 0x7f40f00d1cd0 2024-04-25T10:02:43.164 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 msgr2=0x7f40d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 0x7f40d4069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0ae000 tx=0x7f40ec008040 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 msgr2=0x7f40f0142ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f40e000c700 tx=0x7f40e000cb00 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 shutdown_connections 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40d4066f60 0x7f40d4069420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40f015da80 0x7f40f01607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40f00d1cd0 0x7f40f0142ec0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 --2- 172.21.15.82:0/4048000119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40f00d07f0 0x7f40f0155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 >> 172.21.15.82:0/4048000119 conn(0x7f40f008b000 msgr2=0x7f40f00b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 shutdown_connections 2024-04-25T10:02:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:43.160+0000 7f40f831c640 1 -- 172.21.15.82:0/4048000119 wait complete. 2024-04-25T10:02:43.183 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:43.184 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:43.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.184 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:43.191 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42507sec preferred_lft 42507sec 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:43.192 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.193 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:43.242 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:02:43.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.244 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:02:43.294 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:02:43.294 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:43.294 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:43.294 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:43.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.295 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:43.295 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:43.295 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:43.296 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:43.296 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:43.296 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:43.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.373 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:02:43.419 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:43.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.419 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list-id 2024-04-25T10:02:43.468 INFO:teuthology.orchestra.run.smithi082.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:02:43.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.469 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:43.469 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:43.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:43.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:02:43.469 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:43.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.539 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.595 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:43.596 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether be:1b:41:ae:d3:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:43.596 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:43.596 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.596 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::bc1b:41ff:feae:d321/64 scope link 2024-04-25T10:02:43.596 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.597 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:43.650 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether be:1b:41:ae:d3:21 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::bc1b:41ff:feae:d321/64 scope link 2024-04-25T10:02:43.651 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.652 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:02:43.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:02:43.653 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:43.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.746 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:02:43.747 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:02:43.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.1 up 2024-04-25T10:02:43.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:43.747 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:02:43.767 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T10:02:43.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:43.767 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.813 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:43.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:43.814 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:43.920 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:43.970 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:43.970 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:43.970 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:43.970 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:43.970 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:43.971 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:43.972 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:43.973 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:43.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.974 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:44.023 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:02:44.024 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:02:44.024 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T10:02:44.067 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:44.082 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:44.130 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:44.131 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:44.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:44.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.133 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:44.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi082.stderr:2024-04-25T10:02:44.136+0000 7fc1d7807580 -1 init, newargv = 0x55c3c2137ce0 newargc=15 2024-04-25T10:02:44.145 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi082.stderr:ceph-fuse[36298]: starting ceph client 2024-04-25T10:02:44.149 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:02:44.152 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi082.stderr:ceph-fuse[36298]: starting fuse 2024-04-25T10:02:45.251 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:02:45.268 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:45.315 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:02:45.315 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:02:45.315 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:02:45.315 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:02:45.316 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:45.317 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:45.318 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:45.318 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:45.318 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.319 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:02:45.366 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:02:45.367 INFO:teuthology.orchestra.run.smithi082.stdout:46 2024-04-25T10:02:45.367 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T10:02:45.367 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:45.368 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:02:45.369 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.2")) 2024-04-25T10:02:45.369 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:02:45.491 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.2.36298.asok 2024-04-25T10:02:45.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:45.503 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36298.asok 2024-04-25T10:02:45.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.503 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36298.asok status 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "2", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "36298", 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:02:45.660 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 4800, 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 4800 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4094773729 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:02:45.661 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4094773729 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.4800 192.168.0.2:0/4094773729", 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.2:0/4094773729", 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 8, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 25, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:02:45.662 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:02:45.663 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:02:45.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:45.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 Processor -- start 2024-04-25T10:02:45.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 -- start start 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efcc006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc00ccc70 con 0x7efcc006ec60 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc006dc40 con 0x7efcc00ce510 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcc567e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00cc2b0 con 0x7efcc0144ca0 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efcc006ec60 0x7efcc0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48216/0 (socket says 172.21.15.82:48216) 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 -- 172.21.15.82:0/3676271012 learned_addr learned my addr 172.21.15.82:0/3676271012 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbe7fc640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efcc006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767117514 0 0) 0x7efcc00ccc70 con 0x7efcc006ec60 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efca8003530 con 0x7efcc006ec60 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2956133322 0 0) 0x7efca8003530 con 0x7efcc006ec60 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 msgr2=0x7efcc006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efcc006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 >> v1:172.21.15.196:6789/0 conn(0x7efcc00ce510 legacy=0x7efcc00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc006df20 con 0x7efcc006ec60 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7efcb8001a00 con 0x7efcc006ec60 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 >> v1:172.21.15.82:6789/0 conn(0x7efcc006ec60 legacy=0x7efcc0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efca8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8003fe0 0x7efca8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efca8004d50 0x7efca80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc00cc2b0 con 0x7efca8004d50 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca800bbc0 con 0x7efca8003fe0 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca800bdc0 con 0x7efcc0144ca0 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efca8004d50 0x7efca80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbe7fc640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8003fe0 0x7efca8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbeffd640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efca8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 -- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 msgr2=0x7efca8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc0144ca0 0x7efca8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 -- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8003fe0 msgr2=0x7efca8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8003fe0 0x7efca8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.792+0000 7efcbf7fe640 1 -- 172.21.15.82:0/3676271012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca8003530 con 0x7efca8004d50 2024-04-25T10:02:45.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efca8004d50 0x7efca80051d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcc004f950 tx=0x7efcb801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18bbe94eceb538dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb800a4a0 con 0x7efca8004d50 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb80ad650 con 0x7efca8004d50 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbdffb640 1 -- 172.21.15.82:0/3676271012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80ada80 con 0x7efca8004d50 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- 172.21.15.82:0/3676271012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efca8004d50 msgr2=0x7efca80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 --2- 172.21.15.82:0/3676271012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efca8004d50 0x7efca80051d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7efcc004f950 tx=0x7efcb801d340 comp rx=0 tx=0).stop 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbf7fe640 1 -- 172.21.15.82:0/3676271012 reap_dead start 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- 172.21.15.82:0/3676271012 shutdown_connections 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- 172.21.15.82:0/3676271012 >> 172.21.15.82:0/3676271012 conn(0x7efcc00b11a0 msgr2=0x7efcc00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- 172.21.15.82:0/3676271012 shutdown_connections 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- 172.21.15.82:0/3676271012 wait complete. 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 Processor -- start 2024-04-25T10:02:45.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- start start 2024-04-25T10:02:45.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 0x7efcc015acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc00c4410 0x7efcc00b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0142d90 con 0x7efcc00c4890 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc006e290 con 0x7efca8004d50 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcc567e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc0142f10 con 0x7efcc00c4410 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.796+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39960/0 (socket says 172.21.15.82:39960) 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 -- 172.21.15.82:0/2536693568 learned_addr learned my addr 172.21.15.82:0/2536693568 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbe7fc640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc00c4410 0x7efcc00b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbeffd640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 0x7efcc015acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 -- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc00c4410 msgr2=0x7efcc00b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc00c4410 0x7efcc00b25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 -- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 msgr2=0x7efcc015acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 0x7efcc015acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 -- 172.21.15.82:0/2536693568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc0157000 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbeffd640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 0x7efcc015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbf7fe640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcb801dbe0 tx=0x7efca8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb80b2040 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb80bc070 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc00adb90 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8004b70 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc00b3170 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc004efa0 con 0x7efcc00c4890 2024-04-25T10:02:45.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcb8004d10 con 0x7efcc00c4890 2024-04-25T10:02:45.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 0x7efcac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efcb811fbf0 con 0x7efcc00c4890 2024-04-25T10:02:45.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbeffd640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 0x7efcac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.800+0000 7efcbeffd640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 0x7efcac069450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcb00016a0 tx=0x7efcb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.804+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcb8017050 con 0x7efcc00c4890 2024-04-25T10:02:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.992+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efcc00b1cc0 con 0x7efcc00c4890 2024-04-25T10:02:46.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.992+0000 7efca77fe640 1 -- 172.21.15.82:0/2536693568 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7efcb800a540 con 0x7efcc00c4890 2024-04-25T10:02:46.001 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:46.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 msgr2=0x7efcac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 0x7efcac069450 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efcb00016a0 tx=0x7efcb0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 msgr2=0x7efcc00b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7efcb801dbe0 tx=0x7efca8005840 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 shutdown_connections 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efcac066f90 0x7efcac069450 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efcc00c4410 0x7efcc00b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efcc00c4890 0x7efcc00b2ae0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 --2- 172.21.15.82:0/2536693568 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efca8004d50 0x7efcc015acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 >> 172.21.15.82:0/2536693568 conn(0x7efcc00b11a0 msgr2=0x7efcc0141720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 shutdown_connections 2024-04-25T10:02:46.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:45.996+0000 7efcc567e640 1 -- 172.21.15.82:0/2536693568 wait complete. 2024-04-25T10:02:46.018 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:46.018 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:46.018 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:46.019 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:46.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.019 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:46.026 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:46.027 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42505sec preferred_lft 42505sec 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:4a:fd:50:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.028 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:46.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.028 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:46.028 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:46.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:46.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:46.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set ceph-brx up 2024-04-25T10:02:46.029 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:46.029 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:46.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.113 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:46.166 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T10:02:46.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.168 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-25T10:02:46.230 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-25T10:02:46.230 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:46.231 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:46.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:46.232 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:46.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.305 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T10:02:46.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.354 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-25T10:02:46.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.402 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:46.402 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:46.402 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:46.402 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:02:46.402 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:46.474 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:02:46.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:02:46.474 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:46.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.579 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:46.579 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:46.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.0 up 2024-04-25T10:02:46.579 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:46.579 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:46.598 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T10:02:46.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:46.598 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:46.646 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:46.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:46.647 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:46.693 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:46.749 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:46.798 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:46.799 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:46.800 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:46.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.801 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:46.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:46.850 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T10:02:46.895 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:46.911 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:46.957 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:46.958 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:46.959 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:46.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.961 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:46.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:2024-04-25T10:02:46.967+0000 7ff84a803580 -1 init, newargv = 0x55ce232bcfb0 newargc=15 2024-04-25T10:02:46.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33502]: starting ceph client 2024-04-25T10:02:46.980 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi196.stderr:ceph-fuse[33502]: starting fuse 2024-04-25T10:02:48.079 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:48.095 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:48.142 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:48.143 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:48.144 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:48.145 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:48.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.145 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:48.197 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T10:02:48.198 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T10:02:48.198 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-04-25T10:02:48.199 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:48.200 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:48.200 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.1")) 2024-04-25T10:02:48.200 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-04-25T10:02:48.324 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.1.33502.asok 2024-04-25T10:02:48.328 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:02:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:48.336 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33502.asok 2024-04-25T10:02:48.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.336 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33502.asok status 2024-04-25T10:02:48.483 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T10:02:48.483 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-04-25T10:02:48.483 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:48.483 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "1", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33502", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4654, 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4654 2024-04-25T10:02:48.484 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2984791025 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 2984791025 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4654 192.168.0.1:0/2984791025", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.1:0/2984791025", 2024-04-25T10:02:48.485 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 8, 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 25, 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-04-25T10:02:48.486 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T10:02:48.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.628+0000 7f9c3d3bb640 1 Processor -- start 2024-04-25T10:02:48.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.628+0000 7f9c3d3bb640 1 -- start start 2024-04-25T10:02:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c3d3bb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c3815b950 0x7f9c3809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c3d3bb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 0x7f9c380cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c3d3bb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3815bd30 con 0x7f9c380cfb80 2024-04-25T10:02:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c3d3bb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c380cc0d0 con 0x7f9c3815b950 2024-04-25T10:02:48.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c3d3bb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3809ab10 con 0x7f9c380cf420 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c3815b950 0x7f9c3809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c367fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 0x7f9c380cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c367fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 0x7f9c380cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:53818/0 (socket says 172.21.15.82:53818) 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9c3815b950 0x7f9c3809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:48122/0 (socket says 172.21.15.82:48122) 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c367fc640 1 -- 172.21.15.82:0/4263800495 learned_addr learned my addr 172.21.15.82:0/4263800495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 -- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 msgr2=0x7f9c380cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 --2- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 0x7f9c380cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 -- 172.21.15.82:0/4263800495 >> v1:172.21.15.196:6789/0 conn(0x7f9c380cfb80 legacy=0x7f9c3806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.632+0000 7f9c36ffd640 1 -- 172.21.15.82:0/4263800495 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c380cc370 con 0x7f9c3815b950 2024-04-25T10:02:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c3815b950 0x7f9c3809a480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009d90 tx=0x7f9c3001c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1534f72c4104b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c35ffb640 1 -- 172.21.15.82:0/4263800495 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300b2070 con 0x7f9c3815b950 2024-04-25T10:02:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c35ffb640 1 -- 172.21.15.82:0/4263800495 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300a5910 con 0x7f9c3815b950 2024-04-25T10:02:48.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c35ffb640 1 -- 172.21.15.82:0/4263800495 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300a5ce0 con 0x7f9c3815b950 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c3815b950 msgr2=0x7f9c3809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c3815b950 0x7f9c3809a480 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009d90 tx=0x7f9c3001c940 comp rx=0 tx=0).stop 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/4263800495 shutdown_connections 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c380cf420 0x7f9c380cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/4263800495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c3815b950 0x7f9c3809a480 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/4263800495 >> 172.21.15.82:0/4263800495 conn(0x7f9c380b30b0 msgr2=0x7f9c380b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/4263800495 shutdown_connections 2024-04-25T10:02:48.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/4263800495 wait complete. 2024-04-25T10:02:48.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 Processor -- start 2024-04-25T10:02:48.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- start start 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c380cfb80 0x7f9c38154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c3815b950 0x7f9c38154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380cd390 con 0x7f9c380cf420 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c380d03a0 con 0x7f9c380cfb80 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c3d3bb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3815a9d0 con 0x7f9c3815b950 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39966/0 (socket says 172.21.15.82:39966) 2024-04-25T10:02:48.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 -- 172.21.15.82:0/3147646957 learned_addr learned my addr 172.21.15.82:0/3147646957 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c367fc640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c380cfb80 0x7f9c38154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 -- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c3815b950 msgr2=0x7f9c38154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c377fe640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c3815b950 0x7f9c38154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c3815b950 0x7f9c38154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 -- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c380cfb80 msgr2=0x7f9c38154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c380cfb80 0x7f9c38154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 -- 172.21.15.82:0/3147646957 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c38159dc0 con 0x7f9c380cf420 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.636+0000 7f9c36ffd640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b0040 tx=0x7f9c300196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300b2040 con 0x7f9c380cf420 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300bf070 con 0x7f9c380cf420 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3815a0a0 con 0x7f9c380cf420 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300b88b0 con 0x7f9c380cf420 2024-04-25T10:02:48.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3815a660 con 0x7f9c380cf420 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3804efa0 con 0x7f9c380cf420 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c300b8a50 con 0x7f9c380cf420 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 0x7f9c18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9c30120770 con 0x7f9c380cf420 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c367fc640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 0x7f9c18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:48.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.640+0000 7f9c367fc640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 0x7f9c18069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c380b2360 tx=0x7f9c20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:48.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.644+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c30016560 con 0x7f9c380cf420 2024-04-25T10:02:48.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.848+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9c380d13a0 con 0x7f9c380cf420 2024-04-25T10:02:48.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.848+0000 7f9c2f7fe640 1 -- 172.21.15.82:0/3147646957 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9c300f1480 con 0x7f9c380cf420 2024-04-25T10:02:48.855 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 msgr2=0x7f9c18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 0x7f9c18069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c380b2360 tx=0x7f9c20008040 comp rx=0 tx=0).stop 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 msgr2=0x7f9c38153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9c300b0040 tx=0x7f9c300196e0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 shutdown_connections 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c18066f60 0x7f9c18069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c3815b950 0x7f9c38154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c380cfb80 0x7f9c38154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 --2- 172.21.15.82:0/3147646957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c380cf420 0x7f9c38153900 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:48.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 >> 172.21.15.82:0/3147646957 conn(0x7f9c380b30b0 msgr2=0x7f9c3806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:48.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 shutdown_connections 2024-04-25T10:02:48.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:48.852+0000 7f9c3d3bb640 1 -- 172.21.15.82:0/3147646957 wait complete. 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi196.front.sepia.ceph.com 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:48.875 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:48.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.876 DEBUG:teuthology.orchestra.run.smithi196:> ip addr 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:be brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f0 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:d9:38:bf brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.883 INFO:teuthology.orchestra.run.smithi196.stdout: altname enp7s0f1 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:46 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0c:c4:7a:88:7f:47 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.21.15.196/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft 42502sec preferred_lft 42502sec 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::ec4:7aff:fe88:7f47/64 scope link 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 02:42:4a:fd:50:50 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:48.884 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:48.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.885 DEBUG:teuthology.orchestra.run.smithi196:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:48.937 INFO:teuthology.orchestra.run.smithi196.stdout:1 2024-04-25T10:02:48.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.939 DEBUG:teuthology.orchestra.run.smithi196:> route 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:Kernel IP routing table 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:02:48.990 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:48.991 INFO:teuthology.orchestra.run.smithi196.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:48.991 INFO:teuthology.orchestra.run.smithi196.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:48.991 INFO:teuthology.orchestra.run.smithi196.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:02:48.991 INFO:teuthology.orchestra.run.smithi196.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:48.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:48.991 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:49.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.065 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T10:02:49.114 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:02:49.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.115 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list-id 2024-04-25T10:02:49.161 INFO:teuthology.orchestra.run.smithi196.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:02:49.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.162 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:49.162 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:49.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:49.162 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:02:49.162 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:49.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.229 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:49.287 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:49.287 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether fa:17:ac:4e:98:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::f817:acff:fe4e:9810/64 scope link 2024-04-25T10:02:49.288 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.289 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.345 INFO:teuthology.orchestra.run.smithi196.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:49.346 INFO:teuthology.orchestra.run.smithi196.stdout: link/ether fa:17:ac:4e:98:10 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:49.346 INFO:teuthology.orchestra.run.smithi196.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:49.346 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.346 INFO:teuthology.orchestra.run.smithi196.stdout: inet6 fe80::f817:acff:fe4e:9810/64 scope link 2024-04-25T10:02:49.346 INFO:teuthology.orchestra.run.smithi196.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:49.347 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:02:49.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.347 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:49.347 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:49.347 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:02:49.347 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:49.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:02:49.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:02:49.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:02:49.348 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:49.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.438 DEBUG:teuthology.orchestra.run.smithi196:> (cd / && exec bash -c ' 2024-04-25T10:02:49.438 DEBUG:teuthology.orchestra.run.smithi196:> set -e 2024-04-25T10:02:49.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set brx.1 up 2024-04-25T10:02:49.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:49.439 DEBUG:teuthology.orchestra.run.smithi196:> ') 2024-04-25T10:02:49.457 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T10:02:49.458 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:49.458 DEBUG:teuthology.orchestra.run.smithi196:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:49.505 INFO:teuthology.orchestra.run.smithi196.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:49.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:49.505 DEBUG:teuthology.orchestra.run.smithi196:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:49.552 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:49.610 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:49.657 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:49.658 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:49.659 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:49.660 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:49.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.661 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T10:02:49.710 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T10:02:49.710 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T10:02:49.754 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:49.768 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:49.814 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:49.815 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:49.816 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:49.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.817 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:49.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:2024-04-25T10:02:49.823+0000 7f41a3a0b580 -1 init, newargv = 0x55dd2cbd36c0 newargc=15 2024-04-25T10:02:49.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33617]: starting ceph client 2024-04-25T10:02:49.837 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T10:02:49.839 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi196.stderr:ceph-fuse[33617]: starting fuse 2024-04-25T10:02:50.938 DEBUG:teuthology.orchestra.run.smithi196:> sudo modprobe fuse 2024-04-25T10:02:50.955 DEBUG:teuthology.orchestra.run.smithi196:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:51.002 INFO:teuthology.orchestra.run.smithi196.stdout:/sys 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/proc 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/dev 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/pts 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/run 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/ 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/security 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/shm 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/run/lock 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/cgroup 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/pstore 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/bpf 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/hugepages 2024-04-25T10:02:51.003 INFO:teuthology.orchestra.run.smithi196.stdout:/dev/mqueue 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/debug 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/tracing 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/proc/fs/nfsd 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/sys/kernel/config 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/run/rpc_pipefs 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/run/user/1000 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T10:02:51.004 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:51.005 INFO:teuthology.orchestra.run.smithi196.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.006 DEBUG:teuthology.orchestra.run.smithi196:> ls /sys/fs/fuse/connections 2024-04-25T10:02:51.053 INFO:teuthology.orchestra.run.smithi196.stdout:45 2024-04-25T10:02:51.053 INFO:teuthology.orchestra.run.smithi196.stdout:46 2024-04-25T10:02:51.054 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> import glob 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> import re 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> import os 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> import subprocess 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> def _find_admin_socket(client_name): 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> files = glob.glob(asok_path) 2024-04-25T10:02:51.054 DEBUG:teuthology.orchestra.run.smithi196:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> # Given a non-glob path, it better be there 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> if "*" not in asok_path: 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> assert(len(files) == 1) 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> return files[0] 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> for f in files: 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> contents = proc_f.read() 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> if mountpoint in contents: 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> return f 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> 2024-04-25T10:02:51.055 DEBUG:teuthology.orchestra.run.smithi196:> print(_find_admin_socket("client.3")) 2024-04-25T10:02:51.056 DEBUG:teuthology.orchestra.run.smithi196:> ' 2024-04-25T10:02:51.179 INFO:teuthology.orchestra.run.smithi196.stdout:/var/run/ceph/ceph-client.3.33617.asok 2024-04-25T10:02:51.183 INFO:teuthology.orchestra.run.smithi196.stderr:2024-04-25T10:02:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:51.190 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33617.asok 2024-04-25T10:02:51.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.191 DEBUG:teuthology.orchestra.run.smithi196:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33617.asok status 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi196.stdout:{ 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi196.stdout: "metadata": { 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:51.346 INFO:teuthology.orchestra.run.smithi196.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "entity_id": "3", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "hostname": "smithi196", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "pid": "33617", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "root": "/" 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_count": 0, 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "id": 4508, 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "inst": { 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "name": { 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "client", 2024-04-25T10:02:51.347 INFO:teuthology.orchestra.run.smithi196.stdout: "num": 4508 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1989685712 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: } 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": { 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "type": "v1", 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "addr": "192.168.0.2:0", 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "nonce": 1989685712 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: }, 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "inst_str": "client.4508 192.168.0.2:0/1989685712", 2024-04-25T10:02:51.348 INFO:teuthology.orchestra.run.smithi196.stdout: "addr_str": "192.168.0.2:0/1989685712", 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "inode_count": 1, 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "mds_epoch": 8, 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch": 27, 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "blocklisted": false, 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout: "fs_name": "cephfs" 2024-04-25T10:02:51.349 INFO:teuthology.orchestra.run.smithi196.stdout:} 2024-04-25T10:02:51.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.360 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:02:51.360 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:51.367 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:02:51.367 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:51.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.368 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:51.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.423 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:02:51.423 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:51.471 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:02:51.471 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:51.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.471 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:51.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.527 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-04-25T10:02:51.527 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:51.534 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-04-25T10:02:51.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:51.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.535 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:51.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.590 DEBUG:teuthology.orchestra.run.smithi196:> stat --file-system '--printf=%T 2024-04-25T10:02:51.590 DEBUG:teuthology.orchestra.run.smithi196:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.639 INFO:teuthology.orchestra.run.smithi196.stdout:fuseblk 2024-04-25T10:02:51.639 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.639 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.693 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:02:51.709 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:51.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.832+0000 7f1a62a19640 1 Processor -- start 2024-04-25T10:02:51.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.832+0000 7f1a62a19640 1 -- start start 2024-04-25T10:02:51.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a62a19640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a62a19640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 0x7f1a5c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a62a19640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5c0c7300 con 0x7f1a5c15ed40 2024-04-25T10:02:51.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a62a19640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c0affa0 con 0x7f1a5c0c6e50 2024-04-25T10:02:51.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a62a19640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c15f120 con 0x7f1a5c160080 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a60fb6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 0x7f1a5c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a60fb6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 0x7f1a5c0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:48144/0 (socket says 172.21.15.82:48144) 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a60fb6640 1 -- 172.21.15.82:0/2376775352 learned_addr learned my addr 172.21.15.82:0/2376775352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5bfff640 1 --1- 172.21.15.82:0/2376775352 >> v1:172.21.15.196:6789/0 conn(0x7f1a5c15ed40 0x7f1a5c0d0c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:39746/0 (socket says 172.21.15.82:39746) 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2376775352 >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 msgr2=0x7f1a5c0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:51.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 --2- 172.21.15.82:0/2376775352 >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 0x7f1a5c0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2376775352 >> v1:172.21.15.196:6789/0 conn(0x7f1a5c15ed40 legacy=0x7f1a5c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2376775352 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0b0240 con 0x7f1a5c160080 2024-04-25T10:02:51.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.836+0000 7f1a5b7fe640 1 --2- 172.21.15.82:0/2376775352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0c6910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009d90 tx=0x7f1a4401c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=918792d7f1575b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5affd640 1 -- 172.21.15.82:0/2376775352 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440b2070 con 0x7f1a5c160080 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5affd640 1 -- 172.21.15.82:0/2376775352 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440a5970 con 0x7f1a5c160080 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5affd640 1 -- 172.21.15.82:0/2376775352 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440a5d40 con 0x7f1a5c160080 2024-04-25T10:02:51.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2376775352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 msgr2=0x7f1a5c0c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2376775352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0c6910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1a44009d90 tx=0x7f1a4401c8b0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2376775352 shutdown_connections 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2376775352 >> v2:172.21.15.196:3300/0 conn(0x7f1a5c0c6e50 0x7f1a5c0caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2376775352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0c6910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2376775352 >> 172.21.15.82:0/2376775352 conn(0x7f1a5c099a70 msgr2=0x7f1a5c0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2376775352 shutdown_connections 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2376775352 wait complete. 2024-04-25T10:02:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 Processor -- start 2024-04-25T10:02:51.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- start start 2024-04-25T10:02:51.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a5c0c6e50 0x7f1a5c0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cb9a0 con 0x7f1a5c15ed40 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cb820 con 0x7f1a5c0c6e50 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0cbb20 con 0x7f1a5c160080 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39968/0 (socket says 172.21.15.82:39968) 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2361247560 learned_addr learned my addr 172.21.15.82:0/2361247560 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5bfff640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a5c0c6e50 0x7f1a5c0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a60fb6640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 msgr2=0x7f1a5c0b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0b5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a5c0c6e50 msgr2=0x7f1a5c0ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a5c0c6e50 0x7f1a5c0ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:51.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 -- 172.21.15.82:0/2361247560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c0ca880 con 0x7f1a5c15ed40 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a5b7fe640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a4401a4e0 tx=0x7f1a44019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440bd020 con 0x7f1a5c15ed40 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a440b8670 con 0x7f1a5c15ed40 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5c0b4f30 con 0x7f1a5c15ed40 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a440b2040 con 0x7f1a5c15ed40 2024-04-25T10:02:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.840+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c0b4c50 con 0x7f1a5c15ed40 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5c04efa0 con 0x7f1a5c15ed40 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a440c0020 con 0x7f1a5c15ed40 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a58ff9640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f1a44120500 con 0x7f1a5c15ed40 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a5bfff640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.844+0000 7f1a5bfff640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a54000f70 tx=0x7f1a54007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:51.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:51.848+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a440c7030 con 0x7f1a5c15ed40 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.008+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a5c15b630 con 0x7f1a5c15ed40 2024-04-25T10:02:52.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.012+0000 7f1a58ff9640 1 -- 172.21.15.82:0/2361247560 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f1a5c0ca880 con 0x7f1a5c15ed40 2024-04-25T10:02:52.019 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:52.020 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:52.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.012+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 msgr2=0x7f1a3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.012+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 0x7f1a3c069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1a54000f70 tx=0x7f1a54007cf0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.012+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 msgr2=0x7f1a5c14afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1a4401a4e0 tx=0x7f1a44019650 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 shutdown_connections 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1a3c066f60 0x7f1a3c069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1a5c160080 0x7f1a5c0b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1a5c15ed40 0x7f1a5c14afc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 --2- 172.21.15.82:0/2361247560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1a5c0c6e50 0x7f1a5c0ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 >> 172.21.15.82:0/2361247560 conn(0x7f1a5c099a70 msgr2=0x7f1a5c06e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 shutdown_connections 2024-04-25T10:02:52.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.016+0000 7f1a62a19640 1 -- 172.21.15.82:0/2361247560 wait complete. 2024-04-25T10:02:52.039 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:52.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.156+0000 7f9b3347f640 1 Processor -- start 2024-04-25T10:02:52.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.156+0000 7f9b3347f640 1 -- start start 2024-04-25T10:02:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3347f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3347f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 0x7f9b2c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3347f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b2c0cbd20 con 0x7f9b2c15f130 2024-04-25T10:02:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3347f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2c15ba90 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3347f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cff30 con 0x7f9b2c0cf590 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b30a1a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 0x7f9b2c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:48172/0 (socket says 172.21.15.82:48172) 2024-04-25T10:02:52.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 -- 172.21.15.82:0/2672818378 learned_addr learned my addr 172.21.15.82:0/2672818378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:52.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 -- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 msgr2=0x7f9b2c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 0x7f9b2c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 -- 172.21.15.82:0/2672818378 >> v1:172.21.15.196:6789/0 conn(0x7f9b2c15f130 legacy=0x7f9b2c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b3121b640 1 -- 172.21.15.82:0/2672818378 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0cc000 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.160+0000 7f9b30a1a640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 0x7f9b2c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3121b640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009e50 tx=0x7f9b2401ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0d9a1b45cb0f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b2bfff640 1 -- 172.21.15.82:0/2672818378 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b240a58d0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b2bfff640 1 -- 172.21.15.82:0/2672818378 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b240a5a70 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b2bfff640 1 -- 172.21.15.82:0/2672818378 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b240a5e40 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c15b6b0 msgr2=0x7f9b2c0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009e50 tx=0x7f9b2401ca00 comp rx=0 tx=0).stop 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- 172.21.15.82:0/2672818378 shutdown_connections 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c0cf590 0x7f9b2c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- 172.21.15.82:0/2672818378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0ca3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- 172.21.15.82:0/2672818378 >> 172.21.15.82:0/2672818378 conn(0x7f9b2c06ae60 msgr2=0x7f9b2c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- 172.21.15.82:0/2672818378 shutdown_connections 2024-04-25T10:02:52.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- 172.21.15.82:0/2672818378 wait complete. 2024-04-25T10:02:52.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 Processor -- start 2024-04-25T10:02:52.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- start start 2024-04-25T10:02:52.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 0x7f9b2c1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c15f130 0x7f9b2c0ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0ca900 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cba30 con 0x7f9b2c0cf590 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3347f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c0cadb0 con 0x7f9b2c15f130 2024-04-25T10:02:52.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39970/0 (socket says 172.21.15.82:39970) 2024-04-25T10:02:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 -- 172.21.15.82:0/1358613130 learned_addr learned my addr 172.21.15.82:0/1358613130 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3121b640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 0x7f9b2c1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b31a1c640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c15f130 0x7f9b2c0ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 -- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c15f130 msgr2=0x7f9b2c0ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c15f130 0x7f9b2c0ce0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 -- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 msgr2=0x7f9b2c1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 0x7f9b2c1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 -- 172.21.15.82:0/1358613130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c0984c0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b3121b640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 0x7f9b2c1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:52.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.164+0000 7f9b30a1a640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ec10 tx=0x7f9b2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20017070 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2000cc30 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c0c76a0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b20013470 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c0c7940 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c04efa0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b2001a070 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9b200853c0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b3121b640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.168+0000 7f9b3121b640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 0x7f9b18069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b240ae000 tx=0x7f9b240a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.172+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b20055fb0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b2c0b1a40 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b29ffb640 1 -- 172.21.15.82:0/1358613130 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b200558b0 con 0x7f9b2c15b6b0 2024-04-25T10:02:52.339 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 msgr2=0x7f9b18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 0x7f9b18069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9b240ae000 tx=0x7f9b240a7070 comp rx=0 tx=0).stop 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 msgr2=0x7f9b2c0988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9b2000ec10 tx=0x7f9b2000c520 comp rx=0 tx=0).stop 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 shutdown_connections 2024-04-25T10:02:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9b18066f60 0x7f9b18069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b2c15f130 0x7f9b2c0ce0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b2c15b6b0 0x7f9b2c0988c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 --2- 172.21.15.82:0/1358613130 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b2c0cf590 0x7f9b2c1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 >> 172.21.15.82:0/1358613130 conn(0x7f9b2c06ae60 msgr2=0x7f9b2c0b3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 shutdown_connections 2024-04-25T10:02:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.332+0000 7f9b3347f640 1 -- 172.21.15.82:0/1358613130 wait complete. 2024-04-25T10:02:52.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.464+0000 7f9eaafc7640 1 Processor -- start 2024-04-25T10:02:52.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.464+0000 7f9eaafc7640 1 -- start start 2024-04-25T10:02:52.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9eaafc7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 0x7f9ea4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9eaafc7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40cfe90 con 0x7f9ea415b6b0 2024-04-25T10:02:52.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9eaafc7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40cbd20 con 0x7f9ea40cf590 2024-04-25T10:02:52.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9eaafc7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415ba90 con 0x7f9ea40cfa20 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea8d63640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9ea415b6b0 0x7f9ea40ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48226/0 (socket says 172.21.15.82:48226) 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea8d63640 1 -- 172.21.15.82:0/1308113219 learned_addr learned my addr 172.21.15.82:0/1308113219 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2738461994 0 0) 0x7f9ea40cfe90 con 0x7f9ea415b6b0 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e8c003530 con 0x7f9ea415b6b0 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea9564640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 0x7f9ea4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 849031713 0 0) 0x7f9e8c003530 con 0x7f9ea415b6b0 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 msgr2=0x7f9ea4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 0x7f9ea4072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 >> v1:172.21.15.196:6789/0 conn(0x7f9ea40cf590 legacy=0x7f9ea4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40cc000 con 0x7f9ea415b6b0 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9e94001680 con 0x7f9ea415b6b0 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 >> v1:172.21.15.82:6789/0 conn(0x7f9ea415b6b0 legacy=0x7f9ea40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 0x7f9e8c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8c004090 0x7f9e8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e8c004da0 0x7f9e8c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea415ba90 con 0x7f9e8c004090 2024-04-25T10:02:52.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c00bbe0 con 0x7f9e8c004da0 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c00bde0 con 0x7f9ea40cfa20 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8c004090 0x7f9e8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 -- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 msgr2=0x7f9e8c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ea40cfa20 0x7f9e8c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 -- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e8c004da0 msgr2=0x7f9e8c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e8c004da0 0x7f9e8c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 -- 172.21.15.82:0/1308113219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c003530 con 0x7f9e8c004090 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea3fff640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8c004090 0x7f9e8c004810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001780 tx=0x7f9e9801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fda21cf945bfaec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.468+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980013a0 con 0x7f9e8c004090 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e980a9070 con 0x7f9e8c004090 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea37fe640 1 -- 172.21.15.82:0/1308113219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98003600 con 0x7f9e8c004090 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/1308113219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8c004090 msgr2=0x7f9e8c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/1308113219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e8c004090 0x7f9e8c004810 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001780 tx=0x7f9e9801c420 comp rx=0 tx=0).stop 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 -- 172.21.15.82:0/1308113219 reap_dead start 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/1308113219 shutdown_connections 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/1308113219 >> 172.21.15.82:0/1308113219 conn(0x7f9ea406ae60 msgr2=0x7f9ea406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/1308113219 shutdown_connections 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/1308113219 wait complete. 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 Processor -- start 2024-04-25T10:02:52.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- start start 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e8c004090 0x7f9ea4098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ea40ad1c0 0x7f9ea40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cc8a0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cc5c0 con 0x7f9ea40ad1c0 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cba30 con 0x7f9e8c004090 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39986/0 (socket says 172.21.15.82:39986) 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 -- 172.21.15.82:0/3718588170 learned_addr learned my addr 172.21.15.82:0/3718588170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea3fff640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ea40ad1c0 0x7f9ea40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea8d63640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e8c004090 0x7f9ea4098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 -- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e8c004090 msgr2=0x7f9ea4098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e8c004090 0x7f9ea4098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 -- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ea40ad1c0 msgr2=0x7f9ea40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ea40ad1c0 0x7f9ea40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 -- 172.21.15.82:0/3718588170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40adf40 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea9564640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae740 tx=0x7f9e9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00ed90 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c019070 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea40ae220 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c014660 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.472+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea415caa0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea415cca0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e9c014800 con 0x7f9ea415cfc0 2024-04-25T10:02:52.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9ea17fa640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 0x7f9e90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9e9c085900 con 0x7f9ea415cfc0 2024-04-25T10:02:52.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9ea8d63640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 0x7f9e90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.476+0000 7f9ea8d63640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 0x7f9e90069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e94004880 tx=0x7f9e940014e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.480+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ea415cca0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.628+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9ea404efa0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.628+0000 7f9ea17fa640 1 -- 172.21.15.82:0/3718588170 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e9c0564f0 con 0x7f9ea415cfc0 2024-04-25T10:02:52.636 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:52.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 msgr2=0x7f9e90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 0x7f9e90069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9e94004880 tx=0x7f9e940014e0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 msgr2=0x7f9ea415d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40ae740 tx=0x7f9e9c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 shutdown_connections 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e90066f90 0x7f9e90069450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9ea415cfc0 0x7f9ea415d440 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ea40ad1c0 0x7f9ea40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 --2- 172.21.15.82:0/3718588170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e8c004090 0x7f9ea4098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 >> 172.21.15.82:0/3718588170 conn(0x7f9ea406ae60 msgr2=0x7f9ea4155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 shutdown_connections 2024-04-25T10:02:52.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.632+0000 7f9eaafc7640 1 -- 172.21.15.82:0/3718588170 wait complete. 2024-04-25T10:02:52.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:52.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 Processor -- start 2024-04-25T10:02:52.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 -- start start 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb7300cf590 0x7fb7300cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300cbd20 con 0x7fb730072000 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300cc000 con 0x7fb73015b6b0 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb7351c1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73015ba90 con 0x7fb7300cf590 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72f59e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb730072000 0x7fb730072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48238/0 (socket says 172.21.15.82:48238) 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72f59e640 1 -- 172.21.15.82:0/1241604986 learned_addr learned my addr 172.21.15.82:0/1241604986 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb727fff640 1 --2- 172.21.15.82:0/1241604986 >> v2:172.21.15.196:3300/0 conn(0x7fb7300cf590 0x7fb7300cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344732191 0 0) 0x7fb7300cbd20 con 0x7fb730072000 2024-04-25T10:02:52.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb71c003530 con 0x7fb730072000 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621511007 0 0) 0x7fb71c003530 con 0x7fb730072000 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 >> v2:172.21.15.196:3300/0 conn(0x7fb7300cf590 msgr2=0x7fb7300cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 --2- 172.21.15.82:0/1241604986 >> v2:172.21.15.196:3300/0 conn(0x7fb7300cf590 0x7fb7300cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 >> v1:172.21.15.196:6789/0 conn(0x7fb73015b6b0 legacy=0x7fb7300ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300cc2e0 con 0x7fb730072000 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb728001a00 con 0x7fb730072000 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 >> v1:172.21.15.82:6789/0 conn(0x7fb730072000 legacy=0x7fb730072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb71c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb71c004810 0x7fb71c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb71c0051d0 0x7fb71c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73015ba90 con 0x7fb71c003a30 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c00c100 con 0x7fb71c004810 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71c00c300 con 0x7fb71c0051d0 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb71c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb727fff640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb71c004810 0x7fb71c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 -- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb71c0051d0 msgr2=0x7fb71c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb71c0051d0 0x7fb71c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 -- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb71c004810 msgr2=0x7fb71c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb71c004810 0x7fb71c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72ed9d640 1 -- 172.21.15.82:0/1241604986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c003530 con 0x7fb71c003a30 2024-04-25T10:02:52.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.764+0000 7fb72f59e640 1 -- 172.21.15.82:0/1241604986 reap_dead start 2024-04-25T10:02:52.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb71c003e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb720001780 tx=0x7fb72001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e3aaa93e247b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200013a0 con 0x7fb71c003a30 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7200052f0 con 0x7fb71c003a30 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72e59c640 1 -- 172.21.15.82:0/1241604986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720003840 con 0x7fb71c003a30 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 msgr2=0x7fb71c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb71c003e50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb720001780 tx=0x7fb72001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- 172.21.15.82:0/1241604986 shutdown_connections 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 --2- 172.21.15.82:0/1241604986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb71c003e50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- 172.21.15.82:0/1241604986 >> 172.21.15.82:0/1241604986 conn(0x7fb73006ae60 msgr2=0x7fb73006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- 172.21.15.82:0/1241604986 shutdown_connections 2024-04-25T10:02:52.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- 172.21.15.82:0/1241604986 wait complete. 2024-04-25T10:02:52.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 Processor -- start 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- start start 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7301f8170 0x7fb73015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 0x7fb730159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cff00 con 0x7fb71c003a30 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cfd80 con 0x7fb7301f8170 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb7351c1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300cadb0 con 0x7fb73015e3f0 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40004/0 (socket says 172.21.15.82:40004) 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72ed9d640 1 -- 172.21.15.82:0/2520783260 learned_addr learned my addr 172.21.15.82:0/2520783260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb727fff640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7301f8170 0x7fb73015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.768+0000 7fb72f59e640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 0x7fb730159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 -- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 msgr2=0x7fb730159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 0x7fb730159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 -- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7301f8170 msgr2=0x7fb73015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7301f8170 0x7fb73015c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 -- 172.21.15.82:0/2520783260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7301f8730 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72f59e640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 0x7fb730159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb72ed9d640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb72001c220 tx=0x7fb71c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7200ba070 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb720000ee0 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7301f8a10 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb720007a40 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7301f8fd0 con 0x7fb71c003a30 2024-04-25T10:02:52.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73004efa0 con 0x7fb71c003a30 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7200bd070 con 0x7fb71c003a30 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 0x7fb710069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb727fff640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 0x7fb710069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fb72011e2e0 con 0x7fb71c003a30 2024-04-25T10:02:52.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.772+0000 7fb727fff640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 0x7fb710069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7300b0d90 tx=0x7fb718008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.776+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb720016130 con 0x7fb71c003a30 2024-04-25T10:02:52.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.940+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb73006b3e0 con 0x7fb71c003a30 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.940+0000 7fb7277fe640 1 -- 172.21.15.82:0/2520783260 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb7200eeed0 con 0x7fb71c003a30 2024-04-25T10:02:52.948 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:52.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 msgr2=0x7fb710069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 0x7fb710069450 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb7300b0d90 tx=0x7fb718008040 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 msgr2=0x7fb7300ade40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb72001c220 tx=0x7fb71c005840 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 shutdown_connections 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb710066f90 0x7fb710069450 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb73015e3f0 0x7fb730159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7301f8170 0x7fb73015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 --2- 172.21.15.82:0/2520783260 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb71c003a30 0x7fb7300ade40 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 >> 172.21.15.82:0/2520783260 conn(0x7fb73006ae60 msgr2=0x7fb7300af2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 shutdown_connections 2024-04-25T10:02:52.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:52.944+0000 7fb7351c1640 1 -- 172.21.15.82:0/2520783260 wait complete. 2024-04-25T10:02:52.967 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:53.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 Processor -- start 2024-04-25T10:02:53.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 -- start start 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6cc415da80 0x7f6cc40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc406be60 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc415de60 con 0x7f6cc40d07f0 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cca1a5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc406c440 con 0x7f6cc415da80 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6cc415da80 0x7f6cc40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6cc415da80 0x7f6cc40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40012/0 (socket says 172.21.15.82:40012) 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 -- 172.21.15.82:0/3623257600 learned_addr learned my addr 172.21.15.82:0/3623257600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc2ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6cc40d1cd0 0x7f6cc40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48244/0 (socket says 172.21.15.82:48244) 2024-04-25T10:02:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 -- 172.21.15.82:0/3623257600 >> v1:172.21.15.196:6789/0 conn(0x7f6cc40d07f0 legacy=0x7f6cc406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 -- 172.21.15.82:0/3623257600 >> v1:172.21.15.82:6789/0 conn(0x7f6cc40d1cd0 legacy=0x7f6cc40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc27fc640 1 -- 172.21.15.82:0/3623257600 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310341981 0 0) 0x7f6cc406be60 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.084+0000 7f6cc37fe640 1 -- 172.21.15.82:0/3623257600 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40d2450 con 0x7f6cc415da80 2024-04-25T10:02:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc37fe640 1 --2- 172.21.15.82:0/3623257600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc415da80 0x7f6cc40d4230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6cc406c5c0 tx=0x7f6cb401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39fa53905b1fd80a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc27fc640 1 -- 172.21.15.82:0/3623257600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40ab430 con 0x7f6cc415da80 2024-04-25T10:02:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc27fc640 1 -- 172.21.15.82:0/3623257600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb40b0070 con 0x7f6cc415da80 2024-04-25T10:02:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc27fc640 1 -- 172.21.15.82:0/3623257600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40ab7f0 con 0x7f6cc415da80 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- 172.21.15.82:0/3623257600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc415da80 msgr2=0x7f6cc40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/3623257600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc415da80 0x7f6cc40d4230 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6cc406c5c0 tx=0x7f6cb401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- 172.21.15.82:0/3623257600 shutdown_connections 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/3623257600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc415da80 0x7f6cc40d4230 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- 172.21.15.82:0/3623257600 >> 172.21.15.82:0/3623257600 conn(0x7f6cc408b000 msgr2=0x7f6cc40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- 172.21.15.82:0/3623257600 shutdown_connections 2024-04-25T10:02:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- 172.21.15.82:0/3623257600 wait complete. 2024-04-25T10:02:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 Processor -- start 2024-04-25T10:02:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- start start 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 0x7f6cc415fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6cc415da80 0x7f6cc4160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc406d970 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc406d7f0 con 0x7f6cc40d07f0 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cca1a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc406daf0 con 0x7f6cc415da80 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40022/0 (socket says 172.21.15.82:40022) 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 -- 172.21.15.82:0/4100669409 learned_addr learned my addr 172.21.15.82:0/4100669409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc37fe640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 0x7f6cc415fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 -- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6cc415da80 msgr2=0x7f6cc4160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc3fff640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6cc415da80 0x7f6cc4160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6cc415da80 0x7f6cc4160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 -- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 msgr2=0x7f6cc415fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 0x7f6cc415fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc2ffd640 1 -- 172.21.15.82:0/4100669409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40c80f0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.088+0000 7f6cc37fe640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 0x7f6cc415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cc2ffd640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c490 tx=0x7f6cb800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8007480 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb8007620 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc40c83d0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80059d0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc40c8990 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc404efa0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6cb8005b70 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6cb80850e0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cc37fe640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.092+0000 7f6cc37fe640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40af000 tx=0x7f6cb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.096+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6cb8055cd0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.252+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6cc40b29c0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.252+0000 7f6cabfff640 1 -- 172.21.15.82:0/4100669409 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6cb80555d0 con 0x7f6cc40d1cd0 2024-04-25T10:02:53.260 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 msgr2=0x7f6cb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 0x7f6cb0069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40af000 tx=0x7f6cb4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 msgr2=0x7f6cc4160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800c490 tx=0x7f6cb800c950 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 shutdown_connections 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6cb0066f60 0x7f6cb0069420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6cc415da80 0x7f6cc4160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6cc40d1cd0 0x7f6cc4160400 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 --2- 172.21.15.82:0/4100669409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6cc40d07f0 0x7f6cc415fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 >> 172.21.15.82:0/4100669409 conn(0x7f6cc408b000 msgr2=0x7f6cc4155210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 shutdown_connections 2024-04-25T10:02:53.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.256+0000 7f6cca1a5640 1 -- 172.21.15.82:0/4100669409 wait complete. 2024-04-25T10:02:53.279 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:53.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.384+0000 7fee1f24e640 1 Processor -- start 2024-04-25T10:02:53.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.384+0000 7fee1f24e640 1 -- start start 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee18071740 0x7fee180acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee180ae860 con 0x7fee1815da10 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee1815e210 con 0x7fee180b24a0 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee1815e4f0 con 0x7fee18071740 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee18071740 0x7fee180acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1cfea640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fee1815da10 0x7fee18071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48252/0 (socket says 172.21.15.82:48252) 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fee18071740 0x7fee180acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40030/0 (socket says 172.21.15.82:40030) 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1cfea640 1 -- 172.21.15.82:0/519499512 learned_addr learned my addr 172.21.15.82:0/519499512 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 -- 172.21.15.82:0/519499512 >> v1:172.21.15.196:6789/0 conn(0x7fee180b24a0 legacy=0x7fee180ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 -- 172.21.15.82:0/519499512 >> v1:172.21.15.82:6789/0 conn(0x7fee1815da10 legacy=0x7fee18071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 -- 172.21.15.82:0/519499512 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee180ae4c0 con 0x7fee18071740 2024-04-25T10:02:53.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0f7fe640 1 -- 172.21.15.82:0/519499512 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051133690 0 0) 0x7fee180ae860 con 0x7fee1815da10 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0ffff640 1 --2- 172.21.15.82:0/519499512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee180acbc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee080097c0 tx=0x7fee0801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f746c5bab1cd5b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0f7fe640 1 -- 172.21.15.82:0/519499512 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080b0070 con 0x7fee18071740 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0f7fe640 1 -- 172.21.15.82:0/519499512 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee080ab600 con 0x7fee18071740 2024-04-25T10:02:53.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee0f7fe640 1 -- 172.21.15.82:0/519499512 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee080ab9d0 con 0x7fee18071740 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- 172.21.15.82:0/519499512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 msgr2=0x7fee180acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 --2- 172.21.15.82:0/519499512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee180acbc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fee080097c0 tx=0x7fee0801c340 comp rx=0 tx=0).stop 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- 172.21.15.82:0/519499512 shutdown_connections 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 --2- 172.21.15.82:0/519499512 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee180acbc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- 172.21.15.82:0/519499512 >> 172.21.15.82:0/519499512 conn(0x7fee180b11a0 msgr2=0x7fee180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- 172.21.15.82:0/519499512 shutdown_connections 2024-04-25T10:02:53.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.388+0000 7fee1f24e640 1 -- 172.21.15.82:0/519499512 wait complete. 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 Processor -- start 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- start start 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 0x7fee1815c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee1815da10 0x7fee181587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee180adc20 con 0x7fee18071740 2024-04-25T10:02:53.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1815ef00 con 0x7fee180b24a0 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1816c2c0 con 0x7fee1815da10 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40036/0 (socket says 172.21.15.82:40036) 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 -- 172.21.15.82:0/4033203136 learned_addr learned my addr 172.21.15.82:0/4033203136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0ffff640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 0x7fee1815c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1d7eb640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee1815da10 0x7fee181587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 -- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee1815da10 msgr2=0x7fee181587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee1815da10 0x7fee181587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 -- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 msgr2=0x7fee1815c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 0x7fee1815c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 -- 172.21.15.82:0/4033203136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1815b6c0 con 0x7fee18071740 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0ffff640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 0x7fee1815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1cfea640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee140011f0 tx=0x7fee1400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14018070 con 0x7fee18071740 2024-04-25T10:02:53.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee140074e0 con 0x7fee18071740 2024-04-25T10:02:53.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1815b9a0 con 0x7fee18071740 2024-04-25T10:02:53.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee14004a10 con 0x7fee18071740 2024-04-25T10:02:53.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1815bf60 con 0x7fee18071740 2024-04-25T10:02:53.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee1804efa0 con 0x7fee18071740 2024-04-25T10:02:53.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee1401b070 con 0x7fee18071740 2024-04-25T10:02:53.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.392+0000 7fee0d7fa640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 0x7fee04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.396+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fee14084740 con 0x7fee18071740 2024-04-25T10:02:53.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.396+0000 7fee0ffff640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 0x7fee04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.396+0000 7fee0ffff640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 0x7fee04069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee080ad000 tx=0x7fee08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.400+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee14055330 con 0x7fee18071740 2024-04-25T10:02:53.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.544+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee180b1dc0 con 0x7fee18071740 2024-04-25T10:02:53.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.544+0000 7fee0d7fa640 1 -- 172.21.15.82:0/4033203136 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fee14081a30 con 0x7fee18071740 2024-04-25T10:02:53.552 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:53.552 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 msgr2=0x7fee04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 0x7fee04069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fee080ad000 tx=0x7fee08008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 msgr2=0x7fee18142f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee140011f0 tx=0x7fee1400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 shutdown_connections 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee04066f90 0x7fee04069450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee1815da10 0x7fee181587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee180b24a0 0x7fee1815c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 --2- 172.21.15.82:0/4033203136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee18071740 0x7fee18142f60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 >> 172.21.15.82:0/4033203136 conn(0x7fee180b11a0 msgr2=0x7fee18160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 shutdown_connections 2024-04-25T10:02:53.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.548+0000 7fee1f24e640 1 -- 172.21.15.82:0/4033203136 wait complete. 2024-04-25T10:02:53.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:53.570 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 Processor -- start 2024-04-25T10:02:53.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 -- start start 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f208c0b0b00 0x7f208c0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc650 0x7f208c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f208c0b1910 0x7f208c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c15cfa0 con 0x7f208c0b0b00 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c15fe80 con 0x7f208c0b1910 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f20933bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c160310 con 0x7f208c0cc650 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f208c0b0b00 0x7f208c0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f208c0b0b00 0x7f208c0cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40038/0 (socket says 172.21.15.82:40038) 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 -- 172.21.15.82:0/267913157 learned_addr learned my addr 172.21.15.82:0/267913157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2090957640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc650 0x7f208c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091959640 1 --2- 172.21.15.82:0/267913157 >> v2:172.21.15.196:3300/0 conn(0x7f208c0b1910 0x7f208c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 -- 172.21.15.82:0/267913157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc650 msgr2=0x7f208c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc650 0x7f208c0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 -- 172.21.15.82:0/267913157 >> v2:172.21.15.196:3300/0 conn(0x7f208c0b1910 msgr2=0x7f208c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 --2- 172.21.15.82:0/267913157 >> v2:172.21.15.196:3300/0 conn(0x7f208c0b1910 0x7f208c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.676+0000 7f2091158640 1 -- 172.21.15.82:0/267913157 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c15f550 con 0x7f208c0b0b00 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f2091158640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0b0b00 0x7f208c0cc110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f208c0c7460 tx=0x7f208801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221ae37b9e2e9a7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f2083fff640 1 -- 172.21.15.82:0/267913157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880b0070 con 0x7f208c0b0b00 2024-04-25T10:02:53.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f2083fff640 1 -- 172.21.15.82:0/267913157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20880ab600 con 0x7f208c0b0b00 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f2083fff640 1 -- 172.21.15.82:0/267913157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880ab9d0 con 0x7f208c0b0b00 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- 172.21.15.82:0/267913157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0b0b00 msgr2=0x7f208c0cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0b0b00 0x7f208c0cc110 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f208c0c7460 tx=0x7f208801c340 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- 172.21.15.82:0/267913157 shutdown_connections 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 --2- 172.21.15.82:0/267913157 >> v2:172.21.15.196:3300/0 conn(0x7f208c0b1910 0x7f208c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0cc650 0x7f208c0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 --2- 172.21.15.82:0/267913157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0b0b00 0x7f208c0cc110 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- 172.21.15.82:0/267913157 >> 172.21.15.82:0/267913157 conn(0x7f208c0b1f10 msgr2=0x7f208c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- 172.21.15.82:0/267913157 shutdown_connections 2024-04-25T10:02:53.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- 172.21.15.82:0/267913157 wait complete. 2024-04-25T10:02:53.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 Processor -- start 2024-04-25T10:02:53.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.680+0000 7f20933bc640 1 -- start start 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b0b00 0x7f208c153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0b1910 0x7f208c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0b1060 con 0x7f208c0cc650 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0b0ee0 con 0x7f208c0b0b00 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0cce10 con 0x7f208c0b1910 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40044/0 (socket says 172.21.15.82:40044) 2024-04-25T10:02:53.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 -- 172.21.15.82:0/1678562531 learned_addr learned my addr 172.21.15.82:0/1678562531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2090957640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0b1910 0x7f208c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091158640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b0b00 0x7f208c153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 -- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0b1910 msgr2=0x7f208c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0b1910 0x7f208c153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 -- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b0b00 msgr2=0x7f208c153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b0b00 0x7f208c153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 -- 172.21.15.82:0/1678562531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c154010 con 0x7f208c0cc650 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091959640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2084007e60 tx=0x7f208400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084014620 con 0x7f208c0cc650 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2084019070 con 0x7f208c0cc650 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c1542d0 con 0x7f208c0cc650 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084014a20 con 0x7f208c0cc650 2024-04-25T10:02:53.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c14b040 con 0x7f208c0cc650 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c04efa0 con 0x7f208c0cc650 2024-04-25T10:02:53.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2084014bc0 con 0x7f208c0cc650 2024-04-25T10:02:53.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2084085c80 con 0x7f208c0cc650 2024-04-25T10:02:53.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091158640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.684+0000 7f2091158640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 0x7f2070069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20880ad000 tx=0x7f2088008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.688+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2084056870 con 0x7f208c0cc650 2024-04-25T10:02:53.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.836+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f208c0b6380 con 0x7f208c0cc650 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.836+0000 7f2081ffb640 1 -- 172.21.15.82:0/1678562531 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2084085020 con 0x7f208c0cc650 2024-04-25T10:02:53.844 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:53.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 msgr2=0x7f2070069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 0x7f2070069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20880ad000 tx=0x7f2088008040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 msgr2=0x7f208c149060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2084007e60 tx=0x7f208400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 shutdown_connections 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2070066f60 0x7f2070069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f208c0b1910 0x7f208c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f208c0cc650 0x7f208c149060 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 --2- 172.21.15.82:0/1678562531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f208c0b0b00 0x7f208c153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 >> 172.21.15.82:0/1678562531 conn(0x7f208c0b1f10 msgr2=0x7f208c159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 shutdown_connections 2024-04-25T10:02:53.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.840+0000 7f20933bc640 1 -- 172.21.15.82:0/1678562531 wait complete. 2024-04-25T10:02:53.862 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:53.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 Processor -- start 2024-04-25T10:02:53.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 -- start start 2024-04-25T10:02:53.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415b6b0 0x7f52c40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52c40cf590 0x7f52c40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f52c415f130 0x7f52c415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c415ba90 con 0x7f52c40cf590 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52c40cff30 con 0x7f52c415f130 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52cbc83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40cbd20 con 0x7f52c415b6b0 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52c40cf590 0x7f52c40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f52c40cf590 0x7f52c40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40050/0 (socket says 172.21.15.82:40050) 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 -- 172.21.15.82:0/3882686030 learned_addr learned my addr 172.21.15.82:0/3882686030 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c9a1f640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415b6b0 0x7f52c40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52ca220640 1 --2- 172.21.15.82:0/3882686030 >> v2:172.21.15.196:3300/0 conn(0x7f52c415f130 0x7f52c415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 -- 172.21.15.82:0/3882686030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415b6b0 msgr2=0x7f52c40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415b6b0 0x7f52c40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 -- 172.21.15.82:0/3882686030 >> v2:172.21.15.196:3300/0 conn(0x7f52c415f130 msgr2=0x7f52c415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 --2- 172.21.15.82:0/3882686030 >> v2:172.21.15.196:3300/0 conn(0x7f52c415f130 0x7f52c415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.976+0000 7f52c921e640 1 -- 172.21.15.82:0/3882686030 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40cc000 con 0x7f52c40cf590 2024-04-25T10:02:53.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52c921e640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c40cf590 0x7f52c40cf9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52c40c2de0 tx=0x7f52b401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ecf1f1424e4377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52c8a1d640 1 -- 172.21.15.82:0/3882686030 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40aa610 con 0x7f52c40cf590 2024-04-25T10:02:53.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52c8a1d640 1 -- 172.21.15.82:0/3882686030 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b40af070 con 0x7f52c40cf590 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52c8a1d640 1 -- 172.21.15.82:0/3882686030 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40aa9d0 con 0x7f52c40cf590 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- 172.21.15.82:0/3882686030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c40cf590 msgr2=0x7f52c40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c40cf590 0x7f52c40cf9f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f52c40c2de0 tx=0x7f52b401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- 172.21.15.82:0/3882686030 shutdown_connections 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- 172.21.15.82:0/3882686030 >> v2:172.21.15.196:3300/0 conn(0x7f52c415f130 0x7f52c415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c40cf590 0x7f52c40cf9f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- 172.21.15.82:0/3882686030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415b6b0 0x7f52c40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- 172.21.15.82:0/3882686030 >> 172.21.15.82:0/3882686030 conn(0x7f52c406ae60 msgr2=0x7f52c406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- 172.21.15.82:0/3882686030 shutdown_connections 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- 172.21.15.82:0/3882686030 wait complete. 2024-04-25T10:02:53.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 Processor -- start 2024-04-25T10:02:53.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- start start 2024-04-25T10:02:53.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c40cf590 0x7f52c4098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415f130 0x7f52c40adc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c415fef0 con 0x7f52c415b6b0 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40cba30 con 0x7f52c40cf590 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.980+0000 7f52cbc83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c40ca900 con 0x7f52c415f130 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40064/0 (socket says 172.21.15.82:40064) 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 -- 172.21.15.82:0/2890995349 learned_addr learned my addr 172.21.15.82:0/2890995349 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ca220640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415f130 0x7f52c40adc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c9a1f640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c40cf590 0x7f52c4098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 -- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415f130 msgr2=0x7f52c40adc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415f130 0x7f52c40adc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 -- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c40cf590 msgr2=0x7f52c4098540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c40cf590 0x7f52c4098540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 -- 172.21.15.82:0/2890995349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c40c4760 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c921e640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52b401aba0 tx=0x7f52b401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40bd5f0 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b40bd790 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c40c4a00 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b40bdb80 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c41531d0 con 0x7f52c415b6b0 2024-04-25T10:02:53.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c404efa0 con 0x7f52c415b6b0 2024-04-25T10:02:53.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52b40b33f0 con 0x7f52c415b6b0 2024-04-25T10:02:53.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 0x7f52a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c9a1f640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 0x7f52a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f52b4120210 con 0x7f52c415b6b0 2024-04-25T10:02:53.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.984+0000 7f52c9a1f640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 0x7f52a8069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b9d0 tx=0x7f52c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:53.988+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f52b40c3060 con 0x7f52c415b6b0 2024-04-25T10:02:54.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.148+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52c40b2080 con 0x7f52c415b6b0 2024-04-25T10:02:54.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.148+0000 7f52ba7fc640 1 -- 172.21.15.82:0/2890995349 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52b40a7070 con 0x7f52c415b6b0 2024-04-25T10:02:54.156 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:54.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 msgr2=0x7f52a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 0x7f52a8069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b9d0 tx=0x7f52c0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 msgr2=0x7f52c40ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f52b401aba0 tx=0x7f52b401ad30 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 shutdown_connections 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52a8066f60 0x7f52a8069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f52c415f130 0x7f52c40adc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f52c415b6b0 0x7f52c40ad710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 --2- 172.21.15.82:0/2890995349 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f52c40cf590 0x7f52c4098540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 >> 172.21.15.82:0/2890995349 conn(0x7f52c406ae60 msgr2=0x7f52c41554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 shutdown_connections 2024-04-25T10:02:54.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.152+0000 7f52cbc83640 1 -- 172.21.15.82:0/2890995349 wait complete. 2024-04-25T10:02:54.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:54.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 Processor -- start 2024-04-25T10:02:54.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 -- start start 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 0x7fe2e40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2e40b28c0 0x7fe2e4163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e40c9ef0 con 0x7fe2e4159fb0 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e4072780 con 0x7fe2e40b28c0 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ebde3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e4071b70 con 0x7fe2e4159bd0 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2e40b28c0 0x7fe2e4163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe2e40b28c0 0x7fe2e4163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40068/0 (socket says 172.21.15.82:40068) 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 -- 172.21.15.82:0/2593760452 learned_addr learned my addr 172.21.15.82:0/2593760452 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:54.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2e9b7f640 1 --2- 172.21.15.82:0/2593760452 >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 0x7fe2e40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 -- 172.21.15.82:0/2593760452 >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 msgr2=0x7fe2e40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 --2- 172.21.15.82:0/2593760452 >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 0x7fe2e40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 -- 172.21.15.82:0/2593760452 >> v1:172.21.15.196:6789/0 conn(0x7fe2e4159fb0 legacy=0x7fe2e40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2ea380640 1 -- 172.21.15.82:0/2593760452 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4071e50 con 0x7fe2e40b28c0 2024-04-25T10:02:54.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.288+0000 7fe2e9b7f640 1 --2- 172.21.15.82:0/2593760452 >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 0x7fe2e40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:54.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ea380640 1 --2- 172.21.15.82:0/2593760452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e40b28c0 0x7fe2e4163960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40cf2c0 tx=0x7fe2dc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd89010b2526a38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2e8b7d640 1 -- 172.21.15.82:0/2593760452 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0a5800 con 0x7fe2e40b28c0 2024-04-25T10:02:54.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2e8b7d640 1 -- 172.21.15.82:0/2593760452 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0a59a0 con 0x7fe2e40b28c0 2024-04-25T10:02:54.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2e8b7d640 1 -- 172.21.15.82:0/2593760452 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0a5d70 con 0x7fe2e40b28c0 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- 172.21.15.82:0/2593760452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e40b28c0 msgr2=0x7fe2e4163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/2593760452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e40b28c0 0x7fe2e4163960 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe2e40cf2c0 tx=0x7fe2dc01c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- 172.21.15.82:0/2593760452 shutdown_connections 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/2593760452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e40b28c0 0x7fe2e4163960 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/2593760452 >> v2:172.21.15.196:3300/0 conn(0x7fe2e4159bd0 0x7fe2e40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- 172.21.15.82:0/2593760452 >> 172.21.15.82:0/2593760452 conn(0x7fe2e406ae60 msgr2=0x7fe2e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- 172.21.15.82:0/2593760452 shutdown_connections 2024-04-25T10:02:54.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- 172.21.15.82:0/2593760452 wait complete. 2024-04-25T10:02:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 Processor -- start 2024-04-25T10:02:54.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.292+0000 7fe2ebde3640 1 -- start start 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2e40b28c0 0x7fe2e40987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 0x7fe2e4152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4164020 con 0x7fe2e4159bd0 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e4163ea0 con 0x7fe2e40b28c0 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e41641a0 con 0x7fe2e4159fb0 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40082/0 (socket says 172.21.15.82:40082) 2024-04-25T10:02:54.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 -- 172.21.15.82:0/1981435136 learned_addr learned my addr 172.21.15.82:0/1981435136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e9b7f640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2e40b28c0 0x7fe2e40987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ea380640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 0x7fe2e4152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 -- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 msgr2=0x7fe2e4152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 0x7fe2e4152b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 -- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2e40b28c0 msgr2=0x7fe2e40987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2e40b28c0 0x7fe2e40987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 -- 172.21.15.82:0/1981435136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4098d30 con 0x7fe2e4159bd0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ea380640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 0x7fe2e4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e937e640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c430 tx=0x7fe2d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d40052d0 con 0x7fe2e4159bd0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d4005470 con 0x7fe2e4159bd0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d4007260 con 0x7fe2e4159bd0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e406d960 con 0x7fe2e4159bd0 2024-04-25T10:02:54.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e406df20 con 0x7fe2e4159bd0 2024-04-25T10:02:54.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e404efa0 con 0x7fe2e4159bd0 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2d4007400 con 0x7fe2e4159bd0 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 0x7fe2c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe2d40848a0 con 0x7fe2e4159bd0 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e9b7f640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 0x7fe2c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.296+0000 7fe2e9b7f640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 0x7fe2c8069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4165a00 tx=0x7fe2e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.300+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2d4055520 con 0x7fe2e4159bd0 2024-04-25T10:02:54.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.460+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2e4071110 con 0x7fe2e4159bd0 2024-04-25T10:02:54.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.460+0000 7fe2da7fc640 1 -- 172.21.15.82:0/1981435136 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2d4054e20 con 0x7fe2e4159bd0 2024-04-25T10:02:54.469 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 msgr2=0x7fe2c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 0x7fe2c8069420 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4165a00 tx=0x7fe2e0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 msgr2=0x7fe2e406eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe2d400c430 tx=0x7fe2d400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 shutdown_connections 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe2c8066f60 0x7fe2c8069420 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2e4159fb0 0x7fe2e4152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2e4159bd0 0x7fe2e406eb20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 --2- 172.21.15.82:0/1981435136 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2e40b28c0 0x7fe2e40987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 >> 172.21.15.82:0/1981435136 conn(0x7fe2e406ae60 msgr2=0x7fe2e40cdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 shutdown_connections 2024-04-25T10:02:54.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.464+0000 7fe2ebde3640 1 -- 172.21.15.82:0/1981435136 wait complete. 2024-04-25T10:02:54.487 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:54.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 Processor -- start 2024-04-25T10:02:54.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 -- start start 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae54159fb0 0x7fae540cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae54071b70 con 0x7fae54159bd0 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae54071e50 con 0x7fae540cb8c0 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae59e99640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae54072780 con 0x7fae54159fb0 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae52ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae54159fb0 0x7fae540cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae537fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fae54159bd0 0x7fae540c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48256/0 (socket says 172.21.15.82:48256) 2024-04-25T10:02:54.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae537fe640 1 -- 172.21.15.82:0/1121526140 learned_addr learned my addr 172.21.15.82:0/1121526140 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae52ffd640 1 -- 172.21.15.82:0/1121526140 >> v1:172.21.15.196:6789/0 conn(0x7fae540cb8c0 legacy=0x7fae540cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae52ffd640 1 -- 172.21.15.82:0/1121526140 >> v1:172.21.15.82:6789/0 conn(0x7fae54159bd0 legacy=0x7fae540c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae527fc640 1 -- 172.21.15.82:0/1121526140 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403376596 0 0) 0x7fae54071b70 con 0x7fae54159bd0 2024-04-25T10:02:54.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.608+0000 7fae52ffd640 1 -- 172.21.15.82:0/1121526140 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae54072130 con 0x7fae54159fb0 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae52ffd640 1 --2- 172.21.15.82:0/1121526140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae540cb380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae480097c0 tx=0x7fae4801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f83ca814cb99262 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae527fc640 1 -- 172.21.15.82:0/1121526140 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480b1070 con 0x7fae54159fb0 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae527fc640 1 -- 172.21.15.82:0/1121526140 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae480ac630 con 0x7fae54159fb0 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae527fc640 1 -- 172.21.15.82:0/1121526140 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480a8050 con 0x7fae54159fb0 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- 172.21.15.82:0/1121526140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 msgr2=0x7fae540cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 --2- 172.21.15.82:0/1121526140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae540cb380 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fae480097c0 tx=0x7fae4801c430 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- 172.21.15.82:0/1121526140 shutdown_connections 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 --2- 172.21.15.82:0/1121526140 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae540cb380 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- 172.21.15.82:0/1121526140 >> 172.21.15.82:0/1121526140 conn(0x7fae5406ae60 msgr2=0x7fae5406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- 172.21.15.82:0/1121526140 shutdown_connections 2024-04-25T10:02:54.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- 172.21.15.82:0/1121526140 wait complete. 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 Processor -- start 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- start start 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 0x7fae541579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 0x7fae540b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae54166d10 con 0x7fae54159fb0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae54166b90 con 0x7fae540cb8c0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae59e99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae54166e90 con 0x7fae54159bd0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40098/0 (socket says 172.21.15.82:40098) 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 -- 172.21.15.82:0/1688732677 learned_addr learned my addr 172.21.15.82:0/1688732677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae52ffd640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 0x7fae540b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae537fe640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 0x7fae541579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 -- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 msgr2=0x7fae540b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 0x7fae540b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 -- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 msgr2=0x7fae541579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 0x7fae541579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae53fff640 1 -- 172.21.15.82:0/1688732677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5415c660 con 0x7fae54159fb0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae52ffd640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 0x7fae540b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.612+0000 7fae537fe640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 0x7fae541579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae53fff640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00b970 tx=0x7fae4c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c019070 con 0x7fae54159fb0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae4c00cb40 con 0x7fae54159fb0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5415c8d0 con 0x7fae54159fb0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c014440 con 0x7fae54159fb0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5415ce90 con 0x7fae54159fb0 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5404efa0 con 0x7fae54159fb0 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae4c01c070 con 0x7fae54159fb0 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 0x7fae40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fae4c085010 con 0x7fae54159fb0 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae537fe640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 0x7fae40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.616+0000 7fae537fe640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 0x7fae40069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae44000f70 tx=0x7fae44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.620+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae4c055c00 con 0x7fae54159fb0 2024-04-25T10:02:54.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.768+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fae540cebe0 con 0x7fae54159fb0 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.768+0000 7fae3bfff640 1 -- 172.21.15.82:0/1688732677 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fae4c055500 con 0x7fae54159fb0 2024-04-25T10:02:54.776 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 msgr2=0x7fae40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 0x7fae40069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fae44000f70 tx=0x7fae44009040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 msgr2=0x7fae5415ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00b970 tx=0x7fae4c00be30 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 shutdown_connections 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae40066f90 0x7fae40069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae54159bd0 0x7fae540b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae54159fb0 0x7fae5415ab60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 --2- 172.21.15.82:0/1688732677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae540cb8c0 0x7fae541579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 >> 172.21.15.82:0/1688732677 conn(0x7fae5406ae60 msgr2=0x7fae5406d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 shutdown_connections 2024-04-25T10:02:54.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.772+0000 7fae59e99640 1 -- 172.21.15.82:0/1688732677 wait complete. 2024-04-25T10:02:54.793 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:54.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 Processor -- start 2024-04-25T10:02:54.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 -- start start 2024-04-25T10:02:54.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 0x7f65c80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65c8144ca0 0x7f65c806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f65c806ec60 0x7f65c8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80cc2b0 con 0x7f65c8144ca0 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c806e290 con 0x7f65c806ec60 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65ce285640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80cee30 con 0x7f65c80ce510 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65c8144ca0 0x7f65c806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65c8144ca0 0x7f65c806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44434/0 (socket says 172.21.15.82:44434) 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 -- 172.21.15.82:0/900874214 learned_addr learned my addr 172.21.15.82:0/900874214 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 -- 172.21.15.82:0/900874214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 msgr2=0x7f65c80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65c7fff640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 0x7f65c80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65cc822640 1 --2- 172.21.15.82:0/900874214 >> v2:172.21.15.196:3300/0 conn(0x7f65c806ec60 0x7f65c8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 0x7f65c80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 -- 172.21.15.82:0/900874214 >> v2:172.21.15.196:3300/0 conn(0x7f65c806ec60 msgr2=0x7f65c8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 --2- 172.21.15.82:0/900874214 >> v2:172.21.15.196:3300/0 conn(0x7f65c806ec60 0x7f65c8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65bffff640 1 -- 172.21.15.82:0/900874214 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80ccc70 con 0x7f65c8144ca0 2024-04-25T10:02:54.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.908+0000 7f65c7fff640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 0x7f65c80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c8144ca0 0x7f65c806e720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65c80cdbc0 tx=0x7f65b801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dd58ca746db261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c77fe640 1 -- 172.21.15.82:0/900874214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80b5070 con 0x7f65c8144ca0 2024-04-25T10:02:54.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c77fe640 1 -- 172.21.15.82:0/900874214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80a89d0 con 0x7f65c8144ca0 2024-04-25T10:02:54.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c77fe640 1 -- 172.21.15.82:0/900874214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80a8da0 con 0x7f65c8144ca0 2024-04-25T10:02:54.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/900874214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c8144ca0 msgr2=0x7f65c806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c8144ca0 0x7f65c806e720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f65c80cdbc0 tx=0x7f65b801ed10 comp rx=0 tx=0).stop 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/900874214 shutdown_connections 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- 172.21.15.82:0/900874214 >> v2:172.21.15.196:3300/0 conn(0x7f65c806ec60 0x7f65c8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c8144ca0 0x7f65c806e720 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- 172.21.15.82:0/900874214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c80ce510 0x7f65c80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/900874214 >> 172.21.15.82:0/900874214 conn(0x7f65c80b11a0 msgr2=0x7f65c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/900874214 shutdown_connections 2024-04-25T10:02:54.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/900874214 wait complete. 2024-04-25T10:02:54.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 Processor -- start 2024-04-25T10:02:54.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- start start 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 0x7f65c81331d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 0x7f65c81e69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80cc900 con 0x7f65c80ce510 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c806df20 con 0x7f65c806ec60 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81451f0 con 0x7f65c8144ca0 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44442/0 (socket says 172.21.15.82:44442) 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 -- 172.21.15.82:0/393214714 learned_addr learned my addr 172.21.15.82:0/393214714 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65cc822640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 0x7f65c81e69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c7fff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 0x7f65c81331d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 -- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 msgr2=0x7f65c81e69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 0x7f65c81e69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 -- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 msgr2=0x7f65c81331d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 0x7f65c81331d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 -- 172.21.15.82:0/393214714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8133710 con 0x7f65c80ce510 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65cc822640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 0x7f65c81e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c7fff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 0x7f65c81331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65bffff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f65b801d3c0 tx=0x7f65b801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80b5070 con 0x7f65c80ce510 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80b0c20 con 0x7f65c80ce510 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c81e5540 con 0x7f65c80ce510 2024-04-25T10:02:54.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.912+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80bb410 con 0x7f65c80ce510 2024-04-25T10:02:54.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c81e5b00 con 0x7f65c80ce510 2024-04-25T10:02:54.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c804efa0 con 0x7f65c80ce510 2024-04-25T10:02:54.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65b80c2060 con 0x7f65c80ce510 2024-04-25T10:02:54.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65c57fa640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65c7fff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f65b81226a0 con 0x7f65c80ce510 2024-04-25T10:02:54.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.916+0000 7f65c7fff640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 0x7f65a8069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65b0008930 tx=0x7f65b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:54.920+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65b8018870 con 0x7f65c80ce510 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.068+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65c8071d70 con 0x7f65c80ce510 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.072+0000 7f65c57fa640 1 -- 172.21.15.82:0/393214714 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f65b80af3a0 con 0x7f65c80ce510 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 msgr2=0x7f65a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 0x7f65a8069420 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f65b0008930 tx=0x7f65b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 msgr2=0x7f65c81e4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f65b801d3c0 tx=0x7f65b801bab0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 shutdown_connections 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65a8066f60 0x7f65a8069420 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65c8144ca0 0x7f65c81e69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65c80ce510 0x7f65c81e4a50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 --2- 172.21.15.82:0/393214714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65c806ec60 0x7f65c81331d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 >> 172.21.15.82:0/393214714 conn(0x7f65c80b11a0 msgr2=0x7f65c80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 shutdown_connections 2024-04-25T10:02:55.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.076+0000 7f65ce285640 1 -- 172.21.15.82:0/393214714 wait complete. 2024-04-25T10:02:55.097 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:55.097 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 Processor -- start 2024-04-25T10:02:55.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 -- start start 2024-04-25T10:02:55.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf440d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4406be60 con 0x7fdf440d1cd0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4415de60 con 0x7fdf4415da80 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf4950e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4406c440 con 0x7fdf440d07f0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf437fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdf440d1cd0 0x7fdf4406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41040/0 (socket says 172.21.15.82:41040) 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf437fe640 1 -- 172.21.15.82:0/2092053124 learned_addr learned my addr 172.21.15.82:0/2092053124 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf440d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955987700 0 0) 0x7fdf4406be60 con 0x7fdf440d1cd0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf30003530 con 0x7fdf440d1cd0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584065952 0 0) 0x7fdf30003530 con 0x7fdf440d1cd0 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 msgr2=0x7fdf440d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf440d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 >> v1:172.21.15.196:6789/0 conn(0x7fdf4415da80 legacy=0x7fdf440d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf440d2450 con 0x7fdf440d1cd0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fdf3c001a00 con 0x7fdf440d1cd0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 >> v1:172.21.15.82:6789/0 conn(0x7fdf440d1cd0 legacy=0x7fdf4406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf30003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf30004d50 0x7fdf300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4406c440 con 0x7fdf30003fe0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000bb30 con 0x7fdf30004d50 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000bd30 con 0x7fdf440d07f0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 msgr2=0x7fdf30003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf42ffd640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf30003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf440d07f0 0x7fdf30003a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf437fe640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf30004d50 0x7fdf300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf30004d50 msgr2=0x7fdf300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf30004d50 0x7fdf300051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.216+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2092053124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30003530 con 0x7fdf30003fe0 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009ac0 tx=0x7fdf2c01c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b3a4ca95440687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0af070 con 0x7fdf30003fe0 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c0059a0 con 0x7fdf30003fe0 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf427fc640 1 -- 172.21.15.82:0/2092053124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c009690 con 0x7fdf30003fe0 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- 172.21.15.82:0/2092053124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf30003fe0 msgr2=0x7fdf30004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2092053124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf30003fe0 0x7fdf30004810 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c009ac0 tx=0x7fdf2c01c6d0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf437fe640 1 -- 172.21.15.82:0/2092053124 reap_dead start 2024-04-25T10:02:55.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- 172.21.15.82:0/2092053124 shutdown_connections 2024-04-25T10:02:55.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- 172.21.15.82:0/2092053124 >> 172.21.15.82:0/2092053124 conn(0x7fdf4408b000 msgr2=0x7fdf440d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- 172.21.15.82:0/2092053124 shutdown_connections 2024-04-25T10:02:55.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- 172.21.15.82:0/2092053124 wait complete. 2024-04-25T10:02:55.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 Processor -- start 2024-04-25T10:02:55.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- start start 2024-04-25T10:02:55.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 0x7fdf4414ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 0x7fdf440b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440d26f0 con 0x7fdf44157c70 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440d2160 con 0x7fdf4415a730 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf4950e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf440d2870 con 0x7fdf30003fe0 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf3bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf3bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44458/0 (socket says 172.21.15.82:44458) 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.220+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2642846476 learned_addr learned my addr 172.21.15.82:0/2642846476 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf42ffd640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 0x7fdf4414ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf437fe640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 0x7fdf440b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 msgr2=0x7fdf4414ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 0x7fdf4414ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 msgr2=0x7fdf440b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 0x7fdf440b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 -- 172.21.15.82:0/2642846476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4415aec0 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf42ffd640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 0x7fdf4414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf437fe640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 0x7fdf440b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3bfff640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01c500 tx=0x7fdf30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0b8070 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c006230 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf441f7740 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0af040 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4414ef40 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4404efa0 con 0x7fdf44157c70 2024-04-25T10:02:55.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf2c0bb070 con 0x7fdf44157c70 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 0x7fdf24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fdf2c11f3d0 con 0x7fdf44157c70 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf42ffd640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 0x7fdf24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.224+0000 7fdf42ffd640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 0x7fdf24069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf34000c10 tx=0x7fdf34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.228+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf2c0166a0 con 0x7fdf44157c70 2024-04-25T10:02:55.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.392+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf440d1ab0 con 0x7fdf44157c70 2024-04-25T10:02:55.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.392+0000 7fdf3b7fe640 1 -- 172.21.15.82:0/2642846476 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf2c0f0050 con 0x7fdf44157c70 2024-04-25T10:02:55.400 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:55.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 msgr2=0x7fdf24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 0x7fdf24069450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdf34000c10 tx=0x7fdf34009040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 msgr2=0x7fdf440b0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c01c500 tx=0x7fdf30005840 comp rx=0 tx=0).stop 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 shutdown_connections 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdf24066f90 0x7fdf24069450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdf4415a730 0x7fdf440b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdf44157c70 0x7fdf440b0690 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 --2- 172.21.15.82:0/2642846476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdf30003fe0 0x7fdf4414ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 >> 172.21.15.82:0/2642846476 conn(0x7fdf4408b000 msgr2=0x7fdf4415e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 shutdown_connections 2024-04-25T10:02:55.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.396+0000 7fdf4950e640 1 -- 172.21.15.82:0/2642846476 wait complete. 2024-04-25T10:02:55.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:55.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 Processor -- start 2024-04-25T10:02:55.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 -- start start 2024-04-25T10:02:55.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469c0cf590 0x7f469c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 0x7f469c072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c0cbd20 con 0x7f469c15b6b0 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c15ba90 con 0x7f469c0cf590 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a245c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469c0cc000 con 0x7f469c072000 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469c0cf590 0x7f469c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469c0cf590 0x7f469c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44468/0 (socket says 172.21.15.82:44468) 2024-04-25T10:02:55.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 -- 172.21.15.82:0/2974321224 learned_addr learned my addr 172.21.15.82:0/2974321224 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a09f9640 1 --2- 172.21.15.82:0/2974321224 >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 0x7f469c072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 -- 172.21.15.82:0/2974321224 >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 msgr2=0x7f469c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 --2- 172.21.15.82:0/2974321224 >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 0x7f469c072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 -- 172.21.15.82:0/2974321224 >> v1:172.21.15.196:6789/0 conn(0x7f469c15b6b0 legacy=0x7f469c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f4693fff640 1 -- 172.21.15.82:0/2974321224 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0cc2e0 con 0x7f469c0cf590 2024-04-25T10:02:55.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.532+0000 7f46a09f9640 1 --2- 172.21.15.82:0/2974321224 >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 0x7f469c072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f4693fff640 1 --2- 172.21.15.82:0/2974321224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c0cf590 0x7f469c0cfa20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f469c06f000 tx=0x7f468c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ce7c0ac12be26a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469b7fe640 1 -- 172.21.15.82:0/2974321224 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0b1070 con 0x7f469c0cf590 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469b7fe640 1 -- 172.21.15.82:0/2974321224 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c0ac630 con 0x7f469c0cf590 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469b7fe640 1 -- 172.21.15.82:0/2974321224 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c0a8050 con 0x7f469c0cf590 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2974321224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c0cf590 msgr2=0x7f469c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- 172.21.15.82:0/2974321224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c0cf590 0x7f469c0cfa20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f469c06f000 tx=0x7f468c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2974321224 shutdown_connections 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- 172.21.15.82:0/2974321224 >> v2:172.21.15.196:3300/0 conn(0x7f469c072000 0x7f469c072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- 172.21.15.82:0/2974321224 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c0cf590 0x7f469c0cfa20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2974321224 >> 172.21.15.82:0/2974321224 conn(0x7f469c06ae60 msgr2=0x7f469c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2974321224 shutdown_connections 2024-04-25T10:02:55.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2974321224 wait complete. 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 Processor -- start 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- start start 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f469c0cf590 0x7f469c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469c15b6b0 0x7f469c1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0caf30 con 0x7f469c072000 2024-04-25T10:02:55.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0cadb0 con 0x7f469c0cf590 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c1682a0 con 0x7f469c15b6b0 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44478/0 (socket says 172.21.15.82:44478) 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 -- 172.21.15.82:0/2659473102 learned_addr learned my addr 172.21.15.82:0/2659473102 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f4693fff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f469c0cf590 0x7f469c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a09f9640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469c15b6b0 0x7f469c1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 -- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469c15b6b0 msgr2=0x7f469c1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469c15b6b0 0x7f469c1522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 -- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f469c0cf590 msgr2=0x7f469c15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f469c0cf590 0x7f469c15f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 -- 172.21.15.82:0/2659473102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0b23c0 con 0x7f469c072000 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f469bfff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f468400b440 tx=0x7f468400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684015020 con 0x7f469c072000 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684011890 con 0x7f469c072000 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c06e3f0 con 0x7f469c072000 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684011c80 con 0x7f469c072000 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.536+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c153180 con 0x7f469c072000 2024-04-25T10:02:55.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c04efa0 con 0x7f469c072000 2024-04-25T10:02:55.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4684018020 con 0x7f469c072000 2024-04-25T10:02:55.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f46997fa640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4684085280 con 0x7f469c072000 2024-04-25T10:02:55.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f4693fff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.540+0000 7f4693fff640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f468c0ae000 tx=0x7f468c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.544+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f468401f030 con 0x7f469c072000 2024-04-25T10:02:55.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.704+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f469c153820 con 0x7f469c072000 2024-04-25T10:02:55.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.704+0000 7f46997fa640 1 -- 172.21.15.82:0/2659473102 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f468400d040 con 0x7f469c072000 2024-04-25T10:02:55.713 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 msgr2=0x7f467c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f468c0ae000 tx=0x7f468c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:02:55.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 msgr2=0x7f469c15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f468400b440 tx=0x7f468400b900 comp rx=0 tx=0).stop 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 shutdown_connections 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469c15b6b0 0x7f469c1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f469c0cf590 0x7f469c15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 --2- 172.21.15.82:0/2659473102 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469c072000 0x7f469c15f0c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 >> 172.21.15.82:0/2659473102 conn(0x7f469c06ae60 msgr2=0x7f469c0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 shutdown_connections 2024-04-25T10:02:55.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.708+0000 7f46a245c640 1 -- 172.21.15.82:0/2659473102 wait complete. 2024-04-25T10:02:55.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:55.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 Processor -- start 2024-04-25T10:02:55.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 -- start start 2024-04-25T10:02:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 0x7f90600d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f906006be60 con 0x7f90600d0620 2024-04-25T10:02:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f906006c440 con 0x7f906015da80 2024-04-25T10:02:55.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.836+0000 7f9064909640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906015de60 con 0x7f90600d1cd0 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 0x7f90600d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905e59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905e59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:50114/0 (socket says 172.21.15.82:50114) 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905e59c640 1 -- 172.21.15.82:0/1790649304 learned_addr learned my addr 172.21.15.82:0/1790649304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 -- 172.21.15.82:0/1790649304 >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 msgr2=0x7f90600d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1790649304 >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 -- 172.21.15.82:0/1790649304 >> v1:172.21.15.196:6789/0 conn(0x7f90600d0620 legacy=0x7f90600ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 -- 172.21.15.82:0/1790649304 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600d2450 con 0x7f90600d1cd0 2024-04-25T10:02:55.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905e59c640 1 --2- 172.21.15.82:0/1790649304 >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:55.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1790649304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 0x7f90600d00e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90540097a0 tx=0x7f905401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2595613ecdf1c16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905d59a640 1 -- 172.21.15.82:0/1790649304 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540b2070 con 0x7f90600d1cd0 2024-04-25T10:02:55.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905d59a640 1 -- 172.21.15.82:0/1790649304 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90540ad640 con 0x7f90600d1cd0 2024-04-25T10:02:55.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f905d59a640 1 -- 172.21.15.82:0/1790649304 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540ada10 con 0x7f90600d1cd0 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 -- 172.21.15.82:0/1790649304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 msgr2=0x7f90600d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 --2- 172.21.15.82:0/1790649304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 0x7f90600d00e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f90540097a0 tx=0x7f905401c480 comp rx=0 tx=0).stop 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 -- 172.21.15.82:0/1790649304 shutdown_connections 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 --2- 172.21.15.82:0/1790649304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f90600d1cd0 0x7f90600d00e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 --2- 172.21.15.82:0/1790649304 >> v2:172.21.15.196:3300/0 conn(0x7f906015da80 0x7f90600d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 -- 172.21.15.82:0/1790649304 >> 172.21.15.82:0/1790649304 conn(0x7f906008b000 msgr2=0x7f90600d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 -- 172.21.15.82:0/1790649304 shutdown_connections 2024-04-25T10:02:55.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.840+0000 7f9064909640 1 -- 172.21.15.82:0/1790649304 wait complete. 2024-04-25T10:02:55.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 Processor -- start 2024-04-25T10:02:55.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- start start 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 0x7f90601572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 0x7f90601517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906016a730 con 0x7f90600d0620 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906016a5b0 con 0x7f90600d1cd0 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f906016a8b0 con 0x7f906015da80 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44490/0 (socket says 172.21.15.82:44490) 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 -- 172.21.15.82:0/1522045133 learned_addr learned my addr 172.21.15.82:0/1522045133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905ed9d640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 0x7f90601517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 0x7f90601572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 -- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 msgr2=0x7f90601517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 0x7f90601517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 -- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 msgr2=0x7f90601572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 0x7f90601572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 -- 172.21.15.82:0/1522045133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060157bf0 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905ed9d640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 0x7f90601517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 0x7f90601572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f905e59c640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f905000e960 tx=0x7f905000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050017070 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000ca00 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000cdb0 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9060160620 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906015f600 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.844+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906015fd10 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.848+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f905001a070 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.848+0000 7f904effd640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.848+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9050085270 con 0x7f90600d0620 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.848+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.848+0000 7f905dd9b640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 0x7f9040069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90540af000 tx=0x7f90540a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:55.852+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9050055ef0 con 0x7f90600d0620 2024-04-25T10:02:56.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.000+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90600d42d0 con 0x7f90600d0620 2024-04-25T10:02:56.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f904effd640 1 -- 172.21.15.82:0/1522045133 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90500557f0 con 0x7f90600d0620 2024-04-25T10:02:56.010 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:56.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 msgr2=0x7f9040069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 0x7f9040069420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f90540af000 tx=0x7f90540a7070 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 msgr2=0x7f90601600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f905000e960 tx=0x7f905000ee20 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 shutdown_connections 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9040066f60 0x7f9040069420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f906015da80 0x7f90601517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f90600d1cd0 0x7f90601572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 --2- 172.21.15.82:0/1522045133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90600d0620 0x7f90601600e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 >> 172.21.15.82:0/1522045133 conn(0x7f906008b000 msgr2=0x7f90600b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 shutdown_connections 2024-04-25T10:02:56.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.004+0000 7f9064909640 1 -- 172.21.15.82:0/1522045133 wait complete. 2024-04-25T10:02:56.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 Processor -- start 2024-04-25T10:02:56.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 -- start start 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f931c15b6b0 0x7f931c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c15ff50 con 0x7f931c0cf590 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f931c0cbd20 con 0x7f931c073010 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9322ca7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931c15ba90 con 0x7f931c15b6b0 2024-04-25T10:02:56.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f931c15b6b0 0x7f931c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f931c15b6b0 0x7f931c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44492/0 (socket says 172.21.15.82:44492) 2024-04-25T10:02:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 -- 172.21.15.82:0/532074016 learned_addr learned my addr 172.21.15.82:0/532074016 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 -- 172.21.15.82:0/532074016 >> v1:172.21.15.196:6789/0 conn(0x7f931c073010 legacy=0x7f931c15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 -- 172.21.15.82:0/532074016 >> v1:172.21.15.82:6789/0 conn(0x7f931c0cf590 legacy=0x7f931c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.140+0000 7f9320a43640 1 -- 172.21.15.82:0/532074016 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0cc000 con 0x7f931c15b6b0 2024-04-25T10:02:56.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9320a43640 1 --2- 172.21.15.82:0/532074016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c15b6b0 0x7f931c0ca3c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f930c009870 tx=0x7f930c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d671a930a672679f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f931b7fe640 1 -- 172.21.15.82:0/532074016 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0aa650 con 0x7f931c15b6b0 2024-04-25T10:02:56.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f931b7fe640 1 -- 172.21.15.82:0/532074016 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930c0af070 con 0x7f931c15b6b0 2024-04-25T10:02:56.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f931b7fe640 1 -- 172.21.15.82:0/532074016 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930c0aaa10 con 0x7f931c15b6b0 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- 172.21.15.82:0/532074016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c15b6b0 msgr2=0x7f931c0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 --2- 172.21.15.82:0/532074016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c15b6b0 0x7f931c0ca3c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f930c009870 tx=0x7f930c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- 172.21.15.82:0/532074016 shutdown_connections 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 --2- 172.21.15.82:0/532074016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c15b6b0 0x7f931c0ca3c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- 172.21.15.82:0/532074016 >> 172.21.15.82:0/532074016 conn(0x7f931c06ae60 msgr2=0x7f931c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- 172.21.15.82:0/532074016 shutdown_connections 2024-04-25T10:02:56.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- 172.21.15.82:0/532074016 wait complete. 2024-04-25T10:02:56.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 Processor -- start 2024-04-25T10:02:56.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- start start 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 0x7f931c1525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f931c15b6b0 0x7f931c0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0ca900 con 0x7f931c0cf590 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0cba30 con 0x7f931c073010 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f9322ca7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f931c0cfdb0 con 0x7f931c15b6b0 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.144+0000 7f931bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44496/0 (socket says 172.21.15.82:44496) 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 -- 172.21.15.82:0/1150363918 learned_addr learned my addr 172.21.15.82:0/1150363918 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9321244640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f931c15b6b0 0x7f931c0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9320a43640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 0x7f931c1525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 -- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f931c15b6b0 msgr2=0x7f931c0b0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f931c15b6b0 0x7f931c0b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 -- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 msgr2=0x7f931c1525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 0x7f931c1525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 -- 172.21.15.82:0/1150363918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c152af0 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9320a43640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 0x7f931c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f931bfff640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f931000c450 tx=0x7f931000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310016020 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93100052d0 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931c0c42b0 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310007260 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931c0c4870 con 0x7f931c0cf590 2024-04-25T10:02:56.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931c04efa0 con 0x7f931c0cf590 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9310007400 con 0x7f931c0cf590 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 0x7f9308069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9310084d90 con 0x7f931c0cf590 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9320a43640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 0x7f9308069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.148+0000 7f9320a43640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 0x7f9308069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ae000 tx=0x7f930c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.152+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9310020020 con 0x7f931c0cf590 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f931c0ca3a0 con 0x7f931c0cf590 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f93197fa640 1 -- 172.21.15.82:0/1150363918 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f931c152af0 con 0x7f931c0cf590 2024-04-25T10:02:56.306 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 msgr2=0x7f9308069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 0x7f9308069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f930c0ae000 tx=0x7f930c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 msgr2=0x7f931c0c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f931000c450 tx=0x7f931000c910 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 shutdown_connections 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9308066f90 0x7f9308069450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f931c15b6b0 0x7f931c0b0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f931c0cf590 0x7f931c0c6cd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 --2- 172.21.15.82:0/1150363918 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f931c073010 0x7f931c1525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 >> 172.21.15.82:0/1150363918 conn(0x7f931c06ae60 msgr2=0x7f931c0af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 shutdown_connections 2024-04-25T10:02:56.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.300+0000 7f9322ca7640 1 -- 172.21.15.82:0/1150363918 wait complete. 2024-04-25T10:02:56.324 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:56.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 Processor -- start 2024-04-25T10:02:56.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 -- start start 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b4015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b400b2090 con 0x7f8b400b24a0 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b400b43e0 con 0x7f8b400b3f50 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b46692640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015f200 con 0x7f8b4015a700 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8b400b24a0 0x7f8b4006e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41066/0 (socket says 172.21.15.82:41066) 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 -- 172.21.15.82:0/4037701600 learned_addr learned my addr 172.21.15.82:0/4037701600 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:56.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3ffff640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b4015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913927708 0 0) 0x7f8b400b2090 con 0x7f8b400b24a0 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b2c003530 con 0x7f8b400b24a0 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1402762001 0 0) 0x7f8b2c003530 con 0x7f8b400b24a0 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 msgr2=0x7f8b4015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b4015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 >> v1:172.21.15.196:6789/0 conn(0x7f8b400b3f50 legacy=0x7f8b400cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400b46c0 con 0x7f8b400b24a0 2024-04-25T10:02:56.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8b38001680 con 0x7f8b400b24a0 2024-04-25T10:02:56.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 >> v1:172.21.15.82:6789/0 conn(0x7f8b400b24a0 legacy=0x7f8b4006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c003a60 0x7f8b2c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4015f200 con 0x7f8b2c004d50 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c00bc00 con 0x7f8b2c003a60 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2c00e010 con 0x7f8b4015a700 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b3ffff640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c003a60 0x7f8b2c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b377fe640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 -- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 msgr2=0x7f8b2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b4015a700 0x7f8b2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 -- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c003a60 msgr2=0x7f8b2c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b2c003a60 0x7f8b2c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.436+0000 7f8b44c2f640 1 -- 172.21.15.82:0/4037701600 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c003530 con 0x7f8b2c004d50 2024-04-25T10:02:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b2c0051d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c240 tx=0x7f8b3801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7453c99980639270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38007d20 con 0x7f8b2c004d50 2024-04-25T10:02:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b380a9070 con 0x7f8b2c004d50 2024-04-25T10:02:56.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b3f7fe640 1 -- 172.21.15.82:0/4037701600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38005d60 con 0x7f8b2c004d50 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- 172.21.15.82:0/4037701600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 msgr2=0x7f8b2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 --2- 172.21.15.82:0/4037701600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b2c0051d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8b3800c240 tx=0x7f8b3801cc50 comp rx=0 tx=0).stop 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b44c2f640 1 -- 172.21.15.82:0/4037701600 reap_dead start 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- 172.21.15.82:0/4037701600 shutdown_connections 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- 172.21.15.82:0/4037701600 >> 172.21.15.82:0/4037701600 conn(0x7f8b400b11a0 msgr2=0x7f8b400b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- 172.21.15.82:0/4037701600 shutdown_connections 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- 172.21.15.82:0/4037701600 wait complete. 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 Processor -- start 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- start start 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b400cecc0 0x7f8b400cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 0x7f8b401f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400cd310 con 0x7f8b2c004d50 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400ce600 con 0x7f8b401f65c0 2024-04-25T10:02:56.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.440+0000 7f8b46692640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b400cd490 con 0x7f8b400cecc0 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44518/0 (socket says 172.21.15.82:44518) 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 -- 172.21.15.82:0/179835941 learned_addr learned my addr 172.21.15.82:0/179835941 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b377fe640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b400cecc0 0x7f8b400cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 0x7f8b401f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 -- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b400cecc0 msgr2=0x7f8b400cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b400cecc0 0x7f8b400cf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 -- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 msgr2=0x7f8b401f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 0x7f8b401f6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 -- 172.21.15.82:0/179835941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b400af2c0 con 0x7f8b2c004d50 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b44c2f640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 0x7f8b401f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3ffff640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007f30 tx=0x7f8b30011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3001a020 con 0x7f8b2c004d50 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30016470 con 0x7f8b2c004d50 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b400b0300 con 0x7f8b2c004d50 2024-04-25T10:02:56.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30016860 con 0x7f8b2c004d50 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b400af820 con 0x7f8b2c004d50 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b400cf680 con 0x7f8b2c004d50 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b3001d020 con 0x7f8b2c004d50 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 0x7f8b20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f8b300862f0 con 0x7f8b2c004d50 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b377fe640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 0x7f8b20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.444+0000 7f8b377fe640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 0x7f8b20069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b40159a60 tx=0x7f8b28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.448+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b30024030 con 0x7f8b2c004d50 2024-04-25T10:02:56.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.608+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b4004efa0 con 0x7f8b2c004d50 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.608+0000 7f8b3d7fa640 1 -- 172.21.15.82:0/179835941 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f8b400af2c0 con 0x7f8b2c004d50 2024-04-25T10:02:56.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:56.618 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:56.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 msgr2=0x7f8b20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 0x7f8b20069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8b40159a60 tx=0x7f8b28009040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 msgr2=0x7f8b4014c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8b30007f30 tx=0x7f8b30011660 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 shutdown_connections 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b20066f90 0x7f8b20069450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b400cecc0 0x7f8b400cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b401f65c0 0x7f8b401f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 --2- 172.21.15.82:0/179835941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b2c004d50 0x7f8b4014c560 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 >> 172.21.15.82:0/179835941 conn(0x7f8b400b11a0 msgr2=0x7f8b4006e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 shutdown_connections 2024-04-25T10:02:56.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.612+0000 7f8b46692640 1 -- 172.21.15.82:0/179835941 wait complete. 2024-04-25T10:02:56.635 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:56.635 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 Processor -- start 2024-04-25T10:02:56.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 -- start start 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48f80cfca0 0x7f48f8166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f80cbd20 con 0x7f48f815b800 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f80cc000 con 0x7f48f815b420 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fec87640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f80cc930 con 0x7f48f80cfca0 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48f7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f48f815b800 0x7f48f80cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41076/0 (socket says 172.21.15.82:41076) 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48f7fff640 1 -- 172.21.15.82:0/3608170634 learned_addr learned my addr 172.21.15.82:0/3608170634 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fd224640 1 --2- 172.21.15.82:0/3608170634 >> v2:172.21.15.82:3300/0 conn(0x7f48f80cfca0 0x7f48f8166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48f77fe640 1 -- 172.21.15.82:0/3608170634 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833823756 0 0) 0x7f48f80cbd20 con 0x7f48f815b800 2024-04-25T10:02:56.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48f77fe640 1 -- 172.21.15.82:0/3608170634 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e8003530 con 0x7f48f815b800 2024-04-25T10:02:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fd224640 1 -- 172.21.15.82:0/3608170634 >> v1:172.21.15.196:6789/0 conn(0x7f48f815b420 legacy=0x7f48f80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fd224640 1 -- 172.21.15.82:0/3608170634 >> v1:172.21.15.82:6789/0 conn(0x7f48f815b800 legacy=0x7f48f80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.740+0000 7f48fd224640 1 -- 172.21.15.82:0/3608170634 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80cc2e0 con 0x7f48f80cfca0 2024-04-25T10:02:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fd224640 1 --2- 172.21.15.82:0/3608170634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f8166290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48f80ccab0 tx=0x7f48f001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c765c6e8de4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48f77fe640 1 -- 172.21.15.82:0/3608170634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00b2070 con 0x7f48f80cfca0 2024-04-25T10:02:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48f77fe640 1 -- 172.21.15.82:0/3608170634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f00a5860 con 0x7f48f80cfca0 2024-04-25T10:02:56.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48f77fe640 1 -- 172.21.15.82:0/3608170634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f00a5c30 con 0x7f48f80cfca0 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- 172.21.15.82:0/3608170634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 msgr2=0x7f48f8166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 --2- 172.21.15.82:0/3608170634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f8166290 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48f80ccab0 tx=0x7f48f001c910 comp rx=0 tx=0).stop 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- 172.21.15.82:0/3608170634 shutdown_connections 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 --2- 172.21.15.82:0/3608170634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f8166290 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- 172.21.15.82:0/3608170634 >> 172.21.15.82:0/3608170634 conn(0x7f48f806ae60 msgr2=0x7f48f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- 172.21.15.82:0/3608170634 shutdown_connections 2024-04-25T10:02:56.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- 172.21.15.82:0/3608170634 wait complete. 2024-04-25T10:02:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 Processor -- start 2024-04-25T10:02:56.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.744+0000 7f48fec87640 1 -- start start 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f815b420 0x7f48f8156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48f815b800 0x7f48f81571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8169170 con 0x7f48f80cfca0 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f8168ff0 con 0x7f48f815b420 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f81692f0 con 0x7f48f815b800 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44546/0 (socket says 172.21.15.82:44546) 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 -- 172.21.15.82:0/844787289 learned_addr learned my addr 172.21.15.82:0/844787289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fd224640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48f815b800 0x7f48f81571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 -- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48f815b800 msgr2=0x7f48f81571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f7fff640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f815b420 0x7f48f8156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48f815b800 0x7f48f81571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 -- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f815b420 msgr2=0x7f48f8156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f815b420 0x7f48f8156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 -- 172.21.15.82:0/844787289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80c6970 con 0x7f48f80cfca0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fca23640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c480 tx=0x7f48ec00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec012830 con 0x7f48f80cfca0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec017070 con 0x7f48f80cfca0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f815bf50 con 0x7f48f80cfca0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec012c30 con 0x7f48f80cfca0 2024-04-25T10:02:56.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f80c8e70 con 0x7f48f80cfca0 2024-04-25T10:02:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f804efa0 con 0x7f48f80cfca0 2024-04-25T10:02:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48ec01c460 con 0x7f48f80cfca0 2024-04-25T10:02:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 0x7f48dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f48ec0851d0 con 0x7f48f80cfca0 2024-04-25T10:02:56.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f7fff640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 0x7f48dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.748+0000 7f48f7fff640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 0x7f48dc069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48e4007fd0 tx=0x7f48e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.752+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48f80c6970 con 0x7f48f80cfca0 2024-04-25T10:02:56.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48f815c120 con 0x7f48f80cfca0 2024-04-25T10:02:56.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48f57fa640 1 -- 172.21.15.82:0/844787289 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48ec055e50 con 0x7f48f80cfca0 2024-04-25T10:02:56.907 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:56.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 msgr2=0x7f48dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 0x7f48dc069420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48e4007fd0 tx=0x7f48e4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 msgr2=0x7f48f80c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f48ec00c480 tx=0x7f48ec00c940 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 shutdown_connections 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48dc066f60 0x7f48dc069420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48f815b800 0x7f48f81571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48f815b420 0x7f48f8156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 --2- 172.21.15.82:0/844787289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48f80cfca0 0x7f48f80c6430 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 >> 172.21.15.82:0/844787289 conn(0x7f48f806ae60 msgr2=0x7f48f80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.900+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 shutdown_connections 2024-04-25T10:02:56.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:56.904+0000 7f48fec87640 1 -- 172.21.15.82:0/844787289 wait complete. 2024-04-25T10:02:56.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 Processor -- start 2024-04-25T10:02:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 -- start start 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98b00d1cd0 0x7f98b006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b006be60 con 0x7f98b015da80 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b006c440 con 0x7f98b00d1cd0 2024-04-25T10:02:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.028+0000 7f98b811f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b015de60 con 0x7f98b00d07f0 2024-04-25T10:02:57.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b56ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98b00d1cd0 0x7f98b006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b56ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:46316/0 (socket says 172.21.15.82:46316) 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98b00d1cd0 0x7f98b006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:50160/0 (socket says 172.21.15.82:50160) 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b56ba640 1 -- 172.21.15.82:0/3420287340 learned_addr learned my addr 172.21.15.82:0/3420287340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b5ebb640 1 --1- 172.21.15.82:0/3420287340 >> v1:172.21.15.196:6789/0 conn(0x7f98b015da80 0x7f98b00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:53782/0 (socket says 172.21.15.82:53782) 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 -- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 msgr2=0x7f98b00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 -- 172.21.15.82:0/3420287340 >> v1:172.21.15.196:6789/0 conn(0x7f98b015da80 legacy=0x7f98b00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 -- 172.21.15.82:0/3420287340 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00d2450 con 0x7f98b00d1cd0 2024-04-25T10:02:57.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b56ba640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:57.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.032+0000 7f98b66bc640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d1cd0 0x7f98b006c8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009d90 tx=0x7f98a801c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a28cdcd911c5472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b4eb9640 1 -- 172.21.15.82:0/3420287340 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80b2070 con 0x7f98b00d1cd0 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b4eb9640 1 -- 172.21.15.82:0/3420287340 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a80a59e0 con 0x7f98b00d1cd0 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b4eb9640 1 -- 172.21.15.82:0/3420287340 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80a5db0 con 0x7f98b00d1cd0 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d1cd0 msgr2=0x7f98b006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d1cd0 0x7f98b006c8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009d90 tx=0x7f98a801c940 comp rx=0 tx=0).stop 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/3420287340 shutdown_connections 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d1cd0 0x7f98b006c8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- 172.21.15.82:0/3420287340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d07f0 0x7f98b00d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/3420287340 >> 172.21.15.82:0/3420287340 conn(0x7f98b008b000 msgr2=0x7f98b00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/3420287340 shutdown_connections 2024-04-25T10:02:57.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/3420287340 wait complete. 2024-04-25T10:02:57.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 Processor -- start 2024-04-25T10:02:57.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- start start 2024-04-25T10:02:57.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d07f0 0x7f98b00b7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d1cd0 0x7f98b015f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b006d7a0 con 0x7f98b015da80 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b006d620 con 0x7f98b00d07f0 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b006d920 con 0x7f98b00d1cd0 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44548/0 (socket says 172.21.15.82:44548) 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 -- 172.21.15.82:0/54715415 learned_addr learned my addr 172.21.15.82:0/54715415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b5ebb640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d07f0 0x7f98b00b7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b56ba640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d1cd0 0x7f98b015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 -- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d1cd0 msgr2=0x7f98b015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d1cd0 0x7f98b015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 -- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d07f0 msgr2=0x7f98b00b7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d07f0 0x7f98b00b7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 -- 172.21.15.82:0/54715415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00b6ed0 con 0x7f98b015da80 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b66bc640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b0040 tx=0x7f98a80196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80b2050 con 0x7f98b015da80 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a80be070 con 0x7f98b015da80 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b00b71b0 con 0x7f98b015da80 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80ada90 con 0x7f98b015da80 2024-04-25T10:02:57.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.036+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b00b7770 con 0x7f98b015da80 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b004efa0 con 0x7f98b015da80 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98a80adc30 con 0x7f98b015da80 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98a67fc640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 0x7f9894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f98a8120770 con 0x7f98b015da80 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98b5ebb640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 0x7f9894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.040+0000 7f98b5ebb640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 0x7f9894069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98ac000f70 tx=0x7f98ac007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.044+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98a8016560 con 0x7f98b015da80 2024-04-25T10:02:57.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.188+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f98b00b15d0 con 0x7f98b015da80 2024-04-25T10:02:57.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.188+0000 7f98a67fc640 1 -- 172.21.15.82:0/54715415 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f98a80ab340 con 0x7f98b015da80 2024-04-25T10:02:57.196 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 msgr2=0x7f9894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 0x7f9894069420 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98ac000f70 tx=0x7f98ac007cf0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 msgr2=0x7f98b00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f98a80b0040 tx=0x7f98a80196e0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 shutdown_connections 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9894066f60 0x7f9894069420 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98b00d1cd0 0x7f98b015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98b015da80 0x7f98b00b6310 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 --2- 172.21.15.82:0/54715415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98b00d07f0 0x7f98b00b7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 >> 172.21.15.82:0/54715415 conn(0x7f98b008b000 msgr2=0x7f98b00b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 shutdown_connections 2024-04-25T10:02:57.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.192+0000 7f98b811f640 1 -- 172.21.15.82:0/54715415 wait complete. 2024-04-25T10:02:57.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.316+0000 7f3d2742c640 1 Processor -- start 2024-04-25T10:02:57.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.316+0000 7f3d2742c640 1 -- start start 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d2742c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d200cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d2742c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d20071b70 con 0x7f3d20159fb0 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d2742c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d20071e50 con 0x7f3d20159bd0 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d2742c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20072780 con 0x7f3d200cba90 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d249c7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3d20159fb0 0x7f3d200cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41086/0 (socket says 172.21.15.82:41086) 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d249c7640 1 -- 172.21.15.82:0/2989908742 learned_addr learned my addr 172.21.15.82:0/2989908742 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035448772 0 0) 0x7f3d20071b70 con 0x7f3d20159fb0 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d0c003450 con 0x7f3d20159fb0 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d259c9640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d200cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198730321 0 0) 0x7f3d0c003450 con 0x7f3d20159fb0 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 msgr2=0x7f3d200cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d200cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 >> v1:172.21.15.196:6789/0 conn(0x7f3d20159bd0 legacy=0x7f3d200c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20072130 con 0x7f3d20159fb0 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3d1c001680 con 0x7f3d20159fb0 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 >> v1:172.21.15.82:6789/0 conn(0x7f3d20159fb0 legacy=0x7f3d200cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d0c003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d0c00a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0c00b110 0x7f3d0c00d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20072780 con 0x7f3d0c003b50 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00ddb0 con 0x7f3d0c00b110 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c00dfb0 con 0x7f3d200cba90 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d0c003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d249c7640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d0c00a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d259c9640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0c00b110 0x7f3d0c00d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 -- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 msgr2=0x7f3d0c00a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d200cba90 0x7f3d0c00a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 -- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0c00b110 msgr2=0x7f3d0c00d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d0c00b110 0x7f3d0c00d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.320+0000 7f3d251c8640 1 -- 172.21.15.82:0/2989908742 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0c003450 con 0x7f3d0c003b50 2024-04-25T10:02:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d0c003f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d10001780 tx=0x7f3d1001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b489434fed222b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d100013a0 con 0x7f3d0c003b50 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d100052f0 con 0x7f3d0c003b50 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d17fff640 1 -- 172.21.15.82:0/2989908742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10003840 con 0x7f3d0c003b50 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2989908742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 msgr2=0x7f3d0c003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2989908742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d0c003f70 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3d10001780 tx=0x7f3d1001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d259c9640 1 -- 172.21.15.82:0/2989908742 reap_dead start 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2989908742 shutdown_connections 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2989908742 >> 172.21.15.82:0/2989908742 conn(0x7f3d2006ae60 msgr2=0x7f3d2006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2989908742 shutdown_connections 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2989908742 wait complete. 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 Processor -- start 2024-04-25T10:02:57.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- start start 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d20161d00 0x7f3d201f9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 0x7f3d20155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20164d00 con 0x7f3d0c003b50 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20164b80 con 0x7f3d20155b50 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20164e80 con 0x7f3d20161d00 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44574/0 (socket says 172.21.15.82:44574) 2024-04-25T10:02:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 -- 172.21.15.82:0/2144427015 learned_addr learned my addr 172.21.15.82:0/2144427015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d259c9640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 0x7f3d20155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d249c7640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d20161d00 0x7f3d201f9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 -- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d20161d00 msgr2=0x7f3d201f9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d20161d00 0x7f3d201f9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 -- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 msgr2=0x7f3d20155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 0x7f3d20155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 -- 172.21.15.82:0/2144427015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d201f9960 con 0x7f3d0c003b50 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d259c9640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 0x7f3d20155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d251c8640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d10001780 tx=0x7f3d0c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10007a40 con 0x7f3d0c003b50 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d100bb070 con 0x7f3d0c003b50 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d200c8ac0 con 0x7f3d0c003b50 2024-04-25T10:02:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10000ee0 con 0x7f3d0c003b50 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.324+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d200c9080 con 0x7f3d0c003b50 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d2004efa0 con 0x7f3d0c003b50 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d100b4040 con 0x7f3d0c003b50 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d15ffb640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 0x7f3d040695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3d1011f300 con 0x7f3d0c003b50 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d249c7640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 0x7f3d040695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.328+0000 7f3d249c7640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 0x7f3d040695f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c001060 tx=0x7f3d1c001120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.332+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d10016130 con 0x7f3d0c003b50 2024-04-25T10:02:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.476+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3d200ad020 con 0x7f3d0c003b50 2024-04-25T10:02:57.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d15ffb640 1 -- 172.21.15.82:0/2144427015 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3d100eff80 con 0x7f3d0c003b50 2024-04-25T10:02:57.485 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:57.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 msgr2=0x7f3d040695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 0x7f3d040695f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c001060 tx=0x7f3d1c001120 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 msgr2=0x7f3d200c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d10001780 tx=0x7f3d0c003740 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 shutdown_connections 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d04067130 0x7f3d040695f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d20161d00 0x7f3d201f9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d20155b50 0x7f3d20155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 --2- 172.21.15.82:0/2144427015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d0c003b50 0x7f3d200c84e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 >> 172.21.15.82:0/2144427015 conn(0x7f3d2006ae60 msgr2=0x7f3d200ccfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 shutdown_connections 2024-04-25T10:02:57.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.480+0000 7f3d2742c640 1 -- 172.21.15.82:0/2144427015 wait complete. 2024-04-25T10:02:57.503 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:57.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 Processor -- start 2024-04-25T10:02:57.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 -- start start 2024-04-25T10:02:57.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8cc06b820 0x7fd8cc0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc0afb20 con 0x7fd8cc06b440 2024-04-25T10:02:57.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8cc0afd90 con 0x7fd8cc0b1ac0 2024-04-25T10:02:57.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8d17e2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8cc0b06c0 con 0x7fd8cc06b820 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8cc06b820 0x7fd8cc0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8cc06b820 0x7fd8cc0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44584/0 (socket says 172.21.15.82:44584) 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 -- 172.21.15.82:0/2723895057 learned_addr learned my addr 172.21.15.82:0/2723895057 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8caffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd8cc06b440 0x7fd8cc0b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41090/0 (socket says 172.21.15.82:41090) 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 -- 172.21.15.82:0/2723895057 >> v1:172.21.15.196:6789/0 conn(0x7fd8cc0b1ac0 legacy=0x7fd8cc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 -- 172.21.15.82:0/2723895057 >> v1:172.21.15.82:6789/0 conn(0x7fd8cc06b440 legacy=0x7fd8cc0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8ca7fc640 1 -- 172.21.15.82:0/2723895057 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc0b0070 con 0x7fd8cc06b820 2024-04-25T10:02:57.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.620+0000 7fd8c9ffb640 1 -- 172.21.15.82:0/2723895057 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421855593 0 0) 0x7fd8cc0afb20 con 0x7fd8cc06b440 2024-04-25T10:02:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8ca7fc640 1 --2- 172.21.15.82:0/2723895057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b820 0x7fd8cc0b0b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9bf1a7b1bf4f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8c9ffb640 1 -- 172.21.15.82:0/2723895057 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00b2070 con 0x7fd8cc06b820 2024-04-25T10:02:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8c9ffb640 1 -- 172.21.15.82:0/2723895057 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c00a9070 con 0x7fd8cc06b820 2024-04-25T10:02:57.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8c9ffb640 1 -- 172.21.15.82:0/2723895057 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c00ad880 con 0x7fd8cc06b820 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- 172.21.15.82:0/2723895057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b820 msgr2=0x7fd8cc0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/2723895057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b820 0x7fd8cc0b0b50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009870 tx=0x7fd8c001c560 comp rx=0 tx=0).stop 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- 172.21.15.82:0/2723895057 shutdown_connections 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/2723895057 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b820 0x7fd8cc0b0b50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- 172.21.15.82:0/2723895057 >> 172.21.15.82:0/2723895057 conn(0x7fd8cc0b1f10 msgr2=0x7fd8cc0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- 172.21.15.82:0/2723895057 shutdown_connections 2024-04-25T10:02:57.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- 172.21.15.82:0/2723895057 wait complete. 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 Processor -- start 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- start start 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8cc06b820 0x7fd8cc0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8cc0b1ac0 0x7fd8cc144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc0cd1b0 con 0x7fd8cc06b440 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc0cd030 con 0x7fd8cc0b1ac0 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8d17e2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8cc16b980 con 0x7fd8cc06b820 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44590/0 (socket says 172.21.15.82:44590) 2024-04-25T10:02:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 -- 172.21.15.82:0/1366468471 learned_addr learned my addr 172.21.15.82:0/1366468471 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 -- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8cc06b820 msgr2=0x7fd8cc0c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8ca7fc640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8cc06b820 0x7fd8cc0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8cb7fe640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8cc0b1ac0 0x7fd8cc144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8cc06b820 0x7fd8cc0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 -- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8cc0b1ac0 msgr2=0x7fd8cc144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8cc0b1ac0 0x7fd8cc144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 -- 172.21.15.82:0/1366468471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8cc1424b0 con 0x7fd8cc06b440 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.624+0000 7fd8caffd640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c4a0 tx=0x7fd8bc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc018070 con 0x7fd8cc06b440 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0074e0 con 0x7fd8cc06b440 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc004a30 con 0x7fd8cc06b440 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8cc142790 con 0x7fd8cc06b440 2024-04-25T10:02:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8cc142d50 con 0x7fd8cc06b440 2024-04-25T10:02:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8cc04efa0 con 0x7fd8cc06b440 2024-04-25T10:02:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8bc01b070 con 0x7fd8cc06b440 2024-04-25T10:02:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd8bc084720 con 0x7fd8cc06b440 2024-04-25T10:02:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8ca7fc640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.628+0000 7fd8ca7fc640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 0x7fd8b8069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00af000 tx=0x7fd8c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.632+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8bc0553a0 con 0x7fd8cc06b440 2024-04-25T10:02:57.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.792+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd8cc0cc160 con 0x7fd8cc06b440 2024-04-25T10:02:57.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.792+0000 7fd8b37fe640 1 -- 172.21.15.82:0/1366468471 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd8bc054ca0 con 0x7fd8cc06b440 2024-04-25T10:02:57.801 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 msgr2=0x7fd8b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 0x7fd8b8069420 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00af000 tx=0x7fd8c0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 msgr2=0x7fd8cc1565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00c4a0 tx=0x7fd8bc00c960 comp rx=0 tx=0).stop 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 shutdown_connections 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd8b8066f60 0x7fd8b8069420 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8cc06b820 0x7fd8cc0c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8cc0b1ac0 0x7fd8cc144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 --2- 172.21.15.82:0/1366468471 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8cc06b440 0x7fd8cc1565c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 >> 172.21.15.82:0/1366468471 conn(0x7fd8cc0b1f10 msgr2=0x7fd8cc0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 shutdown_connections 2024-04-25T10:02:57.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.796+0000 7fd8d17e2640 1 -- 172.21.15.82:0/1366468471 wait complete. 2024-04-25T10:02:57.829 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:02:57.829 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 Processor -- start 2024-04-25T10:02:57.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 -- start start 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb048144ca0 0x7fb04806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0480ccc70 con 0x7fb04806ec60 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04806dc40 con 0x7fb0480ce510 2024-04-25T10:02:57.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb04d351640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0480cc2b0 con 0x7fb048144ca0 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb04806ec60 0x7fb048142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41104/0 (socket says 172.21.15.82:41104) 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 -- 172.21.15.82:0/2632237325 learned_addr learned my addr 172.21.15.82:0/2632237325 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb03e7fc640 1 --2- 172.21.15.82:0/2632237325 >> v2:172.21.15.196:3300/0 conn(0x7fb048144ca0 0x7fb04806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543081389 0 0) 0x7fb0480ccc70 con 0x7fb04806ec60 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb034003530 con 0x7fb04806ec60 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713241826 0 0) 0x7fb04806dc40 con 0x7fb0480ce510 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0480ccc70 con 0x7fb0480ce510 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670291317 0 0) 0x7fb034003530 con 0x7fb04806ec60 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 >> v2:172.21.15.196:3300/0 conn(0x7fb048144ca0 msgr2=0x7fb04806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 --2- 172.21.15.82:0/2632237325 >> v2:172.21.15.196:3300/0 conn(0x7fb048144ca0 0x7fb04806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 >> v1:172.21.15.196:6789/0 conn(0x7fb0480ce510 legacy=0x7fb0480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04806df20 con 0x7fb04806ec60 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb040001a00 con 0x7fb04806ec60 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb03e7fc640 1 --2- 172.21.15.82:0/2632237325 >> v2:172.21.15.196:3300/0 conn(0x7fb048144ca0 0x7fb04806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 >> v1:172.21.15.82:6789/0 conn(0x7fb04806ec60 legacy=0x7fb048142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb034003bc0 0x7fb034003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb034004810 0x7fb034004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 0x7fb03400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0480cc2b0 con 0x7fb0340051d0 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03400c020 con 0x7fb034004810 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03400c220 con 0x7fb034003bc0 2024-04-25T10:02:57.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 0x7fb03400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb046ffd640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb034003bc0 0x7fb034003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb03e7fc640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb034004810 0x7fb034004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 -- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb034003bc0 msgr2=0x7fb034003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb034003bc0 0x7fb034003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 -- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb034004810 msgr2=0x7fb034004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb034004810 0x7fb034004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.940+0000 7fb0477fe640 1 -- 172.21.15.82:0/2632237325 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb034003530 con 0x7fb0340051d0 2024-04-25T10:02:57.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb0477fe640 1 -- 172.21.15.82:0/2632237325 reap_dead start 2024-04-25T10:02:57.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb0477fe640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 0x7fb03400b9d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb04000a820 tx=0x7fb04001d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb345133e2afe23f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000a450 con 0x7fb0340051d0 2024-04-25T10:02:57.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040007310 con 0x7fb0340051d0 2024-04-25T10:02:57.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb0467fc640 1 -- 172.21.15.82:0/2632237325 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040007700 con 0x7fb0340051d0 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 msgr2=0x7fb03400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 0x7fb03400b9d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb04000a820 tx=0x7fb04001d300 comp rx=0 tx=0).stop 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- 172.21.15.82:0/2632237325 shutdown_connections 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 --2- 172.21.15.82:0/2632237325 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0340051d0 0x7fb03400b9d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- 172.21.15.82:0/2632237325 >> 172.21.15.82:0/2632237325 conn(0x7fb0480b11a0 msgr2=0x7fb0480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- 172.21.15.82:0/2632237325 shutdown_connections 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- 172.21.15.82:0/2632237325 wait complete. 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 Processor -- start 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.944+0000 7fb04d351640 1 -- start start 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0340051d0 0x7fb0480ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0480ae6f0 0x7fb0480c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0481451f0 con 0x7fb0481f6b90 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04806e290 con 0x7fb0340051d0 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb048145370 con 0x7fb0480ae6f0 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44598/0 (socket says 172.21.15.82:44598) 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 -- 172.21.15.82:0/1634621897 learned_addr learned my addr 172.21.15.82:0/1634621897 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03e7fc640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0480ae6f0 0x7fb0480c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb046ffd640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0340051d0 0x7fb0480ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 -- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0480ae6f0 msgr2=0x7fb0480c9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0480ae6f0 0x7fb0480c9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 -- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0340051d0 msgr2=0x7fb0480ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0340051d0 0x7fb0480ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 -- 172.21.15.82:0/1634621897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480aee90 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb0477fe640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb04001cd70 tx=0x7fb034005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040001fa0 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb040001a10 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb040007700 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0480b2640 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0480b2c00 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04804efa0 con 0x7fb0481f6b90 2024-04-25T10:02:57.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0400b6070 con 0x7fb0481f6b90 2024-04-25T10:02:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 0x7fb028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fb04011ead0 con 0x7fb0481f6b90 2024-04-25T10:02:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb046ffd640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 0x7fb028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.948+0000 7fb046ffd640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 0x7fb028069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb038001620 tx=0x7fb038009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:57.952+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0400bf070 con 0x7fb0481f6b90 2024-04-25T10:02:58.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.100+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0480b1cc0 con 0x7fb0481f6b90 2024-04-25T10:02:58.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.100+0000 7fb03ffff640 1 -- 172.21.15.82:0/1634621897 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fb040017010 con 0x7fb0481f6b90 2024-04-25T10:02:58.109 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:02:58.109 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 msgr2=0x7fb028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 0x7fb028069420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb038001620 tx=0x7fb038009040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 msgr2=0x7fb0480b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb04001cd70 tx=0x7fb034005840 comp rx=0 tx=0).stop 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 shutdown_connections 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb028066f60 0x7fb028069420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb0480ae6f0 0x7fb0480c9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb0481f6b90 0x7fb0480b3400 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 --2- 172.21.15.82:0/1634621897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb0340051d0 0x7fb0480ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 >> 172.21.15.82:0/1634621897 conn(0x7fb0480b11a0 msgr2=0x7fb0480718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 shutdown_connections 2024-04-25T10:02:58.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.104+0000 7fb04d351640 1 -- 172.21.15.82:0/1634621897 wait complete. 2024-04-25T10:02:58.126 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:58.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:58.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 Processor -- start 2024-04-25T10:02:58.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 -- start start 2024-04-25T10:02:58.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97f80cf590 0x7f97f80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f97f815f130 0x7f97f815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f815ba90 con 0x7f97f80cf590 2024-04-25T10:02:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80cff30 con 0x7f97f815f130 2024-04-25T10:02:58.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97ff841640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cbd20 con 0x7f97f815b6b0 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97f80cf590 0x7f97f80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f97f80cf590 0x7f97f80cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44600/0 (socket says 172.21.15.82:44600) 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 -- 172.21.15.82:0/1539341191 learned_addr learned my addr 172.21.15.82:0/1539341191 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fd5dd640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fddde640 1 --2- 172.21.15.82:0/1539341191 >> v2:172.21.15.196:3300/0 conn(0x7f97f815f130 0x7f97f815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 -- 172.21.15.82:0/1539341191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 msgr2=0x7f97f80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 -- 172.21.15.82:0/1539341191 >> v2:172.21.15.196:3300/0 conn(0x7f97f815f130 msgr2=0x7f97f815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 --2- 172.21.15.82:0/1539341191 >> v2:172.21.15.196:3300/0 conn(0x7f97f815f130 0x7f97f815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.248+0000 7f97fcddc640 1 -- 172.21.15.82:0/1539341191 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80cc000 con 0x7f97f80cf590 2024-04-25T10:02:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97fcddc640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f80cf9f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97f80c2de0 tx=0x7f97e801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9f9d46139fffe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97effff640 1 -- 172.21.15.82:0/1539341191 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80aa610 con 0x7f97f80cf590 2024-04-25T10:02:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97effff640 1 -- 172.21.15.82:0/1539341191 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e80af070 con 0x7f97f80cf590 2024-04-25T10:02:58.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97effff640 1 -- 172.21.15.82:0/1539341191 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80aa9d0 con 0x7f97f80cf590 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- 172.21.15.82:0/1539341191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 msgr2=0x7f97f80cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f80cf9f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f97f80c2de0 tx=0x7f97e801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- 172.21.15.82:0/1539341191 shutdown_connections 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 --2- 172.21.15.82:0/1539341191 >> v2:172.21.15.196:3300/0 conn(0x7f97f815f130 0x7f97f815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f80cf9f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 --2- 172.21.15.82:0/1539341191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- 172.21.15.82:0/1539341191 >> 172.21.15.82:0/1539341191 conn(0x7f97f806ae60 msgr2=0x7f97f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- 172.21.15.82:0/1539341191 shutdown_connections 2024-04-25T10:02:58.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- 172.21.15.82:0/1539341191 wait complete. 2024-04-25T10:02:58.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 Processor -- start 2024-04-25T10:02:58.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.252+0000 7f97ff841640 1 -- start start 2024-04-25T10:02:58.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 0x7f97f8098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815fef0 con 0x7f97f80cf590 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cba30 con 0x7f97f815f130 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80ca900 con 0x7f97f815b6b0 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44606/0 (socket says 172.21.15.82:44606) 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 -- 172.21.15.82:0/2228203187 learned_addr learned my addr 172.21.15.82:0/2228203187 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fcddc640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fddde640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 0x7f97f8098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 -- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 msgr2=0x7f97f80ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 -- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 msgr2=0x7f97f8098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 0x7f97f8098880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 -- 172.21.15.82:0/2228203187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8165240 con 0x7f97f80cf590 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fddde640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 0x7f97f8098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fd5dd640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f97f400d830 tx=0x7f97f400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400b6c0 con 0x7f97f80cf590 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f4019070 con 0x7f97f80cf590 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80cef00 con 0x7f97f80cf590 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f400ba60 con 0x7f97f80cf590 2024-04-25T10:02:58.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81657c0 con 0x7f97f80cf590 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f804efa0 con 0x7f97f80cf590 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97f401e460 con 0x7f97f80cf590 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 0x7f97dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f97f4085ba0 con 0x7f97f80cf590 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.256+0000 7f97fcddc640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 0x7f97dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.260+0000 7f97fcddc640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 0x7f97dc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e80ae000 tx=0x7f97e80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.260+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97f4056700 con 0x7f97f80cf590 2024-04-25T10:02:58.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97f80b2080 con 0x7f97f80cf590 2024-04-25T10:02:58.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97edffb640 1 -- 172.21.15.82:0/2228203187 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97f4056000 con 0x7f97f80cf590 2024-04-25T10:02:58.415 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 msgr2=0x7f97dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 0x7f97dc069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97e80ae000 tx=0x7f97e80a7000 comp rx=0 tx=0).stop 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 msgr2=0x7f97f8153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f97f400d830 tx=0x7f97f400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 shutdown_connections 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f97dc066f90 0x7f97dc069450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97f815b6b0 0x7f97f80ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97f815f130 0x7f97f8098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 --2- 172.21.15.82:0/2228203187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97f80cf590 0x7f97f8153920 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 >> 172.21.15.82:0/2228203187 conn(0x7f97f806ae60 msgr2=0x7f97f81554f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 shutdown_connections 2024-04-25T10:02:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.408+0000 7f97ff841640 1 -- 172.21.15.82:0/2228203187 wait complete. 2024-04-25T10:02:58.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.548+0000 7f67a59b1640 1 Processor -- start 2024-04-25T10:02:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.548+0000 7f67a59b1640 1 -- start start 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67a59b1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f67a015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67a59b1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a00cbf10 con 0x7f67a015e4f0 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67a59b1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a00cc1f0 con 0x7f67a00b3cf0 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67a59b1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00cc4d0 con 0x7f67a00cfca0 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f67a015e4f0 0x7f67a00caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41118/0 (socket says 172.21.15.82:41118) 2024-04-25T10:02:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679f7fe640 1 -- 172.21.15.82:0/115237293 learned_addr learned my addr 172.21.15.82:0/115237293 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:02:58.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f67a015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618471743 0 0) 0x7f67a00cbf10 con 0x7f67a015e4f0 2024-04-25T10:02:58.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f678c003530 con 0x7f67a015e4f0 2024-04-25T10:02:58.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524923343 0 0) 0x7f678c003530 con 0x7f67a015e4f0 2024-04-25T10:02:58.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 msgr2=0x7f67a015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f67a015dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 >> v1:172.21.15.196:6789/0 conn(0x7f67a00b3cf0 legacy=0x7f67a00cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00cc7b0 con 0x7f67a015e4f0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6798001a00 con 0x7f67a015e4f0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 >> v1:172.21.15.82:6789/0 conn(0x7f67a015e4f0 legacy=0x7f67a00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c003a60 0x7f678c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f678c004810 0x7f678c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f678c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00cc4d0 con 0x7f678c004810 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678c00bb00 con 0x7f678c003a60 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678c00bd00 con 0x7f67a00cfca0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f678c004810 0x7f678c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679effd640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c003a60 0x7f678c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f679f7fe640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f678c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 -- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 msgr2=0x7f678c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a00cfca0 0x7f678c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 -- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c003a60 msgr2=0x7f678c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c003a60 0x7f678c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.552+0000 7f67967fc640 1 -- 172.21.15.82:0/115237293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678c003530 con 0x7f678c004810 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67967fc640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f678c004810 0x7f678c004c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6788009ae0 tx=0x7f678801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2e7debefe088c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880096b0 con 0x7f678c004810 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67880a6480 con 0x7f678c004810 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679e7fc640 1 -- 172.21.15.82:0/115237293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67880a6870 con 0x7f678c004810 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- 172.21.15.82:0/115237293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f678c004810 msgr2=0x7f678c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 --2- 172.21.15.82:0/115237293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f678c004810 0x7f678c004c90 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6788009ae0 tx=0x7f678801ca70 comp rx=0 tx=0).stop 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 -- 172.21.15.82:0/115237293 reap_dead start 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- 172.21.15.82:0/115237293 shutdown_connections 2024-04-25T10:02:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- 172.21.15.82:0/115237293 >> 172.21.15.82:0/115237293 conn(0x7f67a006ae60 msgr2=0x7f67a006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- 172.21.15.82:0/115237293 shutdown_connections 2024-04-25T10:02:58.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- 172.21.15.82:0/115237293 wait complete. 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 Processor -- start 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- start start 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c004810 0x7f67a00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a0159db0 0x7f67a00b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00b4250 con 0x7f67a015a1d0 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00b40d0 con 0x7f678c004810 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67a59b1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015e920 con 0x7f67a0159db0 2024-04-25T10:02:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44626/0 (socket says 172.21.15.82:44626) 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 -- 172.21.15.82:0/2326324503 learned_addr learned my addr 172.21.15.82:0/2326324503 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679effd640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c004810 0x7f67a00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f67967fc640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a0159db0 0x7f67a00b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 -- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a0159db0 msgr2=0x7f67a00b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a0159db0 0x7f67a00b0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 -- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c004810 msgr2=0x7f67a00710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c004810 0x7f67a00710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 -- 172.21.15.82:0/2326324503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00afd40 con 0x7f67a015a1d0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.556+0000 7f679f7fe640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67a00d01d0 tx=0x7f6798011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798010040 con 0x7f67a015a1d0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679801a070 con 0x7f67a015a1d0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a00b0020 con 0x7f67a015a1d0 2024-04-25T10:02:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798013a40 con 0x7f67a015a1d0 2024-04-25T10:02:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a00b05e0 con 0x7f67a015a1d0 2024-04-25T10:02:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a004efa0 con 0x7f67a015a1d0 2024-04-25T10:02:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6798013be0 con 0x7f67a015a1d0 2024-04-25T10:02:58.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 0x7f6780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f67980882f0 con 0x7f67a015a1d0 2024-04-25T10:02:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679effd640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 0x7f6780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.560+0000 7f679effd640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 0x7f6780069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6790007fd0 tx=0x7f6790009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.564+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6798058ef0 con 0x7f67a015a1d0 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.716+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67a00b3060 con 0x7f67a015a1d0 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.716+0000 7f679cff9640 1 -- 172.21.15.82:0/2326324503 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f67980587f0 con 0x7f67a015a1d0 2024-04-25T10:02:58.724 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 msgr2=0x7f6780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 0x7f6780069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6790007fd0 tx=0x7f6790009040 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 msgr2=0x7f67a0158fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f67a00d01d0 tx=0x7f6798011a70 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 shutdown_connections 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6780066f60 0x7f6780069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67a0159db0 0x7f67a00b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67a015a1d0 0x7f67a0158fa0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 --2- 172.21.15.82:0/2326324503 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f678c004810 0x7f67a00710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 >> 172.21.15.82:0/2326324503 conn(0x7f67a006ae60 msgr2=0x7f67a015ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 shutdown_connections 2024-04-25T10:02:58.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.720+0000 7f67a59b1640 1 -- 172.21.15.82:0/2326324503 wait complete. 2024-04-25T10:02:58.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.856+0000 7fc8f8cf8640 1 Processor -- start 2024-04-25T10:02:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.856+0000 7fc8f8cf8640 1 -- start start 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.856+0000 7fc8f8cf8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8f4159bd0 0x7fc8f40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.856+0000 7fc8f8cf8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8f4159fb0 0x7fc8f40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f40b26f0 0x7fc8f40b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f4072780 con 0x7fc8f4159bd0 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f40c9ef0 con 0x7fc8f4159fb0 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f4071b70 con 0x7fc8f40b26f0 2024-04-25T10:02:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8f4159bd0 0x7fc8f40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8f4159bd0 0x7fc8f40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44642/0 (socket says 172.21.15.82:44642) 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 -- 172.21.15.82:0/710126505 learned_addr learned my addr 172.21.15.82:0/710126505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 -- 172.21.15.82:0/710126505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f40b26f0 msgr2=0x7fc8f40b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 --2- 172.21.15.82:0/710126505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f40b26f0 0x7fc8f40b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f1d9b640 1 --2- 172.21.15.82:0/710126505 >> v2:172.21.15.196:3300/0 conn(0x7fc8f4159fb0 0x7fc8f40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 -- 172.21.15.82:0/710126505 >> v2:172.21.15.196:3300/0 conn(0x7fc8f4159fb0 msgr2=0x7fc8f40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 --2- 172.21.15.82:0/710126505 >> v2:172.21.15.196:3300/0 conn(0x7fc8f4159fb0 0x7fc8f40b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 -- 172.21.15.82:0/710126505 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f4071e50 con 0x7fc8f4159bd0 2024-04-25T10:02:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f259c640 1 --2- 172.21.15.82:0/710126505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f4159bd0 0x7fc8f40c99b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406bcc0 tx=0x7fc8e401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd465eb8f8f0389 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f159a640 1 -- 172.21.15.82:0/710126505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40b1070 con 0x7fc8f4159bd0 2024-04-25T10:02:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f159a640 1 -- 172.21.15.82:0/710126505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e40ac410 con 0x7fc8f4159bd0 2024-04-25T10:02:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f159a640 1 -- 172.21.15.82:0/710126505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40ac7e0 con 0x7fc8f4159bd0 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/710126505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f4159bd0 msgr2=0x7fc8f40c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/710126505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f4159bd0 0x7fc8f40c99b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406bcc0 tx=0x7fc8e401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/710126505 shutdown_connections 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/710126505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f40b26f0 0x7fc8f40b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/710126505 >> v2:172.21.15.196:3300/0 conn(0x7fc8f4159fb0 0x7fc8f40b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/710126505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f4159bd0 0x7fc8f40c99b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/710126505 >> 172.21.15.82:0/710126505 conn(0x7fc8f406ae60 msgr2=0x7fc8f406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/710126505 shutdown_connections 2024-04-25T10:02:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.860+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/710126505 wait complete. 2024-04-25T10:02:58.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 Processor -- start 2024-04-25T10:02:58.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- start start 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f4159bd0 0x7fc8f40c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4159fb0 0x7fc8f40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40cbd10 con 0x7fc8f40b26f0 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40cbb90 con 0x7fc8f4159fb0 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f40cbe90 con 0x7fc8f4159bd0 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44646/0 (socket says 172.21.15.82:44646) 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 -- 172.21.15.82:0/163479222 learned_addr learned my addr 172.21.15.82:0/163479222 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:58.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f1d9b640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f4159bd0 0x7fc8f40c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 -- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f4159bd0 msgr2=0x7fc8f40c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f2d9d640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4159fb0 0x7fc8f40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f4159bd0 0x7fc8f40c5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 -- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4159fb0 msgr2=0x7fc8f40c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4159fb0 0x7fc8f40c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 -- 172.21.15.82:0/163479222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f40987d0 con 0x7fc8f40b26f0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f259c640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc8e401cd90 tx=0x7fc8e4019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40b1040 con 0x7fc8f40b26f0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e40ba070 con 0x7fc8f40b26f0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f40c6e40 con 0x7fc8f40b26f0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40b5440 con 0x7fc8f40b26f0 2024-04-25T10:02:58.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f4098a70 con 0x7fc8f40b26f0 2024-04-25T10:02:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f404efa0 con 0x7fc8f40b26f0 2024-04-25T10:02:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.864+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8e40b55e0 con 0x7fc8f40b26f0 2024-04-25T10:02:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.868+0000 7fc8e2ffd640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 0x7fc8d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.868+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc8e411f180 con 0x7fc8f40b26f0 2024-04-25T10:02:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.868+0000 7fc8f1d9b640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 0x7fc8d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.868+0000 7fc8f1d9b640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 0x7fc8d4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40af240 tx=0x7fc8e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:58.872+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8e4016110 con 0x7fc8f40b26f0 2024-04-25T10:02:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.024+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8f406f2a0 con 0x7fc8f40b26f0 2024-04-25T10:02:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.024+0000 7fc8e2ffd640 1 -- 172.21.15.82:0/163479222 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc8e40efce0 con 0x7fc8f40b26f0 2024-04-25T10:02:59.032 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 msgr2=0x7fc8d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 0x7fc8d4069420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc8f40af240 tx=0x7fc8e8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 msgr2=0x7fc8f40c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc8e401cd90 tx=0x7fc8e4019290 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 shutdown_connections 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f60 0x7fc8d4069420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f4159bd0 0x7fc8f40c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f4159fb0 0x7fc8f40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 --2- 172.21.15.82:0/163479222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f40b26f0 0x7fc8f40c6430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 >> 172.21.15.82:0/163479222 conn(0x7fc8f406ae60 msgr2=0x7fc8f40b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 shutdown_connections 2024-04-25T10:02:59.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.028+0000 7fc8f8cf8640 1 -- 172.21.15.82:0/163479222 wait complete. 2024-04-25T10:02:59.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:59.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 Processor -- start 2024-04-25T10:02:59.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 -- start start 2024-04-25T10:02:59.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83d015b6b0 0x7f83d00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83d00cf590 0x7f83d00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d00cbd20 con 0x7f83d015f130 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d015ba90 con 0x7f83d015b6b0 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83d4d38640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d00cc000 con 0x7f83d00cf590 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83d015b6b0 0x7f83d00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83d015b6b0 0x7f83d00ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44652/0 (socket says 172.21.15.82:44652) 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 -- 172.21.15.82:0/3104371884 learned_addr learned my addr 172.21.15.82:0/3104371884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 -- 172.21.15.82:0/3104371884 >> v2:172.21.15.196:3300/0 conn(0x7f83d00cf590 msgr2=0x7f83d00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 --2- 172.21.15.82:0/3104371884 >> v2:172.21.15.196:3300/0 conn(0x7f83d00cf590 0x7f83d00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 -- 172.21.15.82:0/3104371884 >> v1:172.21.15.196:6789/0 conn(0x7f83d015f130 legacy=0x7f83d015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.164+0000 7f83ce59c640 1 -- 172.21.15.82:0/3104371884 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00cc2e0 con 0x7f83d015b6b0 2024-04-25T10:02:59.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83ce59c640 1 --2- 172.21.15.82:0/3104371884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d00ca3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b4430 tx=0x7f83c401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90ef4d8a5a1a356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cd59a640 1 -- 172.21.15.82:0/3104371884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40b1070 con 0x7f83d015b6b0 2024-04-25T10:02:59.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cd59a640 1 -- 172.21.15.82:0/3104371884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c40ac630 con 0x7f83d015b6b0 2024-04-25T10:02:59.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cd59a640 1 -- 172.21.15.82:0/3104371884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c40a8050 con 0x7f83d015b6b0 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- 172.21.15.82:0/3104371884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 msgr2=0x7f83d00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3104371884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d00ca3c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f83d00b4430 tx=0x7f83c401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- 172.21.15.82:0/3104371884 shutdown_connections 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3104371884 >> v2:172.21.15.196:3300/0 conn(0x7f83d00cf590 0x7f83d00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3104371884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d00ca3c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- 172.21.15.82:0/3104371884 >> 172.21.15.82:0/3104371884 conn(0x7f83d006ae60 msgr2=0x7f83d006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- 172.21.15.82:0/3104371884 shutdown_connections 2024-04-25T10:02:59.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- 172.21.15.82:0/3104371884 wait complete. 2024-04-25T10:02:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 Processor -- start 2024-04-25T10:02:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- start start 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d00cf590 0x7f83d00c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83d015f130 0x7f83d0097d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00cadb0 con 0x7f83d015b6b0 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00ca900 con 0x7f83d00cf590 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83d4d38640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d00caf30 con 0x7f83d015f130 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44664/0 (socket says 172.21.15.82:44664) 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 -- 172.21.15.82:0/3292816483 learned_addr learned my addr 172.21.15.82:0/3292816483 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83ce59c640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d00cf590 0x7f83d00c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83ced9d640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83d015f130 0x7f83d0097d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 -- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83d015f130 msgr2=0x7f83d0097d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83d015f130 0x7f83d0097d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 -- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d00cf590 msgr2=0x7f83d00c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d00cf590 0x7f83d00c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 -- 172.21.15.82:0/3292816483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00ac210 con 0x7f83d015b6b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83cdd9b640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83c000b550 tx=0x7f83c000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.168+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0014070 con 0x7f83d015b6b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c0010890 con 0x7f83d015b6b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d00c2390 con 0x7f83d015b6b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c0010c80 con 0x7f83d015b6b0 2024-04-25T10:02:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d00987e0 con 0x7f83d015b6b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d004efa0 con 0x7f83d015b6b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83c0017070 con 0x7f83d015b6b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83beffd640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 0x7f83b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f83c0085190 con 0x7f83d015b6b0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83ce59c640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 0x7f83b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.172+0000 7f83ce59c640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 0x7f83b0069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ae000 tx=0x7f83c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.176+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83c0055e10 con 0x7f83d015b6b0 2024-04-25T10:02:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.332+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f83d00af490 con 0x7f83d015b6b0 2024-04-25T10:02:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83beffd640 1 -- 172.21.15.82:0/3292816483 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f83c0010e20 con 0x7f83d015b6b0 2024-04-25T10:02:59.342 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:02:59.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 msgr2=0x7f83b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 0x7f83b0069450 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f83c40ae000 tx=0x7f83c4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:59.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 msgr2=0x7f83d0097830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f83c000b550 tx=0x7f83c000ba10 comp rx=0 tx=0).stop 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 shutdown_connections 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83b0066f90 0x7f83b0069450 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83d015f130 0x7f83d0097d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83d015b6b0 0x7f83d0097830 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 --2- 172.21.15.82:0/3292816483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83d00cf590 0x7f83d00c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 >> 172.21.15.82:0/3292816483 conn(0x7f83d006ae60 msgr2=0x7f83d00b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 shutdown_connections 2024-04-25T10:02:59.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.336+0000 7f83d4d38640 1 -- 172.21.15.82:0/3292816483 wait complete. 2024-04-25T10:02:59.360 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:02:59.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 Processor -- start 2024-04-25T10:02:59.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 -- start start 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d07f0 0x7f6c940d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c940d1cd0 0x7f6c9406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9406be60 con 0x7f6c9415da80 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9406c440 con 0x7f6c940d1cd0 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c9b01b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9415de60 con 0x7f6c940d07f0 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c940d1cd0 0x7f6c9406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c940d1cd0 0x7f6c9406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44676/0 (socket says 172.21.15.82:44676) 2024-04-25T10:02:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c98db7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c9415da80 0x7f6c940d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41126/0 (socket says 172.21.15.82:41126) 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 -- 172.21.15.82:0/1551991161 learned_addr learned my addr 172.21.15.82:0/1551991161 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c93fff640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d07f0 0x7f6c940d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 -- 172.21.15.82:0/1551991161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d07f0 msgr2=0x7f6c940d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d07f0 0x7f6c940d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 -- 172.21.15.82:0/1551991161 >> v1:172.21.15.82:6789/0 conn(0x7f6c9415da80 legacy=0x7f6c940d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 -- 172.21.15.82:0/1551991161 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940d2450 con 0x7f6c940d1cd0 2024-04-25T10:02:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c937fe640 1 -- 172.21.15.82:0/1551991161 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704009731 0 0) 0x7f6c9406be60 con 0x7f6c9415da80 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c995b8640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c940d1cd0 0x7f6c9406c8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009e50 tx=0x7f6c8c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9406c382416492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c937fe640 1 -- 172.21.15.82:0/1551991161 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0a58b0 con 0x7f6c940d1cd0 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c937fe640 1 -- 172.21.15.82:0/1551991161 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0a5a50 con 0x7f6c940d1cd0 2024-04-25T10:02:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.468+0000 7f6c937fe640 1 -- 172.21.15.82:0/1551991161 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0a5e20 con 0x7f6c940d1cd0 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/1551991161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c940d1cd0 msgr2=0x7f6c9406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c940d1cd0 0x7f6c9406c8d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c009e50 tx=0x7f6c8c01ca00 comp rx=0 tx=0).stop 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/1551991161 shutdown_connections 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c940d1cd0 0x7f6c9406c8d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/1551991161 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d07f0 0x7f6c940d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/1551991161 >> 172.21.15.82:0/1551991161 conn(0x7f6c9408b000 msgr2=0x7f6c940d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/1551991161 shutdown_connections 2024-04-25T10:02:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/1551991161 wait complete. 2024-04-25T10:02:59.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 Processor -- start 2024-04-25T10:02:59.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- start start 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c940d07f0 0x7f6c940c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d1cd0 0x7f6c940ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9406d7a0 con 0x7f6c9415da80 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9406d620 con 0x7f6c940d07f0 2024-04-25T10:02:59.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9406d920 con 0x7f6c940d1cd0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44682/0 (socket says 172.21.15.82:44682) 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 -- 172.21.15.82:0/2065903823 learned_addr learned my addr 172.21.15.82:0/2065903823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 -- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d1cd0 msgr2=0x7f6c940ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d1cd0 0x7f6c940ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 -- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c940d07f0 msgr2=0x7f6c940c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c940d07f0 0x7f6c940c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 -- 172.21.15.82:0/2065903823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940b77c0 con 0x7f6c9415da80 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c995b8640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01a630 tx=0x7f6c8c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0bc020 con 0x7f6c9415da80 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c0b8470 con 0x7f6c9415da80 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0a58b0 con 0x7f6c9415da80 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c940b7aa0 con 0x7f6c9415da80 2024-04-25T10:02:59.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.472+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c940b8060 con 0x7f6c9415da80 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9404efa0 con 0x7f6c9415da80 2024-04-25T10:02:59.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c8c0bf020 con 0x7f6c9415da80 2024-04-25T10:02:59.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c917fa640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6c8c120a50 con 0x7f6c9415da80 2024-04-25T10:02:59.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c98db7640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.476+0000 7f6c98db7640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 0x7f6c78069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c941f9900 tx=0x7f6c88007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.480+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c8c0c6030 con 0x7f6c9415da80 2024-04-25T10:02:59.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:02:59.680+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f6c940b29e0 con 0x7f6c9415da80 2024-04-25T10:03:00.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.624+0000 7f6c917fa640 1 -- 172.21.15.82:0/2065903823 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v71) v1 ==== 145+0+0 (secure 0 0 0) 0x7f6c8c016620 con 0x7f6c9415da80 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 msgr2=0x7f6c78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 0x7f6c78069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6c941f9900 tx=0x7f6c88007970 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 msgr2=0x7f6c940b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c01a630 tx=0x7f6c8c0197a0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 shutdown_connections 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c78066f60 0x7f6c78069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c940d1cd0 0x7f6c940ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c9415da80 0x7f6c940b43c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 --2- 172.21.15.82:0/2065903823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c940d07f0 0x7f6c940c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 >> 172.21.15.82:0/2065903823 conn(0x7f6c9408b000 msgr2=0x7f6c940ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 shutdown_connections 2024-04-25T10:03:00.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.628+0000 7f6c9b01b640 1 -- 172.21.15.82:0/2065903823 wait complete. 2024-04-25T10:03:00.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:03:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 Processor -- start 2024-04-25T10:03:00.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 -- start start 2024-04-25T10:03:00.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65040cf590 0x7f65040cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6504072000 0x7f6504072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040cbd20 con 0x7f650415b6b0 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f650415ba90 con 0x7f65040cf590 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f650944c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040cc000 con 0x7f6504072000 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65040cf590 0x7f65040cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65040cf590 0x7f65040cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44694/0 (socket says 172.21.15.82:44694) 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 -- 172.21.15.82:0/21817159 learned_addr learned my addr 172.21.15.82:0/21817159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65037fe640 1 --2- 172.21.15.82:0/21817159 >> v2:172.21.15.196:3300/0 conn(0x7f6504072000 0x7f6504072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 -- 172.21.15.82:0/21817159 >> v2:172.21.15.196:3300/0 conn(0x7f6504072000 msgr2=0x7f6504072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 --2- 172.21.15.82:0/21817159 >> v2:172.21.15.196:3300/0 conn(0x7f6504072000 0x7f6504072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 -- 172.21.15.82:0/21817159 >> v1:172.21.15.196:6789/0 conn(0x7f650415b6b0 legacy=0x7f65040ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.764+0000 7f65027fc640 1 -- 172.21.15.82:0/21817159 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040cc2e0 con 0x7f65040cf590 2024-04-25T10:03:00.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f65027fc640 1 --2- 172.21.15.82:0/21817159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65040cf590 0x7f65040cfa20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f650406f000 tx=0x7f64f801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4df1882d20a9387 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f6501ffb640 1 -- 172.21.15.82:0/21817159 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80b1070 con 0x7f65040cf590 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f6501ffb640 1 -- 172.21.15.82:0/21817159 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f80ac630 con 0x7f65040cf590 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f6501ffb640 1 -- 172.21.15.82:0/21817159 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a8050 con 0x7f65040cf590 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- 172.21.15.82:0/21817159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65040cf590 msgr2=0x7f65040cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- 172.21.15.82:0/21817159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65040cf590 0x7f65040cfa20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f650406f000 tx=0x7f64f801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- 172.21.15.82:0/21817159 shutdown_connections 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- 172.21.15.82:0/21817159 >> v2:172.21.15.196:3300/0 conn(0x7f6504072000 0x7f6504072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- 172.21.15.82:0/21817159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65040cf590 0x7f65040cfa20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- 172.21.15.82:0/21817159 >> 172.21.15.82:0/21817159 conn(0x7f650406ae60 msgr2=0x7f650406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- 172.21.15.82:0/21817159 shutdown_connections 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- 172.21.15.82:0/21817159 wait complete. 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 Processor -- start 2024-04-25T10:03:00.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- start start 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 0x7f6504152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650415b6b0 0x7f6504153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040caf30 con 0x7f6504072000 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040cadb0 con 0x7f650415b6b0 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.768+0000 7f650944c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65041682a0 con 0x7f65040cf590 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44704/0 (socket says 172.21.15.82:44704) 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 -- 172.21.15.82:0/4031043190 learned_addr learned my addr 172.21.15.82:0/4031043190 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f65037fe640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650415b6b0 0x7f6504153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f65027fc640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 0x7f6504152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 -- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 msgr2=0x7f6504152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 0x7f6504152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 -- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650415b6b0 msgr2=0x7f6504153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650415b6b0 0x7f6504153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 -- 172.21.15.82:0/4031043190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040b40c0 con 0x7f6504072000 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f65027fc640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 0x7f6504152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f6502ffd640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f64f400b440 tx=0x7f64f400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4015020 con 0x7f6504072000 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f4011890 con 0x7f6504072000 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040c2400 con 0x7f6504072000 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f4011c80 con 0x7f6504072000 2024-04-25T10:03:00.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65041539b0 con 0x7f6504072000 2024-04-25T10:03:00.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650404efa0 con 0x7f6504072000 2024-04-25T10:03:00.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64f4018020 con 0x7f6504072000 2024-04-25T10:03:00.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 0x7f64f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f64f40852a0 con 0x7f6504072000 2024-04-25T10:03:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f65027fc640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 0x7f64f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.772+0000 7f65027fc640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 0x7f64f0069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64f80ae000 tx=0x7f64f80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.776+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64f401f030 con 0x7f6504072000 2024-04-25T10:03:00.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.936+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f65040b2d90 con 0x7f6504072000 2024-04-25T10:03:00.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.944+0000 7f64eb7fe640 1 -- 172.21.15.82:0/4031043190 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f64f400d040 con 0x7f6504072000 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 msgr2=0x7f64f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 0x7f64f0069450 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f64f80ae000 tx=0x7f64f80a7000 comp rx=0 tx=0).stop 2024-04-25T10:03:00.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 msgr2=0x7f65040c1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f64f400b440 tx=0x7f64f400b900 comp rx=0 tx=0).stop 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 shutdown_connections 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64f0066f90 0x7f64f0069450 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65040cf590 0x7f6504152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f650415b6b0 0x7f6504153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 --2- 172.21.15.82:0/4031043190 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504072000 0x7f65040c1ec0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 >> 172.21.15.82:0/4031043190 conn(0x7f650406ae60 msgr2=0x7f650406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 shutdown_connections 2024-04-25T10:03:00.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:00.948+0000 7f650944c640 1 -- 172.21.15.82:0/4031043190 wait complete. 2024-04-25T10:03:00.971 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:00.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:01.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:01.063 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:01.063 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:01.063 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:01.069 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:01.069 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:01.118 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 Processor -- start 2024-04-25T10:03:01.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 -- start start 2024-04-25T10:03:01.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fac64144ca0 0x7fac6406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac6406ec60 0x7fac64142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac640ccc70 con 0x7fac640ce510 2024-04-25T10:03:01.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac640cc2b0 con 0x7fac6406ec60 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6c79e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6406e290 con 0x7fac64144ca0 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac6406ec60 0x7fac64142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fac6406ec60 0x7fac64142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44708/0 (socket says 172.21.15.82:44708) 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 -- 172.21.15.82:0/2296369957 learned_addr learned my addr 172.21.15.82:0/2296369957 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac69d39640 1 --2- 172.21.15.82:0/2296369957 >> v2:172.21.15.196:3300/0 conn(0x7fac64144ca0 0x7fac6406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 -- 172.21.15.82:0/2296369957 >> v2:172.21.15.196:3300/0 conn(0x7fac64144ca0 msgr2=0x7fac6406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 --2- 172.21.15.82:0/2296369957 >> v2:172.21.15.196:3300/0 conn(0x7fac64144ca0 0x7fac6406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 -- 172.21.15.82:0/2296369957 >> v1:172.21.15.196:6789/0 conn(0x7fac640ce510 legacy=0x7fac640ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.272+0000 7fac6ad3b640 1 -- 172.21.15.82:0/2296369957 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6406dc40 con 0x7fac6406ec60 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6ad3b640 1 --2- 172.21.15.82:0/2296369957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6406ec60 0x7fac64142680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fac64070fe0 tx=0x7fac5c01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e934befde6fef0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69538640 1 -- 172.21.15.82:0/2296369957 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0b4070 con 0x7fac6406ec60 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69538640 1 -- 172.21.15.82:0/2296369957 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5c0af400 con 0x7fac6406ec60 2024-04-25T10:03:01.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69538640 1 -- 172.21.15.82:0/2296369957 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5c0af7d0 con 0x7fac6406ec60 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- 172.21.15.82:0/2296369957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6406ec60 msgr2=0x7fac64142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- 172.21.15.82:0/2296369957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6406ec60 0x7fac64142680 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fac64070fe0 tx=0x7fac5c01f1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- 172.21.15.82:0/2296369957 shutdown_connections 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- 172.21.15.82:0/2296369957 >> v2:172.21.15.196:3300/0 conn(0x7fac64144ca0 0x7fac6406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- 172.21.15.82:0/2296369957 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac6406ec60 0x7fac64142680 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- 172.21.15.82:0/2296369957 >> 172.21.15.82:0/2296369957 conn(0x7fac640b11a0 msgr2=0x7fac640b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- 172.21.15.82:0/2296369957 shutdown_connections 2024-04-25T10:03:01.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- 172.21.15.82:0/2296369957 wait complete. 2024-04-25T10:03:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 Processor -- start 2024-04-25T10:03:01.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- start start 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac6406ec60 0x7fac641e5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac64144ca0 0x7fac641e5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac641451f0 con 0x7fac640ce510 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac640cc900 con 0x7fac64144ca0 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6c79e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac64145370 con 0x7fac6406ec60 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44712/0 (socket says 172.21.15.82:44712) 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 -- 172.21.15.82:0/4261098685 learned_addr learned my addr 172.21.15.82:0/4261098685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:01.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6ad3b640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac64144ca0 0x7fac641e5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac6a53a640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac6406ec60 0x7fac641e5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 -- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac6406ec60 msgr2=0x7fac641e5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac6406ec60 0x7fac641e5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 -- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac64144ca0 msgr2=0x7fac641e5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac64144ca0 0x7fac641e5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.276+0000 7fac69d39640 1 -- 172.21.15.82:0/4261098685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac641e6630 con 0x7fac640ce510 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac69d39640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d5d0 tx=0x7fac6000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60004240 con 0x7fac640ce510 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac600043e0 con 0x7fac640ce510 2024-04-25T10:03:01.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac641f27b0 con 0x7fac640ce510 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60003840 con 0x7fac640ce510 2024-04-25T10:03:01.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac641f2d70 con 0x7fac640ce510 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6404efa0 con 0x7fac640ce510 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac600039e0 con 0x7fac640ce510 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 0x7fac48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac6a53a640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 0x7fac48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fac600856b0 con 0x7fac640ce510 2024-04-25T10:03:01.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.280+0000 7fac6a53a640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 0x7fac48069450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac58004e30 tx=0x7fac58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.284+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac6001c360 con 0x7fac640ce510 2024-04-25T10:03:01.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.443+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac6415b9c0 con 0x7fac640ce510 2024-04-25T10:03:01.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.443+0000 7fac56ffd640 1 -- 172.21.15.82:0/4261098685 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fac60082a30 con 0x7fac640ce510 2024-04-25T10:03:01.452 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:01.452 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 msgr2=0x7fac48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 0x7fac48069450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fac58004e30 tx=0x7fac58009040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 msgr2=0x7fac641e5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fac6000d5d0 tx=0x7fac6000da90 comp rx=0 tx=0).stop 2024-04-25T10:03:01.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 shutdown_connections 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fac48066f90 0x7fac48069450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac64144ca0 0x7fac641e5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac640ce510 0x7fac641e5a80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 --2- 172.21.15.82:0/4261098685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fac6406ec60 0x7fac641e5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 >> 172.21.15.82:0/4261098685 conn(0x7fac640b11a0 msgr2=0x7fac640c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 shutdown_connections 2024-04-25T10:03:01.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.447+0000 7fac6c79e640 1 -- 172.21.15.82:0/4261098685 wait complete. 2024-04-25T10:03:01.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:01.474 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:01.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 Processor -- start 2024-04-25T10:03:01.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 -- start start 2024-04-25T10:03:01.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f62ac0cfa20 0x7f62ac072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0cbd20 con 0x7f62ac15b6b0 2024-04-25T10:03:01.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0cc000 con 0x7f62ac0cf590 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62b0e3e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac15ba90 con 0x7f62ac0cfa20 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aa59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f62ac15b6b0 0x7f62ac0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41128/0 (socket says 172.21.15.82:41128) 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aa59c640 1 -- 172.21.15.82:0/187615484 learned_addr learned my addr 172.21.15.82:0/187615484 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aad9d640 1 --2- 172.21.15.82:0/187615484 >> v2:172.21.15.82:3300/0 conn(0x7f62ac0cfa20 0x7f62ac072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 894487585 0 0) 0x7f62ac0cbd20 con 0x7f62ac15b6b0 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6298003530 con 0x7f62ac15b6b0 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aad9d640 1 -- 172.21.15.82:0/187615484 >> v1:172.21.15.196:6789/0 conn(0x7f62ac0cf590 legacy=0x7f62ac072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404111966 0 0) 0x7f6298003530 con 0x7f62ac15b6b0 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aad9d640 1 -- 172.21.15.82:0/187615484 >> v1:172.21.15.82:6789/0 conn(0x7f62ac15b6b0 legacy=0x7f62ac0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.595+0000 7f62aad9d640 1 -- 172.21.15.82:0/187615484 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0cc2e0 con 0x7f62ac0cfa20 2024-04-25T10:03:01.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aad9d640 1 --2- 172.21.15.82:0/187615484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cfa20 0x7f62ac072710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009d90 tx=0x7f62a401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4763e26a3fbd0077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a40b2070 con 0x7f62ac0cfa20 2024-04-25T10:03:01.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a40a56a0 con 0x7f62ac0cfa20 2024-04-25T10:03:01.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62a9d9b640 1 -- 172.21.15.82:0/187615484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c001680 con 0x7f62ac0cfa20 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- 172.21.15.82:0/187615484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cfa20 msgr2=0x7f62ac072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/187615484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cfa20 0x7f62ac072710 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009d90 tx=0x7f62a401c860 comp rx=0 tx=0).stop 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- 172.21.15.82:0/187615484 shutdown_connections 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/187615484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cfa20 0x7f62ac072710 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- 172.21.15.82:0/187615484 >> 172.21.15.82:0/187615484 conn(0x7f62ac06ae60 msgr2=0x7f62ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- 172.21.15.82:0/187615484 shutdown_connections 2024-04-25T10:03:01.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- 172.21.15.82:0/187615484 wait complete. 2024-04-25T10:03:01.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 Processor -- start 2024-04-25T10:03:01.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- start start 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62ac0cfa20 0x7f62ac0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62ac15b6b0 0x7f62ac1656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0cadb0 con 0x7f62ac0cf590 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0cfe50 con 0x7f62ac15b6b0 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62b0e3e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0caf30 con 0x7f62ac0cfa20 2024-04-25T10:03:01.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44724/0 (socket says 172.21.15.82:44724) 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 -- 172.21.15.82:0/1986290436 learned_addr learned my addr 172.21.15.82:0/1986290436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 -- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62ac0cfa20 msgr2=0x7f62ac0ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62ac0cfa20 0x7f62ac0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 -- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62ac15b6b0 msgr2=0x7f62ac1656c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62ac15b6b0 0x7f62ac1656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 -- 172.21.15.82:0/1986290436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac1594c0 con 0x7f62ac0cf590 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62aa59c640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c520 tx=0x7f629c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c017070 con 0x7f62ac0cf590 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.599+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c005d10 con 0x7f62ac0cf590 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac158f00 con 0x7f62ac0cf590 2024-04-25T10:03:01.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c0072b0 con 0x7f62ac0cf590 2024-04-25T10:03:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac0cdae0 con 0x7f62ac0cf590 2024-04-25T10:03:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac158c90 con 0x7f62ac0cf590 2024-04-25T10:03:01.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f629c01a070 con 0x7f62ac0cf590 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a2ffd640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 0x7f628c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a3fff640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 0x7f628c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f629c0849a0 con 0x7f62ac0cf590 2024-04-25T10:03:01.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.603+0000 7f62a3fff640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 0x7f628c069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6294000f70 tx=0x7f62940079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.607+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62ac1594c0 con 0x7f62ac0cf590 2024-04-25T10:03:01.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.767+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62ac04efa0 con 0x7f62ac0cf590 2024-04-25T10:03:01.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.767+0000 7f62a2ffd640 1 -- 172.21.15.82:0/1986290436 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f629c0554f0 con 0x7f62ac0cf590 2024-04-25T10:03:01.777 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 msgr2=0x7f628c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 0x7f628c069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6294000f70 tx=0x7f62940079e0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 msgr2=0x7f62ac0985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c520 tx=0x7f629c00c9e0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 shutdown_connections 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f628c066f60 0x7f628c069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f62ac0cfa20 0x7f62ac0ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62ac15b6b0 0x7f62ac1656c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 --2- 172.21.15.82:0/1986290436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62ac0cf590 0x7f62ac0985f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 >> 172.21.15.82:0/1986290436 conn(0x7f62ac06ae60 msgr2=0x7f62ac0b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 shutdown_connections 2024-04-25T10:03:01.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.771+0000 7f62b0e3e640 1 -- 172.21.15.82:0/1986290436 wait complete. 2024-04-25T10:03:01.794 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.899+0000 7f3cd7175640 1 Processor -- start 2024-04-25T10:03:01.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.899+0000 7f3cd7175640 1 -- start start 2024-04-25T10:03:01.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cd006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd00ca490 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd006f900 con 0x7f3cd006c2e0 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00ca240 con 0x7f3cd015de00 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd5712640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3cd00ce8e0 0x7f3cd006e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41144/0 (socket says 172.21.15.82:41144) 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd5712640 1 -- 172.21.15.82:0/3241539648 learned_addr learned my addr 172.21.15.82:0/3241539648 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84005461 0 0) 0x7f3cd00ca490 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb8003530 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd4f11640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cd006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951914450 0 0) 0x7f3cb8003530 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 msgr2=0x7f3cd006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cd006bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 >> v1:172.21.15.196:6789/0 conn(0x7f3cd006c2e0 legacy=0x7f3cd00ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd00cb780 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3cc8001a00 con 0x7f3cd00ce8e0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 >> v1:172.21.15.82:6789/0 conn(0x7f3cd00ce8e0 legacy=0x7f3cd006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cb8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb8004090 0x7f3cb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cb8004da0 0x7f3cb8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd00ca240 con 0x7f3cb8004090 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb800bbe0 con 0x7f3cb8004da0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb800bde0 con 0x7f3cd015de00 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb8004090 0x7f3cb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd4f11640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cb8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd5712640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cb8004da0 0x7f3cb8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 -- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 msgr2=0x7f3cb8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cd015de00 0x7f3cb8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 -- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cb8004da0 msgr2=0x7f3cb8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cb8004da0 0x7f3cb8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 -- 172.21.15.82:0/3241539648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8003530 con 0x7f3cb8004090 2024-04-25T10:03:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb8004090 0x7f3cb8004810 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001780 tx=0x7f3cc001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6192edbce421da51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00013a0 con 0x7f3cb8004090 2024-04-25T10:03:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc00052f0 con 0x7f3cb8004090 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cc77fe640 1 -- 172.21.15.82:0/3241539648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0003840 con 0x7f3cb8004090 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 -- 172.21.15.82:0/3241539648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb8004090 msgr2=0x7f3cb8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd7175640 1 --2- 172.21.15.82:0/3241539648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cb8004090 0x7f3cb8004810 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0001780 tx=0x7f3cc001c420 comp rx=0 tx=0).stop 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.903+0000 7f3cd5712640 1 -- 172.21.15.82:0/3241539648 reap_dead start 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/3241539648 shutdown_connections 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/3241539648 >> 172.21.15.82:0/3241539648 conn(0x7f3cd00cc670 msgr2=0x7f3cd00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/3241539648 shutdown_connections 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/3241539648 wait complete. 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 Processor -- start 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- start start 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cb8004090 0x7f3cd00c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 0x7f3cd01584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd015e1e0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd006fc70 con 0x7f3cd01f1f90 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd015e360 con 0x7f3cb8004090 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44744/0 (socket says 172.21.15.82:44744) 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 -- 172.21.15.82:0/2341126590 learned_addr learned my addr 172.21.15.82:0/2341126590 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 -- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cb8004090 msgr2=0x7f3cd00c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd4f11640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cb8004090 0x7f3cd00c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd5712640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 0x7f3cd01584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cb8004090 0x7f3cd00c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 -- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 msgr2=0x7f3cd01584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 0x7f3cd01584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 -- 172.21.15.82:0/2341126590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd01569d0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd5712640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 0x7f3cd01584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc7fff640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005040 tx=0x7f3cb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00ba020 con 0x7f3cd00a03d0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc0000ee0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cd00c6cc0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00069e0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd00c6f60 con 0x7f3cd00a03d0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.907+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd004efa0 con 0x7f3cd00a03d0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.911+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cc00bd020 con 0x7f3cd00a03d0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.911+0000 7f3cc57fa640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 0x7f3cbc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.911+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3cc011f730 con 0x7f3cd00a03d0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.911+0000 7f3cd4f11640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 0x7f3cbc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.911+0000 7f3cd4f11640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 0x7f3cbc069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0156440 tx=0x7f3ccc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:01.915+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cc00c4030 con 0x7f3cd00a03d0 2024-04-25T10:03:02.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cd00a0f10 con 0x7f3cd00a03d0 2024-04-25T10:03:02.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cc57fa640 1 -- 172.21.15.82:0/2341126590 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cc0016130 con 0x7f3cd00a03d0 2024-04-25T10:03:02.071 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:02.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 msgr2=0x7f3cbc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 0x7f3cbc069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0156440 tx=0x7f3ccc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 msgr2=0x7f3cd0156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.063+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0005040 tx=0x7f3cb8005840 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 shutdown_connections 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3cbc066f90 0x7f3cbc069450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3cd01f1f90 0x7f3cd01584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cd00a03d0 0x7f3cd0156490 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 --2- 172.21.15.82:0/2341126590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3cb8004090 0x7f3cd00c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 >> 172.21.15.82:0/2341126590 conn(0x7f3cd00cc670 msgr2=0x7f3cd00caec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 shutdown_connections 2024-04-25T10:03:02.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.067+0000 7f3cd7175640 1 -- 172.21.15.82:0/2341126590 wait complete. 2024-04-25T10:03:02.089 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:02.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 Processor -- start 2024-04-25T10:03:02.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 -- start start 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 0x7f11440cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 0x7f1144099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1144099fb0 0x7f114415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440d0d90 con 0x7f1144099fb0 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11440b0e10 con 0x7f11440b0a30 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1149f3b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11440d0180 con 0x7f114406f4a0 2024-04-25T10:03:02.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1144099fb0 0x7f114415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1144099fb0 0x7f114415d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44746/0 (socket says 172.21.15.82:44746) 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 -- 172.21.15.82:0/925054918 learned_addr learned my addr 172.21.15.82:0/925054918 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f11437fe640 1 --2- 172.21.15.82:0/925054918 >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 0x7f11440cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1142ffd640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 0x7f1144099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 -- 172.21.15.82:0/925054918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 msgr2=0x7f1144099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 0x7f1144099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 -- 172.21.15.82:0/925054918 >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 msgr2=0x7f11440cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 --2- 172.21.15.82:0/925054918 >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 0x7f11440cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1143fff640 1 -- 172.21.15.82:0/925054918 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440d0460 con 0x7f1144099fb0 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f11437fe640 1 --2- 172.21.15.82:0/925054918 >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 0x7f11440cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.191+0000 7f1142ffd640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 0x7f1144099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1143fff640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1144099fb0 0x7f114415d230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1144072ac0 tx=0x7f113c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1ed5eea26329576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f11427fc640 1 -- 172.21.15.82:0/925054918 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0a5760 con 0x7f1144099fb0 2024-04-25T10:03:02.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f11427fc640 1 -- 172.21.15.82:0/925054918 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113c0a5900 con 0x7f1144099fb0 2024-04-25T10:03:02.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f11427fc640 1 -- 172.21.15.82:0/925054918 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113c0a5cd0 con 0x7f1144099fb0 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- 172.21.15.82:0/925054918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1144099fb0 msgr2=0x7f114415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1144099fb0 0x7f114415d230 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1144072ac0 tx=0x7f113c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- 172.21.15.82:0/925054918 shutdown_connections 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1144099fb0 0x7f114415d230 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 --2- 172.21.15.82:0/925054918 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114406f4a0 0x7f1144099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 --2- 172.21.15.82:0/925054918 >> v2:172.21.15.196:3300/0 conn(0x7f11440b0a30 0x7f11440cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- 172.21.15.82:0/925054918 >> 172.21.15.82:0/925054918 conn(0x7f11440b30b0 msgr2=0x7f11440b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- 172.21.15.82:0/925054918 shutdown_connections 2024-04-25T10:03:02.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- 172.21.15.82:0/925054918 wait complete. 2024-04-25T10:03:02.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 Processor -- start 2024-04-25T10:03:02.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.195+0000 7f1149f3b640 1 -- start start 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144099fb0 0x7f11441554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11440b0a30 0x7f1144150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144161790 con 0x7f114406f4a0 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144161610 con 0x7f1144099fb0 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1144161910 con 0x7f11440b0a30 2024-04-25T10:03:02.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44748/0 (socket says 172.21.15.82:44748) 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 -- 172.21.15.82:0/1977397167 learned_addr learned my addr 172.21.15.82:0/1977397167 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1142ffd640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144099fb0 0x7f11441554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 -- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11440b0a30 msgr2=0x7f1144150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1143fff640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11440b0a30 0x7f1144150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11440b0a30 0x7f1144150f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 -- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144099fb0 msgr2=0x7f11441554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144099fb0 0x7f11441554d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 -- 172.21.15.82:0/1977397167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1144155a10 con 0x7f114406f4a0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f11437fe640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f113400c820 tx=0x7f113400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134015070 con 0x7f114406f4a0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134007a30 con 0x7f114406f4a0 2024-04-25T10:03:02.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1144154710 con 0x7f114406f4a0 2024-04-25T10:03:02.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134007e00 con 0x7f114406f4a0 2024-04-25T10:03:02.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1144154cd0 con 0x7f114406f4a0 2024-04-25T10:03:02.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114404efa0 con 0x7f114406f4a0 2024-04-25T10:03:02.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1134018070 con 0x7f114406f4a0 2024-04-25T10:03:02.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 0x7f1130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f11340852c0 con 0x7f114406f4a0 2024-04-25T10:03:02.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1142ffd640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 0x7f1130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.199+0000 7f1142ffd640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 0x7f1130069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.203+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1134055eb0 con 0x7f114406f4a0 2024-04-25T10:03:02.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f114406fa80 con 0x7f114406f4a0 2024-04-25T10:03:02.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f112bfff640 1 -- 172.21.15.82:0/1977397167 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11340557b0 con 0x7f114406f4a0 2024-04-25T10:03:02.358 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 msgr2=0x7f1130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 0x7f1130069420 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1138006fd0 tx=0x7f1138008040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 msgr2=0x7f1144159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f113400c820 tx=0x7f113400cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 shutdown_connections 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1130066f60 0x7f1130069420 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11440b0a30 0x7f1144150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1144099fb0 0x7f11441554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 --2- 172.21.15.82:0/1977397167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f114406f4a0 0x7f1144159810 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 >> 172.21.15.82:0/1977397167 conn(0x7f11440b30b0 msgr2=0x7f11440b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 shutdown_connections 2024-04-25T10:03:02.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.351+0000 7f1149f3b640 1 -- 172.21.15.82:0/1977397167 wait complete. 2024-04-25T10:03:02.379 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:02.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 Processor -- start 2024-04-25T10:03:02.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 -- start start 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a5415ecb0 0x7f2a540c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a540c9ef0 0x7f2a540ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a540ca4c0 con 0x7f2a540cf590 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a540ca780 con 0x7f2a540c9ef0 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a5b08b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5415f0f0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a59628640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a540c9ef0 0x7f2a540ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a59628640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a540c9ef0 0x7f2a540ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44752/0 (socket says 172.21.15.82:44752) 2024-04-25T10:03:02.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a58e27640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2a540cf590 0x7f2a540cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41152/0 (socket says 172.21.15.82:41152) 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.479+0000 7f2a59628640 1 -- 172.21.15.82:0/3091810050 learned_addr learned my addr 172.21.15.82:0/3091810050 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a59628640 1 -- 172.21.15.82:0/3091810050 >> v2:172.21.15.196:3300/0 conn(0x7f2a5415ecb0 msgr2=0x7f2a540c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a53fff640 1 --2- 172.21.15.82:0/3091810050 >> v2:172.21.15.196:3300/0 conn(0x7f2a5415ecb0 0x7f2a540c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a59628640 1 --2- 172.21.15.82:0/3091810050 >> v2:172.21.15.196:3300/0 conn(0x7f2a5415ecb0 0x7f2a540c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a537fe640 1 -- 172.21.15.82:0/3091810050 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523194322 0 0) 0x7f2a540ca4c0 con 0x7f2a540cf590 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a59628640 1 -- 172.21.15.82:0/3091810050 >> v1:172.21.15.82:6789/0 conn(0x7f2a540cf590 legacy=0x7f2a540cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a59628640 1 -- 172.21.15.82:0/3091810050 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540cd690 con 0x7f2a540c9ef0 2024-04-25T10:03:02.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a59628640 1 --2- 172.21.15.82:0/3091810050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a540c9ef0 0x7f2a540ce170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a540d00e0 tx=0x7f2a4c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afebfee09da75009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a537fe640 1 -- 172.21.15.82:0/3091810050 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0a56c0 con 0x7f2a540c9ef0 2024-04-25T10:03:02.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a537fe640 1 -- 172.21.15.82:0/3091810050 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c0a5860 con 0x7f2a540c9ef0 2024-04-25T10:03:02.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a537fe640 1 -- 172.21.15.82:0/3091810050 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0a5c30 con 0x7f2a540c9ef0 2024-04-25T10:03:02.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 -- 172.21.15.82:0/3091810050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a540c9ef0 msgr2=0x7f2a540ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/3091810050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a540c9ef0 0x7f2a540ce170 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a540d00e0 tx=0x7f2a4c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 -- 172.21.15.82:0/3091810050 shutdown_connections 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/3091810050 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a540c9ef0 0x7f2a540ce170 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/3091810050 >> v2:172.21.15.196:3300/0 conn(0x7f2a5415ecb0 0x7f2a540c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 -- 172.21.15.82:0/3091810050 >> 172.21.15.82:0/3091810050 conn(0x7f2a5406ae60 msgr2=0x7f2a5406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 -- 172.21.15.82:0/3091810050 shutdown_connections 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.483+0000 7f2a5b08b640 1 -- 172.21.15.82:0/3091810050 wait complete. 2024-04-25T10:03:02.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 Processor -- start 2024-04-25T10:03:02.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- start start 2024-04-25T10:03:02.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a540c9ef0 0x7f2a540af660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 0x7f2a540c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cc170 con 0x7f2a5415ecb0 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cbff0 con 0x7f2a540cf590 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a540cc2f0 con 0x7f2a540c9ef0 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44768/0 (socket says 172.21.15.82:44768) 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 -- 172.21.15.82:0/4028993532 learned_addr learned my addr 172.21.15.82:0/4028993532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a53fff640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 0x7f2a540c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a58e27640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a540c9ef0 0x7f2a540af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 -- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a540c9ef0 msgr2=0x7f2a540af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a540c9ef0 0x7f2a540af660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 -- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 msgr2=0x7f2a540c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 0x7f2a540c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 -- 172.21.15.82:0/4028993532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a540c4040 con 0x7f2a5415ecb0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a53fff640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 0x7f2a540c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a59628640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0af040 tx=0x7f2a4c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0b8670 con 0x7f2a5415ecb0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c0be070 con 0x7f2a5415ecb0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a540c4320 con 0x7f2a5415ecb0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0b8a50 con 0x7f2a5415ecb0 2024-04-25T10:03:02.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a540c48e0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5415aed0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a4c0b8bf0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 0x7f2a40069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2a4c1200e0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a58e27640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 0x7f2a40069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.487+0000 7f2a58e27640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 0x7f2a40069450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a44000f70 tx=0x7f2a44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.491+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a4c0c7020 con 0x7f2a5415ecb0 2024-04-25T10:03:02.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.639+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2a5404efa0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.639+0000 7f2a517fa640 1 -- 172.21.15.82:0/4028993532 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2a4c0165e0 con 0x7f2a5415ecb0 2024-04-25T10:03:02.649 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:02.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 msgr2=0x7f2a40069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 0x7f2a40069450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2a44000f70 tx=0x7f2a44009040 comp rx=0 tx=0).stop 2024-04-25T10:03:02.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 msgr2=0x7f2a54098860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0af040 tx=0x7f2a4c019790 comp rx=0 tx=0).stop 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 shutdown_connections 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a40066f90 0x7f2a40069450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5415ecb0 0x7f2a54098860 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a540cf590 0x7f2a540c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 --2- 172.21.15.82:0/4028993532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a540c9ef0 0x7f2a540af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 >> 172.21.15.82:0/4028993532 conn(0x7f2a5406ae60 msgr2=0x7f2a540b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 shutdown_connections 2024-04-25T10:03:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.643+0000 7f2a5b08b640 1 -- 172.21.15.82:0/4028993532 wait complete. 2024-04-25T10:03:02.667 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 Processor -- start 2024-04-25T10:03:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 -- start start 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89fc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15ae30 con 0x7f89fc06d090 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc15b110 con 0x7f89fc159ea0 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a03b9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b2990 con 0x7f89fc15a280 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a0213c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f89fc06d090 0x7f89fc06d520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41158/0 (socket says 172.21.15.82:41158) 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.775+0000 7f8a0213c640 1 -- 172.21.15.82:0/2868801322 learned_addr learned my addr 172.21.15.82:0/2868801322 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89fc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149238817 0 0) 0x7f89fc15ae30 con 0x7f89fc06d090 2024-04-25T10:03:02.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e8003530 con 0x7f89fc06d090 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4245810000 0 0) 0x7f89e8003530 con 0x7f89fc06d090 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 msgr2=0x7f89fc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89fc15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 >> v1:172.21.15.196:6789/0 conn(0x7f89fc159ea0 legacy=0x7f89fc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0b2620 con 0x7f89fc06d090 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f89f4001a00 con 0x7f89fc06d090 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 >> v1:172.21.15.82:6789/0 conn(0x7f89fc06d090 legacy=0x7f89fc06d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e8003a60 0x7f89e8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b2990 con 0x7f89e8004810 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800bad0 con 0x7f89e8003a60 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800bcd0 con 0x7f89fc15a280 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0193b640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e8003a60 0x7f89e8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0213c640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 -- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 msgr2=0x7f89e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc15a280 0x7f89e80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 -- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e8003a60 msgr2=0x7f89e8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e8003a60 0x7f89e8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 -- 172.21.15.82:0/2868801322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8003530 con 0x7f89e8004810 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89e8004c90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009ae0 tx=0x7f89ec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ac28c384a02eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0b2070 con 0x7f89e8004810 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec0a5580 con 0x7f89e8004810 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a00939640 1 -- 172.21.15.82:0/2868801322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec0096b0 con 0x7f89e8004810 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2868801322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 msgr2=0x7f89e8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2868801322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89e8004c90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f89ec009ae0 tx=0x7f89ec01ca70 comp rx=0 tx=0).stop 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a0213c640 1 -- 172.21.15.82:0/2868801322 reap_dead start 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2868801322 shutdown_connections 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2868801322 >> 172.21.15.82:0/2868801322 conn(0x7f89fc0b11a0 msgr2=0x7f89fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2868801322 shutdown_connections 2024-04-25T10:03:02.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.779+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2868801322 wait complete. 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 Processor -- start 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- start start 2024-04-25T10:03:02.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc156900 0x7f89fc1f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc155080 0x7f89fc1e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc072040 con 0x7f89e8004810 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0b3530 con 0x7f89fc156900 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc0721c0 con 0x7f89fc155080 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44782/0 (socket says 172.21.15.82:44782) 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 -- 172.21.15.82:0/2005985982 learned_addr learned my addr 172.21.15.82:0/2005985982 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0213c640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc155080 0x7f89fc1e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc156900 0x7f89fc1f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 -- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc155080 msgr2=0x7f89fc1e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc155080 0x7f89fc1e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 -- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc156900 msgr2=0x7f89fc1f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc156900 0x7f89fc1f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 -- 172.21.15.82:0/2005985982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc1f3920 con 0x7f89e8004810 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a0193b640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e550 tx=0x7f89f800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f80052e0 con 0x7f89e8004810 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8018070 con 0x7f89e8004810 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc0a8720 con 0x7f89e8004810 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8003820 con 0x7f89e8004810 2024-04-25T10:03:02.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc0a8ce0 con 0x7f89e8004810 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc04efa0 con 0x7f89e8004810 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.783+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89f80039c0 con 0x7f89e8004810 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.787+0000 7f89f27fc640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 0x7f89e0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.787+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f89f80851d0 con 0x7f89e8004810 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.787+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 0x7f89e0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.787+0000 7f8a0113a640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 0x7f89e0069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0af000 tx=0x7f89ec0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.791+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89f8055dc0 con 0x7f89e8004810 2024-04-25T10:03:02.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.959+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f89fc06e6b0 con 0x7f89e8004810 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.959+0000 7f89f27fc640 1 -- 172.21.15.82:0/2005985982 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f89f8005480 con 0x7f89e8004810 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 msgr2=0x7f89e0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 0x7f89e0069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89ec0af000 tx=0x7f89ec0046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 msgr2=0x7f89fc0aba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f89f800e550 tx=0x7f89f800ea10 comp rx=0 tx=0).stop 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 shutdown_connections 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89e0066f90 0x7f89e0069450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89fc155080 0x7f89fc1e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89fc156900 0x7f89fc1f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 --2- 172.21.15.82:0/2005985982 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e8004810 0x7f89fc0aba10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 >> 172.21.15.82:0/2005985982 conn(0x7f89fc0b11a0 msgr2=0x7f89fc15c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 shutdown_connections 2024-04-25T10:03:02.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:02.963+0000 7f8a03b9f640 1 -- 172.21.15.82:0/2005985982 wait complete. 2024-04-25T10:03:02.989 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:02.989 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:02.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:03.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.139+0000 7fd82e4c1640 1 Processor -- start 2024-04-25T10:03:03.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.139+0000 7fd82e4c1640 1 -- start start 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd82815a700 0x7fd82815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 0x7fd8280cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 0x7fd8280b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd82815f200 con 0x7fd82815a700 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8280b2090 con 0x7fd8280ce430 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82e4c1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280b43e0 con 0x7fd8280b3f50 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd82815a700 0x7fd82815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd82815a700 0x7fd82815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44788/0 (socket says 172.21.15.82:44788) 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 -- 172.21.15.82:0/3337623976 learned_addr learned my addr 172.21.15.82:0/3337623976 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:03.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82ca5e640 1 --2- 172.21.15.82:0/3337623976 >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 0x7fd8280b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd8277fe640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 0x7fd8280cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 -- 172.21.15.82:0/3337623976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 msgr2=0x7fd8280cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 0x7fd8280cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 -- 172.21.15.82:0/3337623976 >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 msgr2=0x7fd8280b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 --2- 172.21.15.82:0/3337623976 >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 0x7fd8280b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 -- 172.21.15.82:0/3337623976 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280b46c0 con 0x7fd82815a700 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd8277fe640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 0x7fd8280cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd82ca5e640 1 --2- 172.21.15.82:0/3337623976 >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 0x7fd8280b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd827fff640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd82815ecc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd82806ca20 tx=0x7fd81001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1d8364d8c674d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd826ffd640 1 -- 172.21.15.82:0/3337623976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100b0070 con 0x7fd82815a700 2024-04-25T10:03:03.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd826ffd640 1 -- 172.21.15.82:0/3337623976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8100ab600 con 0x7fd82815a700 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.143+0000 7fd826ffd640 1 -- 172.21.15.82:0/3337623976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8100ab9d0 con 0x7fd82815a700 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/3337623976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 msgr2=0x7fd82815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd82815ecc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd82806ca20 tx=0x7fd81001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/3337623976 shutdown_connections 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/3337623976 >> v2:172.21.15.196:3300/0 conn(0x7fd8280ce430 0x7fd8280b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8280b3f50 0x7fd8280cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/3337623976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd82815ecc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/3337623976 >> 172.21.15.82:0/3337623976 conn(0x7fd8280b11a0 msgr2=0x7fd8280b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/3337623976 shutdown_connections 2024-04-25T10:03:03.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/3337623976 wait complete. 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 Processor -- start 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- start start 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 0x7fd8281fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8281549c0 0x7fd8280b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280cd490 con 0x7fd82815a700 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280cd310 con 0x7fd8280ce430 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd828071030 con 0x7fd8281549c0 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44792/0 (socket says 172.21.15.82:44792) 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 -- 172.21.15.82:0/896076242 learned_addr learned my addr 172.21.15.82:0/896076242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd827fff640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 0x7fd8281fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82ca5e640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8281549c0 0x7fd8280b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 -- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8281549c0 msgr2=0x7fd8280b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8281549c0 0x7fd8280b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 -- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 msgr2=0x7fd8281fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 0x7fd8281fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 -- 172.21.15.82:0/896076242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8281553a0 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd827fff640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 0x7fd8281fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd8277fe640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd81800c7f0 tx=0x7fd81800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818015070 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818007820 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd828155680 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818007c10 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281fe060 con 0x7fd82815a700 2024-04-25T10:03:03.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.147+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82804efa0 con 0x7fd82815a700 2024-04-25T10:03:03.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.151+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd818018070 con 0x7fd82815a700 2024-04-25T10:03:03.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.151+0000 7fd824ff9640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 0x7fd808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.151+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd818085590 con 0x7fd82815a700 2024-04-25T10:03:03.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.151+0000 7fd827fff640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 0x7fd808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.151+0000 7fd827fff640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 0x7fd808069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ad000 tx=0x7fd810008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.155+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd818056120 con 0x7fd82815a700 2024-04-25T10:03:03.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.315+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd82806f260 con 0x7fd82815a700 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.315+0000 7fd824ff9640 1 -- 172.21.15.82:0/896076242 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fd818055a20 con 0x7fd82815a700 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 msgr2=0x7fd808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 0x7fd808069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8100ad000 tx=0x7fd810008040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 msgr2=0x7fd8281fdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd81800c7f0 tx=0x7fd81800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 shutdown_connections 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd808066f60 0x7fd808069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8281549c0 0x7fd8280b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd82815a700 0x7fd8281fdb20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 --2- 172.21.15.82:0/896076242 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8280ce430 0x7fd8281fae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 >> 172.21.15.82:0/896076242 conn(0x7fd8280b11a0 msgr2=0x7fd8280b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 shutdown_connections 2024-04-25T10:03:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.319+0000 7fd82e4c1640 1 -- 172.21.15.82:0/896076242 wait complete. 2024-04-25T10:03:03.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:03.344 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 Processor -- start 2024-04-25T10:03:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 -- start start 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcf0015ecb0 0x7fcf000c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000ca4c0 con 0x7fcf000cf590 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf000cd690 con 0x7fcf000c9ef0 2024-04-25T10:03:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf07f59640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf000ca780 con 0x7fcf0015ecb0 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcf000cf590 0x7fcf000cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41166/0 (socket says 172.21.15.82:41166) 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 -- 172.21.15.82:0/4156970561 learned_addr learned my addr 172.21.15.82:0/4156970561 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf054f4640 1 --2- 172.21.15.82:0/4156970561 >> v2:172.21.15.196:3300/0 conn(0x7fcf0015ecb0 0x7fcf000c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062137924 0 0) 0x7fcf000ca4c0 con 0x7fcf000cf590 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceec005740 con 0x7fcf000cf590 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3372868307 0 0) 0x7fceec005740 con 0x7fcf000cf590 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 >> v2:172.21.15.196:3300/0 conn(0x7fcf0015ecb0 msgr2=0x7fcf000c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 --2- 172.21.15.82:0/4156970561 >> v2:172.21.15.196:3300/0 conn(0x7fcf0015ecb0 0x7fcf000c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 >> v1:172.21.15.196:6789/0 conn(0x7fcf000c9ef0 legacy=0x7fcf000ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000cd970 con 0x7fcf000cf590 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcef0001680 con 0x7fcf000cf590 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 >> v1:172.21.15.82:6789/0 conn(0x7fcf000cf590 legacy=0x7fcf000cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fceec006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fceec006a20 0x7fceec006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceec0073e0 0x7fceec00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00e2e0 con 0x7fceec005c40 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00e490 con 0x7fceec0073e0 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fceec00e690 con 0x7fceec006a20 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fceec006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf064f6640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceec0073e0 0x7fceec00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 -- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fceec006a20 msgr2=0x7fceec006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fceec006a20 0x7fceec006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 -- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceec0073e0 msgr2=0x7fceec00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fceec0073e0 0x7fceec00dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf064f6640 1 -- 172.21.15.82:0/4156970561 reap_dead start 2024-04-25T10:03:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.463+0000 7fcf05cf5640 1 -- 172.21.15.82:0/4156970561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec005740 con 0x7fceec005c40 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fceec006060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c240 tx=0x7fcef001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16127819d5006c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0007d20 con 0x7fceec005c40 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef00a9070 con 0x7fceec005c40 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf04cf3640 1 -- 172.21.15.82:0/4156970561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0005da0 con 0x7fceec005c40 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 msgr2=0x7fceec006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fceec006060 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcef000c240 tx=0x7fcef001cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- 172.21.15.82:0/4156970561 shutdown_connections 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 --2- 172.21.15.82:0/4156970561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fceec006060 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- 172.21.15.82:0/4156970561 >> 172.21.15.82:0/4156970561 conn(0x7fcf0006ae60 msgr2=0x7fcf0006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- 172.21.15.82:0/4156970561 shutdown_connections 2024-04-25T10:03:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- 172.21.15.82:0/4156970561 wait complete. 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 Processor -- start 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- start start 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf000c2e90 0x7fcf0015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0015d350 0x7fcf000c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b2d50 con 0x7fceec005c40 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b2bd0 con 0x7fcf000c2e90 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.467+0000 7fcf07f59640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000b2ed0 con 0x7fcf0015d350 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44798/0 (socket says 172.21.15.82:44798) 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 -- 172.21.15.82:0/3936030573 learned_addr learned my addr 172.21.15.82:0/3936030573 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:03.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf054f4640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf000c2e90 0x7fcf0015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf064f6640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0015d350 0x7fcf000c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 -- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0015d350 msgr2=0x7fcf000c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0015d350 0x7fcf000c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 -- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf000c2e90 msgr2=0x7fcf0015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf000c2e90 0x7fcf0015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 -- 172.21.15.82:0/3936030573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000b1990 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf05cf5640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcef001cab0 tx=0x7fceec007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0007390 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef0007530 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000c5cb0 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef00047f0 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf000c59d0 con 0x7fceec005c40 2024-04-25T10:03:03.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf00157840 con 0x7fceec005c40 2024-04-25T10:03:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcef0004990 con 0x7fceec005c40 2024-04-25T10:03:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 0x7fcee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fcef011ed10 con 0x7fceec005c40 2024-04-25T10:03:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf054f4640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 0x7fcee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.471+0000 7fcf054f4640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 0x7fcee4069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf000c9400 tx=0x7fcef800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.475+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcef0016990 con 0x7fceec005c40 2024-04-25T10:03:03.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.631+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf000c9690 con 0x7fceec005c40 2024-04-25T10:03:03.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcef67fc640 1 -- 172.21.15.82:0/3936030573 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcef00ef990 con 0x7fceec005c40 2024-04-25T10:03:03.642 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 msgr2=0x7fcee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 0x7fcee4069420 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcf000c9400 tx=0x7fcef800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 msgr2=0x7fcf001f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcef001cab0 tx=0x7fceec007a50 comp rx=0 tx=0).stop 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 shutdown_connections 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcee4066f60 0x7fcee4069420 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf0015d350 0x7fcf000c45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf000c2e90 0x7fcf0015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 --2- 172.21.15.82:0/3936030573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fceec005c40 0x7fcf001f6830 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 >> 172.21.15.82:0/3936030573 conn(0x7fcf0006ae60 msgr2=0x7fcf00158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 shutdown_connections 2024-04-25T10:03:03.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.635+0000 7fcf07f59640 1 -- 172.21.15.82:0/3936030573 wait complete. 2024-04-25T10:03:03.660 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.775+0000 7f73ae2b7640 1 Processor -- start 2024-04-25T10:03:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.775+0000 7f73ae2b7640 1 -- start start 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73ae2b7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f73a815b6b0 0x7f73a80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73ae2b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a80cf590 0x7f73a80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73ae2b7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73a80cbd20 con 0x7f73a815f130 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73ae2b7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73a815ba90 con 0x7f73a815b6b0 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73ae2b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80cff30 con 0x7f73a80cf590 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f73a815b6b0 0x7f73a80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f73a815b6b0 0x7f73a80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44802/0 (socket says 172.21.15.82:44802) 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 -- 172.21.15.82:0/1415223069 learned_addr learned my addr 172.21.15.82:0/1415223069 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a77fe640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a80cf590 0x7f73a80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 -- 172.21.15.82:0/1415223069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a80cf590 msgr2=0x7f73a80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a80cf590 0x7f73a80cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 -- 172.21.15.82:0/1415223069 >> v1:172.21.15.196:6789/0 conn(0x7f73a815f130 legacy=0x7f73a815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 -- 172.21.15.82:0/1415223069 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a80cc000 con 0x7f73a815b6b0 2024-04-25T10:03:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a7fff640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a815b6b0 0x7f73a80ca3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009870 tx=0x7f73a001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fcee49350ee117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a6ffd640 1 -- 172.21.15.82:0/1415223069 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00aa610 con 0x7f73a815b6b0 2024-04-25T10:03:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a6ffd640 1 -- 172.21.15.82:0/1415223069 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a00af070 con 0x7f73a815b6b0 2024-04-25T10:03:03.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.779+0000 7f73a6ffd640 1 -- 172.21.15.82:0/1415223069 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00aa9d0 con 0x7f73a815b6b0 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- 172.21.15.82:0/1415223069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a815b6b0 msgr2=0x7f73a80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a815b6b0 0x7f73a80ca3c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f73a0009870 tx=0x7f73a001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- 172.21.15.82:0/1415223069 shutdown_connections 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a80cf590 0x7f73a80cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/1415223069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a815b6b0 0x7f73a80ca3c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- 172.21.15.82:0/1415223069 >> 172.21.15.82:0/1415223069 conn(0x7f73a806ae60 msgr2=0x7f73a806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- 172.21.15.82:0/1415223069 shutdown_connections 2024-04-25T10:03:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- 172.21.15.82:0/1415223069 wait complete. 2024-04-25T10:03:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 Processor -- start 2024-04-25T10:03:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- start start 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 0x7f73a8152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 0x7f73a8153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80ca900 con 0x7f73a80cf590 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80cba30 con 0x7f73a815b6b0 2024-04-25T10:03:03.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ae2b7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80cadb0 con 0x7f73a815f130 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44812/0 (socket says 172.21.15.82:44812) 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 -- 172.21.15.82:0/3583175384 learned_addr learned my addr 172.21.15.82:0/3583175384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ac854640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 0x7f73a8153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a77fe640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 0x7f73a8152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 -- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 msgr2=0x7f73a8153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 0x7f73a8153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 -- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 msgr2=0x7f73a8152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 0x7f73a8152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73a7fff640 1 -- 172.21.15.82:0/3583175384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a815c8d0 con 0x7f73a80cf590 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.783+0000 7f73ac854640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 0x7f73a8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a77fe640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 0x7f73a8152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a7fff640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73a00ac040 tx=0x7f73a001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00b3800 con 0x7f73a80cf590 2024-04-25T10:03:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a00b39c0 con 0x7f73a80cf590 2024-04-25T10:03:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a815cbb0 con 0x7f73a80cf590 2024-04-25T10:03:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00b3db0 con 0x7f73a80cf590 2024-04-25T10:03:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a815d170 con 0x7f73a80cf590 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a804efa0 con 0x7f73a80cf590 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73a00c6390 con 0x7f73a80cf590 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 0x7f7394069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a77fe640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 0x7f7394069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f73a011fbc0 con 0x7f73a80cf590 2024-04-25T10:03:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.787+0000 7f73a77fe640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 0x7f7394069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73a81540d0 tx=0x7f739c0074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.791+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73a00c3060 con 0x7f73a80cf590 2024-04-25T10:03:03.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.947+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73a8098e40 con 0x7f73a80cf590 2024-04-25T10:03:03.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73a4ff9640 1 -- 172.21.15.82:0/3583175384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73a00f6020 con 0x7f73a80cf590 2024-04-25T10:03:03.958 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 msgr2=0x7f7394069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 0x7f7394069450 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73a81540d0 tx=0x7f739c0074a0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 msgr2=0x7f73a8098900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f73a00ac040 tx=0x7f73a001ad30 comp rx=0 tx=0).stop 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 shutdown_connections 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7394066f90 0x7f7394069450 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f73a815f130 0x7f73a8153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73a815b6b0 0x7f73a8152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 --2- 172.21.15.82:0/3583175384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73a80cf590 0x7f73a8098900 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 >> 172.21.15.82:0/3583175384 conn(0x7f73a806ae60 msgr2=0x7f73a8154830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 shutdown_connections 2024-04-25T10:03:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:03.951+0000 7f73ae2b7640 1 -- 172.21.15.82:0/3583175384 wait complete. 2024-04-25T10:03:03.977 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 Processor -- start 2024-04-25T10:03:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 -- start start 2024-04-25T10:03:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad7c06ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c0d0ab0 con 0x7fad7c06c600 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad7c099e50 con 0x7fad7c0b0730 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad80e9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c09aa60 con 0x7fad7c0ccbd0 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad79d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fad7c06c600 0x7fad7c15d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41172/0 (socket says 172.21.15.82:41172) 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad79d9b640 1 -- 172.21.15.82:0/2277447627 learned_addr learned my addr 172.21.15.82:0/2277447627 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad7c06ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239297453 0 0) 0x7fad7c0d0ab0 con 0x7fad7c06c600 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad64003530 con 0x7fad7c06c600 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3375727209 0 0) 0x7fad64003530 con 0x7fad7c06c600 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 msgr2=0x7fad7c06ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad7c06ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 >> v1:172.21.15.196:6789/0 conn(0x7fad7c0b0730 legacy=0x7fad7c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c09a130 con 0x7fad7c06c600 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fad70001680 con 0x7fad7c06c600 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 >> v1:172.21.15.82:6789/0 conn(0x7fad7c06c600 legacy=0x7fad7c15d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad64003a60 0x7fad64003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad64004d50 0x7fad640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c09aa60 con 0x7fad64004d50 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400bb10 con 0x7fad64003a60 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6400bd10 con 0x7fad7c0ccbd0 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad64004d50 0x7fad640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad79d9b640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7a59c640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad64003a60 0x7fad64003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 -- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 msgr2=0x7fad64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad7c0ccbd0 0x7fad64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 -- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad64003a60 msgr2=0x7fad64003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad64003a60 0x7fad64003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.091+0000 7fad7ad9d640 1 -- 172.21.15.82:0/2277447627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad64003530 con 0x7fad64004d50 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad64004d50 0x7fad640051d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad7c072ea0 tx=0x7fad7401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15dad5ddccb2c43e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740b2070 con 0x7fad64004d50 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740a5bf0 con 0x7fad64004d50 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7959a640 1 -- 172.21.15.82:0/2277447627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad74009c10 con 0x7fad64004d50 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- 172.21.15.82:0/2277447627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad64004d50 msgr2=0x7fad640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 --2- 172.21.15.82:0/2277447627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad64004d50 0x7fad640051d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fad7c072ea0 tx=0x7fad7401cbe0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7ad9d640 1 -- 172.21.15.82:0/2277447627 reap_dead start 2024-04-25T10:03:04.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- 172.21.15.82:0/2277447627 shutdown_connections 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- 172.21.15.82:0/2277447627 >> 172.21.15.82:0/2277447627 conn(0x7fad7c0b3060 msgr2=0x7fad7c0b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- 172.21.15.82:0/2277447627 shutdown_connections 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- 172.21.15.82:0/2277447627 wait complete. 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 Processor -- start 2024-04-25T10:03:04.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- start start 2024-04-25T10:03:04.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad64004d50 0x7fad7c158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 0x7fad7c1e9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c099a70 con 0x7fad7c15e820 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c09a6f0 con 0x7fad7c15e3a0 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad80e9f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7c099bf0 con 0x7fad64004d50 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44822/0 (socket says 172.21.15.82:44822) 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.095+0000 7fad7ad9d640 1 -- 172.21.15.82:0/1438793241 learned_addr learned my addr 172.21.15.82:0/1438793241 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7a59c640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad64004d50 0x7fad7c158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad79d9b640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 0x7fad7c1e9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 -- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad64004d50 msgr2=0x7fad7c158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad64004d50 0x7fad7c158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 -- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 msgr2=0x7fad7c1e9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 0x7fad7c1e9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 -- 172.21.15.82:0/1438793241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7c1e9cf0 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad79d9b640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 0x7fad7c1e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7ad9d640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad74002240 tx=0x7fad74009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740bd070 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad740a59f0 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad7c158560 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740b2040 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7c1e84b0 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad7c04efa0 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad740c0070 con 0x7fad7c15e820 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 0x7fad5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7a59c640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 0x7fad5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fad7411f090 con 0x7fad7c15e820 2024-04-25T10:03:04.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.099+0000 7fad7a59c640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 0x7fad5c069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007fd0 tx=0x7fad6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.103+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad740168f0 con 0x7fad7c15e820 2024-04-25T10:03:04.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fad7c06e1b0 con 0x7fad7c15e820 2024-04-25T10:03:04.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad6affd640 1 -- 172.21.15.82:0/1438793241 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fad740ef6c0 con 0x7fad7c15e820 2024-04-25T10:03:04.259 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:04.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 msgr2=0x7fad5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 0x7fad5c069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007fd0 tx=0x7fad6c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 msgr2=0x7fad7c1e97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fad74002240 tx=0x7fad74009750 comp rx=0 tx=0).stop 2024-04-25T10:03:04.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 shutdown_connections 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad5c066f60 0x7fad5c069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad7c15e820 0x7fad7c1e97b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad7c15e3a0 0x7fad7c1e9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 --2- 172.21.15.82:0/1438793241 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad64004d50 0x7fad7c158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 >> 172.21.15.82:0/1438793241 conn(0x7fad7c0b3060 msgr2=0x7fad7c06f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 shutdown_connections 2024-04-25T10:03:04.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.251+0000 7fad80e9f640 1 -- 172.21.15.82:0/1438793241 wait complete. 2024-04-25T10:03:04.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:04.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 Processor -- start 2024-04-25T10:03:04.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 -- start start 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ee00cfa20 0x7f7ee0072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00cbd20 con 0x7f7ee00cf590 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee00cc000 con 0x7f7ee015b6b0 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee7931640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee015ba90 con 0x7f7ee00cfa20 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee5ece640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ee00cfa20 0x7f7ee0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee4ecc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7ee00cf590 0x7f7ee0072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41174/0 (socket says 172.21.15.82:41174) 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee4ecc640 1 -- 172.21.15.82:0/1369878662 learned_addr learned my addr 172.21.15.82:0/1369878662 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:04.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee5ece640 1 -- 172.21.15.82:0/1369878662 >> v1:172.21.15.196:6789/0 conn(0x7f7ee015b6b0 legacy=0x7f7ee00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee5ece640 1 -- 172.21.15.82:0/1369878662 >> v1:172.21.15.82:6789/0 conn(0x7f7ee00cf590 legacy=0x7f7ee0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee5ece640 1 -- 172.21.15.82:0/1369878662 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee00cc2e0 con 0x7f7ee00cfa20 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.379+0000 7f7ee5ece640 1 --2- 172.21.15.82:0/1369878662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cfa20 0x7f7ee0072710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009d90 tx=0x7f7ed801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da2d02e01afbf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ed7fff640 1 -- 172.21.15.82:0/1369878662 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80b2070 con 0x7f7ee00cfa20 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ed7fff640 1 -- 172.21.15.82:0/1369878662 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed80a59c0 con 0x7f7ee00cfa20 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ed7fff640 1 -- 172.21.15.82:0/1369878662 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80a5d90 con 0x7f7ee00cfa20 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- 172.21.15.82:0/1369878662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cfa20 msgr2=0x7f7ee0072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 --2- 172.21.15.82:0/1369878662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cfa20 0x7f7ee0072710 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009d90 tx=0x7f7ed801c900 comp rx=0 tx=0).stop 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- 172.21.15.82:0/1369878662 shutdown_connections 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 --2- 172.21.15.82:0/1369878662 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cfa20 0x7f7ee0072710 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- 172.21.15.82:0/1369878662 >> 172.21.15.82:0/1369878662 conn(0x7f7ee006ae60 msgr2=0x7f7ee006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- 172.21.15.82:0/1369878662 shutdown_connections 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- 172.21.15.82:0/1369878662 wait complete. 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 Processor -- start 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- start start 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00cfa20 0x7f7ee0158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee015b6b0 0x7f7ee01591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00cadb0 con 0x7f7ee00cf590 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00cfe50 con 0x7f7ee015b6b0 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee7931640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00caf30 con 0x7f7ee00cfa20 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44844/0 (socket says 172.21.15.82:44844) 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 -- 172.21.15.82:0/3080632274 learned_addr learned my addr 172.21.15.82:0/3080632274 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:04.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 -- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00cfa20 msgr2=0x7f7ee0158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee4ecc640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00cfa20 0x7f7ee0158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee5ece640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee015b6b0 0x7f7ee01591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00cfa20 0x7f7ee0158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 -- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee015b6b0 msgr2=0x7f7ee01591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee015b6b0 0x7f7ee01591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 -- 172.21.15.82:0/3080632274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee0159df0 con 0x7f7ee00cf590 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.383+0000 7f7ee56cd640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b410 tx=0x7f7ed000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0015030 con 0x7f7ee00cf590 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0004020 con 0x7f7ee00cf590 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0004450 con 0x7f7ee00cf590 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee015a0d0 con 0x7f7ee00cf590 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee015a690 con 0x7f7ee00cf590 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee004efa0 con 0x7f7ee00cf590 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ed0018070 con 0x7f7ee00cf590 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 0x7f7ec4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f7ed0084f90 con 0x7f7ee00cf590 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ee4ecc640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 0x7f7ec4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.387+0000 7f7ee4ecc640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 0x7f7ec4069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0150b30 tx=0x7f7edc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.391+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ed0055b80 con 0x7f7ee00cf590 2024-04-25T10:03:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.535+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7ee00b31e0 con 0x7f7ee00cf590 2024-04-25T10:03:04.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.535+0000 7f7ed5ffb640 1 -- 172.21.15.82:0/3080632274 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7ed0055480 con 0x7f7ee00cf590 2024-04-25T10:03:04.545 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 msgr2=0x7f7ec4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 0x7f7ec4069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0150b30 tx=0x7f7edc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 msgr2=0x7f7ee00980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000b410 tx=0x7f7ed000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 shutdown_connections 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ec4066f90 0x7f7ec4069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ee00cfa20 0x7f7ee0158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ee015b6b0 0x7f7ee01591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 --2- 172.21.15.82:0/3080632274 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ee00cf590 0x7f7ee00980d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 >> 172.21.15.82:0/3080632274 conn(0x7f7ee006ae60 msgr2=0x7f7ee0154710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 shutdown_connections 2024-04-25T10:03:04.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.539+0000 7f7ee7931640 1 -- 172.21.15.82:0/3080632274 wait complete. 2024-04-25T10:03:04.562 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:04.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.663+0000 7f7c5fc2b640 1 Processor -- start 2024-04-25T10:03:04.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.663+0000 7f7c5fc2b640 1 -- start start 2024-04-25T10:03:04.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5fc2b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c580b0a30 0x7f7c580cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5fc2b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 0x7f7c5815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5fc2b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c580b0e10 con 0x7f7c5806f4a0 2024-04-25T10:03:04.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5fc2b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580d0d90 con 0x7f7c580b0a30 2024-04-25T10:03:04.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5fc2b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580d0180 con 0x7f7c5809a180 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c580b0a30 0x7f7c580cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c580b0a30 0x7f7c580cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52442/0 (socket says 172.21.15.82:52442) 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/270932495 learned_addr learned my addr 172.21.15.82:0/270932495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5e1c8640 1 --2- 172.21.15.82:0/270932495 >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 0x7f7c5815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/270932495 >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 msgr2=0x7f7c5815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 --2- 172.21.15.82:0/270932495 >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 0x7f7c5815d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/270932495 >> v1:172.21.15.196:6789/0 conn(0x7f7c5806f4a0 legacy=0x7f7c58099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/270932495 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580d0460 con 0x7f7c580b0a30 2024-04-25T10:03:04.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5e1c8640 1 --2- 172.21.15.82:0/270932495 >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 0x7f7c5815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5d9c7640 1 --2- 172.21.15.82:0/270932495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c580b0a30 0x7f7c580cf890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7c580d1560 tx=0x7f7c5401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd1e64f7d8909c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5c9c5640 1 -- 172.21.15.82:0/270932495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540aa610 con 0x7f7c580b0a30 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5c9c5640 1 -- 172.21.15.82:0/270932495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c540af070 con 0x7f7c580b0a30 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.667+0000 7f7c5c9c5640 1 -- 172.21.15.82:0/270932495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540aa9d0 con 0x7f7c580b0a30 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/270932495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c580b0a30 msgr2=0x7f7c580cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/270932495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c580b0a30 0x7f7c580cf890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f7c580d1560 tx=0x7f7c5401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/270932495 shutdown_connections 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/270932495 >> v2:172.21.15.196:3300/0 conn(0x7f7c5809a180 0x7f7c5815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/270932495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c580b0a30 0x7f7c580cf890 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/270932495 >> 172.21.15.82:0/270932495 conn(0x7f7c580b30b0 msgr2=0x7f7c580b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/270932495 shutdown_connections 2024-04-25T10:03:04.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/270932495 wait complete. 2024-04-25T10:03:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 Processor -- start 2024-04-25T10:03:04.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- start start 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 0x7f7c580b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c580b0a30 0x7f7c580cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cbd90 con 0x7f7c5806f4a0 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cbc10 con 0x7f7c580b0a30 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cbf10 con 0x7f7c5809a180 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52452/0 (socket says 172.21.15.82:52452) 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/3053665725 learned_addr learned my addr 172.21.15.82:0/3053665725 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:04.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5e1c8640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c580b0a30 0x7f7c580cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 msgr2=0x7f7c580b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d1c6640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 0x7f7c580b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 0x7f7c580b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c580b0a30 msgr2=0x7f7c580cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c580b0a30 0x7f7c580cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 -- 172.21.15.82:0/3053665725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c580b2e30 con 0x7f7c5806f4a0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d1c6640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 0x7f7c580b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5d9c7640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7c5401a0e0 tx=0x7f7c5401acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540bd5f0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c540bd7b0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c580b6e20 con 0x7f7c5806f4a0 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540bdba0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.671+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c581567f0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5804efa0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c540b33f0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c4e7fc640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 0x7f7c3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f7c5411fb10 con 0x7f7c5806f4a0 2024-04-25T10:03:04.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c5d1c6640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 0x7f7c3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.675+0000 7f7c5d1c6640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 0x7f7c3c069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c581594b0 tx=0x7f7c48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.679+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c540c3060 con 0x7f7c5806f4a0 2024-04-25T10:03:04.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7c580b3ee0 con 0x7f7c5806f4a0 2024-04-25T10:03:04.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c4e7fc640 1 -- 172.21.15.82:0/3053665725 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7c540a7070 con 0x7f7c5806f4a0 2024-04-25T10:03:04.850 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 msgr2=0x7f7c3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 0x7f7c3c069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c581594b0 tx=0x7f7c48009040 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 msgr2=0x7f7c581549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7c5401a0e0 tx=0x7f7c5401acf0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 shutdown_connections 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c3c066f60 0x7f7c3c069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c5809a180 0x7f7c580b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c580b0a30 0x7f7c580cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 --2- 172.21.15.82:0/3053665725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5806f4a0 0x7f7c581549f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 >> 172.21.15.82:0/3053665725 conn(0x7f7c580b30b0 msgr2=0x7f7c580b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 shutdown_connections 2024-04-25T10:03:04.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:04.843+0000 7f7c5fc2b640 1 -- 172.21.15.82:0/3053665725 wait complete. 2024-04-25T10:03:04.867 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:03:04.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:04.868 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:04.880 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:04.880 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:03:04.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:04.880 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:04.926 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:04.926 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:04.932 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:04.932 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:04.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:05.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.091+0000 7feb033f4640 1 Processor -- start 2024-04-25T10:03:05.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.091+0000 7feb033f4640 1 -- start start 2024-04-25T10:03:05.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feafc0cd350 0x7feafc15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feafc0ce320 con 0x7feafc15aac0 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc0ce0d0 con 0x7feafc0cd350 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0ce600 con 0x7feafc0cf0e0 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7feafc0cd350 0x7feafc15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb01190640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feafc15aac0 0x7feafc0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44732/0 (socket says 172.21.15.82:44732) 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb01190640 1 -- 172.21.15.82:0/1622455169 learned_addr learned my addr 172.21.15.82:0/1622455169 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb01991640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 -- 172.21.15.82:0/1622455169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 msgr2=0x7feafc15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc15b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 -- 172.21.15.82:0/1622455169 >> v1:172.21.15.82:6789/0 conn(0x7feafc15aac0 legacy=0x7feafc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 -- 172.21.15.82:0/1622455169 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0ce8e0 con 0x7feafc0cd350 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb0098f640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc15aea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feaec0097c0 tx=0x7feaec01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69da48d6cc608653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feaf3fff640 1 -- 172.21.15.82:0/1622455169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0b1070 con 0x7feafc0cd350 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feaf3fff640 1 -- 172.21.15.82:0/1622455169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec0ac410 con 0x7feafc0cd350 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feaf3fff640 1 -- 172.21.15.82:0/1622455169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0ac7e0 con 0x7feafc0cd350 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- 172.21.15.82:0/1622455169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 msgr2=0x7feafc15aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc15aea0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7feaec0097c0 tx=0x7feaec01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- 172.21.15.82:0/1622455169 shutdown_connections 2024-04-25T10:03:05.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 --2- 172.21.15.82:0/1622455169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc15aea0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- 172.21.15.82:0/1622455169 >> 172.21.15.82:0/1622455169 conn(0x7feafc099a70 msgr2=0x7feafc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- 172.21.15.82:0/1622455169 shutdown_connections 2024-04-25T10:03:05.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.095+0000 7feb033f4640 1 -- 172.21.15.82:0/1622455169 wait complete. 2024-04-25T10:03:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 Processor -- start 2024-04-25T10:03:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- start start 2024-04-25T10:03:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc0c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feafc15aac0 0x7feafc0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0cbbb0 con 0x7feafc0cd350 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0cba30 con 0x7feafc15aac0 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0cbd30 con 0x7feafc0cf0e0 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52470/0 (socket says 172.21.15.82:52470) 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 -- 172.21.15.82:0/3557789851 learned_addr learned my addr 172.21.15.82:0/3557789851 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb0098f640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc0c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01991640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feafc15aac0 0x7feafc0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 -- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 msgr2=0x7feafc0c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc0c4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 -- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feafc15aac0 msgr2=0x7feafc0c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feafc15aac0 0x7feafc0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 -- 172.21.15.82:0/3557789851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0c8d60 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb0098f640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc0c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb01190640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c3a0 tx=0x7feaf800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf8017070 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf8005cf0 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc0c9040 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf80072f0 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc0c9600 con 0x7feafc0cd350 2024-04-25T10:03:05.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc04efa0 con 0x7feafc0cd350 2024-04-25T10:03:05.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.099+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaf801a070 con 0x7feafc0cd350 2024-04-25T10:03:05.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.103+0000 7feaf1ffb640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.103+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7feaf8084bc0 con 0x7feafc0cd350 2024-04-25T10:03:05.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.103+0000 7feb0098f640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.103+0000 7feb0098f640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 0x7feae0069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feaec0ae000 tx=0x7feaec0a6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.103+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf8021020 con 0x7feafc0cd350 2024-04-25T10:03:05.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.251+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feafc0cdc90 con 0x7feafc0cd350 2024-04-25T10:03:05.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feaf1ffb640 1 -- 172.21.15.82:0/3557789851 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7feaf8055890 con 0x7feafc0cd350 2024-04-25T10:03:05.262 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:05.262 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:05.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 msgr2=0x7feae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 0x7feae0069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7feaec0ae000 tx=0x7feaec0a6040 comp rx=0 tx=0).stop 2024-04-25T10:03:05.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 msgr2=0x7feafc150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7feaf800c3a0 tx=0x7feaf800c860 comp rx=0 tx=0).stop 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 shutdown_connections 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feae0066f60 0x7feae0069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feafc0cf0e0 0x7feafc0c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feafc15aac0 0x7feafc0c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 --2- 172.21.15.82:0/3557789851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feafc0cd350 0x7feafc150540 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 >> 172.21.15.82:0/3557789851 conn(0x7feafc099a70 msgr2=0x7feafc15bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 shutdown_connections 2024-04-25T10:03:05.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.255+0000 7feb033f4640 1 -- 172.21.15.82:0/3557789851 wait complete. 2024-04-25T10:03:05.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:05.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:05.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.395+0000 7f9b17988640 1 Processor -- start 2024-04-25T10:03:05.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.395+0000 7f9b17988640 1 -- start start 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b17988640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9b100cfca0 0x7f9b1015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b17988640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100cbf10 con 0x7f9b1015e4f0 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b17988640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b100cc1f0 con 0x7f9b100b3cf0 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b17988640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b100cc4d0 con 0x7f9b100cfca0 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b15f25640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9b1015e4f0 0x7f9b100caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44736/0 (socket says 172.21.15.82:44736) 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b15f25640 1 -- 172.21.15.82:0/3865180743 learned_addr learned my addr 172.21.15.82:0/3865180743 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b14f23640 1 --2- 172.21.15.82:0/3865180743 >> v2:172.21.15.82:3300/0 conn(0x7f9b100cfca0 0x7f9b1015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b07fff640 1 -- 172.21.15.82:0/3865180743 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2438094360 0 0) 0x7f9b100cbf10 con 0x7f9b1015e4f0 2024-04-25T10:03:05.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b07fff640 1 -- 172.21.15.82:0/3865180743 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9afc003530 con 0x7f9b1015e4f0 2024-04-25T10:03:05.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b14f23640 1 -- 172.21.15.82:0/3865180743 >> v1:172.21.15.196:6789/0 conn(0x7f9b100b3cf0 legacy=0x7f9b100cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b14f23640 1 -- 172.21.15.82:0/3865180743 >> v1:172.21.15.82:6789/0 conn(0x7f9b1015e4f0 legacy=0x7f9b100caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b14f23640 1 -- 172.21.15.82:0/3865180743 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100cc7b0 con 0x7f9b100cfca0 2024-04-25T10:03:05.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b14f23640 1 --2- 172.21.15.82:0/3865180743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b100cfca0 0x7f9b1015dfb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0097c0 tx=0x7f9b0c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a4d5cf490f5f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b07fff640 1 -- 172.21.15.82:0/3865180743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0b0070 con 0x7f9b100cfca0 2024-04-25T10:03:05.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b07fff640 1 -- 172.21.15.82:0/3865180743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0c0ab600 con 0x7f9b100cfca0 2024-04-25T10:03:05.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.399+0000 7f9b07fff640 1 -- 172.21.15.82:0/3865180743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0ab9d0 con 0x7f9b100cfca0 2024-04-25T10:03:05.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/3865180743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b100cfca0 msgr2=0x7f9b1015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 --2- 172.21.15.82:0/3865180743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b100cfca0 0x7f9b1015dfb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0097c0 tx=0x7f9b0c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/3865180743 shutdown_connections 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 --2- 172.21.15.82:0/3865180743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b100cfca0 0x7f9b1015dfb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/3865180743 >> 172.21.15.82:0/3865180743 conn(0x7f9b1006ae60 msgr2=0x7f9b1006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/3865180743 shutdown_connections 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/3865180743 wait complete. 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 Processor -- start 2024-04-25T10:03:05.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- start start 2024-04-25T10:03:05.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 0x7f9b100ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b100cfca0 0x7f9b10098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1016b8a0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1016b720 con 0x7f9b100b3cf0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1016ba20 con 0x7f9b100cfca0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52500/0 (socket says 172.21.15.82:52500) 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 -- 172.21.15.82:0/329893816 learned_addr learned my addr 172.21.15.82:0/329893816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15724640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 0x7f9b100ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b14f23640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b100cfca0 0x7f9b10098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 -- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b100cfca0 msgr2=0x7f9b10098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b100cfca0 0x7f9b10098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 -- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 msgr2=0x7f9b100ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 0x7f9b100ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 -- 172.21.15.82:0/329893816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b100b2a10 con 0x7f9b1015e4f0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15724640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 0x7f9b100ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b15f25640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800caa0 tx=0x7f9b0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08017070 con 0x7f9b1015e4f0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b0800ac00 con 0x7f9b1015e4f0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b101571d0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b08012400 con 0x7f9b1015e4f0 2024-04-25T10:03:05.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b10156ef0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.403+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1004efa0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.407+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b0801a070 con 0x7f9b1015e4f0 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.407+0000 7f9b05ffb640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.407+0000 7f9b15724640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.407+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9b08085150 con 0x7f9b1015e4f0 2024-04-25T10:03:05.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.407+0000 7f9b15724640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 0x7f9af4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9b10152390 tx=0x7f9b00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.411+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b08055dd0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.567+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b100b3060 con 0x7f9b1015e4f0 2024-04-25T10:03:05.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.567+0000 7f9b05ffb640 1 -- 172.21.15.82:0/329893816 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b080556d0 con 0x7f9b1015e4f0 2024-04-25T10:03:05.577 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:05.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 msgr2=0x7f9af4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 0x7f9af4069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9b10152390 tx=0x7f9b00009040 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 msgr2=0x7f9b1006e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800caa0 tx=0x7f9b0800cf60 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 shutdown_connections 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9af4066f60 0x7f9af4069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9b100cfca0 0x7f9b10098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9b1015e4f0 0x7f9b1006e4b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 --2- 172.21.15.82:0/329893816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9b100b3cf0 0x7f9b100ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 >> 172.21.15.82:0/329893816 conn(0x7f9b1006ae60 msgr2=0x7f9b1015fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 shutdown_connections 2024-04-25T10:03:05.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.571+0000 7f9b17988640 1 -- 172.21.15.82:0/329893816 wait complete. 2024-04-25T10:03:05.596 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:05.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 Processor -- start 2024-04-25T10:03:05.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 -- start start 2024-04-25T10:03:05.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33400cf590 0x7f33400cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400cbd20 con 0x7f3340072000 2024-04-25T10:03:05.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33400cc000 con 0x7f334015b6b0 2024-04-25T10:03:05.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f3346c16640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334015ba90 con 0x7f33400cf590 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33451b3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3340072000 0x7f3340072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44744/0 (socket says 172.21.15.82:44744) 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33451b3640 1 -- 172.21.15.82:0/485800595 learned_addr learned my addr 172.21.15.82:0/485800595 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333ffff640 1 --2- 172.21.15.82:0/485800595 >> v2:172.21.15.196:3300/0 conn(0x7f33400cf590 0x7f33400cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681906805 0 0) 0x7f33400cbd20 con 0x7f3340072000 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3328003530 con 0x7f3340072000 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 202304627 0 0) 0x7f3328003530 con 0x7f3340072000 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 >> v2:172.21.15.196:3300/0 conn(0x7f33400cf590 msgr2=0x7f33400cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 --2- 172.21.15.82:0/485800595 >> v2:172.21.15.196:3300/0 conn(0x7f33400cf590 0x7f33400cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 >> v1:172.21.15.196:6789/0 conn(0x7f334015b6b0 legacy=0x7f33400ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33400cc2e0 con 0x7f3340072000 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3338001a00 con 0x7f3340072000 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 >> v1:172.21.15.82:6789/0 conn(0x7f3340072000 legacy=0x7f3340072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f3328003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3328005220 0x7f332800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334015ba90 con 0x7f3328003a80 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332800c150 con 0x7f3328005220 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332800c350 con 0x7f3328004810 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f3328003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f333ffff640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 -- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3328004810 msgr2=0x7f3328004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3328004810 0x7f3328004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 -- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3328005220 msgr2=0x7f332800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3328005220 0x7f332800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33449b2640 1 -- 172.21.15.82:0/485800595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328003530 con 0x7f3328003a80 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.711+0000 7f33451b3640 1 -- 172.21.15.82:0/485800595 reap_dead start 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f3328003ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3330001780 tx=0x7f333001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5111ec7f9f0add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300013a0 con 0x7f3328003a80 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33300052f0 con 0x7f3328003a80 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f333f7fe640 1 -- 172.21.15.82:0/485800595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330003840 con 0x7f3328003a80 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 msgr2=0x7f3328003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f3328003ea0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f3330001780 tx=0x7f333001c3f0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- 172.21.15.82:0/485800595 shutdown_connections 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 --2- 172.21.15.82:0/485800595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f3328003ea0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- 172.21.15.82:0/485800595 >> 172.21.15.82:0/485800595 conn(0x7f334006ae60 msgr2=0x7f334006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- 172.21.15.82:0/485800595 shutdown_connections 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- 172.21.15.82:0/485800595 wait complete. 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 Processor -- start 2024-04-25T10:03:05.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- start start 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33400b0640 0x7f33400c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33401f7a20 0x7f334015d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400cff00 con 0x7f3328003a80 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33400cfd80 con 0x7f33400b0640 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f3346c16640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33400cadb0 con 0x7f33401f7a20 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52516/0 (socket says 172.21.15.82:52516) 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 -- 172.21.15.82:0/1136103339 learned_addr learned my addr 172.21.15.82:0/1136103339 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 -- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33401f7a20 msgr2=0x7f334015d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33401f7a20 0x7f334015d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 -- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33400b0640 msgr2=0x7f33400c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f333ffff640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33400b0640 0x7f33400c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33400b0640 0x7f33400c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.715+0000 7f33449b2640 1 -- 172.21.15.82:0/1136103339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334015e460 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f33449b2640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3330000c10 tx=0x7f3328005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300013a0 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33300054a0 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334015e740 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3330007a40 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33400c9c10 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334004efa0 con 0x7f3328003a80 2024-04-25T10:03:05.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33300067e0 con 0x7f3328003a80 2024-04-25T10:03:05.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 0x7f332c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333ffff640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 0x7f332c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f333011ede0 con 0x7f3328003a80 2024-04-25T10:03:05.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.719+0000 7f333ffff640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 0x7f332c069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f334006d230 tx=0x7f3334008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.723+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3330016130 con 0x7f3328003a80 2024-04-25T10:03:05.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.871+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f334015e910 con 0x7f3328003a80 2024-04-25T10:03:05.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.871+0000 7f333d7fa640 1 -- 172.21.15.82:0/1136103339 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33300ef9c0 con 0x7f3328003a80 2024-04-25T10:03:05.881 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:05.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 msgr2=0x7f332c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 0x7f332c069450 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f334006d230 tx=0x7f3334008040 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 msgr2=0x7f33400c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f3330000c10 tx=0x7f3328005840 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 shutdown_connections 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f332c066f90 0x7f332c069450 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33401f7a20 0x7f334015d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33400b0640 0x7f33400c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 --2- 172.21.15.82:0/1136103339 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3328003a80 0x7f33400c8600 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 >> 172.21.15.82:0/1136103339 conn(0x7f334006ae60 msgr2=0x7f33400afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 shutdown_connections 2024-04-25T10:03:05.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:05.875+0000 7f3346c16640 1 -- 172.21.15.82:0/1136103339 wait complete. 2024-04-25T10:03:05.898 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:06.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 Processor -- start 2024-04-25T10:03:06.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 -- start start 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9d8009a180 0x7f9d800b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d800ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800cbec0 con 0x7f9d800b2c30 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d800cc1a0 con 0x7f9d8009a180 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d87ca0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800cc480 con 0x7f9d8009a7b0 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d85a3c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9d800b2c30 0x7f9d80099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44758/0 (socket says 172.21.15.82:44758) 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d85a3c640 1 -- 172.21.15.82:0/133428898 learned_addr learned my addr 172.21.15.82:0/133428898 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501484402 0 0) 0x7f9d800cbec0 con 0x7f9d800b2c30 2024-04-25T10:03:06.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d6c003530 con 0x7f9d800b2c30 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8523b640 1 --2- 172.21.15.82:0/133428898 >> v2:172.21.15.196:3300/0 conn(0x7f9d8009a180 0x7f9d800b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d800ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3931457764 0 0) 0x7f9d6c003530 con 0x7f9d800b2c30 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 msgr2=0x7f9d800ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d800ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 >> v2:172.21.15.196:3300/0 conn(0x7f9d8009a180 msgr2=0x7f9d800b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 --2- 172.21.15.82:0/133428898 >> v2:172.21.15.196:3300/0 conn(0x7f9d8009a180 0x7f9d800b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d800cc760 con 0x7f9d800b2c30 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9d7c001680 con 0x7f9d800b2c30 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 >> v1:172.21.15.82:6789/0 conn(0x7f9d800b2c30 legacy=0x7f9d80099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c003a30 0x7f9d6c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c004d50 0x7f9d6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800cc1a0 con 0x7f9d6c004d50 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800cc480 con 0x7f9d6c003a30 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c00b9d0 con 0x7f9d8009a7b0 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c004d50 0x7f9d6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8523b640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 -- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 msgr2=0x7f9d6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d8009a7b0 0x7f9d6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 -- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c003a30 msgr2=0x7f9d6c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c003a30 0x7f9d6c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.003+0000 7f9d8623d640 1 -- 172.21.15.82:0/133428898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6c003530 con 0x7f9d6c004d50 2024-04-25T10:03:06.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d8623d640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c004d50 0x7f9d6c0051d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fc0 tx=0x7f9d7801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819a67ec79d26059 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d780b2070 con 0x7f9d6c004d50 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d780a5c90 con 0x7f9d6c004d50 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d84a3a640 1 -- 172.21.15.82:0/133428898 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78009c10 con 0x7f9d6c004d50 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- 172.21.15.82:0/133428898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c004d50 msgr2=0x7f9d6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/133428898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d6c004d50 0x7f9d6c0051d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009fc0 tx=0x7f9d7801cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d8623d640 1 -- 172.21.15.82:0/133428898 reap_dead start 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- 172.21.15.82:0/133428898 shutdown_connections 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- 172.21.15.82:0/133428898 >> 172.21.15.82:0/133428898 conn(0x7f9d800b3010 msgr2=0x7f9d800b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- 172.21.15.82:0/133428898 shutdown_connections 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- 172.21.15.82:0/133428898 wait complete. 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 Processor -- start 2024-04-25T10:03:06.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.007+0000 7f9d87ca0640 1 -- start start 2024-04-25T10:03:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c004d50 0x7f9d800c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d800cfdd0 0x7f9d800d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800cbb70 con 0x7f9d800b4b50 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8009ac40 con 0x7f9d6c004d50 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8006c200 con 0x7f9d800cfdd0 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52526/0 (socket says 172.21.15.82:52526) 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 -- 172.21.15.82:0/1253067393 learned_addr learned my addr 172.21.15.82:0/1253067393 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d85a3c640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c004d50 0x7f9d800c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8623d640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d800cfdd0 0x7f9d800d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 -- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d800cfdd0 msgr2=0x7f9d800d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d800cfdd0 0x7f9d800d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 -- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c004d50 msgr2=0x7f9d800c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c004d50 0x7f9d800c7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 -- 172.21.15.82:0/1253067393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8015b060 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d8523b640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d70011660 tx=0x7f9d70011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7000f9c0 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7000fb60 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d800b5330 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70015460 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d800b58f0 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8004efa0 con 0x7f9d800b4b50 2024-04-25T10:03:06.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d70015600 con 0x7f9d800b4b50 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 0x7f9d64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d85a3c640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 0x7f9d64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9d70085df0 con 0x7f9d800b4b50 2024-04-25T10:03:06.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.011+0000 7f9d85a3c640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 0x7f9d64069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0017f0 tx=0x7f9d7c001600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.015+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d70019030 con 0x7f9d800b4b50 2024-04-25T10:03:06.161 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:06.164 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:06.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9d8006ba80 con 0x7f9d800b4b50 2024-04-25T10:03:06.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d767fc640 1 -- 172.21.15.82:0/1253067393 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9d700569e0 con 0x7f9d800b4b50 2024-04-25T10:03:06.174 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:06.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 msgr2=0x7f9d64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 0x7f9d64069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c0017f0 tx=0x7f9d7c001600 comp rx=0 tx=0).stop 2024-04-25T10:03:06.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 msgr2=0x7f9d8015ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9d70011660 tx=0x7f9d70011b20 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 shutdown_connections 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d64066f60 0x7f9d64069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d800cfdd0 0x7f9d800d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d800b4b50 0x7f9d8015ab20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 --2- 172.21.15.82:0/1253067393 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d6c004d50 0x7f9d800c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 >> 172.21.15.82:0/1253067393 conn(0x7f9d800b3010 msgr2=0x7f9d800b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 shutdown_connections 2024-04-25T10:03:06.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.167+0000 7f9d87ca0640 1 -- 172.21.15.82:0/1253067393 wait complete. 2024-04-25T10:03:06.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:06.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.367+0000 7f910b5ea640 1 Processor -- start 2024-04-25T10:03:06.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.367+0000 7f910b5ea640 1 -- start start 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f910b5ea640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 0x7f910415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f910b5ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91040ce170 0x7f91040ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f910b5ea640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040ca4c0 con 0x7f91040cf970 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f910b5ea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040ca780 con 0x7f91040ce170 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f910b5ea640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91040cd440 con 0x7f91040cf590 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91040ce170 0x7f91040ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91040ce170 0x7f91040ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52542/0 (socket says 172.21.15.82:52542) 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 -- 172.21.15.82:0/3786389914 learned_addr learned my addr 172.21.15.82:0/3786389914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:06.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109386640 1 --2- 172.21.15.82:0/3786389914 >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 0x7f910415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 -- 172.21.15.82:0/3786389914 >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 msgr2=0x7f910415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 --2- 172.21.15.82:0/3786389914 >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 0x7f910415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 -- 172.21.15.82:0/3786389914 >> v1:172.21.15.196:6789/0 conn(0x7f91040cf970 legacy=0x7f91040c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 -- 172.21.15.82:0/3786389914 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91040cd690 con 0x7f91040ce170 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109386640 1 --2- 172.21.15.82:0/3786389914 >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 0x7f910415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9109b87640 1 --2- 172.21.15.82:0/3786389914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040ce170 0x7f91040ce600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91040d0430 tx=0x7f90f401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418fd3cc0cf5e6d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9103fff640 1 -- 172.21.15.82:0/3786389914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40b0070 con 0x7f91040ce170 2024-04-25T10:03:06.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9103fff640 1 -- 172.21.15.82:0/3786389914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f40ab600 con 0x7f91040ce170 2024-04-25T10:03:06.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.371+0000 7f9103fff640 1 -- 172.21.15.82:0/3786389914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40ab9d0 con 0x7f91040ce170 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/3786389914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040ce170 msgr2=0x7f91040ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- 172.21.15.82:0/3786389914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040ce170 0x7f91040ce600 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f91040d0430 tx=0x7f90f401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/3786389914 shutdown_connections 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- 172.21.15.82:0/3786389914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040ce170 0x7f91040ce600 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- 172.21.15.82:0/3786389914 >> v2:172.21.15.196:3300/0 conn(0x7f91040cf590 0x7f910415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/3786389914 >> 172.21.15.82:0/3786389914 conn(0x7f910406ae60 msgr2=0x7f910406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/3786389914 shutdown_connections 2024-04-25T10:03:06.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/3786389914 wait complete. 2024-04-25T10:03:06.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 Processor -- start 2024-04-25T10:03:06.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- start start 2024-04-25T10:03:06.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91040ce170 0x7f91040983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91040cf590 0x7f91040c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040ca240 con 0x7f91040cf970 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91040ca0c0 con 0x7f91040ce170 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91040ac170 con 0x7f91040cf590 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52548/0 (socket says 172.21.15.82:52548) 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 -- 172.21.15.82:0/1936112984 learned_addr learned my addr 172.21.15.82:0/1936112984 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109386640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91040ce170 0x7f91040983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9108b85640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91040cf590 0x7f91040c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 -- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91040cf590 msgr2=0x7f91040c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91040cf590 0x7f91040c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 -- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91040ce170 msgr2=0x7f91040983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91040ce170 0x7f91040983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 -- 172.21.15.82:0/1936112984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910406ea50 con 0x7f91040cf970 2024-04-25T10:03:06.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9109b87640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ae040 tx=0x7f90f401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40b0040 con 0x7f91040cf970 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f40bc070 con 0x7f91040cf970 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910415d380 con 0x7f91040cf970 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40b6860 con 0x7f91040cf970 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.375+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910415ac70 con 0x7f91040cf970 2024-04-25T10:03:06.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910404efa0 con 0x7f91040cf970 2024-04-25T10:03:06.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90f40ab4c0 con 0x7f91040cf970 2024-04-25T10:03:06.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f9101ffb640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 0x7f90f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f90f411f7a0 con 0x7f91040cf970 2024-04-25T10:03:06.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f9109386640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 0x7f90f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.379+0000 7f9109386640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 0x7f90f0069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0075b0 tx=0x7f90fc007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.383+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90f40160d0 con 0x7f91040cf970 2024-04-25T10:03:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f91040b4170 con 0x7f91040cf970 2024-04-25T10:03:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f9101ffb640 1 -- 172.21.15.82:0/1936112984 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f90f40f0370 con 0x7f91040cf970 2024-04-25T10:03:06.539 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 msgr2=0x7f90f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 0x7f90f0069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0075b0 tx=0x7f90fc007540 comp rx=0 tx=0).stop 2024-04-25T10:03:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 msgr2=0x7f91040aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f90f40ae040 tx=0x7f90f401ad20 comp rx=0 tx=0).stop 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 shutdown_connections 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f90f0066f60 0x7f90f0069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91040cf590 0x7f91040c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91040cf970 0x7f91040aedc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.531+0000 7f910b5ea640 1 --2- 172.21.15.82:0/1936112984 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91040ce170 0x7f91040983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.535+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 >> 172.21.15.82:0/1936112984 conn(0x7f910406ae60 msgr2=0x7f91040afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.535+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 shutdown_connections 2024-04-25T10:03:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.535+0000 7f910b5ea640 1 -- 172.21.15.82:0/1936112984 wait complete. 2024-04-25T10:03:06.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:06.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 Processor -- start 2024-04-25T10:03:06.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 -- start start 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fee701451f0 0x7fee70142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee700ccc70 con 0x7fee70144ca0 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7006dc40 con 0x7fee700ce510 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee74fdf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee700cc2b0 con 0x7fee701451f0 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6dd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fee70144ca0 0x7fee7006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44766/0 (socket says 172.21.15.82:44766) 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6dd9b640 1 -- 172.21.15.82:0/4221506268 learned_addr learned my addr 172.21.15.82:0/4221506268 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 90164394 0 0) 0x7fee700ccc70 con 0x7fee70144ca0 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/4221506268 >> v2:172.21.15.196:3300/0 conn(0x7fee701451f0 0x7fee70142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee58003530 con 0x7fee70144ca0 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3522184884 0 0) 0x7fee58003530 con 0x7fee70144ca0 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 >> v2:172.21.15.196:3300/0 conn(0x7fee701451f0 msgr2=0x7fee70142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 --2- 172.21.15.82:0/4221506268 >> v2:172.21.15.196:3300/0 conn(0x7fee701451f0 0x7fee70142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 >> v1:172.21.15.196:6789/0 conn(0x7fee700ce510 legacy=0x7fee700ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7006df20 con 0x7fee70144ca0 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fee64001680 con 0x7fee70144ca0 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 >> v1:172.21.15.82:6789/0 conn(0x7fee70144ca0 legacy=0x7fee7006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee58003a60 0x7fee58003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58004810 0x7fee58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 0x7fee5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee700cc2b0 con 0x7fee580052f0 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5800c1b0 con 0x7fee58004810 2024-04-25T10:03:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5800c3b0 con 0x7fee58003a60 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 0x7fee5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6dd9b640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58004810 0x7fee58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.659+0000 7fee6e59c640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee58003a60 0x7fee58003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 -- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee58003a60 msgr2=0x7fee58003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee58003a60 0x7fee58003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 -- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58004810 msgr2=0x7fee58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58004810 0x7fee58004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 -- 172.21.15.82:0/4221506268 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee58003530 con 0x7fee580052f0 2024-04-25T10:03:06.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6dd9b640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee58004810 0x7fee58004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 -- 172.21.15.82:0/4221506268 reap_dead start 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 0x7fee5800b9d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee701456a0 tx=0x7fee6801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c170a7a9905f8a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6800b610 con 0x7fee580052f0 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee680a9460 con 0x7fee580052f0 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee6d59a640 1 -- 172.21.15.82:0/4221506268 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a9850 con 0x7fee580052f0 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 -- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 msgr2=0x7fee5800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 0x7fee5800b9d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fee701456a0 tx=0x7fee6801c680 comp rx=0 tx=0).stop 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 -- 172.21.15.82:0/4221506268 shutdown_connections 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 --2- 172.21.15.82:0/4221506268 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee580052f0 0x7fee5800b9d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 -- 172.21.15.82:0/4221506268 >> 172.21.15.82:0/4221506268 conn(0x7fee700b11a0 msgr2=0x7fee700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 -- 172.21.15.82:0/4221506268 shutdown_connections 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.663+0000 7fee74fdf640 1 -- 172.21.15.82:0/4221506268 wait complete. 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 Processor -- start 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- start start 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee580052f0 0x7fee701408f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 0x7fee700c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee700ce8f0 con 0x7fee701e65f0 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7006e290 con 0x7fee701400e0 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700cea70 con 0x7fee580052f0 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52570/0 (socket says 172.21.15.82:52570) 2024-04-25T10:03:06.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 -- 172.21.15.82:0/1103111511 learned_addr learned my addr 172.21.15.82:0/1103111511 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 0x7fee700c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 -- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee580052f0 msgr2=0x7fee701408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee580052f0 0x7fee701408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 -- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 msgr2=0x7fee700c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 0x7fee700c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 -- 172.21.15.82:0/1103111511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee70141150 con 0x7fee701e65f0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6ed9d640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 0x7fee700c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6dd9b640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e7b0 tx=0x7fee6400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400ee60 con 0x7fee701e65f0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee64018070 con 0x7fee701e65f0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee701e5550 con 0x7fee701e65f0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400cad0 con 0x7fee701e65f0 2024-04-25T10:03:06.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee701e5b10 con 0x7fee701e65f0 2024-04-25T10:03:06.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7004efa0 con 0x7fee701e65f0 2024-04-25T10:03:06.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee6400cc70 con 0x7fee701e65f0 2024-04-25T10:03:06.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 0x7fee50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fee640852a0 con 0x7fee701e65f0 2024-04-25T10:03:06.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.667+0000 7fee6e59c640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 0x7fee50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.671+0000 7fee6e59c640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 0x7fee50069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee60001b60 tx=0x7fee60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.671+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee64055e90 con 0x7fee701e65f0 2024-04-25T10:03:06.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fee700a8dd0 con 0x7fee701e65f0 2024-04-25T10:03:06.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee5effd640 1 -- 172.21.15.82:0/1103111511 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fee640850c0 con 0x7fee701e65f0 2024-04-25T10:03:06.846 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 msgr2=0x7fee50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 0x7fee50069420 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee60001b60 tx=0x7fee60009040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 msgr2=0x7fee701e4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e7b0 tx=0x7fee6400ec70 comp rx=0 tx=0).stop 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 shutdown_connections 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee50066f60 0x7fee50069420 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee701400e0 0x7fee700c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee701e65f0 0x7fee701e4e70 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 --2- 172.21.15.82:0/1103111511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee580052f0 0x7fee701408f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 >> 172.21.15.82:0/1103111511 conn(0x7fee700b11a0 msgr2=0x7fee70141800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 shutdown_connections 2024-04-25T10:03:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.839+0000 7fee74fdf640 1 -- 172.21.15.82:0/1103111511 wait complete. 2024-04-25T10:03:06.864 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:06.864 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:06.871 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:06.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 Processor -- start 2024-04-25T10:03:06.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 -- start start 2024-04-25T10:03:06.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1db40ce510 0x7f1db40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1db41451f0 0x7f1db4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db40cee30 con 0x7f1db4144ca0 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db40cc2b0 con 0x7f1db41451f0 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db937a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db406e290 con 0x7f1db40ce510 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db27fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1db4144ca0 0x7f1db406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44776/0 (socket says 172.21.15.82:44776) 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db27fc640 1 -- 172.21.15.82:0/2573006754 learned_addr learned my addr 172.21.15.82:0/2573006754 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db37fe640 1 --2- 172.21.15.82:0/2573006754 >> v2:172.21.15.82:3300/0 conn(0x7f1db41451f0 0x7f1db4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db2ffd640 1 --2- 172.21.15.82:0/2573006754 >> v2:172.21.15.196:3300/0 conn(0x7f1db40ce510 0x7f1db40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db37fe640 1 -- 172.21.15.82:0/2573006754 >> v2:172.21.15.196:3300/0 conn(0x7f1db40ce510 msgr2=0x7f1db40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db37fe640 1 --2- 172.21.15.82:0/2573006754 >> v2:172.21.15.196:3300/0 conn(0x7f1db40ce510 0x7f1db40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db37fe640 1 -- 172.21.15.82:0/2573006754 >> v1:172.21.15.82:6789/0 conn(0x7f1db4144ca0 legacy=0x7f1db406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db37fe640 1 -- 172.21.15.82:0/2573006754 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40ccc70 con 0x7f1db41451f0 2024-04-25T10:03:06.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.983+0000 7f1db1ffb640 1 -- 172.21.15.82:0/2573006754 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069247985 0 0) 0x7f1db40cee30 con 0x7f1db4144ca0 2024-04-25T10:03:06.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db37fe640 1 --2- 172.21.15.82:0/2573006754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db41451f0 0x7f1db4142680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1db4144c50 tx=0x7f1dac01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dbc1918d951d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db1ffb640 1 -- 172.21.15.82:0/2573006754 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0b4070 con 0x7f1db41451f0 2024-04-25T10:03:06.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db1ffb640 1 -- 172.21.15.82:0/2573006754 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac0af600 con 0x7f1db41451f0 2024-04-25T10:03:06.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db1ffb640 1 -- 172.21.15.82:0/2573006754 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0af9d0 con 0x7f1db41451f0 2024-04-25T10:03:06.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- 172.21.15.82:0/2573006754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db41451f0 msgr2=0x7f1db4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- 172.21.15.82:0/2573006754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db41451f0 0x7f1db4142680 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1db4144c50 tx=0x7f1dac01f110 comp rx=0 tx=0).stop 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- 172.21.15.82:0/2573006754 shutdown_connections 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- 172.21.15.82:0/2573006754 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db41451f0 0x7f1db4142680 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- 172.21.15.82:0/2573006754 >> v2:172.21.15.196:3300/0 conn(0x7f1db40ce510 0x7f1db40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- 172.21.15.82:0/2573006754 >> 172.21.15.82:0/2573006754 conn(0x7f1db40b11a0 msgr2=0x7f1db40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- 172.21.15.82:0/2573006754 shutdown_connections 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- 172.21.15.82:0/2573006754 wait complete. 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 Processor -- start 2024-04-25T10:03:06.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- start start 2024-04-25T10:03:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 0x7f1db41f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1db41451f0 0x7f1db4133200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db406ee30 con 0x7f1db4144ca0 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db406df20 con 0x7f1db40ce510 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db937a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db40cc900 con 0x7f1db41451f0 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52584/0 (socket says 172.21.15.82:52584) 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 -- 172.21.15.82:0/3887188878 learned_addr learned my addr 172.21.15.82:0/3887188878 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db2ffd640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 0x7f1db41f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 -- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1db41451f0 msgr2=0x7f1db4133200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1db41451f0 0x7f1db4133200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 -- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 msgr2=0x7f1db41f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 0x7f1db41f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 -- 172.21.15.82:0/3887188878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40a8a10 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db2ffd640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 0x7f1db41f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.987+0000 7f1db27fc640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c4b0 tx=0x7f1da800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8018070 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8007500 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db41e2690 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8004a30 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db41e2c50 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db41e3130 con 0x7f1db4144ca0 2024-04-25T10:03:07.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1da801b070 con 0x7f1db4144ca0 2024-04-25T10:03:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f1da80846a0 con 0x7f1db4144ca0 2024-04-25T10:03:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1db2ffd640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.991+0000 7f1db2ffd640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 0x7f1da0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009840 tx=0x7f1da4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:06.995+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1db41e3130 con 0x7f1db4144ca0 2024-04-25T10:03:07.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1db40a8d40 con 0x7f1db4144ca0 2024-04-25T10:03:07.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1d9b7fe640 1 -- 172.21.15.82:0/3887188878 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f1da80553e0 con 0x7f1db4144ca0 2024-04-25T10:03:07.163 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:07.163 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 msgr2=0x7f1da0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 0x7f1da0069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1da4009840 tx=0x7f1da4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 msgr2=0x7f1db40a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1da800c4b0 tx=0x7f1da800c970 comp rx=0 tx=0).stop 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 shutdown_connections 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1da0066f60 0x7f1da0069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1db41451f0 0x7f1db4133200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1db4144ca0 0x7f1db40a84d0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 --2- 172.21.15.82:0/3887188878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1db40ce510 0x7f1db41f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 >> 172.21.15.82:0/3887188878 conn(0x7f1db40b11a0 msgr2=0x7f1db40cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 shutdown_connections 2024-04-25T10:03:07.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.155+0000 7f1db937a640 1 -- 172.21.15.82:0/3887188878 wait complete. 2024-04-25T10:03:07.182 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:07.182 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:07.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.295+0000 7f2ced891640 1 Processor -- start 2024-04-25T10:03:07.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.295+0000 7f2ced891640 1 -- start start 2024-04-25T10:03:07.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ced891640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ce80b0730 0x7f2ce80d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ced891640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2ce806c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ced891640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce8099e50 con 0x7f2ce806cb40 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ced891640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce809aa60 con 0x7f2ce80b0730 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ced891640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce809a130 con 0x7f2ce80d08e0 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2ce806cb40 0x7f2ce815d1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44792/0 (socket says 172.21.15.82:44792) 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce77fe640 1 -- 172.21.15.82:0/3420546135 learned_addr learned my addr 172.21.15.82:0/3420546135 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce6ffd640 1 --2- 172.21.15.82:0/3420546135 >> v2:172.21.15.196:3300/0 conn(0x7f2ce80b0730 0x7f2ce80d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2ce806c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3147361631 0 0) 0x7f2ce8099e50 con 0x7f2ce806cb40 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd4003530 con 0x7f2ce806cb40 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3995622432 0 0) 0x7f2cd4003530 con 0x7f2ce806cb40 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 msgr2=0x7f2ce806c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2ce806c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 >> v2:172.21.15.196:3300/0 conn(0x7f2ce80b0730 msgr2=0x7f2ce80d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 --2- 172.21.15.82:0/3420546135 >> v2:172.21.15.196:3300/0 conn(0x7f2ce80b0730 0x7f2ce80d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce809a410 con 0x7f2ce806cb40 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2ce0001a00 con 0x7f2ce806cb40 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2ce806c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 >> v1:172.21.15.82:6789/0 conn(0x7f2ce806cb40 legacy=0x7f2ce815d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2cd4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2cd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce809aa60 con 0x7f2cd4003f70 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce809a130 con 0x7f2cd4004d50 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd400ba50 con 0x7f2ce80d08e0 2024-04-25T10:03:07.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2cd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce6ffd640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2cd4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2ce77fe640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 -- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 msgr2=0x7f2cd4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80d08e0 0x7f2cd4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 -- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cd4004d50 msgr2=0x7f2cd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2cd4004d50 0x7f2cd40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 -- 172.21.15.82:0/3420546135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd4003530 con 0x7f2cd4003f70 2024-04-25T10:03:07.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.299+0000 7f2cdffff640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2cd4004810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098d0 tx=0x7f2cd001c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4076ddecf5ed7495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd000f040 con 0x7f2cd4003f70 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd00af070 con 0x7f2cd4003f70 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce67fc640 1 -- 172.21.15.82:0/3420546135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00aa870 con 0x7f2cd4003f70 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/3420546135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 msgr2=0x7f2cd4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 --2- 172.21.15.82:0/3420546135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2cd4004810 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00098d0 tx=0x7f2cd001c5d0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce77fe640 1 -- 172.21.15.82:0/3420546135 reap_dead start 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/3420546135 shutdown_connections 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/3420546135 >> 172.21.15.82:0/3420546135 conn(0x7f2ce80b3060 msgr2=0x7f2ce80b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/3420546135 shutdown_connections 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/3420546135 wait complete. 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 Processor -- start 2024-04-25T10:03:07.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- start start 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80b5800 0x7f2ce815f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ce814fae0 0x7f2ce814ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8099bf0 con 0x7f2cd4003f70 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8099a70 con 0x7f2ce814fae0 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80b0b10 con 0x7f2ce80b5800 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52598/0 (socket says 172.21.15.82:52598) 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 -- 172.21.15.82:0/1806411948 learned_addr learned my addr 172.21.15.82:0/1806411948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2cdffff640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80b5800 0x7f2ce815f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce77fe640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ce814fae0 0x7f2ce814ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 -- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80b5800 msgr2=0x7f2ce815f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80b5800 0x7f2ce815f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 -- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ce814fae0 msgr2=0x7f2ce814ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ce814fae0 0x7f2ce814ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 -- 172.21.15.82:0/1806411948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce81515e0 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce6ffd640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011660 tx=0x7f2cd8011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800f9c0 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd800fb60 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8150a80 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8015460 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.303+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce8151040 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce804efa0 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2cd8015600 con 0x7f2cd4003f70 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2ce4ff9640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 0x7f2cc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2cdffff640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 0x7f2cc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2cd8085eb0 con 0x7f2cd4003f70 2024-04-25T10:03:07.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.307+0000 7f2cdffff640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 0x7f2cc8069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00ae000 tx=0x7f2cd00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.311+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2cd8019030 con 0x7f2cd4003f70 2024-04-25T10:03:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ce806f9d0 con 0x7f2cd4003f70 2024-04-25T10:03:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ce4ff9640 1 -- 172.21.15.82:0/1806411948 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cd8056a10 con 0x7f2cd4003f70 2024-04-25T10:03:07.494 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 msgr2=0x7f2cc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 0x7f2cc8069450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2cd00ae000 tx=0x7f2cd00046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 msgr2=0x7f2ce815f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8011660 tx=0x7f2cd8011b20 comp rx=0 tx=0).stop 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 shutdown_connections 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2cc8066f90 0x7f2cc8069450 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ce80b5800 0x7f2ce815f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ce814fae0 0x7f2ce814ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 --2- 172.21.15.82:0/1806411948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2cd4003f70 0x7f2ce815f460 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 >> 172.21.15.82:0/1806411948 conn(0x7f2ce80b3060 msgr2=0x7f2ce80cddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 shutdown_connections 2024-04-25T10:03:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.487+0000 7f2ced891640 1 -- 172.21.15.82:0/1806411948 wait complete. 2024-04-25T10:03:07.512 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 Processor -- start 2024-04-25T10:03:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 -- start start 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc93015b6b0 0x7fc9300ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93015ff50 con 0x7fc9300cf590 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300cbd20 con 0x7fc930073010 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc9369d2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93015ba90 con 0x7fc93015b6b0 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc93015b6b0 0x7fc9300ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc93015b6b0 0x7fc9300ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52602/0 (socket says 172.21.15.82:52602) 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc9300cf590 0x7fc93015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44798/0 (socket says 172.21.15.82:44798) 2024-04-25T10:03:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 -- 172.21.15.82:0/4224101936 learned_addr learned my addr 172.21.15.82:0/4224101936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 -- 172.21.15.82:0/4224101936 >> v1:172.21.15.196:6789/0 conn(0x7fc930073010 legacy=0x7fc93015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92effd640 1 -- 172.21.15.82:0/4224101936 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816790155 0 0) 0x7fc93015ff50 con 0x7fc9300cf590 2024-04-25T10:03:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 -- 172.21.15.82:0/4224101936 >> v1:172.21.15.82:6789/0 conn(0x7fc9300cf590 legacy=0x7fc93015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.627+0000 7fc92ffff640 1 -- 172.21.15.82:0/4224101936 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300cc000 con 0x7fc93015b6b0 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc92ffff640 1 --2- 172.21.15.82:0/4224101936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc93015b6b0 0x7fc9300ca3c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc920009870 tx=0x7fc92001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2c11727bac08c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc92effd640 1 -- 172.21.15.82:0/4224101936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200aa650 con 0x7fc93015b6b0 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc92effd640 1 -- 172.21.15.82:0/4224101936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9200af070 con 0x7fc93015b6b0 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc92effd640 1 -- 172.21.15.82:0/4224101936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9200aaa10 con 0x7fc93015b6b0 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- 172.21.15.82:0/4224101936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc93015b6b0 msgr2=0x7fc9300ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 --2- 172.21.15.82:0/4224101936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc93015b6b0 0x7fc9300ca3c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fc920009870 tx=0x7fc92001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- 172.21.15.82:0/4224101936 shutdown_connections 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 --2- 172.21.15.82:0/4224101936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc93015b6b0 0x7fc9300ca3c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- 172.21.15.82:0/4224101936 >> 172.21.15.82:0/4224101936 conn(0x7fc93006ae60 msgr2=0x7fc93006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- 172.21.15.82:0/4224101936 shutdown_connections 2024-04-25T10:03:07.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- 172.21.15.82:0/4224101936 wait complete. 2024-04-25T10:03:07.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 Processor -- start 2024-04-25T10:03:07.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.631+0000 7fc9369d2640 1 -- start start 2024-04-25T10:03:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc930073010 0x7fc9300c0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc93015b6b0 0x7fc9301648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300ca900 con 0x7fc9300cf590 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300cba30 con 0x7fc930073010 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9300cfdf0 con 0x7fc93015b6b0 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52610/0 (socket says 172.21.15.82:52610) 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 -- 172.21.15.82:0/2247784817 learned_addr learned my addr 172.21.15.82:0/2247784817 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92ffff640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc930073010 0x7fc9300c0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 -- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc93015b6b0 msgr2=0x7fc9301648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc93015b6b0 0x7fc9301648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 -- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc930073010 msgr2=0x7fc9300c0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc930073010 0x7fc9300c0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 -- 172.21.15.82:0/2247784817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300b0a50 con 0x7fc9300cf590 2024-04-25T10:03:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92f7fe640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c520 tx=0x7fc92400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924018070 con 0x7fc9300cf590 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9240074e0 con 0x7fc9300cf590 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc930165c90 con 0x7fc9300cf590 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240049f0 con 0x7fc9300cf590 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc930166250 con 0x7fc9300cf590 2024-04-25T10:03:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93004efa0 con 0x7fc9300cf590 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc92401b070 con 0x7fc9300cf590 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc924084880 con 0x7fc9300cf590 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92ffff640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.635+0000 7fc92ffff640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ae000 tx=0x7fc920008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.639+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc924055500 con 0x7fc9300cf590 2024-04-25T10:03:07.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.787+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc93006b3e0 con 0x7fc9300cf590 2024-04-25T10:03:07.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.787+0000 7fc92cff9640 1 -- 172.21.15.82:0/2247784817 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc924054e00 con 0x7fc9300cf590 2024-04-25T10:03:07.797 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 msgr2=0x7fc91c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 0x7fc91c069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc9200ae000 tx=0x7fc920008040 comp rx=0 tx=0).stop 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 msgr2=0x7fc9300c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c520 tx=0x7fc92400c9e0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 shutdown_connections 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc91c066f60 0x7fc91c069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc93015b6b0 0x7fc9301648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc9300cf590 0x7fc9300c1220 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 --2- 172.21.15.82:0/2247784817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc930073010 0x7fc9300c0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 >> 172.21.15.82:0/2247784817 conn(0x7fc93006ae60 msgr2=0x7fc9300aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 shutdown_connections 2024-04-25T10:03:07.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.791+0000 7fc9369d2640 1 -- 172.21.15.82:0/2247784817 wait complete. 2024-04-25T10:03:07.814 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.927+0000 7f93e6588640 1 Processor -- start 2024-04-25T10:03:07.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.927+0000 7f93e6588640 1 -- start start 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e6588640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93e0072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e6588640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e00cfe90 con 0x7f93e00cf590 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e6588640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e00cbd20 con 0x7f93e015b6b0 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e6588640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e015ba90 con 0x7f93e00cfa20 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f93e00cf590 0x7f93e0072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44800/0 (socket says 172.21.15.82:44800) 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 -- 172.21.15.82:0/3554847376 learned_addr learned my addr 172.21.15.82:0/3554847376 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:07.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024171458 0 0) 0x7f93e00cfe90 con 0x7f93e00cf590 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93c8003530 con 0x7f93e00cf590 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e4b25640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93e0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3373778805 0 0) 0x7f93c8003530 con 0x7f93e00cf590 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 msgr2=0x7f93e0072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93e0072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 >> v1:172.21.15.196:6789/0 conn(0x7f93e015b6b0 legacy=0x7f93e00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00cc000 con 0x7f93e00cf590 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f93d4001680 con 0x7f93e00cf590 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 >> v1:172.21.15.82:6789/0 conn(0x7f93e00cf590 legacy=0x7f93e0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93c8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c8004090 0x7f93c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93c8004da0 0x7f93c8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e015ba90 con 0x7f93c8004090 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800bbe0 con 0x7f93c8004da0 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c800bde0 con 0x7f93e00cfa20 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c8004090 0x7f93c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93e4b25640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93c8004da0 0x7f93c8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93dffff640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93c8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 -- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 msgr2=0x7f93c8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93e00cfa20 0x7f93c8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 -- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93c8004da0 msgr2=0x7f93c8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93c8004da0 0x7f93c8005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.931+0000 7f93df7fe640 1 -- 172.21.15.82:0/3554847376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8003530 con 0x7f93c8004090 2024-04-25T10:03:07.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c8004090 0x7f93c8004810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c210 tx=0x7f93d401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdb7d8366aef19e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4007d40 con 0x7f93c8004090 2024-04-25T10:03:07.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d4005b00 con 0x7f93c8004090 2024-04-25T10:03:07.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93deffd640 1 -- 172.21.15.82:0/3554847376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4007070 con 0x7f93c8004090 2024-04-25T10:03:07.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- 172.21.15.82:0/3554847376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c8004090 msgr2=0x7f93c8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 --2- 172.21.15.82:0/3554847376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93c8004090 0x7f93c8004810 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f93d400c210 tx=0x7f93d401f3f0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e4b25640 1 -- 172.21.15.82:0/3554847376 reap_dead start 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- 172.21.15.82:0/3554847376 shutdown_connections 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- 172.21.15.82:0/3554847376 >> 172.21.15.82:0/3554847376 conn(0x7f93e006ae60 msgr2=0x7f93e006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- 172.21.15.82:0/3554847376 shutdown_connections 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- 172.21.15.82:0/3554847376 wait complete. 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 Processor -- start 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- start start 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93c8004090 0x7f93e0098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e01f4d70 0x7f93e01f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00cc8a0 con 0x7f93e0155650 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00cc5c0 con 0x7f93e01f4d70 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e6588640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00cba30 con 0x7f93c8004090 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52628/0 (socket says 172.21.15.82:52628) 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 -- 172.21.15.82:0/2585673667 learned_addr learned my addr 172.21.15.82:0/2585673667 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93dffff640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93c8004090 0x7f93e0098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93e4b25640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e01f4d70 0x7f93e01f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 -- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93c8004090 msgr2=0x7f93e0098ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93c8004090 0x7f93e0098ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 -- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e01f4d70 msgr2=0x7f93e01f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e01f4d70 0x7f93e01f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 -- 172.21.15.82:0/2585673667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e015cc80 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.935+0000 7f93df7fe640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f93d409bb40 tx=0x7f93c8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4007d20 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d40b6040 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e015c6c0 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40049f0 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e00aff80 con 0x7f93e0155650 2024-04-25T10:03:07.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e004efa0 con 0x7f93e0155650 2024-04-25T10:03:07.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f93d4004b90 con 0x7f93e0155650 2024-04-25T10:03:07.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 0x7f93cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f93d41228c0 con 0x7f93e0155650 2024-04-25T10:03:07.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dffff640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 0x7f93cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.939+0000 7f93dffff640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 0x7f93cc069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93d80051f0 tx=0x7f93d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:07.943+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f93d40c1060 con 0x7f93e0155650 2024-04-25T10:03:08.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.099+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93e006b580 con 0x7f93e0155650 2024-04-25T10:03:08.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.099+0000 7f93dcff9640 1 -- 172.21.15.82:0/2585673667 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93d4019100 con 0x7f93e0155650 2024-04-25T10:03:08.110 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 msgr2=0x7f93cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 0x7f93cc069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f93d80051f0 tx=0x7f93d8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 msgr2=0x7f93e01f4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f93d409bb40 tx=0x7f93c8005840 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 shutdown_connections 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f93cc066f90 0x7f93cc069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f93e01f4d70 0x7f93e01f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f93e0155650 0x7f93e01f4830 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 --2- 172.21.15.82:0/2585673667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f93c8004090 0x7f93e0098ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 >> 172.21.15.82:0/2585673667 conn(0x7f93e006ae60 msgr2=0x7f93e0155be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 shutdown_connections 2024-04-25T10:03:08.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.103+0000 7f93e6588640 1 -- 172.21.15.82:0/2585673667 wait complete. 2024-04-25T10:03:08.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:08.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 Processor -- start 2024-04-25T10:03:08.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 -- start start 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbc415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbc40ce170 0x7fcbc40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc40ca4c0 con 0x7fcbc40cf970 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc40ca780 con 0x7fcbc40ce170 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc92bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40cd440 con 0x7fcbc40cf590 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcbc40cf970 0x7fcbc40c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44812/0 (socket says 172.21.15.82:44812) 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 -- 172.21.15.82:0/1184272409 learned_addr learned my addr 172.21.15.82:0/1184272409 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbc415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/1184272409 >> v2:172.21.15.196:3300/0 conn(0x7fcbc40ce170 0x7fcbc40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597681357 0 0) 0x7fcbc40ca4c0 con 0x7fcbc40cf970 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbac003530 con 0x7fcbc40cf970 2024-04-25T10:03:08.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 813302648 0 0) 0x7fcbac003530 con 0x7fcbc40cf970 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 msgr2=0x7fcbc415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbc415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 >> v2:172.21.15.196:3300/0 conn(0x7fcbc40ce170 msgr2=0x7fcbc40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 --2- 172.21.15.82:0/1184272409 >> v2:172.21.15.196:3300/0 conn(0x7fcbc40ce170 0x7fcbc40ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc40cd690 con 0x7fcbc40cf970 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcbb8001680 con 0x7fcbc40cf970 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 >> v1:172.21.15.82:6789/0 conn(0x7fcbc40cf970 legacy=0x7fcbc40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40cd440 con 0x7fcbac003fb0 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac00b9d0 con 0x7fcbac004d50 2024-04-25T10:03:08.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac00bb80 con 0x7fcbc40cf590 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 -- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 msgr2=0x7fcbac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc40cf590 0x7fcbac003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 -- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbac004d50 msgr2=0x7fcbac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc27fc640 1 -- 172.21.15.82:0/1184272409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac003530 con 0x7fcbac003fb0 2024-04-25T10:03:08.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.239+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbac004d50 0x7fcbac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbac004810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c2c0 tx=0x7fcbb801f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84a4766a807a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8007d60 con 0x7fcbac003fb0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb8005b00 con 0x7fcbac003fb0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc1ffb640 1 -- 172.21.15.82:0/1184272409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8007070 con 0x7fcbac003fb0 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1184272409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 msgr2=0x7fcbac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1184272409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbac004810 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcbb800c2c0 tx=0x7fcbb801f4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc37fe640 1 -- 172.21.15.82:0/1184272409 reap_dead start 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1184272409 shutdown_connections 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1184272409 >> 172.21.15.82:0/1184272409 conn(0x7fcbc406ae60 msgr2=0x7fcbc406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1184272409 shutdown_connections 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1184272409 wait complete. 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 Processor -- start 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- start start 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 0x7fcbc4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4158ed0 0x7fcbc4071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40cfdb0 con 0x7fcbac003fb0 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40cdce0 con 0x7fcbc4159370 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41f6ac0 con 0x7fcbc4158ed0 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52660/0 (socket says 172.21.15.82:52660) 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/1492234866 learned_addr learned my addr 172.21.15.82:0/1492234866 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:08.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 0x7fcbc4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4158ed0 0x7fcbc4071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4158ed0 msgr2=0x7fcbc4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4158ed0 0x7fcbc4071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 msgr2=0x7fcbc4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 0x7fcbc4158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/1492234866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc41f6d10 con 0x7fcbac003fb0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 0x7fcbc4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4010890 tx=0x7fcbb4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400f040 con 0x7fcbac003fb0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb400eb60 con 0x7fcbac003fb0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc41f6ff0 con 0x7fcbac003fb0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb401c400 con 0x7fcbac003fb0 2024-04-25T10:03:08.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.243+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc41f75b0 con 0x7fcbac003fb0 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc41f7990 con 0x7fcbac003fb0 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbb401c5a0 con 0x7fcbac003fb0 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbab7fe640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fcbb4085a00 con 0x7fcbac003fb0 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.247+0000 7fcbc27fc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb8001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.251+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbc41f7990 con 0x7fcbac003fb0 2024-04-25T10:03:08.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcbc404efa0 con 0x7fcbac003fb0 2024-04-25T10:03:08.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbab7fe640 1 -- 172.21.15.82:0/1492234866 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcbb4018030 con 0x7fcbac003fb0 2024-04-25T10:03:08.415 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 msgr2=0x7fcbb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80ae000 tx=0x7fcbb8001a50 comp rx=0 tx=0).stop 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 msgr2=0x7fcbc40c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4010890 tx=0x7fcbb4010d50 comp rx=0 tx=0).stop 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 shutdown_connections 2024-04-25T10:03:08.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4158ed0 0x7fcbc4071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4159370 0x7fcbc4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 --2- 172.21.15.82:0/1492234866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbac003fb0 0x7fcbc40c41a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 >> 172.21.15.82:0/1492234866 conn(0x7fcbc406ae60 msgr2=0x7fcbc415a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 shutdown_connections 2024-04-25T10:03:08.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.407+0000 7fcbc92bc640 1 -- 172.21.15.82:0/1492234866 wait complete. 2024-04-25T10:03:08.433 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:08.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 Processor -- start 2024-04-25T10:03:08.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 -- start start 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fada0099a70 0x7fada00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099e50 0x7fada006c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fada0161510 0x7fada01619a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00cfe20 con 0x7fada0161510 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada00d0050 con 0x7fada0099a70 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fada6055640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada00d0330 con 0x7fada0099e50 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fada0161510 0x7fada01619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fada0161510 0x7fada01619a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52664/0 (socket says 172.21.15.82:52664) 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 -- 172.21.15.82:0/2525738437 learned_addr learned my addr 172.21.15.82:0/2525738437 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9f7fe640 1 --2- 172.21.15.82:0/2525738437 >> v2:172.21.15.196:3300/0 conn(0x7fada0099a70 0x7fada00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9effd640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099e50 0x7fada006c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 -- 172.21.15.82:0/2525738437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099e50 msgr2=0x7fada006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099e50 0x7fada006c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 -- 172.21.15.82:0/2525738437 >> v2:172.21.15.196:3300/0 conn(0x7fada0099a70 msgr2=0x7fada00cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 --2- 172.21.15.82:0/2525738437 >> v2:172.21.15.196:3300/0 conn(0x7fada0099a70 0x7fada00cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.535+0000 7fad9ffff640 1 -- 172.21.15.82:0/2525738437 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada00d0610 con 0x7fada0161510 2024-04-25T10:03:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9ffff640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0161510 0x7fada01619a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fada0072410 tx=0x7fad9801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d68fa14787eed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9e7fc640 1 -- 172.21.15.82:0/2525738437 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980a5760 con 0x7fada0161510 2024-04-25T10:03:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9e7fc640 1 -- 172.21.15.82:0/2525738437 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad980a5900 con 0x7fada0161510 2024-04-25T10:03:08.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9e7fc640 1 -- 172.21.15.82:0/2525738437 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980a5cd0 con 0x7fada0161510 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- 172.21.15.82:0/2525738437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0161510 msgr2=0x7fada01619a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0161510 0x7fada01619a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fada0072410 tx=0x7fad9801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- 172.21.15.82:0/2525738437 shutdown_connections 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0161510 0x7fada01619a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- 172.21.15.82:0/2525738437 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099e50 0x7fada006c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- 172.21.15.82:0/2525738437 >> v2:172.21.15.196:3300/0 conn(0x7fada0099a70 0x7fada00cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- 172.21.15.82:0/2525738437 >> 172.21.15.82:0/2525738437 conn(0x7fada00b3100 msgr2=0x7fada00b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- 172.21.15.82:0/2525738437 shutdown_connections 2024-04-25T10:03:08.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- 172.21.15.82:0/2525738437 wait complete. 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 Processor -- start 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- start start 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099a70 0x7fada01424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 0x7fada00b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada00b2da0 con 0x7fada0099e50 2024-04-25T10:03:08.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada006cf20 con 0x7fada0161510 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fada6055640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada00b2f20 con 0x7fada0099a70 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52674/0 (socket says 172.21.15.82:52674) 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 -- 172.21.15.82:0/387679595 learned_addr learned my addr 172.21.15.82:0/387679595 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9ffff640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 0x7fada00b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9f7fe640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099a70 0x7fada01424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 -- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099a70 msgr2=0x7fada01424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099a70 0x7fada01424c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 -- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 msgr2=0x7fada00b6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 0x7fada00b6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9effd640 1 -- 172.21.15.82:0/387679595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada015cac0 con 0x7fada0099e50 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.539+0000 7fad9ffff640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 0x7fada00b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad9effd640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad9400c7f0 tx=0x7fad9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94015070 con 0x7fada0099e50 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad94004240 con 0x7fada0099e50 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada015cda0 con 0x7fada0099e50 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94003620 con 0x7fada0099e50 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada015d360 con 0x7fada0099e50 2024-04-25T10:03:08.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada004efa0 con 0x7fada0099e50 2024-04-25T10:03:08.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad940039d0 con 0x7fada0099e50 2024-04-25T10:03:08.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 0x7fad8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fad94085410 con 0x7fada0099e50 2024-04-25T10:03:08.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad9f7fe640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 0x7fad8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.543+0000 7fad9f7fe640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 0x7fad8c069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad900096f0 tx=0x7fad90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.547+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad94056090 con 0x7fada0099e50 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.703+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fada00b3cc0 con 0x7fada0099e50 2024-04-25T10:03:08.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fad87fff640 1 -- 172.21.15.82:0/387679595 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fad94055990 con 0x7fada0099e50 2024-04-25T10:03:08.714 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 msgr2=0x7fad8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 0x7fad8c069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fad900096f0 tx=0x7fad90008040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 msgr2=0x7fada015c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad9400c7f0 tx=0x7fad9400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 shutdown_connections 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad8c066f60 0x7fad8c069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fada0161510 0x7fada00b6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fada0099e50 0x7fada015c250 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 --2- 172.21.15.82:0/387679595 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fada0099a70 0x7fada01424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 >> 172.21.15.82:0/387679595 conn(0x7fada00b3100 msgr2=0x7fada015d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 shutdown_connections 2024-04-25T10:03:08.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:08.707+0000 7fada6055640 1 -- 172.21.15.82:0/387679595 wait complete. 2024-04-25T10:03:08.731 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-04-25T10:03:08.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.731 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T10:03:08.758 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:08.758 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:03:08.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:08.758 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:03:08.813 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:08.813 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.819 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:08.819 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:08.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 Processor -- start 2024-04-25T10:03:09.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 -- start start 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4480b3f50 0x7fc44815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480b2090 con 0x7fc44815a700 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4480b43e0 con 0x7fc4480b24a0 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.023+0000 7fc44e8df640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44815f200 con 0x7fc4480b3f50 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4480b3f50 0x7fc44815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4480b3f50 0x7fc44815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52678/0 (socket says 172.21.15.82:52678) 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc447fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc44815a700 0x7fc4480cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44822/0 (socket says 172.21.15.82:44822) 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 -- 172.21.15.82:0/3784427222 learned_addr learned my addr 172.21.15.82:0/3784427222 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 -- 172.21.15.82:0/3784427222 >> v1:172.21.15.196:6789/0 conn(0x7fc4480b24a0 legacy=0x7fc44806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc4477fe640 1 -- 172.21.15.82:0/3784427222 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4257510852 0 0) 0x7fc4480b2090 con 0x7fc44815a700 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 -- 172.21.15.82:0/3784427222 >> v1:172.21.15.82:6789/0 conn(0x7fc44815a700 legacy=0x7fc4480cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 -- 172.21.15.82:0/3784427222 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480b46c0 con 0x7fc4480b3f50 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc43ffff640 1 --2- 172.21.15.82:0/3784427222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4480b3f50 0x7fc44815ecc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc430009870 tx=0x7fc43001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd82c9b5b5094031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc4477fe640 1 -- 172.21.15.82:0/3784427222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300ab610 con 0x7fc4480b3f50 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc4477fe640 1 -- 172.21.15.82:0/3784427222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300b0070 con 0x7fc4480b3f50 2024-04-25T10:03:09.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc4477fe640 1 -- 172.21.15.82:0/3784427222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300ab9d0 con 0x7fc4480b3f50 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 -- 172.21.15.82:0/3784427222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4480b3f50 msgr2=0x7fc44815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3784427222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4480b3f50 0x7fc44815ecc0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc430009870 tx=0x7fc43001c410 comp rx=0 tx=0).stop 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 -- 172.21.15.82:0/3784427222 shutdown_connections 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3784427222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4480b3f50 0x7fc44815ecc0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 -- 172.21.15.82:0/3784427222 >> 172.21.15.82:0/3784427222 conn(0x7fc4480b11a0 msgr2=0x7fc4480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 -- 172.21.15.82:0/3784427222 shutdown_connections 2024-04-25T10:03:09.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.027+0000 7fc44e8df640 1 -- 172.21.15.82:0/3784427222 wait complete. 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 Processor -- start 2024-04-25T10:03:09.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- start start 2024-04-25T10:03:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4480b24a0 0x7fc448140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 0x7fc44806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480cd310 con 0x7fc44815a700 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480ce600 con 0x7fc448159ea0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480cd490 con 0x7fc4480b24a0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52690/0 (socket says 172.21.15.82:52690) 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 -- 172.21.15.82:0/3966211853 learned_addr learned my addr 172.21.15.82:0/3966211853 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc447fff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4480b24a0 0x7fc448140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 -- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4480b24a0 msgr2=0x7fc448140fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4480b24a0 0x7fc448140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44ce7c640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 0x7fc44806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 -- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 msgr2=0x7fc44806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 0x7fc44806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 -- 172.21.15.82:0/3966211853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44815cd00 con 0x7fc44815a700 2024-04-25T10:03:09.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44ce7c640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 0x7fc44806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc43ffff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc4300ad040 tx=0x7fc43001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300b4600 con 0x7fc44815a700 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4300b47c0 con 0x7fc44815a700 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4480a8570 con 0x7fc44815a700 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300b4bb0 con 0x7fc44815a700 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc448158390 con 0x7fc44815a700 2024-04-25T10:03:09.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44804efa0 con 0x7fc44815a700 2024-04-25T10:03:09.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.031+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4300c7440 con 0x7fc44815a700 2024-04-25T10:03:09.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.035+0000 7fc4457fa640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.035+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc43011fdc0 con 0x7fc44815a700 2024-04-25T10:03:09.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.035+0000 7fc447fff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.035+0000 7fc447fff640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 0x7fc428069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc438000f70 tx=0x7fc438009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.039+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4300a8050 con 0x7fc44815a700 2024-04-25T10:03:09.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.199+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc44815d6c0 con 0x7fc44815a700 2024-04-25T10:03:09.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.199+0000 7fc4457fa640 1 -- 172.21.15.82:0/3966211853 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fc430016150 con 0x7fc44815a700 2024-04-25T10:03:09.209 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:09.209 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:09.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 msgr2=0x7fc428069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 0x7fc428069420 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fc438000f70 tx=0x7fc438009040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 msgr2=0x7fc44815c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc4300ad040 tx=0x7fc43001ac50 comp rx=0 tx=0).stop 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 shutdown_connections 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc428066f60 0x7fc428069420 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc448159ea0 0x7fc44806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc44815a700 0x7fc44815c7c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 --2- 172.21.15.82:0/3966211853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4480b24a0 0x7fc448140fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 >> 172.21.15.82:0/3966211853 conn(0x7fc4480b11a0 msgr2=0x7fc4480afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 shutdown_connections 2024-04-25T10:03:09.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.203+0000 7fc44e8df640 1 -- 172.21.15.82:0/3966211853 wait complete. 2024-04-25T10:03:09.228 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:09.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:09.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 Processor -- start 2024-04-25T10:03:09.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 -- start start 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f664806d090 0x7f664806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664806bff0 con 0x7f66480b5270 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f664806c2d0 con 0x7f66480b0380 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664ed7e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f664806cc00 con 0x7f664806d090 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664cb1a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f66480b5270 0x7f66480b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44834/0 (socket says 172.21.15.82:44834) 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664cb1a640 1 -- 172.21.15.82:0/3846481970 learned_addr learned my addr 172.21.15.82:0/3846481970 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:09.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 --2- 172.21.15.82:0/3846481970 >> v2:172.21.15.196:3300/0 conn(0x7f664806d090 0x7f664806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245468684 0 0) 0x7f664806bff0 con 0x7f66480b5270 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6630003530 con 0x7f66480b5270 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166279583 0 0) 0x7f6630003530 con 0x7f66480b5270 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 >> v2:172.21.15.196:3300/0 conn(0x7f664806d090 msgr2=0x7f664806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 --2- 172.21.15.82:0/3846481970 >> v2:172.21.15.196:3300/0 conn(0x7f664806d090 0x7f664806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 >> v1:172.21.15.196:6789/0 conn(0x7f66480b0380 legacy=0x7f66480d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664806c5b0 con 0x7f66480b5270 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f663c001800 con 0x7f66480b5270 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 >> v1:172.21.15.82:6789/0 conn(0x7f66480b5270 legacy=0x7f66480b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630003a80 0x7f6630003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 0x7f663000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664806cc00 con 0x7f6630005220 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663000c150 con 0x7f6630004810 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663000c350 con 0x7f6630003a80 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 0x7f663000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f6647fff640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 -- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630003a80 msgr2=0x7f6630003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630003a80 0x7f6630003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 -- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6630004810 msgr2=0x7f6630004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6630004810 0x7f6630004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.347+0000 7f664d31b640 1 -- 172.21.15.82:0/3846481970 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630003530 con 0x7f6630005220 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664d31b640 1 -- 172.21.15.82:0/3846481970 reap_dead start 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664d31b640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 0x7f663000b9d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6640009fc0 tx=0x7f664001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca363658b17e5efd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640009c10 con 0x7f6630005220 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66400a5d10 con 0x7f6630005220 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f66477fe640 1 -- 172.21.15.82:0/3846481970 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640003be0 con 0x7f6630005220 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 msgr2=0x7f663000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 0x7f663000b9d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6640009fc0 tx=0x7f664001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- 172.21.15.82:0/3846481970 shutdown_connections 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 --2- 172.21.15.82:0/3846481970 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6630005220 0x7f663000b9d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- 172.21.15.82:0/3846481970 >> 172.21.15.82:0/3846481970 conn(0x7f664808b000 msgr2=0x7f66480d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- 172.21.15.82:0/3846481970 shutdown_connections 2024-04-25T10:03:09.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- 172.21.15.82:0/3846481970 wait complete. 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 Processor -- start 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- start start 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630005220 0x7f66480b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 0x7f66480b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664815de20 con 0x7f66480b7e90 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66480d1760 con 0x7f6648159700 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.351+0000 7f664ed7e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664815dfa0 con 0x7f6630005220 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52704/0 (socket says 172.21.15.82:52704) 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 -- 172.21.15.82:0/2822473526 learned_addr learned my addr 172.21.15.82:0/2822473526 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664cb1a640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630005220 0x7f66480b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f6647fff640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 0x7f66480b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 -- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630005220 msgr2=0x7f66480b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630005220 0x7f66480b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 -- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 msgr2=0x7f66480b7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 0x7f66480b7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 -- 172.21.15.82:0/2822473526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480b61f0 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f6647fff640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 0x7f66480b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664d31b640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f664001d1a0 tx=0x7f6640009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66400aed80 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66400bd070 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66480b8310 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640003ba0 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648159b80 con 0x7f66480b7e90 2024-04-25T10:03:09.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f664804efa0 con 0x7f66480b7e90 2024-04-25T10:03:09.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66400a5d10 con 0x7f66480b7e90 2024-04-25T10:03:09.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 0x7f6634069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6640121a90 con 0x7f66480b7e90 2024-04-25T10:03:09.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664cb1a640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 0x7f6634069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.355+0000 7f664cb1a640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 0x7f6634069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f663c001ec0 tx=0x7f663c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.359+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f66400c7020 con 0x7f66480b7e90 2024-04-25T10:03:09.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.523+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f664806dd90 con 0x7f66480b7e90 2024-04-25T10:03:09.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.523+0000 7f66457fa640 1 -- 172.21.15.82:0/2822473526 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66400169d0 con 0x7f66480b7e90 2024-04-25T10:03:09.532 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 msgr2=0x7f6634069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 0x7f6634069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f663c001ec0 tx=0x7f663c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 msgr2=0x7f66480b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f664001d1a0 tx=0x7f6640009750 comp rx=0 tx=0).stop 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 shutdown_connections 2024-04-25T10:03:09.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6634066f60 0x7f6634069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66480b7e90 0x7f66480b21c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6648159700 0x7f66480b7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 --2- 172.21.15.82:0/2822473526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6630005220 0x7f66480b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 >> 172.21.15.82:0/2822473526 conn(0x7f664808b000 msgr2=0x7f66480b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 shutdown_connections 2024-04-25T10:03:09.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.527+0000 7f664ed7e640 1 -- 172.21.15.82:0/2822473526 wait complete. 2024-04-25T10:03:09.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:09.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 Processor -- start 2024-04-25T10:03:09.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 -- start start 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6f9015a750 0x7f6f9006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f90066bb0 0x7f6f9006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9006c1e0 0x7f6f9015e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f90067100 con 0x7f6f90066bb0 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f9015ab30 con 0x7f6f9015a750 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.671+0000 7f6f9739b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900cf560 con 0x7f6f9006c1e0 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f90066bb0 0x7f6f9006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6f90066bb0 0x7f6f9006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52714/0 (socket says 172.21.15.82:52714) 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 -- 172.21.15.82:0/1475601133 learned_addr learned my addr 172.21.15.82:0/1475601133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f95938640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9006c1e0 0x7f6f9015e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f95137640 1 --2- 172.21.15.82:0/1475601133 >> v2:172.21.15.196:3300/0 conn(0x7f6f9015a750 0x7f6f9006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 -- 172.21.15.82:0/1475601133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9006c1e0 msgr2=0x7f6f9015e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9006c1e0 0x7f6f9015e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 -- 172.21.15.82:0/1475601133 >> v2:172.21.15.196:3300/0 conn(0x7f6f9015a750 msgr2=0x7f6f9006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 --2- 172.21.15.82:0/1475601133 >> v2:172.21.15.196:3300/0 conn(0x7f6f9015a750 0x7f6f9006ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 -- 172.21.15.82:0/1475601133 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f900cf840 con 0x7f6f90066bb0 2024-04-25T10:03:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f94936640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90066bb0 0x7f6f9006bca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6f900713a0 tx=0x7f6f8001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d63f7dd46bc6ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f87fff640 1 -- 172.21.15.82:0/1475601133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800b1070 con 0x7f6f90066bb0 2024-04-25T10:03:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f87fff640 1 -- 172.21.15.82:0/1475601133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f800ac440 con 0x7f6f90066bb0 2024-04-25T10:03:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f87fff640 1 -- 172.21.15.82:0/1475601133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800ac810 con 0x7f6f90066bb0 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 -- 172.21.15.82:0/1475601133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90066bb0 msgr2=0x7f6f9006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90066bb0 0x7f6f9006bca0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6f900713a0 tx=0x7f6f8001c140 comp rx=0 tx=0).stop 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 -- 172.21.15.82:0/1475601133 shutdown_connections 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9006c1e0 0x7f6f9015e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 --2- 172.21.15.82:0/1475601133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f90066bb0 0x7f6f9006bca0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 --2- 172.21.15.82:0/1475601133 >> v2:172.21.15.196:3300/0 conn(0x7f6f9015a750 0x7f6f9006ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 -- 172.21.15.82:0/1475601133 >> 172.21.15.82:0/1475601133 conn(0x7f6f90071810 msgr2=0x7f6f90071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 -- 172.21.15.82:0/1475601133 shutdown_connections 2024-04-25T10:03:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.675+0000 7f6f9739b640 1 -- 172.21.15.82:0/1475601133 wait complete. 2024-04-25T10:03:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 Processor -- start 2024-04-25T10:03:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- start start 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f90066bb0 0x7f6f90158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9015a750 0x7f6f901f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900b3000 con 0x7f6f9006c1e0 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900b2e80 con 0x7f6f90066bb0 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f900b3180 con 0x7f6f9015a750 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52724/0 (socket says 172.21.15.82:52724) 2024-04-25T10:03:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 -- 172.21.15.82:0/2095445519 learned_addr learned my addr 172.21.15.82:0/2095445519 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f95137640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f90066bb0 0x7f6f90158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f95938640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9015a750 0x7f6f901f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 -- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9015a750 msgr2=0x7f6f901f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9015a750 0x7f6f901f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 -- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f90066bb0 msgr2=0x7f6f90158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f90066bb0 0x7f6f90158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 -- 172.21.15.82:0/2095445519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f90153950 con 0x7f6f9006c1e0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f94936640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f8001a7f0 tx=0x7f6f80018ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800bc070 con 0x7f6f9006c1e0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f800aa750 con 0x7f6f9006c1e0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f90153c30 con 0x7f6f9006c1e0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f800b1070 con 0x7f6f9006c1e0 2024-04-25T10:03:09.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f901541f0 con 0x7f6f9006c1e0 2024-04-25T10:03:09.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f9004efa0 con 0x7f6f9006c1e0 2024-04-25T10:03:09.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.679+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6f800bf070 con 0x7f6f9006c1e0 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.683+0000 7f6f85ffb640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 0x7f6f74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.683+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6f8011f060 con 0x7f6f9006c1e0 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.683+0000 7f6f95137640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 0x7f6f74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.683+0000 7f6f95137640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 0x7f6f74069450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ce080 tx=0x7f6f8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.687+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6f80016110 con 0x7f6f9006c1e0 2024-04-25T10:03:09.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.839+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f9006c7d0 con 0x7f6f9006c1e0 2024-04-25T10:03:09.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.839+0000 7f6f85ffb640 1 -- 172.21.15.82:0/2095445519 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f800efce0 con 0x7f6f9006c1e0 2024-04-25T10:03:09.849 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 msgr2=0x7f6f74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 0x7f6f74069450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ce080 tx=0x7f6f8c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 msgr2=0x7f6f901f7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6f8001a7f0 tx=0x7f6f80018ee0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 shutdown_connections 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6f74066f90 0x7f6f74069450 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6f9015a750 0x7f6f901f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6f9006c1e0 0x7f6f901f7b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 --2- 172.21.15.82:0/2095445519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6f90066bb0 0x7f6f90158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 >> 172.21.15.82:0/2095445519 conn(0x7f6f90071810 msgr2=0x7f6f900b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 shutdown_connections 2024-04-25T10:03:09.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.843+0000 7f6f9739b640 1 -- 172.21.15.82:0/2095445519 wait complete. 2024-04-25T10:03:09.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:09.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 Processor -- start 2024-04-25T10:03:09.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 -- start start 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 0x7fc5f40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5f4073010 0x7fc5f40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f40cbd20 con 0x7fc5f40cf590 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f415ba90 con 0x7fc5f4073010 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fbadf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f40cc000 con 0x7fc5f415b6b0 2024-04-25T10:03:09.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5f4073010 0x7fc5f40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5f907a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc5f40cf590 0x7fc5f415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44850/0 (socket says 172.21.15.82:44850) 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5f907a640 1 -- 172.21.15.82:0/4280680251 learned_addr learned my addr 172.21.15.82:0/4280680251 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5f987b640 1 --2- 172.21.15.82:0/4280680251 >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 0x7fc5f40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 -- 172.21.15.82:0/4280680251 >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 msgr2=0x7fc5f40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 --2- 172.21.15.82:0/4280680251 >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 0x7fc5f40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 -- 172.21.15.82:0/4280680251 >> v1:172.21.15.82:6789/0 conn(0x7fc5f40cf590 legacy=0x7fc5f415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5f8879640 1 -- 172.21.15.82:0/4280680251 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149271579 0 0) 0x7fc5f40cbd20 con 0x7fc5f40cf590 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 -- 172.21.15.82:0/4280680251 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f40cc2e0 con 0x7fc5f4073010 2024-04-25T10:03:09.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5f987b640 1 --2- 172.21.15.82:0/4280680251 >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 0x7fc5f40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.971+0000 7fc5fa07c640 1 --2- 172.21.15.82:0/4280680251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f40cfa20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40d0180 tx=0x7fc5ec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c4f50a5c54e946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f8879640 1 -- 172.21.15.82:0/4280680251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0b2070 con 0x7fc5f4073010 2024-04-25T10:03:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f8879640 1 -- 172.21.15.82:0/4280680251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec0a5800 con 0x7fc5f4073010 2024-04-25T10:03:09.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f8879640 1 -- 172.21.15.82:0/4280680251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0a5bd0 con 0x7fc5f4073010 2024-04-25T10:03:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/4280680251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 msgr2=0x7fc5f40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/4280680251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f40cfa20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40d0180 tx=0x7fc5ec01c930 comp rx=0 tx=0).stop 2024-04-25T10:03:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/4280680251 shutdown_connections 2024-04-25T10:03:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/4280680251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f40cfa20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/4280680251 >> v2:172.21.15.196:3300/0 conn(0x7fc5f415b6b0 0x7fc5f40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/4280680251 >> 172.21.15.82:0/4280680251 conn(0x7fc5f406ae60 msgr2=0x7fc5f406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/4280680251 shutdown_connections 2024-04-25T10:03:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/4280680251 wait complete. 2024-04-25T10:03:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 Processor -- start 2024-04-25T10:03:09.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- start start 2024-04-25T10:03:09.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 0x7fc5f40989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f415b6b0 0x7fc5f406e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40cadb0 con 0x7fc5f4073010 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40ca900 con 0x7fc5f40cf590 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40caf30 con 0x7fc5f415b6b0 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52750/0 (socket says 172.21.15.82:52750) 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 -- 172.21.15.82:0/2632660610 learned_addr learned my addr 172.21.15.82:0/2632660610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:09.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f907a640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 0x7fc5f40989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fa07c640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f415b6b0 0x7fc5f406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 -- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f415b6b0 msgr2=0x7fc5f406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f415b6b0 0x7fc5f406e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 -- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 msgr2=0x7fc5f40989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 0x7fc5f40989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 -- 172.21.15.82:0/2632660610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f4098ef0 con 0x7fc5f4073010 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f907a640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 0x7fc5f40989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5f987b640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c820 tx=0x7fc5f000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0015070 con 0x7fc5f4073010 2024-04-25T10:03:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f0007a30 con 0x7fc5f4073010 2024-04-25T10:03:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f40b3620 con 0x7fc5f4073010 2024-04-25T10:03:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0007e40 con 0x7fc5f4073010 2024-04-25T10:03:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.975+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f406d760 con 0x7fc5f4073010 2024-04-25T10:03:09.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f404efa0 con 0x7fc5f4073010 2024-04-25T10:03:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5f0018070 con 0x7fc5f4073010 2024-04-25T10:03:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5ea7fc640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 0x7fc5d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5f907a640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 0x7fc5d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc5f0085310 con 0x7fc5f4073010 2024-04-25T10:03:09.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.979+0000 7fc5f907a640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 0x7fc5d8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40b1f20 tx=0x7fc5e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:09.983+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5f0055ef0 con 0x7fc5f4073010 2024-04-25T10:03:10.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.131+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5f4158a70 con 0x7fc5f4073010 2024-04-25T10:03:10.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.131+0000 7fc5ea7fc640 1 -- 172.21.15.82:0/2632660610 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc5f00557f0 con 0x7fc5f4073010 2024-04-25T10:03:10.139 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 msgr2=0x7fc5d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 0x7fc5d8069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40b1f20 tx=0x7fc5e4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 msgr2=0x7fc5f4155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc5f000c820 tx=0x7fc5f000cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 shutdown_connections 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5d8066f60 0x7fc5d8069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f415b6b0 0x7fc5f406e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f40cf590 0x7fc5f40989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 --2- 172.21.15.82:0/2632660610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f4073010 0x7fc5f4155f20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 >> 172.21.15.82:0/2632660610 conn(0x7fc5f406ae60 msgr2=0x7fc5f40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 shutdown_connections 2024-04-25T10:03:10.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.135+0000 7fc5fbadf640 1 -- 172.21.15.82:0/2632660610 wait complete. 2024-04-25T10:03:10.158 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:10.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 Processor -- start 2024-04-25T10:03:10.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 -- start start 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e5c0cf590 0x7f2e5c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0cbd20 con 0x7f2e5c15b6b0 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0cc000 con 0x7f2e5c072000 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e60b68640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c15ba90 con 0x7f2e5c0cf590 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e59d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e5c0cf590 0x7f2e5c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5a59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2e5c15b6b0 0x7f2e5c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44854/0 (socket says 172.21.15.82:44854) 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5a59c640 1 -- 172.21.15.82:0/1962878295 learned_addr learned my addr 172.21.15.82:0/1962878295 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e59d9b640 1 -- 172.21.15.82:0/1962878295 >> v1:172.21.15.196:6789/0 conn(0x7f2e5c072000 legacy=0x7f2e5c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e59d9b640 1 -- 172.21.15.82:0/1962878295 >> v1:172.21.15.82:6789/0 conn(0x7f2e5c15b6b0 legacy=0x7f2e5c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e59d9b640 1 -- 172.21.15.82:0/1962878295 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0cc2e0 con 0x7f2e5c0cf590 2024-04-25T10:03:10.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5959a640 1 -- 172.21.15.82:0/1962878295 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712529868 0 0) 0x7f2e5c0cbd20 con 0x7f2e5c15b6b0 2024-04-25T10:03:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e59d9b640 1 --2- 172.21.15.82:0/1962878295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c0cfa20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e500097c0 tx=0x7f2e5001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d67d4dda21ddf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5959a640 1 -- 172.21.15.82:0/1962878295 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b0070 con 0x7f2e5c0cf590 2024-04-25T10:03:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5959a640 1 -- 172.21.15.82:0/1962878295 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e50004230 con 0x7f2e5c0cf590 2024-04-25T10:03:10.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.271+0000 7f2e5959a640 1 -- 172.21.15.82:0/1962878295 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003620 con 0x7f2e5c0cf590 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/1962878295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 msgr2=0x7f2e5c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 --2- 172.21.15.82:0/1962878295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c0cfa20 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2e500097c0 tx=0x7f2e5001c460 comp rx=0 tx=0).stop 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/1962878295 shutdown_connections 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 --2- 172.21.15.82:0/1962878295 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c0cfa20 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/1962878295 >> 172.21.15.82:0/1962878295 conn(0x7f2e5c06ae60 msgr2=0x7f2e5c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/1962878295 shutdown_connections 2024-04-25T10:03:10.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/1962878295 wait complete. 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 Processor -- start 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- start start 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 0x7f2e5c152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e5c15b6b0 0x7f2e5c1f8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0caf30 con 0x7f2e5c0cf590 2024-04-25T10:03:10.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0cadb0 con 0x7f2e5c072000 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1692b0 con 0x7f2e5c15b6b0 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52774/0 (socket says 172.21.15.82:52774) 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 -- 172.21.15.82:0/2124893303 learned_addr learned my addr 172.21.15.82:0/2124893303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 -- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e5c15b6b0 msgr2=0x7f2e5c1f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e5a59c640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 0x7f2e5c152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e5ad9d640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e5c15b6b0 0x7f2e5c1f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e5c15b6b0 0x7f2e5c1f8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 -- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 msgr2=0x7f2e5c152780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 0x7f2e5c152780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 -- 172.21.15.82:0/2124893303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c15def0 con 0x7f2e5c0cf590 2024-04-25T10:03:10.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e5a59c640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 0x7f2e5c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e59d9b640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ae040 tx=0x7f2e50019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e500b0040 con 0x7f2e5c0cf590 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500bc070 con 0x7f2e5c0cf590 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c15e1d0 con 0x7f2e5c0cf590 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50003e60 con 0x7f2e5c0cf590 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c0b0750 con 0x7f2e5c0cf590 2024-04-25T10:03:10.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.275+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c04efa0 con 0x7f2e5c0cf590 2024-04-25T10:03:10.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.279+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e500b3070 con 0x7f2e5c0cf590 2024-04-25T10:03:10.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.279+0000 7f2e4affd640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 0x7f2e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.279+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2e5011f340 con 0x7f2e5c0cf590 2024-04-25T10:03:10.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.279+0000 7f2e5a59c640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 0x7f2e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.279+0000 7f2e5a59c640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 0x7f2e3c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.283+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e50016050 con 0x7f2e5c0cf590 2024-04-25T10:03:10.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.431+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2e5c06e660 con 0x7f2e5c0cf590 2024-04-25T10:03:10.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.431+0000 7f2e4affd640 1 -- 172.21.15.82:0/2124893303 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2e500eff30 con 0x7f2e5c0cf590 2024-04-25T10:03:10.439 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 msgr2=0x7f2e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 0x7f2e3c069420 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c007fd0 tx=0x7f2e4c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 msgr2=0x7f2e5c1f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2e500ae040 tx=0x7f2e50019200 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 shutdown_connections 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e3c066f60 0x7f2e3c069420 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e5c15b6b0 0x7f2e5c1f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e5c0cf590 0x7f2e5c1f7c50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 --2- 172.21.15.82:0/2124893303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e5c072000 0x7f2e5c152780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 >> 172.21.15.82:0/2124893303 conn(0x7f2e5c06ae60 msgr2=0x7f2e5c14fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 shutdown_connections 2024-04-25T10:03:10.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.435+0000 7f2e60b68640 1 -- 172.21.15.82:0/2124893303 wait complete. 2024-04-25T10:03:10.457 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 Processor -- start 2024-04-25T10:03:10.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 -- start start 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 0x7f2e9c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e9c144ca0 0x7f2e9c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c06ec60 0x7f2e9c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cc2b0 con 0x7f2e9c144ca0 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e9c06e290 con 0x7f2e9c0ce510 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2ea1e4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cee30 con 0x7f2e9c06ec60 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e9c144ca0 0x7f2e9c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2e9c144ca0 0x7f2e9c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52780/0 (socket says 172.21.15.82:52780) 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 -- 172.21.15.82:0/1385152559 learned_addr learned my addr 172.21.15.82:0/1385152559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9b7fe640 1 --2- 172.21.15.82:0/1385152559 >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 0x7f2e9c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 -- 172.21.15.82:0/1385152559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c06ec60 msgr2=0x7f2e9c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9bfff640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c06ec60 0x7f2e9c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c06ec60 0x7f2e9c142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 -- 172.21.15.82:0/1385152559 >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 msgr2=0x7f2e9c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 --2- 172.21.15.82:0/1385152559 >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 0x7f2e9c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9affd640 1 -- 172.21.15.82:0/1385152559 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0ccc70 con 0x7f2e9c144ca0 2024-04-25T10:03:10.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.563+0000 7f2e9b7fe640 1 --2- 172.21.15.82:0/1385152559 >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 0x7f2e9c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2e9affd640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c06e720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c144c50 tx=0x7f2e8c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb9e2a5aab8934f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2e9a7fc640 1 -- 172.21.15.82:0/1385152559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0b5070 con 0x7f2e9c144ca0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2e9a7fc640 1 -- 172.21.15.82:0/1385152559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e8c0a89d0 con 0x7f2e9c144ca0 2024-04-25T10:03:10.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2e9a7fc640 1 -- 172.21.15.82:0/1385152559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0a8da0 con 0x7f2e9c144ca0 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/1385152559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c144ca0 msgr2=0x7f2e9c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c06e720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c144c50 tx=0x7f2e8c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/1385152559 shutdown_connections 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c06ec60 0x7f2e9c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/1385152559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c06e720 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/1385152559 >> v2:172.21.15.196:3300/0 conn(0x7f2e9c0ce510 0x7f2e9c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/1385152559 >> 172.21.15.82:0/1385152559 conn(0x7f2e9c0b11a0 msgr2=0x7f2e9c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/1385152559 shutdown_connections 2024-04-25T10:03:10.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/1385152559 wait complete. 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 Processor -- start 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.567+0000 7f2ea1e4b640 1 -- start start 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c0ce510 0x7f2e9c143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c1e5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c0cc900 con 0x7f2e9c06ec60 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c06df20 con 0x7f2e9c144ca0 2024-04-25T10:03:10.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9c1451f0 con 0x7f2e9c0ce510 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52786/0 (socket says 172.21.15.82:52786) 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 -- 172.21.15.82:0/2263437598 learned_addr learned my addr 172.21.15.82:0/2263437598 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9bfff640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c1e5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 -- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c0ce510 msgr2=0x7f2e9c143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c0ce510 0x7f2e9c143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 -- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e9c144ca0 msgr2=0x7f2e9c1e5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c1e5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 -- 172.21.15.82:0/2263437598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e9c0a84d0 con 0x7f2e9c06ec60 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9b7fe640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e810 tx=0x7f2e9000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e90017070 con 0x7f2e9c06ec60 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9000c8d0 con 0x7f2e9c06ec60 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e9c0a87b0 con 0x7f2e9c06ec60 2024-04-25T10:03:10.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e9000f040 con 0x7f2e9c06ec60 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e9c1e5360 con 0x7f2e9c06ec60 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9c04efa0 con 0x7f2e9c06ec60 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e9001a070 con 0x7f2e9c06ec60 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2e90085070 con 0x7f2e9c06ec60 2024-04-25T10:03:10.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9affd640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.571+0000 7f2e9affd640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 0x7f2e88069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0b2000 tx=0x7f2e8c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.575+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e90055c60 con 0x7f2e9c06ec60 2024-04-25T10:03:10.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2e9c0b1cc0 con 0x7f2e9c06ec60 2024-04-25T10:03:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2e83fff640 1 -- 172.21.15.82:0/2263437598 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2e90055560 con 0x7f2e9c06ec60 2024-04-25T10:03:10.755 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 msgr2=0x7f2e88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 0x7f2e88069420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0b2000 tx=0x7f2e8c004740 comp rx=0 tx=0).stop 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 msgr2=0x7f2e9c1e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000e810 tx=0x7f2e9000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 shutdown_connections 2024-04-25T10:03:10.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e88066f60 0x7f2e88069420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e9c0ce510 0x7f2e9c143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e9c144ca0 0x7f2e9c1e5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 --2- 172.21.15.82:0/2263437598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e9c06ec60 0x7f2e9c1e7480 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 >> 172.21.15.82:0/2263437598 conn(0x7f2e9c0b11a0 msgr2=0x7f2e9c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 shutdown_connections 2024-04-25T10:03:10.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.747+0000 7f2ea1e4b640 1 -- 172.21.15.82:0/2263437598 wait complete. 2024-04-25T10:03:10.774 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:10.774 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.779 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:10.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.923+0000 7f46919a5640 1 Processor -- start 2024-04-25T10:03:10.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.923+0000 7f46919a5640 1 -- start start 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f46919a5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f468c0d07f0 0x7f468c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f46919a5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 0x7f468c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f46919a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c06be60 con 0x7f468c15da80 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f46919a5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c06c440 con 0x7f468c0d07f0 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f46919a5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f468c15de60 con 0x7f468c0d1cd0 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f468c0d07f0 0x7f468c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f468c0d07f0 0x7f468c0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52794/0 (socket says 172.21.15.82:52794) 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 -- 172.21.15.82:0/3026477887 learned_addr learned my addr 172.21.15.82:0/3026477887 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:10.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 -- 172.21.15.82:0/3026477887 >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 msgr2=0x7f468c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468b7fe640 1 --2- 172.21.15.82:0/3026477887 >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 0x7f468c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 --2- 172.21.15.82:0/3026477887 >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 0x7f468c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 -- 172.21.15.82:0/3026477887 >> v1:172.21.15.196:6789/0 conn(0x7f468c15da80 legacy=0x7f468c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 -- 172.21.15.82:0/3026477887 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0d2450 con 0x7f468c0d07f0 2024-04-25T10:03:10.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468b7fe640 1 --2- 172.21.15.82:0/3026477887 >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 0x7f468c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:10.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f468a7fc640 1 --2- 172.21.15.82:0/3026477887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c0d4230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f468c06de30 tx=0x7f468001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fa719f6c8b287b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f4689ffb640 1 -- 172.21.15.82:0/3026477887 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800b1070 con 0x7f468c0d07f0 2024-04-25T10:03:10.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f4689ffb640 1 -- 172.21.15.82:0/3026477887 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46800a8050 con 0x7f468c0d07f0 2024-04-25T10:03:10.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.927+0000 7f4689ffb640 1 -- 172.21.15.82:0/3026477887 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46800ac860 con 0x7f468c0d07f0 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3026477887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 msgr2=0x7f468c0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- 172.21.15.82:0/3026477887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c0d4230 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f468c06de30 tx=0x7f468001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3026477887 shutdown_connections 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- 172.21.15.82:0/3026477887 >> v2:172.21.15.196:3300/0 conn(0x7f468c0d1cd0 0x7f468c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- 172.21.15.82:0/3026477887 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c0d4230 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3026477887 >> 172.21.15.82:0/3026477887 conn(0x7f468c08b000 msgr2=0x7f468c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3026477887 shutdown_connections 2024-04-25T10:03:10.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3026477887 wait complete. 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 Processor -- start 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- start start 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f468c0d1cd0 0x7f468c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f468c15da80 0x7f468c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06d7a0 con 0x7f468c0d07f0 2024-04-25T10:03:10.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06d620 con 0x7f468c0d1cd0 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c06d920 con 0x7f468c15da80 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52796/0 (socket says 172.21.15.82:52796) 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 -- 172.21.15.82:0/3936537700 learned_addr learned my addr 172.21.15.82:0/3936537700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468a7fc640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f468c0d1cd0 0x7f468c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 -- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f468c15da80 msgr2=0x7f468c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f468c15da80 0x7f468c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 -- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f468c0d1cd0 msgr2=0x7f468c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f468c0d1cd0 0x7f468c158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 -- 172.21.15.82:0/3936537700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c15ab60 con 0x7f468c0d07f0 2024-04-25T10:03:10.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f468affd640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b440 tx=0x7f467c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c015020 con 0x7f468c0d07f0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c011890 con 0x7f468c0d07f0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c15ae40 con 0x7f468c0d07f0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c011c80 con 0x7f468c0d07f0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c15b400 con 0x7f468c0d07f0 2024-04-25T10:03:10.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.931+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c04efa0 con 0x7f468c0d07f0 2024-04-25T10:03:10.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.935+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f467c018020 con 0x7f468c0d07f0 2024-04-25T10:03:10.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.935+0000 7f4673fff640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 0x7f4678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.935+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f467c0852e0 con 0x7f468c0d07f0 2024-04-25T10:03:10.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.935+0000 7f468a7fc640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 0x7f4678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.935+0000 7f468a7fc640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 0x7f4678069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46800ae000 tx=0x7f46800a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:10.939+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f467c01f030 con 0x7f468c0d07f0 2024-04-25T10:03:11.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.095+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f468c160e70 con 0x7f468c0d07f0 2024-04-25T10:03:11.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.095+0000 7f4673fff640 1 -- 172.21.15.82:0/3936537700 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f467c00d040 con 0x7f468c0d07f0 2024-04-25T10:03:11.104 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:11.104 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 msgr2=0x7f4678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 0x7f4678069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46800ae000 tx=0x7f46800a7000 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 msgr2=0x7f468c1559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b440 tx=0x7f467c00b900 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 shutdown_connections 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4678066f60 0x7f4678069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f468c15da80 0x7f468c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f468c0d1cd0 0x7f468c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 --2- 172.21.15.82:0/3936537700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f468c0d07f0 0x7f468c1559f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 >> 172.21.15.82:0/3936537700 conn(0x7f468c08b000 msgr2=0x7f468c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 shutdown_connections 2024-04-25T10:03:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.099+0000 7f46919a5640 1 -- 172.21.15.82:0/3936537700 wait complete. 2024-04-25T10:03:11.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:11.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 Processor -- start 2024-04-25T10:03:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 -- start start 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe90c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe90c0cfb80 0x7fe90c0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c15bd30 con 0x7fe90c0cf420 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c0cc0d0 con 0x7fe90c0cfb80 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe912aee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c09ab10 con 0x7fe90c15b950 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe90c0cf420 0x7fe90c0cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44862/0 (socket says 172.21.15.82:44862) 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90bfff640 1 -- 172.21.15.82:0/63074476 learned_addr learned my addr 172.21.15.82:0/63074476 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91088a640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe90c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 --2- 172.21.15.82:0/63074476 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0cfb80 0x7fe90c0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628883410 0 0) 0x7fe90c15bd30 con 0x7fe90c0cf420 2024-04-25T10:03:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8fc003530 con 0x7fe90c0cf420 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1949367069 0 0) 0x7fe8fc003530 con 0x7fe90c0cf420 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 msgr2=0x7fe90c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe90c09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0cfb80 msgr2=0x7fe90c0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 --2- 172.21.15.82:0/63074476 >> v2:172.21.15.196:3300/0 conn(0x7fe90c0cfb80 0x7fe90c0d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0cc370 con 0x7fe90c0cf420 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe8f8001680 con 0x7fe90c0cf420 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 >> v1:172.21.15.82:6789/0 conn(0x7fe90c0cf420 legacy=0x7fe90c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc003a30 0x7fe8fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe8fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc004d50 0x7fe8fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cc0d0 con 0x7fe8fc004d50 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c09ab10 con 0x7fe8fc003a30 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8fc00b9d0 con 0x7fe90c15b950 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc004d50 0x7fe8fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91088a640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc003a30 0x7fe8fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe90bfff640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe8fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 -- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 msgr2=0x7fe8fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c15b950 0x7fe8fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 -- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc003a30 msgr2=0x7fe8fc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc003a30 0x7fe8fc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91108b640 1 -- 172.21.15.82:0/63074476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc003530 con 0x7fe8fc004d50 2024-04-25T10:03:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.231+0000 7fe91088a640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8fc003a30 0x7fe8fc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe91108b640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc004d50 0x7fe8fc0051d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe904009fc0 tx=0x7fe90401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b184e5ee522fa8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040b2070 con 0x7fe8fc004d50 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9040a5cd0 con 0x7fe8fc004d50 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90b7fe640 1 -- 172.21.15.82:0/63074476 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904009c10 con 0x7fe8fc004d50 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- 172.21.15.82:0/63074476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc004d50 msgr2=0x7fe8fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 --2- 172.21.15.82:0/63074476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8fc004d50 0x7fe8fc0051d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe904009fc0 tx=0x7fe90401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe91108b640 1 -- 172.21.15.82:0/63074476 reap_dead start 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- 172.21.15.82:0/63074476 shutdown_connections 2024-04-25T10:03:11.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- 172.21.15.82:0/63074476 >> 172.21.15.82:0/63074476 conn(0x7fe90c0b30b0 msgr2=0x7fe90c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- 172.21.15.82:0/63074476 shutdown_connections 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- 172.21.15.82:0/63074476 wait complete. 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 Processor -- start 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- start start 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c09ec70 0x7fe90c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 0x7fe90c0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cf880 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0cc930 con 0x7fe90c09ec70 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe912aee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c09a480 con 0x7fe90c09f050 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52810/0 (socket says 172.21.15.82:52810) 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.235+0000 7fe90bfff640 1 -- 172.21.15.82:0/2061187011 learned_addr learned my addr 172.21.15.82:0/2061187011 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe91088a640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c09ec70 0x7fe90c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe91108b640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 0x7fe90c0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 -- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 msgr2=0x7fe90c0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 0x7fe90c0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 -- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c09ec70 msgr2=0x7fe90c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c09ec70 0x7fe90c0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 -- 172.21.15.82:0/2061187011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0c9070 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe91108b640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 0x7fe90c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe90bfff640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800eaf0 tx=0x7fe8f80016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800c8d0 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8018070 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0c9350 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800ccd0 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0c9910 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c04efa0 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe8f801d460 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe8f80853f0 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe91088a640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.239+0000 7fe91088a640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 0x7fe8f0069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9000099b0 tx=0x7fe900008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.243+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe8f8056070 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.395+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe90c0d13a0 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe9097fa640 1 -- 172.21.15.82:0/2061187011 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8f8055970 con 0x7fe90c0c8bd0 2024-04-25T10:03:11.405 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:11.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 msgr2=0x7fe8f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 0x7fe8f0069420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe9000099b0 tx=0x7fe900008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 msgr2=0x7fe90c0caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800eaf0 tx=0x7fe8f80016d0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 shutdown_connections 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe8f0066f60 0x7fe8f0069420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe90c09f050 0x7fe90c0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe90c0c8bd0 0x7fe90c0caa80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 --2- 172.21.15.82:0/2061187011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe90c09ec70 0x7fe90c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 >> 172.21.15.82:0/2061187011 conn(0x7fe90c0b30b0 msgr2=0x7fe90c0ccca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 shutdown_connections 2024-04-25T10:03:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.399+0000 7fe912aee640 1 -- 172.21.15.82:0/2061187011 wait complete. 2024-04-25T10:03:11.422 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:11.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 Processor -- start 2024-04-25T10:03:11.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 -- start start 2024-04-25T10:03:11.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 0x7efe940cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe94072000 0x7efe94072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe940cbd20 con 0x7efe9415b6b0 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9415ba90 con 0x7efe94072000 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe9b413640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940cff30 con 0x7efe940cf590 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe989ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 0x7efe940cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe999b0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe94072000 0x7efe94072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe989ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 0x7efe940cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:42452/0 (socket says 172.21.15.82:42452) 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe989ae640 1 -- 172.21.15.82:0/1461618685 learned_addr learned my addr 172.21.15.82:0/1461618685 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe991af640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7efe9415b6b0 0x7efe940ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:54498/0 (socket says 172.21.15.82:54498) 2024-04-25T10:03:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe999b0640 1 -- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 msgr2=0x7efe940cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe999b0640 1 --2- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 0x7efe940cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe999b0640 1 -- 172.21.15.82:0/1461618685 >> v1:172.21.15.196:6789/0 conn(0x7efe9415b6b0 legacy=0x7efe940ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.535+0000 7efe999b0640 1 -- 172.21.15.82:0/1461618685 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940cc000 con 0x7efe94072000 2024-04-25T10:03:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe999b0640 1 --2- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe94072470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c01c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6294dacfab011bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe8bfff640 1 -- 172.21.15.82:0/1461618685 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0b2070 con 0x7efe94072000 2024-04-25T10:03:11.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe8bfff640 1 -- 172.21.15.82:0/1461618685 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c0a5a00 con 0x7efe94072000 2024-04-25T10:03:11.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe8bfff640 1 -- 172.21.15.82:0/1461618685 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0a5dd0 con 0x7efe94072000 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 -- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 msgr2=0x7efe94072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 --2- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe94072470 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009d90 tx=0x7efe8c01c940 comp rx=0 tx=0).stop 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 -- 172.21.15.82:0/1461618685 shutdown_connections 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 --2- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe94072470 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 --2- 172.21.15.82:0/1461618685 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe940cf590 0x7efe940cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 -- 172.21.15.82:0/1461618685 >> 172.21.15.82:0/1461618685 conn(0x7efe9406ae60 msgr2=0x7efe9406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 -- 172.21.15.82:0/1461618685 shutdown_connections 2024-04-25T10:03:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.539+0000 7efe9b413640 1 -- 172.21.15.82:0/1461618685 wait complete. 2024-04-25T10:03:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 Processor -- start 2024-04-25T10:03:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- start start 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe9415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe9415b6b0 0x7efe94154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940cadb0 con 0x7efe940cf590 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe940cba30 con 0x7efe94072000 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe940caf30 con 0x7efe9415b6b0 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52820/0 (socket says 172.21.15.82:52820) 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 -- 172.21.15.82:0/503991502 learned_addr learned my addr 172.21.15.82:0/503991502 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe991af640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe9415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe999b0640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe9415b6b0 0x7efe94154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 -- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe9415b6b0 msgr2=0x7efe94154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe9415b6b0 0x7efe94154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 -- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 msgr2=0x7efe9415d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe9415d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 -- 172.21.15.82:0/503991502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe940c98c0 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe991af640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe9415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe989ae640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe8400e890 tx=0x7efe8400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84017070 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8400c940 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe940b1af0 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8400cd30 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9415de50 con 0x7efe940cf590 2024-04-25T10:03:11.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9404efa0 con 0x7efe940cf590 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.543+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe8401a070 con 0x7efe940cf590 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.547+0000 7efe89ffb640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 0x7efe78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.547+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7efe840856c0 con 0x7efe940cf590 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.547+0000 7efe991af640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 0x7efe78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.547+0000 7efe991af640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 0x7efe78069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe940af820 tx=0x7efe90007d90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.551+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe84056340 con 0x7efe940cf590 2024-04-25T10:03:11.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.707+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efe9406b5c0 con 0x7efe940cf590 2024-04-25T10:03:11.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.707+0000 7efe89ffb640 1 -- 172.21.15.82:0/503991502 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7efe84055c40 con 0x7efe940cf590 2024-04-25T10:03:11.716 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 msgr2=0x7efe78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 0x7efe78069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efe940af820 tx=0x7efe90007d90 comp rx=0 tx=0).stop 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 msgr2=0x7efe94159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7efe8400e890 tx=0x7efe8400ed50 comp rx=0 tx=0).stop 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 shutdown_connections 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe78066f60 0x7efe78069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe9415b6b0 0x7efe94154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe940cf590 0x7efe94159f60 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 --2- 172.21.15.82:0/503991502 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe94072000 0x7efe9415d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 >> 172.21.15.82:0/503991502 conn(0x7efe9406ae60 msgr2=0x7efe940afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 shutdown_connections 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.711+0000 7efe9b413640 1 -- 172.21.15.82:0/503991502 wait complete. 2024-04-25T10:03:11.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:11.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 Processor -- start 2024-04-25T10:03:11.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 -- start start 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0fa40d07f0 0x7f0fa40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 0x7f0fa406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa406be60 con 0x7f0fa415da80 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa406c440 con 0x7f0fa40d07f0 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fab1e7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa415de60 con 0x7f0fa40d1cd0 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa8f83640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0fa415da80 0x7f0fa40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44874/0 (socket says 172.21.15.82:44874) 2024-04-25T10:03:11.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa8f83640 1 -- 172.21.15.82:0/2295452408 learned_addr learned my addr 172.21.15.82:0/2295452408 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/2295452408 >> v2:172.21.15.196:3300/0 conn(0x7f0fa40d07f0 0x7f0fa40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa9784640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 0x7f0fa406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395035509 0 0) 0x7f0fa406be60 con 0x7f0fa415da80 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f8c003530 con 0x7f0fa415da80 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1750761391 0 0) 0x7f0f8c003530 con 0x7f0fa415da80 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 msgr2=0x7f0fa406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 0x7f0fa406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 >> v2:172.21.15.196:3300/0 conn(0x7f0fa40d07f0 msgr2=0x7f0fa40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 --2- 172.21.15.82:0/2295452408 >> v2:172.21.15.196:3300/0 conn(0x7f0fa40d07f0 0x7f0fa40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa40d2450 con 0x7f0fa415da80 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0f940016a0 con 0x7f0fa415da80 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 >> v1:172.21.15.82:6789/0 conn(0x7f0fa415da80 legacy=0x7f0fa40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c003a30 0x7f0f8c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c004810 0x7f0f8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 0x7f0f8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa406c440 con 0x7f0f8c004810 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa415de60 con 0x7f0f8c003a30 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8c00b9d0 con 0x7f0fa40d1cd0 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c004810 0x7f0f8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa8f83640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c003a30 0x7f0f8c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 -- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 msgr2=0x7f0f8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40d1cd0 0x7f0f8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 -- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c003a30 msgr2=0x7f0f8c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c003a30 0x7f0f8c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa3fff640 1 -- 172.21.15.82:0/2295452408 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c003530 con 0x7f0f8c004810 2024-04-25T10:03:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.839+0000 7f0fa8f83640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c003a30 0x7f0f8c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c004810 0x7f0f8c004c90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009af0 tx=0x7f0f9801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07ad62aff1d3f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98009690 con 0x7f0f8c004810 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f980059a0 con 0x7f0f8c004810 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fa37fe640 1 -- 172.21.15.82:0/2295452408 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f98005d90 con 0x7f0f8c004810 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- 172.21.15.82:0/2295452408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c004810 msgr2=0x7f0f8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/2295452408 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0f8c004810 0x7f0f8c004c90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0f98009af0 tx=0x7f0f9801ca70 comp rx=0 tx=0).stop 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fa9784640 1 -- 172.21.15.82:0/2295452408 reap_dead start 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- 172.21.15.82:0/2295452408 shutdown_connections 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- 172.21.15.82:0/2295452408 >> 172.21.15.82:0/2295452408 conn(0x7f0fa408b000 msgr2=0x7f0fa40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- 172.21.15.82:0/2295452408 shutdown_connections 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- 172.21.15.82:0/2295452408 wait complete. 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 Processor -- start 2024-04-25T10:03:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.843+0000 7f0fab1e7640 1 -- start start 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c004810 0x7f0fa40ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40a0a70 0x7f0fa409d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa40d26f0 con 0x7f0fa414aa80 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa40d2140 con 0x7f0f8c004810 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa40d2870 con 0x7f0fa40a0a70 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52832/0 (socket says 172.21.15.82:52832) 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 -- 172.21.15.82:0/345383907 learned_addr learned my addr 172.21.15.82:0/345383907 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa8f83640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c004810 0x7f0fa40ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa3fff640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40a0a70 0x7f0fa409d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 -- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40a0a70 msgr2=0x7f0fa409d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40a0a70 0x7f0fa409d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 -- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c004810 msgr2=0x7f0fa40ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c004810 0x7f0fa40ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 -- 172.21.15.82:0/345383907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa409d910 con 0x7f0fa414aa80 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa9784640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406bc80 tx=0x7f0f9c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018020 con 0x7f0fa414aa80 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f9c00bab0 con 0x7f0fa414aa80 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c014400 con 0x7f0fa414aa80 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa409fcb0 con 0x7f0fa414aa80 2024-04-25T10:03:11.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa40a0270 con 0x7f0fa414aa80 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa404efa0 con 0x7f0fa414aa80 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f9c01b020 con 0x7f0fa414aa80 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 0x7f0f90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f0f9c085800 con 0x7f0fa414aa80 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa8f83640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 0x7f0f90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.847+0000 7f0fa8f83640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 0x7f0f90069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406de60 tx=0x7f0f94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.851+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f9c022030 con 0x7f0fa414aa80 2024-04-25T10:03:12.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.995+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fa406da50 con 0x7f0fa414aa80 2024-04-25T10:03:12.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fa17fa640 1 -- 172.21.15.82:0/345383907 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fa409d910 con 0x7f0fa414aa80 2024-04-25T10:03:12.006 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:12.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 msgr2=0x7f0f90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 0x7f0f90069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406de60 tx=0x7f0f94009040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 msgr2=0x7f0fa40c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0fa406bc80 tx=0x7f0f9c00b480 comp rx=0 tx=0).stop 2024-04-25T10:03:12.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 shutdown_connections 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f90066f90 0x7f0f90069450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa40a0a70 0x7f0fa409d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa414aa80 0x7f0fa40c7820 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 --2- 172.21.15.82:0/345383907 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0f8c004810 0x7f0fa40ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 >> 172.21.15.82:0/345383907 conn(0x7f0fa408b000 msgr2=0x7f0fa40cec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 shutdown_connections 2024-04-25T10:03:12.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:11.999+0000 7f0fab1e7640 1 -- 172.21.15.82:0/345383907 wait complete. 2024-04-25T10:03:12.023 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.131+0000 7fa1aa629640 1 Processor -- start 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.131+0000 7fa1aa629640 1 -- start start 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1aa629640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 0x7fa1a40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1aa629640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1a4073010 0x7fa1a40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1aa629640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a40cbd20 con 0x7fa1a40cf590 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1aa629640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a415ba90 con 0x7fa1a4073010 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1aa629640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a40cc000 con 0x7fa1a415b6b0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1a4073010 0x7fa1a40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1a4073010 0x7fa1a40cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52846/0 (socket says 172.21.15.82:52846) 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 -- 172.21.15.82:0/2556136096 learned_addr learned my addr 172.21.15.82:0/2556136096 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a3fff640 1 --2- 172.21.15.82:0/2556136096 >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 0x7fa1a40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 -- 172.21.15.82:0/2556136096 >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 msgr2=0x7fa1a40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 --2- 172.21.15.82:0/2556136096 >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 0x7fa1a40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 -- 172.21.15.82:0/2556136096 >> v1:172.21.15.196:6789/0 conn(0x7fa1a40cf590 legacy=0x7fa1a415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 -- 172.21.15.82:0/2556136096 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a40cc2e0 con 0x7fa1a4073010 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a3fff640 1 --2- 172.21.15.82:0/2556136096 >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 0x7fa1a40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a8bc6640 1 --2- 172.21.15.82:0/2556136096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a40cfa20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d0180 tx=0x7fa19c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd58e22db771bab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a2ffd640 1 -- 172.21.15.82:0/2556136096 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0a5680 con 0x7fa1a4073010 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a2ffd640 1 -- 172.21.15.82:0/2556136096 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c0a5820 con 0x7fa1a4073010 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.135+0000 7fa1a2ffd640 1 -- 172.21.15.82:0/2556136096 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0a5bf0 con 0x7fa1a4073010 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2556136096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 msgr2=0x7fa1a40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2556136096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a40cfa20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40d0180 tx=0x7fa19c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2556136096 shutdown_connections 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2556136096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a40cfa20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2556136096 >> v2:172.21.15.196:3300/0 conn(0x7fa1a415b6b0 0x7fa1a40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2556136096 >> 172.21.15.82:0/2556136096 conn(0x7fa1a406ae60 msgr2=0x7fa1a406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2556136096 shutdown_connections 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2556136096 wait complete. 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 Processor -- start 2024-04-25T10:03:12.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- start start 2024-04-25T10:03:12.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a40cf590 0x7fa1a4151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 0x7fa1a4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40cadb0 con 0x7fa1a4073010 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40ca900 con 0x7fa1a415b6b0 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40caf30 con 0x7fa1a40cf590 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52854/0 (socket says 172.21.15.82:52854) 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 -- 172.21.15.82:0/2797632677 learned_addr learned my addr 172.21.15.82:0/2797632677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a8bc6640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 0x7fa1a4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 -- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a40cf590 msgr2=0x7fa1a4151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a37fe640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a40cf590 0x7fa1a4151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a40cf590 0x7fa1a4151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 -- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 msgr2=0x7fa1a4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 0x7fa1a4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 -- 172.21.15.82:0/2797632677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a415eaa0 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a8bc6640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 0x7fa1a4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a3fff640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c7f0 tx=0x7fa19400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194015070 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194007820 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a40c2f90 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194007c10 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a415e2a0 con 0x7fa1a4073010 2024-04-25T10:03:12.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.139+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a404efa0 con 0x7fa1a4073010 2024-04-25T10:03:12.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.143+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa194018070 con 0x7fa1a4073010 2024-04-25T10:03:12.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.143+0000 7fa1a0ff9640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 0x7fa190069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.143+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fa1940854d0 con 0x7fa1a4073010 2024-04-25T10:03:12.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.143+0000 7fa1a37fe640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 0x7fa190069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.143+0000 7fa1a37fe640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 0x7fa190069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa198007fd0 tx=0x7fa198009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.147+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa194056150 con 0x7fa1a4073010 2024-04-25T10:03:12.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.291+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa1a40b2090 con 0x7fa1a4073010 2024-04-25T10:03:12.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.291+0000 7fa1a0ff9640 1 -- 172.21.15.82:0/2797632677 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa194055a50 con 0x7fa1a4073010 2024-04-25T10:03:12.300 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 msgr2=0x7fa190069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 0x7fa190069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa198007fd0 tx=0x7fa198009040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 msgr2=0x7fa1a415d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c7f0 tx=0x7fa19400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 shutdown_connections 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa190066f60 0x7fa190069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1a40cf590 0x7fa1a4151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a415b6b0 0x7fa1a4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 --2- 172.21.15.82:0/2797632677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a4073010 0x7fa1a415d650 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 >> 172.21.15.82:0/2797632677 conn(0x7fa1a406ae60 msgr2=0x7fa1a40cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 shutdown_connections 2024-04-25T10:03:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.295+0000 7fa1aa629640 1 -- 172.21.15.82:0/2797632677 wait complete. 2024-04-25T10:03:12.317 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:12.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 Processor -- start 2024-04-25T10:03:12.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 -- start start 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d300d07f0 0x7f3d300d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d300d1cd0 0x7f3d3006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3006be60 con 0x7f3d3015da80 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3006c440 con 0x7f3d300d1cd0 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d34ef9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3015de60 con 0x7f3d300d07f0 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d300d1cd0 0x7f3d3006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d300d1cd0 0x7f3d3006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52864/0 (socket says 172.21.15.82:52864) 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 -- 172.21.15.82:0/433099277 learned_addr learned my addr 172.21.15.82:0/433099277 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2dd9b640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d300d07f0 0x7f3d300d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 -- 172.21.15.82:0/433099277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d300d07f0 msgr2=0x7f3d300d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d300d07f0 0x7f3d300d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 -- 172.21.15.82:0/433099277 >> v1:172.21.15.196:6789/0 conn(0x7f3d3015da80 legacy=0x7f3d300d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.431+0000 7f3d2ed9d640 1 -- 172.21.15.82:0/433099277 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d300d2450 con 0x7f3d300d1cd0 2024-04-25T10:03:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d2ed9d640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d1cd0 0x7f3d3006c8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009e50 tx=0x7f3d2801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326cb398b353aa9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d2d59a640 1 -- 172.21.15.82:0/433099277 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280a58b0 con 0x7f3d300d1cd0 2024-04-25T10:03:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d2d59a640 1 -- 172.21.15.82:0/433099277 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d280a5a50 con 0x7f3d300d1cd0 2024-04-25T10:03:12.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d2d59a640 1 -- 172.21.15.82:0/433099277 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d280a5e20 con 0x7f3d300d1cd0 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- 172.21.15.82:0/433099277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d1cd0 msgr2=0x7f3d3006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d1cd0 0x7f3d3006c8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3d28009e50 tx=0x7f3d2801ca00 comp rx=0 tx=0).stop 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- 172.21.15.82:0/433099277 shutdown_connections 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d300d07f0 0x7f3d300d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/433099277 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d1cd0 0x7f3d3006c8d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- 172.21.15.82:0/433099277 >> 172.21.15.82:0/433099277 conn(0x7f3d3008b000 msgr2=0x7f3d300d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- 172.21.15.82:0/433099277 shutdown_connections 2024-04-25T10:03:12.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- 172.21.15.82:0/433099277 wait complete. 2024-04-25T10:03:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 Processor -- start 2024-04-25T10:03:12.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.435+0000 7f3d34ef9640 1 -- start start 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d300d1cd0 0x7f3d30155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 0x7f3d300b05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3006d7a0 con 0x7f3d300d07f0 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3006d620 con 0x7f3d300d1cd0 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3006d920 con 0x7f3d3015da80 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52878/0 (socket says 172.21.15.82:52878) 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 -- 172.21.15.82:0/2542551335 learned_addr learned my addr 172.21.15.82:0/2542551335 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2ed9d640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 0x7f3d300b05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 -- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 msgr2=0x7f3d300b05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 0x7f3d300b05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2dd9b640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d300d1cd0 0x7f3d30155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 -- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d300d1cd0 msgr2=0x7f3d30155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d300d1cd0 0x7f3d30155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 -- 172.21.15.82:0/2542551335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3015e180 con 0x7f3d300d07f0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2ed9d640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 0x7f3d300b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2e59c640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c370 tx=0x7f3d2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d200052d0 con 0x7f3d300d07f0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d20005470 con 0x7f3d300d07f0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d20007260 con 0x7f3d300d07f0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300b8ff0 con 0x7f3d300d07f0 2024-04-25T10:03:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3015e700 con 0x7f3d300d07f0 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d3004efa0 con 0x7f3d300d07f0 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d20007400 con 0x7f3d300d07f0 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 0x7f3d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3d20084850 con 0x7f3d300d07f0 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2dd9b640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 0x7f3d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.439+0000 7f3d2dd9b640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 0x7f3d10069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d30160370 tx=0x7f3d24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.443+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d200554d0 con 0x7f3d300d07f0 2024-04-25T10:03:12.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.615+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3d30156050 con 0x7f3d300d07f0 2024-04-25T10:03:12.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.615+0000 7f3d1effd640 1 -- 172.21.15.82:0/2542551335 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3d20054dd0 con 0x7f3d300d07f0 2024-04-25T10:03:12.623 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 msgr2=0x7f3d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 0x7f3d10069420 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3d30160370 tx=0x7f3d24008040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 msgr2=0x7f3d300ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3d2000c370 tx=0x7f3d2000c830 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 shutdown_connections 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d10066f60 0x7f3d10069420 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d3015da80 0x7f3d300b05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d300d1cd0 0x7f3d30155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 --2- 172.21.15.82:0/2542551335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d300d07f0 0x7f3d300ca7c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 >> 172.21.15.82:0/2542551335 conn(0x7f3d3008b000 msgr2=0x7f3d30159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 shutdown_connections 2024-04-25T10:03:12.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.619+0000 7f3d34ef9640 1 -- 172.21.15.82:0/2542551335 wait complete. 2024-04-25T10:03:12.642 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:03:12.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:12.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T10:03:12.654 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:12.654 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:03:12.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:12.654 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:03:12.702 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:12.702 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:12.709 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:12.709 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:12.758 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 Processor -- start 2024-04-25T10:03:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 -- start start 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9da80cd220 0x7f9da80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da80b1240 con 0x7f9da80b2740 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da815f360 con 0x7f9da80d0010 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9dad05d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da815f640 con 0x7f9da80cd220 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9d9ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9da80b2740 0x7f9da80cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44882/0 (socket says 172.21.15.82:44882) 2024-04-25T10:03:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9d9ffff640 1 -- 172.21.15.82:0/3102752425 learned_addr learned my addr 172.21.15.82:0/3102752425 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789534299 0 0) 0x7f9d90001120 con 0x7f9da80b2740 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da6d9d640 1 --2- 172.21.15.82:0/3102752425 >> v2:172.21.15.196:3300/0 conn(0x7f9da80cd220 0x7f9da80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d94003530 con 0x7f9da80b2740 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4021325652 0 0) 0x7f9d94003530 con 0x7f9da80b2740 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 >> v2:172.21.15.196:3300/0 conn(0x7f9da80cd220 msgr2=0x7f9da80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 --2- 172.21.15.82:0/3102752425 >> v2:172.21.15.196:3300/0 conn(0x7f9da80cd220 0x7f9da80cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 >> v1:172.21.15.196:6789/0 conn(0x7f9da80d0010 legacy=0x7f9da8099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da815f920 con 0x7f9da80b2740 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9d90001850 con 0x7f9da80b2740 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 >> v1:172.21.15.82:6789/0 conn(0x7f9da80b2740 legacy=0x7f9da80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d94003a30 0x7f9d94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94004810 0x7f9d94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 0x7f9d9400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da815f640 con 0x7f9d94005240 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9400c1a0 con 0x7f9d94004810 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9400c3a0 con 0x7f9d94003a30 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 0x7f9d9400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9d9ffff640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94004810 0x7f9d94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da6d9d640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d94003a30 0x7f9d94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 -- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d94003a30 msgr2=0x7f9d94003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d94003a30 0x7f9d94003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 -- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94004810 msgr2=0x7f9d94004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94004810 0x7f9d94004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9da759e640 1 -- 172.21.15.82:0/3102752425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d94003530 con 0x7f9d94005240 2024-04-25T10:03:12.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.875+0000 7f9d9ffff640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94004810 0x7f9d94004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9da759e640 1 -- 172.21.15.82:0/3102752425 reap_dead start 2024-04-25T10:03:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9da759e640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 0x7f9d9400b9d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001e40 tx=0x7f9da001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1596da2aef3f693a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0001a90 con 0x7f9d94005240 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da0003b80 con 0x7f9d94005240 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9da659c640 1 -- 172.21.15.82:0/3102752425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da00050d0 con 0x7f9d94005240 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 msgr2=0x7f9d9400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 0x7f9d9400b9d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9da0001e40 tx=0x7f9da001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- 172.21.15.82:0/3102752425 shutdown_connections 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 --2- 172.21.15.82:0/3102752425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d94005240 0x7f9d9400b9d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- 172.21.15.82:0/3102752425 >> 172.21.15.82:0/3102752425 conn(0x7f9da80d15b0 msgr2=0x7f9da80d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- 172.21.15.82:0/3102752425 shutdown_connections 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- 172.21.15.82:0/3102752425 wait complete. 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 Processor -- start 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.879+0000 7f9dad05d640 1 -- start start 2024-04-25T10:03:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94005240 0x7f9da80b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9da815c480 0x7f9da815c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80b2bb0 con 0x7f9da8153b40 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da815fee0 con 0x7f9d94005240 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da80b2d30 con 0x7f9da815c480 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52882/0 (socket says 172.21.15.82:52882) 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 -- 172.21.15.82:0/961969436 learned_addr learned my addr 172.21.15.82:0/961969436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9da6d9d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94005240 0x7f9da80b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9da759e640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9da815c480 0x7f9da815c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 -- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9da815c480 msgr2=0x7f9da815c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9da815c480 0x7f9da815c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 -- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94005240 msgr2=0x7f9da80b5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94005240 0x7f9da80b5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 -- 172.21.15.82:0/961969436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da80b4b40 con 0x7f9da8153b40 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9ffff640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d90011600 tx=0x7f9d90011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d90010040 con 0x7f9da8153b40 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9000f8b0 con 0x7f9da8153b40 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da80b4e20 con 0x7f9da8153b40 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000fc80 con 0x7f9da8153b40 2024-04-25T10:03:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da80b53e0 con 0x7f9da8153b40 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da804efa0 con 0x7f9da8153b40 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d90020470 con 0x7f9da8153b40 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 0x7f9d88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f9d900885e0 con 0x7f9da8153b40 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9da6d9d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 0x7f9d88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.883+0000 7f9da6d9d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 0x7f9d88069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9da809a910 tx=0x7f9d98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:12.887+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9da80b1240 con 0x7f9da8153b40 2024-04-25T10:03:13.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.047+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da80cede0 con 0x7f9da8153b40 2024-04-25T10:03:13.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.047+0000 7f9d9f7fe640 1 -- 172.21.15.82:0/961969436 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f9d9000fe20 con 0x7f9da8153b40 2024-04-25T10:03:13.057 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:13.057 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 msgr2=0x7f9d88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 0x7f9d88069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9da809a910 tx=0x7f9d98008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 msgr2=0x7f9da80b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9d90011600 tx=0x7f9d90011ac0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 shutdown_connections 2024-04-25T10:03:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d88066f60 0x7f9d88069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9da815c480 0x7f9da815c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9da8153b40 0x7f9da80b41a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 --2- 172.21.15.82:0/961969436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d94005240 0x7f9da80b5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 >> 172.21.15.82:0/961969436 conn(0x7f9da80d15b0 msgr2=0x7f9da8070830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 shutdown_connections 2024-04-25T10:03:13.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.051+0000 7f9dad05d640 1 -- 172.21.15.82:0/961969436 wait complete. 2024-04-25T10:03:13.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:13.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:13.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 Processor -- start 2024-04-25T10:03:13.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 -- start start 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc006b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dc015ca90 0x7f6dc00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc006b5b0 con 0x7f6dc00cc130 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc015ff70 con 0x7f6dc015ca90 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc72cb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc00d0300 con 0x7f6dc006ac90 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dc015ca90 0x7f6dc00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dc015ca90 0x7f6dc00d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52884/0 (socket says 172.21.15.82:52884) 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 -- 172.21.15.82:0/4030685024 learned_addr learned my addr 172.21.15.82:0/4030685024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc5067640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 -- 172.21.15.82:0/4030685024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 msgr2=0x7f6dc006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc006b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 -- 172.21.15.82:0/4030685024 >> v1:172.21.15.82:6789/0 conn(0x7f6dc00cc130 legacy=0x7f6dc00ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6dc4866640 1 -- 172.21.15.82:0/4030685024 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc015f360 con 0x7f6dc015ca90 2024-04-25T10:03:13.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.187+0000 7f6db7fff640 1 -- 172.21.15.82:0/4030685024 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506257275 0 0) 0x7f6dc006b5b0 con 0x7f6dc00cc130 2024-04-25T10:03:13.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc4866640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc015ca90 0x7f6dc00d0c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009870 tx=0x7f6db001c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0f79e407da4cef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6db7fff640 1 -- 172.21.15.82:0/4030685024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00a9610 con 0x7f6dc015ca90 2024-04-25T10:03:13.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6db7fff640 1 -- 172.21.15.82:0/4030685024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00a97b0 con 0x7f6dc015ca90 2024-04-25T10:03:13.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6db7fff640 1 -- 172.21.15.82:0/4030685024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00a9b80 con 0x7f6dc015ca90 2024-04-25T10:03:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- 172.21.15.82:0/4030685024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc015ca90 msgr2=0x7f6dc00d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc015ca90 0x7f6dc00d0c10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009870 tx=0x7f6db001c550 comp rx=0 tx=0).stop 2024-04-25T10:03:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- 172.21.15.82:0/4030685024 shutdown_connections 2024-04-25T10:03:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc015ca90 0x7f6dc00d0c10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/4030685024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc006b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- 172.21.15.82:0/4030685024 >> 172.21.15.82:0/4030685024 conn(0x7f6dc0099a70 msgr2=0x7f6dc00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- 172.21.15.82:0/4030685024 shutdown_connections 2024-04-25T10:03:13.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- 172.21.15.82:0/4030685024 wait complete. 2024-04-25T10:03:13.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 Processor -- start 2024-04-25T10:03:13.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- start start 2024-04-25T10:03:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc00b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 0x7f6dc01f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0159750 con 0x7f6dc00cc130 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc01595d0 con 0x7f6dc01476d0 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc72cb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc0168660 con 0x7f6dc006ac90 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc4866640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc4866640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52892/0 (socket says 172.21.15.82:52892) 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.191+0000 7f6dc4866640 1 -- 172.21.15.82:0/2012144905 learned_addr learned my addr 172.21.15.82:0/2012144905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5067640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc00b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5868640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 0x7f6dc01f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 -- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 msgr2=0x7f6dc00b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc00b3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 -- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 msgr2=0x7f6dc01f4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 0x7f6dc01f4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 -- 172.21.15.82:0/2012144905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00c73e0 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5067640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc00b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5868640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 0x7f6dc01f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc4866640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6db001ac00 tx=0x7f6db001ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00bc650 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00bc7f0 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc00c5220 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00bcbc0 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc00c57e0 con 0x7f6dc00cc130 2024-04-25T10:03:13.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc004efa0 con 0x7f6dc00cc130 2024-04-25T10:03:13.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6db00c53f0 con 0x7f6dc00cc130 2024-04-25T10:03:13.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 0x7f6da4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6db0120010 con 0x7f6dc00cc130 2024-04-25T10:03:13.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5067640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 0x7f6da4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.195+0000 7f6dc5067640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 0x7f6da4069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6db8007330 tx=0x7f6db800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.199+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6db00ad020 con 0x7f6dc00cc130 2024-04-25T10:03:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.355+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dc00c7060 con 0x7f6dc00cc130 2024-04-25T10:03:13.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.355+0000 7f6db5ffb640 1 -- 172.21.15.82:0/2012144905 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6db0016170 con 0x7f6dc00cc130 2024-04-25T10:03:13.364 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:13.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 msgr2=0x7f6da4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 0x7f6da4069450 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6db8007330 tx=0x7f6db800b040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 msgr2=0x7f6dc0158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6db001ac00 tx=0x7f6db001ad90 comp rx=0 tx=0).stop 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 shutdown_connections 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6da4066f90 0x7f6da4069450 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc01476d0 0x7f6dc01f4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc00cc130 0x7f6dc0158980 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 --2- 172.21.15.82:0/2012144905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc006ac90 0x7f6dc00b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 >> 172.21.15.82:0/2012144905 conn(0x7f6dc0099a70 msgr2=0x7f6dc015d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 shutdown_connections 2024-04-25T10:03:13.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.359+0000 7f6dc72cb640 1 -- 172.21.15.82:0/2012144905 wait complete. 2024-04-25T10:03:13.382 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 Processor -- start 2024-04-25T10:03:13.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 -- start start 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6dc0cfa20 0x7fc6dc072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc0cbd20 con 0x7fc6dc0cf590 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6dc0cc000 con 0x7fc6dc15b6b0 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6e1268640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6dc15ba90 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6db7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc6dc0cfa20 0x7fc6dc072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6da7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc6dc0cf590 0x7fc6dc072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44902/0 (socket says 172.21.15.82:44902) 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6da7fc640 1 -- 172.21.15.82:0/539859546 learned_addr learned my addr 172.21.15.82:0/539859546 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6db7fe640 1 -- 172.21.15.82:0/539859546 >> v1:172.21.15.196:6789/0 conn(0x7fc6dc15b6b0 legacy=0x7fc6dc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6db7fe640 1 -- 172.21.15.82:0/539859546 >> v1:172.21.15.82:6789/0 conn(0x7fc6dc0cf590 legacy=0x7fc6dc072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.491+0000 7fc6db7fe640 1 -- 172.21.15.82:0/539859546 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc0cc2e0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6db7fe640 1 --2- 172.21.15.82:0/539859546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc072710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009d90 tx=0x7fc6d401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9ba191bcab2db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6d9ffb640 1 -- 172.21.15.82:0/539859546 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40b2070 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6d9ffb640 1 -- 172.21.15.82:0/539859546 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d40a5a00 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6d9ffb640 1 -- 172.21.15.82:0/539859546 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d40a5dd0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- 172.21.15.82:0/539859546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 msgr2=0x7fc6dc072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 --2- 172.21.15.82:0/539859546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc072710 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4009d90 tx=0x7fc6d401c940 comp rx=0 tx=0).stop 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- 172.21.15.82:0/539859546 shutdown_connections 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 --2- 172.21.15.82:0/539859546 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc072710 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- 172.21.15.82:0/539859546 >> 172.21.15.82:0/539859546 conn(0x7fc6dc06ae60 msgr2=0x7fc6dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- 172.21.15.82:0/539859546 shutdown_connections 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- 172.21.15.82:0/539859546 wait complete. 2024-04-25T10:03:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 Processor -- start 2024-04-25T10:03:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- start start 2024-04-25T10:03:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6dc0cf590 0x7fc6dc098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 0x7fc6dc0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cadb0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0cfe50 con 0x7fc6dc15b6b0 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6e1268640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6dc0caf30 con 0x7fc6dc0cf590 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52916/0 (socket says 172.21.15.82:52916) 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 -- 172.21.15.82:0/3091923423 learned_addr learned my addr 172.21.15.82:0/3091923423 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6db7fe640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 0x7fc6dc0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 -- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6dc0cf590 msgr2=0x7fc6dc098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6daffd640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6dc0cf590 0x7fc6dc098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6dc0cf590 0x7fc6dc098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 -- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 msgr2=0x7fc6dc0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 0x7fc6dc0cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 -- 172.21.15.82:0/3091923423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc15a000 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6db7fe640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 0x7fc6dc0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.495+0000 7fc6da7fc640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c530 tx=0x7fc6cc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc017070 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc005cf0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6dc15a2e0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc007270 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6dc15a8a0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6dc04efa0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6cc01a070 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc6cc0846d0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6daffd640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.499+0000 7fc6daffd640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 0x7fc6c8069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15afc0 tx=0x7fc6d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.503+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6cc054c70 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.647+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6dc0ca6a0 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.647+0000 7fc6c37fe640 1 -- 172.21.15.82:0/3091923423 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6cc058d10 con 0x7fc6dc0cfa20 2024-04-25T10:03:13.656 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 msgr2=0x7fc6c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 0x7fc6c8069420 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc15afc0 tx=0x7fc6d0009040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 msgr2=0x7fc6dc0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c530 tx=0x7fc6cc00c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 shutdown_connections 2024-04-25T10:03:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc6c8066f60 0x7fc6c8069420 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6dc15b6b0 0x7fc6dc0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6dc0cfa20 0x7fc6dc0ccf30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 --2- 172.21.15.82:0/3091923423 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc6dc0cf590 0x7fc6dc098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 >> 172.21.15.82:0/3091923423 conn(0x7fc6dc06ae60 msgr2=0x7fc6dc15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 shutdown_connections 2024-04-25T10:03:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.651+0000 7fc6e1268640 1 -- 172.21.15.82:0/3091923423 wait complete. 2024-04-25T10:03:13.674 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:13.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 Processor -- start 2024-04-25T10:03:13.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 -- start start 2024-04-25T10:03:13.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4fe40cf970 0x7f4fe40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fe40ca4c0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fe40ca780 con 0x7f4fe40cf970 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4febe93640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe40cd440 con 0x7f4fe40cf590 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4fe40cf970 0x7f4fe40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4fe40cf970 0x7f4fe40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52918/0 (socket says 172.21.15.82:52918) 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 -- 172.21.15.82:0/1563170375 learned_addr learned my addr 172.21.15.82:0/1563170375 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 -- 172.21.15.82:0/1563170375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 msgr2=0x7f4fe415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe9c2f640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 -- 172.21.15.82:0/1563170375 >> v1:172.21.15.196:6789/0 conn(0x7f4fe40c9ef0 legacy=0x7f4fe40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.783+0000 7f4fe942e640 1 -- 172.21.15.82:0/1563170375 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe40cd690 con 0x7f4fe40cf970 2024-04-25T10:03:13.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe942e640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40cf970 0x7f4fe40c99b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009870 tx=0x7f4fd401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03364531b342a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe8c2d640 1 -- 172.21.15.82:0/1563170375 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40b0070 con 0x7f4fe40cf970 2024-04-25T10:03:13.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe8c2d640 1 -- 172.21.15.82:0/1563170375 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd40ab600 con 0x7f4fe40cf970 2024-04-25T10:03:13.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe8c2d640 1 -- 172.21.15.82:0/1563170375 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd40ab9d0 con 0x7f4fe40cf970 2024-04-25T10:03:13.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/1563170375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40cf970 msgr2=0x7f4fe40c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40cf970 0x7f4fe40c99b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4009870 tx=0x7f4fd401c520 comp rx=0 tx=0).stop 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/1563170375 shutdown_connections 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40cf970 0x7f4fe40c99b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- 172.21.15.82:0/1563170375 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/1563170375 >> 172.21.15.82:0/1563170375 conn(0x7f4fe406ae60 msgr2=0x7f4fe406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/1563170375 shutdown_connections 2024-04-25T10:03:13.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/1563170375 wait complete. 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 Processor -- start 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- start start 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40cf970 0x7f4fe4155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe4167660 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41674e0 con 0x7f4fe40cf970 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41677e0 con 0x7f4fe40cf590 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52926/0 (socket says 172.21.15.82:52926) 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 -- 172.21.15.82:0/53121724 learned_addr learned my addr 172.21.15.82:0/53121724 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe942e640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fea430640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40cf970 0x7f4fe4155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 -- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 msgr2=0x7f4fe415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 -- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40cf970 msgr2=0x7f4fe4155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40cf970 0x7f4fe4155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 -- 172.21.15.82:0/53121724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe4156ca0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe942e640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fe9c2f640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c7f0 tx=0x7f4fe000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0015070 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fe0004240 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe4158750 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0003620 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.787+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe415b110 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe404efa0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4fe00039d0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4fda7fc640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 0x7f4fc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4fe0085450 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4fe942e640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 0x7f4fc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.791+0000 7f4fe942e640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 0x7f4fc8069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ad000 tx=0x7f4fd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.795+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4fe00560d0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.939+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4fe40b4170 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.939+0000 7f4fda7fc640 1 -- 172.21.15.82:0/53121724 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4fe00559d0 con 0x7f4fe40c9ef0 2024-04-25T10:03:13.948 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 msgr2=0x7f4fc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 0x7f4fc8069420 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40ad000 tx=0x7f4fd4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 msgr2=0x7f4fe4071e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4fe000c7f0 tx=0x7f4fe000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 shutdown_connections 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fc8066f60 0x7f4fc8069420 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4fe40cf590 0x7f4fe415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4fe40cf970 0x7f4fe4155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 --2- 172.21.15.82:0/53121724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4fe40c9ef0 0x7f4fe4071e00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 >> 172.21.15.82:0/53121724 conn(0x7f4fe406ae60 msgr2=0x7f4fe40712a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 shutdown_connections 2024-04-25T10:03:13.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:13.943+0000 7f4febe93640 1 -- 172.21.15.82:0/53121724 wait complete. 2024-04-25T10:03:13.965 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.067+0000 7fe8b256c640 1 Processor -- start 2024-04-25T10:03:14.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.067+0000 7fe8b256c640 1 -- start start 2024-04-25T10:03:14.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b256c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8ac15da80 0x7fe8ac0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b256c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe8ac0d07f0 0x7fe8ac06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b256c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ac06be60 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b256c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac06c440 con 0x7fe8ac15da80 2024-04-25T10:03:14.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b256c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8ac15de60 con 0x7fe8ac0d07f0 2024-04-25T10:03:14.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8ac15da80 0x7fe8ac0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe8ac15da80 0x7fe8ac0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52930/0 (socket says 172.21.15.82:52930) 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 -- 172.21.15.82:0/2493902162 learned_addr learned my addr 172.21.15.82:0/2493902162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8b0b09640 1 --2- 172.21.15.82:0/2493902162 >> v2:172.21.15.196:3300/0 conn(0x7fe8ac0d07f0 0x7fe8ac06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 -- 172.21.15.82:0/2493902162 >> v2:172.21.15.196:3300/0 conn(0x7fe8ac0d07f0 msgr2=0x7fe8ac06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 --2- 172.21.15.82:0/2493902162 >> v2:172.21.15.196:3300/0 conn(0x7fe8ac0d07f0 0x7fe8ac06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 -- 172.21.15.82:0/2493902162 >> v1:172.21.15.196:6789/0 conn(0x7fe8ac0d1cd0 legacy=0x7fe8ac0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 -- 172.21.15.82:0/2493902162 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac0d2450 con 0x7fe8ac15da80 2024-04-25T10:03:14.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8abfff640 1 --2- 172.21.15.82:0/2493902162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0d4230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009870 tx=0x7fe89c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d943e19a8a14b582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8ab7fe640 1 -- 172.21.15.82:0/2493902162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0ab650 con 0x7fe8ac15da80 2024-04-25T10:03:14.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8ab7fe640 1 -- 172.21.15.82:0/2493902162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89c0b0070 con 0x7fe8ac15da80 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.071+0000 7fe8ab7fe640 1 -- 172.21.15.82:0/2493902162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89c0aba10 con 0x7fe8ac15da80 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/2493902162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac15da80 msgr2=0x7fe8ac0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- 172.21.15.82:0/2493902162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0d4230 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe89c009870 tx=0x7fe89c01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/2493902162 shutdown_connections 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- 172.21.15.82:0/2493902162 >> v2:172.21.15.196:3300/0 conn(0x7fe8ac0d07f0 0x7fe8ac06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- 172.21.15.82:0/2493902162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0d4230 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/2493902162 >> 172.21.15.82:0/2493902162 conn(0x7fe8ac08b000 msgr2=0x7fe8ac0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/2493902162 shutdown_connections 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/2493902162 wait complete. 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 Processor -- start 2024-04-25T10:03:14.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- start start 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8ac0d07f0 0x7fe8ac154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06d7a0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06d620 con 0x7fe8ac15da80 2024-04-25T10:03:14.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac06d920 con 0x7fe8ac0d07f0 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52940/0 (socket says 172.21.15.82:52940) 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 -- 172.21.15.82:0/1280023469 learned_addr learned my addr 172.21.15.82:0/1280023469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 -- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8ac0d07f0 msgr2=0x7fe8ac154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8ac0d07f0 0x7fe8ac154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b0b09640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 -- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac15da80 msgr2=0x7fe8ac0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 -- 172.21.15.82:0/1280023469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac153f50 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a37fe640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c430 tx=0x7fe89400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011790 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894017070 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ac154230 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894011bb0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.075+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ac1547f0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8ac04efa0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe894012420 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8a97fa640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 0x7fe88c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fe894084ed0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8abfff640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 0x7fe88c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.079+0000 7fe8abfff640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 0x7fe88c069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0af000 tx=0x7fe89c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.083+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe894055b50 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.239+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe8ac1549c0 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8a97fa640 1 -- 172.21.15.82:0/1280023469 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe894055450 con 0x7fe8ac0d1cd0 2024-04-25T10:03:14.249 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:14.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 msgr2=0x7fe88c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 0x7fe88c069420 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe89c0af000 tx=0x7fe89c004740 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 msgr2=0x7fe8ac153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe89400c430 tx=0x7fe89400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 shutdown_connections 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe88c066f60 0x7fe88c069420 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe8ac15da80 0x7fe8ac0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe8ac0d1cd0 0x7fe8ac153500 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 --2- 172.21.15.82:0/1280023469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe8ac0d07f0 0x7fe8ac154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 >> 172.21.15.82:0/1280023469 conn(0x7fe8ac08b000 msgr2=0x7fe8ac0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 shutdown_connections 2024-04-25T10:03:14.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.243+0000 7fe8b256c640 1 -- 172.21.15.82:0/1280023469 wait complete. 2024-04-25T10:03:14.268 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:14.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 Processor -- start 2024-04-25T10:03:14.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 -- start start 2024-04-25T10:03:14.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f212815aac0 0x7f21280ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21280cf0e0 0x7f21280cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280ce320 con 0x7f212815aea0 2024-04-25T10:03:14.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21280ce0d0 con 0x7f21280cf0e0 2024-04-25T10:03:14.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f21302bc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21280ce600 con 0x7f212815aac0 2024-04-25T10:03:14.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21280cf0e0 0x7f21280cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21280cf0e0 0x7f21280cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52950/0 (socket says 172.21.15.82:52950) 2024-04-25T10:03:14.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 -- 172.21.15.82:0/1941984314 learned_addr learned my addr 172.21.15.82:0/1941984314 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:14.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 -- 172.21.15.82:0/1941984314 >> v2:172.21.15.196:3300/0 conn(0x7f212815aac0 msgr2=0x7f21280ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 --2- 172.21.15.82:0/1941984314 >> v2:172.21.15.196:3300/0 conn(0x7f212815aac0 0x7f21280ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 -- 172.21.15.82:0/1941984314 >> v1:172.21.15.196:6789/0 conn(0x7f212815aea0 legacy=0x7f212815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.379+0000 7f212d857640 1 -- 172.21.15.82:0/1941984314 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280ce8e0 con 0x7f21280cf0e0 2024-04-25T10:03:14.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f212d857640 1 --2- 172.21.15.82:0/1941984314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f21280cd180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2128072f80 tx=0x7f211c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41294b0b77579b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f212d056640 1 -- 172.21.15.82:0/1941984314 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0ab5f0 con 0x7f21280cf0e0 2024-04-25T10:03:14.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f212d056640 1 -- 172.21.15.82:0/1941984314 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c0b0070 con 0x7f21280cf0e0 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f212d056640 1 -- 172.21.15.82:0/1941984314 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0ab9b0 con 0x7f21280cf0e0 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- 172.21.15.82:0/1941984314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 msgr2=0x7f21280cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 --2- 172.21.15.82:0/1941984314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f21280cd180 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2128072f80 tx=0x7f211c01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- 172.21.15.82:0/1941984314 shutdown_connections 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 --2- 172.21.15.82:0/1941984314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f21280cd180 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 --2- 172.21.15.82:0/1941984314 >> v2:172.21.15.196:3300/0 conn(0x7f212815aac0 0x7f21280ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- 172.21.15.82:0/1941984314 >> 172.21.15.82:0/1941984314 conn(0x7f2128099a70 msgr2=0x7f21280d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- 172.21.15.82:0/1941984314 shutdown_connections 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- 172.21.15.82:0/1941984314 wait complete. 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 Processor -- start 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.383+0000 7f21302bc640 1 -- start start 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 0x7f212806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212815aea0 0x7f212806b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cbbb0 con 0x7f21280cf0e0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cba30 con 0x7f212815aea0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cbd30 con 0x7f212815aac0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52954/0 (socket says 172.21.15.82:52954) 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 -- 172.21.15.82:0/4173829348 learned_addr learned my addr 172.21.15.82:0/4173829348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212d857640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 0x7f212806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e859640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212815aea0 0x7f212806b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 -- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 msgr2=0x7f212806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 0x7f212806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 -- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212815aea0 msgr2=0x7f212806b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212815aea0 0x7f212806b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 -- 172.21.15.82:0/4173829348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280b4510 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212d857640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 0x7f212806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212e058640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f212000ba50 tx=0x7f212000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120019070 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212000cc00 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21280b47f0 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2120014440 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21280b4db0 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212804efa0 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f212001c070 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 0x7f2114069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f2120085140 con 0x7f21280cf0e0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212d857640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 0x7f2114069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.387+0000 7f212d857640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 0x7f2114069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f211c0af000 tx=0x7f211c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.391+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2120055dc0 con 0x7f21280cf0e0 2024-04-25T10:03:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.547+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f21280cdc90 con 0x7f21280cf0e0 2024-04-25T10:03:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.547+0000 7f2126ffd640 1 -- 172.21.15.82:0/4173829348 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f21200556c0 con 0x7f21280cf0e0 2024-04-25T10:03:14.556 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 msgr2=0x7f2114069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 0x7f2114069420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f211c0af000 tx=0x7f211c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 msgr2=0x7f2128154210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f212000ba50 tx=0x7f212000bf10 comp rx=0 tx=0).stop 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 shutdown_connections 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2114066f60 0x7f2114069420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815aac0 0x7f212806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f212815aea0 0x7f212806b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 --2- 172.21.15.82:0/4173829348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21280cf0e0 0x7f2128154210 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 >> 172.21.15.82:0/4173829348 conn(0x7f2128099a70 msgr2=0x7f2128156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 shutdown_connections 2024-04-25T10:03:14.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.551+0000 7f21302bc640 1 -- 172.21.15.82:0/4173829348 wait complete. 2024-04-25T10:03:14.574 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:03:14.574 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:14.580 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:14.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 Processor -- start 2024-04-25T10:03:14.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 -- start start 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c200d07f0 0x7f3c200d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d1cd0 0x7f3c2006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2006be60 con 0x7f3c2015da80 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2006c440 con 0x7f3c200d07f0 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c27a4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2015de60 con 0x7f3c200d1cd0 2024-04-25T10:03:14.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c200d07f0 0x7f3c200d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c257e7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3c2015da80 0x7f3c200d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41260/0 (socket says 172.21.15.82:41260) 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c257e7640 1 -- 172.21.15.82:0/1730324724 learned_addr learned my addr 172.21.15.82:0/1730324724 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 -- 172.21.15.82:0/1730324724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d1cd0 msgr2=0x7f3c2006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c25fe8640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d1cd0 0x7f3c2006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d1cd0 0x7f3c2006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c13fff640 1 -- 172.21.15.82:0/1730324724 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021264242 0 0) 0x7f3c2006be60 con 0x7f3c2015da80 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 -- 172.21.15.82:0/1730324724 >> v1:172.21.15.82:6789/0 conn(0x7f3c2015da80 legacy=0x7f3c200d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 -- 172.21.15.82:0/1730324724 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200d2450 con 0x7f3c200d07f0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c24fe6640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c200d07f0 0x7f3c200d4230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f60446134d3c5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.679+0000 7f3c13fff640 1 -- 172.21.15.82:0/1730324724 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140b1070 con 0x7f3c200d07f0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c13fff640 1 -- 172.21.15.82:0/1730324724 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c140ac410 con 0x7f3c200d07f0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c13fff640 1 -- 172.21.15.82:0/1730324724 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140ac7e0 con 0x7f3c200d07f0 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/1730324724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c200d07f0 msgr2=0x7f3c200d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c200d07f0 0x7f3c200d4230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1401c450 comp rx=0 tx=0).stop 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/1730324724 shutdown_connections 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d1cd0 0x7f3c2006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/1730324724 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c200d07f0 0x7f3c200d4230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/1730324724 >> 172.21.15.82:0/1730324724 conn(0x7f3c2008b000 msgr2=0x7f3c200d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/1730324724 shutdown_connections 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/1730324724 wait complete. 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 Processor -- start 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- start start 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d07f0 0x7f3c2015bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c200d1cd0 0x7f3c200c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2006d7a0 con 0x7f3c2015da80 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2006d620 con 0x7f3c200d1cd0 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2006d920 con 0x7f3c200d07f0 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40724/0 (socket says 172.21.15.82:40724) 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 -- 172.21.15.82:0/3387511 learned_addr learned my addr 172.21.15.82:0/3387511 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c24fe6640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c200d1cd0 0x7f3c200c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c257e7640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d07f0 0x7f3c2015bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 -- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d07f0 msgr2=0x7f3c2015bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d07f0 0x7f3c2015bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 -- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c200d1cd0 msgr2=0x7f3c200c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c200d1cd0 0x7f3c200c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 -- 172.21.15.82:0/3387511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2015ab50 con 0x7f3c2015da80 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c25fe8640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800cda0 tx=0x7f3c1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18003ba0 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c18003d40 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2015ae30 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18004240 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.683+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2015b3f0 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2004efa0 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c180043e0 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c11ffb640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 0x7f3c04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3c18085fb0 con 0x7f3c2015da80 2024-04-25T10:03:14.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c257e7640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 0x7f3c04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.687+0000 7f3c257e7640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 0x7f3c04069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c007fd0 tx=0x7f3c1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.691+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c18056ba0 con 0x7f3c2015da80 2024-04-25T10:03:14.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.839+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c200cf180 con 0x7f3c2015da80 2024-04-25T10:03:14.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c11ffb640 1 -- 172.21.15.82:0/3387511 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f3c180564a0 con 0x7f3c2015da80 2024-04-25T10:03:14.850 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:14.850 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 msgr2=0x7f3c04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 0x7f3c04069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c007fd0 tx=0x7f3c1c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 msgr2=0x7f3c200c74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800cda0 tx=0x7f3c1800a660 comp rx=0 tx=0).stop 2024-04-25T10:03:14.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 shutdown_connections 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c04066f60 0x7f3c04069420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c2015da80 0x7f3c200c74b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c200d1cd0 0x7f3c200c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 --2- 172.21.15.82:0/3387511 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c200d07f0 0x7f3c2015bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 >> 172.21.15.82:0/3387511 conn(0x7f3c2008b000 msgr2=0x7f3c200ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 shutdown_connections 2024-04-25T10:03:14.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.843+0000 7f3c27a4b640 1 -- 172.21.15.82:0/3387511 wait complete. 2024-04-25T10:03:14.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:14.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:14.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.983+0000 7f349c072640 1 Processor -- start 2024-04-25T10:03:14.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349c072640 1 -- start start 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349c072640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f349415ecb0 0x7f34940c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349c072640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34940ca4c0 con 0x7f34940cf590 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349c072640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34940cd690 con 0x7f34940c9ef0 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349c072640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34940ca780 con 0x7f349415ecb0 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349960d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f349415ecb0 0x7f34940c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3499e0e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f34940cf590 0x7f34940cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41270/0 (socket says 172.21.15.82:41270) 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3499e0e640 1 -- 172.21.15.82:0/3389013447 learned_addr learned my addr 172.21.15.82:0/3389013447 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349960d640 1 -- 172.21.15.82:0/3389013447 >> v1:172.21.15.196:6789/0 conn(0x7f34940c9ef0 legacy=0x7f34940ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349960d640 1 -- 172.21.15.82:0/3389013447 >> v1:172.21.15.82:6789/0 conn(0x7f34940cf590 legacy=0x7f34940cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349960d640 1 -- 172.21.15.82:0/3389013447 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940cd970 con 0x7f349415ecb0 2024-04-25T10:03:14.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3498e0c640 1 -- 172.21.15.82:0/3389013447 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488551869 0 0) 0x7f34940ca4c0 con 0x7f34940cf590 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f349960d640 1 --2- 172.21.15.82:0/3389013447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34940c99b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f34840097c0 tx=0x7f348401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94624046ab4d0ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3498e0c640 1 -- 172.21.15.82:0/3389013447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840b0070 con 0x7f349415ecb0 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3498e0c640 1 -- 172.21.15.82:0/3389013447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34840ab600 con 0x7f349415ecb0 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.987+0000 7f3498e0c640 1 -- 172.21.15.82:0/3389013447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840ab9d0 con 0x7f349415ecb0 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/3389013447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 msgr2=0x7f34940c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 --2- 172.21.15.82:0/3389013447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34940c99b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f34840097c0 tx=0x7f348401c460 comp rx=0 tx=0).stop 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/3389013447 shutdown_connections 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 --2- 172.21.15.82:0/3389013447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34940c99b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/3389013447 >> 172.21.15.82:0/3389013447 conn(0x7f349406ae60 msgr2=0x7f349406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/3389013447 shutdown_connections 2024-04-25T10:03:14.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/3389013447 wait complete. 2024-04-25T10:03:14.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 Processor -- start 2024-04-25T10:03:14.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- start start 2024-04-25T10:03:15.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34940c9ef0 0x7f34940b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34940cf590 0x7f3494071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34940cdce0 con 0x7f349415ecb0 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34940cd440 con 0x7f34940cf590 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3494168f10 con 0x7f34940c9ef0 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40746/0 (socket says 172.21.15.82:40746) 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 -- 172.21.15.82:0/594679449 learned_addr learned my addr 172.21.15.82:0/594679449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349960d640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34940cf590 0x7f3494071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f3499e0e640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34940c9ef0 0x7f34940b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 -- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34940c9ef0 msgr2=0x7f34940b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34940c9ef0 0x7f34940b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 -- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34940cf590 msgr2=0x7f3494071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34940cf590 0x7f3494071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 -- 172.21.15.82:0/594679449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34940725a0 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349a60f640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b970 tx=0x7f348c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c019070 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c00cb40 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c014440 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34940b0550 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.991+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349406e3e0 con 0x7f349415ecb0 2024-04-25T10:03:15.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349404efa0 con 0x7f349415ecb0 2024-04-25T10:03:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f348c01c070 con 0x7f349415ecb0 2024-04-25T10:03:15.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f348a7fc640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 0x7f3478069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f348c085070 con 0x7f349415ecb0 2024-04-25T10:03:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f3499e0e640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 0x7f3478069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.995+0000 7f3499e0e640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 0x7f3478069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f70 tx=0x7f3490007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:14.999+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f348c055c60 con 0x7f349415ecb0 2024-04-25T10:03:15.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.159+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34940b4170 con 0x7f349415ecb0 2024-04-25T10:03:15.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.159+0000 7f348a7fc640 1 -- 172.21.15.82:0/594679449 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f348c055560 con 0x7f349415ecb0 2024-04-25T10:03:15.168 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 msgr2=0x7f3478069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 0x7f3478069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3490000f70 tx=0x7f3490007c10 comp rx=0 tx=0).stop 2024-04-25T10:03:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 msgr2=0x7f34941f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b970 tx=0x7f348c00be30 comp rx=0 tx=0).stop 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 shutdown_connections 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3478066f60 0x7f3478069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f349415ecb0 0x7f34941f7300 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f34940cf590 0x7f3494071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 --2- 172.21.15.82:0/594679449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f34940c9ef0 0x7f34940b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 >> 172.21.15.82:0/594679449 conn(0x7f349406ae60 msgr2=0x7f3494152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 shutdown_connections 2024-04-25T10:03:15.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.163+0000 7f349c072640 1 -- 172.21.15.82:0/594679449 wait complete. 2024-04-25T10:03:15.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:15.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 Processor -- start 2024-04-25T10:03:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 -- start start 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83fc15da80 0x7f83fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 0x7f83fc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc06be60 con 0x7f83fc0d0620 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc0d0a70 con 0x7f83fc15da80 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f8402221640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc06c440 con 0x7f83fc0d1cd0 2024-04-25T10:03:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83fc15da80 0x7f83fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83fc15da80 0x7f83fc0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40748/0 (socket says 172.21.15.82:40748) 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 -- 172.21.15.82:0/3619865566 learned_addr learned my addr 172.21.15.82:0/3619865566 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83f3fff640 1 --2- 172.21.15.82:0/3619865566 >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 0x7f83fc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 -- 172.21.15.82:0/3619865566 >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 msgr2=0x7f83fc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 --2- 172.21.15.82:0/3619865566 >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 0x7f83fc0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 -- 172.21.15.82:0/3619865566 >> v1:172.21.15.196:6789/0 conn(0x7f83fc0d0620 legacy=0x7f83fc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 -- 172.21.15.82:0/3619865566 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc15de60 con 0x7f83fc15da80 2024-04-25T10:03:15.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83f3fff640 1 --2- 172.21.15.82:0/3619865566 >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 0x7f83fc0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.307+0000 7f83fb7fe640 1 --2- 172.21.15.82:0/3619865566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc15da80 0x7f83fc0d4230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d893cb4f3bc5e220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83faffd640 1 -- 172.21.15.82:0/3619865566 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0b0070 con 0x7f83fc15da80 2024-04-25T10:03:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83faffd640 1 -- 172.21.15.82:0/3619865566 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec0ab600 con 0x7f83fc15da80 2024-04-25T10:03:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83faffd640 1 -- 172.21.15.82:0/3619865566 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0ab9d0 con 0x7f83fc15da80 2024-04-25T10:03:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/3619865566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc15da80 msgr2=0x7f83fc0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- 172.21.15.82:0/3619865566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc15da80 0x7f83fc0d4230 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f83ec009870 tx=0x7f83ec01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/3619865566 shutdown_connections 2024-04-25T10:03:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- 172.21.15.82:0/3619865566 >> v2:172.21.15.196:3300/0 conn(0x7f83fc0d1cd0 0x7f83fc0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- 172.21.15.82:0/3619865566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc15da80 0x7f83fc0d4230 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/3619865566 >> 172.21.15.82:0/3619865566 conn(0x7f83fc08b000 msgr2=0x7f83fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/3619865566 shutdown_connections 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/3619865566 wait complete. 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 Processor -- start 2024-04-25T10:03:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- start start 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83fc0d1cd0 0x7f83fc1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 0x7f83fc15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc16a730 con 0x7f83fc0d0620 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc16a5b0 con 0x7f83fc0d1cd0 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc16a8b0 con 0x7f83fc15c630 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40754/0 (socket says 172.21.15.82:40754) 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 -- 172.21.15.82:0/2743872128 learned_addr learned my addr 172.21.15.82:0/2743872128 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83f3fff640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83fc0d1cd0 0x7f83fc1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fbfff640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 0x7f83fc15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 -- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 msgr2=0x7f83fc15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 0x7f83fc15cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 -- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83fc0d1cd0 msgr2=0x7f83fc1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83fc0d1cd0 0x7f83fc1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 -- 172.21.15.82:0/2743872128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc157cf0 con 0x7f83fc0d0620 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fbfff640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 0x7f83fc15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83fb7fe640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ae040 tx=0x7f83ec019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0bb070 con 0x7f83fc0d0620 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec0b6430 con 0x7f83fc0d0620 2024-04-25T10:03:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc15e100 con 0x7f83fc0d0620 2024-04-25T10:03:15.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0b0040 con 0x7f83fc0d0620 2024-04-25T10:03:15.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.311+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc15e6c0 con 0x7f83fc0d0620 2024-04-25T10:03:15.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc04efa0 con 0x7f83fc0d0620 2024-04-25T10:03:15.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83ec0be070 con 0x7f83fc0d0620 2024-04-25T10:03:15.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f83f8ff9640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f83f3fff640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f83ec11eac0 con 0x7f83fc0d0620 2024-04-25T10:03:15.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.315+0000 7f83f3fff640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0cbea0 tx=0x7f83e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.319+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f83ec0160d0 con 0x7f83fc0d0620 2024-04-25T10:03:15.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.475+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83fc160650 con 0x7f83fc0d0620 2024-04-25T10:03:15.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f83f8ff9640 1 -- 172.21.15.82:0/2743872128 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83ec0ef610 con 0x7f83fc0d0620 2024-04-25T10:03:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 msgr2=0x7f83dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0cbea0 tx=0x7f83e4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 msgr2=0x7f83fc150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0ae040 tx=0x7f83ec019280 comp rx=0 tx=0).stop 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 shutdown_connections 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83fc15c630 0x7f83fc15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83fc0d1cd0 0x7f83fc1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 --2- 172.21.15.82:0/2743872128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83fc0d0620 0x7f83fc150890 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 >> 172.21.15.82:0/2743872128 conn(0x7f83fc08b000 msgr2=0x7f83fc0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 shutdown_connections 2024-04-25T10:03:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.479+0000 7f8402221640 1 -- 172.21.15.82:0/2743872128 wait complete. 2024-04-25T10:03:15.503 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:15.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 Processor -- start 2024-04-25T10:03:15.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 -- start start 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3cc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd3cc15b800 0x7fd3cc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0cbd20 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cff20 con 0x7fd3cc15b800 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d450d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cc930 con 0x7fd3cc15b420 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd3cc0cfad0 0x7fd3cc15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41274/0 (socket says 172.21.15.82:41274) 2024-04-25T10:03:15.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 -- 172.21.15.82:0/2325317975 learned_addr learned my addr 172.21.15.82:0/2325317975 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/2325317975 >> v2:172.21.15.196:3300/0 conn(0x7fd3cc15b800 0x7fd3cc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d22a9640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3cc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224524188 0 0) 0x7fd3cc0cbd20 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b8003530 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 208748818 0 0) 0x7fd3b8003530 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 msgr2=0x7fd3cc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3cc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 >> v2:172.21.15.196:3300/0 conn(0x7fd3cc15b800 msgr2=0x7fd3cc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 --2- 172.21.15.82:0/2325317975 >> v2:172.21.15.196:3300/0 conn(0x7fd3cc15b800 0x7fd3cc0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0cc000 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fd3c4001a00 con 0x7fd3cc0cfad0 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 >> v1:172.21.15.82:6789/0 conn(0x7fd3cc0cfad0 legacy=0x7fd3cc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cff20 con 0x7fd3b8004d50 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cc930 con 0x7fd3b8003a30 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b800b9d0 con 0x7fd3cc15b420 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d22a9640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 -- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 msgr2=0x7fd3b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 -- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3b8003a30 msgr2=0x7fd3b8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3b8003a30 0x7fd3b8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d2aaa640 1 -- 172.21.15.82:0/2325317975 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8003530 con 0x7fd3b8004d50 2024-04-25T10:03:15.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.607+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc15b420 0x7fd3b8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3b80051d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400a880 tx=0x7fd3c401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f782df058401d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c400a4a0 con 0x7fd3b8004d50 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c40ad650 con 0x7fd3b8004d50 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d12a7640 1 -- 172.21.15.82:0/2325317975 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c40ada40 con 0x7fd3b8004d50 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/2325317975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 msgr2=0x7fd3b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 --2- 172.21.15.82:0/2325317975 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3b80051d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd3c400a880 tx=0x7fd3c401d3a0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d2aaa640 1 -- 172.21.15.82:0/2325317975 reap_dead start 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/2325317975 shutdown_connections 2024-04-25T10:03:15.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/2325317975 >> 172.21.15.82:0/2325317975 conn(0x7fd3cc06ae60 msgr2=0x7fd3cc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/2325317975 shutdown_connections 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/2325317975 wait complete. 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 Processor -- start 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- start start 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 0x7fd3cc15de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 0x7fd3cc15e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cb9f0 con 0x7fd3b8004d50 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cc5c0 con 0x7fd3cc150b90 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0cadb0 con 0x7fd3cc0c7700 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40774/0 (socket says 172.21.15.82:40774) 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 -- 172.21.15.82:0/846363415 learned_addr learned my addr 172.21.15.82:0/846363415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 0x7fd3cc15e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 0x7fd3cc15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 -- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 msgr2=0x7fd3cc15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 0x7fd3cc15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 -- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 msgr2=0x7fd3cc15e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 0x7fd3cc15e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 -- 172.21.15.82:0/846363415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0c8890 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 0x7fd3cc15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d2aaa640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 0x7fd3cc15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d22a9640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8011590 tx=0x7fd3c8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8015930 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c8015ad0 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc0c8b70 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c800f460 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.611+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc0c9130 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc04efa0 con 0x7fd3b8004d50 2024-04-25T10:03:15.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3c800f600 con 0x7fd3b8004d50 2024-04-25T10:03:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3c2ffd640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd3c8085da0 con 0x7fd3b8004d50 2024-04-25T10:03:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.615+0000 7fd3d1aa8640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 0x7fd3b0069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c7ae0 tx=0x7fd3bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.619+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3c8019030 con 0x7fd3b8004d50 2024-04-25T10:03:15.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.779+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3cc06b5f0 con 0x7fd3b8004d50 2024-04-25T10:03:15.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.779+0000 7fd3c2ffd640 1 -- 172.21.15.82:0/846363415 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd3c8056990 con 0x7fd3b8004d50 2024-04-25T10:03:15.787 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:15.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.779+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 msgr2=0x7fd3b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.779+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 0x7fd3b0069450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0c7ae0 tx=0x7fd3bc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 msgr2=0x7fd3cc1f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd3c8011590 tx=0x7fd3c8011a50 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 shutdown_connections 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f90 0x7fd3b0069450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0c7700 0x7fd3cc15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc150b90 0x7fd3cc15e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 --2- 172.21.15.82:0/846363415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3b8004d50 0x7fd3cc1f3bc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 >> 172.21.15.82:0/846363415 conn(0x7fd3cc06ae60 msgr2=0x7fd3cc06e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 shutdown_connections 2024-04-25T10:03:15.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.783+0000 7fd3d450d640 1 -- 172.21.15.82:0/846363415 wait complete. 2024-04-25T10:03:15.805 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:15.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 Processor -- start 2024-04-25T10:03:15.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 -- start start 2024-04-25T10:03:15.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f808415b420 0x7f80840ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f80840cfca0 0x7f8084072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80840cbd20 con 0x7f808415b800 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840cc930 con 0x7f80840cfca0 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808bbea640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80840cc000 con 0x7f808415b420 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f8089185640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f808415b800 0x7f80840cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41284/0 (socket says 172.21.15.82:41284) 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f8089185640 1 -- 172.21.15.82:0/3745177698 learned_addr learned my addr 172.21.15.82:0/3745177698 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808a187640 1 --2- 172.21.15.82:0/3745177698 >> v2:172.21.15.82:3300/0 conn(0x7f80840cfca0 0x7f8084072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f8089986640 1 --2- 172.21.15.82:0/3745177698 >> v2:172.21.15.196:3300/0 conn(0x7f808415b420 0x7f80840ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808a187640 1 -- 172.21.15.82:0/3745177698 >> v2:172.21.15.196:3300/0 conn(0x7f808415b420 msgr2=0x7f80840ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808a187640 1 --2- 172.21.15.82:0/3745177698 >> v2:172.21.15.196:3300/0 conn(0x7f808415b420 0x7f80840ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808a187640 1 -- 172.21.15.82:0/3745177698 >> v1:172.21.15.82:6789/0 conn(0x7f808415b800 legacy=0x7f80840cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f808a187640 1 -- 172.21.15.82:0/3745177698 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cc2e0 con 0x7f80840cfca0 2024-04-25T10:03:15.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.919+0000 7f8088984640 1 -- 172.21.15.82:0/3745177698 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145408257 0 0) 0x7f80840cbd20 con 0x7f808415b800 2024-04-25T10:03:15.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808a187640 1 --2- 172.21.15.82:0/3745177698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80840cfca0 0x7f8084072000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f80840d0180 tx=0x7f807c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7d3adafb514b2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8088984640 1 -- 172.21.15.82:0/3745177698 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b2070 con 0x7f80840cfca0 2024-04-25T10:03:15.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8088984640 1 -- 172.21.15.82:0/3745177698 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0a5960 con 0x7f80840cfca0 2024-04-25T10:03:15.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8088984640 1 -- 172.21.15.82:0/3745177698 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0a5d30 con 0x7f80840cfca0 2024-04-25T10:03:15.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3745177698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80840cfca0 msgr2=0x7f8084072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- 172.21.15.82:0/3745177698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80840cfca0 0x7f8084072000 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f80840d0180 tx=0x7f807c01c900 comp rx=0 tx=0).stop 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3745177698 shutdown_connections 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- 172.21.15.82:0/3745177698 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80840cfca0 0x7f8084072000 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- 172.21.15.82:0/3745177698 >> v2:172.21.15.196:3300/0 conn(0x7f808415b420 0x7f80840ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3745177698 >> 172.21.15.82:0/3745177698 conn(0x7f808406ae60 msgr2=0x7f808406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3745177698 shutdown_connections 2024-04-25T10:03:15.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3745177698 wait complete. 2024-04-25T10:03:15.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 Processor -- start 2024-04-25T10:03:15.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- start start 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80840cfca0 0x7f80840affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 0x7f8084159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840caf30 con 0x7f808415b420 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80840cadb0 con 0x7f808415b800 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8084072d50 con 0x7f80840cfca0 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40788/0 (socket says 172.21.15.82:40788) 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 -- 172.21.15.82:0/3843177199 learned_addr learned my addr 172.21.15.82:0/3843177199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808a187640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 0x7f8084159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 -- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80840cfca0 msgr2=0x7f80840affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80840cfca0 0x7f80840affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 -- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 msgr2=0x7f8084159880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 0x7f8084159880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 -- 172.21.15.82:0/3843177199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840cd940 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808a187640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 0x7f8084159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f8089185640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f807400c410 tx=0x7f807400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740074e0 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80740047f0 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80840cdc20 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074007680 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.923+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80840ce1e0 con 0x7f808415b420 2024-04-25T10:03:15.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808404efa0 con 0x7f808415b420 2024-04-25T10:03:15.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8074011070 con 0x7f808415b420 2024-04-25T10:03:15.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f807a7fc640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f80740850a0 con 0x7f808415b420 2024-04-25T10:03:15.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f8089986640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.927+0000 7f8089986640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 0x7f8068069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80800098f0 tx=0x7f8080008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:15.931+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8074055cc0 con 0x7f808415b420 2024-04-25T10:03:16.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.087+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f80840b2180 con 0x7f808415b420 2024-04-25T10:03:16.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.087+0000 7f807a7fc640 1 -- 172.21.15.82:0/3843177199 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80740555c0 con 0x7f808415b420 2024-04-25T10:03:16.097 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:16.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 msgr2=0x7f8068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 0x7f8068069420 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f80800098f0 tx=0x7f8080008040 comp rx=0 tx=0).stop 2024-04-25T10:03:16.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 msgr2=0x7f80840ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f807400c410 tx=0x7f807400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 shutdown_connections 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8068066f60 0x7f8068069420 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f808415b800 0x7f8084159880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f808415b420 0x7f80840ccf30 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 --2- 172.21.15.82:0/3843177199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f80840cfca0 0x7f80840affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 >> 172.21.15.82:0/3843177199 conn(0x7f808406ae60 msgr2=0x7f808406ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 shutdown_connections 2024-04-25T10:03:16.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.091+0000 7f808bbea640 1 -- 172.21.15.82:0/3843177199 wait complete. 2024-04-25T10:03:16.116 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 Processor -- start 2024-04-25T10:03:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 -- start start 2024-04-25T10:03:16.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc144ca0 0x7fd2fc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2fc06ec60 0x7fd2fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0ccc70 con 0x7fd2fc0ce510 2024-04-25T10:03:16.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0cc2b0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd300add640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc06e290 con 0x7fd2fc144ca0 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd2fad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2fc06ec60 0x7fd2fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd2fad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2fc06ec60 0x7fd2fc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40796/0 (socket says 172.21.15.82:40796) 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd2fad9d640 1 -- 172.21.15.82:0/2608344124 learned_addr learned my addr 172.21.15.82:0/2608344124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.223+0000 7fd2f3fff640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc144ca0 0x7fd2fc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2fad9d640 1 -- 172.21.15.82:0/2608344124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc144ca0 msgr2=0x7fd2fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2fad9d640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc144ca0 0x7fd2fc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2fad9d640 1 -- 172.21.15.82:0/2608344124 >> v1:172.21.15.196:6789/0 conn(0x7fd2fc0ce510 legacy=0x7fd2fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2fad9d640 1 -- 172.21.15.82:0/2608344124 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc06dc40 con 0x7fd2fc06ec60 2024-04-25T10:03:16.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2fad9d640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc142680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e50 tx=0x7fd2f401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bec4495397b63f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2f9d9b640 1 -- 172.21.15.82:0/2608344124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40b4070 con 0x7fd2fc06ec60 2024-04-25T10:03:16.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2f9d9b640 1 -- 172.21.15.82:0/2608344124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f40af400 con 0x7fd2fc06ec60 2024-04-25T10:03:16.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd2f9d9b640 1 -- 172.21.15.82:0/2608344124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f40af7d0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 -- 172.21.15.82:0/2608344124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 msgr2=0x7fd2fc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc142680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009e50 tx=0x7fd2f401f210 comp rx=0 tx=0).stop 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 -- 172.21.15.82:0/2608344124 shutdown_connections 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc144ca0 0x7fd2fc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 --2- 172.21.15.82:0/2608344124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc142680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 -- 172.21.15.82:0/2608344124 >> 172.21.15.82:0/2608344124 conn(0x7fd2fc0b11a0 msgr2=0x7fd2fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 -- 172.21.15.82:0/2608344124 shutdown_connections 2024-04-25T10:03:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.227+0000 7fd300add640 1 -- 172.21.15.82:0/2608344124 wait complete. 2024-04-25T10:03:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 Processor -- start 2024-04-25T10:03:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- start start 2024-04-25T10:03:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0a9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc0aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1451f0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0cc900 con 0x7fd2fc144ca0 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc145370 con 0x7fd2fc0ce510 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40804/0 (socket says 172.21.15.82:40804) 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 -- 172.21.15.82:0/3504837603 learned_addr learned my addr 172.21.15.82:0/3504837603 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fad9d640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc0aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f3fff640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0a9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 -- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc0ce510 msgr2=0x7fd2fc0a9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0a9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 -- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2fc144ca0 msgr2=0x7fd2fc0aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc0aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 -- 172.21.15.82:0/3504837603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc1e6fe0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2fa59c640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00c420 tx=0x7fd2ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0052d0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec005470 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc1e72c0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec007260 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1e7880 con 0x7fd2fc06ec60 2024-04-25T10:03:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc04efa0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2ec007400 con 0x7fd2fc06ec60 2024-04-25T10:03:16.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 0x7fd2dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fd2ec0848c0 con 0x7fd2fc06ec60 2024-04-25T10:03:16.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.231+0000 7fd2f3fff640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 0x7fd2dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.235+0000 7fd2f3fff640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 0x7fd2dc069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b09e0 tx=0x7fd2e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.235+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2ec055540 con 0x7fd2fc06ec60 2024-04-25T10:03:16.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.395+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2fc0b0f80 con 0x7fd2fc06ec60 2024-04-25T10:03:16.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.395+0000 7fd2f2ffd640 1 -- 172.21.15.82:0/3504837603 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd2ec054e40 con 0x7fd2fc06ec60 2024-04-25T10:03:16.403 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 msgr2=0x7fd2dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 0x7fd2dc069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc0b09e0 tx=0x7fd2e4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 msgr2=0x7fd2fc0a9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec00c420 tx=0x7fd2ec00c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 shutdown_connections 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd2dc066f60 0x7fd2dc069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0a9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc0aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 --2- 172.21.15.82:0/3504837603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2fc06ec60 0x7fd2fc0a9000 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 >> 172.21.15.82:0/3504837603 conn(0x7fd2fc0b11a0 msgr2=0x7fd2fc0c75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 shutdown_connections 2024-04-25T10:03:16.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.399+0000 7fd300add640 1 -- 172.21.15.82:0/3504837603 wait complete. 2024-04-25T10:03:16.421 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi196.front.sepia.ceph.com... 2024-04-25T10:03:16.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:16.421 DEBUG:teuthology.orchestra.run.smithi196:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T10:03:16.436 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:16.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:03:16.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:16.436 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:03:16.483 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:16.483 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:16.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 Processor -- start 2024-04-25T10:03:16.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 -- start start 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb80815da80 0x7fb8080d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80806be60 con 0x7fb8080d1cd0 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb80806c440 con 0x7fb80815da80 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80e8de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80815de60 con 0x7fb8080d07f0 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80ce7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb80815da80 0x7fb8080d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb80815da80 0x7fb8080d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:59588/0 (socket says 172.21.15.82:59588) 2024-04-25T10:03:16.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80ce7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:53272/0 (socket says 172.21.15.82:53272) 2024-04-25T10:03:16.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 -- 172.21.15.82:0/2215512158 learned_addr learned my addr 172.21.15.82:0/2215512158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:16.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 -- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 msgr2=0x7fb80806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 -- 172.21.15.82:0/2215512158 >> v1:172.21.15.196:6789/0 conn(0x7fb8080d1cd0 legacy=0x7fb8080d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb807fff640 1 -- 172.21.15.82:0/2215512158 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080d2450 con 0x7fb80815da80 2024-04-25T10:03:16.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.595+0000 7fb80ce7b640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:16.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb807fff640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080d4230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80098a0 tx=0x7fb7f801c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b74490920fcf4961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb8077fe640 1 -- 172.21.15.82:0/2215512158 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80a7070 con 0x7fb80815da80 2024-04-25T10:03:16.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb8077fe640 1 -- 172.21.15.82:0/2215512158 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f80b0070 con 0x7fb80815da80 2024-04-25T10:03:16.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb8077fe640 1 -- 172.21.15.82:0/2215512158 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80ab890 con 0x7fb80815da80 2024-04-25T10:03:16.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 msgr2=0x7fb8080d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080d4230 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80098a0 tx=0x7fb7f801c510 comp rx=0 tx=0).stop 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- 172.21.15.82:0/2215512158 shutdown_connections 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb80806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- 172.21.15.82:0/2215512158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080d4230 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- 172.21.15.82:0/2215512158 >> 172.21.15.82:0/2215512158 conn(0x7fb80808b000 msgr2=0x7fb8080d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- 172.21.15.82:0/2215512158 shutdown_connections 2024-04-25T10:03:16.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- 172.21.15.82:0/2215512158 wait complete. 2024-04-25T10:03:16.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 Processor -- start 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- start start 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb808155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806d7a0 con 0x7fb8080d1cd0 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806d620 con 0x7fb80815da80 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb80e8de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80806d920 con 0x7fb8080d07f0 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb7fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb7fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40816/0 (socket says 172.21.15.82:40816) 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.599+0000 7fb7fffff640 1 -- 172.21.15.82:0/1467517540 learned_addr learned my addr 172.21.15.82:0/1467517540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:16.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb807fff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb808155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb80ce7b640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 -- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 msgr2=0x7fb808155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb808155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 -- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 msgr2=0x7fb8080ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 -- 172.21.15.82:0/1467517540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080b63b0 con 0x7fb8080d1cd0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb7fffff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000c460 tx=0x7fb7f000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0017070 con 0x7fb8080d1cd0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f00052f0 con 0x7fb8080d1cd0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8080b6650 con 0x7fb8080d1cd0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f0003670 con 0x7fb8080d1cd0 2024-04-25T10:03:16.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8080b6c10 con 0x7fb8080d1cd0 2024-04-25T10:03:16.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80804efa0 con 0x7fb8080d1cd0 2024-04-25T10:03:16.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7f001a070 con 0x7fb8080d1cd0 2024-04-25T10:03:16.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 0x7fb7ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fb7f0084840 con 0x7fb8080d1cd0 2024-04-25T10:03:16.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb807fff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 0x7fb7ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.603+0000 7fb807fff640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 0x7fb7ec069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80af000 tx=0x7fb7f80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.607+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7f0055430 con 0x7fb8080d1cd0 2024-04-25T10:03:16.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.799+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb808155b90 con 0x7fb8080d1cd0 2024-04-25T10:03:16.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.799+0000 7fb8057fa640 1 -- 172.21.15.82:0/1467517540 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7fb7f0054d30 con 0x7fb8080d1cd0 2024-04-25T10:03:16.808 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:16.808 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:16.808 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 msgr2=0x7fb7ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 0x7fb7ec069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb7f80af000 tx=0x7fb7f80a7000 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 msgr2=0x7fb80815c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb7f000c460 tx=0x7fb7f000c920 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 shutdown_connections 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7ec066f60 0x7fb7ec069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb80815da80 0x7fb8080ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb8080d1cd0 0x7fb80815c570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 --2- 172.21.15.82:0/1467517540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb8080d07f0 0x7fb808155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 >> 172.21.15.82:0/1467517540 conn(0x7fb80808b000 msgr2=0x7fb8080b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 shutdown_connections 2024-04-25T10:03:16.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.803+0000 7fb80e8de640 1 -- 172.21.15.82:0/1467517540 wait complete. 2024-04-25T10:03:16.826 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:16.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.923+0000 7fc1a7910640 1 Processor -- start 2024-04-25T10:03:16.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.923+0000 7fc1a7910640 1 -- start start 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a7910640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 0x7fc1a00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a7910640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a006be60 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a7910640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1a015de60 con 0x7fc1a00d07f0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a7910640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a006c440 con 0x7fc1a015da80 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a4eab640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc1a00d1cd0 0x7fc1a00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41300/0 (socket says 172.21.15.82:41300) 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a4eab640 1 -- 172.21.15.82:0/1641788806 learned_addr learned my addr 172.21.15.82:0/1641788806 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1607184362 0 0) 0x7fc1a006be60 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc18c003530 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 0x7fc1a00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3236871489 0 0) 0x7fc18c003530 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 msgr2=0x7fc1a00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 0x7fc1a00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 >> v1:172.21.15.196:6789/0 conn(0x7fc1a00d07f0 legacy=0x7fc1a006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00d2450 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc190001800 con 0x7fc1a00d1cd0 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 >> v1:172.21.15.82:6789/0 conn(0x7fc1a00d1cd0 legacy=0x7fc1a00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c003a60 0x7fc18c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 0x7fc18c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c004e70 0x7fc18c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a006c440 con 0x7fc18c003a60 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0054f0 con 0x7fc18c004e70 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c00bf10 con 0x7fc1a015da80 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c003a60 0x7fc18c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a5ead640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c004e70 0x7fc18c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 -- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 msgr2=0x7fc18c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc1a015da80 0x7fc18c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 -- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c004e70 msgr2=0x7fc18c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c004e70 0x7fc18c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.927+0000 7fc1a56ac640 1 -- 172.21.15.82:0/1641788806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c003530 con 0x7fc18c003a60 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c003a60 0x7fc18c003ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ba30 tx=0x7fc19c01c6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee127cc1c732dac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c00b600 con 0x7fc18c003a60 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c0b1070 con 0x7fc18c003a60 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc197fff640 1 -- 172.21.15.82:0/1641788806 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c0a6040 con 0x7fc18c003a60 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/1641788806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c003a60 msgr2=0x7fc18c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 --2- 172.21.15.82:0/1641788806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c003a60 0x7fc18c003ea0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ba30 tx=0x7fc19c01c6b0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a5ead640 1 -- 172.21.15.82:0/1641788806 reap_dead start 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/1641788806 shutdown_connections 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/1641788806 >> 172.21.15.82:0/1641788806 conn(0x7fc1a008b000 msgr2=0x7fc1a00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/1641788806 shutdown_connections 2024-04-25T10:03:16.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/1641788806 wait complete. 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 Processor -- start 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- start start 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c003a60 0x7fc1a00c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1a00c96a0 0x7fc1a00c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a006d970 con 0x7fc1a00b8c00 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a006d7f0 con 0x7fc1a00c96a0 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a006daf0 con 0x7fc18c003a60 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40832/0 (socket says 172.21.15.82:40832) 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 -- 172.21.15.82:0/906603944 learned_addr learned my addr 172.21.15.82:0/906603944 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a5ead640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1a00c96a0 0x7fc1a00c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c003a60 0x7fc1a00c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 -- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c003a60 msgr2=0x7fc1a00c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c003a60 0x7fc1a00c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 -- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1a00c96a0 msgr2=0x7fc1a00c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1a00c96a0 0x7fc1a00c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 -- 172.21.15.82:0/906603944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a00c9270 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a4eab640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c700 tx=0x7fc19000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190017070 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc190007270 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a0157f40 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1900057b0 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.931+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a00c81d0 con 0x7fc1a00b8c00 2024-04-25T10:03:16.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a004efa0 con 0x7fc1a00b8c00 2024-04-25T10:03:16.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc19001a070 con 0x7fc1a00b8c00 2024-04-25T10:03:16.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc195ffb640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 0x7fc184069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fc190084990 con 0x7fc1a00b8c00 2024-04-25T10:03:16.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 0x7fc184069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.935+0000 7fc1a56ac640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 0x7fc184069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0b0000 tx=0x7fc19c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:16.939+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc190055580 con 0x7fc1a00b8c00 2024-04-25T10:03:17.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.083+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a00b1ac0 con 0x7fc1a00b8c00 2024-04-25T10:03:17.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc195ffb640 1 -- 172.21.15.82:0/906603944 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7fc190054e80 con 0x7fc1a00b8c00 2024-04-25T10:03:17.095 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:17.095 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 msgr2=0x7fc184069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 0x7fc184069420 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0b0000 tx=0x7fc19c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 msgr2=0x7fc1a0157a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc19000c700 tx=0x7fc19000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 shutdown_connections 2024-04-25T10:03:17.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc184066f60 0x7fc184069420 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc1a00c96a0 0x7fc1a00c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1a00b8c00 0x7fc1a0157a00 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 --2- 172.21.15.82:0/906603944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c003a60 0x7fc1a00c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 >> 172.21.15.82:0/906603944 conn(0x7fc1a008b000 msgr2=0x7fc1a00ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 shutdown_connections 2024-04-25T10:03:17.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.087+0000 7fc1a7910640 1 -- 172.21.15.82:0/906603944 wait complete. 2024-04-25T10:03:17.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:05.053780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:05.688047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:09.428238+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:17.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 Processor -- start 2024-04-25T10:03:17.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 -- start start 2024-04-25T10:03:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3c280b2ac0 0x7f3c280cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c280ce070 con 0x7f3c2815e1e0 2024-04-25T10:03:17.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c280ce350 con 0x7f3c2815a600 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2f196640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c280ce630 con 0x7f3c280b2ac0 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2d733640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3c2815e1e0 0x7f3c2815e5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41302/0 (socket says 172.21.15.82:41302) 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2d733640 1 -- 172.21.15.82:0/2918725603 learned_addr learned my addr 172.21.15.82:0/2918725603 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> v2:172.21.15.196:3300/0 conn(0x7f3c280b2ac0 0x7f3c280cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243675901 0 0) 0x7f3c280ce070 con 0x7f3c2815e1e0 2024-04-25T10:03:17.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c10003530 con 0x7f3c2815e1e0 2024-04-25T10:03:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423533862 0 0) 0x7f3c10003530 con 0x7f3c2815e1e0 2024-04-25T10:03:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 >> v2:172.21.15.196:3300/0 conn(0x7f3c280b2ac0 msgr2=0x7f3c280cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 --2- 172.21.15.82:0/2918725603 >> v2:172.21.15.196:3300/0 conn(0x7f3c280b2ac0 0x7f3c280cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 >> v1:172.21.15.196:6789/0 conn(0x7f3c2815a600 legacy=0x7f3c280cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c280ce910 con 0x7f3c2815e1e0 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3c20001a00 con 0x7f3c2815e1e0 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> v2:172.21.15.196:3300/0 conn(0x7f3c280b2ac0 0x7f3c280cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 >> v1:172.21.15.82:6789/0 conn(0x7f3c2815e1e0 legacy=0x7f3c2815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c10003a30 0x7f3c10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c100051d0 0x7f3c1000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280ce630 con 0x7f3c10004810 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000e2f0 con 0x7f3c10003a30 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000e4f0 con 0x7f3c100051d0 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2cf32640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c10003a30 0x7f3c10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 -- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c100051d0 msgr2=0x7f3c1000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c100051d0 0x7f3c1000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 -- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c10003a30 msgr2=0x7f3c10003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c10003a30 0x7f3c10003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c27fff640 1 -- 172.21.15.82:0/2918725603 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10003530 con 0x7f3c10004810 2024-04-25T10:03:17.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.227+0000 7f3c2d733640 1 -- 172.21.15.82:0/2918725603 reap_dead start 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c27fff640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c10004c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009cb0 tx=0x7f3c1c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b172680d5653bf9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0b2070 con 0x7f3c10004810 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c0a5960 con 0x7f3c10004810 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c277fe640 1 -- 172.21.15.82:0/2918725603 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c009880 con 0x7f3c10004810 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 msgr2=0x7f3c10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c10004c90 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009cb0 tx=0x7f3c1c01cca0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- 172.21.15.82:0/2918725603 shutdown_connections 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 --2- 172.21.15.82:0/2918725603 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c10004c90 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- 172.21.15.82:0/2918725603 >> 172.21.15.82:0/2918725603 conn(0x7f3c280b1f10 msgr2=0x7f3c280b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- 172.21.15.82:0/2918725603 shutdown_connections 2024-04-25T10:03:17.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- 172.21.15.82:0/2918725603 wait complete. 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 Processor -- start 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.231+0000 7f3c2f196640 1 -- start start 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 0x7f3c281efe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c280b59e0 0x7f3c2806ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815ab60 con 0x7f3c10004810 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2815a9e0 con 0x7f3c280b43f0 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c280cee30 con 0x7f3c280b59e0 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40844/0 (socket says 172.21.15.82:40844) 2024-04-25T10:03:17.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 -- 172.21.15.82:0/782694204 learned_addr learned my addr 172.21.15.82:0/782694204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c27fff640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 0x7f3c281efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2d733640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c280b59e0 0x7f3c2806ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 -- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c280b59e0 msgr2=0x7f3c2806ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c280b59e0 0x7f3c2806ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 -- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 msgr2=0x7f3c281efe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 0x7f3c281efe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 -- 172.21.15.82:0/782694204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28156fc0 con 0x7f3c10004810 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c27fff640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 0x7f3c281efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2cf32640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800e4b0 tx=0x7f3c1800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180052e0 con 0x7f3c10004810 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c18018070 con 0x7f3c10004810 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c281f0350 con 0x7f3c10004810 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18003840 con 0x7f3c10004810 2024-04-25T10:03:17.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c280b4940 con 0x7f3c10004810 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2804efa0 con 0x7f3c10004810 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c180039e0 con 0x7f3c10004810 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 0x7f3c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f3c18085040 con 0x7f3c10004810 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c27fff640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 0x7f3c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.235+0000 7f3c27fff640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 0x7f3c14069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0af000 tx=0x7f3c1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.239+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c281f0350 con 0x7f3c10004810 2024-04-25T10:03:17.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.399+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c280d0650 con 0x7f3c10004810 2024-04-25T10:03:17.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.399+0000 7f3c257fa640 1 -- 172.21.15.82:0/782694204 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c18055cc0 con 0x7f3c10004810 2024-04-25T10:03:17.408 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:17.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 msgr2=0x7f3c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 0x7f3c14069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c0af000 tx=0x7f3c1c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 msgr2=0x7f3c28156a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3c1800e4b0 tx=0x7f3c1800e970 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 shutdown_connections 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c14066f60 0x7f3c14069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c280b59e0 0x7f3c2806ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c280b43f0 0x7f3c281efe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 --2- 172.21.15.82:0/782694204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c10004810 0x7f3c28156a80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 >> 172.21.15.82:0/782694204 conn(0x7f3c280b1f10 msgr2=0x7f3c280d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 shutdown_connections 2024-04-25T10:03:17.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.403+0000 7f3c2f196640 1 -- 172.21.15.82:0/782694204 wait complete. 2024-04-25T10:03:17.426 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:17.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 Processor -- start 2024-04-25T10:03:17.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 -- start start 2024-04-25T10:03:17.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa67815da80 0x7fa6780d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa67806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67806be60 con 0x7fa6780d1cd0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67806c440 con 0x7fa67815da80 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa6800c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815de60 con 0x7fa6780d07f0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67d65f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa6780d1cd0 0x7fa6780d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41314/0 (socket says 172.21.15.82:41314) 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67d65f640 1 -- 172.21.15.82:0/2963592799 learned_addr learned my addr 172.21.15.82:0/2963592799 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67de60640 1 --2- 172.21.15.82:0/2963592799 >> v2:172.21.15.196:3300/0 conn(0x7fa67815da80 0x7fa6780d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa67806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936744324 0 0) 0x7fa67806be60 con 0x7fa6780d1cd0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa664003530 con 0x7fa6780d1cd0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 403535281 0 0) 0x7fa664003530 con 0x7fa6780d1cd0 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 msgr2=0x7fa67806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa67806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 >> v2:172.21.15.196:3300/0 conn(0x7fa67815da80 msgr2=0x7fa6780d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 --2- 172.21.15.82:0/2963592799 >> v2:172.21.15.196:3300/0 conn(0x7fa67815da80 0x7fa6780d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780d2450 con 0x7fa6780d1cd0 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa6680016f0 con 0x7fa6780d1cd0 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 >> v1:172.21.15.82:6789/0 conn(0x7fa6780d1cd0 legacy=0x7fa6780d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa664003a30 0x7fa664003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa664004d50 0x7fa6640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67806c440 con 0x7fa664004d50 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815de60 con 0x7fa664003a30 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa66400ba00 con 0x7fa6780d07f0 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa664004d50 0x7fa6640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67de60640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa664003a30 0x7fa664003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67d65f640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 -- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 msgr2=0x7fa664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa664004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 -- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa664003a30 msgr2=0x7fa664003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa664003a30 0x7fa664003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67e661640 1 -- 172.21.15.82:0/2963592799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa664003530 con 0x7fa664004d50 2024-04-25T10:03:17.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.543+0000 7fa67d65f640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6780d07f0 0x7fa664004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67e661640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa664004d50 0x7fa6640051d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa67808d8a0 tx=0x7fa67001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ffff1e918fc54a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700b2070 con 0x7fa664004d50 2024-04-25T10:03:17.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700a5c30 con 0x7fa664004d50 2024-04-25T10:03:17.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67ce5e640 1 -- 172.21.15.82:0/2963592799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670009c10 con 0x7fa664004d50 2024-04-25T10:03:17.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- 172.21.15.82:0/2963592799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa664004d50 msgr2=0x7fa6640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 --2- 172.21.15.82:0/2963592799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa664004d50 0x7fa6640051d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa67808d8a0 tx=0x7fa67001cc20 comp rx=0 tx=0).stop 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67e661640 1 -- 172.21.15.82:0/2963592799 reap_dead start 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- 172.21.15.82:0/2963592799 shutdown_connections 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- 172.21.15.82:0/2963592799 >> 172.21.15.82:0/2963592799 conn(0x7fa67808b000 msgr2=0x7fa6780d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- 172.21.15.82:0/2963592799 shutdown_connections 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- 172.21.15.82:0/2963592799 wait complete. 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 Processor -- start 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- start start 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67814fad0 0x7fa67809d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa67809d910 0x7fa67809f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780d26f0 con 0x7fa67809f850 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780d2120 con 0x7fa67814fad0 2024-04-25T10:03:17.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa6800c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780d2870 con 0x7fa67809d910 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67e661640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67e661640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40866/0 (socket says 172.21.15.82:40866) 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67e661640 1 -- 172.21.15.82:0/1612503583 learned_addr learned my addr 172.21.15.82:0/1612503583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67d65f640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa67809d910 0x7fa67809f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.547+0000 7fa67de60640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67814fad0 0x7fa67809d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 -- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa67809d910 msgr2=0x7fa67809f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa67809d910 0x7fa67809f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 -- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67814fad0 msgr2=0x7fa67809d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67814fad0 0x7fa67809d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 -- 172.21.15.82:0/1612503583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780a0060 con 0x7fa67809f850 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67e661640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6700b0040 tx=0x7fa670009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700bd070 con 0x7fa67809f850 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700a5a30 con 0x7fa67809f850 2024-04-25T10:03:17.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6780a0340 con 0x7fa67809f850 2024-04-25T10:03:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700b2040 con 0x7fa67809f850 2024-04-25T10:03:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6780a0900 con 0x7fa67809f850 2024-04-25T10:03:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67815c3a0 con 0x7fa67809f850 2024-04-25T10:03:17.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6700c0070 con 0x7fa67809f850 2024-04-25T10:03:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 0x7fa65c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7fa67011f190 con 0x7fa67809f850 2024-04-25T10:03:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67de60640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 0x7fa65c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.551+0000 7fa67de60640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 0x7fa65c069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ab0 tx=0x7fa674008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.555+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa670016930 con 0x7fa67809f850 2024-04-25T10:03:17.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.711+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa67809dcf0 con 0x7fa67809f850 2024-04-25T10:03:17.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.711+0000 7fa66e7fc640 1 -- 172.21.15.82:0/1612503583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6700ef8e0 con 0x7fa67809f850 2024-04-25T10:03:17.720 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 msgr2=0x7fa65c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 0x7fa65c069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa674009ab0 tx=0x7fa674008040 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 msgr2=0x7fa6780c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6700b0040 tx=0x7fa670009750 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 shutdown_connections 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa65c066f60 0x7fa65c069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa67809d910 0x7fa67809f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa67809f850 0x7fa6780c85f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 --2- 172.21.15.82:0/1612503583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa67814fad0 0x7fa67809d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 >> 172.21.15.82:0/1612503583 conn(0x7fa67808b000 msgr2=0x7fa678073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 shutdown_connections 2024-04-25T10:03:17.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.715+0000 7fa6800c4640 1 -- 172.21.15.82:0/1612503583 wait complete. 2024-04-25T10:03:17.738 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:17.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.839+0000 7f0bf149a640 1 Processor -- start 2024-04-25T10:03:17.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.839+0000 7f0bf149a640 1 -- start start 2024-04-25T10:03:17.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0bf149a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0bec0cba90 0x7f0bec0cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0bf149a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec071b70 con 0x7f0bec159bd0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0bf149a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bec071e50 con 0x7f0bec159fb0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0bf149a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bec072780 con 0x7f0bec0cba90 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0bec159bd0 0x7f0bec0c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41322/0 (socket says 172.21.15.82:41322) 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 -- 172.21.15.82:0/3038162623 learned_addr learned my addr 172.21.15.82:0/3038162623 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beb7fe640 1 --2- 172.21.15.82:0/3038162623 >> v2:172.21.15.196:3300/0 conn(0x7f0bec0cba90 0x7f0bec0cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377123914 0 0) 0x7f0bec071b70 con 0x7f0bec159bd0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bd4003530 con 0x7f0bec159bd0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1792380603 0 0) 0x7f0bd4003530 con 0x7f0bec159bd0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 >> v2:172.21.15.196:3300/0 conn(0x7f0bec0cba90 msgr2=0x7f0bec0cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 --2- 172.21.15.82:0/3038162623 >> v2:172.21.15.196:3300/0 conn(0x7f0bec0cba90 0x7f0bec0cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 >> v1:172.21.15.196:6789/0 conn(0x7f0bec159fb0 legacy=0x7f0bec0cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec072130 con 0x7f0bec159bd0 2024-04-25T10:03:17.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0be0001680 con 0x7f0bec159bd0 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 >> v1:172.21.15.82:6789/0 conn(0x7f0bec159bd0 legacy=0x7f0bec0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 0x7f0bd4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bd4004810 0x7f0bd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd40051d0 0x7f0bd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec072780 con 0x7f0bd4003a30 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd400c100 con 0x7f0bd4004810 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bd400c300 con 0x7f0bd40051d0 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 0x7f0bd4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 -- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd40051d0 msgr2=0x7f0bd400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd40051d0 0x7f0bd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 -- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bd4004810 msgr2=0x7f0bd4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bd4004810 0x7f0bd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 -- 172.21.15.82:0/3038162623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd4003530 con 0x7f0bd4003a30 2024-04-25T10:03:17.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beb7fe640 1 -- 172.21.15.82:0/3038162623 reap_dead start 2024-04-25T10:03:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0beaffd640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 0x7f0bd4003e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c210 tx=0x7f0be001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca282928fcc6d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be0007d40 con 0x7f0bd4003a30 2024-04-25T10:03:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be00a9070 con 0x7f0bd4003a30 2024-04-25T10:03:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.843+0000 7f0be9ffb640 1 -- 172.21.15.82:0/3038162623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00a5dd0 con 0x7f0bd4003a30 2024-04-25T10:03:17.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 msgr2=0x7f0bd4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 0x7f0bd4003e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0be000c210 tx=0x7f0be001cc30 comp rx=0 tx=0).stop 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/3038162623 shutdown_connections 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 --2- 172.21.15.82:0/3038162623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bd4003a30 0x7f0bd4003e50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/3038162623 >> 172.21.15.82:0/3038162623 conn(0x7f0bec06ae60 msgr2=0x7f0bec06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/3038162623 shutdown_connections 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/3038162623 wait complete. 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 Processor -- start 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- start start 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd4003a30 0x7f0bec0ad920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 0x7f0bec0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c9f40 con 0x7f0bec15c760 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0c9dc0 con 0x7f0bec15c2e0 2024-04-25T10:03:17.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bec0cb2d0 con 0x7f0bd4003a30 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40882/0 (socket says 172.21.15.82:40882) 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 -- 172.21.15.82:0/2789028142 learned_addr learned my addr 172.21.15.82:0/2789028142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 -- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd4003a30 msgr2=0x7f0bec0ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beaffd640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd4003a30 0x7f0bec0ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd4003a30 0x7f0bec0ad920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bea7fc640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 0x7f0bec0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 -- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 msgr2=0x7f0bec0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 0x7f0bec0c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 -- 172.21.15.82:0/2789028142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec0ade60 con 0x7f0bec15c760 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bea7fc640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 0x7f0bec0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0beb7fe640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0c9940 tx=0x7f0be400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400bcb0 con 0x7f0bec15c760 2024-04-25T10:03:17.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be401a070 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bec0acb80 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be4014640 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.847+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bec0ace40 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bec04efa0 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0be4013070 con 0x7f0bec15c760 2024-04-25T10:03:17.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0bd37fe640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0beaffd640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f0be40856e0 con 0x7f0bec15c760 2024-04-25T10:03:17.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.851+0000 7f0beaffd640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 0x7f0bd8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ab000 tx=0x7f0be00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:17.855+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0be40562d0 con 0x7f0bec15c760 2024-04-25T10:03:18.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bec06bfc0 con 0x7f0bec15c760 2024-04-25T10:03:18.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bd37fe640 1 -- 172.21.15.82:0/2789028142 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0be4055bd0 con 0x7f0bec15c760 2024-04-25T10:03:18.014 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 msgr2=0x7f0bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 0x7f0bd8069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0be00ab000 tx=0x7f0be00a7070 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 msgr2=0x7f0bec0c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0c9940 tx=0x7f0be400b480 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 shutdown_connections 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0bd8066f60 0x7f0bd8069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bec15c760 0x7f0bec0c58b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bec15c2e0 0x7f0bec0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 --2- 172.21.15.82:0/2789028142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bd4003a30 0x7f0bec0ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 >> 172.21.15.82:0/2789028142 conn(0x7f0bec06ae60 msgr2=0x7f0bec0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 shutdown_connections 2024-04-25T10:03:18.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.007+0000 7f0bf149a640 1 -- 172.21.15.82:0/2789028142 wait complete. 2024-04-25T10:03:18.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:18.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 Processor -- start 2024-04-25T10:03:18.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 -- start start 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10500ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f10500cf590 0x7f10500cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10500cbd20 con 0x7f105015f130 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105015ba90 con 0x7f10500cf590 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f1056177640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500cff30 con 0x7f105015b6b0 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f105015f130 0x7f105015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41336/0 (socket says 172.21.15.82:41336) 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104ffff640 1 -- 172.21.15.82:0/2352236815 learned_addr learned my addr 172.21.15.82:0/2352236815 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104effd640 1 --2- 172.21.15.82:0/2352236815 >> v2:172.21.15.196:3300/0 conn(0x7f10500cf590 0x7f10500cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10500ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760357107 0 0) 0x7f10500cbd20 con 0x7f105015f130 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038003530 con 0x7f105015f130 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3435996758 0 0) 0x7f1038003530 con 0x7f105015f130 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 msgr2=0x7f10500ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10500ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 >> v2:172.21.15.196:3300/0 conn(0x7f10500cf590 msgr2=0x7f10500cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 --2- 172.21.15.82:0/2352236815 >> v2:172.21.15.196:3300/0 conn(0x7f10500cf590 0x7f10500cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500cc000 con 0x7f105015f130 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1048001a00 con 0x7f105015f130 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 >> v1:172.21.15.82:6789/0 conn(0x7f105015f130 legacy=0x7f105015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f1038003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038004810 0x7f1038004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105015ba90 con 0x7f1038003a30 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103800b9d0 con 0x7f1038004810 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103800bb50 con 0x7f105015b6b0 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f1038003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104effd640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038004810 0x7f1038004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 -- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 msgr2=0x7f10380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104ffff640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f105015b6b0 0x7f10380051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 -- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038004810 msgr2=0x7f1038004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1038004810 0x7f1038004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.147+0000 7f104f7fe640 1 -- 172.21.15.82:0/2352236815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038003530 con 0x7f1038003a30 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f1038003e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1044009b10 tx=0x7f104401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904311ed859216ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440b2070 con 0x7f1038003a30 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10440a5530 con 0x7f1038003a30 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f104e7fc640 1 -- 172.21.15.82:0/2352236815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440096b0 con 0x7f1038003a30 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- 172.21.15.82:0/2352236815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 msgr2=0x7f1038003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 --2- 172.21.15.82:0/2352236815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f1038003e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1044009b10 tx=0x7f104401ca40 comp rx=0 tx=0).stop 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f104ffff640 1 -- 172.21.15.82:0/2352236815 reap_dead start 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- 172.21.15.82:0/2352236815 shutdown_connections 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- 172.21.15.82:0/2352236815 >> 172.21.15.82:0/2352236815 conn(0x7f105006ae60 msgr2=0x7f105006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- 172.21.15.82:0/2352236815 shutdown_connections 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- 172.21.15.82:0/2352236815 wait complete. 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 Processor -- start 2024-04-25T10:03:18.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.151+0000 7f1056177640 1 -- start start 2024-04-25T10:03:18.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1050096d00 0x7f105015e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1050156580 0x7f10500aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500cadb0 con 0x7f1038003a30 2024-04-25T10:03:18.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500cba30 con 0x7f1050096d00 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500caf30 con 0x7f1050156580 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40906/0 (socket says 172.21.15.82:40906) 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 -- 172.21.15.82:0/2454603744 learned_addr learned my addr 172.21.15.82:0/2454603744 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 -- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1050156580 msgr2=0x7f10500aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1050156580 0x7f10500aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 -- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1050096d00 msgr2=0x7f105015e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1050096d00 0x7f105015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 -- 172.21.15.82:0/2454603744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105015f630 con 0x7f1038003a30 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104f7fe640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1044009a30 tx=0x7f1038005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440bc030 con 0x7f1038003a30 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104400f000 con 0x7f1038003a30 2024-04-25T10:03:18.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10500b40b0 con 0x7f1038003a30 2024-04-25T10:03:18.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440b2040 con 0x7f1038003a30 2024-04-25T10:03:18.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105015e9b0 con 0x7f1038003a30 2024-04-25T10:03:18.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105004efa0 con 0x7f1038003a30 2024-04-25T10:03:18.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10440bf070 con 0x7f1038003a30 2024-04-25T10:03:18.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f104411f130 con 0x7f1038003a30 2024-04-25T10:03:18.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104effd640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.155+0000 7f104effd640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 0x7f103c069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1050097120 tx=0x7f1040008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.159+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10440c7020 con 0x7f1038003a30 2024-04-25T10:03:18.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f1050158a70 con 0x7f1038003a30 2024-04-25T10:03:18.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1037fff640 1 -- 172.21.15.82:0/2454603744 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10440167c0 con 0x7f1038003a30 2024-04-25T10:03:18.326 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 msgr2=0x7f103c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 0x7f103c069450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1050097120 tx=0x7f1040008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 msgr2=0x7f10500b3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1044009a30 tx=0x7f1038005840 comp rx=0 tx=0).stop 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 shutdown_connections 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f103c066f90 0x7f103c069450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1050156580 0x7f10500aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1050096d00 0x7f105015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 --2- 172.21.15.82:0/2454603744 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1038003a30 0x7f10500b3b70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 >> 172.21.15.82:0/2454603744 conn(0x7f105006ae60 msgr2=0x7f10500ae180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 shutdown_connections 2024-04-25T10:03:18.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.319+0000 7f1056177640 1 -- 172.21.15.82:0/2454603744 wait complete. 2024-04-25T10:03:18.346 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 Processor -- start 2024-04-25T10:03:18.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 -- start start 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35f00cf420 0x7f35f00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f015bd30 con 0x7f35f00cfb80 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f00cc0d0 con 0x7f35f00cf420 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35f607a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f009ab10 con 0x7f35f015b950 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35f00cf420 0x7f35f00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35f00cf420 0x7f35f00cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40922/0 (socket says 172.21.15.82:40922) 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 -- 172.21.15.82:0/1315065779 learned_addr learned my addr 172.21.15.82:0/1315065779 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 -- 172.21.15.82:0/1315065779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 msgr2=0x7f35f009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35ef7fe640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f009a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 -- 172.21.15.82:0/1315065779 >> v1:172.21.15.196:6789/0 conn(0x7f35f00cfb80 legacy=0x7f35f006f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 -- 172.21.15.82:0/1315065779 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00cc370 con 0x7f35f00cf420 2024-04-25T10:03:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f00cce50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009870 tx=0x7f35d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d80f92f86e7683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.459+0000 7f35eeffd640 1 -- 172.21.15.82:0/1315065779 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d80b1070 con 0x7f35f00cf420 2024-04-25T10:03:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35eeffd640 1 -- 172.21.15.82:0/1315065779 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d80ac620 con 0x7f35f00cf420 2024-04-25T10:03:18.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35eeffd640 1 -- 172.21.15.82:0/1315065779 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d80ac9f0 con 0x7f35f00cf420 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/1315065779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 msgr2=0x7f35f00cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f00cce50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f35d8009870 tx=0x7f35d801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/1315065779 shutdown_connections 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f00cce50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- 172.21.15.82:0/1315065779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/1315065779 >> 172.21.15.82:0/1315065779 conn(0x7f35f00b30b0 msgr2=0x7f35f00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/1315065779 shutdown_connections 2024-04-25T10:03:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/1315065779 wait complete. 2024-04-25T10:03:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 Processor -- start 2024-04-25T10:03:18.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- start start 2024-04-25T10:03:18.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 0x7f35f00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f014fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00cd390 con 0x7f35f00cf420 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00cf880 con 0x7f35f00cfb80 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f00cb280 con 0x7f35f015b950 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40930/0 (socket says 172.21.15.82:40930) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 -- 172.21.15.82:0/2298844422 learned_addr learned my addr 172.21.15.82:0/2298844422 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 0x7f35f00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 -- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 msgr2=0x7f35f014fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35effff640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f014fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f014fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 -- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 msgr2=0x7f35f00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 0x7f35f00cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 -- 172.21.15.82:0/2298844422 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f00b3a00 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 0x7f35f00cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ef7fe640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35e000c820 tx=0x7f35e000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0015070 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e0004020 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f00c9f80 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e0004450 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.463+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f00ca540 con 0x7f35f00cf420 2024-04-25T10:03:18.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f004efa0 con 0x7f35f00cf420 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35e0007830 con 0x7f35f00cf420 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35ecff9640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 0x7f35d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f35e00851b0 con 0x7f35f00cf420 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 0x7f35d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.467+0000 7f35e6ffd640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 0x7f35d0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35d80ae000 tx=0x7f35d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.471+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35e0055e30 con 0x7f35f00cf420 2024-04-25T10:03:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35f00d13a0 con 0x7f35f00cf420 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35ecff9640 1 -- 172.21.15.82:0/2298844422 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f35e0055730 con 0x7f35f00cf420 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 msgr2=0x7f35d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 0x7f35d0069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f35d80ae000 tx=0x7f35d8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 msgr2=0x7f35f014e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f35e000c820 tx=0x7f35e000cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 shutdown_connections 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35d0066f60 0x7f35d0069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35f015b950 0x7f35f014fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35f00cfb80 0x7f35f00cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 --2- 172.21.15.82:0/2298844422 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35f00cf420 0x7f35f014e1f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 >> 172.21.15.82:0/2298844422 conn(0x7f35f00b30b0 msgr2=0x7f35f00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 shutdown_connections 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.663+0000 7f35f607a640 1 -- 172.21.15.82:0/2298844422 wait complete. 2024-04-25T10:03:18.690 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:18.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.791+0000 7f6a42305640 1 Processor -- start 2024-04-25T10:03:18.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.791+0000 7f6a42305640 1 -- start start 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a42305640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6a3c144ca0 0x7f6a3c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a42305640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c0ccc70 con 0x7f6a3c06ec60 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a42305640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a3c06dc40 con 0x7f6a3c0ce510 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a42305640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3c0cc2b0 con 0x7f6a3c144ca0 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a408a2640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a3c06ec60 0x7f6a3c142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41338/0 (socket says 172.21.15.82:41338) 2024-04-25T10:03:18.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a408a2640 1 -- 172.21.15.82:0/643149459 learned_addr learned my addr 172.21.15.82:0/643149459 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/643149459 >> v2:172.21.15.196:3300/0 conn(0x7f6a3c144ca0 0x7f6a3c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690097457 0 0) 0x7f6a3c0ccc70 con 0x7f6a3c06ec60 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a24003530 con 0x7f6a3c06ec60 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2675324314 0 0) 0x7f6a24003530 con 0x7f6a3c06ec60 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 >> v2:172.21.15.196:3300/0 conn(0x7f6a3c144ca0 msgr2=0x7f6a3c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 --2- 172.21.15.82:0/643149459 >> v2:172.21.15.196:3300/0 conn(0x7f6a3c144ca0 0x7f6a3c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 >> v1:172.21.15.196:6789/0 conn(0x7f6a3c0ce510 legacy=0x7f6a3c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c06df20 con 0x7f6a3c06ec60 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6a34001a00 con 0x7f6a3c06ec60 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 >> v1:172.21.15.82:6789/0 conn(0x7f6a3c06ec60 legacy=0x7f6a3c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a24003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a24004810 0x7f6a24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a240051d0 0x7f6a2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c0cc2b0 con 0x7f6a24003a30 2024-04-25T10:03:18.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400c0e0 con 0x7f6a24004810 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2400c2e0 con 0x7f6a240051d0 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a24003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a24004810 0x7f6a24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 -- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a240051d0 msgr2=0x7f6a2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a240051d0 0x7f6a2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 -- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a24004810 msgr2=0x7f6a24004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a24004810 0x7f6a24004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a3bfff640 1 -- 172.21.15.82:0/643149459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a24003530 con 0x7f6a24003a30 2024-04-25T10:03:18.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.795+0000 7f6a408a2640 1 -- 172.21.15.82:0/643149459 reap_dead start 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a24003e50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c001780 tx=0x7f6a2c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49a1f1232768b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0013a0 con 0x7f6a24003a30 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0052f0 con 0x7f6a24003a30 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3affd640 1 -- 172.21.15.82:0/643149459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c003840 con 0x7f6a24003a30 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 msgr2=0x7f6a24003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a24003e50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c001780 tx=0x7f6a2c01c400 comp rx=0 tx=0).stop 2024-04-25T10:03:18.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- 172.21.15.82:0/643149459 shutdown_connections 2024-04-25T10:03:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 --2- 172.21.15.82:0/643149459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a24003e50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- 172.21.15.82:0/643149459 >> 172.21.15.82:0/643149459 conn(0x7f6a3c0b11a0 msgr2=0x7f6a3c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- 172.21.15.82:0/643149459 shutdown_connections 2024-04-25T10:03:18.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- 172.21.15.82:0/643149459 wait complete. 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 Processor -- start 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- start start 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 0x7f6a3c0acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c0ad510 0x7f6a3c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c1451f0 con 0x7f6a24003a30 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c06e290 con 0x7f6a3c0ad510 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a42305640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c145370 con 0x7f6a3c0ab2b0 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40956/0 (socket says 172.21.15.82:40956) 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 -- 172.21.15.82:0/1355498316 learned_addr learned my addr 172.21.15.82:0/1355498316 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a408a2640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c0ad510 0x7f6a3c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 0x7f6a3c0acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 -- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 msgr2=0x7f6a3c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 0x7f6a3c0acfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 -- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c0ad510 msgr2=0x7f6a3c152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c0ad510 0x7f6a3c152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3bfff640 1 -- 172.21.15.82:0/1355498316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c13e450 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.799+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 0x7f6a3c0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a3bfff640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c01c230 tx=0x7f6a24005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007a40 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c0ba070 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3c155ad0 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c007270 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3c0ac210 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3c04efa0 con 0x7f6a24003a30 2024-04-25T10:03:18.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a2c005280 con 0x7f6a24003a30 2024-04-25T10:03:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 0x7f6a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 0x7f6a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f6a2c11f4d0 con 0x7f6a24003a30 2024-04-25T10:03:18.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.803+0000 7f6a3b7fe640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 0x7f6a28069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c155e90 tx=0x7f6a30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.807+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a2c016110 con 0x7f6a24003a30 2024-04-25T10:03:18.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.951+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a3c0a8f30 con 0x7f6a24003a30 2024-04-25T10:03:18.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a38ff9640 1 -- 172.21.15.82:0/1355498316 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f6a2c0b33b0 con 0x7f6a24003a30 2024-04-25T10:03:18.963 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:18.963 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 msgr2=0x7f6a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 0x7f6a28069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c155e90 tx=0x7f6a30008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 msgr2=0x7f6a3c13df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c01c230 tx=0x7f6a24005840 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 shutdown_connections 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a28066f60 0x7f6a28069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a3c0ab2b0 0x7f6a3c0acfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a3c0ad510 0x7f6a3c152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 --2- 172.21.15.82:0/1355498316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a24003a30 0x7f6a3c13df10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 >> 172.21.15.82:0/1355498316 conn(0x7f6a3c0b11a0 msgr2=0x7f6a3c0a85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 shutdown_connections 2024-04-25T10:03:18.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:18.955+0000 7f6a42305640 1 -- 172.21.15.82:0/1355498316 wait complete. 2024-04-25T10:03:18.980 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:18.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.087+0000 7feac05f2640 1 Processor -- start 2024-04-25T10:03:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 -- start start 2024-04-25T10:03:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feab815b950 0x7feab809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feab815bd30 con 0x7feab80cfb80 2024-04-25T10:03:19.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feab80cc0d0 con 0x7feab815b950 2024-04-25T10:03:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feac05f2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab809ab10 con 0x7feab80cf420 2024-04-25T10:03:19.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabdb8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabe38e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feab815b950 0x7feab809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabdb8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab80cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:53454/0 (socket says 172.21.15.82:53454) 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabdb8d640 1 -- 172.21.15.82:0/20958023 learned_addr learned my addr 172.21.15.82:0/20958023 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabe38e640 1 -- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 msgr2=0x7feab80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabe38e640 1 --2- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabe38e640 1 -- 172.21.15.82:0/20958023 >> v1:172.21.15.196:6789/0 conn(0x7feab80cfb80 legacy=0x7feab806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.091+0000 7feabe38e640 1 -- 172.21.15.82:0/20958023 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab80cc370 con 0x7feab815b950 2024-04-25T10:03:19.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feabe38e640 1 --2- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab809a480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feab40097c0 tx=0x7feab401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cb77194ec26d5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feabd38c640 1 -- 172.21.15.82:0/20958023 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40b2070 con 0x7feab815b950 2024-04-25T10:03:19.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feabd38c640 1 -- 172.21.15.82:0/20958023 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40ad680 con 0x7feab815b950 2024-04-25T10:03:19.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feabd38c640 1 -- 172.21.15.82:0/20958023 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40ada50 con 0x7feab815b950 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 msgr2=0x7feab809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab809a480 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feab40097c0 tx=0x7feab401c470 comp rx=0 tx=0).stop 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- 172.21.15.82:0/20958023 shutdown_connections 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- 172.21.15.82:0/20958023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab809a480 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- 172.21.15.82:0/20958023 >> 172.21.15.82:0/20958023 conn(0x7feab80b30b0 msgr2=0x7feab80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- 172.21.15.82:0/20958023 shutdown_connections 2024-04-25T10:03:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- 172.21.15.82:0/20958023 wait complete. 2024-04-25T10:03:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 Processor -- start 2024-04-25T10:03:19.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 -- start start 2024-04-25T10:03:19.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab8153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.095+0000 7feac05f2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab8154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab80cd390 con 0x7feab80cfb80 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab80d03a0 con 0x7feab815b950 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab815a9d0 con 0x7feab80cf420 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40968/0 (socket says 172.21.15.82:40968) 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 -- 172.21.15.82:0/1977914091 learned_addr learned my addr 172.21.15.82:0/1977914091 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabe38e640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab8153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabeb8f640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab8154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 -- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 msgr2=0x7feab8153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab8153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 -- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 msgr2=0x7feab8154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab8154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 -- 172.21.15.82:0/1977914091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8159dc0 con 0x7feab80cfb80 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabeb8f640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab8154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabdb8d640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feaa800e7f0 tx=0x7feaa800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa8017070 con 0x7feab80cfb80 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa800c940 con 0x7feab80cfb80 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab815a0a0 con 0x7feab80cfb80 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa800cd30 con 0x7feab80cfb80 2024-04-25T10:03:19.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab815a660 con 0x7feab80cfb80 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab804efa0 con 0x7feab80cfb80 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaa801a070 con 0x7feab80cfb80 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 0x7fea9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7feaa8085600 con 0x7feab80cfb80 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabe38e640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 0x7fea9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.099+0000 7feabe38e640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 0x7fea9c069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feab40af000 tx=0x7feab40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.103+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaa8055ba0 con 0x7feab80cfb80 2024-04-25T10:03:19.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feab80d13a0 con 0x7feab80cfb80 2024-04-25T10:03:19.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feaaeffd640 1 -- 172.21.15.82:0/1977914091 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7feaa8059c40 con 0x7feab80cfb80 2024-04-25T10:03:19.291 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:19.291 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:19.291 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 msgr2=0x7fea9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 0x7fea9c069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7feab40af000 tx=0x7feab40a7000 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 msgr2=0x7feab8154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feaa800e7f0 tx=0x7feaa800ecb0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 shutdown_connections 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea9c066f60 0x7fea9c069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feab815b950 0x7feab8154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feab80cfb80 0x7feab8154920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 --2- 172.21.15.82:0/1977914091 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feab80cf420 0x7feab8153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 >> 172.21.15.82:0/1977914091 conn(0x7feab80b30b0 msgr2=0x7feab806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 shutdown_connections 2024-04-25T10:03:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.283+0000 7feac05f2640 1 -- 172.21.15.82:0/1977914091 wait complete. 2024-04-25T10:03:19.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:19.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.407+0000 7f4c93ac1640 1 Processor -- start 2024-04-25T10:03:19.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 -- start start 2024-04-25T10:03:19.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c15da10 0x7f4c8c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c8c0b24a0 0x7f4c8c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 0x7f4c8c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c0ae860 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c8c15e210 con 0x7f4c8c071e90 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c93ac1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c15e4f0 con 0x7f4c8c15da10 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c8c0b24a0 0x7f4c8c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4c8c0b24a0 0x7f4c8c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40974/0 (socket says 172.21.15.82:40974) 2024-04-25T10:03:19.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 -- 172.21.15.82:0/3424108626 learned_addr learned my addr 172.21.15.82:0/3424108626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9205e640 1 --2- 172.21.15.82:0/3424108626 >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 0x7f4c8c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 -- 172.21.15.82:0/3424108626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c15da10 msgr2=0x7f4c8c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 --2- 172.21.15.82:0/3424108626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c15da10 0x7f4c8c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 -- 172.21.15.82:0/3424108626 >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 msgr2=0x7f4c8c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 --2- 172.21.15.82:0/3424108626 >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 0x7f4c8c1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 -- 172.21.15.82:0/3424108626 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c0ae4c0 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9205e640 1 --2- 172.21.15.82:0/3424108626 >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 0x7f4c8c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9105c640 1 --2- 172.21.15.82:0/3424108626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c0b24a0 0x7f4c8c071570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c070fc0 tx=0x7f4c7c01c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9040235d84a57109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9085b640 1 -- 172.21.15.82:0/3424108626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0b1070 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9085b640 1 -- 172.21.15.82:0/3424108626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c0ac440 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.411+0000 7f4c9085b640 1 -- 172.21.15.82:0/3424108626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0ac810 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/3424108626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c0b24a0 msgr2=0x7f4c8c071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/3424108626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c0b24a0 0x7f4c8c071570 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c070fc0 tx=0x7f4c7c01c140 comp rx=0 tx=0).stop 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/3424108626 shutdown_connections 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/3424108626 >> v2:172.21.15.196:3300/0 conn(0x7f4c8c071e90 0x7f4c8c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/3424108626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c0b24a0 0x7f4c8c071570 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/3424108626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c15da10 0x7f4c8c071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/3424108626 >> 172.21.15.82:0/3424108626 conn(0x7f4c8c0b11a0 msgr2=0x7f4c8c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/3424108626 shutdown_connections 2024-04-25T10:03:19.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/3424108626 wait complete. 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 Processor -- start 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- start start 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c0b24a0 0x7f4c8c1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c8c15da10 0x7f4c8c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0acf50 con 0x7f4c8c071e90 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0acdd0 con 0x7f4c8c15da10 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8c0ad0d0 con 0x7f4c8c0b24a0 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40976/0 (socket says 172.21.15.82:40976) 2024-04-25T10:03:19.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 -- 172.21.15.82:0/199269690 learned_addr learned my addr 172.21.15.82:0/199269690 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9205e640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c8c15da10 0x7f4c8c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9105c640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c0b24a0 0x7f4c8c1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 -- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c0b24a0 msgr2=0x7f4c8c1503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c0b24a0 0x7f4c8c1503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 -- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c8c15da10 msgr2=0x7f4c8c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c8c15da10 0x7f4c8c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 -- 172.21.15.82:0/199269690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c15fac0 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c9185d640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b410 tx=0x7f4c8800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88015020 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88011830 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c15fda0 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88011c20 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.415+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8c160360 con 0x7f4c8c071e90 2024-04-25T10:03:19.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c8c04efa0 con 0x7f4c8c071e90 2024-04-25T10:03:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c88018020 con 0x7f4c8c071e90 2024-04-25T10:03:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c827fc640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f4c880853b0 con 0x7f4c8c071e90 2024-04-25T10:03:19.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c9105c640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.419+0000 7f4c9105c640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 0x7f4c70069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0ae000 tx=0x7f4c7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.423+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c8801f020 con 0x7f4c8c071e90 2024-04-25T10:03:19.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.599+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c8c13d210 con 0x7f4c8c071e90 2024-04-25T10:03:19.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.599+0000 7f4c827fc640 1 -- 172.21.15.82:0/199269690 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4934 (secure 0 0 0) 0x7f4c880172c0 con 0x7f4c8c071e90 2024-04-25T10:03:19.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:19.608 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/692019525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":692019525},{"type":"v1","addr":"172.21.15.82:6835","nonce":692019525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2236802031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2236802031},{"type":"v1","addr":"172.21.15.82:6837","nonce":2236802031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4269,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/354743300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":354743300},{"type":"v1","addr":"172.21.15.196:6833","nonce":354743300}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:06.717111+0000","modified":"2024-04-25T10:02:29.823823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.196:6835/3824448217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3824448217},{"type":"v1","addr":"172.21.15.196:6835","nonce":3824448217}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:19.609 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 8 2024-04-25T10:03:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 msgr2=0x7f4c70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 0x7f4c70069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0ae000 tx=0x7f4c7c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 msgr2=0x7f4c8c13ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800b410 tx=0x7f4c8800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 shutdown_connections 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c70066f60 0x7f4c70069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c8c0b24a0 0x7f4c8c1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c8c15da10 0x7f4c8c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 --2- 172.21.15.82:0/199269690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c8c071e90 0x7f4c8c13ccd0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 >> 172.21.15.82:0/199269690 conn(0x7f4c8c0b11a0 msgr2=0x7f4c8c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 shutdown_connections 2024-04-25T10:03:19.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.603+0000 7f4c93ac1640 1 -- 172.21.15.82:0/199269690 wait complete. 2024-04-25T10:03:19.626 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 Processor -- start 2024-04-25T10:03:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 -- start start 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8bbc0d0870 0x7f8bbc0d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8bbc15dc30 0x7f8bbc0d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc0ce6b0 con 0x7f8bbc15d850 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc0ce990 con 0x7f8bbc15dc30 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bc10d4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc0cec70 con 0x7f8bbc0d0870 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbad9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8bbc15d850 0x7f8bbc0d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41342/0 (socket says 172.21.15.82:41342) 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbad9d640 1 -- 172.21.15.82:0/574372242 learned_addr learned my addr 172.21.15.82:0/574372242 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:19.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 --2- 172.21.15.82:0/574372242 >> v2:172.21.15.82:3300/0 conn(0x7f8bbc15dc30 0x7f8bbc0d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bba59c640 1 --2- 172.21.15.82:0/574372242 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc0d0870 0x7f8bbc0d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 -- 172.21.15.82:0/574372242 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc0d0870 msgr2=0x7f8bbc0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 --2- 172.21.15.82:0/574372242 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc0d0870 0x7f8bbc0d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 -- 172.21.15.82:0/574372242 >> v1:172.21.15.82:6789/0 conn(0x7f8bbc15d850 legacy=0x7f8bbc0d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 -- 172.21.15.82:0/574372242 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0cef50 con 0x7f8bbc15dc30 2024-04-25T10:03:19.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.727+0000 7f8bbb59e640 1 --2- 172.21.15.82:0/574372242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc0d2320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc08b930 tx=0x7f8bb401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b69b0b5e1fb8a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bb9d9b640 1 -- 172.21.15.82:0/574372242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40b2070 con 0x7f8bbc15dc30 2024-04-25T10:03:19.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bb9d9b640 1 -- 172.21.15.82:0/574372242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb40a5900 con 0x7f8bbc15dc30 2024-04-25T10:03:19.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bb9d9b640 1 -- 172.21.15.82:0/574372242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb40a5cd0 con 0x7f8bbc15dc30 2024-04-25T10:03:19.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/574372242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc15dc30 msgr2=0x7f8bbc0d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/574372242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc0d2320 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc08b930 tx=0x7f8bb401c930 comp rx=0 tx=0).stop 2024-04-25T10:03:19.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/574372242 shutdown_connections 2024-04-25T10:03:19.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/574372242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc0d2320 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/574372242 >> v2:172.21.15.196:3300/0 conn(0x7f8bbc0d0870 0x7f8bbc0d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/574372242 >> 172.21.15.82:0/574372242 conn(0x7f8bbc08b000 msgr2=0x7f8bbc0d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/574372242 shutdown_connections 2024-04-25T10:03:19.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/574372242 wait complete. 2024-04-25T10:03:19.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 Processor -- start 2024-04-25T10:03:19.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- start start 2024-04-25T10:03:19.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15d850 0x7f8bbc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc16a4c0 con 0x7f8bbc0d0870 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc16a340 con 0x7f8bbc15dc30 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc16a640 con 0x7f8bbc15d850 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40982/0 (socket says 172.21.15.82:40982) 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 -- 172.21.15.82:0/3146241682 learned_addr learned my addr 172.21.15.82:0/3146241682 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bba59c640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15d850 0x7f8bbc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbb59e640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 -- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15d850 msgr2=0x7f8bbc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15d850 0x7f8bbc15cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 -- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 msgr2=0x7f8bbc152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 -- 172.21.15.82:0/3146241682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc15c230 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbb59e640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bbad9d640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c450 tx=0x7f8bac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac017070 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac005cf0 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac007290 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc15c4d0 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.731+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc0b2390 con 0x7f8bbc0d0870 2024-04-25T10:03:19.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc04efa0 con 0x7f8bbc0d0870 2024-04-25T10:03:19.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bac01a070 con 0x7f8bbc0d0870 2024-04-25T10:03:19.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8ba37fe640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f8bac0846c0 con 0x7f8bbc0d0870 2024-04-25T10:03:19.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8bba59c640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.735+0000 7f8bba59c640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 0x7f8ba8069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc06c580 tx=0x7f8bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.739+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bac055340 con 0x7f8bbc0d0870 2024-04-25T10:03:19.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.887+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bbc0d0e60 con 0x7f8bbc0d0870 2024-04-25T10:03:19.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8ba37fe640 1 -- 172.21.15.82:0/3146241682 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18961 (secure 0 0 0) 0x7f8bac054c40 con 0x7f8bbc0d0870 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:19.898 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":27,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:02:48.561322+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:05.053780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:05.688047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:09.428238+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.26","mappings":[{"from":7,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 msgr2=0x7f8ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 0x7f8ba8069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc06c580 tx=0x7f8bb0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 msgr2=0x7f8bbc0cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c450 tx=0x7f8bac00c910 comp rx=0 tx=0).stop 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 shutdown_connections 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ba8066f60 0x7f8ba8069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bbc15d850 0x7f8bbc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bbc15dc30 0x7f8bbc152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 --2- 172.21.15.82:0/3146241682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bbc0d0870 0x7f8bbc0cac00 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 >> 172.21.15.82:0/3146241682 conn(0x7f8bbc08b000 msgr2=0x7f8bbc15e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 shutdown_connections 2024-04-25T10:03:19.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:19.891+0000 7f8bc10d4640 1 -- 172.21.15.82:0/3146241682 wait complete. 2024-04-25T10:03:19.915 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:20.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 Processor -- start 2024-04-25T10:03:20.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 -- start start 2024-04-25T10:03:20.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f19940cc890 0x7f19940af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f19940af3c0 0x7f19940afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19940ccd50 con 0x7f199406aef0 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19940cd080 con 0x7f19940cc890 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199c15b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19940cfbb0 con 0x7f19940af3c0 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f19996f6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f19940cc890 0x7f19940af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f1999ef7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f199406aef0 0x7f19940cd510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41350/0 (socket says 172.21.15.82:41350) 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f1999ef7640 1 -- 172.21.15.82:0/2748171855 learned_addr learned my addr 172.21.15.82:0/2748171855 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f199a6f8640 1 --2- 172.21.15.82:0/2748171855 >> v2:172.21.15.196:3300/0 conn(0x7f19940af3c0 0x7f19940afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f19996f6640 1 -- 172.21.15.82:0/2748171855 >> v2:172.21.15.196:3300/0 conn(0x7f19940af3c0 msgr2=0x7f19940afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f19996f6640 1 --2- 172.21.15.82:0/2748171855 >> v2:172.21.15.196:3300/0 conn(0x7f19940af3c0 0x7f19940afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f19996f6640 1 -- 172.21.15.82:0/2748171855 >> v1:172.21.15.82:6789/0 conn(0x7f199406aef0 legacy=0x7f19940cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f19996f6640 1 -- 172.21.15.82:0/2748171855 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19940cfe90 con 0x7f19940cc890 2024-04-25T10:03:20.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.031+0000 7f1998ef5640 1 -- 172.21.15.82:0/2748171855 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648843901 0 0) 0x7f19940ccd50 con 0x7f199406aef0 2024-04-25T10:03:20.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f19996f6640 1 --2- 172.21.15.82:0/2748171855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19940cc890 0x7f19940af910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19940b2c50 tx=0x7f198401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850db0eef781c61c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1998ef5640 1 -- 172.21.15.82:0/2748171855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840b1070 con 0x7f19940cc890 2024-04-25T10:03:20.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1998ef5640 1 -- 172.21.15.82:0/2748171855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19840ac440 con 0x7f19940cc890 2024-04-25T10:03:20.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1998ef5640 1 -- 172.21.15.82:0/2748171855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840ac810 con 0x7f19940cc890 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- 172.21.15.82:0/2748171855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19940cc890 msgr2=0x7f19940af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- 172.21.15.82:0/2748171855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19940cc890 0x7f19940af910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f19940b2c50 tx=0x7f198401c480 comp rx=0 tx=0).stop 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- 172.21.15.82:0/2748171855 shutdown_connections 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- 172.21.15.82:0/2748171855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19940cc890 0x7f19940af910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- 172.21.15.82:0/2748171855 >> v2:172.21.15.196:3300/0 conn(0x7f19940af3c0 0x7f19940afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- 172.21.15.82:0/2748171855 >> 172.21.15.82:0/2748171855 conn(0x7f19940b1fa0 msgr2=0x7f19940b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- 172.21.15.82:0/2748171855 shutdown_connections 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- 172.21.15.82:0/2748171855 wait complete. 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 Processor -- start 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- start start 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19940af3c0 0x7f1994158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19940cc890 0x7f19940ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199416ada0 con 0x7f199406aef0 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199416ac20 con 0x7f19940cc890 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f199c15b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199416af20 con 0x7f19940af3c0 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40998/0 (socket says 172.21.15.82:40998) 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 -- 172.21.15.82:0/3078174052 learned_addr learned my addr 172.21.15.82:0/3078174052 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 -- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19940af3c0 msgr2=0x7f1994158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19940af3c0 0x7f1994158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 -- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19940cc890 msgr2=0x7f19940ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.035+0000 7f1999ef7640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19940cc890 0x7f19940ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f1999ef7640 1 -- 172.21.15.82:0/3078174052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19940c7020 con 0x7f199406aef0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f1999ef7640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f199000c520 tx=0x7f199000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990018070 con 0x7f199406aef0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19900074e0 con 0x7f199406aef0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19940b3e70 con 0x7f199406aef0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990004a10 con 0x7f199406aef0 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19940b4430 con 0x7f199406aef0 2024-04-25T10:03:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199404efa0 con 0x7f199406aef0 2024-04-25T10:03:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f199001b070 con 0x7f199406aef0 2024-04-25T10:03:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 0x7f1978069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6764+0+0 (secure 0 0 0) 0x7f1990084710 con 0x7f199406aef0 2024-04-25T10:03:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f19996f6640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 0x7f1978069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.039+0000 7f19996f6640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 0x7f1978069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f19840ae000 tx=0x7f1984004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.043+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1990055390 con 0x7f199406aef0 2024-04-25T10:03:20.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.227+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f19940b6410 con 0x7f199406aef0 2024-04-25T10:03:20.236 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:03:20.227+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:20.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.231+0000 7f198a7fc640 1 -- 172.21.15.82:0/3078174052 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1990054c90 con 0x7f199406aef0 2024-04-25T10:03:20.239 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 msgr2=0x7f1978069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 0x7f1978069450 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f19840ae000 tx=0x7f1984004740 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 msgr2=0x7f1994155c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f199000c520 tx=0x7f199000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 shutdown_connections 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1978066f90 0x7f1978069450 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19940af3c0 0x7f1994158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19940cc890 0x7f19940ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 --2- 172.21.15.82:0/3078174052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f199406aef0 0x7f1994155c00 secure :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f199000c520 tx=0x7f199000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 >> 172.21.15.82:0/3078174052 conn(0x7f19940b1fa0 msgr2=0x7f19941562a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 shutdown_connections 2024-04-25T10:03:20.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.235+0000 7f199c15b640 1 -- 172.21.15.82:0/3078174052 wait complete. 2024-04-25T10:03:20.256 INFO:tasks.ceph.mds.d.smithi196.stderr: -4759> 2024-04-25T10:02:11.624+0000 7fe652c1c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:20.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:20.323 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:03:20.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 Processor -- start 2024-04-25T10:03:20.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 -- start start 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 0x7f204c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f204c0cf590 0x7f204c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0cbd20 con 0x7f204c15f130 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c15ba90 con 0x7f204c0cf590 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f205140b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cff30 con 0x7f204c15b6b0 2024-04-25T10:03:20.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f2043fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f204c0cf590 0x7f204c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f204b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f204c15f130 0x7f204c15f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41354/0 (socket says 172.21.15.82:41354) 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f204b7fe640 1 -- 172.21.15.82:0/2112357072 learned_addr learned my addr 172.21.15.82:0/2112357072 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f204affd640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 0x7f204c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f2043fff640 1 -- 172.21.15.82:0/2112357072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 msgr2=0x7f204c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f2043fff640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 0x7f204c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f2043fff640 1 -- 172.21.15.82:0/2112357072 >> v1:172.21.15.82:6789/0 conn(0x7f204c15f130 legacy=0x7f204c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f204a7fc640 1 -- 172.21.15.82:0/2112357072 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511692003 0 0) 0x7f204c0cbd20 con 0x7f204c15f130 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f2043fff640 1 -- 172.21.15.82:0/2112357072 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0cc000 con 0x7f204c0cf590 2024-04-25T10:03:20.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.375+0000 7f204affd640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 0x7f204c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f2043fff640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c0cf590 0x7f204c0cf9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20340097c0 tx=0x7f203401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4071920fbe936fae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204a7fc640 1 -- 172.21.15.82:0/2112357072 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340a7070 con 0x7f204c0cf590 2024-04-25T10:03:20.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204a7fc640 1 -- 172.21.15.82:0/2112357072 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20340b0070 con 0x7f204c0cf590 2024-04-25T10:03:20.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204a7fc640 1 -- 172.21.15.82:0/2112357072 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20340ab890 con 0x7f204c0cf590 2024-04-25T10:03:20.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- 172.21.15.82:0/2112357072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c0cf590 msgr2=0x7f204c0cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c0cf590 0x7f204c0cf9f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f20340097c0 tx=0x7f203401c470 comp rx=0 tx=0).stop 2024-04-25T10:03:20.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- 172.21.15.82:0/2112357072 shutdown_connections 2024-04-25T10:03:20.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c0cf590 0x7f204c0cf9f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- 172.21.15.82:0/2112357072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c15b6b0 0x7f204c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- 172.21.15.82:0/2112357072 >> 172.21.15.82:0/2112357072 conn(0x7f204c06ae60 msgr2=0x7f204c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- 172.21.15.82:0/2112357072 shutdown_connections 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- 172.21.15.82:0/2112357072 wait complete. 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 Processor -- start 2024-04-25T10:03:20.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- start start 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 0x7f204c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f204c15b6b0 0x7f204c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0ca900 con 0x7f204c15f130 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cba30 con 0x7f204c15b6b0 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f205140b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cadb0 con 0x7f204c0cf590 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41008/0 (socket says 172.21.15.82:41008) 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 -- 172.21.15.82:0/3581698772 learned_addr learned my addr 172.21.15.82:0/3581698772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f2043fff640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f204c15b6b0 0x7f204c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204affd640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 0x7f204c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 -- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 msgr2=0x7f204c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 0x7f204c1557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 -- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f204c15b6b0 msgr2=0x7f204c0c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f204c15b6b0 0x7f204c0c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 -- 172.21.15.82:0/3581698772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0ace90 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204affd640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 0x7f204c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.379+0000 7f204b7fe640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f204400c700 tx=0x7f204400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044018070 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204400ce10 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c0c5580 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204400a860 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c0c5b40 con 0x7f204c15f130 2024-04-25T10:03:20.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c04efa0 con 0x7f204c15f130 2024-04-25T10:03:20.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f204401b070 con 0x7f204c15f130 2024-04-25T10:03:20.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 0x7f202c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f204affd640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 0x7f202c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6850+0+0 (secure 0 0 0) 0x7f2044084920 con 0x7f204c15f130 2024-04-25T10:03:20.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.383+0000 7f204affd640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 0x7f202c069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203c008930 tx=0x7f203c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.387+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2044055540 con 0x7f204c15f130 2024-04-25T10:03:20.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:20.567+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f204c158a70 con 0x7f204c15f130 2024-04-25T10:03:21.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f20437fe640 1 -- 172.21.15.82:0/3581698772 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2044054e40 con 0x7f204c15f130 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 msgr2=0x7f202c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 0x7f202c069450 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f203c008930 tx=0x7f203c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 msgr2=0x7f204c15bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f204400c700 tx=0x7f204400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 shutdown_connections 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f202c066f90 0x7f202c069450 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f204c15f130 0x7f204c15bc10 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f204c15b6b0 0x7f204c0c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 --2- 172.21.15.82:0/3581698772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f204c0cf590 0x7f204c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 >> 172.21.15.82:0/3581698772 conn(0x7f204c06ae60 msgr2=0x7f204c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 shutdown_connections 2024-04-25T10:03:21.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.235+0000 7f205140b640 1 -- 172.21.15.82:0/3581698772 wait complete. 2024-04-25T10:03:21.265 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 Processor -- start 2024-04-25T10:03:21.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 -- start start 2024-04-25T10:03:21.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0cf510 0x7fbe3c0cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbe3c0ac170 0x7fbe3c0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0b28c0 con 0x7fbe3c0adc70 2024-04-25T10:03:21.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cfe90 con 0x7fbe3c0ac170 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe43220640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c15bb20 con 0x7fbe3c0cf510 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe40fbc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbe3c0adc70 0x7fbe3c06e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41362/0 (socket says 172.21.15.82:41362) 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe40fbc640 1 -- 172.21.15.82:0/1152171093 learned_addr learned my addr 172.21.15.82:0/1152171093 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe417bd640 1 --2- 172.21.15.82:0/1152171093 >> v2:172.21.15.82:3300/0 conn(0x7fbe3c0ac170 0x7fbe3c0ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe3b7fe640 1 -- 172.21.15.82:0/1152171093 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656600946 0 0) 0x7fbe3c0b28c0 con 0x7fbe3c0adc70 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe3b7fe640 1 -- 172.21.15.82:0/1152171093 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe28003530 con 0x7fbe3c0adc70 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.379+0000 7fbe3bfff640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0cf510 0x7fbe3c0cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe417bd640 1 -- 172.21.15.82:0/1152171093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0cf510 msgr2=0x7fbe3c0cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe417bd640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0cf510 0x7fbe3c0cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe417bd640 1 -- 172.21.15.82:0/1152171093 >> v1:172.21.15.82:6789/0 conn(0x7fbe3c0adc70 legacy=0x7fbe3c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe417bd640 1 -- 172.21.15.82:0/1152171093 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0b2b60 con 0x7fbe3c0ac170 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe417bd640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0ac600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097c0 tx=0x7fbe3401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f8269c95810d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe3b7fe640 1 -- 172.21.15.82:0/1152171093 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340b2070 con 0x7fbe3c0ac170 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe3b7fe640 1 -- 172.21.15.82:0/1152171093 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe340ad640 con 0x7fbe3c0ac170 2024-04-25T10:03:21.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe3b7fe640 1 -- 172.21.15.82:0/1152171093 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340ada10 con 0x7fbe3c0ac170 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 -- 172.21.15.82:0/1152171093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 msgr2=0x7fbe3c0ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0ac600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fbe340097c0 tx=0x7fbe3401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 -- 172.21.15.82:0/1152171093 shutdown_connections 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0ac600 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 --2- 172.21.15.82:0/1152171093 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0cf510 0x7fbe3c0cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 -- 172.21.15.82:0/1152171093 >> 172.21.15.82:0/1152171093 conn(0x7fbe3c06ae60 msgr2=0x7fbe3c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 -- 172.21.15.82:0/1152171093 shutdown_connections 2024-04-25T10:03:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.383+0000 7fbe43220640 1 -- 172.21.15.82:0/1152171093 wait complete. 2024-04-25T10:03:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 Processor -- start 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- start start 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0adc70 0x7fbe3c0c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe3c0cf510 0x7fbe3c098520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0b3c00 con 0x7fbe3c0ac170 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0b36e0 con 0x7fbe3c0cf510 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0c5ad0 con 0x7fbe3c0adc70 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41024/0 (socket says 172.21.15.82:41024) 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 -- 172.21.15.82:0/3089923926 learned_addr learned my addr 172.21.15.82:0/3089923926 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe417bd640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe3c0cf510 0x7fbe3c098520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe3bfff640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0adc70 0x7fbe3c0c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 -- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0adc70 msgr2=0x7fbe3c0c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0adc70 0x7fbe3c0c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 -- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe3c0cf510 msgr2=0x7fbe3c098520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe3c0cf510 0x7fbe3c098520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 -- 172.21.15.82:0/3089923926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c15e6f0 con 0x7fbe3c0ac170 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe40fbc640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00cb40 tx=0x7fbe2c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c017070 con 0x7fbe3c0ac170 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c00ac40 con 0x7fbe3c0ac170 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c012400 con 0x7fbe3c0ac170 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c15e9d0 con 0x7fbe3c0ac170 2024-04-25T10:03:21.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1521c0 con 0x7fbe3c0ac170 2024-04-25T10:03:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.387+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c151cd0 con 0x7fbe3c0ac170 2024-04-25T10:03:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.391+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe2c01a070 con 0x7fbe3c0ac170 2024-04-25T10:03:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.391+0000 7fbe397fa640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.391+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6850+0+0 (secure 0 0 0) 0x7fbe2c085370 con 0x7fbe3c0ac170 2024-04-25T10:03:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.391+0000 7fbe3bfff640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.391+0000 7fbe3bfff640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 0x7fbe1c069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c1f7f70 tx=0x7fbe24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.395+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe2c055e80 con 0x7fbe3c0ac170 2024-04-25T10:03:21.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:21.547+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe3c067c30 con 0x7fbe3c0ac170 2024-04-25T10:03:22.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.247+0000 7fbe397fa640 1 -- 172.21.15.82:0/3089923926 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbe2c055780 con 0x7fbe3c0ac170 2024-04-25T10:03:22.254 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:22.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 msgr2=0x7fbe1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 0x7fbe1c069420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c1f7f70 tx=0x7fbe24008040 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 msgr2=0x7fbe3c0c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00cb40 tx=0x7fbe2c00a060 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 shutdown_connections 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbe1c066f60 0x7fbe1c069420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbe3c0adc70 0x7fbe3c0c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbe3c0cf510 0x7fbe3c098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 --2- 172.21.15.82:0/3089923926 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbe3c0ac170 0x7fbe3c0c5590 secure :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00cb40 tx=0x7fbe2c00a060 comp rx=0 tx=0).stop 2024-04-25T10:03:22.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 >> 172.21.15.82:0/3089923926 conn(0x7fbe3c06ae60 msgr2=0x7fbe3c154120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 shutdown_connections 2024-04-25T10:03:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.251+0000 7fbe43220640 1 -- 172.21.15.82:0/3089923926 wait complete. 2024-04-25T10:03:22.281 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 Processor -- start 2024-04-25T10:03:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 -- start start 2024-04-25T10:03:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64600d12c0 0x7f64600d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d21f0 0x7f646006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f646015dd60 con 0x7f646015d980 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f646006c810 con 0x7f64600d12c0 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f6467b42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64600d45e0 con 0x7f64600d21f0 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64600d12c0 0x7f64600d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64600d12c0 0x7f64600d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41034/0 (socket says 172.21.15.82:41034) 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 -- 172.21.15.82:0/3290983078 learned_addr learned my addr 172.21.15.82:0/3290983078 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64660df640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d21f0 0x7f646006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 -- 172.21.15.82:0/3290983078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d21f0 msgr2=0x7f646006cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d21f0 0x7f646006cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 -- 172.21.15.82:0/3290983078 >> v1:172.21.15.196:6789/0 conn(0x7f646015d980 legacy=0x7f64600d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.399+0000 7f64650dd640 1 -- 172.21.15.82:0/3290983078 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600b02c0 con 0x7f64600d12c0 2024-04-25T10:03:22.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f64650dd640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d12c0 0x7f64600d1cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f646006c990 tx=0x7f645001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3208983edca17ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f64648dc640 1 -- 172.21.15.82:0/3290983078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b0070 con 0x7f64600d12c0 2024-04-25T10:03:22.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f64648dc640 1 -- 172.21.15.82:0/3290983078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64500ab600 con 0x7f64600d12c0 2024-04-25T10:03:22.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f64648dc640 1 -- 172.21.15.82:0/3290983078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500ab9d0 con 0x7f64600d12c0 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 -- 172.21.15.82:0/3290983078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d12c0 msgr2=0x7f64600d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d12c0 0x7f64600d1cb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f646006c990 tx=0x7f645001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 -- 172.21.15.82:0/3290983078 shutdown_connections 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d21f0 0x7f646006cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 --2- 172.21.15.82:0/3290983078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d12c0 0x7f64600d1cb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 -- 172.21.15.82:0/3290983078 >> 172.21.15.82:0/3290983078 conn(0x7f646008b000 msgr2=0x7f64600d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 -- 172.21.15.82:0/3290983078 shutdown_connections 2024-04-25T10:03:22.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.403+0000 7f6467b42640 1 -- 172.21.15.82:0/3290983078 wait complete. 2024-04-25T10:03:22.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 Processor -- start 2024-04-25T10:03:22.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- start start 2024-04-25T10:03:22.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d12c0 0x7f64600c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f646015d980 0x7f6460155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646006db70 con 0x7f64600d21f0 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f646006d9f0 con 0x7f646015d980 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f646006dcf0 con 0x7f64600d12c0 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41036/0 (socket says 172.21.15.82:41036) 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 -- 172.21.15.82:0/1373646489 learned_addr learned my addr 172.21.15.82:0/1373646489 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64658de640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d12c0 0x7f64600c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64660df640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f646015d980 0x7f6460155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 -- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d12c0 msgr2=0x7f64600c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d12c0 0x7f64600c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 -- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f646015d980 msgr2=0x7f6460155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f646015d980 0x7f6460155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 -- 172.21.15.82:0/1373646489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6460140880 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64650dd640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64500ae040 tx=0x7f645001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500ba020 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64500b6640 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6460140b60 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500b0040 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460141120 con 0x7f64600d21f0 2024-04-25T10:03:22.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460141320 con 0x7f64600d21f0 2024-04-25T10:03:22.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64500bd020 con 0x7f64600d21f0 2024-04-25T10:03:22.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6412+0+0 (secure 0 0 0) 0x7f645011f3f0 con 0x7f64600d21f0 2024-04-25T10:03:22.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.407+0000 7f64658de640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.411+0000 7f64658de640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 0x7f6444069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f645c0015b0 tx=0x7f645c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.415+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6460141320 con 0x7f64600d21f0 2024-04-25T10:03:22.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:22.575+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6460067c30 con 0x7f64600d21f0 2024-04-25T10:03:23.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.247+0000 7f64567fc640 1 -- 172.21.15.82:0/1373646489 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f64500c4030 con 0x7f64600d21f0 2024-04-25T10:03:23.258 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 msgr2=0x7f6444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 0x7f6444069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f645c0015b0 tx=0x7f645c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 msgr2=0x7f6460155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f64500ae040 tx=0x7f645001acf0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 shutdown_connections 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f646015d980 0x7f6460155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64600d21f0 0x7f6460155110 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 --2- 172.21.15.82:0/1373646489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64600d12c0 0x7f64600c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 >> 172.21.15.82:0/1373646489 conn(0x7f646008b000 msgr2=0x7f646015ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 shutdown_connections 2024-04-25T10:03:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.251+0000 7f6467b42640 1 -- 172.21.15.82:0/1373646489 wait complete. 2024-04-25T10:03:23.287 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T10:03:23.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 Processor -- start 2024-04-25T10:03:23.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 -- start start 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a3c15f330 0x7f0a3c15f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a3c160160 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a3c0cdd20 con 0x7f0a3c15f330 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a4304b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c15bbf0 con 0x7f0a3c15b810 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a3c15f330 0x7f0a3c15f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a415e8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0a3c15fcd0 0x7f0a3c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41370/0 (socket says 172.21.15.82:41370) 2024-04-25T10:03:23.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a415e8640 1 -- 172.21.15.82:0/3158563415 learned_addr learned my addr 172.21.15.82:0/3158563415 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3bfff640 1 -- 172.21.15.82:0/3158563415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 msgr2=0x7f0a3c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3bfff640 1 --2- 172.21.15.82:0/3158563415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3bfff640 1 -- 172.21.15.82:0/3158563415 >> v1:172.21.15.82:6789/0 conn(0x7f0a3c15fcd0 legacy=0x7f0a3c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3bfff640 1 -- 172.21.15.82:0/3158563415 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c0cd110 con 0x7f0a3c15f330 2024-04-25T10:03:23.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.407+0000 7f0a3b7fe640 1 -- 172.21.15.82:0/3158563415 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581086597 0 0) 0x7f0a340014a0 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a3bfff640 1 --2- 172.21.15.82:0/3158563415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15f330 0x7f0a3c15f790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097c0 tx=0x7f0a3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6dd1a868088828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a3b7fe640 1 -- 172.21.15.82:0/3158563415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300a7070 con 0x7f0a3c15f330 2024-04-25T10:03:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a3b7fe640 1 -- 172.21.15.82:0/3158563415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a300b0070 con 0x7f0a3c15f330 2024-04-25T10:03:23.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a3b7fe640 1 -- 172.21.15.82:0/3158563415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a300ab890 con 0x7f0a3c15f330 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- 172.21.15.82:0/3158563415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15f330 msgr2=0x7f0a3c15f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- 172.21.15.82:0/3158563415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15f330 0x7f0a3c15f790 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0a300097c0 tx=0x7f0a3001c470 comp rx=0 tx=0).stop 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- 172.21.15.82:0/3158563415 shutdown_connections 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- 172.21.15.82:0/3158563415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15f330 0x7f0a3c15f790 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- 172.21.15.82:0/3158563415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- 172.21.15.82:0/3158563415 >> 172.21.15.82:0/3158563415 conn(0x7f0a3c0b1fa0 msgr2=0x7f0a3c0b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- 172.21.15.82:0/3158563415 shutdown_connections 2024-04-25T10:03:23.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- 172.21.15.82:0/3158563415 wait complete. 2024-04-25T10:03:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 Processor -- start 2024-04-25T10:03:23.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- start start 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 0x7f0a3c14f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c0c3bb0 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c168670 con 0x7f0a3c15f330 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a4304b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3c159580 con 0x7f0a3c15b810 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a415e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a415e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41064/0 (socket says 172.21.15.82:41064) 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a415e8640 1 -- 172.21.15.82:0/662426186 learned_addr learned my addr 172.21.15.82:0/662426186 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:23.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a3bfff640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 0x7f0a3c14f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.411+0000 7f0a40de7640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 -- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 msgr2=0x7f0a3c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 -- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 msgr2=0x7f0a3c14f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 0x7f0a3c14f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 -- 172.21.15.82:0/662426186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c159860 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a3bfff640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 0x7f0a3c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a40de7640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a415e8640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c730 tx=0x7f0a3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a34018070 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3400ce40 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3c159b40 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3400a860 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3c15a100 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3c068340 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a3401b070 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 0x7f0a28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a40de7640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 0x7f0a28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0a34084620 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.415+0000 7f0a40de7640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 0x7f0a28069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00a8b0 tx=0x7f0a2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.419+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a3c160160 con 0x7f0a3c15fcd0 2024-04-25T10:03:23.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:23.583+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0a3c0caae0 con 0x7f0a3c15fcd0 2024-04-25T10:03:24.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.255+0000 7f0a397fa640 1 -- 172.21.15.82:0/662426186 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f0a340555a0 con 0x7f0a3c15fcd0 2024-04-25T10:03:24.266 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data_ec' removed 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 msgr2=0x7f0a28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 0x7f0a28069450 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c00a8b0 tx=0x7f0a2c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 msgr2=0x7f0a3c14f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0a3400c730 tx=0x7f0a3400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 shutdown_connections 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a28066f90 0x7f0a28069450 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a3c15fcd0 0x7f0a3c14f910 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a3c15f330 0x7f0a3c14f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 --2- 172.21.15.82:0/662426186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a3c15b810 0x7f0a3c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 >> 172.21.15.82:0/662426186 conn(0x7f0a3c0b1fa0 msgr2=0x7f0a3c0addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 shutdown_connections 2024-04-25T10:03:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.263+0000 7f0a4304b640 1 -- 172.21.15.82:0/662426186 wait complete. 2024-04-25T10:03:24.296 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:24.296 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:03:24.297 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:24.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:24.297 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:24.297 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:24.297 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:24.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:24.297 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:24.298 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:24.298 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:24.298 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:24.298 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:24.298 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:24.298 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:24.299 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:24.308 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:24.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:24.309 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:24.310 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:24.318 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:24.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:24.319 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:24.319 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:24.351 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:24.351 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:24.352 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:24.352 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:24.363 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:24.363 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:24.364 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:24.364 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:24.364 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:24.417 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:03:24.420 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:03:24.431 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:03:24.436 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:03:24.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.479+0000 7f6c84cec640 1 Processor -- start 2024-04-25T10:03:24.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.479+0000 7f6c84cec640 1 -- start start 2024-04-25T10:03:24.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c84cec640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c84cec640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8009a950 con 0x7f6c800b0440 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c84cec640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8009ac30 con 0x7f6c80099fb0 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c84cec640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8009a400 con 0x7f6c8015b920 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7dd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c800b0440 0x7f6c800aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41392/0 (socket says 172.21.15.82:41392) 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7dd9b640 1 -- 172.21.15.82:0/1751897658 learned_addr learned my addr 172.21.15.82:0/1751897658 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1751897658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7d59a640 1 -- 172.21.15.82:0/1751897658 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573133875 0 0) 0x7f6c8009a950 con 0x7f6c800b0440 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7d59a640 1 -- 172.21.15.82:0/1751897658 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c68003530 con 0x7f6c800b0440 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7e59c640 1 -- 172.21.15.82:0/1751897658 >> v1:172.21.15.196:6789/0 conn(0x7f6c80099fb0 legacy=0x7f6c800af290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7e59c640 1 -- 172.21.15.82:0/1751897658 >> v1:172.21.15.82:6789/0 conn(0x7f6c800b0440 legacy=0x7f6c800aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7e59c640 1 -- 172.21.15.82:0/1751897658 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c800ae380 con 0x7f6c8015b920 2024-04-25T10:03:24.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1751897658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80099a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097f0 tx=0x7f6c7001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd548a3d7d303dac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7d59a640 1 -- 172.21.15.82:0/1751897658 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700b0070 con 0x7f6c8015b920 2024-04-25T10:03:24.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7d59a640 1 -- 172.21.15.82:0/1751897658 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700ab600 con 0x7f6c8015b920 2024-04-25T10:03:24.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.483+0000 7f6c7d59a640 1 -- 172.21.15.82:0/1751897658 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700ab9b0 con 0x7f6c8015b920 2024-04-25T10:03:24.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1751897658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 msgr2=0x7f6c80099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1751897658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80099a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c700097f0 tx=0x7f6c7001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1751897658 shutdown_connections 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1751897658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80099a70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1751897658 >> 172.21.15.82:0/1751897658 conn(0x7f6c800b3010 msgr2=0x7f6c800b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1751897658 shutdown_connections 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1751897658 wait complete. 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 Processor -- start 2024-04-25T10:03:24.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- start start 2024-04-25T10:03:24.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 0x7f6c800c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80169ce0 con 0x7f6c800b0440 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c800cd550 con 0x7f6c80099fb0 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c80169e60 con 0x7f6c8015b920 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41086/0 (socket says 172.21.15.82:41086) 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 -- 172.21.15.82:0/1653048965 learned_addr learned my addr 172.21.15.82:0/1653048965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 0x7f6c800c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7ed9d640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 -- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 msgr2=0x7f6c80157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 -- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 msgr2=0x7f6c800c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 0x7f6c800c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 -- 172.21.15.82:0/1653048965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c800c7f60 con 0x7f6c800b0440 2024-04-25T10:03:24.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 0x7f6c800c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c7dd9b640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c420 tx=0x7f6c7400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74017070 con 0x7f6c800b0440 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c74005cf0 con 0x7f6c800b0440 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c800ca250 con 0x7f6c800b0440 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c740072d0 con 0x7f6c800b0440 2024-04-25T10:03:24.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.487+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c800ca810 con 0x7f6c800b0440 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8004efa0 con 0x7f6c800b0440 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c7401a070 con 0x7f6c800b0440 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c6effd640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 0x7f6c60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 0x7f6c60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c74084190 con 0x7f6c800b0440 2024-04-25T10:03:24.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.491+0000 7f6c7e59c640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 0x7f6c60069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c700ad000 tx=0x7f6c700a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.495+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c74054cc0 con 0x7f6c800b0440 2024-04-25T10:03:24.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:24.663+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6c800cba50 con 0x7f6c800b0440 2024-04-25T10:03:25.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.279+0000 7f6c6effd640 1 -- 172.21.15.82:0/1653048965 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6c74058d60 con 0x7f6c800b0440 2024-04-25T10:03:25.287 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 msgr2=0x7f6c60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 0x7f6c60069420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c700ad000 tx=0x7f6c700a6000 comp rx=0 tx=0).stop 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 msgr2=0x7f6c801573c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6c7400c420 tx=0x7f6c7400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 shutdown_connections 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c60066f60 0x7f6c60069420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c8015b920 0x7f6c80157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c800b0440 0x7f6c801573c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 --2- 172.21.15.82:0/1653048965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c80099fb0 0x7f6c800c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 >> 172.21.15.82:0/1653048965 conn(0x7f6c800b3010 msgr2=0x7f6c800b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 shutdown_connections 2024-04-25T10:03:25.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.283+0000 7f6c84cec640 1 -- 172.21.15.82:0/1653048965 wait complete. 2024-04-25T10:03:25.312 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:25.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.423+0000 7f2b06377640 1 Processor -- start 2024-04-25T10:03:25.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.423+0000 7f2b06377640 1 -- start start 2024-04-25T10:03:25.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b06377640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2b0015a700 0x7f2b0015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b06377640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b000b3f50 0x7f2b000cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b06377640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000b2090 con 0x7f2b000ce430 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b06377640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b0015f200 con 0x7f2b000b3f50 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b06377640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b000b43e0 con 0x7f2b0015a700 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2b000b3f50 0x7f2b000cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b04914640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2b000ce430 0x7f2b000b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52558/0 (socket says 172.21.15.82:52558) 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2b04914640 1 -- 172.21.15.82:0/2940286203 learned_addr learned my addr 172.21.15.82:0/2940286203 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2afffff640 1 --2- 172.21.15.82:0/2940286203 >> v2:172.21.15.196:3300/0 conn(0x7f2b0015a700 0x7f2b0015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 -- 172.21.15.82:0/2940286203 >> v2:172.21.15.196:3300/0 conn(0x7f2b0015a700 msgr2=0x7f2b0015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2940286203 >> v2:172.21.15.196:3300/0 conn(0x7f2b0015a700 0x7f2b0015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 -- 172.21.15.82:0/2940286203 >> v1:172.21.15.82:6789/0 conn(0x7f2b000ce430 legacy=0x7f2b000b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2afeffd640 1 -- 172.21.15.82:0/2940286203 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1577529987 0 0) 0x7f2b000b2090 con 0x7f2b000ce430 2024-04-25T10:03:25.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 -- 172.21.15.82:0/2940286203 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000b46c0 con 0x7f2b000b3f50 2024-04-25T10:03:25.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2940286203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000b3f50 0x7f2b000cdef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006bd20 tx=0x7f2af401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c43aa54abdb66c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2afeffd640 1 -- 172.21.15.82:0/2940286203 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40b0070 con 0x7f2b000b3f50 2024-04-25T10:03:25.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2afeffd640 1 -- 172.21.15.82:0/2940286203 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af40ab600 con 0x7f2b000b3f50 2024-04-25T10:03:25.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.427+0000 7f2afeffd640 1 -- 172.21.15.82:0/2940286203 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40ab9d0 con 0x7f2b000b3f50 2024-04-25T10:03:25.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2940286203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000b3f50 msgr2=0x7f2b000cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- 172.21.15.82:0/2940286203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000b3f50 0x7f2b000cdef0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006bd20 tx=0x7f2af401c330 comp rx=0 tx=0).stop 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2940286203 shutdown_connections 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- 172.21.15.82:0/2940286203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000b3f50 0x7f2b000cdef0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- 172.21.15.82:0/2940286203 >> v2:172.21.15.196:3300/0 conn(0x7f2b0015a700 0x7f2b0015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2940286203 >> 172.21.15.82:0/2940286203 conn(0x7f2b000b11a0 msgr2=0x7f2b000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2940286203 shutdown_connections 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2940286203 wait complete. 2024-04-25T10:03:25.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 Processor -- start 2024-04-25T10:03:25.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- start start 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 0x7f2b0015b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b00154f60 0x7f2b001f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000cd490 con 0x7f2b000ce430 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000cd310 con 0x7f2b0015a700 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000b2bb0 con 0x7f2b00154f60 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33924/0 (socket says 172.21.15.82:33924) 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 -- 172.21.15.82:0/2544663684 learned_addr learned my addr 172.21.15.82:0/2544663684 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b04914640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b00154f60 0x7f2b001f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 0x7f2b0015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 -- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b00154f60 msgr2=0x7f2b001f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b00154f60 0x7f2b001f6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 -- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 msgr2=0x7f2b0015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 0x7f2b0015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 -- 172.21.15.82:0/2544663684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0015bfe0 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 0x7f2b0015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afffff640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d830 tx=0x7f2af000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000b660 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af0019070 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b0015c2c0 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ba00 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b0015c880 con 0x7f2b000ce430 2024-04-25T10:03:25.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.431+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0004efa0 con 0x7f2b000ce430 2024-04-25T10:03:25.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.435+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2af001e460 con 0x7f2b000ce430 2024-04-25T10:03:25.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.435+0000 7f2afcff9640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.435+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2af0085e90 con 0x7f2b000ce430 2024-04-25T10:03:25.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.435+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.435+0000 7f2aff7fe640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 0x7f2aec069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2af40ad000 tx=0x7f2af4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.439+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2af00571d0 con 0x7f2b000ce430 2024-04-25T10:03:25.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.659+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b00071ff0 con 0x7f2b000ce430 2024-04-25T10:03:25.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.659+0000 7f2afcff9640 1 -- 172.21.15.82:0/2544663684 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2af0056ad0 con 0x7f2b000ce430 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:03:25.669 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:03:25.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.671 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.672 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:03:25.673 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.674 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:03:25.675 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.676 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:03:25.677 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:03:25.678 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.679 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:03:25.680 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipmH9DJKhAA31qUGiQIiDcvlepo2/iEFA==", 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipm2OzJKhAAw8fJ4hy+xQkvjrMIenyz6w==", 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.681 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipmagTKKhAAXJkwBuO9tdeCaXRwIOLxHg==", 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipmZRvKKhAAIUY0OoSEHD/1bN0mGhpYYg==", 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.682 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipm4jLKKhAAc8NmRSNktrFsfWlrVTgI3Q==", 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAGKipmpkvKKhAA/Ng/9bBEiUXIblfAhmA5FQ==", 2024-04-25T10:03:25.683 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:25.684 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:03:25.685 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 msgr2=0x7f2aec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 0x7f2aec069420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2af40ad000 tx=0x7f2af4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 msgr2=0x7f2b0015ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2af000d830 tx=0x7f2af000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 shutdown_connections 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aec066f60 0x7f2aec069420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2b00154f60 0x7f2b001f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2b0015a700 0x7f2b0015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 --2- 172.21.15.82:0/2544663684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2b000ce430 0x7f2b0015ff00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 >> 172.21.15.82:0/2544663684 conn(0x7f2b000b11a0 msgr2=0x7f2b0015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 shutdown_connections 2024-04-25T10:03:25.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.663+0000 7f2b06377640 1 -- 172.21.15.82:0/2544663684 wait complete. 2024-04-25T10:03:25.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:03:25.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 Processor -- start 2024-04-25T10:03:25.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 -- start start 2024-04-25T10:03:25.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071030 0x7fdab0161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdab0071e90 0x7fdab00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdab00b29e0 con 0x7fdab0071410 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdab0162680 con 0x7fdab0071e90 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdab48c6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab0162960 con 0x7fdab0071030 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdab0071e90 0x7fdab00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdab0071e90 0x7fdab00b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33936/0 (socket says 172.21.15.82:33936) 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 -- 172.21.15.82:0/3957203454 learned_addr learned my addr 172.21.15.82:0/3957203454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaae59c640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071030 0x7fdab0161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 -- 172.21.15.82:0/3957203454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071030 msgr2=0x7fdab0161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071030 0x7fdab0161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 -- 172.21.15.82:0/3957203454 >> v1:172.21.15.82:6789/0 conn(0x7fdab0071410 legacy=0x7fdab0165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaad59a640 1 -- 172.21.15.82:0/3957203454 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238760036 0 0) 0x7fdab00b29e0 con 0x7fdab0071410 2024-04-25T10:03:25.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.795+0000 7fdaadd9b640 1 -- 172.21.15.82:0/3957203454 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0162c40 con 0x7fdab0071e90 2024-04-25T10:03:25.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdaadd9b640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab00b24a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68282a1229251249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdaad59a640 1 -- 172.21.15.82:0/3957203454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40b0070 con 0x7fdab0071e90 2024-04-25T10:03:25.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdaad59a640 1 -- 172.21.15.82:0/3957203454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa40ab600 con 0x7fdab0071e90 2024-04-25T10:03:25.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdaad59a640 1 -- 172.21.15.82:0/3957203454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40ab9d0 con 0x7fdab0071e90 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- 172.21.15.82:0/3957203454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 msgr2=0x7fdab00b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab00b24a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4009870 tx=0x7fdaa401c4d0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- 172.21.15.82:0/3957203454 shutdown_connections 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab00b24a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 --2- 172.21.15.82:0/3957203454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071030 0x7fdab0161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- 172.21.15.82:0/3957203454 >> 172.21.15.82:0/3957203454 conn(0x7fdab00b11a0 msgr2=0x7fdab00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- 172.21.15.82:0/3957203454 shutdown_connections 2024-04-25T10:03:25.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- 172.21.15.82:0/3957203454 wait complete. 2024-04-25T10:03:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 Processor -- start 2024-04-25T10:03:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.799+0000 7fdab48c6640 1 -- start start 2024-04-25T10:03:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0071030 0x7fdab015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 0x7fdab015c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab016f050 con 0x7fdab0071e90 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab016eed0 con 0x7fdab0071030 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab016f1d0 con 0x7fdab0071410 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33946/0 (socket says 172.21.15.82:33946) 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 -- 172.21.15.82:0/1162214430 learned_addr learned my addr 172.21.15.82:0/1162214430 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaadd9b640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 0x7fdab015c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaae59c640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0071030 0x7fdab015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 -- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 msgr2=0x7fdab015c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 0x7fdab015c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 -- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0071030 msgr2=0x7fdab015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0071030 0x7fdab015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 -- 172.21.15.82:0/1162214430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0156fc0 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaadd9b640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 0x7fdab015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaaed9d640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8001570 tx=0x7fdaa800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8017070 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa8003db0 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab0157260 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80052b0 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab0157820 con 0x7fdab0071e90 2024-04-25T10:03:25.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdab004efa0 con 0x7fdab0071e90 2024-04-25T10:03:25.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdaa801a070 con 0x7fdab0071e90 2024-04-25T10:03:25.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 0x7fda90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:25.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdaa8084520 con 0x7fdab0071e90 2024-04-25T10:03:25.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaae59c640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 0x7fda90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:25.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.803+0000 7fdaae59c640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 0x7fda90069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b9d0 tx=0x7fdaa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:25.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:25.807+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdaa80556f0 con 0x7fdab0071e90 2024-04-25T10:03:26.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.023+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fdab00b1dc0 con 0x7fdab0071e90 2024-04-25T10:03:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fda9effd640 1 -- 172.21.15.82:0/1162214430 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdaa8054ff0 con 0x7fdab0071e90 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 msgr2=0x7fda90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 0x7fda90069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000b9d0 tx=0x7fdaa0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 msgr2=0x7fdab01402c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8001570 tx=0x7fdaa800cd20 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 shutdown_connections 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda90066f60 0x7fda90069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdab0071410 0x7fdab015c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdab0071e90 0x7fdab01402c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 --2- 172.21.15.82:0/1162214430 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdab0071030 0x7fdab015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 >> 172.21.15.82:0/1162214430 conn(0x7fdab00b11a0 msgr2=0x7fdab0164bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 shutdown_connections 2024-04-25T10:03:26.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.027+0000 7fdab48c6640 1 -- 172.21.15.82:0/1162214430 wait complete. 2024-04-25T10:03:26.053 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:03:26.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 Processor -- start 2024-04-25T10:03:26.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 -- start start 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe420071fd0 0x7fe42015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015ae30 con 0x7fe420159ea0 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015b110 con 0x7fe42015a280 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe427616640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4200b2990 con 0x7fe420071fd0 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe420071fd0 0x7fe42015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe420071fd0 0x7fe42015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33958/0 (socket says 172.21.15.82:33958) 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 -- 172.21.15.82:0/1242941503 learned_addr learned my addr 172.21.15.82:0/1242941503 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 -- 172.21.15.82:0/1242941503 >> v1:172.21.15.196:6789/0 conn(0x7fe42015a280 legacy=0x7fe4200718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 -- 172.21.15.82:0/1242941503 >> v1:172.21.15.82:6789/0 conn(0x7fe420159ea0 legacy=0x7fe4200b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe417fff640 1 -- 172.21.15.82:0/1242941503 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778726854 0 0) 0x7fe42015ae30 con 0x7fe420159ea0 2024-04-25T10:03:26.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.171+0000 7fe425bb3640 1 -- 172.21.15.82:0/1242941503 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200b2620 con 0x7fe420071fd0 2024-04-25T10:03:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe425bb3640 1 --2- 172.21.15.82:0/1242941503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420071fd0 0x7fe42015a700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe418009e50 tx=0x7fe41801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e510dc4dd81aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe417fff640 1 -- 172.21.15.82:0/1242941503 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180a5750 con 0x7fe420071fd0 2024-04-25T10:03:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe417fff640 1 -- 172.21.15.82:0/1242941503 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4180a58f0 con 0x7fe420071fd0 2024-04-25T10:03:26.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe417fff640 1 -- 172.21.15.82:0/1242941503 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180a5cc0 con 0x7fe420071fd0 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- 172.21.15.82:0/1242941503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420071fd0 msgr2=0x7fe42015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 --2- 172.21.15.82:0/1242941503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420071fd0 0x7fe42015a700 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe418009e50 tx=0x7fe41801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- 172.21.15.82:0/1242941503 shutdown_connections 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 --2- 172.21.15.82:0/1242941503 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420071fd0 0x7fe42015a700 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- 172.21.15.82:0/1242941503 >> 172.21.15.82:0/1242941503 conn(0x7fe4200b11a0 msgr2=0x7fe4200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- 172.21.15.82:0/1242941503 shutdown_connections 2024-04-25T10:03:26.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- 172.21.15.82:0/1242941503 wait complete. 2024-04-25T10:03:26.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 Processor -- start 2024-04-25T10:03:26.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- start start 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe420071fd0 0x7fe42013eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe42015a280 0x7fe4200aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4200b3530 con 0x7fe420159ea0 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006dab0 con 0x7fe420071fd0 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe427616640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe420168ae0 con 0x7fe42015a280 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.175+0000 7fe424bb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33960/0 (socket says 172.21.15.82:33960) 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 -- 172.21.15.82:0/331578473 learned_addr learned my addr 172.21.15.82:0/331578473 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 -- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe42015a280 msgr2=0x7fe4200aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe42015a280 0x7fe4200aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 -- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe420071fd0 msgr2=0x7fe42013eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe420071fd0 0x7fe42013eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 -- 172.21.15.82:0/331578473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe420150ef0 con 0x7fe420159ea0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe424bb1640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b410 tx=0x7fe41000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410015030 con 0x7fe420159ea0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410004020 con 0x7fe420159ea0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4201511d0 con 0x7fe420159ea0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410004450 con 0x7fe420159ea0 2024-04-25T10:03:26.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe420151790 con 0x7fe420159ea0 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42004efa0 con 0x7fe420159ea0 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe410018070 con 0x7fe420159ea0 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe410084a00 con 0x7fe420159ea0 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe4253b2640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.179+0000 7fe4253b2640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe41c007fd0 tx=0x7fe41c007920 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.183+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe410055bd0 con 0x7fe420159ea0 2024-04-25T10:03:26.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.403+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe42006e6b0 con 0x7fe420159ea0 2024-04-25T10:03:26.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe415ffb640 1 -- 172.21.15.82:0/331578473 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe4100554d0 con 0x7fe420159ea0 2024-04-25T10:03:26.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 msgr2=0x7fe404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe41c007fd0 tx=0x7fe41c007920 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 msgr2=0x7fe4201505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b410 tx=0x7fe41000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 shutdown_connections 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe42015a280 0x7fe4200aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe420159ea0 0x7fe4201505c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 --2- 172.21.15.82:0/331578473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe420071fd0 0x7fe42013eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 >> 172.21.15.82:0/331578473 conn(0x7fe4200b11a0 msgr2=0x7fe420152780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.411+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 shutdown_connections 2024-04-25T10:03:26.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.415+0000 7fe427616640 1 -- 172.21.15.82:0/331578473 wait complete. 2024-04-25T10:03:26.437 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:03:26.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 Processor -- start 2024-04-25T10:03:26.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 -- start start 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c15a700 0x7f318c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f318c0b3f50 0x7f318c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f318c0ce430 0x7f318c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c15f200 con 0x7f318c0b3f50 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f318c0b2090 con 0x7f318c0ce430 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f3191a20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0b43e0 con 0x7f318c15a700 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f318c0b3f50 0x7f318c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f318c0b3f50 0x7f318c0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33974/0 (socket says 172.21.15.82:33974) 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 -- 172.21.15.82:0/3815728534 learned_addr learned my addr 172.21.15.82:0/3815728534 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318b7fe640 1 --2- 172.21.15.82:0/3815728534 >> v2:172.21.15.196:3300/0 conn(0x7f318c0ce430 0x7f318c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 -- 172.21.15.82:0/3815728534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c15a700 msgr2=0x7f318c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 --2- 172.21.15.82:0/3815728534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c15a700 0x7f318c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 -- 172.21.15.82:0/3815728534 >> v2:172.21.15.196:3300/0 conn(0x7f318c0ce430 msgr2=0x7f318c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 --2- 172.21.15.82:0/3815728534 >> v2:172.21.15.196:3300/0 conn(0x7f318c0ce430 0x7f318c0b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.551+0000 7f318a7fc640 1 -- 172.21.15.82:0/3815728534 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c0b46c0 con 0x7f318c0b3f50 2024-04-25T10:03:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f318a7fc640 1 --2- 172.21.15.82:0/3815728534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c0b3f50 0x7f318c0cdef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f318c06ca20 tx=0x7f317c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a797cc699f7dcc18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3189ffb640 1 -- 172.21.15.82:0/3815728534 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0b0070 con 0x7f318c0b3f50 2024-04-25T10:03:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3189ffb640 1 -- 172.21.15.82:0/3815728534 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c0ab600 con 0x7f318c0b3f50 2024-04-25T10:03:26.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3189ffb640 1 -- 172.21.15.82:0/3815728534 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0ab9d0 con 0x7f318c0b3f50 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 -- 172.21.15.82:0/3815728534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c0b3f50 msgr2=0x7f318c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 --2- 172.21.15.82:0/3815728534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c0b3f50 0x7f318c0cdef0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f318c06ca20 tx=0x7f317c01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 -- 172.21.15.82:0/3815728534 shutdown_connections 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 --2- 172.21.15.82:0/3815728534 >> v2:172.21.15.196:3300/0 conn(0x7f318c0ce430 0x7f318c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 --2- 172.21.15.82:0/3815728534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c0b3f50 0x7f318c0cdef0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 --2- 172.21.15.82:0/3815728534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c15a700 0x7f318c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 -- 172.21.15.82:0/3815728534 >> 172.21.15.82:0/3815728534 conn(0x7f318c0b11a0 msgr2=0x7f318c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 -- 172.21.15.82:0/3815728534 shutdown_connections 2024-04-25T10:03:26.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.555+0000 7f3191a20640 1 -- 172.21.15.82:0/3815728534 wait complete. 2024-04-25T10:03:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 Processor -- start 2024-04-25T10:03:26.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- start start 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c0ce430 0x7f318c165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f318c160e50 0x7f318c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0cd490 con 0x7f318c15a700 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f318c0cd310 con 0x7f318c160e50 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c071030 con 0x7f318c0ce430 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33978/0 (socket says 172.21.15.82:33978) 2024-04-25T10:03:26.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 -- 172.21.15.82:0/161138227 learned_addr learned my addr 172.21.15.82:0/161138227 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 -- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c0ce430 msgr2=0x7f318c165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c0ce430 0x7f318c165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 -- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f318c160e50 msgr2=0x7f318c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f318c160e50 0x7f318c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 -- 172.21.15.82:0/161138227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c161830 con 0x7f318c15a700 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f318a7fc640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f317c0ae040 tx=0x7f317c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0b0040 con 0x7f318c15a700 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c0b9070 con 0x7f318c15a700 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c0b48a0 con 0x7f318c15a700 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c15f4d0 con 0x7f318c15a700 2024-04-25T10:03:26.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c161b10 con 0x7f318c15a700 2024-04-25T10:03:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318c04efa0 con 0x7f318c15a700 2024-04-25T10:03:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f317c0ab4c0 con 0x7f318c15a700 2024-04-25T10:03:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.559+0000 7f3173fff640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 0x7f3178069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.563+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f317c11ed00 con 0x7f318c15a700 2024-04-25T10:03:26.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.563+0000 7f318affd640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 0x7f3178069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.563+0000 7f318affd640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 0x7f3178069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3180009730 tx=0x7f3180006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.567+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f317c016110 con 0x7f318c15a700 2024-04-25T10:03:26.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.779+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f318c15dbc0 con 0x7f318c15a700 2024-04-25T10:03:26.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.783+0000 7f3173fff640 1 -- 172.21.15.82:0/161138227 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f317c0efe30 con 0x7f318c15a700 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 msgr2=0x7f3178069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 0x7f3178069420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3180009730 tx=0x7f3180006ca0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 msgr2=0x7f318c15ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f317c0ae040 tx=0x7f317c019170 comp rx=0 tx=0).stop 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 shutdown_connections 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3178066f60 0x7f3178069420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f318c160e50 0x7f318c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f318c15a700 0x7f318c15ffb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 --2- 172.21.15.82:0/161138227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f318c0ce430 0x7f318c165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 >> 172.21.15.82:0/161138227 conn(0x7f318c0b11a0 msgr2=0x7f318c06da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 shutdown_connections 2024-04-25T10:03:26.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.787+0000 7f3191a20640 1 -- 172.21.15.82:0/161138227 wait complete. 2024-04-25T10:03:26.812 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:03:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 Processor -- start 2024-04-25T10:03:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 -- start start 2024-04-25T10:03:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe4180b3f50 0x7fe41815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180b2090 con 0x7fe4180b24a0 2024-04-25T10:03:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4180b43e0 con 0x7fe41815a700 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41f0c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41815f200 con 0x7fe4180b3f50 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41d666640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe4180b24a0 0x7fe41806e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52570/0 (socket says 172.21.15.82:52570) 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41d666640 1 -- 172.21.15.82:0/1248111963 learned_addr learned my addr 172.21.15.82:0/1248111963 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe417fff640 1 --2- 172.21.15.82:0/1248111963 >> v2:172.21.15.196:3300/0 conn(0x7fe4180b3f50 0x7fe41815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284259866 0 0) 0x7fe4180b2090 con 0x7fe4180b24a0 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe400003530 con 0x7fe4180b24a0 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270557442 0 0) 0x7fe400003530 con 0x7fe4180b24a0 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 >> v2:172.21.15.196:3300/0 conn(0x7fe4180b3f50 msgr2=0x7fe41815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 --2- 172.21.15.82:0/1248111963 >> v2:172.21.15.196:3300/0 conn(0x7fe4180b3f50 0x7fe41815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 >> v1:172.21.15.196:6789/0 conn(0x7fe41815a700 legacy=0x7fe4180cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180b46c0 con 0x7fe4180b24a0 2024-04-25T10:03:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe410001a20 con 0x7fe4180b24a0 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 >> v1:172.21.15.82:6789/0 conn(0x7fe4180b24a0 legacy=0x7fe41806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 0x7fe400003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe400004810 0x7fe400004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400005380 0x7fe40000b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41815f200 con 0x7fe400003a30 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000c2d0 con 0x7fe400004810 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40000c4d0 con 0x7fe400005380 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 0x7fe400003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe417fff640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe400004810 0x7fe400004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41d666640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400005380 0x7fe40000b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 -- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400005380 msgr2=0x7fe40000b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400005380 0x7fe40000b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 -- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe400004810 msgr2=0x7fe400004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe400004810 0x7fe400004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41ce65640 1 -- 172.21.15.82:0/1248111963 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe400003530 con 0x7fe400003a30 2024-04-25T10:03:26.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.919+0000 7fe41d666640 1 -- 172.21.15.82:0/1248111963 reap_dead start 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 0x7fe400003e50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe408001770 tx=0x7fe40801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5965a010dafe3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080013c0 con 0x7fe400003a30 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4080052d0 con 0x7fe400003a30 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe4177fe640 1 -- 172.21.15.82:0/1248111963 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080054c0 con 0x7fe400003a30 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 msgr2=0x7fe400003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 0x7fe400003e50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe408001770 tx=0x7fe40801c3f0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1248111963 shutdown_connections 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1248111963 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe400003a30 0x7fe400003e50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1248111963 >> 172.21.15.82:0/1248111963 conn(0x7fe4180b11a0 msgr2=0x7fe4180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1248111963 shutdown_connections 2024-04-25T10:03:26.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1248111963 wait complete. 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 Processor -- start 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.923+0000 7fe41f0c9640 1 -- start start 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400003a30 0x7fe4180ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4180a8df0 0x7fe418156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180b2c90 con 0x7fe4181486e0 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180ce700 con 0x7fe4180a8df0 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180cd2d0 con 0x7fe400003a30 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33990/0 (socket says 172.21.15.82:33990) 2024-04-25T10:03:26.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 -- 172.21.15.82:0/1393401505 learned_addr learned my addr 172.21.15.82:0/1393401505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400003a30 0x7fe4180ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe417fff640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4180a8df0 0x7fe418156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 -- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400003a30 msgr2=0x7fe4180ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400003a30 0x7fe4180ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 -- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4180a8df0 msgr2=0x7fe418156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4180a8df0 0x7fe418156ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 -- 172.21.15.82:0/1393401505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4180a94b0 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41d666640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f90 tx=0x7fe41000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000b8d0 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41000ba90 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4180a9790 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000be80 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4180a9d50 con 0x7fe4181486e0 2024-04-25T10:03:26.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41804efa0 con 0x7fe4181486e0 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe41001b060 con 0x7fe4181486e0 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe410084cc0 con 0x7fe4181486e0 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.927+0000 7fe41ce65640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4080ac000 tx=0x7fe4080011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:26.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:26.931+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe410055e90 con 0x7fe4181486e0 2024-04-25T10:03:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.143+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fe418072040 con 0x7fe4181486e0 2024-04-25T10:03:27.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.147+0000 7fe4157fa640 1 -- 172.21.15.82:0/1393401505 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe410055790 con 0x7fe4181486e0 2024-04-25T10:03:27.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 msgr2=0x7fe404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe4080ac000 tx=0x7fe4080011a0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 msgr2=0x7fe41806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe410000f90 tx=0x7fe41000b0d0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 shutdown_connections 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4181486e0 0x7fe41806d090 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4180a8df0 0x7fe418156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 --2- 172.21.15.82:0/1393401505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe400003a30 0x7fe4180ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 >> 172.21.15.82:0/1393401505 conn(0x7fe4180b11a0 msgr2=0x7fe41815dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 shutdown_connections 2024-04-25T10:03:27.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.151+0000 7fe41f0c9640 1 -- 172.21.15.82:0/1393401505 wait complete. 2024-04-25T10:03:27.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:03:27.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 Processor -- start 2024-04-25T10:03:27.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 -- start start 2024-04-25T10:03:27.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e2006bca0 0x7f9e200cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e20066e30 0x7f9e2006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2006ec30 con 0x7f9e200af370 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e200669a0 con 0x7f9e20066e30 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e2604e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e200cd5f0 con 0x7f9e2006bca0 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e20066e30 0x7f9e2006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e20066e30 0x7f9e2006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33994/0 (socket says 172.21.15.82:33994) 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e200af370 0x7f9e2006d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52574/0 (socket says 172.21.15.82:52574) 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 -- 172.21.15.82:0/1980680464 learned_addr learned my addr 172.21.15.82:0/1980680464 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1f7fe640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e2006bca0 0x7f9e200cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 -- 172.21.15.82:0/1980680464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e2006bca0 msgr2=0x7f9e200cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e2006bca0 0x7f9e200cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 -- 172.21.15.82:0/1980680464 >> v1:172.21.15.82:6789/0 conn(0x7f9e200af370 legacy=0x7f9e2006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1effd640 1 -- 172.21.15.82:0/1980680464 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2006c5c0 con 0x7f9e20066e30 2024-04-25T10:03:27.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.287+0000 7f9e1e7fc640 1 -- 172.21.15.82:0/1980680464 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739843399 0 0) 0x7f9e2006ec30 con 0x7f9e200af370 2024-04-25T10:03:27.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e1effd640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e20066e30 0x7f9e2006c080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3394ab73fcbc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e1e7fc640 1 -- 172.21.15.82:0/1980680464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080b1070 con 0x7f9e20066e30 2024-04-25T10:03:27.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e1e7fc640 1 -- 172.21.15.82:0/1980680464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080ac620 con 0x7f9e20066e30 2024-04-25T10:03:27.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e1e7fc640 1 -- 172.21.15.82:0/1980680464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080ac9f0 con 0x7f9e20066e30 2024-04-25T10:03:27.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- 172.21.15.82:0/1980680464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e20066e30 msgr2=0x7f9e2006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e20066e30 0x7f9e2006c080 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009870 tx=0x7f9e0801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- 172.21.15.82:0/1980680464 shutdown_connections 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e20066e30 0x7f9e2006c080 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1980680464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e2006bca0 0x7f9e200cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- 172.21.15.82:0/1980680464 >> 172.21.15.82:0/1980680464 conn(0x7f9e20071810 msgr2=0x7f9e20071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- 172.21.15.82:0/1980680464 shutdown_connections 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- 172.21.15.82:0/1980680464 wait complete. 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 Processor -- start 2024-04-25T10:03:27.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.291+0000 7f9e2604e640 1 -- start start 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e20066e30 0x7f9e2013e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 0x7f9e20157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2015cc80 con 0x7f9e200af370 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2015cb00 con 0x7f9e2006bca0 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20168630 con 0x7f9e20066e30 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34006/0 (socket says 172.21.15.82:34006) 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 -- 172.21.15.82:0/1358274784 learned_addr learned my addr 172.21.15.82:0/1358274784 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1f7fe640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e20066e30 0x7f9e2013e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1effd640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 0x7f9e20157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 -- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e20066e30 msgr2=0x7f9e2013e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e20066e30 0x7f9e2013e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 -- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 msgr2=0x7f9e20157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 0x7f9e20157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 -- 172.21.15.82:0/1358274784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e201575b0 con 0x7f9e200af370 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1effd640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 0x7f9e20157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1ffff640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800c7d0 tx=0x7f9e1800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e18018070 con 0x7f9e200af370 2024-04-25T10:03:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1800a640 con 0x7f9e200af370 2024-04-25T10:03:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e200b3280 con 0x7f9e200af370 2024-04-25T10:03:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1800aa70 con 0x7f9e200af370 2024-04-25T10:03:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e20156870 con 0x7f9e200af370 2024-04-25T10:03:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e20156380 con 0x7f9e200af370 2024-04-25T10:03:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e1801b070 con 0x7f9e200af370 2024-04-25T10:03:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 0x7f9e00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1f7fe640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 0x7f9e00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9e18084430 con 0x7f9e200af370 2024-04-25T10:03:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.295+0000 7f9e1f7fe640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 0x7f9e00069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000b9d0 tx=0x7f9e10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.299+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e18055600 con 0x7f9e200af370 2024-04-25T10:03:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.519+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f9e20072910 con 0x7f9e200af370 2024-04-25T10:03:27.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.523+0000 7f9e17fff640 1 -- 172.21.15.82:0/1358274784 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9e18054f00 con 0x7f9e200af370 2024-04-25T10:03:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 msgr2=0x7f9e00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 0x7f9e00069420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000b9d0 tx=0x7f9e10008040 comp rx=0 tx=0).stop 2024-04-25T10:03:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 msgr2=0x7f9e200b08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9e1800c7d0 tx=0x7f9e1800cc90 comp rx=0 tx=0).stop 2024-04-25T10:03:27.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 shutdown_connections 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e00066f60 0x7f9e00069420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e200af370 0x7f9e200b08a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e2006bca0 0x7f9e20157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 --2- 172.21.15.82:0/1358274784 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e20066e30 0x7f9e2013e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 >> 172.21.15.82:0/1358274784 conn(0x7f9e20071810 msgr2=0x7f9e20159af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 shutdown_connections 2024-04-25T10:03:27.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.527+0000 7f9e2604e640 1 -- 172.21.15.82:0/1358274784 wait complete. 2024-04-25T10:03:27.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:03:27.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 Processor -- start 2024-04-25T10:03:27.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 -- start start 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feda00b5270 0x7feda00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7feda00d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feda006bff0 con 0x7feda00d4440 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feda006cc00 con 0x7feda00b5270 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda7962640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda006c2d0 con 0x7feda00b5650 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.663+0000 7feda5eff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feda00d4440 0x7feda006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52586/0 (socket says 172.21.15.82:52586) 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda5eff640 1 -- 172.21.15.82:0/1616011873 learned_addr learned my addr 172.21.15.82:0/1616011873 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142268697 0 0) 0x7feda006bff0 con 0x7feda00d4440 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda4efd640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7feda00d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed8c003530 con 0x7feda00d4440 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> v2:172.21.15.196:3300/0 conn(0x7feda00b5270 0x7feda00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798870401 0 0) 0x7fed8c003530 con 0x7feda00d4440 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 msgr2=0x7feda00d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7feda00d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 >> v2:172.21.15.196:3300/0 conn(0x7feda00b5270 msgr2=0x7feda00b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 --2- 172.21.15.82:0/1616011873 >> v2:172.21.15.196:3300/0 conn(0x7feda00b5270 0x7feda00b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda006c5b0 con 0x7feda00d4440 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fed98001a00 con 0x7feda00d4440 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 >> v1:172.21.15.82:6789/0 conn(0x7feda00d4440 legacy=0x7feda006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed8c003a30 0x7fed8c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7fed8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c004d50 0x7fed8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda006cc00 con 0x7fed8c003a30 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda006c2d0 con 0x7fed8c004d50 2024-04-25T10:03:27.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c00bab0 con 0x7feda00b5650 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> v2:172.21.15.196:3300/0 conn(0x7feda00b5270 0x7feda00b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed8c003a30 0x7fed8c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda4efd640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7fed8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda5eff640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c004d50 0x7fed8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 -- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 msgr2=0x7fed8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b5650 0x7fed8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 -- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c004d50 msgr2=0x7fed8c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed8c004d50 0x7fed8c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 -- 172.21.15.82:0/1616011873 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c003530 con 0x7fed8c003a30 2024-04-25T10:03:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7feda56fe640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed8c003a30 0x7fed8c003e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0098a0 tx=0x7fed9c01c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ad7f99a09732f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c00f040 con 0x7fed8c003a30 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9c0a9610 con 0x7fed8c003a30 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.667+0000 7fed97fff640 1 -- 172.21.15.82:0/1616011873 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0a99c0 con 0x7fed8c003a30 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/1616011873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed8c003a30 msgr2=0x7fed8c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 --2- 172.21.15.82:0/1616011873 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed8c003a30 0x7fed8c003e50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0098a0 tx=0x7fed9c01c610 comp rx=0 tx=0).stop 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda5eff640 1 -- 172.21.15.82:0/1616011873 reap_dead start 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/1616011873 shutdown_connections 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/1616011873 >> 172.21.15.82:0/1616011873 conn(0x7feda008b000 msgr2=0x7feda00d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/1616011873 shutdown_connections 2024-04-25T10:03:27.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/1616011873 wait complete. 2024-04-25T10:03:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 Processor -- start 2024-04-25T10:03:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- start start 2024-04-25T10:03:27.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b66e0 0x7feda00b1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda015c950 0x7feda015cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda015dfa0 con 0x7feda00b6ac0 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feda015de20 con 0x7feda015c950 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feda0168710 con 0x7feda00b66e0 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34032/0 (socket says 172.21.15.82:34032) 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 -- 172.21.15.82:0/2986483195 learned_addr learned my addr 172.21.15.82:0/2986483195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda5eff640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda015c950 0x7feda015cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda56fe640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b66e0 0x7feda00b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 -- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b66e0 msgr2=0x7feda00b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b66e0 0x7feda00b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 -- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda015c950 msgr2=0x7feda015cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda015c950 0x7feda015cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 -- 172.21.15.82:0/2986483195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00b8a90 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda4efd640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fed90011660 tx=0x7fed90011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90019070 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90005ba0 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000f760 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feda00b8d70 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.671+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feda00b9330 con 0x7feda00b6ac0 2024-04-25T10:03:27.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feda004efa0 con 0x7feda00b6ac0 2024-04-25T10:03:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed9001c070 con 0x7feda00b6ac0 2024-04-25T10:03:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7fed95ffb640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 0x7fed84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fed900852c0 con 0x7feda00b6ac0 2024-04-25T10:03:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7feda56fe640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 0x7fed84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:27.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.675+0000 7feda56fe640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 0x7fed84069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ae000 tx=0x7fed9c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:27.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.679+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed90056490 con 0x7feda00b6ac0 2024-04-25T10:03:27.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.883+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7feda00c7070 con 0x7feda00b6ac0 2024-04-25T10:03:27.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7fed95ffb640 1 -- 172.21.15.82:0/2986483195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fed90055d90 con 0x7feda00b6ac0 2024-04-25T10:03:27.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 msgr2=0x7fed84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 0x7fed84069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0ae000 tx=0x7fed9c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 msgr2=0x7feda00b81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fed90011660 tx=0x7fed90011b20 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 shutdown_connections 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed84066f60 0x7fed84069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feda015c950 0x7feda015cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feda00b6ac0 0x7feda00b81f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 --2- 172.21.15.82:0/2986483195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feda00b66e0 0x7feda00b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 >> 172.21.15.82:0/2986483195 conn(0x7feda008b000 msgr2=0x7feda00b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 shutdown_connections 2024-04-25T10:03:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:27.887+0000 7feda7962640 1 -- 172.21.15.82:0/2986483195 wait complete. 2024-04-25T10:03:27.911 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:28.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.019+0000 7f71a8730640 1 Processor -- start 2024-04-25T10:03:28.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.019+0000 7f71a8730640 1 -- start start 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a8730640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f71a015a700 0x7f71a015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a8730640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00b2090 con 0x7f71a00b3f50 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a8730640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a00b43e0 con 0x7f71a00b24a0 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a8730640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a015f200 con 0x7f71a015a700 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a5ccb640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71a00b3f50 0x7f71a00cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52590/0 (socket says 172.21.15.82:52590) 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a5ccb640 1 -- 172.21.15.82:0/3614484604 learned_addr learned my addr 172.21.15.82:0/3614484604 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:28.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529897742 0 0) 0x7f71a00b2090 con 0x7f71a00b3f50 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f718c005740 con 0x7f71a00b3f50 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a64cc640 1 --2- 172.21.15.82:0/3614484604 >> v2:172.21.15.196:3300/0 conn(0x7f71a015a700 0x7f71a015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2235747471 0 0) 0x7f718c005740 con 0x7f71a00b3f50 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 >> v2:172.21.15.196:3300/0 conn(0x7f71a015a700 msgr2=0x7f71a015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 --2- 172.21.15.82:0/3614484604 >> v2:172.21.15.196:3300/0 conn(0x7f71a015a700 0x7f71a015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 >> v1:172.21.15.196:6789/0 conn(0x7f71a00b24a0 legacy=0x7f71a006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a00b46c0 con 0x7f71a00b3f50 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f719c0016a0 con 0x7f71a00b3f50 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a64cc640 1 --2- 172.21.15.82:0/3614484604 >> v2:172.21.15.196:3300/0 conn(0x7f71a015a700 0x7f71a015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 >> v1:172.21.15.82:6789/0 conn(0x7f71a00b3f50 legacy=0x7f71a00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f718c005c40 0x7f718c006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c006a20 0x7f718c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 0x7f718c00dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015f200 con 0x7f718c007590 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00e4e0 con 0x7f718c006a20 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c00e6e0 con 0x7f718c005c40 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 0x7f718c00dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a5ccb640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c006a20 0x7f718c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a64cc640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f718c005c40 0x7f718c006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 -- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f718c005c40 msgr2=0x7f718c006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f718c005c40 0x7f718c006060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 -- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c006a20 msgr2=0x7f718c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c006a20 0x7f718c006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a6ccd640 1 -- 172.21.15.82:0/3614484604 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c005740 con 0x7f718c007590 2024-04-25T10:03:28.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.023+0000 7f71a5ccb640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c006a20 0x7f718c006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 -- 172.21.15.82:0/3614484604 reap_dead start 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 0x7f718c00dc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7198001e70 tx=0x7f719801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f469f516df28cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198001a90 con 0x7f718c007590 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7198003bc0 con 0x7f718c007590 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a54ca640 1 -- 172.21.15.82:0/3614484604 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71980050d0 con 0x7f718c007590 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 msgr2=0x7f718c00dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 0x7f718c00dc00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7198001e70 tx=0x7f719801c750 comp rx=0 tx=0).stop 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- 172.21.15.82:0/3614484604 shutdown_connections 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 --2- 172.21.15.82:0/3614484604 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f718c007590 0x7f718c00dc00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- 172.21.15.82:0/3614484604 >> 172.21.15.82:0/3614484604 conn(0x7f71a00b11a0 msgr2=0x7f71a00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- 172.21.15.82:0/3614484604 shutdown_connections 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- 172.21.15.82:0/3614484604 wait complete. 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 Processor -- start 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- start start 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c007590 0x7f71a00a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71a015a090 0x7f71a00ae700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cd490 con 0x7f71a0151580 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a00cd310 con 0x7f718c007590 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a8730640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a015ecc0 con 0x7f71a015a090 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34058/0 (socket says 172.21.15.82:34058) 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 -- 172.21.15.82:0/1614877203 learned_addr learned my addr 172.21.15.82:0/1614877203 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a64cc640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c007590 0x7f71a00a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a5ccb640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71a015a090 0x7f71a00ae700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 -- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71a015a090 msgr2=0x7f71a00ae700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71a015a090 0x7f71a00ae700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 -- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c007590 msgr2=0x7f71a00a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c007590 0x7f71a00a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 -- 172.21.15.82:0/1614877203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0152310 con 0x7f71a0151580 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.027+0000 7f71a6ccd640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f719801cff0 tx=0x7f718c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198001a90 con 0x7f71a0151580 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71980057d0 con 0x7f71a0151580 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a014e0f0 con 0x7f71a0151580 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198007dc0 con 0x7f71a0151580 2024-04-25T10:03:28.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a01525f0 con 0x7f71a0151580 2024-04-25T10:03:28.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a004efa0 con 0x7f71a0151580 2024-04-25T10:03:28.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7198007030 con 0x7f71a0151580 2024-04-25T10:03:28.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 0x7f7184069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f719811e3d0 con 0x7f71a0151580 2024-04-25T10:03:28.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f71a64cc640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 0x7f7184069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.031+0000 7f71a64cc640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 0x7f7184069450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71a0150800 tx=0x7f7190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.035+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7198016490 con 0x7f71a0151580 2024-04-25T10:03:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.195+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71a014e5a0 con 0x7f71a0151580 2024-04-25T10:03:28.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.195+0000 7f7196ffd640 1 -- 172.21.15.82:0/1614877203 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13176 (secure 0 0 0) 0x7f71980ef5a0 con 0x7f71a0151580 2024-04-25T10:03:28.204 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:28.204 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":35,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:25.296570+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 msgr2=0x7f7184069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 0x7f7184069450 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f71a0150800 tx=0x7f7190008040 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 msgr2=0x7f71a0151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f719801cff0 tx=0x7f718c007a50 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 shutdown_connections 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7184066f90 0x7f7184069450 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71a015a090 0x7f71a00ae700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71a0151580 0x7f71a0151a10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 --2- 172.21.15.82:0/1614877203 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f718c007590 0x7f71a00a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 >> 172.21.15.82:0/1614877203 conn(0x7f71a00b11a0 msgr2=0x7f71a00af490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 shutdown_connections 2024-04-25T10:03:28.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.199+0000 7f71a8730640 1 -- 172.21.15.82:0/1614877203 wait complete. 2024-04-25T10:03:28.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:28.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:28.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 Processor -- start 2024-04-25T10:03:28.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 -- start start 2024-04-25T10:03:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa720073010 0x7fa7200cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7200cbd20 con 0x7fa7200cf590 2024-04-25T10:03:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72015ba90 con 0x7fa720073010 2024-04-25T10:03:28.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa727dcc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cff30 con 0x7fa72015b6b0 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa720073010 0x7fa7200cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa725367640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa7200cf590 0x7fa72015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52604/0 (socket says 172.21.15.82:52604) 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa725367640 1 -- 172.21.15.82:0/576791492 learned_addr learned my addr 172.21.15.82:0/576791492 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa725b68640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 -- 172.21.15.82:0/576791492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 msgr2=0x7fa7200ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 -- 172.21.15.82:0/576791492 >> v1:172.21.15.82:6789/0 conn(0x7fa7200cf590 legacy=0x7fa72015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 -- 172.21.15.82:0/576791492 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200cc000 con 0x7fa720073010 2024-04-25T10:03:28.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa724b66640 1 -- 172.21.15.82:0/576791492 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547054804 0 0) 0x7fa7200cbd20 con 0x7fa7200cf590 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.335+0000 7fa726369640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa720073010 0x7fa7200cf9f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa718009d90 tx=0x7fa71801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100c6e8580269534 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa724b66640 1 -- 172.21.15.82:0/576791492 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180b2070 con 0x7fa720073010 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa724b66640 1 -- 172.21.15.82:0/576791492 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7180a5920 con 0x7fa720073010 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa724b66640 1 -- 172.21.15.82:0/576791492 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180a5cf0 con 0x7fa720073010 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/576791492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa720073010 msgr2=0x7fa7200cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa720073010 0x7fa7200cf9f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa718009d90 tx=0x7fa71801c940 comp rx=0 tx=0).stop 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/576791492 shutdown_connections 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa720073010 0x7fa7200cf9f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- 172.21.15.82:0/576791492 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/576791492 >> 172.21.15.82:0/576791492 conn(0x7fa72006ae60 msgr2=0x7fa72006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/576791492 shutdown_connections 2024-04-25T10:03:28.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/576791492 wait complete. 2024-04-25T10:03:28.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 Processor -- start 2024-04-25T10:03:28.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- start start 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa720073010 0x7fa720098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200ca900 con 0x7fa7200cf590 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cba30 con 0x7fa720073010 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7200cadb0 con 0x7fa72015b6b0 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34074/0 (socket says 172.21.15.82:34074) 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 -- 172.21.15.82:0/3475448884 learned_addr learned my addr 172.21.15.82:0/3475448884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 -- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 msgr2=0x7fa7200ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa726369640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725b68640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa720073010 0x7fa720098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 -- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa720073010 msgr2=0x7fa720098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa720073010 0x7fa720098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 -- 172.21.15.82:0/3475448884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720158ec0 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa725367640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7100011f0 tx=0x7fa71000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710018070 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7100074e0 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa720151d20 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710004a10 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.339+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7200acf90 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa720158cb0 con 0x7fa7200cf590 2024-04-25T10:03:28.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa71001b070 con 0x7fa7200cf590 2024-04-25T10:03:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa7167fc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 0x7fa704069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa725b68640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 0x7fa704069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa7100840d0 con 0x7fa7200cf590 2024-04-25T10:03:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.343+0000 7fa725b68640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 0x7fa704069450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0096f0 tx=0x7fa71c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.347+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa710055330 con 0x7fa7200cf590 2024-04-25T10:03:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.503+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa72004efa0 con 0x7fa7200cf590 2024-04-25T10:03:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.503+0000 7fa7167fc640 1 -- 172.21.15.82:0/3475448884 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa710054c30 con 0x7fa7200cf590 2024-04-25T10:03:28.512 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 msgr2=0x7fa704069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 0x7fa704069450 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0096f0 tx=0x7fa71c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 msgr2=0x7fa7201517e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa7100011f0 tx=0x7fa71000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 shutdown_connections 2024-04-25T10:03:28.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa704066f90 0x7fa704069450 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa72015b6b0 0x7fa7200ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7200cf590 0x7fa7201517e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 --2- 172.21.15.82:0/3475448884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa720073010 0x7fa720098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 >> 172.21.15.82:0/3475448884 conn(0x7fa72006ae60 msgr2=0x7fa7200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 shutdown_connections 2024-04-25T10:03:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.507+0000 7fa727dcc640 1 -- 172.21.15.82:0/3475448884 wait complete. 2024-04-25T10:03:28.529 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:28.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 Processor -- start 2024-04-25T10:03:28.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 -- start start 2024-04-25T10:03:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d380ce510 0x7f6d380ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d381451f0 0x7f6d38142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d380cee30 con 0x7f6d38144ca0 2024-04-25T10:03:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d380cc2b0 con 0x7f6d381451f0 2024-04-25T10:03:28.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3efa9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3806e290 con 0x7f6d380ce510 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d381451f0 0x7f6d38142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d381451f0 0x7f6d38142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34086/0 (socket says 172.21.15.82:34086) 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 -- 172.21.15.82:0/2782436287 learned_addr learned my addr 172.21.15.82:0/2782436287 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3cd45640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d380ce510 0x7f6d380ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 -- 172.21.15.82:0/2782436287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d380ce510 msgr2=0x7f6d380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d380ce510 0x7f6d380ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 -- 172.21.15.82:0/2782436287 >> v1:172.21.15.196:6789/0 conn(0x7f6d38144ca0 legacy=0x7f6d3806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.631+0000 7f6d3d546640 1 -- 172.21.15.82:0/2782436287 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380ccc70 con 0x7f6d381451f0 2024-04-25T10:03:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3d546640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d381451f0 0x7f6d38142680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009e50 tx=0x7f6d3001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4cba32ff7618284 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d377fe640 1 -- 172.21.15.82:0/2782436287 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300b4070 con 0x7f6d381451f0 2024-04-25T10:03:28.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d377fe640 1 -- 172.21.15.82:0/2782436287 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d300af400 con 0x7f6d381451f0 2024-04-25T10:03:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d377fe640 1 -- 172.21.15.82:0/2782436287 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d300af7d0 con 0x7f6d381451f0 2024-04-25T10:03:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/2782436287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d381451f0 msgr2=0x7f6d38142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d381451f0 0x7f6d38142680 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009e50 tx=0x7f6d3001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/2782436287 shutdown_connections 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d381451f0 0x7f6d38142680 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/2782436287 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d380ce510 0x7f6d380ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/2782436287 >> 172.21.15.82:0/2782436287 conn(0x7f6d380b11a0 msgr2=0x7f6d380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/2782436287 shutdown_connections 2024-04-25T10:03:28.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/2782436287 wait complete. 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 Processor -- start 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- start start 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d380ce510 0x7f6d380a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d381451f0 0x7f6d381e1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3806ee30 con 0x7f6d38144ca0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3806df20 con 0x7f6d380ce510 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d380cc900 con 0x7f6d381451f0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34088/0 (socket says 172.21.15.82:34088) 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 -- 172.21.15.82:0/1175344506 learned_addr learned my addr 172.21.15.82:0/1175344506 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3cd45640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d380ce510 0x7f6d380a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3d546640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d381451f0 0x7f6d381e1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 -- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d381451f0 msgr2=0x7f6d381e1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d381451f0 0x7f6d381e1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 -- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d380ce510 msgr2=0x7f6d380a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d380ce510 0x7f6d380a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 -- 172.21.15.82:0/1175344506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d381e1740 con 0x7f6d38144ca0 2024-04-25T10:03:28.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d37fff640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c410 tx=0x7f6d2c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c011790 con 0x7f6d38144ca0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c017070 con 0x7f6d38144ca0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d381e19b0 con 0x7f6d38144ca0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c011bb0 con 0x7f6d38144ca0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.635+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d381e1f70 con 0x7f6d38144ca0 2024-04-25T10:03:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3804efa0 con 0x7f6d38144ca0 2024-04-25T10:03:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d2c01c460 con 0x7f6d38144ca0 2024-04-25T10:03:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d357fa640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 0x7f6d24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d3cd45640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 0x7f6d24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6d2c084ad0 con 0x7f6d38144ca0 2024-04-25T10:03:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.639+0000 7f6d3cd45640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 0x7f6d24069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800b9d0 tx=0x7f6d28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.643+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d2c021020 con 0x7f6d38144ca0 2024-04-25T10:03:28.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6d380b1cc0 con 0x7f6d38144ca0 2024-04-25T10:03:28.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d357fa640 1 -- 172.21.15.82:0/1175344506 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6d2c055ca0 con 0x7f6d38144ca0 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:28.807 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 msgr2=0x7f6d24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 0x7f6d24069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800b9d0 tx=0x7f6d28008040 comp rx=0 tx=0).stop 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 msgr2=0x7f6d381e0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00c410 tx=0x7f6d2c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 shutdown_connections 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d24066f90 0x7f6d24069450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d381451f0 0x7f6d381e1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d38144ca0 0x7f6d381e0cc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 --2- 172.21.15.82:0/1175344506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d380ce510 0x7f6d380a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.799+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 >> 172.21.15.82:0/1175344506 conn(0x7f6d380b11a0 msgr2=0x7f6d380b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.803+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 shutdown_connections 2024-04-25T10:03:28.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.803+0000 7f6d3efa9640 1 -- 172.21.15.82:0/1175344506 wait complete. 2024-04-25T10:03:28.824 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:28.824 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:28.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 Processor -- start 2024-04-25T10:03:28.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 -- start start 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcac0adc70 0x7fbcac06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac06e9f0 0x7fbcac0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 0x7fbcac15f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac0b28c0 con 0x7fbcac0adc70 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac15bb20 con 0x7fbcac0cfa50 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb3b42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0b2b60 con 0x7fbcac06e9f0 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcac0adc70 0x7fbcac06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcac0adc70 0x7fbcac06e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34094/0 (socket says 172.21.15.82:34094) 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 -- 172.21.15.82:0/3113722135 learned_addr learned my addr 172.21.15.82:0/3113722135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb10dd640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac06e9f0 0x7fbcac0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb20df640 1 --2- 172.21.15.82:0/3113722135 >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 0x7fbcac15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 -- 172.21.15.82:0/3113722135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac06e9f0 msgr2=0x7fbcac0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac06e9f0 0x7fbcac0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 -- 172.21.15.82:0/3113722135 >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 msgr2=0x7fbcac15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:28.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3113722135 >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 0x7fbcac15f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb18de640 1 -- 172.21.15.82:0/3113722135 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0b2e40 con 0x7fbcac0adc70 2024-04-25T10:03:28.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.939+0000 7fbcb20df640 1 --2- 172.21.15.82:0/3113722135 >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 0x7fbcac15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:28.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0adc70 0x7fbcac06e4b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0d03b0 tx=0x7fbca801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595085ba05b637fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb08dc640 1 -- 172.21.15.82:0/3113722135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80b0070 con 0x7fbcac0adc70 2024-04-25T10:03:28.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb08dc640 1 -- 172.21.15.82:0/3113722135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80ab600 con 0x7fbcac0adc70 2024-04-25T10:03:28.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb08dc640 1 -- 172.21.15.82:0/3113722135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80ab9d0 con 0x7fbcac0adc70 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3113722135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0adc70 msgr2=0x7fbcac06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0adc70 0x7fbcac06e4b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fbcac0d03b0 tx=0x7fbca801c260 comp rx=0 tx=0).stop 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3113722135 shutdown_connections 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3113722135 >> v2:172.21.15.196:3300/0 conn(0x7fbcac0cfa50 0x7fbcac15f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac06e9f0 0x7fbcac0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3113722135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0adc70 0x7fbcac06e4b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3113722135 >> 172.21.15.82:0/3113722135 conn(0x7fbcac06ae60 msgr2=0x7fbcac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3113722135 shutdown_connections 2024-04-25T10:03:28.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3113722135 wait complete. 2024-04-25T10:03:28.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 Processor -- start 2024-04-25T10:03:28.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.943+0000 7fbcb3b42640 1 -- start start 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 0x7fbcac154030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 0x7fbcac15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0b3c00 con 0x7fbcac0cfa50 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0cfe60 con 0x7fbcac06e9f0 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac071b90 con 0x7fbcac0adc70 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34102/0 (socket says 172.21.15.82:34102) 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 -- 172.21.15.82:0/3668765183 learned_addr learned my addr 172.21.15.82:0/3668765183 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb10dd640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 0x7fbcac15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 0x7fbcac154030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 -- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 msgr2=0x7fbcac15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 0x7fbcac15a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 -- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 msgr2=0x7fbcac154030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 0x7fbcac154030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 -- 172.21.15.82:0/3668765183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac159e20 con 0x7fbcac0cfa50 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb10dd640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 0x7fbcac15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 0x7fbcac154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb20df640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbca400cd50 tx=0x7fbca400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca400f680 con 0x7fbcac0cfa50 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca400f840 con 0x7fbcac0cfa50 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac15a100 con 0x7fbcac0cfa50 2024-04-25T10:03:28.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca400fc30 con 0x7fbcac0cfa50 2024-04-25T10:03:28.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac155510 con 0x7fbcac0cfa50 2024-04-25T10:03:28.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac067c30 con 0x7fbcac0cfa50 2024-04-25T10:03:28.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbca40213f0 con 0x7fbcac0cfa50 2024-04-25T10:03:28.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 0x7fbc90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:28.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbca4085c90 con 0x7fbcac0cfa50 2024-04-25T10:03:28.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 0x7fbc90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:28.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.947+0000 7fbcb18de640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 0x7fbc90069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbca80ad000 tx=0x7fbca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:28.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:28.951+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbca4014030 con 0x7fbcac0cfa50 2024-04-25T10:03:29.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.115+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbcac06e7d0 con 0x7fbcac0cfa50 2024-04-25T10:03:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.299+0000 7fbca27fc640 1 -- 172.21.15.82:0/3668765183 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbca4056e60 con 0x7fbcac0cfa50 2024-04-25T10:03:29.308 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 msgr2=0x7fbc90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 0x7fbc90069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbca80ad000 tx=0x7fbca8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 msgr2=0x7fbcac15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fbca400cd50 tx=0x7fbca400a650 comp rx=0 tx=0).stop 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 shutdown_connections 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc90066f60 0x7fbc90069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcac0adc70 0x7fbcac15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcac0cfa50 0x7fbcac15af10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 --2- 172.21.15.82:0/3668765183 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcac06e9f0 0x7fbcac154030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 >> 172.21.15.82:0/3668765183 conn(0x7fbcac06ae60 msgr2=0x7fbcac073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 shutdown_connections 2024-04-25T10:03:29.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.303+0000 7fbcb3b42640 1 -- 172.21.15.82:0/3668765183 wait complete. 2024-04-25T10:03:29.337 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:29.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 Processor -- start 2024-04-25T10:03:29.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 -- start start 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70a40b3cf0 0x7f70a40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70a415e6c0 0x7f70a40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a40cbf10 con 0x7f70a40cfad0 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a40cff40 con 0x7f70a415e6c0 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70ab125640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a415eae0 con 0x7f70a40b3cf0 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70a415e6c0 0x7f70a40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70a415e6c0 0x7f70a40caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34118/0 (socket says 172.21.15.82:34118) 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 -- 172.21.15.82:0/1318016720 learned_addr learned my addr 172.21.15.82:0/1318016720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a8ec1640 1 --2- 172.21.15.82:0/1318016720 >> v2:172.21.15.196:3300/0 conn(0x7f70a40b3cf0 0x7f70a40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 -- 172.21.15.82:0/1318016720 >> v2:172.21.15.196:3300/0 conn(0x7f70a40b3cf0 msgr2=0x7f70a40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:29.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 --2- 172.21.15.82:0/1318016720 >> v2:172.21.15.196:3300/0 conn(0x7f70a40b3cf0 0x7f70a40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 -- 172.21.15.82:0/1318016720 >> v1:172.21.15.196:6789/0 conn(0x7f70a40cfad0 legacy=0x7f70a415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.455+0000 7f70a96c2640 1 -- 172.21.15.82:0/1318016720 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a40cc1f0 con 0x7f70a415e6c0 2024-04-25T10:03:29.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70a96c2640 1 --2- 172.21.15.82:0/1318016720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a415e6c0 0x7f70a40caaf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f70a40cbd40 tx=0x7f709c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eda6011aa4a6319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709b7fe640 1 -- 172.21.15.82:0/1318016720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a56c0 con 0x7f70a415e6c0 2024-04-25T10:03:29.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709b7fe640 1 -- 172.21.15.82:0/1318016720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c0a5860 con 0x7f70a415e6c0 2024-04-25T10:03:29.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709b7fe640 1 -- 172.21.15.82:0/1318016720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0a5c30 con 0x7f70a415e6c0 2024-04-25T10:03:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- 172.21.15.82:0/1318016720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a415e6c0 msgr2=0x7f70a40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- 172.21.15.82:0/1318016720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a415e6c0 0x7f70a40caaf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f70a40cbd40 tx=0x7f709c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- 172.21.15.82:0/1318016720 shutdown_connections 2024-04-25T10:03:29.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- 172.21.15.82:0/1318016720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a415e6c0 0x7f70a40caaf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- 172.21.15.82:0/1318016720 >> v2:172.21.15.196:3300/0 conn(0x7f70a40b3cf0 0x7f70a40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- 172.21.15.82:0/1318016720 >> 172.21.15.82:0/1318016720 conn(0x7f70a406ae60 msgr2=0x7f70a406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- 172.21.15.82:0/1318016720 shutdown_connections 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- 172.21.15.82:0/1318016720 wait complete. 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 Processor -- start 2024-04-25T10:03:29.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- start start 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 0x7f70a4157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70a415e6c0 0x7f70a415cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b4250 con 0x7f70a40cfad0 2024-04-25T10:03:29.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40b40d0 con 0x7f70a415e6c0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f70ab125640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a40cb840 con 0x7f70a40b3cf0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34132/0 (socket says 172.21.15.82:34132) 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.459+0000 7f709bfff640 1 -- 172.21.15.82:0/3952248151 learned_addr learned my addr 172.21.15.82:0/3952248151 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70a8ec1640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 0x7f70a4157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70a96c2640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70a415e6c0 0x7f70a415cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 -- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 msgr2=0x7f70a4157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 0x7f70a4157300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 -- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70a415e6c0 msgr2=0x7f70a415cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70a415e6c0 0x7f70a415cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 -- 172.21.15.82:0/3952248151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4156260 con 0x7f70a40cfad0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70a8ec1640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 0x7f70a4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f709bfff640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f709400c420 tx=0x7f709400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70940052d0 con 0x7f70a40cfad0 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7094005470 con 0x7f70a40cfad0 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a4156540 con 0x7f70a40cfad0 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7094007260 con 0x7f70a40cfad0 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a4156de0 con 0x7f70a40cfad0 2024-04-25T10:03:29.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4156fe0 con 0x7f70a40cfad0 2024-04-25T10:03:29.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7094007400 con 0x7f70a40cfad0 2024-04-25T10:03:29.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:29.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f7094084cb0 con 0x7f70a40cfad0 2024-04-25T10:03:29.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70a8ec1640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:29.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.463+0000 7f70a8ec1640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 0x7f7090069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009840 tx=0x7f70a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:29.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.467+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70a4156fe0 con 0x7f70a40cfad0 2024-04-25T10:03:29.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:29.615+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f70a404efa0 con 0x7f70a40cfad0 2024-04-25T10:03:30.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70997fa640 1 -- 172.21.15.82:0/3952248151 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7094055df0 con 0x7f70a40cfad0 2024-04-25T10:03:30.322 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 msgr2=0x7f7090069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 0x7f7090069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70a0009840 tx=0x7f70a0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 msgr2=0x7f70a415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f709400c420 tx=0x7f709400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 shutdown_connections 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7090066f60 0x7f7090069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70a415e6c0 0x7f70a415cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70a40cfad0 0x7f70a415af50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 --2- 172.21.15.82:0/3952248151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70a40b3cf0 0x7f70a4157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 >> 172.21.15.82:0/3952248151 conn(0x7f70a406ae60 msgr2=0x7f70a415ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.315+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 shutdown_connections 2024-04-25T10:03:30.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.319+0000 7f70ab125640 1 -- 172.21.15.82:0/3952248151 wait complete. 2024-04-25T10:03:30.355 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 Processor -- start 2024-04-25T10:03:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 -- start start 2024-04-25T10:03:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f035015da80 0x7f03500d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035006be60 con 0x7f03500d07f0 2024-04-25T10:03:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f035015de60 con 0x7f03500d1cd0 2024-04-25T10:03:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f03568b0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035006c440 con 0x7f035015da80 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f034ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f035015da80 0x7f03500d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f0354e4d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f03500d07f0 0x7f035006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52620/0 (socket says 172.21.15.82:52620) 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f0354e4d640 1 -- 172.21.15.82:0/478148200 learned_addr learned my addr 172.21.15.82:0/478148200 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f034ffff640 1 -- 172.21.15.82:0/478148200 >> v1:172.21.15.196:6789/0 conn(0x7f03500d1cd0 legacy=0x7f03500d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f034ffff640 1 -- 172.21.15.82:0/478148200 >> v1:172.21.15.82:6789/0 conn(0x7f03500d07f0 legacy=0x7f035006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f034ffff640 1 -- 172.21.15.82:0/478148200 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500d2450 con 0x7f035015da80 2024-04-25T10:03:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.467+0000 7f034ffff640 1 --2- 172.21.15.82:0/478148200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f035015da80 0x7f03500d4230 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f035006c5c0 tx=0x7f034001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1767ca96e25a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034effd640 1 -- 172.21.15.82:0/478148200 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400b1070 con 0x7f035015da80 2024-04-25T10:03:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034effd640 1 -- 172.21.15.82:0/478148200 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03400ac410 con 0x7f035015da80 2024-04-25T10:03:30.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034effd640 1 -- 172.21.15.82:0/478148200 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03400ac7e0 con 0x7f035015da80 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/478148200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f035015da80 msgr2=0x7f03500d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 --2- 172.21.15.82:0/478148200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f035015da80 0x7f03500d4230 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f035006c5c0 tx=0x7f034001c410 comp rx=0 tx=0).stop 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/478148200 shutdown_connections 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 --2- 172.21.15.82:0/478148200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f035015da80 0x7f03500d4230 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/478148200 >> 172.21.15.82:0/478148200 conn(0x7f035008b000 msgr2=0x7f03500d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/478148200 shutdown_connections 2024-04-25T10:03:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/478148200 wait complete. 2024-04-25T10:03:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 Processor -- start 2024-04-25T10:03:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- start start 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03500d07f0 0x7f0350154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f035015da80 0x7f035015e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035006d970 con 0x7f03500d1cd0 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035006d7f0 con 0x7f035015da80 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035006daf0 con 0x7f03500d07f0 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34148/0 (socket says 172.21.15.82:34148) 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 -- 172.21.15.82:0/1970374855 learned_addr learned my addr 172.21.15.82:0/1970374855 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034ffff640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03500d07f0 0x7f0350154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f0354e4d640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f035015da80 0x7f035015e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 -- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03500d07f0 msgr2=0x7f0350154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03500d07f0 0x7f0350154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 -- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f035015da80 msgr2=0x7f035015e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f035015da80 0x7f035015e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 -- 172.21.15.82:0/1970374855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03500cd790 con 0x7f03500d1cd0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034f7fe640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f034400c430 tx=0x7f034400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011790 con 0x7f03500d1cd0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344017070 con 0x7f03500d1cd0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03500c75e0 con 0x7f03500d1cd0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344011b70 con 0x7f03500d1cd0 2024-04-25T10:03:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.471+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03500c7b60 con 0x7f03500d1cd0 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035004efa0 con 0x7f03500d1cd0 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0344011d10 con 0x7f03500d1cd0 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f034cff9640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f0344085570 con 0x7f03500d1cd0 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f034ffff640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.475+0000 7f034ffff640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 0x7f033c069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f03400ae000 tx=0x7f0340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.479+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03440102e0 con 0x7f03500d1cd0 2024-04-25T10:03:30.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:30.671+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f035006cd70 con 0x7f03500d1cd0 2024-04-25T10:03:31.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.343+0000 7f034cff9640 1 -- 172.21.15.82:0/1970374855 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f03440565c0 con 0x7f03500d1cd0 2024-04-25T10:03:31.355 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T10:03:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 msgr2=0x7f033c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 0x7f033c069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f03400ae000 tx=0x7f0340008040 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 msgr2=0x7f035015e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f034400c430 tx=0x7f034400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 shutdown_connections 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f033c066f60 0x7f033c069420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f035015da80 0x7f035015e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f03500d1cd0 0x7f035015e060 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 --2- 172.21.15.82:0/1970374855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f03500d07f0 0x7f0350154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.351+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 >> 172.21.15.82:0/1970374855 conn(0x7f035008b000 msgr2=0x7f0350073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.355+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 shutdown_connections 2024-04-25T10:03:31.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.355+0000 7f03568b0640 1 -- 172.21.15.82:0/1970374855 wait complete. 2024-04-25T10:03:31.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 Processor -- start 2024-04-25T10:03:31.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 -- start start 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe4380d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe4380d1cd0 0x7fe43806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43806be60 con 0x7fe43815da80 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43806c440 con 0x7fe4380d1cd0 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43f80d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43815de60 con 0x7fe4380d07f0 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43d5a9640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe43815da80 0x7fe4380d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52636/0 (socket says 172.21.15.82:52636) 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.499+0000 7fe43d5a9640 1 -- 172.21.15.82:0/616476278 learned_addr learned my addr 172.21.15.82:0/616476278 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43ddaa640 1 --2- 172.21.15.82:0/616476278 >> v2:172.21.15.196:3300/0 conn(0x7fe4380d1cd0 0x7fe43806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe4380d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525560626 0 0) 0x7fe43806be60 con 0x7fe43815da80 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe424003530 con 0x7fe43815da80 2024-04-25T10:03:31.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195559919 0 0) 0x7fe424003530 con 0x7fe43815da80 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 msgr2=0x7fe4380d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe4380d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 >> v2:172.21.15.196:3300/0 conn(0x7fe4380d1cd0 msgr2=0x7fe43806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 --2- 172.21.15.82:0/616476278 >> v2:172.21.15.196:3300/0 conn(0x7fe4380d1cd0 0x7fe43806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380d2450 con 0x7fe43815da80 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe434001680 con 0x7fe43815da80 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 >> v1:172.21.15.82:6789/0 conn(0x7fe43815da80 legacy=0x7fe4380d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe424003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe424003fb0 0x7fe424004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43806c440 con 0x7fe424003fb0 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43815de60 con 0x7fe424004d50 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42400bab0 con 0x7fe4380d07f0 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43ddaa640 1 --2- 172.21.15.82:0/616476278 >> v2:172.21.15.196:3300/0 conn(0x7fe4380d1cd0 0x7fe43806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe424003fb0 0x7fe424004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43d5a9640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe424003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 -- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 msgr2=0x7fe424003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe4380d07f0 0x7fe424003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43ddaa640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 -- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe424004d50 msgr2=0x7fe4240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe424004d50 0x7fe4240051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 -- 172.21.15.82:0/616476278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424003530 con 0x7fe424003fb0 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43cda8640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe424003fb0 0x7fe424004810 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe428009ac0 tx=0x7fe42801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdcca46295707b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428009690 con 0x7fe424003fb0 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4280059a0 con 0x7fe424003fb0 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe42ffff640 1 -- 172.21.15.82:0/616476278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe428005d90 con 0x7fe424003fb0 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43f80d640 1 -- 172.21.15.82:0/616476278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe424003fb0 msgr2=0x7fe424004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43f80d640 1 --2- 172.21.15.82:0/616476278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe424003fb0 0x7fe424004810 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe428009ac0 tx=0x7fe42801ca50 comp rx=0 tx=0).stop 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.503+0000 7fe43ddaa640 1 -- 172.21.15.82:0/616476278 reap_dead start 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/616476278 shutdown_connections 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/616476278 >> 172.21.15.82:0/616476278 conn(0x7fe43808b000 msgr2=0x7fe4380d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/616476278 shutdown_connections 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/616476278 wait complete. 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 Processor -- start 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- start start 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe424003fb0 0x7fe438151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 0x7fe4381f3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380d26f0 con 0x7fe4380b8d70 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380d2160 con 0x7fe4380cc1e0 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4380d2870 con 0x7fe424003fb0 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34178/0 (socket says 172.21.15.82:34178) 2024-04-25T10:03:31.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 -- 172.21.15.82:0/2426984962 learned_addr learned my addr 172.21.15.82:0/2426984962 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43d5a9640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe424003fb0 0x7fe438151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43ddaa640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 0x7fe4381f3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 -- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe424003fb0 msgr2=0x7fe438151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe424003fb0 0x7fe438151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 -- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 msgr2=0x7fe4381f3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 0x7fe4381f3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 -- 172.21.15.82:0/2426984962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe438152080 con 0x7fe4380b8d70 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43ddaa640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 0x7fe4381f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43cda8640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe42801c880 tx=0x7fe424005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280bc070 con 0x7fe4380b8d70 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe428004240 con 0x7fe4380b8d70 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4280a5850 con 0x7fe4380b8d70 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4380cb420 con 0x7fe4380b8d70 2024-04-25T10:03:31.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.507+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4380cb9e0 con 0x7fe4380b8d70 2024-04-25T10:03:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43804efa0 con 0x7fe4380b8d70 2024-04-25T10:03:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4280a59f0 con 0x7fe4380b8d70 2024-04-25T10:03:31.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe42dffb640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 0x7fe41c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe42811fa10 con 0x7fe4380b8d70 2024-04-25T10:03:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe43d5a9640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 0x7fe41c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.511+0000 7fe43d5a9640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 0x7fe41c069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4340018f0 tx=0x7fe4340015e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.515+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe428016760 con 0x7fe4380b8d70 2024-04-25T10:03:31.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.675+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4380b0440 con 0x7fe4380b8d70 2024-04-25T10:03:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe42dffb640 1 -- 172.21.15.82:0/2426984962 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16967 (secure 0 0 0) 0x7fe4280f0a40 con 0x7fe4380b8d70 2024-04-25T10:03:31.685 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:31.686 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:30.679194+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:31.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 msgr2=0x7fe41c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 0x7fe41c069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe4340018f0 tx=0x7fe4340015e0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 msgr2=0x7fe4380ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe42801c880 tx=0x7fe424005840 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 shutdown_connections 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe41c066f90 0x7fe41c069450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe4380cc1e0 0x7fe4381f3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe4380b8d70 0x7fe4380ccd10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 --2- 172.21.15.82:0/2426984962 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe424003fb0 0x7fe438151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 >> 172.21.15.82:0/2426984962 conn(0x7fe43808b000 msgr2=0x7fe4380ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 shutdown_connections 2024-04-25T10:03:31.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.679+0000 7fe43f80d640 1 -- 172.21.15.82:0/2426984962 wait complete. 2024-04-25T10:03:31.704 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:31.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.819+0000 7f562f512640 1 Processor -- start 2024-04-25T10:03:31.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.819+0000 7f562f512640 1 -- start start 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562f512640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5628071740 0x7f56280acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562f512640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f56280ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562f512640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56280ae860 con 0x7f562815da10 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562f512640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f562815e210 con 0x7f5628071740 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562f512640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562815e4f0 con 0x7f56280b24a0 2024-04-25T10:03:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562d2ae640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f562815da10 0x7f5628071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52638/0 (socket says 172.21.15.82:52638) 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562d2ae640 1 -- 172.21.15.82:0/2365220222 learned_addr learned my addr 172.21.15.82:0/2365220222 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562caad640 1 --2- 172.21.15.82:0/2365220222 >> v2:172.21.15.196:3300/0 conn(0x7f5628071740 0x7f56280acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f56280ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307207308 0 0) 0x7f56280ae860 con 0x7f562815da10 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5614003530 con 0x7f562815da10 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 7544416 0 0) 0x7f5614003530 con 0x7f562815da10 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 msgr2=0x7f56280ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f56280ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 >> v2:172.21.15.196:3300/0 conn(0x7f5628071740 msgr2=0x7f56280acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 --2- 172.21.15.82:0/2365220222 >> v2:172.21.15.196:3300/0 conn(0x7f5628071740 0x7f56280acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56280ae4c0 con 0x7f562815da10 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5624001680 con 0x7f562815da10 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 >> v1:172.21.15.82:6789/0 conn(0x7f562815da10 legacy=0x7f5628071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5614003a30 0x7f5614003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f5614004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f56140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562815e210 con 0x7f5614004d50 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562815e4f0 con 0x7f5614003a30 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561400bad0 con 0x7f56280b24a0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f56140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562caad640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f5614004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562d2ae640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5614003a30 0x7f5614003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 -- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 msgr2=0x7f5614004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56280b24a0 0x7f5614004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 -- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5614003a30 msgr2=0x7f5614003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5614003a30 0x7f5614003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.823+0000 7f562daaf640 1 -- 172.21.15.82:0/2365220222 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614003530 con 0x7f5614004d50 2024-04-25T10:03:31.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562daaf640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f56140051d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f562000e170 tx=0x7f562001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ec740fddbf0f0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200b2070 con 0x7f5614004d50 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56200a5a50 con 0x7f5614004d50 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f561bfff640 1 -- 172.21.15.82:0/2365220222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620009c10 con 0x7f5614004d50 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/2365220222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 msgr2=0x7f56140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 --2- 172.21.15.82:0/2365220222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f56140051d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f562000e170 tx=0x7f562001cc60 comp rx=0 tx=0).stop 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562daaf640 1 -- 172.21.15.82:0/2365220222 reap_dead start 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/2365220222 shutdown_connections 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/2365220222 >> 172.21.15.82:0/2365220222 conn(0x7f56280b11a0 msgr2=0x7f56280b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/2365220222 shutdown_connections 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/2365220222 wait complete. 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 Processor -- start 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- start start 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562815cad0 0x7f5628151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f562815cf50 0x7f5628151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562815ef00 con 0x7f5614004d50 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56280adb20 con 0x7f562815cf50 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5628168880 con 0x7f562815cad0 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34188/0 (socket says 172.21.15.82:34188) 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 -- 172.21.15.82:0/752617417 learned_addr learned my addr 172.21.15.82:0/752617417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562caad640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562815cad0 0x7f5628151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562daaf640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f562815cf50 0x7f5628151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 -- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562815cad0 msgr2=0x7f5628151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562815cad0 0x7f5628151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 -- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f562815cf50 msgr2=0x7f5628151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f562815cf50 0x7f5628151c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 -- 172.21.15.82:0/752617417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562814ba00 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562d2ae640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f562400d770 tx=0x7f562400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5624019070 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56240074e0 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562815b010 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56240049f0 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.827+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562815b5b0 con 0x7f5614004d50 2024-04-25T10:03:31.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562815aac0 con 0x7f5614004d50 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f562401c070 con 0x7f5614004d50 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f5619ffb640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 0x7f560c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5624084be0 con 0x7f5614004d50 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f562caad640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 0x7f560c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.831+0000 7f562caad640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 0x7f560c069450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5628152a80 tx=0x7f561c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.835+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5624055a60 con 0x7f5614004d50 2024-04-25T10:03:32.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.995+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f562804ef80 con 0x7f5614004d50 2024-04-25T10:03:32.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f5619ffb640 1 -- 172.21.15.82:0/752617417 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f5624055360 con 0x7f5614004d50 2024-04-25T10:03:32.005 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:32.006 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":38,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:30.679194+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 msgr2=0x7f560c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 0x7f560c069450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5628152a80 tx=0x7f561c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 msgr2=0x7f562814b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f562400d770 tx=0x7f562400dc30 comp rx=0 tx=0).stop 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 shutdown_connections 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f90 0x7f560c069450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562815cad0 0x7f5628151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f562815cf50 0x7f5628151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 --2- 172.21.15.82:0/752617417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5614004d50 0x7f562814b4c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 >> 172.21.15.82:0/752617417 conn(0x7f56280b11a0 msgr2=0x7f562815f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 shutdown_connections 2024-04-25T10:03:32.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:31.999+0000 7f562f512640 1 -- 172.21.15.82:0/752617417 wait complete. 2024-04-25T10:03:32.025 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:32.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 Processor -- start 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 -- start start 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700cf590 0x7f577015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57700ce170 0x7f57700ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700ca4c0 con 0x7f57700cf970 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57700ca780 con 0x7f57700ce170 2024-04-25T10:03:32.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5776bda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700cd440 con 0x7f57700cf590 2024-04-25T10:03:32.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57700ce170 0x7f57700ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5774976640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700cf590 0x7f577015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f57700ce170 0x7f57700ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:47988/0 (socket says 172.21.15.82:47988) 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 -- 172.21.15.82:0/1330810650 learned_addr learned my addr 172.21.15.82:0/1330810650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 -- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700cf590 msgr2=0x7f577015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 --2- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700cf590 0x7f577015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 -- 172.21.15.82:0/1330810650 >> v1:172.21.15.196:6789/0 conn(0x7f57700cf970 legacy=0x7f57700c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 -- 172.21.15.82:0/1330810650 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700cd690 con 0x7f57700ce170 2024-04-25T10:03:32.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f5775177640 1 --2- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700ce170 0x7f57700ce600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5768009e50 tx=0x7f576801ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c34b5e7c052bc0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f576f7fe640 1 -- 172.21.15.82:0/1330810650 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680a5810 con 0x7f57700ce170 2024-04-25T10:03:32.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f576f7fe640 1 -- 172.21.15.82:0/1330810650 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57680a59b0 con 0x7f57700ce170 2024-04-25T10:03:32.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.143+0000 7f576f7fe640 1 -- 172.21.15.82:0/1330810650 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680a5d80 con 0x7f57700ce170 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700ce170 msgr2=0x7f57700ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700ce170 0x7f57700ce600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5768009e50 tx=0x7f576801ca30 comp rx=0 tx=0).stop 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/1330810650 shutdown_connections 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700ce170 0x7f57700ce600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- 172.21.15.82:0/1330810650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700cf590 0x7f577015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/1330810650 >> 172.21.15.82:0/1330810650 conn(0x7f577006ae60 msgr2=0x7f577006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/1330810650 shutdown_connections 2024-04-25T10:03:32.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/1330810650 wait complete. 2024-04-25T10:03:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 Processor -- start 2024-04-25T10:03:32.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- start start 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700ce170 0x7f57700c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 0x7f57700c5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700cfed0 con 0x7f57700cf970 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700cfd50 con 0x7f57700cf590 2024-04-25T10:03:32.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700ac170 con 0x7f57700ce170 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34190/0 (socket says 172.21.15.82:34190) 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 -- 172.21.15.82:0/3984348939 learned_addr learned my addr 172.21.15.82:0/3984348939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5774976640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700ce170 0x7f57700c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f576ffff640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 0x7f57700c5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 -- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700ce170 msgr2=0x7f57700c2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700ce170 0x7f57700c2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 -- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 msgr2=0x7f57700c5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 0x7f57700c5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 -- 172.21.15.82:0/3984348939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700c3750 con 0x7f57700cf970 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f576ffff640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 0x7f57700c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:32.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5775177640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f576801a660 tx=0x7f57680197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680b8670 con 0x7f57700cf970 2024-04-25T10:03:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57680be070 con 0x7f57700cf970 2024-04-25T10:03:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57700c5c90 con 0x7f57700cf970 2024-04-25T10:03:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680b8a50 con 0x7f57700cf970 2024-04-25T10:03:32.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.147+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5770152810 con 0x7f57700cf970 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577004efa0 con 0x7f57700cf970 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57680b8bf0 con 0x7f57700cf970 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f576d7fa640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 0x7f575c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f57681201b0 con 0x7f57700cf970 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f5774976640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 0x7f575c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:32.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.151+0000 7f5774976640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 0x7f575c069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5760009af0 tx=0x7f5760008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:32.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.155+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57700c3750 con 0x7f57700cf970 2024-04-25T10:03:32.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:32.359+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f57700c9bf0 con 0x7f57700cf970 2024-04-25T10:03:33.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.351+0000 7f576d7fa640 1 -- 172.21.15.82:0/3984348939 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5768016620 con 0x7f57700cf970 2024-04-25T10:03:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 msgr2=0x7f575c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 0x7f575c069420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5760009af0 tx=0x7f5760008040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 msgr2=0x7f57701532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f576801a660 tx=0x7f57680197d0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 shutdown_connections 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f575c066f60 0x7f575c069420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57700cf970 0x7f57701532f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57700cf590 0x7f57700c5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 --2- 172.21.15.82:0/3984348939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57700ce170 0x7f57700c2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 >> 172.21.15.82:0/3984348939 conn(0x7f577006ae60 msgr2=0x7f5770156a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 shutdown_connections 2024-04-25T10:03:33.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.355+0000 7f5776bda640 1 -- 172.21.15.82:0/3984348939 wait complete. 2024-04-25T10:03:33.384 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:33.384 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:33.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 Processor -- start 2024-04-25T10:03:33.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- start start 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 0x7fd70c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd70c0cf590 0x7fd70c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70c0cbd20 con 0x7fd70c15f130 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70c15ba90 con 0x7fd70c0cf590 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd70c0cc000 con 0x7fd70c15b6b0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd70c0cf590 0x7fd70c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd70c0cf590 0x7fd70c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34204/0 (socket says 172.21.15.82:34204) 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 -- 172.21.15.82:0/4151176301 learned_addr learned my addr 172.21.15.82:0/4151176301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd7115d4640 1 --2- 172.21.15.82:0/4151176301 >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 0x7fd70c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 -- 172.21.15.82:0/4151176301 >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 msgr2=0x7fd70c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 --2- 172.21.15.82:0/4151176301 >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 0x7fd70c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 -- 172.21.15.82:0/4151176301 >> v1:172.21.15.196:6789/0 conn(0x7fd70c15f130 legacy=0x7fd70c15f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 -- 172.21.15.82:0/4151176301 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0cc2e0 con 0x7fd70c0cf590 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd7115d4640 1 --2- 172.21.15.82:0/4151176301 >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 0x7fd70c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd710dd3640 1 --2- 172.21.15.82:0/4151176301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c0cf590 0x7fd70c0cfa20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0d0180 tx=0x7fd6fc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7731692ec0127b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd703fff640 1 -- 172.21.15.82:0/4151176301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0b1070 con 0x7fd70c0cf590 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd703fff640 1 -- 172.21.15.82:0/4151176301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc0ac630 con 0x7fd70c0cf590 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd703fff640 1 -- 172.21.15.82:0/4151176301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc0a8050 con 0x7fd70c0cf590 2024-04-25T10:03:33.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- 172.21.15.82:0/4151176301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c0cf590 msgr2=0x7fd70c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 --2- 172.21.15.82:0/4151176301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c0cf590 0x7fd70c0cfa20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0d0180 tx=0x7fd6fc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- 172.21.15.82:0/4151176301 shutdown_connections 2024-04-25T10:03:33.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 --2- 172.21.15.82:0/4151176301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c0cf590 0x7fd70c0cfa20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 --2- 172.21.15.82:0/4151176301 >> v2:172.21.15.196:3300/0 conn(0x7fd70c15b6b0 0x7fd70c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- 172.21.15.82:0/4151176301 >> 172.21.15.82:0/4151176301 conn(0x7fd70c06ae60 msgr2=0x7fd70c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- 172.21.15.82:0/4151176301 shutdown_connections 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- 172.21.15.82:0/4151176301 wait complete. 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 Processor -- start 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.511+0000 7fd713838640 1 -- start start 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70c0cf590 0x7fd70c152bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70c15b6b0 0x7fd70c0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c0cadb0 con 0x7fd70c15f130 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c0ca900 con 0x7fd70c0cf590 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70c0caf30 con 0x7fd70c15b6b0 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34212/0 (socket says 172.21.15.82:34212) 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 -- 172.21.15.82:0/153354814 learned_addr learned my addr 172.21.15.82:0/153354814 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:33.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd7115d4640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70c0cf590 0x7fd70c152bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd710dd3640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70c15b6b0 0x7fd70c0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 -- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70c15b6b0 msgr2=0x7fd70c0c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70c15b6b0 0x7fd70c0c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 -- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70c0cf590 msgr2=0x7fd70c152bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70c0cf590 0x7fd70c152bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 -- 172.21.15.82:0/153354814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70c0c2490 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd711dd5640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd70400b9a0 tx=0x7fd70400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704019070 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70400cba0 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70c097730 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704014440 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd70c097cf0 con 0x7fd70c15f130 2024-04-25T10:03:33.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70c04efa0 con 0x7fd70c15f130 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd70401c070 con 0x7fd70c15f130 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 0x7fd6f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd704084dc0 con 0x7fd70c15f130 2024-04-25T10:03:33.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd7115d4640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 0x7fd6f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.515+0000 7fd7115d4640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 0x7fd6f0069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd70800a9a0 tx=0x7fd708008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.519+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd704055c40 con 0x7fd70c15f130 2024-04-25T10:03:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:33.715+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd70c165df0 con 0x7fd70c15f130 2024-04-25T10:03:34.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.363+0000 7fd701ffb640 1 -- 172.21.15.82:0/153354814 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd704055540 con 0x7fd70c15f130 2024-04-25T10:03:34.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 msgr2=0x7fd6f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 0x7fd6f0069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fd70800a9a0 tx=0x7fd708008040 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 msgr2=0x7fd70c156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fd70400b9a0 tx=0x7fd70400be60 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 shutdown_connections 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6f0066f60 0x7fd6f0069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd70c15b6b0 0x7fd70c0c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd70c15f130 0x7fd70c156dd0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 --2- 172.21.15.82:0/153354814 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd70c0cf590 0x7fd70c152bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 >> 172.21.15.82:0/153354814 conn(0x7fd70c06ae60 msgr2=0x7fd70c06cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 shutdown_connections 2024-04-25T10:03:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.367+0000 7fd713838640 1 -- 172.21.15.82:0/153354814 wait complete. 2024-04-25T10:03:34.392 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:34.392 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 Processor -- start 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 -- start start 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf015ed40 0x7ffaf00b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf00b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf00b1000 con 0x7ffaf009ac10 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf00b1610 con 0x7ffaf015ed40 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaf4d95640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00d0500 con 0x7ffaf015f120 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf015ed40 0x7ffaf00b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf015ed40 0x7ffaf00b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34224/0 (socket says 172.21.15.82:34224) 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 -- 172.21.15.82:0/3397453600 learned_addr learned my addr 172.21.15.82:0/3397453600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaedd9b640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf00b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 -- 172.21.15.82:0/3397453600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 msgr2=0x7ffaf00b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf00b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 -- 172.21.15.82:0/3397453600 >> v1:172.21.15.196:6789/0 conn(0x7ffaf009ac10 legacy=0x7ffaf00cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaee59c640 1 -- 172.21.15.82:0/3397453600 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00d0220 con 0x7ffaf015ed40 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.507+0000 7ffaedd9b640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf00b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:34.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaee59c640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf00b1aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e50 tx=0x7ffae001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aa02cf3897c4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaed59a640 1 -- 172.21.15.82:0/3397453600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00a5860 con 0x7ffaf015ed40 2024-04-25T10:03:34.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaed59a640 1 -- 172.21.15.82:0/3397453600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae00a5a00 con 0x7ffaf015ed40 2024-04-25T10:03:34.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaed59a640 1 -- 172.21.15.82:0/3397453600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00a5dd0 con 0x7ffaf015ed40 2024-04-25T10:03:34.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/3397453600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 msgr2=0x7ffaf00b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf00b1aa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009e50 tx=0x7ffae001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/3397453600 shutdown_connections 2024-04-25T10:03:34.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf00b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/3397453600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf00b1aa0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/3397453600 >> 172.21.15.82:0/3397453600 conn(0x7ffaf00b3020 msgr2=0x7ffaf00b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/3397453600 shutdown_connections 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/3397453600 wait complete. 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 Processor -- start 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- start start 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf009ac10 0x7ffaf006ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf0070360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0168810 con 0x7ffaf015ed40 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00b2cb0 con 0x7ffaf009ac10 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf0168990 con 0x7ffaf015f120 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34238/0 (socket says 172.21.15.82:34238) 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 -- 172.21.15.82:0/1678125939 learned_addr learned my addr 172.21.15.82:0/1678125939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaee59c640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf009ac10 0x7ffaf006ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaeed9d640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf0070360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 -- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 msgr2=0x7ffaf0070360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf0070360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 -- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf009ac10 msgr2=0x7ffaf006ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf009ac10 0x7ffaf006ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 -- 172.21.15.82:0/1678125939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0164fe0 con 0x7ffaf015ed40 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaeed9d640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf0070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaedd9b640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c7f0 tx=0x7ffad800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8015070 con 0x7ffaf015ed40 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8007820 con 0x7ffaf015ed40 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf01652c0 con 0x7ffaf015ed40 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8007c10 con 0x7ffaf015ed40 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf0163db0 con 0x7ffaf015ed40 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf004efa0 con 0x7ffaf015ed40 2024-04-25T10:03:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffad8018070 con 0x7ffaf015ed40 2024-04-25T10:03:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffad8085740 con 0x7ffaf015ed40 2024-04-25T10:03:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaee59c640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.515+0000 7ffaee59c640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 0x7ffad0069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffae00ae000 tx=0x7ffae00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.519+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffad80566a0 con 0x7ffaf015ed40 2024-04-25T10:03:34.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.711+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaf0165490 con 0x7ffaf015ed40 2024-04-25T10:03:34.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffae6ffd640 1 -- 172.21.15.82:0/1678125939 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7ffad8055fa0 con 0x7ffaf015ed40 2024-04-25T10:03:34.721 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:34.722 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:34.722 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:03:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 msgr2=0x7ffad0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 0x7ffad0069420 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffae00ae000 tx=0x7ffae00a7070 comp rx=0 tx=0).stop 2024-04-25T10:03:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 msgr2=0x7ffaf0152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ffad800c7f0 tx=0x7ffad800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 shutdown_connections 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf015f120 0x7ffaf0070360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf015ed40 0x7ffaf0152350 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 --2- 172.21.15.82:0/1678125939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf009ac10 0x7ffaf006ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 >> 172.21.15.82:0/1678125939 conn(0x7ffaf00b3020 msgr2=0x7ffaf00af2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 shutdown_connections 2024-04-25T10:03:34.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.715+0000 7ffaf4d95640 1 -- 172.21.15.82:0/1678125939 wait complete. 2024-04-25T10:03:34.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 Processor -- start 2024-04-25T10:03:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 -- start start 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bbc144ca0 0x7f3bbc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bbc0ccc70 con 0x7f3bbc0ce510 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bbc0cc2b0 con 0x7f3bbc144ca0 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bc0ec6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc06e290 con 0x7f3bbc06ec60 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bbc144ca0 0x7f3bbc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bbc144ca0 0x7f3bbc06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42066/0 (socket says 172.21.15.82:42066) 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/3225363049 learned_addr learned my addr 172.21.15.82:0/3225363049 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bbad9d640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/3225363049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 msgr2=0x7f3bbc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/3225363049 >> v1:172.21.15.196:6789/0 conn(0x7f3bbc0ce510 legacy=0x7f3bbc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/3225363049 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc06dc40 con 0x7f3bbc144ca0 2024-04-25T10:03:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bbad9d640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.847+0000 7f3bb1d9b640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc144ca0 0x7f3bbc06e720 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009870 tx=0x7f3ba401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7732cecf74c1377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb9d9b640 1 -- 172.21.15.82:0/3225363049 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40b5070 con 0x7f3bbc144ca0 2024-04-25T10:03:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb9d9b640 1 -- 172.21.15.82:0/3225363049 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba40a89d0 con 0x7f3bbc144ca0 2024-04-25T10:03:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb9d9b640 1 -- 172.21.15.82:0/3225363049 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40a8da0 con 0x7f3bbc144ca0 2024-04-25T10:03:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/3225363049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc144ca0 msgr2=0x7f3bbc06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc144ca0 0x7f3bbc06e720 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009870 tx=0x7f3ba401ed10 comp rx=0 tx=0).stop 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/3225363049 shutdown_connections 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/3225363049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc144ca0 0x7f3bbc06e720 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/3225363049 >> 172.21.15.82:0/3225363049 conn(0x7f3bbc0b11a0 msgr2=0x7f3bbc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/3225363049 shutdown_connections 2024-04-25T10:03:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/3225363049 wait complete. 2024-04-25T10:03:34.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 Processor -- start 2024-04-25T10:03:34.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- start start 2024-04-25T10:03:34.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc1eede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 0x7f3bbc1f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc1451f0 con 0x7f3bbc0ce510 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc0cc900 con 0x7f3bbc1e3050 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bbc145370 con 0x7f3bbc06ec60 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42074/0 (socket says 172.21.15.82:42074) 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/648470945 learned_addr learned my addr 172.21.15.82:0/648470945 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bbad9d640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 0x7f3bbc1f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 msgr2=0x7f3bbc1eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bba59c640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc1eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc1eede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 msgr2=0x7f3bbc1f4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 0x7f3bbc1f4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 -- 172.21.15.82:0/648470945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc1e3500 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bbad9d640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 0x7f3bbc1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb1d9b640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b3040 tx=0x7f3ba401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40c0070 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba40b0b60 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bbc1e37e0 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40b5070 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.851+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bbc1f2ae0 con 0x7f3bbc0ce510 2024-04-25T10:03:34.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bbc04efa0 con 0x7f3bbc0ce510 2024-04-25T10:03:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ba40c3070 con 0x7f3bbc0ce510 2024-04-25T10:03:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bb37fe640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 0x7f3b9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ba4122320 con 0x7f3bbc0ce510 2024-04-25T10:03:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bba59c640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 0x7f3b9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.855+0000 7f3bba59c640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 0x7f3b9c069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00a750 tx=0x7f3bac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:34.859+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ba4018870 con 0x7f3bbc0ce510 2024-04-25T10:03:35.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.015+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bbc0b04c0 con 0x7f3bbc0ce510 2024-04-25T10:03:35.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bb37fe640 1 -- 172.21.15.82:0/648470945 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f3ba40f3230 con 0x7f3bbc0ce510 2024-04-25T10:03:35.025 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":39,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:32.322932+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 msgr2=0x7f3b9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 0x7f3b9c069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00a750 tx=0x7f3bac008040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 msgr2=0x7f3bbc1e2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40b3040 tx=0x7f3ba401bab0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 shutdown_connections 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b9c066f60 0x7f3b9c069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bbc1e3050 0x7f3bbc1f4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bbc0ce510 0x7f3bbc1e2b10 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 --2- 172.21.15.82:0/648470945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bbc06ec60 0x7f3bbc1eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 >> 172.21.15.82:0/648470945 conn(0x7f3bbc0b11a0 msgr2=0x7f3bbc0c76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 shutdown_connections 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.019+0000 7f3bc0ec6640 1 -- 172.21.15.82:0/648470945 wait complete. 2024-04-25T10:03:35.043 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:35.043 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 Processor -- start 2024-04-25T10:03:35.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 -- start start 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f651c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0b2090 con 0x7f651c0b24a0 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c0b43e0 con 0x7f651c0b3f50 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f6522402640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15f200 con 0x7f651c15a700 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f651c0b24a0 0x7f651c06e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54756/0 (socket says 172.21.15.82:54756) 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 -- 172.21.15.82:0/2638484403 learned_addr learned my addr 172.21.15.82:0/2638484403 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651bfff640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f651c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078549684 0 0) 0x7f651c0b2090 con 0x7f651c0b24a0 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6504003530 con 0x7f651c0b24a0 2024-04-25T10:03:35.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 253333947 0 0) 0x7f6504003530 con 0x7f651c0b24a0 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 msgr2=0x7f651c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f651c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 >> v1:172.21.15.196:6789/0 conn(0x7f651c0b3f50 legacy=0x7f651c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0b46c0 con 0x7f651c0b24a0 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6514001a00 con 0x7f651c0b24a0 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 >> v1:172.21.15.82:6789/0 conn(0x7f651c0b24a0 legacy=0x7f651c06e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f6504003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6504003fe0 0x7f6504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504004d50 0x7f65040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c15f200 con 0x7f6504004d50 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650400bc60 con 0x7f6504003fe0 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650400be60 con 0x7f651c15a700 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504004d50 0x7f65040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651bfff640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f6504003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f651b7fe640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6504003fe0 0x7f6504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 -- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 msgr2=0x7f6504003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c15a700 0x7f6504003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 -- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6504003fe0 msgr2=0x7f6504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6504003fe0 0x7f6504004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.143+0000 7f652099f640 1 -- 172.21.15.82:0/2638484403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504003530 con 0x7f6504004d50 2024-04-25T10:03:35.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f652099f640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504004d50 0x7f65040051d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f651400a860 tx=0x7f651401d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc37cbd0b5704fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400a480 con 0x7f6504004d50 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140ad630 con 0x7f6504004d50 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651affd640 1 -- 172.21.15.82:0/2638484403 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140adaa0 con 0x7f6504004d50 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/2638484403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504004d50 msgr2=0x7f65040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 --2- 172.21.15.82:0/2638484403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6504004d50 0x7f65040051d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f651400a860 tx=0x7f651401d350 comp rx=0 tx=0).stop 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f652099f640 1 -- 172.21.15.82:0/2638484403 reap_dead start 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/2638484403 shutdown_connections 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/2638484403 >> 172.21.15.82:0/2638484403 conn(0x7f651c0b11a0 msgr2=0x7f651c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/2638484403 shutdown_connections 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/2638484403 wait complete. 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 Processor -- start 2024-04-25T10:03:35.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- start start 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c157d10 0x7f651c1597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 0x7f651c0a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0cd310 con 0x7f651c157930 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0ce600 con 0x7f651c0a9940 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0cd490 con 0x7f651c157d10 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42092/0 (socket says 172.21.15.82:42092) 2024-04-25T10:03:35.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 -- 172.21.15.82:0/361999024 learned_addr learned my addr 172.21.15.82:0/361999024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f652099f640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 0x7f651c0a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651b7fe640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c157d10 0x7f651c1597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 -- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c157d10 msgr2=0x7f651c1597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c157d10 0x7f651c1597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 -- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 msgr2=0x7f651c0a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 0x7f651c0a9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 -- 172.21.15.82:0/361999024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c0aa730 con 0x7f651c157930 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f652099f640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 0x7f651c0a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f651bfff640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f650c0115e0 tx=0x7f650c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00f8f0 con 0x7f651c157930 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c00fa90 con 0x7f651c157930 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0aaa10 con 0x7f651c157930 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c00fe40 con 0x7f651c157930 2024-04-25T10:03:35.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.147+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c0aafd0 con 0x7f651c157930 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c04efa0 con 0x7f651c157930 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f650c015460 con 0x7f651c157930 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f6518ff9640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 0x7f6508069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f650c085bb0 con 0x7f651c157930 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f651b7fe640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 0x7f6508069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.151+0000 7f651b7fe640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 0x7f6508069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f651c158110 tx=0x7f6510009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.155+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f650c019030 con 0x7f651c157930 2024-04-25T10:03:35.282 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:03:35.275+0000 7fdeb2eac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:35.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.299+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f651c06f260 con 0x7f6508066f60 2024-04-25T10:03:35.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6518ff9640 1 -- 172.21.15.82:0/361999024 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7f651c06f260 con 0x7f6508066f60 2024-04-25T10:03:35.330 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:35.332 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 msgr2=0x7f6508069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 0x7f6508069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f651c158110 tx=0x7f6510009040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 msgr2=0x7f651c1592b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f650c0115e0 tx=0x7f650c011aa0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 shutdown_connections 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6508066f60 0x7f6508069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f651c157d10 0x7f651c1597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f651c0a9940 0x7f651c0a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 --2- 172.21.15.82:0/361999024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f651c157930 0x7f651c1592b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 >> 172.21.15.82:0/361999024 conn(0x7f651c0b11a0 msgr2=0x7f651c06e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 shutdown_connections 2024-04-25T10:03:35.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.319+0000 7f6522402640 1 -- 172.21.15.82:0/361999024 wait complete. 2024-04-25T10:03:35.344 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-25T10:03:35.287633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225084,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749636,"statfs":{"total":773094113280,"available":772863627264,"internally_reserved":0,"allocated":4857856,"data_stored":2744444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.978692"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335193+0000","last_change":"2024-04-25T10:03:31.351112+0000","last_active":"2024-04-25T10:03:32.335193+0000","last_peered":"2024-04-25T10:03:32.335193+0000","last_clean":"2024-04-25T10:03:32.335193+0000","last_became_active":"2024-04-25T10:03:31.350738+0000","last_became_peered":"2024-04-25T10:03:31.350738+0000","last_unstale":"2024-04-25T10:03:32.335193+0000","last_undegraded":"2024-04-25T10:03:32.335193+0000","last_fullsized":"2024-04-25T10:03:32.335193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:05.515973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335099+0000","last_change":"2024-04-25T10:03:31.368660+0000","last_active":"2024-04-25T10:03:32.335099+0000","last_peered":"2024-04-25T10:03:32.335099+0000","last_clean":"2024-04-25T10:03:32.335099+0000","last_became_active":"2024-04-25T10:03:31.364478+0000","last_became_peered":"2024-04-25T10:03:31.364478+0000","last_unstale":"2024-04-25T10:03:32.335099+0000","last_undegraded":"2024-04-25T10:03:32.335099+0000","last_fullsized":"2024-04-25T10:03:32.335099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:30.553923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181190+0000","last_change":"2024-04-25T10:03:31.360312+0000","last_active":"2024-04-25T10:03:33.181190+0000","last_peered":"2024-04-25T10:03:33.181190+0000","last_clean":"2024-04-25T10:03:33.181190+0000","last_became_active":"2024-04-25T10:03:31.359362+0000","last_became_peered":"2024-04-25T10:03:31.359362+0000","last_unstale":"2024-04-25T10:03:33.181190+0000","last_undegraded":"2024-04-25T10:03:33.181190+0000","last_fullsized":"2024-04-25T10:03:33.181190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:20.714126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359998+0000","last_change":"2024-04-25T10:03:31.349686+0000","last_active":"2024-04-25T10:03:32.359998+0000","last_peered":"2024-04-25T10:03:32.359998+0000","last_clean":"2024-04-25T10:03:32.359998+0000","last_became_active":"2024-04-25T10:03:31.348361+0000","last_became_peered":"2024-04-25T10:03:31.348361+0000","last_unstale":"2024-04-25T10:03:32.359998+0000","last_undegraded":"2024-04-25T10:03:32.359998+0000","last_fullsized":"2024-04-25T10:03:32.359998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:25.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359994+0000","last_change":"2024-04-25T10:03:31.345856+0000","last_active":"2024-04-25T10:03:32.359994+0000","last_peered":"2024-04-25T10:03:32.359994+0000","last_clean":"2024-04-25T10:03:32.359994+0000","last_became_active":"2024-04-25T10:03:31.344526+0000","last_became_peered":"2024-04-25T10:03:31.344526+0000","last_unstale":"2024-04-25T10:03:32.359994+0000","last_undegraded":"2024-04-25T10:03:32.359994+0000","last_fullsized":"2024-04-25T10:03:32.359994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:48.559110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181876+0000","last_change":"2024-04-25T10:03:31.369315+0000","last_active":"2024-04-25T10:03:33.181876+0000","last_peered":"2024-04-25T10:03:33.181876+0000","last_clean":"2024-04-25T10:03:33.181876+0000","last_became_active":"2024-04-25T10:03:31.369018+0000","last_became_peered":"2024-04-25T10:03:31.369018+0000","last_unstale":"2024-04-25T10:03:33.181876+0000","last_undegraded":"2024-04-25T10:03:33.181876+0000","last_fullsized":"2024-04-25T10:03:33.181876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362207+0000","last_change":"2024-04-25T10:03:31.369966+0000","last_active":"2024-04-25T10:03:32.362207+0000","last_peered":"2024-04-25T10:03:32.362207+0000","last_clean":"2024-04-25T10:03:32.362207+0000","last_became_active":"2024-04-25T10:03:31.369177+0000","last_became_peered":"2024-04-25T10:03:31.369177+0000","last_unstale":"2024-04-25T10:03:32.362207+0000","last_undegraded":"2024-04-25T10:03:32.362207+0000","last_fullsized":"2024-04-25T10:03:32.362207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334524+0000","last_change":"2024-04-25T10:03:31.371142+0000","last_active":"2024-04-25T10:03:32.334524+0000","last_peered":"2024-04-25T10:03:32.334524+0000","last_clean":"2024-04-25T10:03:32.334524+0000","last_became_active":"2024-04-25T10:03:31.370803+0000","last_became_peered":"2024-04-25T10:03:31.370803+0000","last_unstale":"2024-04-25T10:03:32.334524+0000","last_undegraded":"2024-04-25T10:03:32.334524+0000","last_fullsized":"2024-04-25T10:03:32.334524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180677+0000","last_change":"2024-04-25T10:03:31.359788+0000","last_active":"2024-04-25T10:03:33.180677+0000","last_peered":"2024-04-25T10:03:33.180677+0000","last_clean":"2024-04-25T10:03:33.180677+0000","last_became_active":"2024-04-25T10:03:31.359453+0000","last_became_peered":"2024-04-25T10:03:31.359453+0000","last_unstale":"2024-04-25T10:03:33.180677+0000","last_undegraded":"2024-04-25T10:03:33.180677+0000","last_fullsized":"2024-04-25T10:03:33.180677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:50.626046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363150+0000","last_change":"2024-04-25T10:03:31.359931+0000","last_active":"2024-04-25T10:03:32.363150+0000","last_peered":"2024-04-25T10:03:32.363150+0000","last_clean":"2024-04-25T10:03:32.363150+0000","last_became_active":"2024-04-25T10:03:31.359227+0000","last_became_peered":"2024-04-25T10:03:31.359227+0000","last_unstale":"2024-04-25T10:03:32.363150+0000","last_undegraded":"2024-04-25T10:03:32.363150+0000","last_fullsized":"2024-04-25T10:03:32.363150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:04.839212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363209+0000","last_change":"2024-04-25T10:03:31.379913+0000","last_active":"2024-04-25T10:03:32.363209+0000","last_peered":"2024-04-25T10:03:32.363209+0000","last_clean":"2024-04-25T10:03:32.363209+0000","last_became_active":"2024-04-25T10:03:31.375798+0000","last_became_peered":"2024-04-25T10:03:31.375798+0000","last_unstale":"2024-04-25T10:03:32.363209+0000","last_undegraded":"2024-04-25T10:03:32.363209+0000","last_fullsized":"2024-04-25T10:03:32.363209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.899769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180767+0000","last_change":"2024-04-25T10:03:31.352737+0000","last_active":"2024-04-25T10:03:33.180767+0000","last_peered":"2024-04-25T10:03:33.180767+0000","last_clean":"2024-04-25T10:03:33.180767+0000","last_became_active":"2024-04-25T10:03:31.352457+0000","last_became_peered":"2024-04-25T10:03:31.352457+0000","last_unstale":"2024-04-25T10:03:33.180767+0000","last_undegraded":"2024-04-25T10:03:33.180767+0000","last_fullsized":"2024-04-25T10:03:33.180767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:06.476635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362364+0000","last_change":"2024-04-25T10:03:31.374237+0000","last_active":"2024-04-25T10:03:32.362364+0000","last_peered":"2024-04-25T10:03:32.362364+0000","last_clean":"2024-04-25T10:03:32.362364+0000","last_became_active":"2024-04-25T10:03:31.370286+0000","last_became_peered":"2024-04-25T10:03:31.370286+0000","last_unstale":"2024-04-25T10:03:32.362364+0000","last_undegraded":"2024-04-25T10:03:32.362364+0000","last_fullsized":"2024-04-25T10:03:32.362364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336233+0000","last_change":"2024-04-25T10:03:31.358324+0000","last_active":"2024-04-25T10:03:32.336233+0000","last_peered":"2024-04-25T10:03:32.336233+0000","last_clean":"2024-04-25T10:03:32.336233+0000","last_became_active":"2024-04-25T10:03:31.357584+0000","last_became_peered":"2024-04-25T10:03:31.357584+0000","last_unstale":"2024-04-25T10:03:32.336233+0000","last_undegraded":"2024-04-25T10:03:32.336233+0000","last_fullsized":"2024-04-25T10:03:32.336233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:19.413525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362239+0000","last_change":"2024-04-25T10:03:31.352205+0000","last_active":"2024-04-25T10:03:32.362239+0000","last_peered":"2024-04-25T10:03:32.362239+0000","last_clean":"2024-04-25T10:03:32.362239+0000","last_became_active":"2024-04-25T10:03:31.351452+0000","last_became_peered":"2024-04-25T10:03:31.351452+0000","last_unstale":"2024-04-25T10:03:32.362239+0000","last_undegraded":"2024-04-25T10:03:32.362239+0000","last_fullsized":"2024-04-25T10:03:32.362239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584884+0000","last_change":"2024-04-25T10:03:31.393451+0000","last_active":"2024-04-25T10:03:32.584884+0000","last_peered":"2024-04-25T10:03:32.584884+0000","last_clean":"2024-04-25T10:03:32.584884+0000","last_became_active":"2024-04-25T10:03:31.388042+0000","last_became_peered":"2024-04-25T10:03:31.388042+0000","last_unstale":"2024-04-25T10:03:32.584884+0000","last_undegraded":"2024-04-25T10:03:32.584884+0000","last_fullsized":"2024-04-25T10:03:32.584884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:06.476635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585939+0000","last_change":"2024-04-25T10:03:31.392272+0000","last_active":"2024-04-25T10:03:32.585939+0000","last_peered":"2024-04-25T10:03:32.585939+0000","last_clean":"2024-04-25T10:03:32.585939+0000","last_became_active":"2024-04-25T10:03:31.391511+0000","last_became_peered":"2024-04-25T10:03:31.391511+0000","last_unstale":"2024-04-25T10:03:32.585939+0000","last_undegraded":"2024-04-25T10:03:32.585939+0000","last_fullsized":"2024-04-25T10:03:32.585939+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335383+0000","last_change":"2024-04-25T10:03:31.368583+0000","last_active":"2024-04-25T10:03:32.335383+0000","last_peered":"2024-04-25T10:03:32.335383+0000","last_clean":"2024-04-25T10:03:32.335383+0000","last_became_active":"2024-04-25T10:03:31.367850+0000","last_became_peered":"2024-04-25T10:03:31.367850+0000","last_unstale":"2024-04-25T10:03:32.335383+0000","last_undegraded":"2024-04-25T10:03:32.335383+0000","last_fullsized":"2024-04-25T10:03:32.335383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:20.288540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360121+0000","last_change":"2024-04-25T10:03:31.360504+0000","last_active":"2024-04-25T10:03:32.360121+0000","last_peered":"2024-04-25T10:03:32.360121+0000","last_clean":"2024-04-25T10:03:32.360121+0000","last_became_active":"2024-04-25T10:03:31.360189+0000","last_became_peered":"2024-04-25T10:03:31.360189+0000","last_unstale":"2024-04-25T10:03:32.360121+0000","last_undegraded":"2024-04-25T10:03:32.360121+0000","last_fullsized":"2024-04-25T10:03:32.360121+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:26.742376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334543+0000","last_change":"2024-04-25T10:03:31.350162+0000","last_active":"2024-04-25T10:03:32.334543+0000","last_peered":"2024-04-25T10:03:32.334543+0000","last_clean":"2024-04-25T10:03:32.334543+0000","last_became_active":"2024-04-25T10:03:31.347504+0000","last_became_peered":"2024-04-25T10:03:31.347504+0000","last_unstale":"2024-04-25T10:03:32.334543+0000","last_undegraded":"2024-04-25T10:03:32.334543+0000","last_fullsized":"2024-04-25T10:03:32.334543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:50.030249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362524+0000","last_change":"2024-04-25T10:03:31.358283+0000","last_active":"2024-04-25T10:03:32.362524+0000","last_peered":"2024-04-25T10:03:32.362524+0000","last_clean":"2024-04-25T10:03:32.362524+0000","last_became_active":"2024-04-25T10:03:31.357410+0000","last_became_peered":"2024-04-25T10:03:31.357410+0000","last_unstale":"2024-04-25T10:03:32.362524+0000","last_undegraded":"2024-04-25T10:03:32.362524+0000","last_fullsized":"2024-04-25T10:03:32.362524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:43.548567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336843+0000","last_change":"2024-04-25T10:03:31.356340+0000","last_active":"2024-04-25T10:03:32.336843+0000","last_peered":"2024-04-25T10:03:32.336843+0000","last_clean":"2024-04-25T10:03:32.336843+0000","last_became_active":"2024-04-25T10:03:31.355625+0000","last_became_peered":"2024-04-25T10:03:31.355625+0000","last_unstale":"2024-04-25T10:03:32.336843+0000","last_undegraded":"2024-04-25T10:03:32.336843+0000","last_fullsized":"2024-04-25T10:03:32.336843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:20.288540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584980+0000","last_change":"2024-04-25T10:03:31.387372+0000","last_active":"2024-04-25T10:03:32.584980+0000","last_peered":"2024-04-25T10:03:32.584980+0000","last_clean":"2024-04-25T10:03:32.584980+0000","last_became_active":"2024-04-25T10:03:31.382677+0000","last_became_peered":"2024-04-25T10:03:31.382677+0000","last_unstale":"2024-04-25T10:03:32.584980+0000","last_undegraded":"2024-04-25T10:03:32.584980+0000","last_fullsized":"2024-04-25T10:03:32.584980+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:20.714126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359751+0000","last_change":"2024-04-25T10:03:31.394429+0000","last_active":"2024-04-25T10:03:32.359751+0000","last_peered":"2024-04-25T10:03:32.359751+0000","last_clean":"2024-04-25T10:03:32.359751+0000","last_became_active":"2024-04-25T10:03:31.393945+0000","last_became_peered":"2024-04-25T10:03:31.393945+0000","last_unstale":"2024-04-25T10:03:32.359751+0000","last_undegraded":"2024-04-25T10:03:32.359751+0000","last_fullsized":"2024-04-25T10:03:32.359751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:03.652716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335384+0000","last_change":"2024-04-25T10:03:31.349115+0000","last_active":"2024-04-25T10:03:32.335384+0000","last_peered":"2024-04-25T10:03:32.335384+0000","last_clean":"2024-04-25T10:03:32.335384+0000","last_became_active":"2024-04-25T10:03:31.347571+0000","last_became_peered":"2024-04-25T10:03:31.347571+0000","last_unstale":"2024-04-25T10:03:32.335384+0000","last_undegraded":"2024-04-25T10:03:32.335384+0000","last_fullsized":"2024-04-25T10:03:32.335384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181908+0000","last_change":"2024-04-25T10:03:31.360221+0000","last_active":"2024-04-25T10:03:33.181908+0000","last_peered":"2024-04-25T10:03:33.181908+0000","last_clean":"2024-04-25T10:03:33.181908+0000","last_became_active":"2024-04-25T10:03:31.359834+0000","last_became_peered":"2024-04-25T10:03:31.359834+0000","last_unstale":"2024-04-25T10:03:33.181908+0000","last_undegraded":"2024-04-25T10:03:33.181908+0000","last_fullsized":"2024-04-25T10:03:33.181908+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335063+0000","last_change":"2024-04-25T10:03:31.337659+0000","last_active":"2024-04-25T10:03:32.335063+0000","last_peered":"2024-04-25T10:03:32.335063+0000","last_clean":"2024-04-25T10:03:32.335063+0000","last_became_active":"2024-04-25T10:03:31.336844+0000","last_became_peered":"2024-04-25T10:03:31.336844+0000","last_unstale":"2024-04-25T10:03:32.335063+0000","last_undegraded":"2024-04-25T10:03:32.335063+0000","last_fullsized":"2024-04-25T10:03:32.335063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:04.546401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362986+0000","last_change":"2024-04-25T10:03:31.374141+0000","last_active":"2024-04-25T10:03:32.362986+0000","last_peered":"2024-04-25T10:03:32.362986+0000","last_clean":"2024-04-25T10:03:32.362986+0000","last_became_active":"2024-04-25T10:03:31.370907+0000","last_became_peered":"2024-04-25T10:03:31.370907+0000","last_unstale":"2024-04-25T10:03:32.362986+0000","last_undegraded":"2024-04-25T10:03:32.362986+0000","last_fullsized":"2024-04-25T10:03:32.362986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:05.515973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334633+0000","last_change":"2024-04-25T10:03:31.354041+0000","last_active":"2024-04-25T10:03:32.334633+0000","last_peered":"2024-04-25T10:03:32.334633+0000","last_clean":"2024-04-25T10:03:32.334633+0000","last_became_active":"2024-04-25T10:03:31.353237+0000","last_became_peered":"2024-04-25T10:03:31.353237+0000","last_unstale":"2024-04-25T10:03:32.334633+0000","last_undegraded":"2024-04-25T10:03:32.334633+0000","last_fullsized":"2024-04-25T10:03:32.334633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:19.413525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334551+0000","last_change":"2024-04-25T10:03:31.369260+0000","last_active":"2024-04-25T10:03:32.334551+0000","last_peered":"2024-04-25T10:03:32.334551+0000","last_clean":"2024-04-25T10:03:32.334551+0000","last_became_active":"2024-04-25T10:03:31.368275+0000","last_became_peered":"2024-04-25T10:03:31.368275+0000","last_unstale":"2024-04-25T10:03:32.334551+0000","last_undegraded":"2024-04-25T10:03:32.334551+0000","last_fullsized":"2024-04-25T10:03:32.334551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.518646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585652+0000","last_change":"2024-04-25T10:03:31.378622+0000","last_active":"2024-04-25T10:03:32.585652+0000","last_peered":"2024-04-25T10:03:32.585652+0000","last_clean":"2024-04-25T10:03:32.585652+0000","last_became_active":"2024-04-25T10:03:31.377926+0000","last_became_peered":"2024-04-25T10:03:31.377926+0000","last_unstale":"2024-04-25T10:03:32.585652+0000","last_undegraded":"2024-04-25T10:03:32.585652+0000","last_fullsized":"2024-04-25T10:03:32.585652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:47.576388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180059+0000","last_change":"2024-04-25T10:03:31.359774+0000","last_active":"2024-04-25T10:03:33.180059+0000","last_peered":"2024-04-25T10:03:33.180059+0000","last_clean":"2024-04-25T10:03:33.180059+0000","last_became_active":"2024-04-25T10:03:31.359427+0000","last_became_peered":"2024-04-25T10:03:31.359427+0000","last_unstale":"2024-04-25T10:03:33.180059+0000","last_undegraded":"2024-04-25T10:03:33.180059+0000","last_fullsized":"2024-04-25T10:03:33.180059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585426+0000","last_change":"2024-04-25T10:03:31.392140+0000","last_active":"2024-04-25T10:03:32.585426+0000","last_peered":"2024-04-25T10:03:32.585426+0000","last_clean":"2024-04-25T10:03:32.585426+0000","last_became_active":"2024-04-25T10:03:31.388839+0000","last_became_peered":"2024-04-25T10:03:31.388839+0000","last_unstale":"2024-04-25T10:03:32.585426+0000","last_undegraded":"2024-04-25T10:03:32.585426+0000","last_fullsized":"2024-04-25T10:03:32.585426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:38.711873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335466+0000","last_change":"2024-04-25T10:03:30.359055+0000","last_active":"2024-04-25T10:03:32.335466+0000","last_peered":"2024-04-25T10:03:32.335466+0000","last_clean":"2024-04-25T10:03:32.335466+0000","last_became_active":"2024-04-25T10:03:30.358355+0000","last_became_peered":"2024-04-25T10:03:30.358355+0000","last_unstale":"2024-04-25T10:03:32.335466+0000","last_undegraded":"2024-04-25T10:03:32.335466+0000","last_fullsized":"2024-04-25T10:03:32.335466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:18.428515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182019+0000","last_change":"2024-04-25T10:03:31.376286+0000","last_active":"2024-04-25T10:03:33.182019+0000","last_peered":"2024-04-25T10:03:33.182019+0000","last_clean":"2024-04-25T10:03:33.182019+0000","last_became_active":"2024-04-25T10:03:31.376009+0000","last_became_peered":"2024-04-25T10:03:31.376009+0000","last_unstale":"2024-04-25T10:03:33.182019+0000","last_undegraded":"2024-04-25T10:03:33.182019+0000","last_fullsized":"2024-04-25T10:03:33.182019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:50.626046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180867+0000","last_change":"2024-04-25T10:03:30.341884+0000","last_active":"2024-04-25T10:03:33.180867+0000","last_peered":"2024-04-25T10:03:33.180867+0000","last_clean":"2024-04-25T10:03:33.180867+0000","last_became_active":"2024-04-25T10:03:30.341386+0000","last_became_peered":"2024-04-25T10:03:30.341386+0000","last_unstale":"2024-04-25T10:03:33.180867+0000","last_undegraded":"2024-04-25T10:03:33.180867+0000","last_fullsized":"2024-04-25T10:03:33.180867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:02.640058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337335+0000","last_change":"2024-04-25T10:03:31.361251+0000","last_active":"2024-04-25T10:03:32.337335+0000","last_peered":"2024-04-25T10:03:32.337335+0000","last_clean":"2024-04-25T10:03:32.337335+0000","last_became_active":"2024-04-25T10:03:31.359515+0000","last_became_peered":"2024-04-25T10:03:31.359515+0000","last_unstale":"2024-04-25T10:03:32.337335+0000","last_undegraded":"2024-04-25T10:03:32.337335+0000","last_fullsized":"2024-04-25T10:03:32.337335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182149+0000","last_change":"2024-04-25T10:03:30.356646+0000","last_active":"2024-04-25T10:03:33.182149+0000","last_peered":"2024-04-25T10:03:33.182149+0000","last_clean":"2024-04-25T10:03:33.182149+0000","last_became_active":"2024-04-25T10:03:30.356150+0000","last_became_peered":"2024-04-25T10:03:30.356150+0000","last_unstale":"2024-04-25T10:03:33.182149+0000","last_undegraded":"2024-04-25T10:03:33.182149+0000","last_fullsized":"2024-04-25T10:03:33.182149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:24.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334900+0000","last_change":"2024-04-25T10:03:31.363890+0000","last_active":"2024-04-25T10:03:32.334900+0000","last_peered":"2024-04-25T10:03:32.334900+0000","last_clean":"2024-04-25T10:03:32.334900+0000","last_became_active":"2024-04-25T10:03:31.363405+0000","last_became_peered":"2024-04-25T10:03:31.363405+0000","last_unstale":"2024-04-25T10:03:32.334900+0000","last_undegraded":"2024-04-25T10:03:32.334900+0000","last_fullsized":"2024-04-25T10:03:32.334900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336641+0000","last_change":"2024-04-25T10:03:30.349762+0000","last_active":"2024-04-25T10:03:32.336641+0000","last_peered":"2024-04-25T10:03:32.336641+0000","last_clean":"2024-04-25T10:03:32.336641+0000","last_became_active":"2024-04-25T10:03:30.349044+0000","last_became_peered":"2024-04-25T10:03:30.349044+0000","last_unstale":"2024-04-25T10:03:32.336641+0000","last_undegraded":"2024-04-25T10:03:32.336641+0000","last_fullsized":"2024-04-25T10:03:32.336641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:08.615248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.179970+0000","last_change":"2024-04-25T10:03:31.360522+0000","last_active":"2024-04-25T10:03:33.179970+0000","last_peered":"2024-04-25T10:03:33.179970+0000","last_clean":"2024-04-25T10:03:33.179970+0000","last_became_active":"2024-04-25T10:03:31.360178+0000","last_became_peered":"2024-04-25T10:03:31.360178+0000","last_unstale":"2024-04-25T10:03:33.179970+0000","last_undegraded":"2024-04-25T10:03:33.179970+0000","last_fullsized":"2024-04-25T10:03:33.179970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362453+0000","last_change":"2024-04-25T10:03:30.351283+0000","last_active":"2024-04-25T10:03:32.362453+0000","last_peered":"2024-04-25T10:03:32.362453+0000","last_clean":"2024-04-25T10:03:32.362453+0000","last_became_active":"2024-04-25T10:03:30.349036+0000","last_became_peered":"2024-04-25T10:03:30.349036+0000","last_unstale":"2024-04-25T10:03:32.362453+0000","last_undegraded":"2024-04-25T10:03:32.362453+0000","last_fullsized":"2024-04-25T10:03:32.362453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:19.275882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362661+0000","last_change":"2024-04-25T10:03:31.378874+0000","last_active":"2024-04-25T10:03:32.362661+0000","last_peered":"2024-04-25T10:03:32.362661+0000","last_clean":"2024-04-25T10:03:32.362661+0000","last_became_active":"2024-04-25T10:03:31.375200+0000","last_became_peered":"2024-04-25T10:03:31.375200+0000","last_unstale":"2024-04-25T10:03:32.362661+0000","last_undegraded":"2024-04-25T10:03:32.362661+0000","last_fullsized":"2024-04-25T10:03:32.362661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:30.553923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585179+0000","last_change":"2024-04-25T10:03:30.360622+0000","last_active":"2024-04-25T10:03:32.585179+0000","last_peered":"2024-04-25T10:03:32.585179+0000","last_clean":"2024-04-25T10:03:32.585179+0000","last_became_active":"2024-04-25T10:03:30.360011+0000","last_became_peered":"2024-04-25T10:03:30.360011+0000","last_unstale":"2024-04-25T10:03:32.585179+0000","last_undegraded":"2024-04-25T10:03:32.585179+0000","last_fullsized":"2024-04-25T10:03:32.585179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:24.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336884+0000","last_change":"2024-04-25T10:03:31.354798+0000","last_active":"2024-04-25T10:03:32.336884+0000","last_peered":"2024-04-25T10:03:32.336884+0000","last_clean":"2024-04-25T10:03:32.336884+0000","last_became_active":"2024-04-25T10:03:31.346353+0000","last_became_peered":"2024-04-25T10:03:31.346353+0000","last_unstale":"2024-04-25T10:03:32.336884+0000","last_undegraded":"2024-04-25T10:03:32.336884+0000","last_fullsized":"2024-04-25T10:03:32.336884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181221+0000","last_change":"2024-04-25T10:03:30.365631+0000","last_active":"2024-04-25T10:03:33.181221+0000","last_peered":"2024-04-25T10:03:33.181221+0000","last_clean":"2024-04-25T10:03:33.181221+0000","last_became_active":"2024-04-25T10:03:30.365299+0000","last_became_peered":"2024-04-25T10:03:30.365299+0000","last_unstale":"2024-04-25T10:03:33.181221+0000","last_undegraded":"2024-04-25T10:03:33.181221+0000","last_fullsized":"2024-04-25T10:03:33.181221+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334364+0000","last_change":"2024-04-25T10:03:31.389546+0000","last_active":"2024-04-25T10:03:32.334364+0000","last_peered":"2024-04-25T10:03:32.334364+0000","last_clean":"2024-04-25T10:03:32.334364+0000","last_became_active":"2024-04-25T10:03:31.389164+0000","last_became_peered":"2024-04-25T10:03:31.389164+0000","last_unstale":"2024-04-25T10:03:32.334364+0000","last_undegraded":"2024-04-25T10:03:32.334364+0000","last_fullsized":"2024-04-25T10:03:32.334364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335753+0000","last_change":"2024-04-25T10:03:30.360937+0000","last_active":"2024-04-25T10:03:32.335753+0000","last_peered":"2024-04-25T10:03:32.335753+0000","last_clean":"2024-04-25T10:03:32.335753+0000","last_became_active":"2024-04-25T10:03:30.360285+0000","last_became_peered":"2024-04-25T10:03:30.360285+0000","last_unstale":"2024-04-25T10:03:32.335753+0000","last_undegraded":"2024-04-25T10:03:32.335753+0000","last_fullsized":"2024-04-25T10:03:32.335753+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360297+0000","last_change":"2024-04-25T10:03:31.349045+0000","last_active":"2024-04-25T10:03:32.360297+0000","last_peered":"2024-04-25T10:03:32.360297+0000","last_clean":"2024-04-25T10:03:32.360297+0000","last_became_active":"2024-04-25T10:03:31.347424+0000","last_became_peered":"2024-04-25T10:03:31.347424+0000","last_unstale":"2024-04-25T10:03:32.360297+0000","last_undegraded":"2024-04-25T10:03:32.360297+0000","last_fullsized":"2024-04-25T10:03:32.360297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.231950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335156+0000","last_change":"2024-04-25T10:03:30.341060+0000","last_active":"2024-04-25T10:03:32.335156+0000","last_peered":"2024-04-25T10:03:32.335156+0000","last_clean":"2024-04-25T10:03:32.335156+0000","last_became_active":"2024-04-25T10:03:30.340582+0000","last_became_peered":"2024-04-25T10:03:30.340582+0000","last_unstale":"2024-04-25T10:03:32.335156+0000","last_undegraded":"2024-04-25T10:03:32.335156+0000","last_fullsized":"2024-04-25T10:03:32.335156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:18.400867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585773+0000","last_change":"2024-04-25T10:03:31.392364+0000","last_active":"2024-04-25T10:03:32.585773+0000","last_peered":"2024-04-25T10:03:32.585773+0000","last_clean":"2024-04-25T10:03:32.585773+0000","last_became_active":"2024-04-25T10:03:31.391563+0000","last_became_peered":"2024-04-25T10:03:31.391563+0000","last_unstale":"2024-04-25T10:03:32.585773+0000","last_undegraded":"2024-04-25T10:03:32.585773+0000","last_fullsized":"2024-04-25T10:03:32.585773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:24.867273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363540+0000","last_change":"2024-04-25T10:03:30.363343+0000","last_active":"2024-04-25T10:03:32.363540+0000","last_peered":"2024-04-25T10:03:32.363540+0000","last_clean":"2024-04-25T10:03:32.363540+0000","last_became_active":"2024-04-25T10:03:30.362630+0000","last_became_peered":"2024-04-25T10:03:30.362630+0000","last_unstale":"2024-04-25T10:03:32.363540+0000","last_undegraded":"2024-04-25T10:03:32.363540+0000","last_fullsized":"2024-04-25T10:03:32.363540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:23.918419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180384+0000","last_change":"2024-04-25T10:03:31.393934+0000","last_active":"2024-04-25T10:03:33.180384+0000","last_peered":"2024-04-25T10:03:33.180384+0000","last_clean":"2024-04-25T10:03:33.180384+0000","last_became_active":"2024-04-25T10:03:31.393636+0000","last_became_peered":"2024-04-25T10:03:31.393636+0000","last_unstale":"2024-04-25T10:03:33.180384+0000","last_undegraded":"2024-04-25T10:03:33.180384+0000","last_fullsized":"2024-04-25T10:03:33.180384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363356+0000","last_change":"2024-04-25T10:03:30.360288+0000","last_active":"2024-04-25T10:03:32.363356+0000","last_peered":"2024-04-25T10:03:32.363356+0000","last_clean":"2024-04-25T10:03:32.363356+0000","last_became_active":"2024-04-25T10:03:30.359605+0000","last_became_peered":"2024-04-25T10:03:30.359605+0000","last_unstale":"2024-04-25T10:03:32.363356+0000","last_undegraded":"2024-04-25T10:03:32.363356+0000","last_fullsized":"2024-04-25T10:03:32.363356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:20.024687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337279+0000","last_change":"2024-04-25T10:03:31.353931+0000","last_active":"2024-04-25T10:03:32.337279+0000","last_peered":"2024-04-25T10:03:32.337279+0000","last_clean":"2024-04-25T10:03:32.337279+0000","last_became_active":"2024-04-25T10:03:31.353097+0000","last_became_peered":"2024-04-25T10:03:31.353097+0000","last_unstale":"2024-04-25T10:03:32.337279+0000","last_undegraded":"2024-04-25T10:03:32.337279+0000","last_fullsized":"2024-04-25T10:03:32.337279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334830+0000","last_change":"2024-04-25T10:03:30.346414+0000","last_active":"2024-04-25T10:03:32.334830+0000","last_peered":"2024-04-25T10:03:32.334830+0000","last_clean":"2024-04-25T10:03:32.334830+0000","last_became_active":"2024-04-25T10:03:30.346187+0000","last_became_peered":"2024-04-25T10:03:30.346187+0000","last_unstale":"2024-04-25T10:03:32.334830+0000","last_undegraded":"2024-04-25T10:03:32.334830+0000","last_fullsized":"2024-04-25T10:03:32.334830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:35.505988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180099+0000","last_change":"2024-04-25T10:03:31.359851+0000","last_active":"2024-04-25T10:03:33.180099+0000","last_peered":"2024-04-25T10:03:33.180099+0000","last_clean":"2024-04-25T10:03:33.180099+0000","last_became_active":"2024-04-25T10:03:31.359545+0000","last_became_peered":"2024-04-25T10:03:31.359545+0000","last_unstale":"2024-04-25T10:03:33.180099+0000","last_undegraded":"2024-04-25T10:03:33.180099+0000","last_fullsized":"2024-04-25T10:03:33.180099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334321+0000","last_change":"2024-04-25T10:03:30.348763+0000","last_active":"2024-04-25T10:03:32.334321+0000","last_peered":"2024-04-25T10:03:32.334321+0000","last_clean":"2024-04-25T10:03:32.334321+0000","last_became_active":"2024-04-25T10:03:30.348505+0000","last_became_peered":"2024-04-25T10:03:30.348505+0000","last_unstale":"2024-04-25T10:03:32.334321+0000","last_undegraded":"2024-04-25T10:03:32.334321+0000","last_fullsized":"2024-04-25T10:03:32.334321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:23.918419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334404+0000","last_change":"2024-04-25T10:03:31.368145+0000","last_active":"2024-04-25T10:03:32.334404+0000","last_peered":"2024-04-25T10:03:32.334404+0000","last_clean":"2024-04-25T10:03:32.334404+0000","last_became_active":"2024-04-25T10:03:31.363662+0000","last_became_peered":"2024-04-25T10:03:31.363662+0000","last_unstale":"2024-04-25T10:03:32.334404+0000","last_undegraded":"2024-04-25T10:03:32.334404+0000","last_fullsized":"2024-04-25T10:03:32.334404+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:24.867273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181187+0000","last_change":"2024-04-25T10:03:30.353157+0000","last_active":"2024-04-25T10:03:33.181187+0000","last_peered":"2024-04-25T10:03:33.181187+0000","last_clean":"2024-04-25T10:03:33.181187+0000","last_became_active":"2024-04-25T10:03:30.352824+0000","last_became_peered":"2024-04-25T10:03:30.352824+0000","last_unstale":"2024-04-25T10:03:33.181187+0000","last_undegraded":"2024-04-25T10:03:33.181187+0000","last_fullsized":"2024-04-25T10:03:33.181187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:47.546452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585574+0000","last_change":"2024-04-25T10:03:31.379271+0000","last_active":"2024-04-25T10:03:32.585574+0000","last_peered":"2024-04-25T10:03:32.585574+0000","last_clean":"2024-04-25T10:03:32.585574+0000","last_became_active":"2024-04-25T10:03:31.377835+0000","last_became_peered":"2024-04-25T10:03:31.377835+0000","last_unstale":"2024-04-25T10:03:32.585574+0000","last_undegraded":"2024-04-25T10:03:32.585574+0000","last_fullsized":"2024-04-25T10:03:32.585574+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:45.405473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.333964+0000","last_change":"2024-04-25T10:03:30.358881+0000","last_active":"2024-04-25T10:03:32.333964+0000","last_peered":"2024-04-25T10:03:32.333964+0000","last_clean":"2024-04-25T10:03:32.333964+0000","last_became_active":"2024-04-25T10:03:30.358164+0000","last_became_peered":"2024-04-25T10:03:30.358164+0000","last_unstale":"2024-04-25T10:03:32.333964+0000","last_undegraded":"2024-04-25T10:03:32.333964+0000","last_fullsized":"2024-04-25T10:03:32.333964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.280508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334040+0000","last_change":"2024-04-25T10:03:31.364194+0000","last_active":"2024-04-25T10:03:32.334040+0000","last_peered":"2024-04-25T10:03:32.334040+0000","last_clean":"2024-04-25T10:03:32.334040+0000","last_became_active":"2024-04-25T10:03:31.363883+0000","last_became_peered":"2024-04-25T10:03:31.363883+0000","last_unstale":"2024-04-25T10:03:32.334040+0000","last_undegraded":"2024-04-25T10:03:32.334040+0000","last_fullsized":"2024-04-25T10:03:32.334040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:20.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584681+0000","last_change":"2024-04-25T10:03:30.358682+0000","last_active":"2024-04-25T10:03:32.584681+0000","last_peered":"2024-04-25T10:03:32.584681+0000","last_clean":"2024-04-25T10:03:32.584681+0000","last_became_active":"2024-04-25T10:03:30.357983+0000","last_became_peered":"2024-04-25T10:03:30.357983+0000","last_unstale":"2024-04-25T10:03:32.584681+0000","last_undegraded":"2024-04-25T10:03:32.584681+0000","last_fullsized":"2024-04-25T10:03:32.584681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:47.546452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334916+0000","last_change":"2024-04-25T10:03:31.368414+0000","last_active":"2024-04-25T10:03:32.334916+0000","last_peered":"2024-04-25T10:03:32.334916+0000","last_clean":"2024-04-25T10:03:32.334916+0000","last_became_active":"2024-04-25T10:03:31.364462+0000","last_became_peered":"2024-04-25T10:03:31.364462+0000","last_unstale":"2024-04-25T10:03:32.334916+0000","last_undegraded":"2024-04-25T10:03:32.334916+0000","last_fullsized":"2024-04-25T10:03:32.334916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:26.194936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363518+0000","last_change":"2024-04-25T10:03:30.364424+0000","last_active":"2024-04-25T10:03:32.363518+0000","last_peered":"2024-04-25T10:03:32.363518+0000","last_clean":"2024-04-25T10:03:32.363518+0000","last_became_active":"2024-04-25T10:03:30.363258+0000","last_became_peered":"2024-04-25T10:03:30.363258+0000","last_unstale":"2024-04-25T10:03:32.363518+0000","last_undegraded":"2024-04-25T10:03:32.363518+0000","last_fullsized":"2024-04-25T10:03:32.363518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:18.400867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363293+0000","last_change":"2024-04-25T10:03:31.370216+0000","last_active":"2024-04-25T10:03:32.363293+0000","last_peered":"2024-04-25T10:03:32.363293+0000","last_clean":"2024-04-25T10:03:32.363293+0000","last_became_active":"2024-04-25T10:03:31.367905+0000","last_became_peered":"2024-04-25T10:03:31.367905+0000","last_unstale":"2024-04-25T10:03:32.363293+0000","last_undegraded":"2024-04-25T10:03:32.363293+0000","last_fullsized":"2024-04-25T10:03:32.363293+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:23.768194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337188+0000","last_change":"2024-04-25T10:03:30.355967+0000","last_active":"2024-04-25T10:03:32.337188+0000","last_peered":"2024-04-25T10:03:32.337188+0000","last_clean":"2024-04-25T10:03:32.337188+0000","last_became_active":"2024-04-25T10:03:30.354860+0000","last_became_peered":"2024-04-25T10:03:30.354860+0000","last_unstale":"2024-04-25T10:03:32.337188+0000","last_undegraded":"2024-04-25T10:03:32.337188+0000","last_fullsized":"2024-04-25T10:03:32.337188+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336981+0000","last_change":"2024-04-25T10:03:31.365262+0000","last_active":"2024-04-25T10:03:32.336981+0000","last_peered":"2024-04-25T10:03:32.336981+0000","last_clean":"2024-04-25T10:03:32.336981+0000","last_became_active":"2024-04-25T10:03:31.360300+0000","last_became_peered":"2024-04-25T10:03:31.360300+0000","last_unstale":"2024-04-25T10:03:32.336981+0000","last_undegraded":"2024-04-25T10:03:32.336981+0000","last_fullsized":"2024-04-25T10:03:32.336981+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359994+0000","last_change":"2024-04-25T10:03:30.343858+0000","last_active":"2024-04-25T10:03:32.359994+0000","last_peered":"2024-04-25T10:03:32.359994+0000","last_clean":"2024-04-25T10:03:32.359994+0000","last_became_active":"2024-04-25T10:03:30.343569+0000","last_became_peered":"2024-04-25T10:03:30.343569+0000","last_unstale":"2024-04-25T10:03:32.359994+0000","last_undegraded":"2024-04-25T10:03:32.359994+0000","last_fullsized":"2024-04-25T10:03:32.359994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:06.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360526+0000","last_change":"2024-04-25T10:03:31.353858+0000","last_active":"2024-04-25T10:03:32.360526+0000","last_peered":"2024-04-25T10:03:32.360526+0000","last_clean":"2024-04-25T10:03:32.360526+0000","last_became_active":"2024-04-25T10:03:31.353497+0000","last_became_peered":"2024-04-25T10:03:31.353497+0000","last_unstale":"2024-04-25T10:03:32.360526+0000","last_undegraded":"2024-04-25T10:03:32.360526+0000","last_fullsized":"2024-04-25T10:03:32.360526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:50.030249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181485+0000","last_change":"2024-04-25T10:03:30.357045+0000","last_active":"2024-04-25T10:03:33.181485+0000","last_peered":"2024-04-25T10:03:33.181485+0000","last_clean":"2024-04-25T10:03:33.181485+0000","last_became_active":"2024-04-25T10:03:30.356262+0000","last_became_peered":"2024-04-25T10:03:30.356262+0000","last_unstale":"2024-04-25T10:03:33.181485+0000","last_undegraded":"2024-04-25T10:03:33.181485+0000","last_fullsized":"2024-04-25T10:03:33.181485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:02.640058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585899+0000","last_change":"2024-04-25T10:03:31.387911+0000","last_active":"2024-04-25T10:03:32.585899+0000","last_peered":"2024-04-25T10:03:32.585899+0000","last_clean":"2024-04-25T10:03:32.585899+0000","last_became_active":"2024-04-25T10:03:31.387394+0000","last_became_peered":"2024-04-25T10:03:31.387394+0000","last_unstale":"2024-04-25T10:03:32.585899+0000","last_undegraded":"2024-04-25T10:03:32.585899+0000","last_fullsized":"2024-04-25T10:03:32.585899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359830+0000","last_change":"2024-04-25T10:03:30.344668+0000","last_active":"2024-04-25T10:03:32.359830+0000","last_peered":"2024-04-25T10:03:32.359830+0000","last_clean":"2024-04-25T10:03:32.359830+0000","last_became_active":"2024-04-25T10:03:30.344302+0000","last_became_peered":"2024-04-25T10:03:30.344302+0000","last_unstale":"2024-04-25T10:03:32.359830+0000","last_undegraded":"2024-04-25T10:03:32.359830+0000","last_fullsized":"2024-04-25T10:03:32.359830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.987811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180325+0000","last_change":"2024-04-25T10:03:31.365872+0000","last_active":"2024-04-25T10:03:33.180325+0000","last_peered":"2024-04-25T10:03:33.180325+0000","last_clean":"2024-04-25T10:03:33.180325+0000","last_became_active":"2024-04-25T10:03:31.365380+0000","last_became_peered":"2024-04-25T10:03:31.365380+0000","last_unstale":"2024-04-25T10:03:33.180325+0000","last_undegraded":"2024-04-25T10:03:33.180325+0000","last_fullsized":"2024-04-25T10:03:33.180325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359837+0000","last_change":"2024-04-25T10:03:30.356681+0000","last_active":"2024-04-25T10:03:32.359837+0000","last_peered":"2024-04-25T10:03:32.359837+0000","last_clean":"2024-04-25T10:03:32.359837+0000","last_became_active":"2024-04-25T10:03:30.356008+0000","last_became_peered":"2024-04-25T10:03:30.356008+0000","last_unstale":"2024-04-25T10:03:32.359837+0000","last_undegraded":"2024-04-25T10:03:32.359837+0000","last_fullsized":"2024-04-25T10:03:32.359837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:45.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362465+0000","last_change":"2024-04-25T10:03:31.352703+0000","last_active":"2024-04-25T10:03:32.362465+0000","last_peered":"2024-04-25T10:03:32.362465+0000","last_clean":"2024-04-25T10:03:32.362465+0000","last_became_active":"2024-04-25T10:03:31.351963+0000","last_became_peered":"2024-04-25T10:03:31.351963+0000","last_unstale":"2024-04-25T10:03:32.362465+0000","last_undegraded":"2024-04-25T10:03:32.362465+0000","last_fullsized":"2024-04-25T10:03:32.362465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:26.194936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585250+0000","last_change":"2024-04-25T10:03:30.359453+0000","last_active":"2024-04-25T10:03:32.585250+0000","last_peered":"2024-04-25T10:03:32.585250+0000","last_clean":"2024-04-25T10:03:32.585250+0000","last_became_active":"2024-04-25T10:03:30.358890+0000","last_became_peered":"2024-04-25T10:03:30.358890+0000","last_unstale":"2024-04-25T10:03:32.585250+0000","last_undegraded":"2024-04-25T10:03:32.585250+0000","last_fullsized":"2024-04-25T10:03:32.585250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.219292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334266+0000","last_change":"2024-04-25T10:03:31.381014+0000","last_active":"2024-04-25T10:03:32.334266+0000","last_peered":"2024-04-25T10:03:32.334266+0000","last_clean":"2024-04-25T10:03:32.334266+0000","last_became_active":"2024-04-25T10:03:31.380759+0000","last_became_peered":"2024-04-25T10:03:31.380759+0000","last_unstale":"2024-04-25T10:03:32.334266+0000","last_undegraded":"2024-04-25T10:03:32.334266+0000","last_fullsized":"2024-04-25T10:03:32.334266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:45.405473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359808+0000","last_change":"2024-04-25T10:03:30.344451+0000","last_active":"2024-04-25T10:03:32.359808+0000","last_peered":"2024-04-25T10:03:32.359808+0000","last_clean":"2024-04-25T10:03:32.359808+0000","last_became_active":"2024-04-25T10:03:30.344119+0000","last_became_peered":"2024-04-25T10:03:30.344119+0000","last_unstale":"2024-04-25T10:03:32.359808+0000","last_undegraded":"2024-04-25T10:03:32.359808+0000","last_fullsized":"2024-04-25T10:03:32.359808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:29.424997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181690+0000","last_change":"2024-04-25T10:03:31.359583+0000","last_active":"2024-04-25T10:03:33.181690+0000","last_peered":"2024-04-25T10:03:33.181690+0000","last_clean":"2024-04-25T10:03:33.181690+0000","last_became_active":"2024-04-25T10:03:31.359237+0000","last_became_peered":"2024-04-25T10:03:31.359237+0000","last_unstale":"2024-04-25T10:03:33.181690+0000","last_undegraded":"2024-04-25T10:03:33.181690+0000","last_fullsized":"2024-04-25T10:03:33.181690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363806+0000","last_change":"2024-04-25T10:03:30.358809+0000","last_active":"2024-04-25T10:03:32.363806+0000","last_peered":"2024-04-25T10:03:32.363806+0000","last_clean":"2024-04-25T10:03:32.363806+0000","last_became_active":"2024-04-25T10:03:30.358091+0000","last_became_peered":"2024-04-25T10:03:30.358091+0000","last_unstale":"2024-04-25T10:03:32.363806+0000","last_undegraded":"2024-04-25T10:03:32.363806+0000","last_fullsized":"2024-04-25T10:03:32.363806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.533743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585457+0000","last_change":"2024-04-25T10:03:31.388203+0000","last_active":"2024-04-25T10:03:32.585457+0000","last_peered":"2024-04-25T10:03:32.585457+0000","last_clean":"2024-04-25T10:03:32.585457+0000","last_became_active":"2024-04-25T10:03:31.387559+0000","last_became_peered":"2024-04-25T10:03:31.387559+0000","last_unstale":"2024-04-25T10:03:32.585457+0000","last_undegraded":"2024-04-25T10:03:32.585457+0000","last_fullsized":"2024-04-25T10:03:32.585457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:20.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360128+0000","last_change":"2024-04-25T10:03:30.356327+0000","last_active":"2024-04-25T10:03:32.360128+0000","last_peered":"2024-04-25T10:03:32.360128+0000","last_clean":"2024-04-25T10:03:32.360128+0000","last_became_active":"2024-04-25T10:03:30.355964+0000","last_became_peered":"2024-04-25T10:03:30.355964+0000","last_unstale":"2024-04-25T10:03:32.360128+0000","last_undegraded":"2024-04-25T10:03:32.360128+0000","last_fullsized":"2024-04-25T10:03:32.360128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:00.451048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334996+0000","last_change":"2024-04-25T10:03:31.368328+0000","last_active":"2024-04-25T10:03:32.334996+0000","last_peered":"2024-04-25T10:03:32.334996+0000","last_clean":"2024-04-25T10:03:32.334996+0000","last_became_active":"2024-04-25T10:03:31.367800+0000","last_became_peered":"2024-04-25T10:03:31.367800+0000","last_unstale":"2024-04-25T10:03:32.334996+0000","last_undegraded":"2024-04-25T10:03:32.334996+0000","last_fullsized":"2024-04-25T10:03:32.334996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:43.548567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181133+0000","last_change":"2024-04-25T10:03:30.341416+0000","last_active":"2024-04-25T10:03:33.181133+0000","last_peered":"2024-04-25T10:03:33.181133+0000","last_clean":"2024-04-25T10:03:33.181133+0000","last_became_active":"2024-04-25T10:03:30.341007+0000","last_became_peered":"2024-04-25T10:03:30.341007+0000","last_unstale":"2024-04-25T10:03:33.181133+0000","last_undegraded":"2024-04-25T10:03:33.181133+0000","last_fullsized":"2024-04-25T10:03:33.181133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180454+0000","last_change":"2024-04-25T10:03:31.365910+0000","last_active":"2024-04-25T10:03:33.180454+0000","last_peered":"2024-04-25T10:03:33.180454+0000","last_clean":"2024-04-25T10:03:33.180454+0000","last_became_active":"2024-04-25T10:03:31.365532+0000","last_became_peered":"2024-04-25T10:03:31.365532+0000","last_unstale":"2024-04-25T10:03:33.180454+0000","last_undegraded":"2024-04-25T10:03:33.180454+0000","last_fullsized":"2024-04-25T10:03:33.180454+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.405550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180640+0000","last_change":"2024-04-25T10:03:30.357850+0000","last_active":"2024-04-25T10:03:33.180640+0000","last_peered":"2024-04-25T10:03:33.180640+0000","last_clean":"2024-04-25T10:03:33.180640+0000","last_became_active":"2024-04-25T10:03:30.357332+0000","last_became_peered":"2024-04-25T10:03:30.357332+0000","last_unstale":"2024-04-25T10:03:33.180640+0000","last_undegraded":"2024-04-25T10:03:33.180640+0000","last_fullsized":"2024-04-25T10:03:33.180640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:08.615248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182001+0000","last_change":"2024-04-25T10:03:31.359781+0000","last_active":"2024-04-25T10:03:33.182001+0000","last_peered":"2024-04-25T10:03:33.182001+0000","last_clean":"2024-04-25T10:03:33.182001+0000","last_became_active":"2024-04-25T10:03:31.359433+0000","last_became_peered":"2024-04-25T10:03:31.359433+0000","last_unstale":"2024-04-25T10:03:33.182001+0000","last_undegraded":"2024-04-25T10:03:33.182001+0000","last_fullsized":"2024-04-25T10:03:33.182001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.405550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334251+0000","last_change":"2024-04-25T10:03:30.344524+0000","last_active":"2024-04-25T10:03:32.334251+0000","last_peered":"2024-04-25T10:03:32.334251+0000","last_clean":"2024-04-25T10:03:32.334251+0000","last_became_active":"2024-04-25T10:03:30.343370+0000","last_became_peered":"2024-04-25T10:03:30.343370+0000","last_unstale":"2024-04-25T10:03:32.334251+0000","last_undegraded":"2024-04-25T10:03:32.334251+0000","last_fullsized":"2024-04-25T10:03:32.334251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336373+0000","last_change":"2024-04-25T10:03:31.369390+0000","last_active":"2024-04-25T10:03:32.336373+0000","last_peered":"2024-04-25T10:03:32.336373+0000","last_clean":"2024-04-25T10:03:32.336373+0000","last_became_active":"2024-04-25T10:03:31.368682+0000","last_became_peered":"2024-04-25T10:03:31.368682+0000","last_unstale":"2024-04-25T10:03:32.336373+0000","last_undegraded":"2024-04-25T10:03:32.336373+0000","last_fullsized":"2024-04-25T10:03:32.336373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:04.546401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335919+0000","last_change":"2024-04-25T10:03:30.355381+0000","last_active":"2024-04-25T10:03:32.335919+0000","last_peered":"2024-04-25T10:03:32.335919+0000","last_clean":"2024-04-25T10:03:32.335919+0000","last_became_active":"2024-04-25T10:03:30.354430+0000","last_became_peered":"2024-04-25T10:03:30.354430+0000","last_unstale":"2024-04-25T10:03:32.335919+0000","last_undegraded":"2024-04-25T10:03:32.335919+0000","last_fullsized":"2024-04-25T10:03:32.335919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.280508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585962+0000","last_change":"2024-04-25T10:03:31.388793+0000","last_active":"2024-04-25T10:03:32.585962+0000","last_peered":"2024-04-25T10:03:32.585962+0000","last_clean":"2024-04-25T10:03:32.585962+0000","last_became_active":"2024-04-25T10:03:31.387890+0000","last_became_peered":"2024-04-25T10:03:31.387890+0000","last_unstale":"2024-04-25T10:03:32.585962+0000","last_undegraded":"2024-04-25T10:03:32.585962+0000","last_fullsized":"2024-04-25T10:03:32.585962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334040+0000","last_change":"2024-04-25T10:03:30.340638+0000","last_active":"2024-04-25T10:03:32.334040+0000","last_peered":"2024-04-25T10:03:32.334040+0000","last_clean":"2024-04-25T10:03:32.334040+0000","last_became_active":"2024-04-25T10:03:30.340365+0000","last_became_peered":"2024-04-25T10:03:30.340365+0000","last_unstale":"2024-04-25T10:03:32.334040+0000","last_undegraded":"2024-04-25T10:03:32.334040+0000","last_fullsized":"2024-04-25T10:03:32.334040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.533743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363526+0000","last_change":"2024-04-25T10:03:31.360025+0000","last_active":"2024-04-25T10:03:32.363526+0000","last_peered":"2024-04-25T10:03:32.363526+0000","last_clean":"2024-04-25T10:03:32.363526+0000","last_became_active":"2024-04-25T10:03:31.354209+0000","last_became_peered":"2024-04-25T10:03:31.354209+0000","last_unstale":"2024-04-25T10:03:32.363526+0000","last_undegraded":"2024-04-25T10:03:32.363526+0000","last_fullsized":"2024-04-25T10:03:32.363526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:59.932672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585700+0000","last_change":"2024-04-25T10:03:30.359055+0000","last_active":"2024-04-25T10:03:32.585700+0000","last_peered":"2024-04-25T10:03:32.585700+0000","last_clean":"2024-04-25T10:03:32.585700+0000","last_became_active":"2024-04-25T10:03:30.358297+0000","last_became_peered":"2024-04-25T10:03:30.358297+0000","last_unstale":"2024-04-25T10:03:32.585700+0000","last_undegraded":"2024-04-25T10:03:32.585700+0000","last_fullsized":"2024-04-25T10:03:32.585700+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:25.729718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585114+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:03:32.585114+0000","last_peered":"2024-04-25T10:03:32.585114+0000","last_clean":"2024-04-25T10:03:32.585114+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:03:32.585114+0000","last_undegraded":"2024-04-25T10:03:32.585114+0000","last_fullsized":"2024-04-25T10:03:32.585114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":786911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":195465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":196507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":785263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:35.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.487+0000 7ffb78602640 1 Processor -- start 2024-04-25T10:03:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb78602640 1 -- start start 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb78602640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb7015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb78602640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7015ae30 con 0x7ffb700718c0 2024-04-25T10:03:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb78602640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7015b110 con 0x7ffb70159ea0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb78602640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700b2990 con 0x7ffb7015a280 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb76b9f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffb700718c0 0x7ffb7006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54772/0 (socket says 172.21.15.82:54772) 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb76b9f640 1 -- 172.21.15.82:0/1901954877 learned_addr learned my addr 172.21.15.82:0/1901954877 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1405186379 0 0) 0x7ffb7015ae30 con 0x7ffb700718c0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb7015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb58003450 con 0x7ffb700718c0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 642642195 0 0) 0x7ffb58003450 con 0x7ffb700718c0 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 msgr2=0x7ffb7015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb7015a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 >> v1:172.21.15.196:6789/0 conn(0x7ffb70159ea0 legacy=0x7ffb700b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700b2620 con 0x7ffb700718c0 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ffb60001680 con 0x7ffb700718c0 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 >> v1:172.21.15.82:6789/0 conn(0x7ffb700718c0 legacy=0x7ffb7006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb58003b50 0x7ffb58003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb58008960 0x7ffb5800add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb5800d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700b2990 con 0x7ffb58008960 2024-04-25T10:03:35.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5800dd80 con 0x7ffb58003b50 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5800df80 con 0x7ffb7015a280 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb58008960 0x7ffb5800add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb7639e640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb58003b50 0x7ffb58003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb76b9f640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb5800d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 -- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 msgr2=0x7ffb5800d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb7015a280 0x7ffb5800d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 -- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb58003b50 msgr2=0x7ffb58003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb58003b50 0x7ffb58003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.491+0000 7ffb75b9d640 1 -- 172.21.15.82:0/1901954877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb58003450 con 0x7ffb58008960 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb58008960 0x7ffb5800add0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400ba30 tx=0x7ffb6401c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ea01f220a514af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640ae070 con 0x7ffb58008960 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb640a9680 con 0x7ffb58008960 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb7539c640 1 -- 172.21.15.82:0/1901954877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400b600 con 0x7ffb58008960 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/1901954877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb58008960 msgr2=0x7ffb5800add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 --2- 172.21.15.82:0/1901954877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb58008960 0x7ffb5800add0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400ba30 tx=0x7ffb6401c6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb76b9f640 1 -- 172.21.15.82:0/1901954877 reap_dead start 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/1901954877 shutdown_connections 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/1901954877 >> 172.21.15.82:0/1901954877 conn(0x7ffb700b11a0 msgr2=0x7ffb700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/1901954877 shutdown_connections 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/1901954877 wait complete. 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 Processor -- start 2024-04-25T10:03:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- start start 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 0x7ffb700ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb70159290 0x7ffb701f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7006d7a0 con 0x7ffb700c4530 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb700b3530 con 0x7ffb7014aee0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7006d920 con 0x7ffb70159290 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42124/0 (socket says 172.21.15.82:42124) 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 -- 172.21.15.82:0/4087929258 learned_addr learned my addr 172.21.15.82:0/4087929258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb7639e640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 0x7ffb700ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 -- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb70159290 msgr2=0x7ffb701f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb70159290 0x7ffb701f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 -- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 msgr2=0x7ffb700ab0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 0x7ffb700ab0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 -- 172.21.15.82:0/4087929258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb700abc90 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb7639e640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 0x7ffb700ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb75b9d640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb640980b0 tx=0x7ffb58003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640ae040 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb640b7070 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb700abf70 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb640a9e60 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.495+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb700ac530 con 0x7ffb700c4530 2024-04-25T10:03:35.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb701596b0 con 0x7ffb700c4530 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb640b1070 con 0x7ffb700c4530 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb6effd640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb7639e640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb6411fa20 con 0x7ffb700c4530 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.499+0000 7ffb7639e640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 0x7ffb5c069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb68001dd0 tx=0x7ffb68009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.503+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb640163b0 con 0x7ffb700c4530 2024-04-25T10:03:35.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.655+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb7006e6b0 con 0x7ffb5c066f60 2024-04-25T10:03:35.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.663+0000 7ffb6effd640 1 -- 172.21.15.82:0/4087929258 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251960 (secure 0 0 0) 0x7ffb7006e6b0 con 0x7ffb5c066f60 2024-04-25T10:03:35.668 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:35.670 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 msgr2=0x7ffb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 0x7ffb5c069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ffb68001dd0 tx=0x7ffb68009040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 msgr2=0x7ffb70158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ffb640980b0 tx=0x7ffb58003740 comp rx=0 tx=0).stop 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 shutdown_connections 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb70159290 0x7ffb701f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb700c4530 0x7ffb70158d50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 --2- 172.21.15.82:0/4087929258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb7014aee0 0x7ffb700ab0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 >> 172.21.15.82:0/4087929258 conn(0x7ffb700b11a0 msgr2=0x7ffb700b04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 shutdown_connections 2024-04-25T10:03:35.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.667+0000 7ffb78602640 1 -- 172.21.15.82:0/4087929258 wait complete. 2024-04-25T10:03:35.685 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-25T10:03:35.287633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":128,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225084,"kb_used_data":4744,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754749636,"statfs":{"total":773094113280,"available":772863627264,"internally_reserved":0,"allocated":4857856,"data_stored":2744444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.978692"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335193+0000","last_change":"2024-04-25T10:03:31.351112+0000","last_active":"2024-04-25T10:03:32.335193+0000","last_peered":"2024-04-25T10:03:32.335193+0000","last_clean":"2024-04-25T10:03:32.335193+0000","last_became_active":"2024-04-25T10:03:31.350738+0000","last_became_peered":"2024-04-25T10:03:31.350738+0000","last_unstale":"2024-04-25T10:03:32.335193+0000","last_undegraded":"2024-04-25T10:03:32.335193+0000","last_fullsized":"2024-04-25T10:03:32.335193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:05.515973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335099+0000","last_change":"2024-04-25T10:03:31.368660+0000","last_active":"2024-04-25T10:03:32.335099+0000","last_peered":"2024-04-25T10:03:32.335099+0000","last_clean":"2024-04-25T10:03:32.335099+0000","last_became_active":"2024-04-25T10:03:31.364478+0000","last_became_peered":"2024-04-25T10:03:31.364478+0000","last_unstale":"2024-04-25T10:03:32.335099+0000","last_undegraded":"2024-04-25T10:03:32.335099+0000","last_fullsized":"2024-04-25T10:03:32.335099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:30.553923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181190+0000","last_change":"2024-04-25T10:03:31.360312+0000","last_active":"2024-04-25T10:03:33.181190+0000","last_peered":"2024-04-25T10:03:33.181190+0000","last_clean":"2024-04-25T10:03:33.181190+0000","last_became_active":"2024-04-25T10:03:31.359362+0000","last_became_peered":"2024-04-25T10:03:31.359362+0000","last_unstale":"2024-04-25T10:03:33.181190+0000","last_undegraded":"2024-04-25T10:03:33.181190+0000","last_fullsized":"2024-04-25T10:03:33.181190+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:20.714126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359998+0000","last_change":"2024-04-25T10:03:31.349686+0000","last_active":"2024-04-25T10:03:32.359998+0000","last_peered":"2024-04-25T10:03:32.359998+0000","last_clean":"2024-04-25T10:03:32.359998+0000","last_became_active":"2024-04-25T10:03:31.348361+0000","last_became_peered":"2024-04-25T10:03:31.348361+0000","last_unstale":"2024-04-25T10:03:32.359998+0000","last_undegraded":"2024-04-25T10:03:32.359998+0000","last_fullsized":"2024-04-25T10:03:32.359998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:25.400328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359994+0000","last_change":"2024-04-25T10:03:31.345856+0000","last_active":"2024-04-25T10:03:32.359994+0000","last_peered":"2024-04-25T10:03:32.359994+0000","last_clean":"2024-04-25T10:03:32.359994+0000","last_became_active":"2024-04-25T10:03:31.344526+0000","last_became_peered":"2024-04-25T10:03:31.344526+0000","last_unstale":"2024-04-25T10:03:32.359994+0000","last_undegraded":"2024-04-25T10:03:32.359994+0000","last_fullsized":"2024-04-25T10:03:32.359994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:48.559110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181876+0000","last_change":"2024-04-25T10:03:31.369315+0000","last_active":"2024-04-25T10:03:33.181876+0000","last_peered":"2024-04-25T10:03:33.181876+0000","last_clean":"2024-04-25T10:03:33.181876+0000","last_became_active":"2024-04-25T10:03:31.369018+0000","last_became_peered":"2024-04-25T10:03:31.369018+0000","last_unstale":"2024-04-25T10:03:33.181876+0000","last_undegraded":"2024-04-25T10:03:33.181876+0000","last_fullsized":"2024-04-25T10:03:33.181876+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362207+0000","last_change":"2024-04-25T10:03:31.369966+0000","last_active":"2024-04-25T10:03:32.362207+0000","last_peered":"2024-04-25T10:03:32.362207+0000","last_clean":"2024-04-25T10:03:32.362207+0000","last_became_active":"2024-04-25T10:03:31.369177+0000","last_became_peered":"2024-04-25T10:03:31.369177+0000","last_unstale":"2024-04-25T10:03:32.362207+0000","last_undegraded":"2024-04-25T10:03:32.362207+0000","last_fullsized":"2024-04-25T10:03:32.362207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334524+0000","last_change":"2024-04-25T10:03:31.371142+0000","last_active":"2024-04-25T10:03:32.334524+0000","last_peered":"2024-04-25T10:03:32.334524+0000","last_clean":"2024-04-25T10:03:32.334524+0000","last_became_active":"2024-04-25T10:03:31.370803+0000","last_became_peered":"2024-04-25T10:03:31.370803+0000","last_unstale":"2024-04-25T10:03:32.334524+0000","last_undegraded":"2024-04-25T10:03:32.334524+0000","last_fullsized":"2024-04-25T10:03:32.334524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180677+0000","last_change":"2024-04-25T10:03:31.359788+0000","last_active":"2024-04-25T10:03:33.180677+0000","last_peered":"2024-04-25T10:03:33.180677+0000","last_clean":"2024-04-25T10:03:33.180677+0000","last_became_active":"2024-04-25T10:03:31.359453+0000","last_became_peered":"2024-04-25T10:03:31.359453+0000","last_unstale":"2024-04-25T10:03:33.180677+0000","last_undegraded":"2024-04-25T10:03:33.180677+0000","last_fullsized":"2024-04-25T10:03:33.180677+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:50.626046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363150+0000","last_change":"2024-04-25T10:03:31.359931+0000","last_active":"2024-04-25T10:03:32.363150+0000","last_peered":"2024-04-25T10:03:32.363150+0000","last_clean":"2024-04-25T10:03:32.363150+0000","last_became_active":"2024-04-25T10:03:31.359227+0000","last_became_peered":"2024-04-25T10:03:31.359227+0000","last_unstale":"2024-04-25T10:03:32.363150+0000","last_undegraded":"2024-04-25T10:03:32.363150+0000","last_fullsized":"2024-04-25T10:03:32.363150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:04.839212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363209+0000","last_change":"2024-04-25T10:03:31.379913+0000","last_active":"2024-04-25T10:03:32.363209+0000","last_peered":"2024-04-25T10:03:32.363209+0000","last_clean":"2024-04-25T10:03:32.363209+0000","last_became_active":"2024-04-25T10:03:31.375798+0000","last_became_peered":"2024-04-25T10:03:31.375798+0000","last_unstale":"2024-04-25T10:03:32.363209+0000","last_undegraded":"2024-04-25T10:03:32.363209+0000","last_fullsized":"2024-04-25T10:03:32.363209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:03.899769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180767+0000","last_change":"2024-04-25T10:03:31.352737+0000","last_active":"2024-04-25T10:03:33.180767+0000","last_peered":"2024-04-25T10:03:33.180767+0000","last_clean":"2024-04-25T10:03:33.180767+0000","last_became_active":"2024-04-25T10:03:31.352457+0000","last_became_peered":"2024-04-25T10:03:31.352457+0000","last_unstale":"2024-04-25T10:03:33.180767+0000","last_undegraded":"2024-04-25T10:03:33.180767+0000","last_fullsized":"2024-04-25T10:03:33.180767+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:06.476635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362364+0000","last_change":"2024-04-25T10:03:31.374237+0000","last_active":"2024-04-25T10:03:32.362364+0000","last_peered":"2024-04-25T10:03:32.362364+0000","last_clean":"2024-04-25T10:03:32.362364+0000","last_became_active":"2024-04-25T10:03:31.370286+0000","last_became_peered":"2024-04-25T10:03:31.370286+0000","last_unstale":"2024-04-25T10:03:32.362364+0000","last_undegraded":"2024-04-25T10:03:32.362364+0000","last_fullsized":"2024-04-25T10:03:32.362364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336233+0000","last_change":"2024-04-25T10:03:31.358324+0000","last_active":"2024-04-25T10:03:32.336233+0000","last_peered":"2024-04-25T10:03:32.336233+0000","last_clean":"2024-04-25T10:03:32.336233+0000","last_became_active":"2024-04-25T10:03:31.357584+0000","last_became_peered":"2024-04-25T10:03:31.357584+0000","last_unstale":"2024-04-25T10:03:32.336233+0000","last_undegraded":"2024-04-25T10:03:32.336233+0000","last_fullsized":"2024-04-25T10:03:32.336233+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:19.413525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362239+0000","last_change":"2024-04-25T10:03:31.352205+0000","last_active":"2024-04-25T10:03:32.362239+0000","last_peered":"2024-04-25T10:03:32.362239+0000","last_clean":"2024-04-25T10:03:32.362239+0000","last_became_active":"2024-04-25T10:03:31.351452+0000","last_became_peered":"2024-04-25T10:03:31.351452+0000","last_unstale":"2024-04-25T10:03:32.362239+0000","last_undegraded":"2024-04-25T10:03:32.362239+0000","last_fullsized":"2024-04-25T10:03:32.362239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584884+0000","last_change":"2024-04-25T10:03:31.393451+0000","last_active":"2024-04-25T10:03:32.584884+0000","last_peered":"2024-04-25T10:03:32.584884+0000","last_clean":"2024-04-25T10:03:32.584884+0000","last_became_active":"2024-04-25T10:03:31.388042+0000","last_became_peered":"2024-04-25T10:03:31.388042+0000","last_unstale":"2024-04-25T10:03:32.584884+0000","last_undegraded":"2024-04-25T10:03:32.584884+0000","last_fullsized":"2024-04-25T10:03:32.584884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:06.476635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585939+0000","last_change":"2024-04-25T10:03:31.392272+0000","last_active":"2024-04-25T10:03:32.585939+0000","last_peered":"2024-04-25T10:03:32.585939+0000","last_clean":"2024-04-25T10:03:32.585939+0000","last_became_active":"2024-04-25T10:03:31.391511+0000","last_became_peered":"2024-04-25T10:03:31.391511+0000","last_unstale":"2024-04-25T10:03:32.585939+0000","last_undegraded":"2024-04-25T10:03:32.585939+0000","last_fullsized":"2024-04-25T10:03:32.585939+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335383+0000","last_change":"2024-04-25T10:03:31.368583+0000","last_active":"2024-04-25T10:03:32.335383+0000","last_peered":"2024-04-25T10:03:32.335383+0000","last_clean":"2024-04-25T10:03:32.335383+0000","last_became_active":"2024-04-25T10:03:31.367850+0000","last_became_peered":"2024-04-25T10:03:31.367850+0000","last_unstale":"2024-04-25T10:03:32.335383+0000","last_undegraded":"2024-04-25T10:03:32.335383+0000","last_fullsized":"2024-04-25T10:03:32.335383+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:20.288540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360121+0000","last_change":"2024-04-25T10:03:31.360504+0000","last_active":"2024-04-25T10:03:32.360121+0000","last_peered":"2024-04-25T10:03:32.360121+0000","last_clean":"2024-04-25T10:03:32.360121+0000","last_became_active":"2024-04-25T10:03:31.360189+0000","last_became_peered":"2024-04-25T10:03:31.360189+0000","last_unstale":"2024-04-25T10:03:32.360121+0000","last_undegraded":"2024-04-25T10:03:32.360121+0000","last_fullsized":"2024-04-25T10:03:32.360121+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:26.742376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334543+0000","last_change":"2024-04-25T10:03:31.350162+0000","last_active":"2024-04-25T10:03:32.334543+0000","last_peered":"2024-04-25T10:03:32.334543+0000","last_clean":"2024-04-25T10:03:32.334543+0000","last_became_active":"2024-04-25T10:03:31.347504+0000","last_became_peered":"2024-04-25T10:03:31.347504+0000","last_unstale":"2024-04-25T10:03:32.334543+0000","last_undegraded":"2024-04-25T10:03:32.334543+0000","last_fullsized":"2024-04-25T10:03:32.334543+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:50.030249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362524+0000","last_change":"2024-04-25T10:03:31.358283+0000","last_active":"2024-04-25T10:03:32.362524+0000","last_peered":"2024-04-25T10:03:32.362524+0000","last_clean":"2024-04-25T10:03:32.362524+0000","last_became_active":"2024-04-25T10:03:31.357410+0000","last_became_peered":"2024-04-25T10:03:31.357410+0000","last_unstale":"2024-04-25T10:03:32.362524+0000","last_undegraded":"2024-04-25T10:03:32.362524+0000","last_fullsized":"2024-04-25T10:03:32.362524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:43.548567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336843+0000","last_change":"2024-04-25T10:03:31.356340+0000","last_active":"2024-04-25T10:03:32.336843+0000","last_peered":"2024-04-25T10:03:32.336843+0000","last_clean":"2024-04-25T10:03:32.336843+0000","last_became_active":"2024-04-25T10:03:31.355625+0000","last_became_peered":"2024-04-25T10:03:31.355625+0000","last_unstale":"2024-04-25T10:03:32.336843+0000","last_undegraded":"2024-04-25T10:03:32.336843+0000","last_fullsized":"2024-04-25T10:03:32.336843+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:20.288540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584980+0000","last_change":"2024-04-25T10:03:31.387372+0000","last_active":"2024-04-25T10:03:32.584980+0000","last_peered":"2024-04-25T10:03:32.584980+0000","last_clean":"2024-04-25T10:03:32.584980+0000","last_became_active":"2024-04-25T10:03:31.382677+0000","last_became_peered":"2024-04-25T10:03:31.382677+0000","last_unstale":"2024-04-25T10:03:32.584980+0000","last_undegraded":"2024-04-25T10:03:32.584980+0000","last_fullsized":"2024-04-25T10:03:32.584980+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:20.714126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359751+0000","last_change":"2024-04-25T10:03:31.394429+0000","last_active":"2024-04-25T10:03:32.359751+0000","last_peered":"2024-04-25T10:03:32.359751+0000","last_clean":"2024-04-25T10:03:32.359751+0000","last_became_active":"2024-04-25T10:03:31.393945+0000","last_became_peered":"2024-04-25T10:03:31.393945+0000","last_unstale":"2024-04-25T10:03:32.359751+0000","last_undegraded":"2024-04-25T10:03:32.359751+0000","last_fullsized":"2024-04-25T10:03:32.359751+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:03.652716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335384+0000","last_change":"2024-04-25T10:03:31.349115+0000","last_active":"2024-04-25T10:03:32.335384+0000","last_peered":"2024-04-25T10:03:32.335384+0000","last_clean":"2024-04-25T10:03:32.335384+0000","last_became_active":"2024-04-25T10:03:31.347571+0000","last_became_peered":"2024-04-25T10:03:31.347571+0000","last_unstale":"2024-04-25T10:03:32.335384+0000","last_undegraded":"2024-04-25T10:03:32.335384+0000","last_fullsized":"2024-04-25T10:03:32.335384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181908+0000","last_change":"2024-04-25T10:03:31.360221+0000","last_active":"2024-04-25T10:03:33.181908+0000","last_peered":"2024-04-25T10:03:33.181908+0000","last_clean":"2024-04-25T10:03:33.181908+0000","last_became_active":"2024-04-25T10:03:31.359834+0000","last_became_peered":"2024-04-25T10:03:31.359834+0000","last_unstale":"2024-04-25T10:03:33.181908+0000","last_undegraded":"2024-04-25T10:03:33.181908+0000","last_fullsized":"2024-04-25T10:03:33.181908+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335063+0000","last_change":"2024-04-25T10:03:31.337659+0000","last_active":"2024-04-25T10:03:32.335063+0000","last_peered":"2024-04-25T10:03:32.335063+0000","last_clean":"2024-04-25T10:03:32.335063+0000","last_became_active":"2024-04-25T10:03:31.336844+0000","last_became_peered":"2024-04-25T10:03:31.336844+0000","last_unstale":"2024-04-25T10:03:32.335063+0000","last_undegraded":"2024-04-25T10:03:32.335063+0000","last_fullsized":"2024-04-25T10:03:32.335063+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:04.546401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362986+0000","last_change":"2024-04-25T10:03:31.374141+0000","last_active":"2024-04-25T10:03:32.362986+0000","last_peered":"2024-04-25T10:03:32.362986+0000","last_clean":"2024-04-25T10:03:32.362986+0000","last_became_active":"2024-04-25T10:03:31.370907+0000","last_became_peered":"2024-04-25T10:03:31.370907+0000","last_unstale":"2024-04-25T10:03:32.362986+0000","last_undegraded":"2024-04-25T10:03:32.362986+0000","last_fullsized":"2024-04-25T10:03:32.362986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:05.515973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334633+0000","last_change":"2024-04-25T10:03:31.354041+0000","last_active":"2024-04-25T10:03:32.334633+0000","last_peered":"2024-04-25T10:03:32.334633+0000","last_clean":"2024-04-25T10:03:32.334633+0000","last_became_active":"2024-04-25T10:03:31.353237+0000","last_became_peered":"2024-04-25T10:03:31.353237+0000","last_unstale":"2024-04-25T10:03:32.334633+0000","last_undegraded":"2024-04-25T10:03:32.334633+0000","last_fullsized":"2024-04-25T10:03:32.334633+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:19.413525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334551+0000","last_change":"2024-04-25T10:03:31.369260+0000","last_active":"2024-04-25T10:03:32.334551+0000","last_peered":"2024-04-25T10:03:32.334551+0000","last_clean":"2024-04-25T10:03:32.334551+0000","last_became_active":"2024-04-25T10:03:31.368275+0000","last_became_peered":"2024-04-25T10:03:31.368275+0000","last_unstale":"2024-04-25T10:03:32.334551+0000","last_undegraded":"2024-04-25T10:03:32.334551+0000","last_fullsized":"2024-04-25T10:03:32.334551+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:36.518646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585652+0000","last_change":"2024-04-25T10:03:31.378622+0000","last_active":"2024-04-25T10:03:32.585652+0000","last_peered":"2024-04-25T10:03:32.585652+0000","last_clean":"2024-04-25T10:03:32.585652+0000","last_became_active":"2024-04-25T10:03:31.377926+0000","last_became_peered":"2024-04-25T10:03:31.377926+0000","last_unstale":"2024-04-25T10:03:32.585652+0000","last_undegraded":"2024-04-25T10:03:32.585652+0000","last_fullsized":"2024-04-25T10:03:32.585652+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:47.576388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180059+0000","last_change":"2024-04-25T10:03:31.359774+0000","last_active":"2024-04-25T10:03:33.180059+0000","last_peered":"2024-04-25T10:03:33.180059+0000","last_clean":"2024-04-25T10:03:33.180059+0000","last_became_active":"2024-04-25T10:03:31.359427+0000","last_became_peered":"2024-04-25T10:03:31.359427+0000","last_unstale":"2024-04-25T10:03:33.180059+0000","last_undegraded":"2024-04-25T10:03:33.180059+0000","last_fullsized":"2024-04-25T10:03:33.180059+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585426+0000","last_change":"2024-04-25T10:03:31.392140+0000","last_active":"2024-04-25T10:03:32.585426+0000","last_peered":"2024-04-25T10:03:32.585426+0000","last_clean":"2024-04-25T10:03:32.585426+0000","last_became_active":"2024-04-25T10:03:31.388839+0000","last_became_peered":"2024-04-25T10:03:31.388839+0000","last_unstale":"2024-04-25T10:03:32.585426+0000","last_undegraded":"2024-04-25T10:03:32.585426+0000","last_fullsized":"2024-04-25T10:03:32.585426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:38.711873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335466+0000","last_change":"2024-04-25T10:03:30.359055+0000","last_active":"2024-04-25T10:03:32.335466+0000","last_peered":"2024-04-25T10:03:32.335466+0000","last_clean":"2024-04-25T10:03:32.335466+0000","last_became_active":"2024-04-25T10:03:30.358355+0000","last_became_peered":"2024-04-25T10:03:30.358355+0000","last_unstale":"2024-04-25T10:03:32.335466+0000","last_undegraded":"2024-04-25T10:03:32.335466+0000","last_fullsized":"2024-04-25T10:03:32.335466+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:18.428515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182019+0000","last_change":"2024-04-25T10:03:31.376286+0000","last_active":"2024-04-25T10:03:33.182019+0000","last_peered":"2024-04-25T10:03:33.182019+0000","last_clean":"2024-04-25T10:03:33.182019+0000","last_became_active":"2024-04-25T10:03:31.376009+0000","last_became_peered":"2024-04-25T10:03:31.376009+0000","last_unstale":"2024-04-25T10:03:33.182019+0000","last_undegraded":"2024-04-25T10:03:33.182019+0000","last_fullsized":"2024-04-25T10:03:33.182019+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:50.626046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180867+0000","last_change":"2024-04-25T10:03:30.341884+0000","last_active":"2024-04-25T10:03:33.180867+0000","last_peered":"2024-04-25T10:03:33.180867+0000","last_clean":"2024-04-25T10:03:33.180867+0000","last_became_active":"2024-04-25T10:03:30.341386+0000","last_became_peered":"2024-04-25T10:03:30.341386+0000","last_unstale":"2024-04-25T10:03:33.180867+0000","last_undegraded":"2024-04-25T10:03:33.180867+0000","last_fullsized":"2024-04-25T10:03:33.180867+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:02.640058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337335+0000","last_change":"2024-04-25T10:03:31.361251+0000","last_active":"2024-04-25T10:03:32.337335+0000","last_peered":"2024-04-25T10:03:32.337335+0000","last_clean":"2024-04-25T10:03:32.337335+0000","last_became_active":"2024-04-25T10:03:31.359515+0000","last_became_peered":"2024-04-25T10:03:31.359515+0000","last_unstale":"2024-04-25T10:03:32.337335+0000","last_undegraded":"2024-04-25T10:03:32.337335+0000","last_fullsized":"2024-04-25T10:03:32.337335+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182149+0000","last_change":"2024-04-25T10:03:30.356646+0000","last_active":"2024-04-25T10:03:33.182149+0000","last_peered":"2024-04-25T10:03:33.182149+0000","last_clean":"2024-04-25T10:03:33.182149+0000","last_became_active":"2024-04-25T10:03:30.356150+0000","last_became_peered":"2024-04-25T10:03:30.356150+0000","last_unstale":"2024-04-25T10:03:33.182149+0000","last_undegraded":"2024-04-25T10:03:33.182149+0000","last_fullsized":"2024-04-25T10:03:33.182149+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:24.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334900+0000","last_change":"2024-04-25T10:03:31.363890+0000","last_active":"2024-04-25T10:03:32.334900+0000","last_peered":"2024-04-25T10:03:32.334900+0000","last_clean":"2024-04-25T10:03:32.334900+0000","last_became_active":"2024-04-25T10:03:31.363405+0000","last_became_peered":"2024-04-25T10:03:31.363405+0000","last_unstale":"2024-04-25T10:03:32.334900+0000","last_undegraded":"2024-04-25T10:03:32.334900+0000","last_fullsized":"2024-04-25T10:03:32.334900+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:45.391664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336641+0000","last_change":"2024-04-25T10:03:30.349762+0000","last_active":"2024-04-25T10:03:32.336641+0000","last_peered":"2024-04-25T10:03:32.336641+0000","last_clean":"2024-04-25T10:03:32.336641+0000","last_became_active":"2024-04-25T10:03:30.349044+0000","last_became_peered":"2024-04-25T10:03:30.349044+0000","last_unstale":"2024-04-25T10:03:32.336641+0000","last_undegraded":"2024-04-25T10:03:32.336641+0000","last_fullsized":"2024-04-25T10:03:32.336641+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:08.615248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.179970+0000","last_change":"2024-04-25T10:03:31.360522+0000","last_active":"2024-04-25T10:03:33.179970+0000","last_peered":"2024-04-25T10:03:33.179970+0000","last_clean":"2024-04-25T10:03:33.179970+0000","last_became_active":"2024-04-25T10:03:31.360178+0000","last_became_peered":"2024-04-25T10:03:31.360178+0000","last_unstale":"2024-04-25T10:03:33.179970+0000","last_undegraded":"2024-04-25T10:03:33.179970+0000","last_fullsized":"2024-04-25T10:03:33.179970+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362453+0000","last_change":"2024-04-25T10:03:30.351283+0000","last_active":"2024-04-25T10:03:32.362453+0000","last_peered":"2024-04-25T10:03:32.362453+0000","last_clean":"2024-04-25T10:03:32.362453+0000","last_became_active":"2024-04-25T10:03:30.349036+0000","last_became_peered":"2024-04-25T10:03:30.349036+0000","last_unstale":"2024-04-25T10:03:32.362453+0000","last_undegraded":"2024-04-25T10:03:32.362453+0000","last_fullsized":"2024-04-25T10:03:32.362453+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:19.275882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362661+0000","last_change":"2024-04-25T10:03:31.378874+0000","last_active":"2024-04-25T10:03:32.362661+0000","last_peered":"2024-04-25T10:03:32.362661+0000","last_clean":"2024-04-25T10:03:32.362661+0000","last_became_active":"2024-04-25T10:03:31.375200+0000","last_became_peered":"2024-04-25T10:03:31.375200+0000","last_unstale":"2024-04-25T10:03:32.362661+0000","last_undegraded":"2024-04-25T10:03:32.362661+0000","last_fullsized":"2024-04-25T10:03:32.362661+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:30.553923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585179+0000","last_change":"2024-04-25T10:03:30.360622+0000","last_active":"2024-04-25T10:03:32.585179+0000","last_peered":"2024-04-25T10:03:32.585179+0000","last_clean":"2024-04-25T10:03:32.585179+0000","last_became_active":"2024-04-25T10:03:30.360011+0000","last_became_peered":"2024-04-25T10:03:30.360011+0000","last_unstale":"2024-04-25T10:03:32.585179+0000","last_undegraded":"2024-04-25T10:03:32.585179+0000","last_fullsized":"2024-04-25T10:03:32.585179+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:24.387670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336884+0000","last_change":"2024-04-25T10:03:31.354798+0000","last_active":"2024-04-25T10:03:32.336884+0000","last_peered":"2024-04-25T10:03:32.336884+0000","last_clean":"2024-04-25T10:03:32.336884+0000","last_became_active":"2024-04-25T10:03:31.346353+0000","last_became_peered":"2024-04-25T10:03:31.346353+0000","last_unstale":"2024-04-25T10:03:32.336884+0000","last_undegraded":"2024-04-25T10:03:32.336884+0000","last_fullsized":"2024-04-25T10:03:32.336884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:24.931077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181221+0000","last_change":"2024-04-25T10:03:30.365631+0000","last_active":"2024-04-25T10:03:33.181221+0000","last_peered":"2024-04-25T10:03:33.181221+0000","last_clean":"2024-04-25T10:03:33.181221+0000","last_became_active":"2024-04-25T10:03:30.365299+0000","last_became_peered":"2024-04-25T10:03:30.365299+0000","last_unstale":"2024-04-25T10:03:33.181221+0000","last_undegraded":"2024-04-25T10:03:33.181221+0000","last_fullsized":"2024-04-25T10:03:33.181221+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334364+0000","last_change":"2024-04-25T10:03:31.389546+0000","last_active":"2024-04-25T10:03:32.334364+0000","last_peered":"2024-04-25T10:03:32.334364+0000","last_clean":"2024-04-25T10:03:32.334364+0000","last_became_active":"2024-04-25T10:03:31.389164+0000","last_became_peered":"2024-04-25T10:03:31.389164+0000","last_unstale":"2024-04-25T10:03:32.334364+0000","last_undegraded":"2024-04-25T10:03:32.334364+0000","last_fullsized":"2024-04-25T10:03:32.334364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335753+0000","last_change":"2024-04-25T10:03:30.360937+0000","last_active":"2024-04-25T10:03:32.335753+0000","last_peered":"2024-04-25T10:03:32.335753+0000","last_clean":"2024-04-25T10:03:32.335753+0000","last_became_active":"2024-04-25T10:03:30.360285+0000","last_became_peered":"2024-04-25T10:03:30.360285+0000","last_unstale":"2024-04-25T10:03:32.335753+0000","last_undegraded":"2024-04-25T10:03:32.335753+0000","last_fullsized":"2024-04-25T10:03:32.335753+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360297+0000","last_change":"2024-04-25T10:03:31.349045+0000","last_active":"2024-04-25T10:03:32.360297+0000","last_peered":"2024-04-25T10:03:32.360297+0000","last_clean":"2024-04-25T10:03:32.360297+0000","last_became_active":"2024-04-25T10:03:31.347424+0000","last_became_peered":"2024-04-25T10:03:31.347424+0000","last_unstale":"2024-04-25T10:03:32.360297+0000","last_undegraded":"2024-04-25T10:03:32.360297+0000","last_fullsized":"2024-04-25T10:03:32.360297+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:25.231950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335156+0000","last_change":"2024-04-25T10:03:30.341060+0000","last_active":"2024-04-25T10:03:32.335156+0000","last_peered":"2024-04-25T10:03:32.335156+0000","last_clean":"2024-04-25T10:03:32.335156+0000","last_became_active":"2024-04-25T10:03:30.340582+0000","last_became_peered":"2024-04-25T10:03:30.340582+0000","last_unstale":"2024-04-25T10:03:32.335156+0000","last_undegraded":"2024-04-25T10:03:32.335156+0000","last_fullsized":"2024-04-25T10:03:32.335156+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:18.400867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585773+0000","last_change":"2024-04-25T10:03:31.392364+0000","last_active":"2024-04-25T10:03:32.585773+0000","last_peered":"2024-04-25T10:03:32.585773+0000","last_clean":"2024-04-25T10:03:32.585773+0000","last_became_active":"2024-04-25T10:03:31.391563+0000","last_became_peered":"2024-04-25T10:03:31.391563+0000","last_unstale":"2024-04-25T10:03:32.585773+0000","last_undegraded":"2024-04-25T10:03:32.585773+0000","last_fullsized":"2024-04-25T10:03:32.585773+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:24.867273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363540+0000","last_change":"2024-04-25T10:03:30.363343+0000","last_active":"2024-04-25T10:03:32.363540+0000","last_peered":"2024-04-25T10:03:32.363540+0000","last_clean":"2024-04-25T10:03:32.363540+0000","last_became_active":"2024-04-25T10:03:30.362630+0000","last_became_peered":"2024-04-25T10:03:30.362630+0000","last_unstale":"2024-04-25T10:03:32.363540+0000","last_undegraded":"2024-04-25T10:03:32.363540+0000","last_fullsized":"2024-04-25T10:03:32.363540+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:23.918419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180384+0000","last_change":"2024-04-25T10:03:31.393934+0000","last_active":"2024-04-25T10:03:33.180384+0000","last_peered":"2024-04-25T10:03:33.180384+0000","last_clean":"2024-04-25T10:03:33.180384+0000","last_became_active":"2024-04-25T10:03:31.393636+0000","last_became_peered":"2024-04-25T10:03:31.393636+0000","last_unstale":"2024-04-25T10:03:33.180384+0000","last_undegraded":"2024-04-25T10:03:33.180384+0000","last_fullsized":"2024-04-25T10:03:33.180384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:18.787990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363356+0000","last_change":"2024-04-25T10:03:30.360288+0000","last_active":"2024-04-25T10:03:32.363356+0000","last_peered":"2024-04-25T10:03:32.363356+0000","last_clean":"2024-04-25T10:03:32.363356+0000","last_became_active":"2024-04-25T10:03:30.359605+0000","last_became_peered":"2024-04-25T10:03:30.359605+0000","last_unstale":"2024-04-25T10:03:32.363356+0000","last_undegraded":"2024-04-25T10:03:32.363356+0000","last_fullsized":"2024-04-25T10:03:32.363356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:20.024687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337279+0000","last_change":"2024-04-25T10:03:31.353931+0000","last_active":"2024-04-25T10:03:32.337279+0000","last_peered":"2024-04-25T10:03:32.337279+0000","last_clean":"2024-04-25T10:03:32.337279+0000","last_became_active":"2024-04-25T10:03:31.353097+0000","last_became_peered":"2024-04-25T10:03:31.353097+0000","last_unstale":"2024-04-25T10:03:32.337279+0000","last_undegraded":"2024-04-25T10:03:32.337279+0000","last_fullsized":"2024-04-25T10:03:32.337279+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334830+0000","last_change":"2024-04-25T10:03:30.346414+0000","last_active":"2024-04-25T10:03:32.334830+0000","last_peered":"2024-04-25T10:03:32.334830+0000","last_clean":"2024-04-25T10:03:32.334830+0000","last_became_active":"2024-04-25T10:03:30.346187+0000","last_became_peered":"2024-04-25T10:03:30.346187+0000","last_unstale":"2024-04-25T10:03:32.334830+0000","last_undegraded":"2024-04-25T10:03:32.334830+0000","last_fullsized":"2024-04-25T10:03:32.334830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:35.505988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180099+0000","last_change":"2024-04-25T10:03:31.359851+0000","last_active":"2024-04-25T10:03:33.180099+0000","last_peered":"2024-04-25T10:03:33.180099+0000","last_clean":"2024-04-25T10:03:33.180099+0000","last_became_active":"2024-04-25T10:03:31.359545+0000","last_became_peered":"2024-04-25T10:03:31.359545+0000","last_unstale":"2024-04-25T10:03:33.180099+0000","last_undegraded":"2024-04-25T10:03:33.180099+0000","last_fullsized":"2024-04-25T10:03:33.180099+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334321+0000","last_change":"2024-04-25T10:03:30.348763+0000","last_active":"2024-04-25T10:03:32.334321+0000","last_peered":"2024-04-25T10:03:32.334321+0000","last_clean":"2024-04-25T10:03:32.334321+0000","last_became_active":"2024-04-25T10:03:30.348505+0000","last_became_peered":"2024-04-25T10:03:30.348505+0000","last_unstale":"2024-04-25T10:03:32.334321+0000","last_undegraded":"2024-04-25T10:03:32.334321+0000","last_fullsized":"2024-04-25T10:03:32.334321+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:23.918419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334404+0000","last_change":"2024-04-25T10:03:31.368145+0000","last_active":"2024-04-25T10:03:32.334404+0000","last_peered":"2024-04-25T10:03:32.334404+0000","last_clean":"2024-04-25T10:03:32.334404+0000","last_became_active":"2024-04-25T10:03:31.363662+0000","last_became_peered":"2024-04-25T10:03:31.363662+0000","last_unstale":"2024-04-25T10:03:32.334404+0000","last_undegraded":"2024-04-25T10:03:32.334404+0000","last_fullsized":"2024-04-25T10:03:32.334404+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:24.867273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181187+0000","last_change":"2024-04-25T10:03:30.353157+0000","last_active":"2024-04-25T10:03:33.181187+0000","last_peered":"2024-04-25T10:03:33.181187+0000","last_clean":"2024-04-25T10:03:33.181187+0000","last_became_active":"2024-04-25T10:03:30.352824+0000","last_became_peered":"2024-04-25T10:03:30.352824+0000","last_unstale":"2024-04-25T10:03:33.181187+0000","last_undegraded":"2024-04-25T10:03:33.181187+0000","last_fullsized":"2024-04-25T10:03:33.181187+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:47.546452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585574+0000","last_change":"2024-04-25T10:03:31.379271+0000","last_active":"2024-04-25T10:03:32.585574+0000","last_peered":"2024-04-25T10:03:32.585574+0000","last_clean":"2024-04-25T10:03:32.585574+0000","last_became_active":"2024-04-25T10:03:31.377835+0000","last_became_peered":"2024-04-25T10:03:31.377835+0000","last_unstale":"2024-04-25T10:03:32.585574+0000","last_undegraded":"2024-04-25T10:03:32.585574+0000","last_fullsized":"2024-04-25T10:03:32.585574+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:45.405473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.333964+0000","last_change":"2024-04-25T10:03:30.358881+0000","last_active":"2024-04-25T10:03:32.333964+0000","last_peered":"2024-04-25T10:03:32.333964+0000","last_clean":"2024-04-25T10:03:32.333964+0000","last_became_active":"2024-04-25T10:03:30.358164+0000","last_became_peered":"2024-04-25T10:03:30.358164+0000","last_unstale":"2024-04-25T10:03:32.333964+0000","last_undegraded":"2024-04-25T10:03:32.333964+0000","last_fullsized":"2024-04-25T10:03:32.333964+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.280508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334040+0000","last_change":"2024-04-25T10:03:31.364194+0000","last_active":"2024-04-25T10:03:32.334040+0000","last_peered":"2024-04-25T10:03:32.334040+0000","last_clean":"2024-04-25T10:03:32.334040+0000","last_became_active":"2024-04-25T10:03:31.363883+0000","last_became_peered":"2024-04-25T10:03:31.363883+0000","last_unstale":"2024-04-25T10:03:32.334040+0000","last_undegraded":"2024-04-25T10:03:32.334040+0000","last_fullsized":"2024-04-25T10:03:32.334040+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:20.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.584681+0000","last_change":"2024-04-25T10:03:30.358682+0000","last_active":"2024-04-25T10:03:32.584681+0000","last_peered":"2024-04-25T10:03:32.584681+0000","last_clean":"2024-04-25T10:03:32.584681+0000","last_became_active":"2024-04-25T10:03:30.357983+0000","last_became_peered":"2024-04-25T10:03:30.357983+0000","last_unstale":"2024-04-25T10:03:32.584681+0000","last_undegraded":"2024-04-25T10:03:32.584681+0000","last_fullsized":"2024-04-25T10:03:32.584681+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:47.546452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334916+0000","last_change":"2024-04-25T10:03:31.368414+0000","last_active":"2024-04-25T10:03:32.334916+0000","last_peered":"2024-04-25T10:03:32.334916+0000","last_clean":"2024-04-25T10:03:32.334916+0000","last_became_active":"2024-04-25T10:03:31.364462+0000","last_became_peered":"2024-04-25T10:03:31.364462+0000","last_unstale":"2024-04-25T10:03:32.334916+0000","last_undegraded":"2024-04-25T10:03:32.334916+0000","last_fullsized":"2024-04-25T10:03:32.334916+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:26.194936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363518+0000","last_change":"2024-04-25T10:03:30.364424+0000","last_active":"2024-04-25T10:03:32.363518+0000","last_peered":"2024-04-25T10:03:32.363518+0000","last_clean":"2024-04-25T10:03:32.363518+0000","last_became_active":"2024-04-25T10:03:30.363258+0000","last_became_peered":"2024-04-25T10:03:30.363258+0000","last_unstale":"2024-04-25T10:03:32.363518+0000","last_undegraded":"2024-04-25T10:03:32.363518+0000","last_fullsized":"2024-04-25T10:03:32.363518+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:18.400867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363293+0000","last_change":"2024-04-25T10:03:31.370216+0000","last_active":"2024-04-25T10:03:32.363293+0000","last_peered":"2024-04-25T10:03:32.363293+0000","last_clean":"2024-04-25T10:03:32.363293+0000","last_became_active":"2024-04-25T10:03:31.367905+0000","last_became_peered":"2024-04-25T10:03:31.367905+0000","last_unstale":"2024-04-25T10:03:32.363293+0000","last_undegraded":"2024-04-25T10:03:32.363293+0000","last_fullsized":"2024-04-25T10:03:32.363293+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:23.768194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.337188+0000","last_change":"2024-04-25T10:03:30.355967+0000","last_active":"2024-04-25T10:03:32.337188+0000","last_peered":"2024-04-25T10:03:32.337188+0000","last_clean":"2024-04-25T10:03:32.337188+0000","last_became_active":"2024-04-25T10:03:30.354860+0000","last_became_peered":"2024-04-25T10:03:30.354860+0000","last_unstale":"2024-04-25T10:03:32.337188+0000","last_undegraded":"2024-04-25T10:03:32.337188+0000","last_fullsized":"2024-04-25T10:03:32.337188+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336981+0000","last_change":"2024-04-25T10:03:31.365262+0000","last_active":"2024-04-25T10:03:32.336981+0000","last_peered":"2024-04-25T10:03:32.336981+0000","last_clean":"2024-04-25T10:03:32.336981+0000","last_became_active":"2024-04-25T10:03:31.360300+0000","last_became_peered":"2024-04-25T10:03:31.360300+0000","last_unstale":"2024-04-25T10:03:32.336981+0000","last_undegraded":"2024-04-25T10:03:32.336981+0000","last_fullsized":"2024-04-25T10:03:32.336981+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359994+0000","last_change":"2024-04-25T10:03:30.343858+0000","last_active":"2024-04-25T10:03:32.359994+0000","last_peered":"2024-04-25T10:03:32.359994+0000","last_clean":"2024-04-25T10:03:32.359994+0000","last_became_active":"2024-04-25T10:03:30.343569+0000","last_became_peered":"2024-04-25T10:03:30.343569+0000","last_unstale":"2024-04-25T10:03:32.359994+0000","last_undegraded":"2024-04-25T10:03:32.359994+0000","last_fullsized":"2024-04-25T10:03:32.359994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:06.585096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360526+0000","last_change":"2024-04-25T10:03:31.353858+0000","last_active":"2024-04-25T10:03:32.360526+0000","last_peered":"2024-04-25T10:03:32.360526+0000","last_clean":"2024-04-25T10:03:32.360526+0000","last_became_active":"2024-04-25T10:03:31.353497+0000","last_became_peered":"2024-04-25T10:03:31.353497+0000","last_unstale":"2024-04-25T10:03:32.360526+0000","last_undegraded":"2024-04-25T10:03:32.360526+0000","last_fullsized":"2024-04-25T10:03:32.360526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:50.030249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181485+0000","last_change":"2024-04-25T10:03:30.357045+0000","last_active":"2024-04-25T10:03:33.181485+0000","last_peered":"2024-04-25T10:03:33.181485+0000","last_clean":"2024-04-25T10:03:33.181485+0000","last_became_active":"2024-04-25T10:03:30.356262+0000","last_became_peered":"2024-04-25T10:03:30.356262+0000","last_unstale":"2024-04-25T10:03:33.181485+0000","last_undegraded":"2024-04-25T10:03:33.181485+0000","last_fullsized":"2024-04-25T10:03:33.181485+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:02.640058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585899+0000","last_change":"2024-04-25T10:03:31.387911+0000","last_active":"2024-04-25T10:03:32.585899+0000","last_peered":"2024-04-25T10:03:32.585899+0000","last_clean":"2024-04-25T10:03:32.585899+0000","last_became_active":"2024-04-25T10:03:31.387394+0000","last_became_peered":"2024-04-25T10:03:31.387394+0000","last_unstale":"2024-04-25T10:03:32.585899+0000","last_undegraded":"2024-04-25T10:03:32.585899+0000","last_fullsized":"2024-04-25T10:03:32.585899+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:37.060632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359830+0000","last_change":"2024-04-25T10:03:30.344668+0000","last_active":"2024-04-25T10:03:32.359830+0000","last_peered":"2024-04-25T10:03:32.359830+0000","last_clean":"2024-04-25T10:03:32.359830+0000","last_became_active":"2024-04-25T10:03:30.344302+0000","last_became_peered":"2024-04-25T10:03:30.344302+0000","last_unstale":"2024-04-25T10:03:32.359830+0000","last_undegraded":"2024-04-25T10:03:32.359830+0000","last_fullsized":"2024-04-25T10:03:32.359830+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:35.987811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180325+0000","last_change":"2024-04-25T10:03:31.365872+0000","last_active":"2024-04-25T10:03:33.180325+0000","last_peered":"2024-04-25T10:03:33.180325+0000","last_clean":"2024-04-25T10:03:33.180325+0000","last_became_active":"2024-04-25T10:03:31.365380+0000","last_became_peered":"2024-04-25T10:03:31.365380+0000","last_unstale":"2024-04-25T10:03:33.180325+0000","last_undegraded":"2024-04-25T10:03:33.180325+0000","last_fullsized":"2024-04-25T10:03:33.180325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:17.561800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359837+0000","last_change":"2024-04-25T10:03:30.356681+0000","last_active":"2024-04-25T10:03:32.359837+0000","last_peered":"2024-04-25T10:03:32.359837+0000","last_clean":"2024-04-25T10:03:32.359837+0000","last_became_active":"2024-04-25T10:03:30.356008+0000","last_became_peered":"2024-04-25T10:03:30.356008+0000","last_unstale":"2024-04-25T10:03:32.359837+0000","last_undegraded":"2024-04-25T10:03:32.359837+0000","last_fullsized":"2024-04-25T10:03:32.359837+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:45.509971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.362465+0000","last_change":"2024-04-25T10:03:31.352703+0000","last_active":"2024-04-25T10:03:32.362465+0000","last_peered":"2024-04-25T10:03:32.362465+0000","last_clean":"2024-04-25T10:03:32.362465+0000","last_became_active":"2024-04-25T10:03:31.351963+0000","last_became_peered":"2024-04-25T10:03:31.351963+0000","last_unstale":"2024-04-25T10:03:32.362465+0000","last_undegraded":"2024-04-25T10:03:32.362465+0000","last_fullsized":"2024-04-25T10:03:32.362465+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:26.194936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585250+0000","last_change":"2024-04-25T10:03:30.359453+0000","last_active":"2024-04-25T10:03:32.585250+0000","last_peered":"2024-04-25T10:03:32.585250+0000","last_clean":"2024-04-25T10:03:32.585250+0000","last_became_active":"2024-04-25T10:03:30.358890+0000","last_became_peered":"2024-04-25T10:03:30.358890+0000","last_unstale":"2024-04-25T10:03:32.585250+0000","last_undegraded":"2024-04-25T10:03:32.585250+0000","last_fullsized":"2024-04-25T10:03:32.585250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:24.219292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334266+0000","last_change":"2024-04-25T10:03:31.381014+0000","last_active":"2024-04-25T10:03:32.334266+0000","last_peered":"2024-04-25T10:03:32.334266+0000","last_clean":"2024-04-25T10:03:32.334266+0000","last_became_active":"2024-04-25T10:03:31.380759+0000","last_became_peered":"2024-04-25T10:03:31.380759+0000","last_unstale":"2024-04-25T10:03:32.334266+0000","last_undegraded":"2024-04-25T10:03:32.334266+0000","last_fullsized":"2024-04-25T10:03:32.334266+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:45.405473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.359808+0000","last_change":"2024-04-25T10:03:30.344451+0000","last_active":"2024-04-25T10:03:32.359808+0000","last_peered":"2024-04-25T10:03:32.359808+0000","last_clean":"2024-04-25T10:03:32.359808+0000","last_became_active":"2024-04-25T10:03:30.344119+0000","last_became_peered":"2024-04-25T10:03:30.344119+0000","last_unstale":"2024-04-25T10:03:32.359808+0000","last_undegraded":"2024-04-25T10:03:32.359808+0000","last_fullsized":"2024-04-25T10:03:32.359808+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:29.424997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181690+0000","last_change":"2024-04-25T10:03:31.359583+0000","last_active":"2024-04-25T10:03:33.181690+0000","last_peered":"2024-04-25T10:03:33.181690+0000","last_clean":"2024-04-25T10:03:33.181690+0000","last_became_active":"2024-04-25T10:03:31.359237+0000","last_became_peered":"2024-04-25T10:03:31.359237+0000","last_unstale":"2024-04-25T10:03:33.181690+0000","last_undegraded":"2024-04-25T10:03:33.181690+0000","last_fullsized":"2024-04-25T10:03:33.181690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:21.037345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363806+0000","last_change":"2024-04-25T10:03:30.358809+0000","last_active":"2024-04-25T10:03:32.363806+0000","last_peered":"2024-04-25T10:03:32.363806+0000","last_clean":"2024-04-25T10:03:32.363806+0000","last_became_active":"2024-04-25T10:03:30.358091+0000","last_became_peered":"2024-04-25T10:03:30.358091+0000","last_unstale":"2024-04-25T10:03:32.363806+0000","last_undegraded":"2024-04-25T10:03:32.363806+0000","last_fullsized":"2024-04-25T10:03:32.363806+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.533743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585457+0000","last_change":"2024-04-25T10:03:31.388203+0000","last_active":"2024-04-25T10:03:32.585457+0000","last_peered":"2024-04-25T10:03:32.585457+0000","last_clean":"2024-04-25T10:03:32.585457+0000","last_became_active":"2024-04-25T10:03:31.387559+0000","last_became_peered":"2024-04-25T10:03:31.387559+0000","last_unstale":"2024-04-25T10:03:32.585457+0000","last_undegraded":"2024-04-25T10:03:32.585457+0000","last_fullsized":"2024-04-25T10:03:32.585457+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:20.844892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.360128+0000","last_change":"2024-04-25T10:03:30.356327+0000","last_active":"2024-04-25T10:03:32.360128+0000","last_peered":"2024-04-25T10:03:32.360128+0000","last_clean":"2024-04-25T10:03:32.360128+0000","last_became_active":"2024-04-25T10:03:30.355964+0000","last_became_peered":"2024-04-25T10:03:30.355964+0000","last_unstale":"2024-04-25T10:03:32.360128+0000","last_undegraded":"2024-04-25T10:03:32.360128+0000","last_fullsized":"2024-04-25T10:03:32.360128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:00.451048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334996+0000","last_change":"2024-04-25T10:03:31.368328+0000","last_active":"2024-04-25T10:03:32.334996+0000","last_peered":"2024-04-25T10:03:32.334996+0000","last_clean":"2024-04-25T10:03:32.334996+0000","last_became_active":"2024-04-25T10:03:31.367800+0000","last_became_peered":"2024-04-25T10:03:31.367800+0000","last_unstale":"2024-04-25T10:03:32.334996+0000","last_undegraded":"2024-04-25T10:03:32.334996+0000","last_fullsized":"2024-04-25T10:03:32.334996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:43.548567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.181133+0000","last_change":"2024-04-25T10:03:30.341416+0000","last_active":"2024-04-25T10:03:33.181133+0000","last_peered":"2024-04-25T10:03:33.181133+0000","last_clean":"2024-04-25T10:03:33.181133+0000","last_became_active":"2024-04-25T10:03:30.341007+0000","last_became_peered":"2024-04-25T10:03:30.341007+0000","last_unstale":"2024-04-25T10:03:33.181133+0000","last_undegraded":"2024-04-25T10:03:33.181133+0000","last_fullsized":"2024-04-25T10:03:33.181133+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:50.584857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180454+0000","last_change":"2024-04-25T10:03:31.365910+0000","last_active":"2024-04-25T10:03:33.180454+0000","last_peered":"2024-04-25T10:03:33.180454+0000","last_clean":"2024-04-25T10:03:33.180454+0000","last_became_active":"2024-04-25T10:03:31.365532+0000","last_became_peered":"2024-04-25T10:03:31.365532+0000","last_unstale":"2024-04-25T10:03:33.180454+0000","last_undegraded":"2024-04-25T10:03:33.180454+0000","last_fullsized":"2024-04-25T10:03:33.180454+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.405550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.180640+0000","last_change":"2024-04-25T10:03:30.357850+0000","last_active":"2024-04-25T10:03:33.180640+0000","last_peered":"2024-04-25T10:03:33.180640+0000","last_clean":"2024-04-25T10:03:33.180640+0000","last_became_active":"2024-04-25T10:03:30.357332+0000","last_became_peered":"2024-04-25T10:03:30.357332+0000","last_unstale":"2024-04-25T10:03:33.180640+0000","last_undegraded":"2024-04-25T10:03:33.180640+0000","last_fullsized":"2024-04-25T10:03:33.180640+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:08.615248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:33.182001+0000","last_change":"2024-04-25T10:03:31.359781+0000","last_active":"2024-04-25T10:03:33.182001+0000","last_peered":"2024-04-25T10:03:33.182001+0000","last_clean":"2024-04-25T10:03:33.182001+0000","last_became_active":"2024-04-25T10:03:31.359433+0000","last_became_peered":"2024-04-25T10:03:31.359433+0000","last_unstale":"2024-04-25T10:03:33.182001+0000","last_undegraded":"2024-04-25T10:03:33.182001+0000","last_fullsized":"2024-04-25T10:03:33.182001+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:06.405550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334251+0000","last_change":"2024-04-25T10:03:30.344524+0000","last_active":"2024-04-25T10:03:32.334251+0000","last_peered":"2024-04-25T10:03:32.334251+0000","last_clean":"2024-04-25T10:03:32.334251+0000","last_became_active":"2024-04-25T10:03:30.343370+0000","last_became_peered":"2024-04-25T10:03:30.343370+0000","last_unstale":"2024-04-25T10:03:32.334251+0000","last_undegraded":"2024-04-25T10:03:32.334251+0000","last_fullsized":"2024-04-25T10:03:32.334251+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:14.456470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.336373+0000","last_change":"2024-04-25T10:03:31.369390+0000","last_active":"2024-04-25T10:03:32.336373+0000","last_peered":"2024-04-25T10:03:32.336373+0000","last_clean":"2024-04-25T10:03:32.336373+0000","last_became_active":"2024-04-25T10:03:31.368682+0000","last_became_peered":"2024-04-25T10:03:31.368682+0000","last_unstale":"2024-04-25T10:03:32.336373+0000","last_undegraded":"2024-04-25T10:03:32.336373+0000","last_fullsized":"2024-04-25T10:03:32.336373+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:04.546401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.335919+0000","last_change":"2024-04-25T10:03:30.355381+0000","last_active":"2024-04-25T10:03:32.335919+0000","last_peered":"2024-04-25T10:03:32.335919+0000","last_clean":"2024-04-25T10:03:32.335919+0000","last_became_active":"2024-04-25T10:03:30.354430+0000","last_became_peered":"2024-04-25T10:03:30.354430+0000","last_unstale":"2024-04-25T10:03:32.335919+0000","last_undegraded":"2024-04-25T10:03:32.335919+0000","last_fullsized":"2024-04-25T10:03:32.335919+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:56.280508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585962+0000","last_change":"2024-04-25T10:03:31.388793+0000","last_active":"2024-04-25T10:03:32.585962+0000","last_peered":"2024-04-25T10:03:32.585962+0000","last_clean":"2024-04-25T10:03:32.585962+0000","last_became_active":"2024-04-25T10:03:31.387890+0000","last_became_peered":"2024-04-25T10:03:31.387890+0000","last_unstale":"2024-04-25T10:03:32.585962+0000","last_undegraded":"2024-04-25T10:03:32.585962+0000","last_fullsized":"2024-04-25T10:03:32.585962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:16.562290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.334040+0000","last_change":"2024-04-25T10:03:30.340638+0000","last_active":"2024-04-25T10:03:32.334040+0000","last_peered":"2024-04-25T10:03:32.334040+0000","last_clean":"2024-04-25T10:03:32.334040+0000","last_became_active":"2024-04-25T10:03:30.340365+0000","last_became_peered":"2024-04-25T10:03:30.340365+0000","last_unstale":"2024-04-25T10:03:32.334040+0000","last_undegraded":"2024-04-25T10:03:32.334040+0000","last_fullsized":"2024-04-25T10:03:32.334040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:03.533743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.363526+0000","last_change":"2024-04-25T10:03:31.360025+0000","last_active":"2024-04-25T10:03:32.363526+0000","last_peered":"2024-04-25T10:03:32.363526+0000","last_clean":"2024-04-25T10:03:32.363526+0000","last_became_active":"2024-04-25T10:03:31.354209+0000","last_became_peered":"2024-04-25T10:03:31.354209+0000","last_unstale":"2024-04-25T10:03:32.363526+0000","last_undegraded":"2024-04-25T10:03:32.363526+0000","last_fullsized":"2024-04-25T10:03:32.363526+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:30.306869+0000","last_clean_scrub_stamp":"2024-04-25T10:03:30.306869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:59.932672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585700+0000","last_change":"2024-04-25T10:03:30.359055+0000","last_active":"2024-04-25T10:03:32.585700+0000","last_peered":"2024-04-25T10:03:32.585700+0000","last_clean":"2024-04-25T10:03:32.585700+0000","last_became_active":"2024-04-25T10:03:30.358297+0000","last_became_peered":"2024-04-25T10:03:30.358297+0000","last_unstale":"2024-04-25T10:03:32.585700+0000","last_undegraded":"2024-04-25T10:03:32.585700+0000","last_fullsized":"2024-04-25T10:03:32.585700+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:29.294211+0000","last_clean_scrub_stamp":"2024-04-25T10:03:29.294211+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:25.729718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":204,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:32.585114+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:03:32.585114+0000","last_peered":"2024-04-25T10:03:32.585114+0000","last_clean":"2024-04-25T10:03:32.585114+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:03:32.585114+0000","last_undegraded":"2024-04-25T10:03:32.585114+0000","last_fullsized":"2024-04-25T10:03:32.585114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1060864,"data_stored":786911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":194973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":458752,"data_stored":195465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":466944,"data_stored":196507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":194931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":194951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":785263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:35.689 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:35.689 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:35.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 Processor -- start 2024-04-25T10:03:35.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 -- start start 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91e015ecb0 0x7f91e00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e00ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e00ca4c0 con 0x7f91e00cf590 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e00ca780 con 0x7f91e015ecb0 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e7912640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e015f0f0 con 0x7f91e00c9ef0 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91e015ecb0 0x7f91e00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91e015ecb0 0x7f91e00c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42140/0 (socket says 172.21.15.82:42140) 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 -- 172.21.15.82:0/4255389748 learned_addr learned my addr 172.21.15.82:0/4255389748 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e5eaf640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e00ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 -- 172.21.15.82:0/4255389748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 msgr2=0x7f91e00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e00ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 -- 172.21.15.82:0/4255389748 >> v1:172.21.15.196:6789/0 conn(0x7f91e00cf590 legacy=0x7f91e00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.847+0000 7f91e4ead640 1 -- 172.21.15.82:0/4255389748 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e00cd690 con 0x7f91e015ecb0 2024-04-25T10:03:35.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e4ead640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e015ecb0 0x7f91e00c99b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009870 tx=0x7f91d001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244708e25614dfd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91d7fff640 1 -- 172.21.15.82:0/4255389748 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00a7070 con 0x7f91e015ecb0 2024-04-25T10:03:35.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91d7fff640 1 -- 172.21.15.82:0/4255389748 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00b0070 con 0x7f91e015ecb0 2024-04-25T10:03:35.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91d7fff640 1 -- 172.21.15.82:0/4255389748 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00ab890 con 0x7f91e015ecb0 2024-04-25T10:03:35.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 -- 172.21.15.82:0/4255389748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e015ecb0 msgr2=0x7f91e00c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e015ecb0 0x7f91e00c99b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009870 tx=0x7f91d001c560 comp rx=0 tx=0).stop 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 -- 172.21.15.82:0/4255389748 shutdown_connections 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e00ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 --2- 172.21.15.82:0/4255389748 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e015ecb0 0x7f91e00c99b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 -- 172.21.15.82:0/4255389748 >> 172.21.15.82:0/4255389748 conn(0x7f91e006ae60 msgr2=0x7f91e006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 -- 172.21.15.82:0/4255389748 shutdown_connections 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.851+0000 7f91e7912640 1 -- 172.21.15.82:0/4255389748 wait complete. 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 Processor -- start 2024-04-25T10:03:35.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- start start 2024-04-25T10:03:35.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e01570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91e015ecb0 0x7f91e00989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e01676f0 con 0x7f91e00cf590 2024-04-25T10:03:35.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e00cdce0 con 0x7f91e015ecb0 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0167870 con 0x7f91e00c9ef0 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42148/0 (socket says 172.21.15.82:42148) 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 -- 172.21.15.82:0/2724945061 learned_addr learned my addr 172.21.15.82:0/2724945061 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 -- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 msgr2=0x7f91e01570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e01570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 -- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91e015ecb0 msgr2=0x7f91e00989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91e015ecb0 0x7f91e00989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 -- 172.21.15.82:0/2724945061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e0098ef0 con 0x7f91e00cf590 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e4ead640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f91d001ac10 tx=0x7f91d0019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00bf610 con 0x7f91e00cf590 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d00bf7d0 con 0x7f91e00cf590 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e00c5b70 con 0x7f91e00cf590 2024-04-25T10:03:35.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d00bfbc0 con 0x7f91e00cf590 2024-04-25T10:03:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0072440 con 0x7f91e00cf590 2024-04-25T10:03:35.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e004efa0 con 0x7f91e00cf590 2024-04-25T10:03:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91d00bd020 con 0x7f91e00cf590 2024-04-25T10:03:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.855+0000 7f91d5ffb640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 0x7f91c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.859+0000 7f91e56ae640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 0x7f91c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.859+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91d011fb40 con 0x7f91e00cf590 2024-04-25T10:03:35.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.859+0000 7f91e56ae640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 0x7f91c4069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91dc000f70 tx=0x7f91dc0079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:35.863+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91d0016150 con 0x7f91e00cf590 2024-04-25T10:03:36.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.059+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f91e00b4170 con 0x7f91e00cf590 2024-04-25T10:03:36.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.063+0000 7f91d5ffb640 1 -- 172.21.15.82:0/2724945061 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f91d00f0ae0 con 0x7f91e00cf590 2024-04-25T10:03:36.068 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:03:36.068 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:03:36.068 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:03:36.068 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:03:36.069 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:36.069 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:03:36.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 msgr2=0x7f91c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 0x7f91c4069450 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f91dc000f70 tx=0x7f91dc0079e0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 msgr2=0x7f91e00c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f91d001ac10 tx=0x7f91d0019300 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 shutdown_connections 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91c4066f90 0x7f91c4069450 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91e015ecb0 0x7f91e00989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91e00cf590 0x7f91e00c54e0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 --2- 172.21.15.82:0/2724945061 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91e00c9ef0 0x7f91e01570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 >> 172.21.15.82:0/2724945061 conn(0x7f91e006ae60 msgr2=0x7f91e006e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 shutdown_connections 2024-04-25T10:03:36.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.067+0000 7f91e7912640 1 -- 172.21.15.82:0/2724945061 wait complete. 2024-04-25T10:03:36.086 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:36.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 Processor -- start 2024-04-25T10:03:36.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 -- start start 2024-04-25T10:03:36.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff90415dd50 0x7ff90406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90406c5f0 con 0x7ff9040af390 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90415e2e0 con 0x7ff90406c1e0 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90bae1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9040b2f10 con 0x7ff90415dd50 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff9040af390 0x7ff904066cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54780/0 (socket says 172.21.15.82:54780) 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 -- 172.21.15.82:0/4119648160 learned_addr learned my addr 172.21.15.82:0/4119648160 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:36.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90907c640 1 --2- 172.21.15.82:0/4119648160 >> v2:172.21.15.196:3300/0 conn(0x7ff90415dd50 0x7ff90406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059571781 0 0) 0x7ff8f4001120 con 0x7ff9040af390 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8f0003530 con 0x7ff9040af390 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 47379125 0 0) 0x7ff8f0003530 con 0x7ff9040af390 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 >> v2:172.21.15.196:3300/0 conn(0x7ff90415dd50 msgr2=0x7ff90406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 --2- 172.21.15.82:0/4119648160 >> v2:172.21.15.196:3300/0 conn(0x7ff90415dd50 0x7ff90406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 >> v1:172.21.15.196:6789/0 conn(0x7ff90406c1e0 legacy=0x7ff90406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90415e5c0 con 0x7ff9040af390 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff8f4001850 con 0x7ff9040af390 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 >> v1:172.21.15.82:6789/0 conn(0x7ff9040af390 legacy=0x7ff904066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff8f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f0004810 0x7ff8f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8f00052d0 0x7ff8f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f000c1e0 con 0x7ff8f0003a30 2024-04-25T10:03:36.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f000c390 con 0x7ff8f0004810 2024-04-25T10:03:36.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f000c590 con 0x7ff8f00052d0 2024-04-25T10:03:36.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff8f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90907c640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f0004810 0x7ff8f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90a07e640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8f00052d0 0x7ff8f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 -- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8f00052d0 msgr2=0x7ff8f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8f00052d0 0x7ff8f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 -- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f0004810 msgr2=0x7ff8f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8f0004810 0x7ff8f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90987d640 1 -- 172.21.15.82:0/4119648160 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0003530 con 0x7ff8f0003a30 2024-04-25T10:03:36.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90a07e640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8f00052d0 0x7ff8f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.191+0000 7ff90a07e640 1 -- 172.21.15.82:0/4119648160 reap_dead start 2024-04-25T10:03:36.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90987d640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff8f0003e50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c4c0 tx=0x7ff8f401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d446a183837677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f400c0e0 con 0x7ff8f0003a30 2024-04-25T10:03:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40a9070 con 0x7ff8f0003a30 2024-04-25T10:03:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90887b640 1 -- 172.21.15.82:0/4119648160 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4005d80 con 0x7ff8f0003a30 2024-04-25T10:03:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 msgr2=0x7ff8f0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff8f0003e50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff8f400c4c0 tx=0x7ff8f401cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:36.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- 172.21.15.82:0/4119648160 shutdown_connections 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 --2- 172.21.15.82:0/4119648160 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff8f0003e50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- 172.21.15.82:0/4119648160 >> 172.21.15.82:0/4119648160 conn(0x7ff9040717f0 msgr2=0x7ff904071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- 172.21.15.82:0/4119648160 shutdown_connections 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- 172.21.15.82:0/4119648160 wait complete. 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 Processor -- start 2024-04-25T10:03:36.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.195+0000 7ff90bae1640 1 -- start start 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 0x7ff9041e95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9040afeb0 0x7ff9040b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff904161d30 con 0x7ff8f0003a30 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff904161bb0 con 0x7ff904151b00 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff904161eb0 con 0x7ff9040afeb0 2024-04-25T10:03:36.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42164/0 (socket says 172.21.15.82:42164) 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 -- 172.21.15.82:0/2003378826 learned_addr learned my addr 172.21.15.82:0/2003378826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90907c640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 0x7ff9041e95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 -- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9040afeb0 msgr2=0x7ff9040b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9040afeb0 0x7ff9040b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 -- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 msgr2=0x7ff9041e95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 0x7ff9041e95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 -- 172.21.15.82:0/2003378826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9041520c0 con 0x7ff8f0003a30 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90907c640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 0x7ff9041e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90987d640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff8f401c6f0 tx=0x7ff8f0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4001650 con 0x7ff8f0003a30 2024-04-25T10:03:36.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f40017f0 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9041523a0 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4007380 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff904152960 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90404efa0 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8f4007520 con 0x7ff8f0003a30 2024-04-25T10:03:36.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8f411e5e0 con 0x7ff8f0003a30 2024-04-25T10:03:36.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90907c640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.199+0000 7ff90907c640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9041ea020 tx=0x7ff900008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.203+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff90406c5f0 con 0x7ff8f0003a30 2024-04-25T10:03:36.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:36.219 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:36.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9040ae450 con 0x7ff8f0003a30 2024-04-25T10:03:36.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff8fa7fc640 1 -- 172.21.15.82:0/2003378826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7ff8f40ef3d0 con 0x7ff8f0003a30 2024-04-25T10:03:36.399 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:36.399 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:36.399 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 msgr2=0x7ff8e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 0x7ff8e8069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff9041ea020 tx=0x7ff900008040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 msgr2=0x7ff904142880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff8f401c6f0 tx=0x7ff8f0005840 comp rx=0 tx=0).stop 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 shutdown_connections 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8e8066f60 0x7ff8e8069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9040afeb0 0x7ff9040b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff904151b00 0x7ff9041e95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 --2- 172.21.15.82:0/2003378826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8f0003a30 0x7ff904142880 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 >> 172.21.15.82:0/2003378826 conn(0x7ff9040717f0 msgr2=0x7ff90415eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 shutdown_connections 2024-04-25T10:03:36.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.395+0000 7ff90bae1640 1 -- 172.21.15.82:0/2003378826 wait complete. 2024-04-25T10:03:36.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:30.679984+0000', 'modified': '2024-04-25T10:03:33.721743+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5439}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5439': {'gid': 5439, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/560740519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 560740519}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 560740519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:36.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:36.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:03:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 Processor -- start 2024-04-25T10:03:36.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 -- start start 2024-04-25T10:03:36.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c80ce510 0x7f29c80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f29c8144ca0 0x7f29c806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f29c806ec60 0x7f29c8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c80cc2b0 con 0x7f29c806ec60 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c806e290 con 0x7f29c8144ca0 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29cccff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80cee30 con 0x7f29c80ce510 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f29c806ec60 0x7f29c8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f29c806ec60 0x7f29c8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42172/0 (socket says 172.21.15.82:42172) 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 -- 172.21.15.82:0/308170376 learned_addr learned my addr 172.21.15.82:0/308170376 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:36.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29bffff640 1 --2- 172.21.15.82:0/308170376 >> v2:172.21.15.196:3300/0 conn(0x7f29c8144ca0 0x7f29c806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c659c640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c80ce510 0x7f29c80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 -- 172.21.15.82:0/308170376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c80ce510 msgr2=0x7f29c80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c80ce510 0x7f29c80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 -- 172.21.15.82:0/308170376 >> v2:172.21.15.196:3300/0 conn(0x7f29c8144ca0 msgr2=0x7f29c806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/308170376 >> v2:172.21.15.196:3300/0 conn(0x7f29c8144ca0 0x7f29c806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.947+0000 7f29c6d9d640 1 -- 172.21.15.82:0/308170376 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c80ccc70 con 0x7f29c806ec60 2024-04-25T10:03:36.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c806ec60 0x7f29c8142680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29c806cfe0 tx=0x7f29c001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886a5507d841c223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29c5d9b640 1 -- 172.21.15.82:0/308170376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00b4070 con 0x7f29c806ec60 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29c5d9b640 1 -- 172.21.15.82:0/308170376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c00af400 con 0x7f29c806ec60 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29c5d9b640 1 -- 172.21.15.82:0/308170376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c00af7d0 con 0x7f29c806ec60 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- 172.21.15.82:0/308170376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c806ec60 msgr2=0x7f29c8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c806ec60 0x7f29c8142680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29c806cfe0 tx=0x7f29c001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- 172.21.15.82:0/308170376 shutdown_connections 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c806ec60 0x7f29c8142680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 --2- 172.21.15.82:0/308170376 >> v2:172.21.15.196:3300/0 conn(0x7f29c8144ca0 0x7f29c806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 --2- 172.21.15.82:0/308170376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c80ce510 0x7f29c80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- 172.21.15.82:0/308170376 >> 172.21.15.82:0/308170376 conn(0x7f29c80b11a0 msgr2=0x7f29c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- 172.21.15.82:0/308170376 shutdown_connections 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- 172.21.15.82:0/308170376 wait complete. 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 Processor -- start 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.951+0000 7f29cccff640 1 -- start start 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29c806ec60 0x7f29c8134be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c8144ca0 0x7f29c81e61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c80cc900 con 0x7f29c80ce510 2024-04-25T10:03:36.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c806df20 con 0x7f29c806ec60 2024-04-25T10:03:36.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81451f0 con 0x7f29c8144ca0 2024-04-25T10:03:36.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42182/0 (socket says 172.21.15.82:42182) 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 -- 172.21.15.82:0/3109549111 learned_addr learned my addr 172.21.15.82:0/3109549111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29c659c640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29c806ec60 0x7f29c8134be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c8144ca0 0x7f29c81e61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 -- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c8144ca0 msgr2=0x7f29c81e61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c8144ca0 0x7f29c81e61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 -- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29c806ec60 msgr2=0x7f29c8134be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29c806ec60 0x7f29c8134be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 -- 172.21.15.82:0/3109549111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c81e2420 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29bffff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d5d0 tx=0x7f29b000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 -- 172.21.15.82:0/3109549111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0004240 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 -- 172.21.15.82:0/3109549111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b00043e0 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c81e2700 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 -- 172.21.15.82:0/3109549111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0003840 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c81e38e0 con 0x7f29c80ce510 2024-04-25T10:03:36.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 -- 172.21.15.82:0/3109549111 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29b00039e0 con 0x7f29c80ce510 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29c659c640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.955+0000 7f29beffd640 1 -- 172.21.15.82:0/3109549111 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b0085520 con 0x7f29c80ce510 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29c659c640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 0x7f29a8069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f29b800ba30 tx=0x7f29b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 Processor -- start 2024-04-25T10:03:36.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- start start 2024-04-25T10:03:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0cffd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0cb6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0b9d5f0 con 0x5646b0d00240 2024-04-25T10:03:36.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0b04da0 con 0x5646b0cff920 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646b0a8d150 con 0x5646b0cb5f30 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42190/0 (socket says 172.21.15.82:42190) 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 -- 172.21.15.82:0/490439384 learned_addr learned my addr 172.21.15.82:0/490439384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29c659c640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0cffd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0cb6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 -- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 msgr2=0x5646b0cb6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0cb6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 -- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 msgr2=0x5646b0cffd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0cffd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 -- 172.21.15.82:0/490439384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646b0aa6ca0 con 0x5646b0d00240 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29c659c640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0cffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0cb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29bffff640 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f29b0083790 tx=0x7f29b0055b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b364809e4c8a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29a7fff640 1 -- 172.21.15.82:0/490439384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0055c00 con 0x5646b0d00240 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29a7fff640 1 -- 172.21.15.82:0/490439384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b007e020 con 0x5646b0d00240 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29a7fff640 1 -- 172.21.15.82:0/490439384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0059d50 con 0x5646b0d00240 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- 172.21.15.82:0/490439384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 msgr2=0x5646b0cb59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f29b0083790 tx=0x7f29b0055b00 comp rx=0 tx=0).stop 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- 172.21.15.82:0/490439384 shutdown_connections 2024-04-25T10:03:36.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0cb6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0d00240 0x5646b0cb59f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/490439384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0cffd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- 172.21.15.82:0/490439384 >> 172.21.15.82:0/490439384 conn(0x5646b0cbc2a0 msgr2=0x5646b0cb42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- 172.21.15.82:0/490439384 shutdown_connections 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- 172.21.15.82:0/490439384 wait complete. 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 -- start start 2024-04-25T10:03:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.959+0000 7f29cfe6c000 1 Processor -- start 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0d00240 0x5646b0cb9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0abf990 con 0x5646b0cb5f30 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0b00e40 con 0x5646b0d00240 2024-04-25T10:03:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646b0a90900 con 0x5646b0cff920 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 10 client.0 _ll_register_callbacks cb 0x5646b0bb81c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42196/0 (socket says 172.21.15.82:42196) 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 -- 172.21.15.82:0/2595020113 learned_addr learned my addr 172.21.15.82:0/2595020113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29bffff640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c6d9d640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0d00240 0x5646b0cb9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 -- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 msgr2=0x5646b0d071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d071c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 -- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0d00240 msgr2=0x5646b0cb9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0d00240 0x5646b0cb9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 -- 172.21.15.82:0/2595020113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646b0abd630 con 0x5646b0cb5f30 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29bffff640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29c659c640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f29b80146e0 tx=0x7f29b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea2fdb2a2f3118b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8015a20 con 0x5646b0cb5f30 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b8015bc0 con 0x5646b0cb5f30 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b007e020 con 0x5646b0cb5f30 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b802e050 con 0x5646b0cb5f30 2024-04-25T10:03:36.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 10 client.5625 fetch_fsmap learned FSMap version 17 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5646b0a8d3a0 con 0x5646b0cb5f30 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b007e020 con 0x5646b0cb5f30 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29b8031020 con 0x5646b0cb5f30 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 10 client.5625 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 10 client.5625 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 20 client.5625 populate_metadata read hostname 'smithi082' 2024-04-25T10:03:36.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0cbb160 0x5646b0d10ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 4 client.5625 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 --> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646b0aa0a50 con 0x5646b0cbb160 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29bffff640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0cbb160 0x5646b0d10ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 10 client.5625 ms_handle_connect on v2:172.21.15.82:6836/560740519 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29bffff640 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0cbb160 0x5646b0d10ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=127186efecde25f2 server_cookie=f3dc3e41b189915b in_seq=0 out_seq=0 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 1 -- 172.21.15.82:0/2595020113 <== mds.0 v2:172.21.15.82:6836/560740519 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5646b0aa0a50 con 0x5646b0cbb160 2024-04-25T10:03:36.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 10 client.5625 handle_command_reply: tid=0 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 client.5625 shutdown 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 msgr2=0x5646b0d06c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f29b80146e0 tx=0x7f29b8004740 comp rx=0 tx=0).stop 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 shutdown_connections 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0cbb160 0x5646b0d10ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0d00240 0x5646b0cb9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29a57fa640 0 client.5625 ms_handle_reset on v2:172.21.15.82:6836/560740519 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2595020113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cb5f30 0x5646b0d06c80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 >> 172.21.15.82:0/2595020113 conn(0x5646b0cbc2a0 msgr2=0x5646b0cb49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 shutdown_connections 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2595020113 wait complete. 2024-04-25T10:03:36.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.963+0000 7f29cfe6c000 20 client.5625 trim_cache size 0 max 16384 2024-04-25T10:03:37.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 Processor -- start 2024-04-25T10:03:37.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 -- start start 2024-04-25T10:03:37.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cb5f30 0x5646b0d5b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d5be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81fd160 con 0x5646b0cbb160 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81e4be0 con 0x5646b0cb5f30 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c81fcc80 con 0x5646b0cff920 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42202/0 (socket says 172.21.15.82:42202) 2024-04-25T10:03:37.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 -- 172.21.15.82:0/2736534108 learned_addr learned my addr 172.21.15.82:0/2736534108 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 -- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 msgr2=0x5646b0d5be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d5be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 -- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cb5f30 msgr2=0x5646b0d5b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cb5f30 0x5646b0d5b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 -- 172.21.15.82:0/2736534108 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646b0d00070 con 0x5646b0cbb160 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29bffff640 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29b000df90 tx=0x7f29b001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11470b40594b9585 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29a77fe640 1 -- 172.21.15.82:0/2736534108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0055c00 con 0x5646b0cbb160 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29a77fe640 1 -- 172.21.15.82:0/2736534108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b007e040 con 0x5646b0cbb160 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.979+0000 7f29a77fe640 1 -- 172.21.15.82:0/2736534108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0053ca0 con 0x5646b0cbb160 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2736534108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 msgr2=0x5646b0d5b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f29b000df90 tx=0x7f29b001e000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2736534108 shutdown_connections 2024-04-25T10:03:37.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cff920 0x5646b0d5be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d5b8e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2736534108 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cb5f30 0x5646b0d5b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2736534108 >> 172.21.15.82:0/2736534108 conn(0x5646b0cbc2a0 msgr2=0x5646b0cb9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2736534108 shutdown_connections 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2736534108 wait complete. 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- start start 2024-04-25T10:03:37.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 Processor -- start 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0d60a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0d44730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0b093d0 con 0x5646b0cbb160 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646b0bc0d50 con 0x5646b0cff920 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5646b0a9f980 con 0x5646b0cb5f30 2024-04-25T10:03:37.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 10 client.0 _ll_register_callbacks cb 0x7f29b0051ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42206/0 (socket says 172.21.15.82:42206) 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 -- 172.21.15.82:0/2231145372 learned_addr learned my addr 172.21.15.82:0/2231145372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 -- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 msgr2=0x5646b0d60a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0d60a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 -- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 msgr2=0x5646b0d44730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0d44730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 -- 172.21.15.82:0/2231145372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5646b0b86600 con 0x5646b0cbb160 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29bffff640 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29b0055b90 tx=0x7f29b0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22649c19ecf6d272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b0055c00 con 0x5646b0cbb160 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29b0053f00 con 0x5646b0cbb160 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29b007e040 con 0x5646b0cbb160 2024-04-25T10:03:37.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29b0083430 con 0x5646b0cbb160 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 10 client.5631 fetch_fsmap learned FSMap version 17 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x5646b0a999b0 con 0x5646b0cbb160 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b007e040 con 0x5646b0cbb160 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29b0013070 con 0x5646b0cbb160 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 10 client.5631 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 10 client.5631 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:03:37.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 20 client.5631 populate_metadata read hostname 'smithi082' 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0d00240 0x5646b0d44c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 4 client.5631 mds_command: new command op to 5439 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 --> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5646b0b122e0 con 0x5646b0d00240 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29c659c640 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0d00240 0x5646b0d44c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 10 client.5631 ms_handle_connect on v2:172.21.15.82:6836/560740519 2024-04-25T10:03:37.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29c659c640 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0d00240 0x5646b0d44c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ef9ce0ea8d42d3 server_cookie=bce8717b0d45cf2a in_seq=0 out_seq=0 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 1 -- 172.21.15.82:0/2231145372 <== mds.0 v2:172.21.15.82:6836/560740519 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5646b0b122e0 con 0x5646b0d00240 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29a57fa640 10 client.5631 handle_command_reply: tid=0 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 client.5631 shutdown 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 msgr2=0x5646b0d60f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.983+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f29b0055b90 tx=0x7f29b0000bd0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 shutdown_connections 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:6836/560740519,v1:172.21.15.82:6837/560740519] conn(0x5646b0d00240 0x5646b0d44c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5646b0cff920 0x5646b0d44730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5646b0cbb160 0x5646b0d60f90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29a57fa640 0 client.5631 ms_handle_reset on v2:172.21.15.82:6836/560740519 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 --2- 172.21.15.82:0/2231145372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5646b0cb5f30 0x5646b0d60a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 >> 172.21.15.82:0/2231145372 conn(0x5646b0cbc2a0 msgr2=0x5646b0d07840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 shutdown_connections 2024-04-25T10:03:37.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 1 -- 172.21.15.82:0/2231145372 wait complete. 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cfe6c000 20 client.5631 trim_cache size 0 max 16384 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5439, 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:03:37.016 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 5.6374295700000001, 2024-04-25T10:03:37.017 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 17, 2024-04-25T10:03:37.017 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 39, 2024-04-25T10:03:37.017 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T10:03:37.017 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 12.567117826 2024-04-25T10:03:37.017 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:03:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 msgr2=0x7f29a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 0x7f29a8069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f29b800ba30 tx=0x7f29b8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 msgr2=0x7f29c81e5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f29b000d5d0 tx=0x7f29b000da90 comp rx=0 tx=0).stop 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 shutdown_connections 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29a8066f60 0x7f29a8069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29c8144ca0 0x7f29c81e61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29c80ce510 0x7f29c81e5c60 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 --2- 172.21.15.82:0/3109549111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29c806ec60 0x7f29c8134be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 >> 172.21.15.82:0/3109549111 conn(0x7f29c80b11a0 msgr2=0x7f29c80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 shutdown_connections 2024-04-25T10:03:37.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:36.987+0000 7f29cccff640 1 -- 172.21.15.82:0/3109549111 wait complete. 2024-04-25T10:03:37.019 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:37.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.127+0000 7f70631bb640 1 Processor -- start 2024-04-25T10:03:37.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.127+0000 7f70631bb640 1 -- start start 2024-04-25T10:03:37.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f70631bb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c143790 0x7f705c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f70631bb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c06c3b0 0x7f705c0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f70631bb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c143b70 con 0x7f705c15da40 2024-04-25T10:03:37.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f70631bb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c06e170 con 0x7f705c143790 2024-04-25T10:03:37.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f70631bb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c06e450 con 0x7f705c06c3b0 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c143790 0x7f705c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f705c143790 0x7f705c06d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42222/0 (socket says 172.21.15.82:42222) 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 -- 172.21.15.82:0/581582756 learned_addr learned my addr 172.21.15.82:0/581582756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7061758640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c06c3b0 0x7f705c0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 -- 172.21.15.82:0/581582756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c06c3b0 msgr2=0x7f705c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c06c3b0 0x7f705c0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 -- 172.21.15.82:0/581582756 >> v1:172.21.15.196:6789/0 conn(0x7f705c15da40 legacy=0x7f705c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 -- 172.21.15.82:0/581582756 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c06e730 con 0x7f705c143790 2024-04-25T10:03:37.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.131+0000 7f7060f57640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c143790 0x7f705c06d870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f705400af60 tx=0x7f705401daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd60655bb1df138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f705b7fe640 1 -- 172.21.15.82:0/581582756 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400a2d0 con 0x7f705c143790 2024-04-25T10:03:37.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f705b7fe640 1 -- 172.21.15.82:0/581582756 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70540a6710 con 0x7f705c143790 2024-04-25T10:03:37.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f705b7fe640 1 -- 172.21.15.82:0/581582756 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540a6ae0 con 0x7f705c143790 2024-04-25T10:03:37.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/581582756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c143790 msgr2=0x7f705c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c143790 0x7f705c06d870 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f705400af60 tx=0x7f705401daf0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/581582756 shutdown_connections 2024-04-25T10:03:37.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c06c3b0 0x7f705c0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- 172.21.15.82:0/581582756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c143790 0x7f705c06d870 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/581582756 >> 172.21.15.82:0/581582756 conn(0x7f705c071810 msgr2=0x7f705c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/581582756 shutdown_connections 2024-04-25T10:03:37.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/581582756 wait complete. 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 Processor -- start 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- start start 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 0x7f705c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 0x7f705c140360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b3b00 con 0x7f705c06c3b0 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b3980 con 0x7f705c143790 2024-04-25T10:03:37.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b3c80 con 0x7f705c15da40 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42238/0 (socket says 172.21.15.82:42238) 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 -- 172.21.15.82:0/374525821 learned_addr learned my addr 172.21.15.82:0/374525821 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7061758640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 0x7f705c140360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f705bfff640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 0x7f705c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 -- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 msgr2=0x7f705c140360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 0x7f705c140360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 -- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 msgr2=0x7f705c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 0x7f705c158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 -- 172.21.15.82:0/374525821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c13f5e0 con 0x7f705c06c3b0 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7061758640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 0x7f705c140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f705bfff640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 0x7f705c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f7060f57640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70540b0040 tx=0x7f705401a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540a6dc0 con 0x7f705c06c3b0 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054008500 con 0x7f705c06c3b0 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c159120 con 0x7f705c06c3b0 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70540088f0 con 0x7f705c06c3b0 2024-04-25T10:03:37.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.135+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c13fb60 con 0x7f705c06c3b0 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c04efa0 con 0x7f705c06c3b0 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70540bf020 con 0x7f705c06c3b0 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f70597fa640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f705411fca0 con 0x7f705c06c3b0 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f705bfff640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.139+0000 7f705bfff640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 0x7f703c069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f705c142fa0 tx=0x7f7044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.143+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70540176e0 con 0x7f705c06c3b0 2024-04-25T10:03:37.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.331+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f705c072670 con 0x7f705c06c3b0 2024-04-25T10:03:37.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.331+0000 7f70597fa640 1 -- 172.21.15.82:0/374525821 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f70540f0b10 con 0x7f705c06c3b0 2024-04-25T10:03:37.337 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 msgr2=0x7f703c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 0x7f703c069420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f705c142fa0 tx=0x7f7044008040 comp rx=0 tx=0).stop 2024-04-25T10:03:37.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 msgr2=0x7f705c15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f70540b0040 tx=0x7f705401a890 comp rx=0 tx=0).stop 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 shutdown_connections 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f703c066f60 0x7f703c069420 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f705c15da40 0x7f705c140360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f705c143790 0x7f705c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 --2- 172.21.15.82:0/374525821 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f705c06c3b0 0x7f705c15c250 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 >> 172.21.15.82:0/374525821 conn(0x7f705c071810 msgr2=0x7f705c1641c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 shutdown_connections 2024-04-25T10:03:37.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:37.335+0000 7f70631bb640 1 -- 172.21.15.82:0/374525821 wait complete. 2024-04-25T10:03:37.353 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:03:37.354 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:03:37.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:37.355 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:37.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.355 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:03:37.363 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:37.363 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:37.363 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:37.363 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:03:37.364 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:37.365 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.366 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:03:37.366 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42452sec preferred_lft 42452sec 2024-04-25T10:03:37.366 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:03:37.366 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.366 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:37.367 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:37.368 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:03:37.369 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.369 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:37.369 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:37.369 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:03:37.369 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:37.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.370 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:37.414 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:03:37.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.415 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:37.465 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:03:37.466 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:37.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.466 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:03:37.466 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:03:37.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:03:37.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:03:37.466 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:03:37.467 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:03:37.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.541 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:03:37.590 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:37.591 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:37.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:37.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.591 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:03:37.647 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:03:37.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:37.647 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:37.693 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:37.694 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:37.694 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:37.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:03:37.799 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:37.848 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:03:37.848 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:03:37.848 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:03:37.848 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:03:37.848 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:03:37.849 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:37.850 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:37.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:37.852 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:03:37.898 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:03:37.898 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:03:37.943 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:03:37.957 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:38.007 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:03:38.008 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:03:38.009 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:03:38.009 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:03:38.009 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:03:38.009 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:03:38.009 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:38.010 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:38.011 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:38.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:38.012 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:03:38.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:03:38.015+0000 7f2b26311580 -1 init, newargv = 0x55fb5a3205d0 newargc=15 2024-04-25T10:03:38.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[39588]: starting ceph client 2024-04-25T10:03:38.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[39588]: starting fuse 2024-04-25T10:03:39.126 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:03:39.143 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:03:39.192 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:03:39.192 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:03:39.192 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:03:39.192 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:03:39.193 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:03:39.194 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:03:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:39.195 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:39.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.196 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:03:39.242 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:03:39.242 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:03:39.242 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:03:39.242 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:03:39.243 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:03:39.244 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:39.245 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:03:39.245 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:39.366 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.39588.asok 2024-04-25T10:03:39.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:39.378 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39588.asok 2024-04-25T10:03:39.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39588.asok status 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:03:39.536 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "39588", 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:03:39.537 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5643, 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 5643 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:39.538 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2773104693 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:03:39.539 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2773104693 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.5643 192.168.0.1:0/2773104693", 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2773104693", 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:03:39.540 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 17, 2024-04-25T10:03:39.541 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 39, 2024-04-25T10:03:39.541 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:03:39.541 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:03:39.541 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:03:39.541 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:03:39.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.549 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:03:39.549 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:39.556 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:03:39.556 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:39.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:39.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.609 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T10:03:39.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.667 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T10:03:39.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.717 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T10:03:39.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.768 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T10:03:39.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.823 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T10:03:39.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:39.874 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:03:39.935 INFO:teuthology.orchestra.run.smithi082.stdout:./ 2024-04-25T10:03:39.935 INFO:teuthology.orchestra.run.smithi082.stdout:./rfile 2024-04-25T10:03:39.935 INFO:teuthology.orchestra.run.smithi082.stdout:./parentdir 2024-04-25T10:03:39.935 INFO:teuthology.orchestra.run.smithi082.stdout:./parentdir/pfile 2024-04-25T10:03:39.935 INFO:teuthology.orchestra.run.smithi082.stdout:./parentdir/childdir 2024-04-25T10:03:39.936 INFO:teuthology.orchestra.run.smithi082.stdout:./parentdir/childdir/cfile 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:03:39.936 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.040 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:03:40.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T10:03:40.050 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.157 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627778 2024-04-25T10:03:40.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T10:03:40.168 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.269 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:03:40.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.278 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:40.278 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:40.278 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:40.279 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:40.279 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T10:03:40.279 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.389 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628279 2024-04-25T10:03:40.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.399 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:40.399 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:40.400 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:40.400 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:40.400 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T10:03:40.400 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.507 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:03:40.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T10:03:40.517 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:03:40.620 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628280 2024-04-25T10:03:40.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:03:40.630 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:40.630 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:40.680 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:40.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 Processor -- start 2024-04-25T10:03:40.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 -- start start 2024-04-25T10:03:40.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43840afb90 0x7f43840cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f438409a950 con 0x7f438415b920 2024-04-25T10:03:40.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f438409ac30 con 0x7f43840afb90 2024-04-25T10:03:40.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f438b305640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43840ae380 con 0x7f438409a180 2024-04-25T10:03:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43898a2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43840afb90 0x7f43840cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43890a1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f438415b920 0x7f4384099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54784/0 (socket says 172.21.15.82:54784) 2024-04-25T10:03:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43890a1640 1 -- 172.21.15.82:0/51717380 learned_addr learned my addr 172.21.15.82:0/51717380 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43888a0640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43898a2640 1 -- 172.21.15.82:0/51717380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 msgr2=0x7f43840cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43898a2640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43898a2640 1 -- 172.21.15.82:0/51717380 >> v1:172.21.15.82:6789/0 conn(0x7f438415b920 legacy=0x7f4384099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f4383fff640 1 -- 172.21.15.82:0/51717380 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809210102 0 0) 0x7f438409a950 con 0x7f438415b920 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43898a2640 1 -- 172.21.15.82:0/51717380 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840ae660 con 0x7f43840afb90 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.847+0000 7f43888a0640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43898a2640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840cc670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f437c0097c0 tx=0x7f437c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37afe4839ac814ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f4383fff640 1 -- 172.21.15.82:0/51717380 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c0b0070 con 0x7f43840afb90 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f4383fff640 1 -- 172.21.15.82:0/51717380 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f437c0ab600 con 0x7f43840afb90 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f4383fff640 1 -- 172.21.15.82:0/51717380 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f437c0ab9d0 con 0x7f43840afb90 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- 172.21.15.82:0/51717380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 msgr2=0x7f43840cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840cc670 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f437c0097c0 tx=0x7f437c01c410 comp rx=0 tx=0).stop 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- 172.21.15.82:0/51717380 shutdown_connections 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840cc670 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- 172.21.15.82:0/51717380 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- 172.21.15.82:0/51717380 >> 172.21.15.82:0/51717380 conn(0x7f43840b3010 msgr2=0x7f43840b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- 172.21.15.82:0/51717380 shutdown_connections 2024-04-25T10:03:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- 172.21.15.82:0/51717380 wait complete. 2024-04-25T10:03:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 Processor -- start 2024-04-25T10:03:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- start start 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 0x7f438415e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43841684c0 con 0x7f43840afb90 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43840cd5d0 con 0x7f438415b920 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f438b305640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4384168640 con 0x7f438409a180 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42266/0 (socket says 172.21.15.82:42266) 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 -- 172.21.15.82:0/3817227505 learned_addr learned my addr 172.21.15.82:0/3817227505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43890a1640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43898a2640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 0x7f438415e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 -- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 msgr2=0x7f43840b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 -- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 msgr2=0x7f438415e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 0x7f438415e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 -- 172.21.15.82:0/3817227505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43840b6b60 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43898a2640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 0x7f438415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.851+0000 7f43888a0640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f437800c820 tx=0x7f437800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378015070 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378007a30 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43840b6e00 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378007e40 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f438415c350 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43840b7320 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4378018070 con 0x7f43840afb90 2024-04-25T10:03:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4378085470 con 0x7f43840afb90 2024-04-25T10:03:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f43890a1640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.855+0000 7f43890a1640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 0x7f4370069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4374007fd0 tx=0x7f4374007ee0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:40.859+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43840b7320 con 0x7f43840afb90 2024-04-25T10:03:41.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.011+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43840cd960 con 0x7f43840afb90 2024-04-25T10:03:41.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f4381ffb640 1 -- 172.21.15.82:0/3817227505 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f43780563d0 con 0x7f43840afb90 2024-04-25T10:03:41.019 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:41.019 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":39,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:32.322932+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 msgr2=0x7f4370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 0x7f4370069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4374007fd0 tx=0x7f4374007ee0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 msgr2=0x7f43840b6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f437800c820 tx=0x7f437800cce0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 shutdown_connections 2024-04-25T10:03:41.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4370066f60 0x7f4370069420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f438415b920 0x7f438415e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43840afb90 0x7f43840b6620 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 --2- 172.21.15.82:0/3817227505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f438409a180 0x7f43840b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 >> 172.21.15.82:0/3817227505 conn(0x7f43840b3010 msgr2=0x7f43840b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 shutdown_connections 2024-04-25T10:03:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.015+0000 7f438b305640 1 -- 172.21.15.82:0/3817227505 wait complete. 2024-04-25T10:03:41.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:29.159335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:29.660329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:41.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:41.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 Processor -- start 2024-04-25T10:03:41.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 -- start start 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f549815da80 0x7f54980d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549806be60 con 0x7f54980d1cd0 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549815de60 con 0x7f54980d07f0 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549e791640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549806c440 con 0x7f549815da80 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f54980d1cd0 0x7f54980d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54786/0 (socket says 172.21.15.82:54786) 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 -- 172.21.15.82:0/538818739 learned_addr learned my addr 172.21.15.82:0/538818739 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5497fff640 1 --2- 172.21.15.82:0/538818739 >> v2:172.21.15.196:3300/0 conn(0x7f549815da80 0x7f54980d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3575294386 0 0) 0x7f549806be60 con 0x7f54980d1cd0 2024-04-25T10:03:41.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5480003530 con 0x7f54980d1cd0 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37889104 0 0) 0x7f5480003530 con 0x7f54980d1cd0 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 >> v2:172.21.15.196:3300/0 conn(0x7f549815da80 msgr2=0x7f54980d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 --2- 172.21.15.82:0/538818739 >> v2:172.21.15.196:3300/0 conn(0x7f549815da80 0x7f54980d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 >> v1:172.21.15.196:6789/0 conn(0x7f54980d07f0 legacy=0x7f549806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980d2450 con 0x7f54980d1cd0 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f548c001800 con 0x7f54980d1cd0 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 >> v1:172.21.15.82:6789/0 conn(0x7f54980d1cd0 legacy=0x7f54980d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480003a30 0x7f5480003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 0x7f5480004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54800051d0 0x7f548000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549806c440 con 0x7f5480004810 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548000c100 con 0x7f5480003a30 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548000c300 con 0x7f54800051d0 2024-04-25T10:03:41.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 0x7f5480004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f5497fff640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480003a30 0x7f5480003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 -- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54800051d0 msgr2=0x7f548000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54800051d0 0x7f548000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 -- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480003a30 msgr2=0x7f5480003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480003a30 0x7f5480003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f54977fe640 1 -- 172.21.15.82:0/538818739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480003530 con 0x7f5480004810 2024-04-25T10:03:41.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.143+0000 7f549cd2e640 1 -- 172.21.15.82:0/538818739 reap_dead start 2024-04-25T10:03:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f54977fe640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 0x7f5480004c90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c210 tx=0x7f548c01f390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f5104af440bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c007d10 con 0x7f5480004810 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c007290 con 0x7f5480004810 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f5496ffd640 1 -- 172.21.15.82:0/538818739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0057b0 con 0x7f5480004810 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 msgr2=0x7f5480004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 0x7f5480004c90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c210 tx=0x7f548c01f390 comp rx=0 tx=0).stop 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- 172.21.15.82:0/538818739 shutdown_connections 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 --2- 172.21.15.82:0/538818739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5480004810 0x7f5480004c90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- 172.21.15.82:0/538818739 >> 172.21.15.82:0/538818739 conn(0x7f549808b000 msgr2=0x7f54980d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- 172.21.15.82:0/538818739 shutdown_connections 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- 172.21.15.82:0/538818739 wait complete. 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 Processor -- start 2024-04-25T10:03:41.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.147+0000 7f549e791640 1 -- start start 2024-04-25T10:03:41.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 0x7f54980cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f549814d820 0x7f5498160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980d26f0 con 0x7f549814d0d0 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980d2120 con 0x7f5480004810 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54980d2870 con 0x7f549814d820 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42278/0 (socket says 172.21.15.82:42278) 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 -- 172.21.15.82:0/2284809489 learned_addr learned my addr 172.21.15.82:0/2284809489 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5497fff640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 0x7f54980cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549cd2e640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f549814d820 0x7f5498160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 -- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f549814d820 msgr2=0x7f5498160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f549814d820 0x7f5498160300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 -- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 msgr2=0x7f54980cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 0x7f54980cd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 -- 172.21.15.82:0/2284809489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549814d530 con 0x7f549814d0d0 2024-04-25T10:03:41.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5497fff640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 0x7f54980cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f54977fe640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c210 tx=0x7f5480005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0070f0 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c0b6070 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54981507c0 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0057b0 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54980b7d90 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549815cc20 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f548c0b8020 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 0x7f5484069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f548c121eb0 con 0x7f549814d0d0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5497fff640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 0x7f5484069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.151+0000 7f5497fff640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 0x7f5484069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5488009aa0 tx=0x7f5488008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.155+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f548c0190a0 con 0x7f549814d0d0 2024-04-25T10:03:41.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.315+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54980b3280 con 0x7f549814d0d0 2024-04-25T10:03:41.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.315+0000 7f5494ff9640 1 -- 172.21.15.82:0/2284809489 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f548c0f2ca0 con 0x7f549814d0d0 2024-04-25T10:03:41.319 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:41.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 msgr2=0x7f5484069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 0x7f5484069420 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5488009aa0 tx=0x7f5488008040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 msgr2=0x7f5498150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f548c00c210 tx=0x7f5480005840 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 shutdown_connections 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5484066f60 0x7f5484069420 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f549814d820 0x7f5498160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f549814d0d0 0x7f5498150280 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 --2- 172.21.15.82:0/2284809489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5480004810 0x7f54980cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 >> 172.21.15.82:0/2284809489 conn(0x7f549808b000 msgr2=0x7f54980b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 shutdown_connections 2024-04-25T10:03:41.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.319+0000 7f549e791640 1 -- 172.21.15.82:0/2284809489 wait complete. 2024-04-25T10:03:41.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:41.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.459+0000 7fd6cc502640 1 Processor -- start 2024-04-25T10:03:41.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.459+0000 7fd6cc502640 1 -- start start 2024-04-25T10:03:41.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.459+0000 7fd6cc502640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c415b950 0x7fd6c409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.459+0000 7fd6cc502640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd6c40cf420 0x7fd6c40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.459+0000 7fd6cc502640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c415bd30 con 0x7fd6c40cfb80 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6c40cc0d0 con 0x7fd6c40cf420 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c409ab10 con 0x7fd6c415b950 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6caa9f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd6c40cfb80 0x7fd6c406f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54790/0 (socket says 172.21.15.82:54790) 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6caa9f640 1 -- 172.21.15.82:0/3976918149 learned_addr learned my addr 172.21.15.82:0/3976918149 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/3976918149 >> v2:172.21.15.82:3300/0 conn(0x7fd6c40cf420 0x7fd6c40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6ca29e640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c415b950 0x7fd6c409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/3976918149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c415b950 msgr2=0x7fd6c409a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c415b950 0x7fd6c409a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/3976918149 >> v1:172.21.15.82:6789/0 conn(0x7fd6c40cfb80 legacy=0x7fd6c406f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/3976918149 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c40cc370 con 0x7fd6c40cf420 2024-04-25T10:03:41.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c929c640 1 -- 172.21.15.82:0/3976918149 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256178757 0 0) 0x7fd6c415bd30 con 0x7fd6c40cfb80 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cf420 0x7fd6c40cce50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40097c0 tx=0x7fd6b401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaf4e2e75b2393e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c929c640 1 -- 172.21.15.82:0/3976918149 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40b2070 con 0x7fd6c40cf420 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c929c640 1 -- 172.21.15.82:0/3976918149 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b40ad680 con 0x7fd6c40cf420 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6c929c640 1 -- 172.21.15.82:0/3976918149 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40ada50 con 0x7fd6c40cf420 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 -- 172.21.15.82:0/3976918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cf420 msgr2=0x7fd6c40cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cf420 0x7fd6c40cce50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40097c0 tx=0x7fd6b401c470 comp rx=0 tx=0).stop 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 -- 172.21.15.82:0/3976918149 shutdown_connections 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cf420 0x7fd6c40cce50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 --2- 172.21.15.82:0/3976918149 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c415b950 0x7fd6c409a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.463+0000 7fd6cc502640 1 -- 172.21.15.82:0/3976918149 >> 172.21.15.82:0/3976918149 conn(0x7fd6c40b30b0 msgr2=0x7fd6c40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- 172.21.15.82:0/3976918149 shutdown_connections 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- 172.21.15.82:0/3976918149 wait complete. 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 Processor -- start 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- start start 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 0x7fd6c4146aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 0x7fd6c40b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c40cd390 con 0x7fd6c40cfb80 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c40d03e0 con 0x7fd6c415b950 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c4165870 con 0x7fd6c40cf420 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42292/0 (socket says 172.21.15.82:42292) 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/4261338403 learned_addr learned my addr 172.21.15.82:0/4261338403 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6caa9f640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 0x7fd6c40b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6ca29e640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 0x7fd6c4146aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 msgr2=0x7fd6c4146aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 0x7fd6c4146aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 msgr2=0x7fd6c40b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 0x7fd6c40b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 -- 172.21.15.82:0/4261338403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c4166240 con 0x7fd6c40cfb80 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6ca29e640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 0x7fd6c4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6caa9f640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 0x7fd6c40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6c9a9d640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40b0040 tx=0x7fd6b4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b40b2040 con 0x7fd6c40cfb80 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b40adbf0 con 0x7fd6c40cfb80 2024-04-25T10:03:41.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c40c7300 con 0x7fd6c40cfb80 2024-04-25T10:03:41.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b4004020 con 0x7fd6c40cfb80 2024-04-25T10:03:41.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c4165500 con 0x7fd6c40cfb80 2024-04-25T10:03:41.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.467+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c4165320 con 0x7fd6c40cfb80 2024-04-25T10:03:41.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.471+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6b40bf060 con 0x7fd6c40cfb80 2024-04-25T10:03:41.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.471+0000 7fd6baffd640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 0x7fd6a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.471+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6b411e260 con 0x7fd6c40cfb80 2024-04-25T10:03:41.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.471+0000 7fd6ca29e640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 0x7fd6a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.471+0000 7fd6ca29e640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 0x7fd6a8069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006fd0 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.475+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6c4165320 con 0x7fd6c40cfb80 2024-04-25T10:03:41.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.619+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6c404efa0 con 0x7fd6c40cfb80 2024-04-25T10:03:41.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.619+0000 7fd6baffd640 1 -- 172.21.15.82:0/4261338403 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6b4016090 con 0x7fd6c40cfb80 2024-04-25T10:03:41.624 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:41.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 msgr2=0x7fd6a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 0x7fd6a8069420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0006fd0 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 msgr2=0x7fd6c4165d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd6b40b0040 tx=0x7fd6b4019210 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 shutdown_connections 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd6a8066f60 0x7fd6a8069420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd6c415b950 0x7fd6c40b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd6c40cfb80 0x7fd6c4165d00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 --2- 172.21.15.82:0/4261338403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd6c40cf420 0x7fd6c4146aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 >> 172.21.15.82:0/4261338403 conn(0x7fd6c40b30b0 msgr2=0x7fd6c406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 shutdown_connections 2024-04-25T10:03:41.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.623+0000 7fd6cc502640 1 -- 172.21.15.82:0/4261338403 wait complete. 2024-04-25T10:03:41.641 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:41.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 Processor -- start 2024-04-25T10:03:41.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 -- start start 2024-04-25T10:03:41.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 0x7eff780cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7815bd30 con 0x7eff780cf420 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780d0510 con 0x7eff7815b950 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7f31b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cc0d0 con 0x7eff780cfb80 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d0b7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7c8b6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7eff780cf420 0x7eff780cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54804/0 (socket says 172.21.15.82:54804) 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7c8b6640 1 -- 172.21.15.82:0/63788699 learned_addr learned my addr 172.21.15.82:0/63788699 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d0b7640 1 -- 172.21.15.82:0/63788699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 msgr2=0x7eff780cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d8b8640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 0x7eff780cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d0b7640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 0x7eff780cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff6ffff640 1 -- 172.21.15.82:0/63788699 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345759091 0 0) 0x7eff7815bd30 con 0x7eff780cf420 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d0b7640 1 -- 172.21.15.82:0/63788699 >> v1:172.21.15.82:6789/0 conn(0x7eff780cf420 legacy=0x7eff780cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d0b7640 1 -- 172.21.15.82:0/63788699 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7809ab10 con 0x7eff7815b950 2024-04-25T10:03:41.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.755+0000 7eff7d8b8640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 0x7eff780cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7d0b7640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7815b950 0x7eff7809a480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff740097c0 tx=0x7eff7401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c16e196f4335a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6ffff640 1 -- 172.21.15.82:0/63788699 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740b2070 con 0x7eff7815b950 2024-04-25T10:03:41.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6ffff640 1 -- 172.21.15.82:0/63788699 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff740ad640 con 0x7eff7815b950 2024-04-25T10:03:41.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6ffff640 1 -- 172.21.15.82:0/63788699 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740ada10 con 0x7eff7815b950 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/63788699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7815b950 msgr2=0x7eff7809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7815b950 0x7eff7809a480 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7eff740097c0 tx=0x7eff7401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/63788699 shutdown_connections 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cfb80 0x7eff780cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- 172.21.15.82:0/63788699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/63788699 >> 172.21.15.82:0/63788699 conn(0x7eff780b30b0 msgr2=0x7eff780b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/63788699 shutdown_connections 2024-04-25T10:03:41.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/63788699 wait complete. 2024-04-25T10:03:41.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 Processor -- start 2024-04-25T10:03:41.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- start start 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cf420 0x7eff780c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 0x7eff7815dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7806faa0 con 0x7eff780cfb80 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cf870 con 0x7eff78155620 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7806fc20 con 0x7eff780cf420 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42300/0 (socket says 172.21.15.82:42300) 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 -- 172.21.15.82:0/2038147425 learned_addr learned my addr 172.21.15.82:0/2038147425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7d0b7640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cf420 0x7eff780c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 -- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cf420 msgr2=0x7eff780c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cf420 0x7eff780c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7d8b8640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 0x7eff7815dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 -- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 msgr2=0x7eff7815dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 0x7eff7815dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 -- 172.21.15.82:0/2038147425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff78156aa0 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7d8b8640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 0x7eff7815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7c8b6640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c480 tx=0x7eff6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68018070 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68007500 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff78156d20 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff68004a30 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.759+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7815d580 con 0x7eff780cfb80 2024-04-25T10:03:41.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7804efa0 con 0x7eff780cfb80 2024-04-25T10:03:41.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff6801b070 con 0x7eff780cfb80 2024-04-25T10:03:41.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff6dffb640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff680844f0 con 0x7eff780cfb80 2024-04-25T10:03:41.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff7d0b7640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.763+0000 7eff7d0b7640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff740af000 tx=0x7eff74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.767+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff78156aa0 con 0x7eff780cfb80 2024-04-25T10:03:41.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.919+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff7815ef70 con 0x7eff780cfb80 2024-04-25T10:03:41.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff6dffb640 1 -- 172.21.15.82:0/2038147425 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff68055400 con 0x7eff780cfb80 2024-04-25T10:03:41.926 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 msgr2=0x7eff5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7eff740af000 tx=0x7eff74008040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 msgr2=0x7eff78156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c480 tx=0x7eff6800c940 comp rx=0 tx=0).stop 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 shutdown_connections 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff78155620 0x7eff7815dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff780cfb80 0x7eff78156560 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 --2- 172.21.15.82:0/2038147425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff780cf420 0x7eff780c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 >> 172.21.15.82:0/2038147425 conn(0x7eff780b30b0 msgr2=0x7eff781580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 shutdown_connections 2024-04-25T10:03:41.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:41.923+0000 7eff7f31b640 1 -- 172.21.15.82:0/2038147425 wait complete. 2024-04-25T10:03:41.943 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:42.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 Processor -- start 2024-04-25T10:03:42.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 -- start start 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f680c0d07f0 0x7f680c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f680c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680c06be60 con 0x7f680c15da80 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f680c06c440 con 0x7f680c0d07f0 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f6812227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c15de60 con 0x7f680c0d1cd0 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f680c15da80 0x7f680c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54816/0 (socket says 172.21.15.82:54816) 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680b7fe640 1 -- 172.21.15.82:0/2301858573 learned_addr learned my addr 172.21.15.82:0/2301858573 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 --2- 172.21.15.82:0/2301858573 >> v2:172.21.15.196:3300/0 conn(0x7f680c0d07f0 0x7f680c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680bfff640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f680c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009906547 0 0) 0x7f680c06be60 con 0x7f680c15da80 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f8003530 con 0x7f680c15da80 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3020821721 0 0) 0x7f67f8003530 con 0x7f680c15da80 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 msgr2=0x7f680c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f680c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 >> v2:172.21.15.196:3300/0 conn(0x7f680c0d07f0 msgr2=0x7f680c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 --2- 172.21.15.82:0/2301858573 >> v2:172.21.15.196:3300/0 conn(0x7f680c0d07f0 0x7f680c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0d2450 con 0x7f680c15da80 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f67f40016a0 con 0x7f680c15da80 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 >> v1:172.21.15.82:6789/0 conn(0x7f680c15da80 legacy=0x7f680c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8003a30 0x7f67f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67f8004810 0x7f67f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f67f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c06c440 con 0x7f67f8004810 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c15de60 con 0x7f67f8003a30 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f800b9d0 con 0x7f680c0d1cd0 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67f8004810 0x7f67f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680bfff640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f67f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 -- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 msgr2=0x7f67f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f680c0d1cd0 0x7f67f80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680b7fe640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8003a30 0x7f67f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 -- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8003a30 msgr2=0x7f67f8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67f8003a30 0x7f67f8003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.059+0000 7f680affd640 1 -- 172.21.15.82:0/2301858573 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f8003530 con 0x7f67f8004810 2024-04-25T10:03:42.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f680affd640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67f8004810 0x7f67f8004c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009af0 tx=0x7f67fc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d81bb348a69a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc009690 con 0x7f67f8004810 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc0059a0 con 0x7f67f8004810 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f680a7fc640 1 -- 172.21.15.82:0/2301858573 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc005b90 con 0x7f67f8004810 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- 172.21.15.82:0/2301858573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67f8004810 msgr2=0x7f67f8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 --2- 172.21.15.82:0/2301858573 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67f8004810 0x7f67f8004c90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009af0 tx=0x7f67fc01ca70 comp rx=0 tx=0).stop 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f680bfff640 1 -- 172.21.15.82:0/2301858573 reap_dead start 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- 172.21.15.82:0/2301858573 shutdown_connections 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- 172.21.15.82:0/2301858573 >> 172.21.15.82:0/2301858573 conn(0x7f680c08b000 msgr2=0x7f680c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- 172.21.15.82:0/2301858573 shutdown_connections 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- 172.21.15.82:0/2301858573 wait complete. 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 Processor -- start 2024-04-25T10:03:42.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.063+0000 7f6812227640 1 -- start start 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 0x7f680c0cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f680c1f3840 0x7f680c1f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0d26f0 con 0x7f680c0c7960 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0d2140 con 0x7f680c1f3840 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680c0d2870 con 0x7f67f8004810 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42318/0 (socket says 172.21.15.82:42318) 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 -- 172.21.15.82:0/2502747193 learned_addr learned my addr 172.21.15.82:0/2502747193 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680bfff640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f680c1f3840 0x7f680c1f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680b7fe640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 0x7f680c0cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 -- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 msgr2=0x7f680c0cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 0x7f680c0cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 -- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f680c1f3840 msgr2=0x7f680c1f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f680c1f3840 0x7f680c1f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 -- 172.21.15.82:0/2502747193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c0c81b0 con 0x7f680c0c7960 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680b7fe640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 0x7f680c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680affd640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009af0 tx=0x7f67f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc0bc070 con 0x7f680c0c7960 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67fc004220 con 0x7f680c0c7960 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67fc006240 con 0x7f680c0c7960 2024-04-25T10:03:42.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c0c8490 con 0x7f680c0c7960 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c0caa10 con 0x7f680c0c7960 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c04efa0 con 0x7f680c0c7960 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67fc0063e0 con 0x7f680c0c7960 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 0x7f67ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f680b7fe640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 0x7f67ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.067+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67fc11fb10 con 0x7f680c0c7960 2024-04-25T10:03:42.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.071+0000 7f680b7fe640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 0x7f67ec069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001950 tx=0x7f67f4001cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.075+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67fc0167c0 con 0x7f680c0c7960 2024-04-25T10:03:42.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.235+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f680c0ca190 con 0x7f680c0c7960 2024-04-25T10:03:42.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.235+0000 7f6803fff640 1 -- 172.21.15.82:0/2502747193 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f67fc0f0b40 con 0x7f680c0c7960 2024-04-25T10:03:42.240 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 msgr2=0x7f67ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 0x7f67ec069450 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f67f4001950 tx=0x7f67f4001cf0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 msgr2=0x7f680c1f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009af0 tx=0x7f67f8005840 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 shutdown_connections 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f67ec066f90 0x7f67ec069450 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f680c1f3840 0x7f680c1f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f680c0c7960 0x7f680c1f3300 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 --2- 172.21.15.82:0/2502747193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67f8004810 0x7f680c0cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 >> 172.21.15.82:0/2502747193 conn(0x7f680c08b000 msgr2=0x7f680c0d12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 shutdown_connections 2024-04-25T10:03:42.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.239+0000 7f6812227640 1 -- 172.21.15.82:0/2502747193 wait complete. 2024-04-25T10:03:42.258 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:03:42.259 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:42.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:42.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.407+0000 7f130ec9a640 1 Processor -- start 2024-04-25T10:03:42.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ec9a640 1 -- start start 2024-04-25T10:03:42.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ec9a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f13081451f0 0x7f1308142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ec9a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13080ccc70 con 0x7f13080ce510 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ec9a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f130806dc40 con 0x7f1308144ca0 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ec9a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13080cc2b0 con 0x7f13081451f0 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ca36640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f13080ce510 0x7f13080ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54824/0 (socket says 172.21.15.82:54824) 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130ca36640 1 -- 172.21.15.82:0/51090683 learned_addr learned my addr 172.21.15.82:0/51090683 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130d237640 1 --2- 172.21.15.82:0/51090683 >> v2:172.21.15.82:3300/0 conn(0x7f13081451f0 0x7f1308142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130d237640 1 -- 172.21.15.82:0/51090683 >> v1:172.21.15.196:6789/0 conn(0x7f1308144ca0 legacy=0x7f130806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130d237640 1 -- 172.21.15.82:0/51090683 >> v1:172.21.15.82:6789/0 conn(0x7f13080ce510 legacy=0x7f13080ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130d237640 1 -- 172.21.15.82:0/51090683 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130806df20 con 0x7f13081451f0 2024-04-25T10:03:42.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f13077fe640 1 -- 172.21.15.82:0/51090683 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228391321 0 0) 0x7f13080ccc70 con 0x7f13080ce510 2024-04-25T10:03:42.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f130d237640 1 --2- 172.21.15.82:0/51090683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13081451f0 0x7f1308142680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1300009dc0 tx=0x7f130001f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe31b4bfe9e05660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f13077fe640 1 -- 172.21.15.82:0/51090683 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000b4070 con 0x7f13081451f0 2024-04-25T10:03:42.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f13077fe640 1 -- 172.21.15.82:0/51090683 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13000af600 con 0x7f13081451f0 2024-04-25T10:03:42.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.411+0000 7f13077fe640 1 -- 172.21.15.82:0/51090683 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000af9d0 con 0x7f13081451f0 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/51090683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13081451f0 msgr2=0x7f1308142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 --2- 172.21.15.82:0/51090683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13081451f0 0x7f1308142680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f1300009dc0 tx=0x7f130001f100 comp rx=0 tx=0).stop 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/51090683 shutdown_connections 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 --2- 172.21.15.82:0/51090683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f13081451f0 0x7f1308142680 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/51090683 >> 172.21.15.82:0/51090683 conn(0x7f13080b11a0 msgr2=0x7f13080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/51090683 shutdown_connections 2024-04-25T10:03:42.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/51090683 wait complete. 2024-04-25T10:03:42.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 Processor -- start 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- start start 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13080ce510 0x7f130815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13081451f0 0x7f1308141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13080cc900 con 0x7f1308144ca0 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f130806e290 con 0x7f13080ce510 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1308142bc0 con 0x7f13081451f0 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42324/0 (socket says 172.21.15.82:42324) 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 -- 172.21.15.82:0/3174343045 learned_addr learned my addr 172.21.15.82:0/3174343045 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ca36640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13080ce510 0x7f130815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130d237640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13081451f0 0x7f1308141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 -- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13081451f0 msgr2=0x7f1308141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13081451f0 0x7f1308141e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 -- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13080ce510 msgr2=0x7f130815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13080ce510 0x7f130815b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 -- 172.21.15.82:0/3174343045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13080b3c70 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f1307fff640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c410 tx=0x7f12f000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011790 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0017070 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0011bb0 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13080b3f50 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.415+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13081f3710 con 0x7f1308144ca0 2024-04-25T10:03:42.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f130804efa0 con 0x7f1308144ca0 2024-04-25T10:03:42.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12f001c460 con 0x7f1308144ca0 2024-04-25T10:03:42.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f13057fa640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 0x7f12e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f12f0084e60 con 0x7f1308144ca0 2024-04-25T10:03:42.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f130ca36640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 0x7f12e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.419+0000 7f130ca36640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 0x7f12e8069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000f70 tx=0x7f12f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.423+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12f0021020 con 0x7f1308144ca0 2024-04-25T10:03:42.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13081f38e0 con 0x7f1308144ca0 2024-04-25T10:03:42.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f13057fa640 1 -- 172.21.15.82:0/3174343045 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f12f0055ce0 con 0x7f1308144ca0 2024-04-25T10:03:42.583 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:42.583 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":39,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:32.322932+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 msgr2=0x7f12e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 0x7f12e8069420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f12f8000f70 tx=0x7f12f8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 msgr2=0x7f13081418e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f12f000c410 tx=0x7f12f000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 shutdown_connections 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12e8066f60 0x7f12e8069420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f13081451f0 0x7f1308141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1308144ca0 0x7f13081418e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 --2- 172.21.15.82:0/3174343045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f13080ce510 0x7f130815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 >> 172.21.15.82:0/3174343045 conn(0x7f13080b11a0 msgr2=0x7f1308071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 shutdown_connections 2024-04-25T10:03:42.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.579+0000 7f130ec9a640 1 -- 172.21.15.82:0/3174343045 wait complete. 2024-04-25T10:03:42.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:29.159335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:29.660329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:42.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:42.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 Processor -- start 2024-04-25T10:03:42.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 -- start start 2024-04-25T10:03:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 0x7fbf7c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbf7c0ce170 0x7fbf7c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf7c0ca4c0 con 0x7fbf7c0cf970 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ca780 con 0x7fbf7c0ce170 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf8245e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0cd440 con 0x7fbf7c0cf590 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbf7c0ce170 0x7fbf7c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf73fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbf7c0cf970 0x7fbf7c0c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54840/0 (socket says 172.21.15.82:54840) 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf73fff640 1 -- 172.21.15.82:0/2494261026 learned_addr learned my addr 172.21.15.82:0/2494261026 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf7bfff640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 0x7fbf7c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 -- 172.21.15.82:0/2494261026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 msgr2=0x7fbf7c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 0x7fbf7c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 -- 172.21.15.82:0/2494261026 >> v1:172.21.15.82:6789/0 conn(0x7fbf7c0cf970 legacy=0x7fbf7c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 -- 172.21.15.82:0/2494261026 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0cd690 con 0x7fbf7c0ce170 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf7b7fe640 1 -- 172.21.15.82:0/2494261026 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732507141 0 0) 0x7fbf7c0ca4c0 con 0x7fbf7c0cf970 2024-04-25T10:03:42.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf7bfff640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 0x7fbf7c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:42.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.719+0000 7fbf809fb640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0ce600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400bfa0 tx=0x7fbf7401eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=272574587b5d9689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7b7fe640 1 -- 172.21.15.82:0/2494261026 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7400b2d0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7b7fe640 1 -- 172.21.15.82:0/2494261026 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf740a7850 con 0x7fbf7c0ce170 2024-04-25T10:03:42.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7b7fe640 1 -- 172.21.15.82:0/2494261026 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf740a7c20 con 0x7fbf7c0ce170 2024-04-25T10:03:42.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2494261026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 msgr2=0x7fbf7c0ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0ce600 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbf7400bfa0 tx=0x7fbf7401eb70 comp rx=0 tx=0).stop 2024-04-25T10:03:42.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2494261026 shutdown_connections 2024-04-25T10:03:42.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0ce600 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2494261026 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf590 0x7fbf7c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2494261026 >> 172.21.15.82:0/2494261026 conn(0x7fbf7c06ae60 msgr2=0x7fbf7c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2494261026 shutdown_connections 2024-04-25T10:03:42.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2494261026 wait complete. 2024-04-25T10:03:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 Processor -- start 2024-04-25T10:03:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- start start 2024-04-25T10:03:42.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf7c0cf590 0x7fbf7c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf970 0x7fbf7c1f6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ca240 con 0x7fbf7c0ce170 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ca0c0 con 0x7fbf7c0cf590 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7c0ac170 con 0x7fbf7c0cf970 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42342/0 (socket says 172.21.15.82:42342) 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 -- 172.21.15.82:0/2505733018 learned_addr learned my addr 172.21.15.82:0/2505733018 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf73fff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf7c0cf590 0x7fbf7c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 -- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf970 msgr2=0x7fbf7c1f6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf970 0x7fbf7c1f6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 -- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf7c0cf590 msgr2=0x7fbf7c1f6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf7c0cf590 0x7fbf7c1f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 -- 172.21.15.82:0/2505733018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0c8fd0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf7bfff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00c7c0 tx=0x7fbf6c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c007a30 con 0x7fbf7c0ce170 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6c007bd0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf7c0c7f30 con 0x7fbf7c0ce170 2024-04-25T10:03:42.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0113f0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.723+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf7c0c84f0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf7c04efa0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf6c011590 con 0x7fbf7c0ce170 2024-04-25T10:03:42.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf797fa640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 0x7fbf5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:42.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf73fff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 0x7fbf5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:42.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf6c085820 con 0x7fbf7c0ce170 2024-04-25T10:03:42.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.727+0000 7fbf73fff640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 0x7fbf5c069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c152c30 tx=0x7fbf64009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:42.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.731+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf6c015020 con 0x7fbf7c0ce170 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.875+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf7c0721a0 con 0x7fbf7c0ce170 2024-04-25T10:03:42.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.875+0000 7fbf797fa640 1 -- 172.21.15.82:0/2505733018 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf6c056730 con 0x7fbf7c0ce170 2024-04-25T10:03:42.881 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 msgr2=0x7fbf5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 0x7fbf5c069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c152c30 tx=0x7fbf64009040 comp rx=0 tx=0).stop 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 msgr2=0x7fbf7c0c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00c7c0 tx=0x7fbf6c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 shutdown_connections 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbf5c066f60 0x7fbf5c069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbf7c0cf970 0x7fbf7c1f6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbf7c0cf590 0x7fbf7c1f6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 --2- 172.21.15.82:0/2505733018 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbf7c0ce170 0x7fbf7c0c55d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 >> 172.21.15.82:0/2505733018 conn(0x7fbf7c06ae60 msgr2=0x7fbf7c06ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 shutdown_connections 2024-04-25T10:03:42.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:42.879+0000 7fbf8245e640 1 -- 172.21.15.82:0/2505733018 wait complete. 2024-04-25T10:03:42.898 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:43.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 Processor -- start 2024-04-25T10:03:43.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 -- start start 2024-04-25T10:03:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 0x7fc4ac0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4ac0b28c0 0x7fc4ac163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac0c9ef0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac072780 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b30f9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac071b70 con 0x7fc4ac159bd0 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b1696640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4ac0b28c0 0x7fc4ac163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4abfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc4ac159fb0 0x7fc4ac0b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54850/0 (socket says 172.21.15.82:54850) 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4abfff640 1 -- 172.21.15.82:0/1824544020 learned_addr learned my addr 172.21.15.82:0/1824544020 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b0e95640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 0x7fc4ac0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b1696640 1 -- 172.21.15.82:0/1824544020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 msgr2=0x7fc4ac0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b1696640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 0x7fc4ac0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b1696640 1 -- 172.21.15.82:0/1824544020 >> v1:172.21.15.82:6789/0 conn(0x7fc4ac159fb0 legacy=0x7fc4ac0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b1696640 1 -- 172.21.15.82:0/1824544020 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac071e50 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4ab7fe640 1 -- 172.21.15.82:0/1824544020 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213238615 0 0) 0x7fc4ac0c9ef0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.019+0000 7fc4b0e95640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 0x7fc4ac0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b1696640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac0b28c0 0x7fc4ac163960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c01da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e583135fead1774f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4ab7fe640 1 -- 172.21.15.82:0/1824544020 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0b3070 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4ab7fe640 1 -- 172.21.15.82:0/1824544020 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c0a6a90 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4ab7fe640 1 -- 172.21.15.82:0/1824544020 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0a6e60 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- 172.21.15.82:0/1824544020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac0b28c0 msgr2=0x7fc4ac163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac0b28c0 0x7fc4ac163960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c01da50 comp rx=0 tx=0).stop 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- 172.21.15.82:0/1824544020 shutdown_connections 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac0b28c0 0x7fc4ac163960 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/1824544020 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac159bd0 0x7fc4ac0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- 172.21.15.82:0/1824544020 >> 172.21.15.82:0/1824544020 conn(0x7fc4ac06ae60 msgr2=0x7fc4ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- 172.21.15.82:0/1824544020 shutdown_connections 2024-04-25T10:03:43.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- 172.21.15.82:0/1824544020 wait complete. 2024-04-25T10:03:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 Processor -- start 2024-04-25T10:03:43.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- start start 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac0b28c0 0x7fc4ac1574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 0x7fc4ac157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac166030 con 0x7fc4ac159fb0 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac165eb0 con 0x7fc4ac159bd0 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.023+0000 7fc4b30f9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1661b0 con 0x7fc4ac0b28c0 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42350/0 (socket says 172.21.15.82:42350) 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 -- 172.21.15.82:0/3344682635 learned_addr learned my addr 172.21.15.82:0/3344682635 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b0e95640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac0b28c0 0x7fc4ac1574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4abfff640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 0x7fc4ac157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 -- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac0b28c0 msgr2=0x7fc4ac1574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac0b28c0 0x7fc4ac1574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 -- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 msgr2=0x7fc4ac157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 0x7fc4ac157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 -- 172.21.15.82:0/3344682635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac158130 con 0x7fc4ac159fb0 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4abfff640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 0x7fc4ac157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b1696640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c01c290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0b3040 con 0x7fc4ac159fb0 2024-04-25T10:03:43.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c003ba0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac0c7ff0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0ae680 con 0x7fc4ac159fb0 2024-04-25T10:03:43.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac0c85b0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac04efa0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc49c0c0060 con 0x7fc4ac159fb0 2024-04-25T10:03:43.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc49c11f180 con 0x7fc4ac159fb0 2024-04-25T10:03:43.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b0e95640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.027+0000 7fc4b0e95640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 0x7fc498069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4006fd0 tx=0x7fc4a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.031+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc49c017670 con 0x7fc4ac159fb0 2024-04-25T10:03:43.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc4ac1634e0 con 0x7fc4ac159fb0 2024-04-25T10:03:43.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4a97fa640 1 -- 172.21.15.82:0/3344682635 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc49c0eff70 con 0x7fc4ac159fb0 2024-04-25T10:03:43.194 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 msgr2=0x7fc498069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 0x7fc498069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4006fd0 tx=0x7fc4a4008040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 msgr2=0x7fc4ac15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009d90 tx=0x7fc49c01c290 comp rx=0 tx=0).stop 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 shutdown_connections 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4ac159fb0 0x7fc4ac15a420 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4ac159bd0 0x7fc4ac157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 --2- 172.21.15.82:0/3344682635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4ac0b28c0 0x7fc4ac1574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 >> 172.21.15.82:0/3344682635 conn(0x7fc4ac06ae60 msgr2=0x7fc4ac0b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 shutdown_connections 2024-04-25T10:03:43.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.191+0000 7fc4b30f9640 1 -- 172.21.15.82:0/3344682635 wait complete. 2024-04-25T10:03:43.213 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:43.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.331+0000 7fee39e78640 1 Processor -- start 2024-04-25T10:03:43.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee39e78640 1 -- start start 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee39e78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 0x7fee3406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee39e78640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3415ab30 con 0x7fee3415a750 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee39e78640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee340cf560 con 0x7fee3406c3b0 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee39e78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34067100 con 0x7fee34066bb0 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fee3415a750 0x7fee3406bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54852/0 (socket says 172.21.15.82:54852) 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 -- 172.21.15.82:0/980791632 learned_addr learned my addr 172.21.15.82:0/980791632 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1631379826 0 0) 0x7fee3415ab30 con 0x7fee3415a750 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee33fff640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 0x7fee3406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee1c003530 con 0x7fee3415a750 2024-04-25T10:03:43.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2348766076 0 0) 0x7fee1c003530 con 0x7fee3415a750 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 msgr2=0x7fee3406ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 0x7fee3406ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 >> v1:172.21.15.196:6789/0 conn(0x7fee3406c3b0 legacy=0x7fee340b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340cf840 con 0x7fee3415a750 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fee24001680 con 0x7fee3415a750 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 >> v1:172.21.15.82:6789/0 conn(0x7fee3415a750 legacy=0x7fee3406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee1c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 0x7fee1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c004d50 0x7fee1c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee34067100 con 0x7fee1c003a60 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c00bc00 con 0x7fee1c004d50 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee1c00be00 con 0x7fee34066bb0 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee1c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee33fff640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c004d50 0x7fee1c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 -- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 msgr2=0x7fee1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee34066bb0 0x7fee1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 -- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c004d50 msgr2=0x7fee1c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee1c004d50 0x7fee1c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.335+0000 7fee337fe640 1 -- 172.21.15.82:0/980791632 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1c003530 con 0x7fee1c003a60 2024-04-25T10:03:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee1c003e80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c210 tx=0x7fee2401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a86cab744b3f369 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24007d40 con 0x7fee1c003a60 2024-04-25T10:03:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee240a9070 con 0x7fee1c003a60 2024-04-25T10:03:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee327fc640 1 -- 172.21.15.82:0/980791632 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24005ae0 con 0x7fee1c003a60 2024-04-25T10:03:43.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- 172.21.15.82:0/980791632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 msgr2=0x7fee1c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 --2- 172.21.15.82:0/980791632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee1c003e80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c210 tx=0x7fee2401cca0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee33fff640 1 -- 172.21.15.82:0/980791632 reap_dead start 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- 172.21.15.82:0/980791632 shutdown_connections 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- 172.21.15.82:0/980791632 >> 172.21.15.82:0/980791632 conn(0x7fee34071810 msgr2=0x7fee34071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- 172.21.15.82:0/980791632 shutdown_connections 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- 172.21.15.82:0/980791632 wait complete. 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 Processor -- start 2024-04-25T10:03:43.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- start start 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee341516e0 0x7fee34149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee340b3b50 0x7fee340b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406dc40 con 0x7fee1c003a60 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406dac0 con 0x7fee341516e0 2024-04-25T10:03:43.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee39e78640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3406ddc0 con 0x7fee340b3b50 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42360/0 (socket says 172.21.15.82:42360) 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 -- 172.21.15.82:0/3419528015 learned_addr learned my addr 172.21.15.82:0/3419528015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee32ffd640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee341516e0 0x7fee34149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee33fff640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee340b3b50 0x7fee340b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 -- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee340b3b50 msgr2=0x7fee340b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee340b3b50 0x7fee340b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 -- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee341516e0 msgr2=0x7fee34149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee341516e0 0x7fee34149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 -- 172.21.15.82:0/3419528015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34149b70 con 0x7fee1c003a60 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.339+0000 7fee337fe640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c210 tx=0x7fee1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24000ee0 con 0x7fee1c003a60 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24001080 con 0x7fee1c003a60 2024-04-25T10:03:43.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee34149e50 con 0x7fee1c003a60 2024-04-25T10:03:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee240049f0 con 0x7fee1c003a60 2024-04-25T10:03:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee3414a410 con 0x7fee1c003a60 2024-04-25T10:03:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee3404efa0 con 0x7fee1c003a60 2024-04-25T10:03:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fee24004b90 con 0x7fee1c003a60 2024-04-25T10:03:43.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 0x7fee20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fee2411ea70 con 0x7fee1c003a60 2024-04-25T10:03:43.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee32ffd640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 0x7fee20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.343+0000 7fee32ffd640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 0x7fee20069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fee34151b00 tx=0x7fee28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.347+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fee240169b0 con 0x7fee1c003a60 2024-04-25T10:03:43.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.507+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fee340cd160 con 0x7fee1c003a60 2024-04-25T10:03:43.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.507+0000 7fee1bfff640 1 -- 172.21.15.82:0/3419528015 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fee240ef900 con 0x7fee1c003a60 2024-04-25T10:03:43.512 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 msgr2=0x7fee20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 0x7fee20069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fee34151b00 tx=0x7fee28009040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 msgr2=0x7fee3414d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c210 tx=0x7fee1c005840 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 shutdown_connections 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fee20066f60 0x7fee20069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fee340b3b50 0x7fee340b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fee341516e0 0x7fee34149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 --2- 172.21.15.82:0/3419528015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fee1c003a60 0x7fee3414d970 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 >> 172.21.15.82:0/3419528015 conn(0x7fee34071810 msgr2=0x7fee340cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 shutdown_connections 2024-04-25T10:03:43.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.511+0000 7fee39e78640 1 -- 172.21.15.82:0/3419528015 wait complete. 2024-04-25T10:03:43.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:43.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 Processor -- start 2024-04-25T10:03:43.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 -- start start 2024-04-25T10:03:43.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94f0071740 0x7f94f00acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f00b24a0 0x7f94f00ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f00ae860 con 0x7f94f015da10 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f015e210 con 0x7f94f00b24a0 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f6621640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f015e4f0 con 0x7f94f0071740 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f00b24a0 0x7f94f00ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f00b24a0 0x7f94f00ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42364/0 (socket says 172.21.15.82:42364) 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 -- 172.21.15.82:0/3463036243 learned_addr learned my addr 172.21.15.82:0/3463036243 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94ef7fe640 1 --2- 172.21.15.82:0/3463036243 >> v2:172.21.15.196:3300/0 conn(0x7f94f0071740 0x7f94f00acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 -- 172.21.15.82:0/3463036243 >> v2:172.21.15.196:3300/0 conn(0x7f94f0071740 msgr2=0x7f94f00acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 --2- 172.21.15.82:0/3463036243 >> v2:172.21.15.196:3300/0 conn(0x7f94f0071740 0x7f94f00acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 -- 172.21.15.82:0/3463036243 >> v1:172.21.15.196:6789/0 conn(0x7f94f015da10 legacy=0x7f94f0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.643+0000 7f94f4bbe640 1 -- 172.21.15.82:0/3463036243 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f00ae4c0 con 0x7f94f00b24a0 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f4bbe640 1 --2- 172.21.15.82:0/3463036243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f00b24a0 0x7f94f00ad100 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94f006c980 tx=0x7f94e801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe28ae7ec911448d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94eeffd640 1 -- 172.21.15.82:0/3463036243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80a5550 con 0x7f94f00b24a0 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94eeffd640 1 -- 172.21.15.82:0/3463036243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e80a56f0 con 0x7f94f00b24a0 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94eeffd640 1 -- 172.21.15.82:0/3463036243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e80a5ac0 con 0x7f94f00b24a0 2024-04-25T10:03:43.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- 172.21.15.82:0/3463036243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f00b24a0 msgr2=0x7f94f00ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- 172.21.15.82:0/3463036243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f00b24a0 0x7f94f00ad100 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f94f006c980 tx=0x7f94e801c890 comp rx=0 tx=0).stop 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- 172.21.15.82:0/3463036243 shutdown_connections 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- 172.21.15.82:0/3463036243 >> v2:172.21.15.196:3300/0 conn(0x7f94f0071740 0x7f94f00acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- 172.21.15.82:0/3463036243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f00b24a0 0x7f94f00ad100 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- 172.21.15.82:0/3463036243 >> 172.21.15.82:0/3463036243 conn(0x7f94f00b11a0 msgr2=0x7f94f00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- 172.21.15.82:0/3463036243 shutdown_connections 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- 172.21.15.82:0/3463036243 wait complete. 2024-04-25T10:03:43.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 Processor -- start 2024-04-25T10:03:43.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- start start 2024-04-25T10:03:43.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f00b24a0 0x7f94f014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f015da10 0x7f94f015b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f015ef00 con 0x7f94f0071740 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00adc60 con 0x7f94f015da10 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f6621640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f016b0a0 con 0x7f94f00b24a0 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42368/0 (socket says 172.21.15.82:42368) 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 -- 172.21.15.82:0/4076517559 learned_addr learned my addr 172.21.15.82:0/4076517559 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94ef7fe640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f00b24a0 0x7f94f014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 -- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f00b24a0 msgr2=0x7f94f014dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f00b24a0 0x7f94f014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 -- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f015da10 msgr2=0x7f94f015b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94f4bbe640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f015da10 0x7f94f015b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f015da10 0x7f94f015b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 -- 172.21.15.82:0/4076517559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f015cc30 con 0x7f94f0071740 2024-04-25T10:03:43.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.647+0000 7f94effff640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b440 tx=0x7f94e400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4015020 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e4011890 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f015cf10 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4011c80 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f015d1f0 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f004efa0 con 0x7f94f0071740 2024-04-25T10:03:43.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94e4018020 con 0x7f94f0071740 2024-04-25T10:03:43.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 0x7f94dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94e4085020 con 0x7f94f0071740 2024-04-25T10:03:43.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ef7fe640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 0x7f94dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.651+0000 7f94ef7fe640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 0x7f94dc069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94f01fa400 tx=0x7f94e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.655+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f94e401f030 con 0x7f94f0071740 2024-04-25T10:03:43.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f94f00b1dc0 con 0x7f94f0071740 2024-04-25T10:03:43.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94ecff9640 1 -- 172.21.15.82:0/4076517559 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v39) v1 ==== 80+0+0 (secure 0 0 0) 0x7f94e400d040 con 0x7f94f0071740 2024-04-25T10:03:43.830 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:03:43.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 msgr2=0x7f94dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 0x7f94dc069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f94f01fa400 tx=0x7f94e0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 msgr2=0x7f94f015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b440 tx=0x7f94e400b900 comp rx=0 tx=0).stop 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 shutdown_connections 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94dc066f60 0x7f94dc069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f00b24a0 0x7f94f014dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f015da10 0x7f94f015b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 --2- 172.21.15.82:0/4076517559 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f0071740 0x7f94f015c3e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 >> 172.21.15.82:0/4076517559 conn(0x7f94f00b11a0 msgr2=0x7f94f00b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 shutdown_connections 2024-04-25T10:03:43.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:43.827+0000 7f94f6621640 1 -- 172.21.15.82:0/4076517559 wait complete. 2024-04-25T10:03:43.848 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:03:43.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:43.848 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:43.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[39588]: fuse finished with error 0 and tester_r 0 2024-04-25T10:03:43.861 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:49.963 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:03:49.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:49.963 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:49.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:50.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.139+0000 7f464a7f0640 1 Processor -- start 2024-04-25T10:03:50.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.139+0000 7f464a7f0640 1 -- start start 2024-04-25T10:03:50.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f464a7f0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46440ce510 0x7f46440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f464a7f0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46440cee30 con 0x7f46441451f0 2024-04-25T10:03:50.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f464a7f0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46440ccc70 con 0x7f4644144ca0 2024-04-25T10:03:50.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f464a7f0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46440cc2b0 con 0x7f46440ce510 2024-04-25T10:03:50.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f46441451f0 0x7f4644142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56450/0 (socket says 172.21.15.82:56450) 2024-04-25T10:03:50.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 -- 172.21.15.82:0/2387001944 learned_addr learned my addr 172.21.15.82:0/2387001944 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:03:50.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4643fff640 1 --2- 172.21.15.82:0/2387001944 >> v2:172.21.15.196:3300/0 conn(0x7f46440ce510 0x7f46440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511287459 0 0) 0x7f46440cee30 con 0x7f46441451f0 2024-04-25T10:03:50.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f462c003530 con 0x7f46441451f0 2024-04-25T10:03:50.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432375017 0 0) 0x7f462c003530 con 0x7f46441451f0 2024-04-25T10:03:50.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 >> v2:172.21.15.196:3300/0 conn(0x7f46440ce510 msgr2=0x7f46440ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 --2- 172.21.15.82:0/2387001944 >> v2:172.21.15.196:3300/0 conn(0x7f46440ce510 0x7f46440ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 >> v1:172.21.15.196:6789/0 conn(0x7f4644144ca0 legacy=0x7f464406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464406dc40 con 0x7f46441451f0 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f463c001a00 con 0x7f46441451f0 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 >> v1:172.21.15.82:6789/0 conn(0x7f46441451f0 legacy=0x7f4644142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f462c003a30 0x7f462c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462c004810 0x7f462c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f462c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440cc2b0 con 0x7f462c0051d0 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c00c0e0 con 0x7f462c004810 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462c00c2e0 con 0x7f462c003a30 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f462c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f46437fe640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462c004810 0x7f462c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4643fff640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f462c003a30 0x7f462c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 -- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f462c003a30 msgr2=0x7f462c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f462c003a30 0x7f462c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 -- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462c004810 msgr2=0x7f462c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f462c004810 0x7f462c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.143+0000 7f4648d8d640 1 -- 172.21.15.82:0/2387001944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462c003530 con 0x7f462c0051d0 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f4648d8d640 1 -- 172.21.15.82:0/2387001944 reap_dead start 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f4648d8d640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f462c00b9d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f46441456a0 tx=0x7f463c01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834690f78e96e03e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c00a4a0 con 0x7f462c0051d0 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f463c0ad650 con 0x7f462c0051d0 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f4642ffd640 1 -- 172.21.15.82:0/2387001944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463c0ada40 con 0x7f462c0051d0 2024-04-25T10:03:50.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 msgr2=0x7f462c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f462c00b9d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f46441456a0 tx=0x7f463c01d360 comp rx=0 tx=0).stop 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- 172.21.15.82:0/2387001944 shutdown_connections 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 --2- 172.21.15.82:0/2387001944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f462c00b9d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- 172.21.15.82:0/2387001944 >> 172.21.15.82:0/2387001944 conn(0x7f46440b11a0 msgr2=0x7f46440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- 172.21.15.82:0/2387001944 shutdown_connections 2024-04-25T10:03:50.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- 172.21.15.82:0/2387001944 wait complete. 2024-04-25T10:03:50.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 Processor -- start 2024-04-25T10:03:50.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.147+0000 7f464a7f0640 1 -- start start 2024-04-25T10:03:50.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 0x7f46440c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46440c8420 0x7f46440c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46440cc900 con 0x7f462c0051d0 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464406e290 con 0x7f46440c4410 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46440cca80 con 0x7f46440c8420 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37130/0 (socket says 172.21.15.82:37130) 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 -- 172.21.15.82:0/1427286614 learned_addr learned my addr 172.21.15.82:0/1427286614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f46437fe640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 0x7f46440c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4648d8d640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46440c8420 0x7f46440c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 -- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46440c8420 msgr2=0x7f46440c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46440c8420 0x7f46440c88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 -- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 msgr2=0x7f46440c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 0x7f46440c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 -- 172.21.15.82:0/1427286614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46440ca1e0 con 0x7f462c0051d0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f46437fe640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 0x7f46440c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4643fff640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4634011590 tx=0x7f4634011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634015870 con 0x7f462c0051d0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4634015a10 con 0x7f462c0051d0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4634015e00 con 0x7f462c0051d0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46440c4ed0 con 0x7f462c0051d0 2024-04-25T10:03:50.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46440ca480 con 0x7f462c0051d0 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464404efa0 con 0x7f462c0051d0 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.151+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f463401d400 con 0x7f462c0051d0 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.155+0000 7f4640ff9640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.155+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4634085aa0 con 0x7f462c0051d0 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.155+0000 7f46437fe640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.155+0000 7f46437fe640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 0x7f4630069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46440718c0 tx=0x7f4638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.155+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4634019030 con 0x7f462c0051d0 2024-04-25T10:03:50.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.347+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46440b1cc0 con 0x7f462c0051d0 2024-04-25T10:03:50.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f4640ff9640 1 -- 172.21.15.82:0/1427286614 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f4634056a70 con 0x7f462c0051d0 2024-04-25T10:03:50.353 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:50.353 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:50.354 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:03:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 msgr2=0x7f4630069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 0x7f4630069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f46440718c0 tx=0x7f4638009040 comp rx=0 tx=0).stop 2024-04-25T10:03:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 msgr2=0x7f4644142a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4634011590 tx=0x7f4634011a50 comp rx=0 tx=0).stop 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 shutdown_connections 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4630066f60 0x7f4630069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46440c8420 0x7f46440c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46440c4410 0x7f46440c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 --2- 172.21.15.82:0/1427286614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f462c0051d0 0x7f4644142a20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 >> 172.21.15.82:0/1427286614 conn(0x7f46440b11a0 msgr2=0x7f46440ae8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 shutdown_connections 2024-04-25T10:03:50.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.351+0000 7f464a7f0640 1 -- 172.21.15.82:0/1427286614 wait complete. 2024-04-25T10:03:50.371 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:03:50.528 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:03:50.529 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:03:50.529 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:50.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.639+0000 7f18a2abb640 1 Processor -- start 2024-04-25T10:03:50.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 -- start start 2024-04-25T10:03:50.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f189c15da10 0x7f189c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 0x7f189c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c0ae860 con 0x7f189c15da10 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c15e210 con 0x7f189c071e90 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a2abb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c15e4f0 con 0x7f189c0b24a0 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f189c15da10 0x7f189c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f189c15da10 0x7f189c071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37134/0 (socket says 172.21.15.82:37134) 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 -- 172.21.15.82:0/2120539315 learned_addr learned my addr 172.21.15.82:0/2120539315 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a1058640 1 --2- 172.21.15.82:0/2120539315 >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 0x7f189c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 -- 172.21.15.82:0/2120539315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 msgr2=0x7f189c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f189bfff640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 -- 172.21.15.82:0/2120539315 >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 msgr2=0x7f189c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:50.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 --2- 172.21.15.82:0/2120539315 >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 0x7f189c1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 -- 172.21.15.82:0/2120539315 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c0ae4c0 con 0x7f189c15da10 2024-04-25T10:03:50.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a1058640 1 --2- 172.21.15.82:0/2120539315 >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 0x7f189c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:50.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.643+0000 7f18a0857640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c071030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f189c070fc0 tx=0x7f189401c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee81843f385a743 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f189b7fe640 1 -- 172.21.15.82:0/2120539315 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940b1070 con 0x7f189c15da10 2024-04-25T10:03:50.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f189b7fe640 1 -- 172.21.15.82:0/2120539315 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18940ac440 con 0x7f189c15da10 2024-04-25T10:03:50.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f189b7fe640 1 -- 172.21.15.82:0/2120539315 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940ac810 con 0x7f189c15da10 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/2120539315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 msgr2=0x7f189c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c071030 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f189c070fc0 tx=0x7f189401c140 comp rx=0 tx=0).stop 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/2120539315 shutdown_connections 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- 172.21.15.82:0/2120539315 >> v2:172.21.15.196:3300/0 conn(0x7f189c071e90 0x7f189c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- 172.21.15.82:0/2120539315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c071030 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/2120539315 >> 172.21.15.82:0/2120539315 conn(0x7f189c0b11a0 msgr2=0x7f189c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/2120539315 shutdown_connections 2024-04-25T10:03:50.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/2120539315 wait complete. 2024-04-25T10:03:50.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 Processor -- start 2024-04-25T10:03:50.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- start start 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f189c071e90 0x7f189c14fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0acf50 con 0x7f189c15da10 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0acdd0 con 0x7f189c071e90 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0ad0d0 con 0x7f189c0b24a0 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:37150/0 (socket says 172.21.15.82:37150) 2024-04-25T10:03:50.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 -- 172.21.15.82:0/104309304 learned_addr learned my addr 172.21.15.82:0/104309304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a0857640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f189c071e90 0x7f189c14fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f189bfff640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 -- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 msgr2=0x7f189c158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 -- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f189c071e90 msgr2=0x7f189c14fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f189c071e90 0x7f189c14fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 -- 172.21.15.82:0/104309304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c15bef0 con 0x7f189c15da10 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a1058640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f188c00cef0 tx=0x7f188c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c01a070 con 0x7f189c15da10 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c00d840 con 0x7f189c15da10 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c150320 con 0x7f189c15da10 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c00dc30 con 0x7f189c15da10 2024-04-25T10:03:50.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.647+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c15c750 con 0x7f189c15da10 2024-04-25T10:03:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c04efa0 con 0x7f189c15da10 2024-04-25T10:03:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f188c01d070 con 0x7f189c15da10 2024-04-25T10:03:50.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18997fa640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 0x7f187c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f188c085a10 con 0x7f189c15da10 2024-04-25T10:03:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18a0857640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 0x7f187c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:50.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.651+0000 7f18a0857640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 0x7f187c069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18940ae000 tx=0x7f18940a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:50.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.655+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f188c056870 con 0x7f189c15da10 2024-04-25T10:03:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.847+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f189c157100 con 0x7f189c15da10 2024-04-25T10:03:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.847+0000 7f18997fa640 1 -- 172.21.15.82:0/104309304 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f188c085070 con 0x7f189c15da10 2024-04-25T10:03:50.852 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:03:50.852 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 msgr2=0x7f187c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 0x7f187c069420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f18940ae000 tx=0x7f18940a6070 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 msgr2=0x7f189c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f188c00cef0 tx=0x7f188c00a710 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 shutdown_connections 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f187c066f60 0x7f187c069420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f189c0b24a0 0x7f189c158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f189c15da10 0x7f189c06d090 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 --2- 172.21.15.82:0/104309304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f189c071e90 0x7f189c14fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 >> 172.21.15.82:0/104309304 conn(0x7f189c0b11a0 msgr2=0x7f189c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 shutdown_connections 2024-04-25T10:03:50.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:03:50.851+0000 7f18a2abb640 1 -- 172.21.15.82:0/104309304 wait complete. 2024-04-25T10:03:50.870 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parentdir mytag 2024-04-25T10:03:50.996 INFO:teuthology.orchestra.run.smithi082.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T10:03:50.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T10:04:00.998 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T10:04:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.035+0000 7f0ade8cf080 1 Processor -- start 2024-04-25T10:04:01.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.035+0000 7f0ade8cf080 1 -- start start 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583128c4c40 0x55831294d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 0x5583129e6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831287b100 con 0x558312951210 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55831287af80 con 0x5583129515f0 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55831287b280 con 0x5583128c4c40 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51120/0 (socket says 172.21.15.82:51120) 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 -- 172.21.15.82:0/2249172215 learned_addr learned my addr 172.21.15.82:0/2249172215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adcbee640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 0x5583129e6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 -- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583128c4c40 msgr2=0x55831294d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adc3ed640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583128c4c40 0x55831294d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583128c4c40 0x55831294d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 -- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 msgr2=0x5583129e6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 0x5583129e6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 -- 172.21.15.82:0/2249172215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55831295b820 con 0x558312951210 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adcbee640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 0x5583129e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adbbec640 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400a7b0 tx=0x7f0ac400ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72a7efd31483d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adabea640 1 -- 172.21.15.82:0/2249172215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac401a070 con 0x558312951210 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adabea640 1 -- 172.21.15.82:0/2249172215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac400ada0 con 0x558312951210 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adabea640 1 -- 172.21.15.82:0/2249172215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4008bb0 con 0x558312951210 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- 172.21.15.82:0/2249172215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 msgr2=0x5583129e6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0ac400a7b0 tx=0x7f0ac400ac70 comp rx=0 tx=0).stop 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- 172.21.15.82:0/2249172215 shutdown_connections 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583129515f0 0x5583129e6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558312951210 0x5583129e6300 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/2249172215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583128c4c40 0x55831294d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- 172.21.15.82:0/2249172215 >> 172.21.15.82:0/2249172215 conn(0x5583128a6ab0 msgr2=0x558312953ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- 172.21.15.82:0/2249172215 shutdown_connections 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- 172.21.15.82:0/2249172215 wait complete. 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 Processor -- start 2024-04-25T10:04:01.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- start start 2024-04-25T10:04:01.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583128c4c40 0x5583129e7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558312951210 0x55831294c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 0x55831287e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583129e5bc0 con 0x5583129515f0 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583129e5920 con 0x5583128c4c40 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0ade8cf080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55831295bac0 con 0x558312951210 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adcbee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 0x55831287e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adcbee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 0x55831287e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51134/0 (socket says 172.21.15.82:51134) 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.039+0000 7f0adcbee640 1 -- 172.21.15.82:0/4027676457 learned_addr learned my addr 172.21.15.82:0/4027676457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adc3ed640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583128c4c40 0x5583129e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adbbec640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558312951210 0x55831294c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 -- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558312951210 msgr2=0x55831294c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558312951210 0x55831294c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 -- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583128c4c40 msgr2=0x5583129e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583128c4c40 0x5583129e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 -- 172.21.15.82:0/4027676457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583129ed550 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adbbec640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558312951210 0x55831294c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adcbee640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 0x55831287e300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ad401b8a0 tx=0x7f0ad4018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 -- 172.21.15.82:0/4027676457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40bb070 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 -- 172.21.15.82:0/4027676457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad40aae40 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5583129ed7f0 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 -- 172.21.15.82:0/4027676457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40af040 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583129edd30 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 -- 172.21.15.82:0/4027676457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ad40be070 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0abc065940 0x7f0abc067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ad9be8640 1 -- 172.21.15.82:0/4027676457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0ad411ee40 con 0x5583129515f0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adc3ed640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0abc065940 0x7f0abc067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55831294ce00 0x5583129ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x5583129559c0 con 0x55831294ce00 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adbbec640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55831294ce00 0x5583129ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.043+0000 7f0adc3ed640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0abc065940 0x7f0abc067e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005fd0 tx=0x7f0acc007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0adbbec640 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55831294ce00 0x5583129ec4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0adbbec640 1 -- 172.21.15.82:0/4027676457 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f0ac401d040 con 0x55831294ce00 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55831294ce00 msgr2=0x5583129ec4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55831294ce00 0x5583129ec4d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0abc065940 msgr2=0x7f0abc067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0abc065940 0x7f0abc067e00 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0acc005fd0 tx=0x7f0acc007040 comp rx=0 tx=0).stop 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 msgr2=0x55831287e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 --2- 172.21.15.82:0/4027676457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583129515f0 0x55831287e300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ad401b8a0 tx=0x7f0ad4018160 comp rx=0 tx=0).stop 2024-04-25T10:04:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0adcbee640 1 -- 172.21.15.82:0/4027676457 reap_dead start 2024-04-25T10:04:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 shutdown_connections 2024-04-25T10:04:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 >> 172.21.15.82:0/4027676457 conn(0x5583128a6ab0 msgr2=0x5583128ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 shutdown_connections 2024-04-25T10:04:01.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.047+0000 7f0ade8cf080 1 -- 172.21.15.82:0/4027676457 wait complete. 2024-04-25T10:04:01.055 INFO:teuthology.orchestra.run.smithi082.stdout:mytag 2024-04-25T10:04:01.056 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T10:04:01.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 Processor -- start 2024-04-25T10:04:01.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 -- start start 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1e7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da1449f0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da148950 con 0x55e1da272320 2024-04-25T10:04:01.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d0adab080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1da19db90 con 0x55e1da260190 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.127+0000 7f3d088c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51154/0 (socket says 172.21.15.82:51154) 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 -- 172.21.15.82:0/3213445094 learned_addr learned my addr 172.21.15.82:0/3213445094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d080c8640 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 -- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 msgr2=0x55e1da1e7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1e7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 -- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 msgr2=0x55e1da1cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 -- 172.21.15.82:0/3213445094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1da1e1a30 con 0x7f3cf4003ac0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009850 tx=0x7f3cf8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da76a18e5afa0f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d070c6640 1 -- 172.21.15.82:0/3213445094 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8008b90 con 0x7f3cf4003ac0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d070c6640 1 -- 172.21.15.82:0/3213445094 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf801a070 con 0x7f3cf4003ac0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d070c6640 1 -- 172.21.15.82:0/3213445094 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8015460 con 0x7f3cf4003ac0 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/3213445094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 msgr2=0x55e1da1e1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009850 tx=0x7f3cf8005a10 comp rx=0 tx=0).stop 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/3213445094 shutdown_connections 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1e7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- 172.21.15.82:0/3213445094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da1e1fb0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/3213445094 >> 172.21.15.82:0/3213445094 conn(0x55e1da1a3c90 msgr2=0x55e1da273d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/3213445094 shutdown_connections 2024-04-25T10:04:01.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/3213445094 wait complete. 2024-04-25T10:04:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 Processor -- start 2024-04-25T10:04:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- start start 2024-04-25T10:04:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1e3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da30f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da1cfbf0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1da2608b0 con 0x55e1da272320 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1da1e24f0 con 0x55e1da260190 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da30f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da30f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51164/0 (socket says 172.21.15.82:51164) 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 -- 172.21.15.82:0/1112898359 learned_addr learned my addr 172.21.15.82:0/1112898359 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d080c8640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1e3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d088c9640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 -- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 msgr2=0x55e1da1c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e1da260190 0x55e1da1c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 -- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 msgr2=0x55e1da1e3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1e3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 -- 172.21.15.82:0/1112898359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1da1c5bc0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d080c8640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e1da272320 0x55e1da1e3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d090ca640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da30f0b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3d0001b920 tx=0x7f3d000aba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d060c4640 1 -- 172.21.15.82:0/1112898359 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d000012e0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d060c4640 1 -- 172.21.15.82:0/1112898359 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d000b1070 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d060c4640 1 -- 172.21.15.82:0/1112898359 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d00006030 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1da1c5e60 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.131+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1da1c63a0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d060c4640 1 -- 172.21.15.82:0/1112898359 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d000b3030 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d060c4640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ce8065970 0x7f3ce8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d060c4640 1 -- 172.21.15.82:0/1112898359 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3d0011d5b0 con 0x7f3cf4003ac0 2024-04-25T10:04:01.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d088c9640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ce8065970 0x7f3ce8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55e1da1e3f90 0x55e1da1c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x55e1da1c8d50 con 0x55e1da1e3f90 2024-04-25T10:04:01.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d088c9640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ce8065970 0x7f3ce8067e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009740 tx=0x7f3cf8014000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d080c8640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55e1da1e3f90 0x55e1da1c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d080c8640 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55e1da1e3f90 0x55e1da1c34a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d080c8640 1 -- 172.21.15.82:0/1112898359 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f3cf0008040 con 0x55e1da1e3f90 2024-04-25T10:04:01.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55e1da1e3f90 msgr2=0x55e1da1c34a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55e1da1e3f90 0x55e1da1c34a0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ce8065970 msgr2=0x7f3ce8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ce8065970 0x7f3ce8067e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009740 tx=0x7f3cf8014000 comp rx=0 tx=0).stop 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 msgr2=0x55e1da30f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 --2- 172.21.15.82:0/1112898359 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3cf4003ac0 0x55e1da30f0b0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3d0001b920 tx=0x7f3d000aba30 comp rx=0 tx=0).stop 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d090ca640 1 -- 172.21.15.82:0/1112898359 reap_dead start 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 shutdown_connections 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 >> 172.21.15.82:0/1112898359 conn(0x55e1da1a3c90 msgr2=0x55e1da1cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.135+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 shutdown_connections 2024-04-25T10:04:01.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.139+0000 7f3d0adab080 1 -- 172.21.15.82:0/1112898359 wait complete. 2024-04-25T10:04:01.145 INFO:teuthology.orchestra.run.smithi082.stdout:mytag 2024-04-25T10:04:01.145 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:04:01.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 Processor -- start 2024-04-25T10:04:01.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- start start 2024-04-25T10:04:01.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x5562838fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55628385cf80 con 0x55628385d960 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562837d29f0 con 0x5562839042a0 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55628385d220 con 0x556283901060 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51178/0 (socket says 172.21.15.82:51178) 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 -- 172.21.15.82:0/201736098 learned_addr learned my addr 172.21.15.82:0/201736098 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c6faa640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c5fa8640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x5562838fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 -- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 msgr2=0x5562838fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x5562838fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 -- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 msgr2=0x5562838750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838750f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 -- 172.21.15.82:0/201736098 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5562838fbac0 con 0x55628385d960 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c5fa8640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x5562838fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0019780 tx=0x7fc9c00033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f41c09c3e91deb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c4fa6640 1 -- 172.21.15.82:0/201736098 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00ac660 con 0x55628385d960 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c4fa6640 1 -- 172.21.15.82:0/201736098 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00ac800 con 0x55628385d960 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c4fa6640 1 -- 172.21.15.82:0/201736098 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00acbd0 con 0x55628385d960 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/201736098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 msgr2=0x5562838fdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0019780 tx=0x7fc9c00033b0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/201736098 shutdown_connections 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x5562838fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/201736098 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fdbb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/201736098 >> 172.21.15.82:0/201736098 conn(0x556283857c20 msgr2=0x55628382e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/201736098 shutdown_connections 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.227+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/201736098 wait complete. 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 Processor -- start 2024-04-25T10:04:01.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- start start 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x556283909f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562838fbd60 con 0x55628385d960 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5562838fb860 con 0x5562839042a0 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5562838fc000 con 0x556283901060 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51184/0 (socket says 172.21.15.82:51184) 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 -- 172.21.15.82:0/3327809795 learned_addr learned my addr 172.21.15.82:0/3327809795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c6faa640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c5fa8640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x556283909f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 -- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 msgr2=0x556283909f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556283901060 0x556283909f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 -- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 msgr2=0x5562838f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5562839042a0 0x5562838f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 -- 172.21.15.82:0/3327809795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556283906080 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c67a9640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fd730 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0017ce0 tx=0x7fc9c0017e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9b77fe640 1 -- 172.21.15.82:0/3327809795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00ac660 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9b77fe640 1 -- 172.21.15.82:0/3327809795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00c4070 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556283906320 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9b77fe640 1 -- 172.21.15.82:0/3327809795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c00b3c60 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556283906860 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9b77fe640 1 -- 172.21.15.82:0/3327809795 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9c00a9070 con 0x55628385d960 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.231+0000 7fc9b77fe640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9a8065970 0x7fc9a8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9b77fe640 1 -- 172.21.15.82:0/3327809795 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc9c0120000 con 0x55628385d960 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c5fa8640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9a8065970 0x7fc9a8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55628390a4a0 0x5562839074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x556283905c20 con 0x55628390a4a0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c5fa8640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9a8065970 0x7fc9a8067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005620 tx=0x7fc9bc005530 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c6faa640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55628390a4a0 0x5562839074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c6faa640 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55628390a4a0 0x5562839074b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c6faa640 1 -- 172.21.15.82:0/3327809795 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fc9b8003020 con 0x55628390a4a0 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55628390a4a0 msgr2=0x5562839074b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55628390a4a0 0x5562839074b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9a8065970 msgr2=0x7fc9a8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9a8065970 0x7fc9a8067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc005620 tx=0x7fc9bc005530 comp rx=0 tx=0).stop 2024-04-25T10:04:01.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 msgr2=0x5562838fd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 --2- 172.21.15.82:0/3327809795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55628385d960 0x5562838fd730 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0017ce0 tx=0x7fc9c0017e70 comp rx=0 tx=0).stop 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c6faa640 1 -- 172.21.15.82:0/3327809795 reap_dead start 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 shutdown_connections 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 >> 172.21.15.82:0/3327809795 conn(0x556283857c20 msgr2=0x55628382faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 shutdown_connections 2024-04-25T10:04:01.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.235+0000 7fc9c8c8b080 1 -- 172.21.15.82:0/3327809795 wait complete. 2024-04-25T10:04:01.244 INFO:teuthology.orchestra.run.smithi082.stdout:mytag 2024-04-25T10:04:01.244 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:04:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 Processor -- start 2024-04-25T10:04:01.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 -- start start 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af11f360 0x5607af187650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af1a91f0 0x5607af23abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607af105800 con 0x5607af0d1570 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607af0789f0 con 0x5607af11f360 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9650c080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607af105980 con 0x5607af1a91f0 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51196/0 (socket says 172.21.15.82:51196) 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 -- 172.21.15.82:0/1702486153 learned_addr learned my addr 172.21.15.82:0/1702486153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9482b640 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af1a91f0 0x5607af23abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b93829640 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af11f360 0x5607af187650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 -- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af1a91f0 msgr2=0x5607af23abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af1a91f0 0x5607af23abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 -- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af11f360 msgr2=0x5607af187650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af11f360 0x5607af187650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 -- 172.21.15.82:0/1702486153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607af116620 con 0x5607af0d1570 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b9402a640 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400a7e0 tx=0x7f3b8400aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513e6d7e6e14bc00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b92827640 1 -- 172.21.15.82:0/1702486153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84008c00 con 0x5607af0d1570 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b92827640 1 -- 172.21.15.82:0/1702486153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b84008da0 con 0x5607af0d1570 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.319+0000 7f3b92827640 1 -- 172.21.15.82:0/1702486153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b84013610 con 0x5607af0d1570 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/1702486153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 msgr2=0x5607af185150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400a7e0 tx=0x7f3b8400aca0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/1702486153 shutdown_connections 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af1a91f0 0x5607af23abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af11f360 0x5607af187650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- 172.21.15.82:0/1702486153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af0d1570 0x5607af185150 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/1702486153 >> 172.21.15.82:0/1702486153 conn(0x5607af0d7c90 msgr2=0x5607af1a4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/1702486153 shutdown_connections 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/1702486153 wait complete. 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 Processor -- start 2024-04-25T10:04:01.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- start start 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af0d1570 0x5607af189ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 0x5607af1a82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af1a91f0 0x5607af188630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607af1168c0 con 0x5607af11f360 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5607af186cb0 con 0x5607af1a91f0 2024-04-25T10:04:01.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5607af186f10 con 0x5607af0d1570 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 0x5607af1a82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 0x5607af1a82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51210/0 (socket says 172.21.15.82:51210) 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 -- 172.21.15.82:0/3307785319 learned_addr learned my addr 172.21.15.82:0/3307785319 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9482b640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af1a91f0 0x5607af188630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9402a640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af0d1570 0x5607af189ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 -- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af0d1570 msgr2=0x5607af189ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af0d1570 0x5607af189ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 -- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af1a91f0 msgr2=0x5607af188630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5607af1a91f0 0x5607af188630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 -- 172.21.15.82:0/3307785319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5607af188fb0 con 0x5607af11f360 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9402a640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5607af0d1570 0x5607af189ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b93829640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 0x5607af1a82e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800df90 tx=0x7f3b88003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b91825640 1 -- 172.21.15.82:0/3307785319 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88014020 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b91825640 1 -- 172.21.15.82:0/3307785319 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b880032e0 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5607af189250 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b91825640 1 -- 172.21.15.82:0/3307785319 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800a040 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.323+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5607af189a30 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b91825640 1 -- 172.21.15.82:0/3307785319 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b88017020 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b91825640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b80065970 0x7f3b80067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b91825640 1 -- 172.21.15.82:0/3307785319 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3b88083d70 con 0x5607af11f360 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9402a640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b80065970 0x7f3b80067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5607af119d90 0x5607af241ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x5607af188b70 con 0x5607af119d90 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9402a640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b80065970 0x7f3b80067e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400ace0 tx=0x7f3b8400c000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9482b640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5607af119d90 0x5607af241ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9482b640 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5607af119d90 0x5607af241ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9482b640 1 -- 172.21.15.82:0/3307785319 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f3b8c0aa490 con 0x5607af119d90 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5607af119d90 msgr2=0x5607af241ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5607af119d90 0x5607af241ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b80065970 msgr2=0x7f3b80067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b80065970 0x7f3b80067e30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b8400ace0 tx=0x7f3b8400c000 comp rx=0 tx=0).stop 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 msgr2=0x5607af1a82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 --2- 172.21.15.82:0/3307785319 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5607af11f360 0x5607af1a82e0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800df90 tx=0x7f3b88003ee0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9482b640 1 -- 172.21.15.82:0/3307785319 reap_dead start 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 shutdown_connections 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 >> 172.21.15.82:0/3307785319 conn(0x5607af0d7c90 msgr2=0x5607af185150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 shutdown_connections 2024-04-25T10:04:01.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.327+0000 7f3b9650c080 1 -- 172.21.15.82:0/3307785319 wait complete. 2024-04-25T10:04:01.336 INFO:teuthology.orchestra.run.smithi082.stdout:mytag 2024-04-25T10:04:01.337 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:04:01.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 Processor -- start 2024-04-25T10:04:01.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 -- start start 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1ba6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1b9e970 con 0x561ce1b9d210 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1a6e9f0 con 0x561ce1b10c40 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.415+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ce1b9eaf0 con 0x561ce1b9d5f0 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51238/0 (socket says 172.21.15.82:51238) 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/2803298389 learned_addr learned my addr 172.21.15.82:0/2803298389 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7975640 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d8176640 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1ba6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 msgr2=0x561ce1ba6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1ba6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 msgr2=0x561ce1ba17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/2803298389 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce1ba7740 con 0x561ce1b9d210 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0018fb0 tx=0x7fe4c0003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6533be8fbb7a3408 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d6172640 1 -- 172.21.15.82:0/2803298389 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00af040 con 0x561ce1b9d210 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d6172640 1 -- 172.21.15.82:0/2803298389 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c00b9070 con 0x561ce1b9d210 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d6172640 1 -- 172.21.15.82:0/2803298389 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00aad50 con 0x561ce1b9d210 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/2803298389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 msgr2=0x561ce1ba6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0018fb0 tx=0x7fe4c0003030 comp rx=0 tx=0).stop 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/2803298389 shutdown_connections 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1ba6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1ba6690 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/2803298389 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/2803298389 >> 172.21.15.82:0/2803298389 conn(0x561ce1af2ab0 msgr2=0x561ce1acb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/2803298389 shutdown_connections 2024-04-25T10:04:01.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/2803298389 wait complete. 2024-04-25T10:04:01.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 Processor -- start 2024-04-25T10:04:01.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- start start 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1b92b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1b96720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1ac9f40 con 0x561ce1b9d210 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ce1ba79e0 con 0x561ce1b10c40 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ce1aca1e0 con 0x561ce1b9d5f0 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1b92b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1b92b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51252/0 (socket says 172.21.15.82:51252) 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/1826838749 learned_addr learned my addr 172.21.15.82:0/1826838749 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d8176640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1b96720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7975640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 msgr2=0x561ce1b96720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1b96720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 msgr2=0x561ce1ba1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ce1b10c40 0x561ce1ba1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 -- 172.21.15.82:0/1826838749 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ce1b977d0 con 0x561ce1b9d210 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d8176640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ce1b9d5f0 0x561ce1b96720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d7174640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1b92b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00b6040 tx=0x7fe4c0018120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d5170640 1 -- 172.21.15.82:0/1826838749 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00af040 con 0x561ce1b9d210 2024-04-25T10:04:01.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d5170640 1 -- 172.21.15.82:0/1826838749 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c00b4820 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ce1b0ed80 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.419+0000 7fe4d5170640 1 -- 172.21.15.82:0/1826838749 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00b4bd0 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ce1b0f2c0 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d5170640 1 -- 172.21.15.82:0/1826838749 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4c00c5020 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d5170640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe4b8065940 0x7fe4b8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d5170640 1 -- 172.21.15.82:0/1826838749 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe4c01205c0 con 0x561ce1b9d210 2024-04-25T10:04:01.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d7975640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe4b8065940 0x7fe4b8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x561ce1b93870 0x561ce1b93c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) v9 -- 0x561ce1b94190 con 0x561ce1b93870 2024-04-25T10:04:01.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d7975640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe4b8065940 0x7fe4b8067e00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005fd0 tx=0x7fe4d0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d8176640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x561ce1b93870 0x561ce1b93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d8176640 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x561ce1b93870 0x561ce1b93c50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:01.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d8176640 1 -- 172.21.15.82:0/1826838749 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe4c800d430 con 0x561ce1b93870 2024-04-25T10:04:01.433 INFO:teuthology.orchestra.run.smithi082.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T10:04:01.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x561ce1b93870 msgr2=0x561ce1b93c50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.423+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x561ce1b93870 0x561ce1b93c50 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe4b8065940 msgr2=0x7fe4b8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe4b8065940 0x7fe4b8067e00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0005fd0 tx=0x7fe4d0007040 comp rx=0 tx=0).stop 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 msgr2=0x561ce1b92b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 --2- 172.21.15.82:0/1826838749 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ce1b9d210 0x561ce1b92b90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fe4c00b6040 tx=0x7fe4c0018120 comp rx=0 tx=0).stop 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d8176640 1 -- 172.21.15.82:0/1826838749 reap_dead start 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 shutdown_connections 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 >> 172.21.15.82:0/1826838749 conn(0x561ce1af2ab0 msgr2=0x561ce1b9eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 shutdown_connections 2024-04-25T10:04:01.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.427+0000 7fe4d9e57080 1 -- 172.21.15.82:0/1826838749 wait complete. 2024-04-25T10:04:01.435 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:01.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:01.507 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:01.563 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:01.563 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:04:01.564 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:01.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:01.569 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:04:01.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.570 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:04:01.617 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:04:01.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:01.619 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:04:01.619 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:01.624 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:01.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:04:01.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.624 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:04:01.670 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:04:01.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:01.671 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:04:01.671 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:01.677 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:01.678 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:04:01.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.678 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:04:01.725 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:04:01.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:01.726 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:04:01.727 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:01.733 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:01.733 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:04:01.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:01.733 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:04:01.782 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:04:01.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:04:01.783 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.907+0000 7fe03e470640 1 Processor -- start 2024-04-25T10:04:01.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.907+0000 7fe03e470640 1 -- start start 2024-04-25T10:04:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03e470640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe038142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03e470640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0380ccc70 con 0x7fe0380ce510 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03e470640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03806dc40 con 0x7fe038144ca0 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03e470640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc2b0 con 0x7fe0381451f0 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe037fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe0380ce510 0x7fe0380ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37464/0 (socket says 172.21.15.82:37464) 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe037fff640 1 -- 172.21.15.82:0/267172935 learned_addr learned my addr 172.21.15.82:0/267172935 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748319909 0 0) 0x7fe0380ccc70 con 0x7fe0380ce510 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe020003530 con 0x7fe0380ce510 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe038142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3465067122 0 0) 0x7fe020003530 con 0x7fe0380ce510 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 msgr2=0x7fe038142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe038142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 >> v1:172.21.15.196:6789/0 conn(0x7fe038144ca0 legacy=0x7fe03806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03806df20 con 0x7fe0380ce510 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe028001680 con 0x7fe0380ce510 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 >> v1:172.21.15.82:6789/0 conn(0x7fe0380ce510 legacy=0x7fe0380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe020003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe020004090 0x7fe020004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe020004da0 0x7fe020005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc2b0 con 0x7fe020004da0 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02000bc70 con 0x7fe020004090 2024-04-25T10:04:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02000be70 con 0x7fe0381451f0 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe020004da0 0x7fe020005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe0377fe640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe020004090 0x7fe020004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe037fff640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe020003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 -- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 msgr2=0x7fe020003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe020003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 -- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe020004090 msgr2=0x7fe020004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe020004090 0x7fe020004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe03ca0d640 1 -- 172.21.15.82:0/267172935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020003530 con 0x7fe020004da0 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe037fff640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0381451f0 0x7fe020003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.911+0000 7fe0377fe640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe020004090 0x7fe020004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03ca0d640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe020004da0 0x7fe020005220 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe030007360 tx=0x7fe03001ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c692e07e8f6bb4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300b2070 con 0x7fe020004da0 2024-04-25T10:04:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300a5d40 con 0x7fe020004da0 2024-04-25T10:04:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe036ffd640 1 -- 172.21.15.82:0/267172935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030009be0 con 0x7fe020004da0 2024-04-25T10:04:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- 172.21.15.82:0/267172935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe020004da0 msgr2=0x7fe020005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 --2- 172.21.15.82:0/267172935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe020004da0 0x7fe020005220 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe030007360 tx=0x7fe03001ce40 comp rx=0 tx=0).stop 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03ca0d640 1 -- 172.21.15.82:0/267172935 reap_dead start 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- 172.21.15.82:0/267172935 shutdown_connections 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- 172.21.15.82:0/267172935 >> 172.21.15.82:0/267172935 conn(0x7fe0380b11a0 msgr2=0x7fe0380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- 172.21.15.82:0/267172935 shutdown_connections 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- 172.21.15.82:0/267172935 wait complete. 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 Processor -- start 2024-04-25T10:04:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.915+0000 7fe03e470640 1 -- start start 2024-04-25T10:04:01.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0380ae120 0x7fe0381e1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0380acb00 0x7fe0381e16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc900 con 0x7fe038134e80 2024-04-25T10:04:01.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03806e290 con 0x7fe0380ae120 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe038142680 con 0x7fe0380acb00 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51270/0 (socket says 172.21.15.82:51270) 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 -- 172.21.15.82:0/1776859905 learned_addr learned my addr 172.21.15.82:0/1776859905 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe0377fe640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0380ae120 0x7fe0381e1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 -- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0380acb00 msgr2=0x7fe0381e16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0380acb00 0x7fe0381e16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 -- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0380ae120 msgr2=0x7fe0381e1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0380ae120 0x7fe0381e1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 -- 172.21.15.82:0/1776859905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe038135260 con 0x7fe038134e80 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe037fff640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe02800eb50 tx=0x7fe02800c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028019070 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02800ee50 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0380ad080 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02800cd80 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0380ad640 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03804efa0 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0280048b0 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 0x7fe024069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe028084b70 con 0x7fe038134e80 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.919+0000 7fe0377fe640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 0x7fe024069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.923+0000 7fe0377fe640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 0x7fe024069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0380ab5d0 tx=0x7fe02c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:01.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:01.927+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe028055380 con 0x7fe038134e80 2024-04-25T10:04:02.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.119+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe03806c030 con 0x7fe038134e80 2024-04-25T10:04:02.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.119+0000 7fe034ff9640 1 -- 172.21.15.82:0/1776859905 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fe028059420 con 0x7fe038134e80 2024-04-25T10:04:02.123 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:02.123 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:02.124 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:04:02.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 msgr2=0x7fe024069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 0x7fe024069450 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0380ab5d0 tx=0x7fe02c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:02.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 msgr2=0x7fe0380ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe02800eb50 tx=0x7fe02800c4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 shutdown_connections 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe024066f90 0x7fe024069450 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0380acb00 0x7fe0381e16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0380ae120 0x7fe0381e1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 --2- 172.21.15.82:0/1776859905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe038134e80 0x7fe0380ac5c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 >> 172.21.15.82:0/1776859905 conn(0x7fe0380b11a0 msgr2=0x7fe038142c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 shutdown_connections 2024-04-25T10:04:02.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.123+0000 7fe03e470640 1 -- 172.21.15.82:0/1776859905 wait complete. 2024-04-25T10:04:02.141 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:02.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 Processor -- start 2024-04-25T10:04:02.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 -- start start 2024-04-25T10:04:02.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4ba415a700 0x7f4ba415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba40b2090 con 0x7f4ba40b3f50 2024-04-25T10:04:02.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba40b43e0 con 0x7f4ba40b24a0 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4baa741640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba415f200 con 0x7f4ba415a700 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4ba415a700 0x7f4ba415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4ba40b3f50 0x7f4ba40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37478/0 (socket says 172.21.15.82:37478) 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba37fe640 1 -- 172.21.15.82:0/2554739727 learned_addr learned my addr 172.21.15.82:0/2554739727 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba3fff640 1 -- 172.21.15.82:0/2554739727 >> v1:172.21.15.196:6789/0 conn(0x7f4ba40b24a0 legacy=0x7f4ba406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba3fff640 1 -- 172.21.15.82:0/2554739727 >> v1:172.21.15.82:6789/0 conn(0x7f4ba40b3f50 legacy=0x7f4ba40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba3fff640 1 -- 172.21.15.82:0/2554739727 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba40b46c0 con 0x7f4ba415a700 2024-04-25T10:04:02.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.259+0000 7f4ba2ffd640 1 -- 172.21.15.82:0/2554739727 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471940980 0 0) 0x7f4ba40b2090 con 0x7f4ba40b3f50 2024-04-25T10:04:02.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba3fff640 1 --2- 172.21.15.82:0/2554739727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba415a700 0x7f4ba415ecc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4b940097c0 tx=0x7f4b9401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ccb4ebf9c479f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba2ffd640 1 -- 172.21.15.82:0/2554739727 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940b0070 con 0x7f4ba415a700 2024-04-25T10:04:02.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba2ffd640 1 -- 172.21.15.82:0/2554739727 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94004230 con 0x7f4ba415a700 2024-04-25T10:04:02.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba2ffd640 1 -- 172.21.15.82:0/2554739727 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94003620 con 0x7f4ba415a700 2024-04-25T10:04:02.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- 172.21.15.82:0/2554739727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba415a700 msgr2=0x7f4ba415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 --2- 172.21.15.82:0/2554739727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba415a700 0x7f4ba415ecc0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4b940097c0 tx=0x7f4b9401c350 comp rx=0 tx=0).stop 2024-04-25T10:04:02.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- 172.21.15.82:0/2554739727 shutdown_connections 2024-04-25T10:04:02.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 --2- 172.21.15.82:0/2554739727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba415a700 0x7f4ba415ecc0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- 172.21.15.82:0/2554739727 >> 172.21.15.82:0/2554739727 conn(0x7f4ba40b11a0 msgr2=0x7f4ba40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- 172.21.15.82:0/2554739727 shutdown_connections 2024-04-25T10:04:02.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- 172.21.15.82:0/2554739727 wait complete. 2024-04-25T10:04:02.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 Processor -- start 2024-04-25T10:04:02.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- start start 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 0x7f4ba4156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba413fd00 0x7f4ba4158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba40cd310 con 0x7f4ba40b24a0 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba40ce600 con 0x7f4ba415a700 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4baa741640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba40cd490 con 0x7f4ba413fd00 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51290/0 (socket says 172.21.15.82:51290) 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.263+0000 7f4ba3fff640 1 -- 172.21.15.82:0/595862863 learned_addr learned my addr 172.21.15.82:0/595862863 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:02.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba37fe640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 0x7f4ba4156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba8cde640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba413fd00 0x7f4ba4158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 -- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba413fd00 msgr2=0x7f4ba4158dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba413fd00 0x7f4ba4158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 -- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 msgr2=0x7f4ba4156f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 0x7f4ba4156f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 -- 172.21.15.82:0/595862863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba41576a0 con 0x7f4ba40b24a0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba37fe640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 0x7f4ba4156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba3fff640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4b940ae040 tx=0x7f4b940190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94003b70 con 0x7f4ba40b24a0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b94003d10 con 0x7f4ba40b24a0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4157980 con 0x7f4ba40b24a0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b940bb680 con 0x7f4ba40b24a0 2024-04-25T10:04:02.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba4157f40 con 0x7f4ba40b24a0 2024-04-25T10:04:02.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba404efa0 con 0x7f4ba40b24a0 2024-04-25T10:04:02.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b940c4440 con 0x7f4ba40b24a0 2024-04-25T10:04:02.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 0x7f4b90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4b94121260 con 0x7f4ba40b24a0 2024-04-25T10:04:02.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba37fe640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 0x7f4b90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.267+0000 7f4ba37fe640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 0x7f4b90069450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ba41405c0 tx=0x7f4b98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.271+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b94016090 con 0x7f4ba40b24a0 2024-04-25T10:04:02.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.435+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ba406f260 con 0x7f4ba40b24a0 2024-04-25T10:04:02.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4ba0ff9640 1 -- 172.21.15.82:0/595862863 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f4b940f2030 con 0x7f4ba40b24a0 2024-04-25T10:04:02.442 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:02.442 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:49.436349+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 msgr2=0x7f4b90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 0x7f4b90069450 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4ba41405c0 tx=0x7f4b98009040 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 msgr2=0x7f4ba415d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4b940ae040 tx=0x7f4b940190f0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 shutdown_connections 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b90066f90 0x7f4b90069450 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ba413fd00 0x7f4ba4158dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ba415a700 0x7f4ba4156f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 --2- 172.21.15.82:0/595862863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ba40b24a0 0x7f4ba415d250 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 >> 172.21.15.82:0/595862863 conn(0x7f4ba40b11a0 msgr2=0x7f4ba40af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 shutdown_connections 2024-04-25T10:04:02.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.439+0000 7f4baa741640 1 -- 172.21.15.82:0/595862863 wait complete. 2024-04-25T10:04:02.461 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:29.159335+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:29.660329+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:02.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.571+0000 7f3e55201640 1 Processor -- start 2024-04-25T10:04:02.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.571+0000 7f3e55201640 1 -- start start 2024-04-25T10:04:02.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e55201640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 0x7f3e5015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e55201640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e55201640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e500ca4c0 con 0x7f3e500c9ef0 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e55201640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e500ca780 con 0x7f3e500cf590 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e55201640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500cd440 con 0x7f3e500cf970 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4ed9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 0x7f3e5015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:54894/0 (socket says 172.21.15.82:54894) 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4ed9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 0x7f3e5015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:33720/0 (socket says 172.21.15.82:33720) 2024-04-25T10:04:02.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 -- 172.21.15.82:0/3499685338 learned_addr learned my addr 172.21.15.82:0/3499685338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 -- 172.21.15.82:0/3499685338 >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 msgr2=0x7f3e5015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 --2- 172.21.15.82:0/3499685338 >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 0x7f3e5015ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 -- 172.21.15.82:0/3499685338 >> v1:172.21.15.196:6789/0 conn(0x7f3e500c9ef0 legacy=0x7f3e500b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 -- 172.21.15.82:0/3499685338 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e500cd690 con 0x7f3e500cf970 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4e59c640 1 --2- 172.21.15.82:0/3499685338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400bb10 tx=0x7f3e4401ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0176ac24269da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4dd9b640 1 -- 172.21.15.82:0/3499685338 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440b4070 con 0x7f3e500cf970 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4dd9b640 1 -- 172.21.15.82:0/3499685338 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e440a7b30 con 0x7f3e500cf970 2024-04-25T10:04:02.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.575+0000 7f3e4dd9b640 1 -- 172.21.15.82:0/3499685338 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e440af440 con 0x7f3e500cf970 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/3499685338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 msgr2=0x7f3e500c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- 172.21.15.82:0/3499685338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3e4400bb10 tx=0x7f3e4401ec30 comp rx=0 tx=0).stop 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/3499685338 shutdown_connections 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- 172.21.15.82:0/3499685338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e500c99b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- 172.21.15.82:0/3499685338 >> v2:172.21.15.196:3300/0 conn(0x7f3e500cf590 0x7f3e5015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/3499685338 >> 172.21.15.82:0/3499685338 conn(0x7f3e5006ae60 msgr2=0x7f3e5006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/3499685338 shutdown_connections 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/3499685338 wait complete. 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 Processor -- start 2024-04-25T10:04:02.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- start start 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e500cf590 0x7f3e5015be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e501f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500cbf60 con 0x7f3e500c9ef0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500cbde0 con 0x7f3e500cf590 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500cc0e0 con 0x7f3e500cf970 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51298/0 (socket says 172.21.15.82:51298) 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 -- 172.21.15.82:0/143116757 learned_addr learned my addr 172.21.15.82:0/143116757 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4e59c640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e500cf590 0x7f3e5015be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 -- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 msgr2=0x7f3e501f9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4f59e640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e501f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e501f9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 -- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e500cf590 msgr2=0x7f3e5015be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e500cf590 0x7f3e5015be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 -- 172.21.15.82:0/143116757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e5015c6b0 con 0x7f3e500c9ef0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e4ed9d640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000e960 tx=0x7f3e4000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e40017070 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000ca00 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5015c990 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000cdb0 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.579+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5015cf50 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5004efa0 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e4001a070 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e377fe640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f3e400855f0 con 0x7f3e500c9ef0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e4e59c640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.583+0000 7f3e4e59c640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 0x7f3e3c069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e440b1000 tx=0x7f3e44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.587+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e40056450 con 0x7f3e500c9ef0 2024-04-25T10:04:02.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.735+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e500b4170 con 0x7f3e500c9ef0 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.735+0000 7f3e377fe640 1 -- 172.21.15.82:0/143116757 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e40055d50 con 0x7f3e500c9ef0 2024-04-25T10:04:02.740 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 msgr2=0x7f3e3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 0x7f3e3c069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3e440b1000 tx=0x7f3e44008040 comp rx=0 tx=0).stop 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 msgr2=0x7f3e500c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f3e4000e960 tx=0x7f3e4000ee20 comp rx=0 tx=0).stop 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 shutdown_connections 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e3c066f60 0x7f3e3c069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e500cf970 0x7f3e501f9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e500cf590 0x7f3e5015be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 --2- 172.21.15.82:0/143116757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e500c9ef0 0x7f3e500c3f30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 >> 172.21.15.82:0/143116757 conn(0x7f3e5006ae60 msgr2=0x7f3e5006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 shutdown_connections 2024-04-25T10:04:02.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.739+0000 7f3e55201640 1 -- 172.21.15.82:0/143116757 wait complete. 2024-04-25T10:04:02.757 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:04:02.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 Processor -- start 2024-04-25T10:04:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 -- start start 2024-04-25T10:04:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 0x7fb9d006c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d00ca3e0 con 0x7fb9d015d9e0 2024-04-25T10:04:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d006c630 con 0x7fb9d0066a80 2024-04-25T10:04:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d86c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d006b6a0 con 0x7fb9d015ddc0 2024-04-25T10:04:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d5c62640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 0x7fb9d006c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:54912/0 (socket says 172.21.15.82:54912) 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 -- 172.21.15.82:0/394238933 learned_addr learned my addr 172.21.15.82:0/394238933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 -- 172.21.15.82:0/394238933 >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 msgr2=0x7fb9d006c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 --2- 172.21.15.82:0/394238933 >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 0x7fb9d006c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 -- 172.21.15.82:0/394238933 >> v1:172.21.15.196:6789/0 conn(0x7fb9d015d9e0 legacy=0x7fb9d00cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d6c64640 1 -- 172.21.15.82:0/394238933 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00ca6c0 con 0x7fb9d015ddc0 2024-04-25T10:04:02.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.875+0000 7fb9d5c62640 1 --2- 172.21.15.82:0/394238933 >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 0x7fb9d006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:02.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d6c64640 1 --2- 172.21.15.82:0/394238933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009e50 tx=0x7fb9c801ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09568875fb10486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d5461640 1 -- 172.21.15.82:0/394238933 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80a5700 con 0x7fb9d015ddc0 2024-04-25T10:04:02.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d5461640 1 -- 172.21.15.82:0/394238933 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c80a58a0 con 0x7fb9d015ddc0 2024-04-25T10:04:02.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d5461640 1 -- 172.21.15.82:0/394238933 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c80a5c70 con 0x7fb9d015ddc0 2024-04-25T10:04:02.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- 172.21.15.82:0/394238933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 msgr2=0x7fb9d00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/394238933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8009e50 tx=0x7fb9c801ca30 comp rx=0 tx=0).stop 2024-04-25T10:04:02.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- 172.21.15.82:0/394238933 shutdown_connections 2024-04-25T10:04:02.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/394238933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d015ddc0 0x7fb9d00cefc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/394238933 >> v2:172.21.15.196:3300/0 conn(0x7fb9d0066a80 0x7fb9d006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- 172.21.15.82:0/394238933 >> 172.21.15.82:0/394238933 conn(0x7fb9d00cc670 msgr2=0x7fb9d00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- 172.21.15.82:0/394238933 shutdown_connections 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- 172.21.15.82:0/394238933 wait complete. 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 Processor -- start 2024-04-25T10:04:02.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.879+0000 7fb9d86c7640 1 -- start start 2024-04-25T10:04:02.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d0066a80 0x7fb9d0157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9d015ddc0 0x7fb9d00c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d006e3b0 con 0x7fb9d015d9e0 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d006e230 con 0x7fb9d015ddc0 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d00cf910 con 0x7fb9d0066a80 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51300/0 (socket says 172.21.15.82:51300) 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 -- 172.21.15.82:0/2509166988 learned_addr learned my addr 172.21.15.82:0/2509166988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d6463640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d0066a80 0x7fb9d0157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d6c64640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9d015ddc0 0x7fb9d00c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 -- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d0066a80 msgr2=0x7fb9d0157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d0066a80 0x7fb9d0157f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 -- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9d015ddc0 msgr2=0x7fb9d00c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9d015ddc0 0x7fb9d00c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 -- 172.21.15.82:0/2509166988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d00683d0 con 0x7fb9d015d9e0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d5c62640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00ec10 tx=0x7fb9cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc017070 con 0x7fb9d015d9e0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9cc00cc30 con 0x7fb9d015d9e0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9cc013470 con 0x7fb9d015d9e0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d006b490 con 0x7fb9d015d9e0 2024-04-25T10:04:02.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d00c8f80 con 0x7fb9d015d9e0 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d004efa0 con 0x7fb9d015d9e0 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9cc01a070 con 0x7fb9d015d9e0 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 0x7fb9b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb9cc085150 con 0x7fb9d015d9e0 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d6463640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 0x7fb9b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:02.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.883+0000 7fb9d6463640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 0x7fb9b4069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001650 tx=0x7fb9c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:02.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:02.887+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9cc056040 con 0x7fb9d015d9e0 2024-04-25T10:04:03.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9d0160560 con 0x7fb9d015d9e0 2024-04-25T10:04:03.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9c6ffd640 1 -- 172.21.15.82:0/2509166988 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9cc055940 con 0x7fb9d015d9e0 2024-04-25T10:04:03.039 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 msgr2=0x7fb9b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 0x7fb9b4069420 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0001650 tx=0x7fb9c0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 msgr2=0x7fb9d00c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb9cc00ec10 tx=0x7fb9cc00c520 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 shutdown_connections 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb9b4066f60 0x7fb9b4069420 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb9d015ddc0 0x7fb9d00c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9d015d9e0 0x7fb9d00c8130 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 --2- 172.21.15.82:0/2509166988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9d0066a80 0x7fb9d0157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 >> 172.21.15.82:0/2509166988 conn(0x7fb9d00cc670 msgr2=0x7fb9d00cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 shutdown_connections 2024-04-25T10:04:03.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.035+0000 7fb9d86c7640 1 -- 172.21.15.82:0/2509166988 wait complete. 2024-04-25T10:04:03.056 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:04:03.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 Processor -- start 2024-04-25T10:04:03.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 -- start start 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dec15b950 0x7f6dec09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 0x7f6dec0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dec0d04d0 con 0x7f6dec0cf420 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec15bd30 con 0x7f6dec15b950 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df3fe4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec0cc0d0 con 0x7f6dec0cfb80 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dec15b950 0x7f6dec09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6dec15b950 0x7f6dec09a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51308/0 (socket says 172.21.15.82:51308) 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 -- 172.21.15.82:0/273361180 learned_addr learned my addr 172.21.15.82:0/273361180 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df157f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6dec0cf420 0x7f6dec0cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37494/0 (socket says 172.21.15.82:37494) 2024-04-25T10:04:03.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df2581640 1 --2- 172.21.15.82:0/273361180 >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 0x7f6dec0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 -- 172.21.15.82:0/273361180 >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 msgr2=0x7f6dec0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 --2- 172.21.15.82:0/273361180 >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 0x7f6dec0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 -- 172.21.15.82:0/273361180 >> v1:172.21.15.82:6789/0 conn(0x7f6dec0cf420 legacy=0x7f6dec0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df0d7e640 1 -- 172.21.15.82:0/273361180 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033814744 0 0) 0x7f6de8001120 con 0x7f6dec0cf420 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df1d80640 1 -- 172.21.15.82:0/273361180 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec09ab10 con 0x7f6dec15b950 2024-04-25T10:04:03.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.163+0000 7f6df2581640 1 --2- 172.21.15.82:0/273361180 >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 0x7f6dec0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:03.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df1d80640 1 --2- 172.21.15.82:0/273361180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec09a480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0b3bd0 tx=0x7f6ddc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbb091fc3bfd8a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df0d7e640 1 -- 172.21.15.82:0/273361180 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0ab610 con 0x7f6dec15b950 2024-04-25T10:04:03.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df0d7e640 1 -- 172.21.15.82:0/273361180 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc0b0070 con 0x7f6dec15b950 2024-04-25T10:04:03.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df0d7e640 1 -- 172.21.15.82:0/273361180 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0ab9d0 con 0x7f6dec15b950 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- 172.21.15.82:0/273361180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 msgr2=0x7f6dec09a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/273361180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec09a480 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6dec0b3bd0 tx=0x7f6ddc01c520 comp rx=0 tx=0).stop 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- 172.21.15.82:0/273361180 shutdown_connections 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/273361180 >> v2:172.21.15.196:3300/0 conn(0x7f6dec0cfb80 0x7f6dec0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/273361180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec09a480 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- 172.21.15.82:0/273361180 >> 172.21.15.82:0/273361180 conn(0x7f6dec0b30b0 msgr2=0x7f6dec0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- 172.21.15.82:0/273361180 shutdown_connections 2024-04-25T10:04:03.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- 172.21.15.82:0/273361180 wait complete. 2024-04-25T10:04:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 Processor -- start 2024-04-25T10:04:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- start start 2024-04-25T10:04:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 0x7f6dec0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 0x7f6dec154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec06faa0 con 0x7f6dec15b950 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec0cf870 con 0x7f6dec0cf420 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df3fe4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec06fc20 con 0x7f6dec0cfb80 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51310/0 (socket says 172.21.15.82:51310) 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 -- 172.21.15.82:0/2631933210 learned_addr learned my addr 172.21.15.82:0/2631933210 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df1d80640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 0x7f6dec0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df157f640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 0x7f6dec154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 -- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 msgr2=0x7f6dec154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 0x7f6dec154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 -- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 msgr2=0x7f6dec0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 0x7f6dec0c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 -- 172.21.15.82:0/2631933210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec0cdb20 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df157f640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 0x7f6dec154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df1d80640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 0x7f6dec0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.167+0000 7f6df2581640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cd50 tx=0x7f6de400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400f680 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de400f840 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec0c5b60 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de400fc30 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec0c5dd0 con 0x7f6dec15b950 2024-04-25T10:04:03.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec04efa0 con 0x7f6dec15b950 2024-04-25T10:04:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6de40213f0 con 0x7f6dec15b950 2024-04-25T10:04:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 0x7f6dd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6de4085ec0 con 0x7f6dec15b950 2024-04-25T10:04:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6df1d80640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 0x7f6dd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.171+0000 7f6df1d80640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 0x7f6dd0069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0af000 tx=0x7f6ddc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.175+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6de4014030 con 0x7f6dec15b950 2024-04-25T10:04:03.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.319+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6dec0b0cb0 con 0x7f6dec15b950 2024-04-25T10:04:03.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6de27fc640 1 -- 172.21.15.82:0/2631933210 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6de4056db0 con 0x7f6dec15b950 2024-04-25T10:04:03.325 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:04:03.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 msgr2=0x7f6dd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 0x7f6dd0069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0af000 tx=0x7f6ddc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 msgr2=0x7f6dec0b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6de400cd50 tx=0x7f6de400a650 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 shutdown_connections 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dd0066f90 0x7f6dd0069450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dec0cfb80 0x7f6dec154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dec15b950 0x7f6dec0b4190 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 --2- 172.21.15.82:0/2631933210 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dec0cf420 0x7f6dec0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 >> 172.21.15.82:0/2631933210 conn(0x7f6dec0b30b0 msgr2=0x7f6dec0b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 shutdown_connections 2024-04-25T10:04:03.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.323+0000 7f6df3fe4640 1 -- 172.21.15.82:0/2631933210 wait complete. 2024-04-25T10:04:03.343 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:03.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.455+0000 7ffa0cd25640 1 Processor -- start 2024-04-25T10:04:03.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0cd25640 1 -- start start 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0cd25640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa0815ecc0 0x7ffa0806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0cd25640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa080b2090 con 0x7ffa080b3f50 2024-04-25T10:04:03.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0cd25640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa080b43e0 con 0x7ffa0815a700 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0cd25640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa080b46c0 con 0x7ffa0815ecc0 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa080b3f50 0x7ffa080b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37498/0 (socket says 172.21.15.82:37498) 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 -- 172.21.15.82:0/2726292729 learned_addr learned my addr 172.21.15.82:0/2726292729 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa06d9d640 1 --2- 172.21.15.82:0/2726292729 >> v2:172.21.15.196:3300/0 conn(0x7ffa0815ecc0 0x7ffa0806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1115355307 0 0) 0x7ffa080b2090 con 0x7ffa080b3f50 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9f0003530 con 0x7ffa080b3f50 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3787298010 0 0) 0x7ff9f0003530 con 0x7ffa080b3f50 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 >> v2:172.21.15.196:3300/0 conn(0x7ffa0815ecc0 msgr2=0x7ffa0806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 --2- 172.21.15.82:0/2726292729 >> v2:172.21.15.196:3300/0 conn(0x7ffa0815ecc0 0x7ffa0806e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 >> v1:172.21.15.196:6789/0 conn(0x7ffa0815a700 legacy=0x7ffa080cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080cd6f0 con 0x7ffa080b3f50 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff9f8001a20 con 0x7ffa080b3f50 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 >> v1:172.21.15.82:6789/0 conn(0x7ffa080b3f50 legacy=0x7ffa080b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f0003a30 0x7ff9f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ff9f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9f00052d0 0x7ff9f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b46c0 con 0x7ff9f0004810 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f000c210 con 0x7ff9f0003a30 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f000c410 con 0x7ff9f00052d0 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ff9f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 -- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9f00052d0 msgr2=0x7ff9f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9f00052d0 0x7ff9f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0659c640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f0003a30 0x7ff9f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 -- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f0003a30 msgr2=0x7ff9f0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f0003a30 0x7ff9f0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa05d9b640 1 -- 172.21.15.82:0/2726292729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f0003530 con 0x7ff9f0004810 2024-04-25T10:04:03.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa06d9d640 1 -- 172.21.15.82:0/2726292729 reap_dead start 2024-04-25T10:04:03.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.459+0000 7ffa0659c640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9f0003a30 0x7ff9f0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:03.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ff9f0004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800a830 tx=0x7ff9f801d0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3663f40ddec87123 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800a480 con 0x7ff9f0004810 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f80ad650 con 0x7ff9f0004810 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0559a640 1 -- 172.21.15.82:0/2726292729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80ada80 con 0x7ff9f0004810 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 msgr2=0x7ff9f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ff9f0004c90 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800a830 tx=0x7ff9f801d0f0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- 172.21.15.82:0/2726292729 shutdown_connections 2024-04-25T10:04:03.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/2726292729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ff9f0004c90 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- 172.21.15.82:0/2726292729 >> 172.21.15.82:0/2726292729 conn(0x7ffa080b11a0 msgr2=0x7ffa080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- 172.21.15.82:0/2726292729 shutdown_connections 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- 172.21.15.82:0/2726292729 wait complete. 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 Processor -- start 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- start start 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa08152810 0x7ffa08157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa08152c70 0x7ffa080a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080cd450 con 0x7ff9f0004810 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080cd2d0 con 0x7ffa08152c70 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0cd25640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b24a0 con 0x7ffa08152810 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51322/0 (socket says 172.21.15.82:51322) 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 -- 172.21.15.82:0/1871747007 learned_addr learned my addr 172.21.15.82:0/1871747007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 -- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa08152810 msgr2=0x7ffa08157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa08152810 0x7ffa08157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 -- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa08152c70 msgr2=0x7ffa080a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa08152c70 0x7ffa080a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.463+0000 7ffa0659c640 1 -- 172.21.15.82:0/1871747007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa08153a40 con 0x7ff9f0004810 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa0659c640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00d4d0 tx=0x7ff9fc00d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc018070 con 0x7ff9f0004810 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc003620 con 0x7ff9f0004810 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa08155260 con 0x7ff9f0004810 2024-04-25T10:04:03.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc003a10 con 0x7ff9f0004810 2024-04-25T10:04:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa08153480 con 0x7ff9f0004810 2024-04-25T10:04:03.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0804efa0 con 0x7ff9f0004810 2024-04-25T10:04:03.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9fc01b070 con 0x7ff9f0004810 2024-04-25T10:04:03.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 0x7ff9e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff9fc084810 con 0x7ff9f0004810 2024-04-25T10:04:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 0x7ff9e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.467+0000 7ffa05d9b640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 0x7ff9e8069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80ad000 tx=0x7ffa080cd8c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.471+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9fc055020 con 0x7ff9f0004810 2024-04-25T10:04:03.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.663+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa080afbc0 con 0x7ff9f0004810 2024-04-25T10:04:03.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.663+0000 7ff9f6ffd640 1 -- 172.21.15.82:0/1871747007 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7ff9fc0590c0 con 0x7ff9f0004810 2024-04-25T10:04:03.668 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:03.668 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:03.668 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:04:03.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 msgr2=0x7ff9e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 0x7ff9e8069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9f80ad000 tx=0x7ffa080cd8c0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 msgr2=0x7ffa08151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00d4d0 tx=0x7ff9fc00d990 comp rx=0 tx=0).stop 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 shutdown_connections 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9e8066f60 0x7ff9e8069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa08152810 0x7ffa08157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa08152c70 0x7ffa080a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 --2- 172.21.15.82:0/1871747007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9f0004810 0x7ffa08151f30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 >> 172.21.15.82:0/1871747007 conn(0x7ffa080b11a0 msgr2=0x7ffa080cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 shutdown_connections 2024-04-25T10:04:03.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.667+0000 7ffa0cd25640 1 -- 172.21.15.82:0/1871747007 wait complete. 2024-04-25T10:04:03.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 Processor -- start 2024-04-25T10:04:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 -- start start 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c06bca0 0x7f4d3c0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d3c066e30 0x7f4d3c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c06ec30 con 0x7f4d3c0af370 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c0669a0 con 0x7f4d3c066e30 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d41f99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0cd5f0 con 0x7f4d3c06bca0 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d3c066e30 0x7f4d3c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4d3c0af370 0x7f4d3c06d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37504/0 (socket says 172.21.15.82:37504) 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3bfff640 1 -- 172.21.15.82:0/690369601 learned_addr learned my addr 172.21.15.82:0/690369601 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3affd640 1 -- 172.21.15.82:0/690369601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c06bca0 msgr2=0x7f4d3c0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3affd640 1 --2- 172.21.15.82:0/690369601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c06bca0 0x7f4d3c0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3affd640 1 -- 172.21.15.82:0/690369601 >> v1:172.21.15.82:6789/0 conn(0x7f4d3c0af370 legacy=0x7f4d3c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3a7fc640 1 -- 172.21.15.82:0/690369601 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874109786 0 0) 0x7f4d3c06ec30 con 0x7f4d3c0af370 2024-04-25T10:04:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.795+0000 7f4d3affd640 1 -- 172.21.15.82:0/690369601 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c06c5c0 con 0x7f4d3c066e30 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- 172.21.15.82:0/690369601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c066e30 0x7f4d3c06c080 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4d240097c0 tx=0x7f4d2401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=612569dcb9fe1b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3a7fc640 1 -- 172.21.15.82:0/690369601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240b0070 con 0x7f4d3c066e30 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3a7fc640 1 -- 172.21.15.82:0/690369601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d240ab600 con 0x7f4d3c066e30 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3a7fc640 1 -- 172.21.15.82:0/690369601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240ab9d0 con 0x7f4d3c066e30 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- 172.21.15.82:0/690369601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c066e30 msgr2=0x7f4d3c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- 172.21.15.82:0/690369601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c066e30 0x7f4d3c06c080 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4d240097c0 tx=0x7f4d2401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- 172.21.15.82:0/690369601 shutdown_connections 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- 172.21.15.82:0/690369601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c066e30 0x7f4d3c06c080 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- 172.21.15.82:0/690369601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c06bca0 0x7f4d3c0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- 172.21.15.82:0/690369601 >> 172.21.15.82:0/690369601 conn(0x7f4d3c071810 msgr2=0x7f4d3c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- 172.21.15.82:0/690369601 shutdown_connections 2024-04-25T10:04:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- 172.21.15.82:0/690369601 wait complete. 2024-04-25T10:04:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 Processor -- start 2024-04-25T10:04:03.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- start start 2024-04-25T10:04:03.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d3c066e30 0x7f4d3c158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c0af370 0x7f4d3c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15cc80 con 0x7f4d3c06bca0 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15cb00 con 0x7f4d3c066e30 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d41f99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c168630 con 0x7f4d3c0af370 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51338/0 (socket says 172.21.15.82:51338) 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 -- 172.21.15.82:0/566140264 learned_addr learned my addr 172.21.15.82:0/566140264 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 -- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c0af370 msgr2=0x7f4d3c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3bfff640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c0af370 0x7f4d3c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3b7fe640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d3c066e30 0x7f4d3c158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c0af370 0x7f4d3c0b08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 -- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d3c066e30 msgr2=0x7f4d3c158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d3c066e30 0x7f4d3c158580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 -- 172.21.15.82:0/566140264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c149560 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d3affd640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4d2401a010 tx=0x7f4d2401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.799+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240b0050 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d240bc070 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c157090 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d240b5810 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c157650 con 0x7f4d3c06bca0 2024-04-25T10:04:03.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c04efa0 con 0x7f4d3c06bca0 2024-04-25T10:04:03.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d240b59b0 con 0x7f4d3c06bca0 2024-04-25T10:04:03.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d33fff640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 0x7f4d1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:03.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4d2411f9e0 con 0x7f4d3c06bca0 2024-04-25T10:04:03.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d3b7fe640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 0x7f4d1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:03.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.803+0000 7f4d3b7fe640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 0x7f4d1c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c15a940 tx=0x7f4d34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:03.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.807+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d24016090 con 0x7f4d3c06bca0 2024-04-25T10:04:03.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.959+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d3c072400 con 0x7f4d3c06bca0 2024-04-25T10:04:03.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.959+0000 7f4d33fff640 1 -- 172.21.15.82:0/566140264 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f4d240f6020 con 0x7f4d3c06bca0 2024-04-25T10:04:03.964 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:03.964 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:49.436349+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 msgr2=0x7f4d1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 0x7f4d1c069420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c15a940 tx=0x7f4d34008040 comp rx=0 tx=0).stop 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 msgr2=0x7f4d3c146ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4d2401a010 tx=0x7f4d2401ac20 comp rx=0 tx=0).stop 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 shutdown_connections 2024-04-25T10:04:03.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d1c066f60 0x7f4d1c069420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d3c0af370 0x7f4d3c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d3c06bca0 0x7f4d3c146ac0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 --2- 172.21.15.82:0/566140264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d3c066e30 0x7f4d3c158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 >> 172.21.15.82:0/566140264 conn(0x7f4d3c071810 msgr2=0x7f4d3c0cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 shutdown_connections 2024-04-25T10:04:03.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:03.963+0000 7f4d41f99640 1 -- 172.21.15.82:0/566140264 wait complete. 2024-04-25T10:04:03.981 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:04:03.982 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:04.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.083+0000 7fb5d889b640 1 Processor -- start 2024-04-25T10:04:04.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 -- start start 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5d4159ea0 0x7fb5d40b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 0x7fb5d415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d40718c0 0x7fb5d4071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d40b2990 con 0x7fb5d4159ea0 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d415ae30 con 0x7fb5d415a280 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d889b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d415b110 con 0x7fb5d40718c0 2024-04-25T10:04:04.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5d4159ea0 0x7fb5d40b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb5d4159ea0 0x7fb5d40b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51340/0 (socket says 172.21.15.82:51340) 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 -- 172.21.15.82:0/2190870656 learned_addr learned my addr 172.21.15.82:0/2190870656 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d2d9d640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d40718c0 0x7fb5d4071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5cbfff640 1 --2- 172.21.15.82:0/2190870656 >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 0x7fb5d415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 -- 172.21.15.82:0/2190870656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d40718c0 msgr2=0x7fb5d4071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d40718c0 0x7fb5d4071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 -- 172.21.15.82:0/2190870656 >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 msgr2=0x7fb5d415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 --2- 172.21.15.82:0/2190870656 >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 0x7fb5d415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 -- 172.21.15.82:0/2190870656 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40b2620 con 0x7fb5d4159ea0 2024-04-25T10:04:04.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5cbfff640 1 --2- 172.21.15.82:0/2190870656 >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 0x7fb5d415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:04.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.087+0000 7fb5d259c640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4159ea0 0x7fb5d40b2e20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40728e0 tx=0x7fb5c401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15ccada30a04f221 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d1d9b640 1 -- 172.21.15.82:0/2190870656 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40b0070 con 0x7fb5d4159ea0 2024-04-25T10:04:04.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d1d9b640 1 -- 172.21.15.82:0/2190870656 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c40ab600 con 0x7fb5d4159ea0 2024-04-25T10:04:04.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d1d9b640 1 -- 172.21.15.82:0/2190870656 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40ab9d0 con 0x7fb5d4159ea0 2024-04-25T10:04:04.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- 172.21.15.82:0/2190870656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4159ea0 msgr2=0x7fb5d40b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4159ea0 0x7fb5d40b2e20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40728e0 tx=0x7fb5c401c520 comp rx=0 tx=0).stop 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- 172.21.15.82:0/2190870656 shutdown_connections 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d40718c0 0x7fb5d4071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- 172.21.15.82:0/2190870656 >> v2:172.21.15.196:3300/0 conn(0x7fb5d415a280 0x7fb5d415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- 172.21.15.82:0/2190870656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d4159ea0 0x7fb5d40b2e20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- 172.21.15.82:0/2190870656 >> 172.21.15.82:0/2190870656 conn(0x7fb5d40b11a0 msgr2=0x7fb5d40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- 172.21.15.82:0/2190870656 shutdown_connections 2024-04-25T10:04:04.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- 172.21.15.82:0/2190870656 wait complete. 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 Processor -- start 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- start start 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d4159ea0 0x7fb5d41490b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 0x7fb5d415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d406d420 con 0x7fb5d40718c0 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d406d2a0 con 0x7fb5d415a280 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d889b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d406d5a0 con 0x7fb5d4159ea0 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51350/0 (socket says 172.21.15.82:51350) 2024-04-25T10:04:04.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 -- 172.21.15.82:0/721660376 learned_addr learned my addr 172.21.15.82:0/721660376 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5cbfff640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d4159ea0 0x7fb5d41490b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d2d9d640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 0x7fb5d415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 -- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d4159ea0 msgr2=0x7fb5d41490b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d4159ea0 0x7fb5d41490b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 -- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 msgr2=0x7fb5d415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 0x7fb5d415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 -- 172.21.15.82:0/721660376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d414b260 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d2d9d640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 0x7fb5d415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.091+0000 7fb5d259c640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40ae040 tx=0x7fb5c401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40b0040 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5c40bc070 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d414b540 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c40b6880 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d414bb00 con 0x7fb5d40718c0 2024-04-25T10:04:04.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d404efa0 con 0x7fb5d40718c0 2024-04-25T10:04:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5c40ab4c0 con 0x7fb5d40718c0 2024-04-25T10:04:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 0x7fb5b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb5c411f4c0 con 0x7fb5d40718c0 2024-04-25T10:04:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5cbfff640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 0x7fb5b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.095+0000 7fb5cbfff640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 0x7fb5b4069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc006fd0 tx=0x7fb5bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.099+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5c4016110 con 0x7fb5d40718c0 2024-04-25T10:04:04.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d406e6b0 con 0x7fb5d40718c0 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5caffd640 1 -- 172.21.15.82:0/721660376 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fb5c40f04d0 con 0x7fb5d40718c0 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:04.290 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:04:04.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 msgr2=0x7fb5b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 0x7fb5b4069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc006fd0 tx=0x7fb5bc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 msgr2=0x7fb5d414d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb5c40ae040 tx=0x7fb5c401ad60 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 shutdown_connections 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb5b4066f60 0x7fb5b4069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5d4159ea0 0x7fb5d41490b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5d415a280 0x7fb5d415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 --2- 172.21.15.82:0/721660376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5d40718c0 0x7fb5d414d830 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 >> 172.21.15.82:0/721660376 conn(0x7fb5d40b11a0 msgr2=0x7fb5d414cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 shutdown_connections 2024-04-25T10:04:04.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.287+0000 7fb5d889b640 1 -- 172.21.15.82:0/721660376 wait complete. 2024-04-25T10:04:04.308 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:04.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.411+0000 7f11fa50f640 1 Processor -- start 2024-04-25T10:04:04.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.411+0000 7f11fa50f640 1 -- start start 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f11f415da80 0x7f11f40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f11f40d07f0 0x7f11f406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f406be60 con 0x7f11f40d1cd0 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f406c440 con 0x7f11f415da80 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f415de60 con 0x7f11f40d07f0 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f11f415da80 0x7f11f40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f11f415da80 0x7f11f40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51364/0 (socket says 172.21.15.82:51364) 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 -- 172.21.15.82:0/4098879226 learned_addr learned my addr 172.21.15.82:0/4098879226 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f8aac640 1 --2- 172.21.15.82:0/4098879226 >> v2:172.21.15.196:3300/0 conn(0x7f11f40d07f0 0x7f11f406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 -- 172.21.15.82:0/4098879226 >> v2:172.21.15.196:3300/0 conn(0x7f11f40d07f0 msgr2=0x7f11f406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 --2- 172.21.15.82:0/4098879226 >> v2:172.21.15.196:3300/0 conn(0x7f11f40d07f0 0x7f11f406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 -- 172.21.15.82:0/4098879226 >> v1:172.21.15.196:6789/0 conn(0x7f11f40d1cd0 legacy=0x7f11f40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 -- 172.21.15.82:0/4098879226 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40d2450 con 0x7f11f415da80 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f3fff640 1 --2- 172.21.15.82:0/4098879226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40d4230 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009870 tx=0x7f11dc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c9417d68b08f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f2ffd640 1 -- 172.21.15.82:0/4098879226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0ab650 con 0x7f11f415da80 2024-04-25T10:04:04.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f2ffd640 1 -- 172.21.15.82:0/4098879226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc0b0070 con 0x7f11f415da80 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11f2ffd640 1 -- 172.21.15.82:0/4098879226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0aba10 con 0x7f11f415da80 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- 172.21.15.82:0/4098879226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 msgr2=0x7f11f40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 --2- 172.21.15.82:0/4098879226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40d4230 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009870 tx=0x7f11dc01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- 172.21.15.82:0/4098879226 shutdown_connections 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 --2- 172.21.15.82:0/4098879226 >> v2:172.21.15.196:3300/0 conn(0x7f11f40d07f0 0x7f11f406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 --2- 172.21.15.82:0/4098879226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40d4230 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- 172.21.15.82:0/4098879226 >> 172.21.15.82:0/4098879226 conn(0x7f11f408b000 msgr2=0x7f11f40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- 172.21.15.82:0/4098879226 shutdown_connections 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.415+0000 7f11fa50f640 1 -- 172.21.15.82:0/4098879226 wait complete. 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 Processor -- start 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- start start 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11f40d07f0 0x7f11f4156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11f40d1cd0 0x7f11f40b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f406d7a0 con 0x7f11f415da80 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f406d620 con 0x7f11f40d1cd0 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f406d920 con 0x7f11f40d07f0 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51380/0 (socket says 172.21.15.82:51380) 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 -- 172.21.15.82:0/1956403053 learned_addr learned my addr 172.21.15.82:0/1956403053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f3fff640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11f40d07f0 0x7f11f4156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f37fe640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11f40d1cd0 0x7f11f40b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 -- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11f40d07f0 msgr2=0x7f11f4156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11f40d07f0 0x7f11f4156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 -- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11f40d1cd0 msgr2=0x7f11f40b82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11f40d1cd0 0x7f11f40b82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 -- 172.21.15.82:0/1956403053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40b6d20 con 0x7f11f415da80 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f8aac640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00cd70 tx=0x7f11ec00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec003ba0 con 0x7f11f415da80 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ec003d40 con 0x7f11f415da80 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ec004240 con 0x7f11f415da80 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f40b7000 con 0x7f11f415da80 2024-04-25T10:04:04.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.419+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f40b75c0 con 0x7f11f415da80 2024-04-25T10:04:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f404efa0 con 0x7f11f415da80 2024-04-25T10:04:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11ec0043e0 con 0x7f11f415da80 2024-04-25T10:04:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11f0ff9640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 0x7f11d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f11ec085d10 con 0x7f11f415da80 2024-04-25T10:04:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11f3fff640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 0x7f11d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.423+0000 7f11f3fff640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 0x7f11d4069450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0af000 tx=0x7f11dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.427+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11ec056b70 con 0x7f11f415da80 2024-04-25T10:04:04.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11f406ad40 con 0x7f11f415da80 2024-04-25T10:04:04.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11f0ff9640 1 -- 172.21.15.82:0/1956403053 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f11ec056470 con 0x7f11f415da80 2024-04-25T10:04:04.610 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:04.610 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:30.679984+0000","modified":"2024-04-25T10:03:33.721743+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5439},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5439":{"gid":5439,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/560740519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":560740519},{"type":"v1","addr":"172.21.15.82:6837","nonce":560740519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:04:04.610 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 17 2024-04-25T10:04:04.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 msgr2=0x7f11d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 0x7f11d4069450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11dc0af000 tx=0x7f11dc008040 comp rx=0 tx=0).stop 2024-04-25T10:04:04.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 msgr2=0x7f11f40b87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f11ec00cd70 tx=0x7f11ec00a660 comp rx=0 tx=0).stop 2024-04-25T10:04:04.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 shutdown_connections 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11d4066f90 0x7f11d4069450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11f415da80 0x7f11f40b87f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11f40d1cd0 0x7f11f40b82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 --2- 172.21.15.82:0/1956403053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11f40d07f0 0x7f11f4156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 >> 172.21.15.82:0/1956403053 conn(0x7f11f408b000 msgr2=0x7f11f40b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 shutdown_connections 2024-04-25T10:04:04.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.607+0000 7f11fa50f640 1 -- 172.21.15.82:0/1956403053 wait complete. 2024-04-25T10:04:04.628 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.743+0000 7f6ee7ae6640 1 Processor -- start 2024-04-25T10:04:04.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.743+0000 7f6ee7ae6640 1 -- start start 2024-04-25T10:04:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee7ae6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee7ae6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ee006ec60 0x7f6ee0142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee7ae6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee00ccc70 con 0x7f6ee00ce510 2024-04-25T10:04:04.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee7ae6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee00cc2b0 con 0x7f6ee006ec60 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee7ae6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee006e290 con 0x7f6ee0144ca0 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34038/0 (socket says 172.21.15.82:34038) 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee6083640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ee006ec60 0x7f6ee0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 -- 172.21.15.82:0/1205268975 learned_addr learned my addr 172.21.15.82:0/1205268975 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 -- 172.21.15.82:0/1205268975 >> v2:172.21.15.196:3300/0 conn(0x7f6ee006ec60 msgr2=0x7f6ee0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 --2- 172.21.15.82:0/1205268975 >> v2:172.21.15.196:3300/0 conn(0x7f6ee006ec60 0x7f6ee0142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 -- 172.21.15.82:0/1205268975 >> v1:172.21.15.196:6789/0 conn(0x7f6ee00ce510 legacy=0x7f6ee00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 -- 172.21.15.82:0/1205268975 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee006dc40 con 0x7f6ee0144ca0 2024-04-25T10:04:04.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.747+0000 7f6ee5081640 1 --2- 172.21.15.82:0/1205268975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009870 tx=0x7f6ed401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=149b07dcc9d6b3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee4880640 1 -- 172.21.15.82:0/1205268975 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40b5070 con 0x7f6ee0144ca0 2024-04-25T10:04:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee4880640 1 -- 172.21.15.82:0/1205268975 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed40a8a50 con 0x7f6ee0144ca0 2024-04-25T10:04:04.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee4880640 1 -- 172.21.15.82:0/1205268975 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40a8e20 con 0x7f6ee0144ca0 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1205268975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 msgr2=0x7f6ee006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1205268975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4009870 tx=0x7f6ed401ed50 comp rx=0 tx=0).stop 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1205268975 shutdown_connections 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1205268975 >> v2:172.21.15.196:3300/0 conn(0x7f6ee006ec60 0x7f6ee0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1205268975 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee0144ca0 0x7f6ee006e720 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1205268975 >> 172.21.15.82:0/1205268975 conn(0x7f6ee00b11a0 msgr2=0x7f6ee00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1205268975 shutdown_connections 2024-04-25T10:04:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1205268975 wait complete. 2024-04-25T10:04:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 Processor -- start 2024-04-25T10:04:04.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- start start 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee006ec60 0x7f6ee015b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee00ce510 0x7f6ee01e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee01451f0 con 0x7f6ee0144ca0 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee00cc900 con 0x7f6ee006ec60 2024-04-25T10:04:04.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee7ae6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee0145370 con 0x7f6ee00ce510 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee6083640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee6083640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40354/0 (socket says 172.21.15.82:40354) 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee6083640 1 -- 172.21.15.82:0/1926969717 learned_addr learned my addr 172.21.15.82:0/1926969717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee5882640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee006ec60 0x7f6ee015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.751+0000 7f6ee5081640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee00ce510 0x7f6ee01e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 -- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee00ce510 msgr2=0x7f6ee01e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee00ce510 0x7f6ee01e3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 -- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee006ec60 msgr2=0x7f6ee015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee006ec60 0x7f6ee015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 -- 172.21.15.82:0/1926969717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee015b770 con 0x7f6ee0144ca0 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee6083640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800ef90 tx=0x7f6ed800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8018020 con 0x7f6ee0144ca0 2024-04-25T10:04:04.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed8014480 con 0x7f6ee0144ca0 2024-04-25T10:04:04.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee01e4810 con 0x7f6ee0144ca0 2024-04-25T10:04:04.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed8014870 con 0x7f6ee0144ca0 2024-04-25T10:04:04.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee01e4dd0 con 0x7f6ee0144ca0 2024-04-25T10:04:04.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee004efa0 con 0x7f6ee0144ca0 2024-04-25T10:04:04.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ed801b020 con 0x7f6ee0144ca0 2024-04-25T10:04:04.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 0x7f6ecc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:04.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6ed8086160 con 0x7f6ee0144ca0 2024-04-25T10:04:04.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee5882640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 0x7f6ecc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:04.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.755+0000 7f6ee5882640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 0x7f6ecc069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0001650 tx=0x7f6ed0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:04.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.759+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ed8022030 con 0x7f6ee0144ca0 2024-04-25T10:04:04.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.907+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ee00b1cc0 con 0x7f6ee0144ca0 2024-04-25T10:04:04.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ede7fc640 1 -- 172.21.15.82:0/1926969717 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+17003 (secure 0 0 0) 0x7f6ed8010040 con 0x7f6ee0144ca0 2024-04-25T10:04:04.914 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:04.915 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:03:49.436349+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:29.159335+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:29.660329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.2f","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:04.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 msgr2=0x7f6ecc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 0x7f6ecc069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0001650 tx=0x7f6ed0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:04.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 msgr2=0x7f6ee01e58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:04.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6ed800ef90 tx=0x7f6ed800c490 comp rx=0 tx=0).stop 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 shutdown_connections 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ecc066f60 0x7f6ecc069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ee00ce510 0x7f6ee01e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ee0144ca0 0x7f6ee01e58b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 --2- 172.21.15.82:0/1926969717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ee006ec60 0x7f6ee015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 >> 172.21.15.82:0/1926969717 conn(0x7f6ee00b11a0 msgr2=0x7f6ee00c7510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 shutdown_connections 2024-04-25T10:04:04.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:04.911+0000 7f6ee7ae6640 1 -- 172.21.15.82:0/1926969717 wait complete. 2024-04-25T10:04:04.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:04:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 Processor -- start 2024-04-25T10:04:05.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 -- start start 2024-04-25T10:04:05.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d48143790 0x7f4d4806d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d4815da40 0x7f4d4806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 0x7f4d4815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d48143b70 con 0x7f4d4815da40 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d4806e170 con 0x7f4d4806c1e0 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d4d8c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4806e450 con 0x7f4d48143790 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d4815da40 0x7f4d4806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4d4815da40 0x7f4d4806bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40370/0 (socket says 172.21.15.82:40370) 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 -- 172.21.15.82:0/101908397 learned_addr learned my addr 172.21.15.82:0/101908397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d46ffd640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d48143790 0x7f4d4806d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d477fe640 1 --2- 172.21.15.82:0/101908397 >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 0x7f4d4815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 -- 172.21.15.82:0/101908397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d48143790 msgr2=0x7f4d4806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d48143790 0x7f4d4806d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 -- 172.21.15.82:0/101908397 >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 msgr2=0x7f4d4815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 --2- 172.21.15.82:0/101908397 >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 0x7f4d4815a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d467fc640 1 -- 172.21.15.82:0/101908397 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4806e730 con 0x7f4d4815da40 2024-04-25T10:04:05.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.043+0000 7f4d477fe640 1 --2- 172.21.15.82:0/101908397 >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 0x7f4d4815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d467fc640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4806bca0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4d48067980 tx=0x7f4d3801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b0616584d6d8a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d45ffb640 1 -- 172.21.15.82:0/101908397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380b0070 con 0x7f4d4815da40 2024-04-25T10:04:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d45ffb640 1 -- 172.21.15.82:0/101908397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d380ab600 con 0x7f4d4815da40 2024-04-25T10:04:05.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d45ffb640 1 -- 172.21.15.82:0/101908397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d380ab9d0 con 0x7f4d4815da40 2024-04-25T10:04:05.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/101908397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 msgr2=0x7f4d4806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4806bca0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4d48067980 tx=0x7f4d3801c520 comp rx=0 tx=0).stop 2024-04-25T10:04:05.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/101908397 shutdown_connections 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/101908397 >> v2:172.21.15.196:3300/0 conn(0x7f4d4806c1e0 0x7f4d4815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4806bca0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/101908397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d48143790 0x7f4d4806d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/101908397 >> 172.21.15.82:0/101908397 conn(0x7f4d48071810 msgr2=0x7f4d48071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/101908397 shutdown_connections 2024-04-25T10:04:05.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/101908397 wait complete. 2024-04-25T10:04:05.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 Processor -- start 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.047+0000 7f4d4d8c7640 1 -- start start 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 0x7f4d48163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d48143790 0x7f4d4815ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4815acc0 con 0x7f4d4815da40 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4815ab40 con 0x7f4d48143790 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4815ae40 con 0x7f4d4806c1e0 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40380/0 (socket says 172.21.15.82:40380) 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 -- 172.21.15.82:0/3448913195 learned_addr learned my addr 172.21.15.82:0/3448913195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d467fc640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d48143790 0x7f4d4815ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d46ffd640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 0x7f4d48163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 -- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 msgr2=0x7f4d48163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 0x7f4d48163bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 -- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d48143790 msgr2=0x7f4d4815ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d48143790 0x7f4d4815ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 -- 172.21.15.82:0/3448913195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d481f7020 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d46ffd640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 0x7f4d48163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d477fe640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c7f0 tx=0x7f4d3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30015070 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d30007820 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d481f7300 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d30007c10 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4815dee0 con 0x7f4d4815da40 2024-04-25T10:04:05.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d4804efa0 con 0x7f4d4815da40 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d30018070 con 0x7f4d4815da40 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4d30085100 con 0x7f4d4815da40 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d46ffd640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.051+0000 7f4d46ffd640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 0x7f4d28069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d40008330 tx=0x7f4d400082c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.055+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d30056020 con 0x7f4d4815da40 2024-04-25T10:04:05.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.235+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4d480b4030 con 0x7f4d4815da40 2024-04-25T10:04:05.238 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:04:05.235+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:04:05.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d3ffff640 1 -- 172.21.15.82:0/3448913195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4d30055920 con 0x7f4d4815da40 2024-04-25T10:04:05.241 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:04:05.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 msgr2=0x7f4d28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 0x7f4d28069420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4d40008330 tx=0x7f4d400082c0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 msgr2=0x7f4d4815d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4d3000c7f0 tx=0x7f4d3000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 shutdown_connections 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d28066f60 0x7f4d28069420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d4815da40 0x7f4d4815d210 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d48143790 0x7f4d4815ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 --2- 172.21.15.82:0/3448913195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d4806c1e0 0x7f4d48163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.239+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 >> 172.21.15.82:0/3448913195 conn(0x7f4d48071810 msgr2=0x7f4d480cf0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.243+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 shutdown_connections 2024-04-25T10:04:05.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.243+0000 7f4d4d8c7640 1 -- 172.21.15.82:0/3448913195 wait complete. 2024-04-25T10:04:05.255 INFO:tasks.ceph.mds.c.smithi082.stderr: -5881> 2024-04-25T10:03:35.275+0000 7fdeb2eac640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:05.263 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:04:05.310 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:04:05.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 Processor -- start 2024-04-25T10:04:05.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 -- start start 2024-04-25T10:04:05.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6f815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80ca4c0 con 0x7fe6f80ce170 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80cd690 con 0x7fe6f80cf970 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6fd089640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80ca780 con 0x7fe6f80cf590 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f759e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe6f80ce170 0x7fe6f80ce600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49010/0 (socket says 172.21.15.82:49010) 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f759e640 1 -- 172.21.15.82:0/2904268148 learned_addr learned my addr 172.21.15.82:0/2904268148 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f6d9d640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6f815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739755922 0 0) 0x7fe6f80ca4c0 con 0x7fe6f80ce170 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e0003530 con 0x7fe6f80ce170 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132072202 0 0) 0x7fe6e0003530 con 0x7fe6f80ce170 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 msgr2=0x7fe6f815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6f815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 >> v1:172.21.15.196:6789/0 conn(0x7fe6f80cf970 legacy=0x7fe6f80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80cd970 con 0x7fe6f80ce170 2024-04-25T10:04:05.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe6f0001a00 con 0x7fe6f80ce170 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 >> v1:172.21.15.82:6789/0 conn(0x7fe6f80ce170 legacy=0x7fe6f80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e0003a60 0x7fe6e0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6e0005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000bb60 con 0x7fe6e0004810 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000bd10 con 0x7fe6e0003a60 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000bf10 con 0x7fe6f80cf590 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f759e640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6e0005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 -- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 msgr2=0x7fe6e0005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f80cf590 0x7fe6e0005280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f6d9d640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e0003a60 0x7fe6e0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 -- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e0003a60 msgr2=0x7fe6e0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:05.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6e0003a60 0x7fe6e0003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.375+0000 7fe6f659c640 1 -- 172.21.15.82:0/2904268148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0003530 con 0x7fe6e0004810 2024-04-25T10:04:05.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f659c640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6e0004c90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001780 tx=0x7fe6ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0b7ed9873cb949 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0013a0 con 0x7fe6e0004810 2024-04-25T10:04:05.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec0a9070 con 0x7fe6e0004810 2024-04-25T10:04:05.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f5d9b640 1 -- 172.21.15.82:0/2904268148 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003600 con 0x7fe6e0004810 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- 172.21.15.82:0/2904268148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 msgr2=0x7fe6e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 --2- 172.21.15.82:0/2904268148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6e0004c90 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001780 tx=0x7fe6ec01c420 comp rx=0 tx=0).stop 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f759e640 1 -- 172.21.15.82:0/2904268148 reap_dead start 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- 172.21.15.82:0/2904268148 shutdown_connections 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- 172.21.15.82:0/2904268148 >> 172.21.15.82:0/2904268148 conn(0x7fe6f806ae60 msgr2=0x7fe6f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- 172.21.15.82:0/2904268148 shutdown_connections 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- 172.21.15.82:0/2904268148 wait complete. 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 Processor -- start 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- start start 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f8071de0 0x7fe6f81f2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6f80ce600 0x7fe6f80cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80cdce0 con 0x7fe6e0004810 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80cd440 con 0x7fe6f80ce600 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6fd089640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80ca0c0 con 0x7fe6f8071de0 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40404/0 (socket says 172.21.15.82:40404) 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 -- 172.21.15.82:0/3480270612 learned_addr learned my addr 172.21.15.82:0/3480270612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f659c640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f8071de0 0x7fe6f81f2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f759e640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6f80ce600 0x7fe6f80cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 -- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f8071de0 msgr2=0x7fe6f81f2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f8071de0 0x7fe6f81f2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 -- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6f80ce600 msgr2=0x7fe6f80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6f80ce600 0x7fe6f80cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.379+0000 7fe6f6d9d640 1 -- 172.21.15.82:0/3480270612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f815c6d0 con 0x7fe6e0004810 2024-04-25T10:04:05.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6f6d9d640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800e840 tx=0x7fe6e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8017070 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800c6d0 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800f040 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f815c9b0 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f815cf70 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f804efa0 con 0x7fe6e0004810 2024-04-25T10:04:05.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6e801a070 con 0x7fe6e0004810 2024-04-25T10:04:05.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 0x7fe6e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:05.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fe6e80852d0 con 0x7fe6e0004810 2024-04-25T10:04:05.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6f659c640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 0x7fe6e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:05.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.383+0000 7fe6f659c640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 0x7fe6e4069450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001750 tx=0x7fe6ec0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:05.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.387+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6e80560c0 con 0x7fe6e0004810 2024-04-25T10:04:05.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:05.575+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6f80b4170 con 0x7fe6e0004810 2024-04-25T10:04:06.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.239+0000 7fe6df7fe640 1 -- 172.21.15.82:0/3480270612 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe6e80559c0 con 0x7fe6e0004810 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 msgr2=0x7fe6e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 0x7fe6e4069450 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001750 tx=0x7fe6ec0011a0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 msgr2=0x7fe6f815c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800e840 tx=0x7fe6e800ed00 comp rx=0 tx=0).stop 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 shutdown_connections 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6e4066f90 0x7fe6e4069450 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6f8071de0 0x7fe6f81f2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6f80ce600 0x7fe6f80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 --2- 172.21.15.82:0/3480270612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6e0004810 0x7fe6f815c0d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 >> 172.21.15.82:0/3480270612 conn(0x7fe6f806ae60 msgr2=0x7fe6f80ad190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 shutdown_connections 2024-04-25T10:04:06.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.243+0000 7fe6fd089640 1 -- 172.21.15.82:0/3480270612 wait complete. 2024-04-25T10:04:06.270 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:04:06.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.318 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:06.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 Processor -- start 2024-04-25T10:04:06.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 -- start start 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd31815ddd0 0x7fd31806bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31806d5d0 con 0x7fd3180d3e00 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31806cd30 con 0x7fd31815d9f0 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31ecb3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31806d010 con 0x7fd31815ddd0 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd317fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd31815ddd0 0x7fd31806bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31d250640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd3180d3e00 0x7fd31806c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49020/0 (socket says 172.21.15.82:49020) 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd31d250640 1 -- 172.21.15.82:0/3600967861 learned_addr learned my addr 172.21.15.82:0/3600967861 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd3177fe640 1 -- 172.21.15.82:0/3600967861 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224440085 0 0) 0x7fd31806d5d0 con 0x7fd3180d3e00 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd3177fe640 1 -- 172.21.15.82:0/3600967861 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd300003530 con 0x7fd3180d3e00 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd317fff640 1 -- 172.21.15.82:0/3600967861 >> v1:172.21.15.196:6789/0 conn(0x7fd31815d9f0 legacy=0x7fd3180d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd317fff640 1 -- 172.21.15.82:0/3600967861 >> v1:172.21.15.82:6789/0 conn(0x7fd3180d3e00 legacy=0x7fd31806c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.391+0000 7fd317fff640 1 -- 172.21.15.82:0/3600967861 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31806d2f0 con 0x7fd31815ddd0 2024-04-25T10:04:06.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd317fff640 1 --2- 172.21.15.82:0/3600967861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd31806bfc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0097c0 tx=0x7fd30c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0717f0c54e0090 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd3177fe640 1 -- 172.21.15.82:0/3600967861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0ab650 con 0x7fd31815ddd0 2024-04-25T10:04:06.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd3177fe640 1 -- 172.21.15.82:0/3600967861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30c0b0070 con 0x7fd31815ddd0 2024-04-25T10:04:06.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd3177fe640 1 -- 172.21.15.82:0/3600967861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30c0aba10 con 0x7fd31815ddd0 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- 172.21.15.82:0/3600967861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 msgr2=0x7fd31806bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/3600967861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd31806bfc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd30c0097c0 tx=0x7fd30c01c430 comp rx=0 tx=0).stop 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- 172.21.15.82:0/3600967861 shutdown_connections 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/3600967861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd31806bfc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- 172.21.15.82:0/3600967861 >> 172.21.15.82:0/3600967861 conn(0x7fd31809ca80 msgr2=0x7fd31808b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- 172.21.15.82:0/3600967861 shutdown_connections 2024-04-25T10:04:06.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- 172.21.15.82:0/3600967861 wait complete. 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 Processor -- start 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.395+0000 7fd31ecb3640 1 -- start start 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3180d3e00 0x7fd3180b9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd31815d9f0 0x7fd318160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31816b2a0 con 0x7fd31815ddd0 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31816b120 con 0x7fd3180d3e00 2024-04-25T10:04:06.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31816b420 con 0x7fd31815d9f0 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40418/0 (socket says 172.21.15.82:40418) 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 -- 172.21.15.82:0/1198400382 learned_addr learned my addr 172.21.15.82:0/1198400382 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:06.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd317fff640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd31815d9f0 0x7fd318160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 -- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd31815d9f0 msgr2=0x7fd318160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd31815d9f0 0x7fd318160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 -- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3180d3e00 msgr2=0x7fd3180b9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ca4f640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3180d3e00 0x7fd3180b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3180d3e00 0x7fd3180b9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 -- 172.21.15.82:0/1198400382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3180b8220 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31d250640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd31000cbb0 tx=0x7fd31000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310017070 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31000ad40 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310012600 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3180b8500 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3180ca830 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd318068340 con 0x7fd31815ddd0 2024-04-25T10:04:06.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd31001a070 con 0x7fd31815ddd0 2024-04-25T10:04:06.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 0x7fd304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:06.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fd310084fe0 con 0x7fd31815ddd0 2024-04-25T10:04:06.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ca4f640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 0x7fd304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:06.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.399+0000 7fd31ca4f640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 0x7fd304069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd308003d90 tx=0x7fd308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:06.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.403+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd310055720 con 0x7fd31815ddd0 2024-04-25T10:04:06.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:06.551+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3180b43e0 con 0x7fd31815ddd0 2024-04-25T10:04:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.499+0000 7fd3157fa640 1 -- 172.21.15.82:0/1198400382 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd3100597c0 con 0x7fd31815ddd0 2024-04-25T10:04:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.503+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 msgr2=0x7fd304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.503+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 0x7fd304069420 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd308003d90 tx=0x7fd308009040 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.503+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 msgr2=0x7fd3180b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.503+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd31000cbb0 tx=0x7fd31000a430 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 shutdown_connections 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd304066f60 0x7fd304069420 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd31815d9f0 0x7fd318160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd31815ddd0 0x7fd3180b78b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 --2- 172.21.15.82:0/1198400382 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3180d3e00 0x7fd3180b9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 >> 172.21.15.82:0/1198400382 conn(0x7fd31809ca80 msgr2=0x7fd3180b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 shutdown_connections 2024-04-25T10:04:07.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.507+0000 7fd31ecb3640 1 -- 172.21.15.82:0/1198400382 wait complete. 2024-04-25T10:04:07.532 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:04:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 Processor -- start 2024-04-25T10:04:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 -- start start 2024-04-25T10:04:07.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd86415d980 0x7fd8640d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8640d12c0 0x7fd8640d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86415dd60 con 0x7fd8640d21f0 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640d2650 con 0x7fd8640d12c0 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd868a20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86406c810 con 0x7fd86415d980 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8640d12c0 0x7fd8640d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd8640d12c0 0x7fd8640d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40422/0 (socket says 172.21.15.82:40422) 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 -- 172.21.15.82:0/2814853525 learned_addr learned my addr 172.21.15.82:0/2814853525 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 -- 172.21.15.82:0/2814853525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd86415d980 msgr2=0x7fd8640d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd86259c640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd86415d980 0x7fd8640d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd86415d980 0x7fd8640d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 -- 172.21.15.82:0/2814853525 >> v1:172.21.15.82:6789/0 conn(0x7fd8640d21f0 legacy=0x7fd8640ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd861d9b640 1 -- 172.21.15.82:0/2814853525 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097776859 0 0) 0x7fd86415dd60 con 0x7fd8640d21f0 2024-04-25T10:04:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.663+0000 7fd85bfff640 1 -- 172.21.15.82:0/2814853525 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640d4340 con 0x7fd8640d12c0 2024-04-25T10:04:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd85bfff640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8640d12c0 0x7fd8640d1cb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd86406c990 tx=0x7fd84c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5090d71569a8db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd861d9b640 1 -- 172.21.15.82:0/2814853525 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0b1070 con 0x7fd8640d12c0 2024-04-25T10:04:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd861d9b640 1 -- 172.21.15.82:0/2814853525 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c0ac620 con 0x7fd8640d12c0 2024-04-25T10:04:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd861d9b640 1 -- 172.21.15.82:0/2814853525 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0ac9f0 con 0x7fd8640d12c0 2024-04-25T10:04:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- 172.21.15.82:0/2814853525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8640d12c0 msgr2=0x7fd8640d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8640d12c0 0x7fd8640d1cb0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd86406c990 tx=0x7fd84c01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- 172.21.15.82:0/2814853525 shutdown_connections 2024-04-25T10:04:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd8640d12c0 0x7fd8640d1cb0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 --2- 172.21.15.82:0/2814853525 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd86415d980 0x7fd8640d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- 172.21.15.82:0/2814853525 >> 172.21.15.82:0/2814853525 conn(0x7fd86408b000 msgr2=0x7fd8640d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- 172.21.15.82:0/2814853525 shutdown_connections 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- 172.21.15.82:0/2814853525 wait complete. 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 Processor -- start 2024-04-25T10:04:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.667+0000 7fd868a20640 1 -- start start 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8640d12c0 0x7fd864155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8640d21f0 0x7fd86415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86406c4a0 con 0x7fd86415d980 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86406c1c0 con 0x7fd8640d12c0 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86406dcb0 con 0x7fd8640d21f0 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40430/0 (socket says 172.21.15.82:40430) 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 -- 172.21.15.82:0/3531767593 learned_addr learned my addr 172.21.15.82:0/3531767593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85bfff640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8640d21f0 0x7fd86415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd86259c640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8640d12c0 0x7fd864155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 -- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8640d21f0 msgr2=0x7fd86415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8640d21f0 0x7fd86415c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 -- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8640d12c0 msgr2=0x7fd864155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8640d12c0 0x7fd864155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 -- 172.21.15.82:0/3531767593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8641560a0 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd862d9d640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c800 tx=0x7fd85c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c003b80 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c003d20 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0052b0 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86415fe30 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8641603f0 con 0x7fd86415d980 2024-04-25T10:04:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd864067c30 con 0x7fd86415d980 2024-04-25T10:04:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd85c005450 con 0x7fd86415d980 2024-04-25T10:04:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 0x7fd844069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5778+0+0 (secure 0 0 0) 0x7fd85c084ef0 con 0x7fd86415d980 2024-04-25T10:04:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd86259c640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 0x7fd844069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.671+0000 7fd86259c640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 0x7fd844069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd85400b800 tx=0x7fd854008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.675+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd85c055f70 con 0x7fd86415d980 2024-04-25T10:04:07.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:07.839+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8640d08f0 con 0x7fd86415d980 2024-04-25T10:04:08.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.503+0000 7fd85affd640 1 -- 172.21.15.82:0/3531767593 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd85c055870 con 0x7fd86415d980 2024-04-25T10:04:08.508 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 msgr2=0x7fd844069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 0x7fd844069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd85400b800 tx=0x7fd854008040 comp rx=0 tx=0).stop 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 msgr2=0x7fd86415c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00c800 tx=0x7fd85c00ccc0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 shutdown_connections 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd844066f60 0x7fd844069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd8640d21f0 0x7fd86415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd86415d980 0x7fd86415c970 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 --2- 172.21.15.82:0/3531767593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd8640d12c0 0x7fd864155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 >> 172.21.15.82:0/3531767593 conn(0x7fd86408b000 msgr2=0x7fd8640b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 shutdown_connections 2024-04-25T10:04:08.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.507+0000 7fd868a20640 1 -- 172.21.15.82:0/3531767593 wait complete. 2024-04-25T10:04:08.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:04:08.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 Processor -- start 2024-04-25T10:04:08.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 -- start start 2024-04-25T10:04:08.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c40b3cf0 0x7fa0c40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c415e6c0 0x7fa0c40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40cbf10 con 0x7fa0c40cfad0 2024-04-25T10:04:08.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40cff40 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0cbbf7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c415eae0 con 0x7fa0c415e6c0 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0c9993640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c40b3cf0 0x7fa0c40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0c9993640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c40b3cf0 0x7fa0c40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40438/0 (socket says 172.21.15.82:40438) 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.651+0000 7fa0c9993640 1 -- 172.21.15.82:0/598833200 learned_addr learned my addr 172.21.15.82:0/598833200 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c9993640 1 -- 172.21.15.82:0/598833200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c415e6c0 msgr2=0x7fa0c40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c9993640 1 --2- 172.21.15.82:0/598833200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c415e6c0 0x7fa0c40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c9993640 1 -- 172.21.15.82:0/598833200 >> v1:172.21.15.196:6789/0 conn(0x7fa0c40cfad0 legacy=0x7fa0c415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c9993640 1 -- 172.21.15.82:0/598833200 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40cc1f0 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c9993640 1 --2- 172.21.15.82:0/598833200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c40cf590 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc64848d64e68e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c8991640 1 -- 172.21.15.82:0/598833200 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b0070 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c8991640 1 -- 172.21.15.82:0/598833200 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40ab600 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0c8991640 1 -- 172.21.15.82:0/598833200 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40ab9d0 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/598833200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 msgr2=0x7fa0c40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/598833200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c40cf590 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009870 tx=0x7fa0b401c520 comp rx=0 tx=0).stop 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/598833200 shutdown_connections 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/598833200 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c415e6c0 0x7fa0c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/598833200 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c40cf590 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/598833200 >> 172.21.15.82:0/598833200 conn(0x7fa0c406ae60 msgr2=0x7fa0c406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/598833200 shutdown_connections 2024-04-25T10:04:08.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.655+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/598833200 wait complete. 2024-04-25T10:04:08.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 Processor -- start 2024-04-25T10:04:08.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- start start 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c40cfad0 0x7fa0c41550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c415e6c0 0x7fa0c4158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b4250 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b40d0 con 0x7fa0c415e6c0 2024-04-25T10:04:08.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40cb840 con 0x7fa0c40cfad0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40446/0 (socket says 172.21.15.82:40446) 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 -- 172.21.15.82:0/2525328435 learned_addr learned my addr 172.21.15.82:0/2525328435 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9192640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c40cfad0 0x7fa0c41550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 -- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c40cfad0 msgr2=0x7fa0c41550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c40cfad0 0x7fa0c41550f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0ca194640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c415e6c0 0x7fa0c4158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 -- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c415e6c0 msgr2=0x7fa0c4158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c415e6c0 0x7fa0c4158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 -- 172.21.15.82:0/2525328435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4155ec0 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0c9993640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ae040 tx=0x7fa0b401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40ba020 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40b6640 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4156180 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40b0040 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4156740 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0b40bd020 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.659+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c404efa0 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.663+0000 7fa0ba7fc640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.663+0000 7fa0c9192640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.663+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa0b411f100 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.663+0000 7fa0c9192640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 0x7fa0a8069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40ceeb0 tx=0x7fa0c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.667+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0b40c4030 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.839+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0c40b3060 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0ba7fc640 1 -- 172.21.15.82:0/2525328435 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0b4016110 con 0x7fa0c40b3cf0 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 msgr2=0x7fa0a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 0x7fa0a8069420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40ceeb0 tx=0x7fa0c0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 msgr2=0x7fa0c41592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40ae040 tx=0x7fa0b401ad60 comp rx=0 tx=0).stop 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 shutdown_connections 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0a8066f60 0x7fa0a8069420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c40cfad0 0x7fa0c41550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c415e6c0 0x7fa0c4158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 --2- 172.21.15.82:0/2525328435 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c40b3cf0 0x7fa0c41592c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 >> 172.21.15.82:0/2525328435 conn(0x7fa0c406ae60 msgr2=0x7fa0c415f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:08.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 shutdown_connections 2024-04-25T10:04:08.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.847+0000 7fa0cbbf7640 1 -- 172.21.15.82:0/2525328435 wait complete. 2024-04-25T10:04:08.869 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T10:04:08.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 Processor -- start 2024-04-25T10:04:08.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 -- start start 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5880ce510 0x7fb5880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb58806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5880cee30 con 0x7fb58806ec60 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880cc2b0 con 0x7fb5880ce510 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58eda3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58806e290 con 0x7fb588144ca0 2024-04-25T10:04:08.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb58806ec60 0x7fb588142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49026/0 (socket says 172.21.15.82:49026) 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 -- 172.21.15.82:0/3937300384 learned_addr learned my addr 172.21.15.82:0/3937300384 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58cb3f640 1 --2- 172.21.15.82:0/3937300384 >> v2:172.21.15.196:3300/0 conn(0x7fb5880ce510 0x7fb5880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb587fff640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb58806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2805363824 0 0) 0x7fb5880cee30 con 0x7fb58806ec60 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb570003530 con 0x7fb58806ec60 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1767286546 0 0) 0x7fb570003530 con 0x7fb58806ec60 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 msgr2=0x7fb58806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb58806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 >> v2:172.21.15.196:3300/0 conn(0x7fb5880ce510 msgr2=0x7fb5880ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 --2- 172.21.15.82:0/3937300384 >> v2:172.21.15.196:3300/0 conn(0x7fb5880ce510 0x7fb5880ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880ccc70 con 0x7fb58806ec60 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb580001a00 con 0x7fb58806ec60 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 >> v1:172.21.15.82:6789/0 conn(0x7fb58806ec60 legacy=0x7fb588142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb570003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb570004000 0x7fb570004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb570004d50 0x7fb5700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cc2b0 con 0x7fb570004d50 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58806e290 con 0x7fb570004000 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57000bae0 con 0x7fb588144ca0 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb570004d50 0x7fb5700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb587fff640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb570004000 0x7fb570004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58cb3f640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb570003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 -- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 msgr2=0x7fb570003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb588144ca0 0x7fb570003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 -- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb570004000 msgr2=0x7fb570004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:08.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb570004000 0x7fb570004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:08.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.987+0000 7fb58d340640 1 -- 172.21.15.82:0/3937300384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb570003530 con 0x7fb570004d50 2024-04-25T10:04:08.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58d340640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb570004d50 0x7fb5700051d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb5800013e0 tx=0x7fb58001d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd82da3a3d82da96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:08.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb58000a410 con 0x7fb570004d50 2024-04-25T10:04:08.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5800ad630 con 0x7fb570004d50 2024-04-25T10:04:08.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb5877fe640 1 -- 172.21.15.82:0/3937300384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5800a5e60 con 0x7fb570004d50 2024-04-25T10:04:08.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- 172.21.15.82:0/3937300384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb570004d50 msgr2=0x7fb5700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3937300384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb570004d50 0x7fb5700051d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb5800013e0 tx=0x7fb58001d280 comp rx=0 tx=0).stop 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58d340640 1 -- 172.21.15.82:0/3937300384 reap_dead start 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- 172.21.15.82:0/3937300384 shutdown_connections 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- 172.21.15.82:0/3937300384 >> 172.21.15.82:0/3937300384 conn(0x7fb5880b11a0 msgr2=0x7fb5880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- 172.21.15.82:0/3937300384 shutdown_connections 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- 172.21.15.82:0/3937300384 wait complete. 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 Processor -- start 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.991+0000 7fb58eda3640 1 -- start start 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588140100 0x7fb5880ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 0x7fb5880aad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cc900 con 0x7fb5880ab240 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58806df20 con 0x7fb588140100 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb588142d90 con 0x7fb5881404e0 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40474/0 (socket says 172.21.15.82:40474) 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 -- 172.21.15.82:0/3548659914 learned_addr learned my addr 172.21.15.82:0/3548659914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58cb3f640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588140100 0x7fb5880ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb587fff640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 0x7fb5880aad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 -- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 msgr2=0x7fb5880aad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 0x7fb5880aad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 -- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588140100 msgr2=0x7fb5880ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588140100 0x7fb5880ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 -- 172.21.15.82:0/3548659914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880ab8e0 con 0x7fb5880ab240 2024-04-25T10:04:09.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb587fff640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 0x7fb5880aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58d340640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb58001db20 tx=0x7fb570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5800bb070 con 0x7fb5880ab240 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58000a3f0 con 0x7fb5880ab240 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5880abbc0 con 0x7fb5880ab240 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580001f80 con 0x7fb5880ab240 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5880ac180 con 0x7fb5880ab240 2024-04-25T10:04:09.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58804efa0 con 0x7fb5880ab240 2024-04-25T10:04:09.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb5800be070 con 0x7fb5880ab240 2024-04-25T10:04:09.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 0x7fb574069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb58011ec70 con 0x7fb5880ab240 2024-04-25T10:04:09.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.995+0000 7fb58cb3f640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 0x7fb574069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.999+0000 7fb58cb3f640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 0x7fb574069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ab0 tx=0x7fb578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:08.999+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb580016f90 con 0x7fb5880ab240 2024-04-25T10:04:09.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.219+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7fb5880b1cc0 con 0x7fb5880ab240 2024-04-25T10:04:09.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.519+0000 7fb5857fa640 1 -- 172.21.15.82:0/3548659914 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v131) v1 ==== 142+0+0 (secure 0 0 0) 0x7fb5800efe60 con 0x7fb5880ab240 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 msgr2=0x7fb574069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 0x7fb574069450 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb578009ab0 tx=0x7fb578008040 comp rx=0 tx=0).stop 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 msgr2=0x7fb588154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb58001db20 tx=0x7fb570005840 comp rx=0 tx=0).stop 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 shutdown_connections 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb574066f90 0x7fb574069450 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb5881404e0 0x7fb5880aad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb5880ab240 0x7fb588154680 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 --2- 172.21.15.82:0/3548659914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb588140100 0x7fb5880ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 >> 172.21.15.82:0/3548659914 conn(0x7fb5880b11a0 msgr2=0x7fb588071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 shutdown_connections 2024-04-25T10:04:09.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.523+0000 7fb58eda3640 1 -- 172.21.15.82:0/3548659914 wait complete. 2024-04-25T10:04:09.547 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:04:09.547 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T10:04:09.547 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:04:09.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:09.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 Processor -- start 2024-04-25T10:04:09.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 -- start start 2024-04-25T10:04:09.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f511c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c0ccc70 con 0x7f511c144ca0 2024-04-25T10:04:09.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c06dc40 con 0x7f511c0ce510 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5124676640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0cc2b0 con 0x7f511c1451f0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121c11640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f511c144ca0 0x7f511c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49028/0 (socket says 172.21.15.82:49028) 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121c11640 1 -- 172.21.15.82:0/1852503048 learned_addr learned my addr 172.21.15.82:0/1852503048 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122c13640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f511c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295910838 0 0) 0x7f511c0ccc70 con 0x7f511c144ca0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5108003530 con 0x7f511c144ca0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2745282793 0 0) 0x7f5108003530 con 0x7f511c144ca0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 msgr2=0x7f511c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f511c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 >> v1:172.21.15.196:6789/0 conn(0x7f511c0ce510 legacy=0x7f511c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c06df20 con 0x7f511c144ca0 2024-04-25T10:04:09.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5110001680 con 0x7f511c144ca0 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 >> v1:172.21.15.82:6789/0 conn(0x7f511c144ca0 legacy=0x7f511c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5108003a60 0x7f5108003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f5108004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108004d50 0x7f51080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0cc2b0 con 0x7f5108003a60 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510800bba0 con 0x7f5108004d50 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510800bda0 con 0x7f511c1451f0 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5108003a60 0x7f5108003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121c11640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f5108004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122c13640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108004d50 0x7f51080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 -- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 msgr2=0x7f5108004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f5108004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 -- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108004d50 msgr2=0x7f51080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108004d50 0x7f51080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122412640 1 -- 172.21.15.82:0/1852503048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108003530 con 0x7f5108003a60 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5121c11640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c1451f0 0x7f5108004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:09.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.671+0000 7f5122c13640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108004d50 0x7f51080051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:09.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5122412640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5108003a60 0x7f5108003e80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5118001750 tx=0x7f511801c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddbbc2852cee68d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51180013a0 con 0x7f5108003a60 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51180052f0 con 0x7f5108003a60 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5121410640 1 -- 172.21.15.82:0/1852503048 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5118003840 con 0x7f5108003a60 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- 172.21.15.82:0/1852503048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5108003a60 msgr2=0x7f5108003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 --2- 172.21.15.82:0/1852503048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5108003a60 0x7f5108003e80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5118001750 tx=0x7f511801c320 comp rx=0 tx=0).stop 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5122c13640 1 -- 172.21.15.82:0/1852503048 reap_dead start 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- 172.21.15.82:0/1852503048 shutdown_connections 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- 172.21.15.82:0/1852503048 >> 172.21.15.82:0/1852503048 conn(0x7f511c0b11a0 msgr2=0x7f511c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- 172.21.15.82:0/1852503048 shutdown_connections 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- 172.21.15.82:0/1852503048 wait complete. 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 Processor -- start 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.675+0000 7f5124676640 1 -- start start 2024-04-25T10:04:09.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108003a60 0x7f511c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c0c9140 0x7f511c0c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c0cc900 con 0x7f511c150500 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c06e290 con 0x7f5108003a60 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c142680 con 0x7f511c0c9140 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40480/0 (socket says 172.21.15.82:40480) 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 -- 172.21.15.82:0/4118206138 learned_addr learned my addr 172.21.15.82:0/4118206138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5122412640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108003a60 0x7f511c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5122c13640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c0c9140 0x7f511c0c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 -- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c0c9140 msgr2=0x7f511c0c95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c0c9140 0x7f511c0c95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 -- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108003a60 msgr2=0x7f511c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108003a60 0x7f511c14ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 -- 172.21.15.82:0/4118206138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c14f2d0 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5121c11640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5110010b40 tx=0x7f5110005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511001b070 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110011040 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c1510e0 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511000ecd0 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c152b70 con 0x7f511c150500 2024-04-25T10:04:09.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c04efa0 con 0x7f511c150500 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f511001e070 con 0x7f511c150500 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 0x7f5100069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5110087770 con 0x7f511c150500 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5122412640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 0x7f5100069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.679+0000 7f5122412640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 0x7f5100069450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5108004790 tx=0x7f51180011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.683+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5110058980 con 0x7f511c150500 2024-04-25T10:04:09.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.847+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f511c0b1cc0 con 0x7f511c150500 2024-04-25T10:04:09.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f510effd640 1 -- 172.21.15.82:0/4118206138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+13303 (secure 0 0 0) 0x7f5110058280 con 0x7f511c150500 2024-04-25T10:04:09.854 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:09.854 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":44,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:08.503150+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/560740519":"2024-04-26T10:04:05.237204+0000","172.21.15.82:6836/560740519":"2024-04-26T10:04:05.237204+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:09.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 msgr2=0x7f5100069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 0x7f5100069450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5108004790 tx=0x7f51180011a0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 msgr2=0x7f511c150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5110010b40 tx=0x7f5110005720 comp rx=0 tx=0).stop 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 shutdown_connections 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5100066f90 0x7f5100069450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f511c0c9140 0x7f511c0c95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f511c150500 0x7f511c150980 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 --2- 172.21.15.82:0/4118206138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5108003a60 0x7f511c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 >> 172.21.15.82:0/4118206138 conn(0x7f511c0b11a0 msgr2=0x7f511c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 shutdown_connections 2024-04-25T10:04:09.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.851+0000 7f5124676640 1 -- 172.21.15.82:0/4118206138 wait complete. 2024-04-25T10:04:09.873 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:09.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:09.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 Processor -- start 2024-04-25T10:04:09.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 -- start start 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7280cf590 0x7fc7280cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280cbd20 con 0x7fc728072000 2024-04-25T10:04:09.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280cc000 con 0x7fc72815b6b0 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72ed0f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc72815ba90 con 0x7fc7280cf590 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc727fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7280cf590 0x7fc7280cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72d2ac640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc728072000 0x7fc728072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49030/0 (socket says 172.21.15.82:49030) 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc727fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc7280cf590 0x7fc7280cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40494/0 (socket says 172.21.15.82:40494) 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc72d2ac640 1 -- 172.21.15.82:0/1210215078 learned_addr learned my addr 172.21.15.82:0/1210215078 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc727fff640 1 -- 172.21.15.82:0/1210215078 >> v1:172.21.15.196:6789/0 conn(0x7fc72815b6b0 legacy=0x7fc7280ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc727fff640 1 -- 172.21.15.82:0/1210215078 >> v1:172.21.15.82:6789/0 conn(0x7fc728072000 legacy=0x7fc728072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.987+0000 7fc727fff640 1 -- 172.21.15.82:0/1210215078 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280cc2e0 con 0x7fc7280cf590 2024-04-25T10:04:09.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc727fff640 1 --2- 172.21.15.82:0/1210215078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7280cf590 0x7fc7280cfa20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d90 tx=0x7fc72001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2808f437a78faae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc7277fe640 1 -- 172.21.15.82:0/1210215078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200b2070 con 0x7fc7280cf590 2024-04-25T10:04:09.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc7277fe640 1 -- 172.21.15.82:0/1210215078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7200a58e0 con 0x7fc7280cf590 2024-04-25T10:04:09.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc7277fe640 1 -- 172.21.15.82:0/1210215078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7200a5cb0 con 0x7fc7280cf590 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- 172.21.15.82:0/1210215078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7280cf590 msgr2=0x7fc7280cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/1210215078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7280cf590 0x7fc7280cfa20 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc720009d90 tx=0x7fc72001c930 comp rx=0 tx=0).stop 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- 172.21.15.82:0/1210215078 shutdown_connections 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/1210215078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7280cf590 0x7fc7280cfa20 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- 172.21.15.82:0/1210215078 >> 172.21.15.82:0/1210215078 conn(0x7fc72806ae60 msgr2=0x7fc72806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- 172.21.15.82:0/1210215078 shutdown_connections 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- 172.21.15.82:0/1210215078 wait complete. 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 Processor -- start 2024-04-25T10:04:09.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- start start 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7280cf590 0x7fc728159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc72815b6b0 0x7fc7280ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:09.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280caf30 con 0x7fc728072000 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280cadb0 con 0x7fc7280cf590 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72ed0f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281692b0 con 0x7fc72815b6b0 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72caab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72caab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40498/0 (socket says 172.21.15.82:40498) 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.991+0000 7fc72caab640 1 -- 172.21.15.82:0/3062884262 learned_addr learned my addr 172.21.15.82:0/3062884262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 -- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc72815b6b0 msgr2=0x7fc7280ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc72815b6b0 0x7fc7280ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 -- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7280cf590 msgr2=0x7fc728159680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7280cf590 0x7fc728159680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 -- 172.21.15.82:0/3062884262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728158cb0 con 0x7fc728072000 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72caab640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b3e0 tx=0x7fc71800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718015020 con 0x7fc728072000 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc718011850 con 0x7fc728072000 2024-04-25T10:04:09.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc728158f90 con 0x7fc728072000 2024-04-25T10:04:10.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718011c40 con 0x7fc728072000 2024-04-25T10:04:10.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc72814e6b0 con 0x7fc728072000 2024-04-25T10:04:10.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72804efa0 con 0x7fc728072000 2024-04-25T10:04:10.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc718018020 con 0x7fc728072000 2024-04-25T10:04:10.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 0x7fc708069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc718084e30 con 0x7fc728072000 2024-04-25T10:04:10.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc727fff640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 0x7fc708069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.995+0000 7fc727fff640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 0x7fc708069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc7200af000 tx=0x7fc7200a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:09.999+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc71801f020 con 0x7fc728072000 2024-04-25T10:04:10.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7280680e0 con 0x7fc728072000 2024-04-25T10:04:10.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc7257fa640 1 -- 172.21.15.82:0/3062884262 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7180172c0 con 0x7fc728072000 2024-04-25T10:04:10.162 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:04:10.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 msgr2=0x7fc708069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 0x7fc708069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc7200af000 tx=0x7fc7200a7070 comp rx=0 tx=0).stop 2024-04-25T10:04:10.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 msgr2=0x7fc7280c2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc71800b3e0 tx=0x7fc71800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 shutdown_connections 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc708066f90 0x7fc708069450 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc72815b6b0 0x7fc7280ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7280cf590 0x7fc728159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 --2- 172.21.15.82:0/3062884262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc728072000 0x7fc7280c2270 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 >> 172.21.15.82:0/3062884262 conn(0x7fc72806ae60 msgr2=0x7fc7280b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 shutdown_connections 2024-04-25T10:04:10.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.159+0000 7fc72ed0f640 1 -- 172.21.15.82:0/3062884262 wait complete. 2024-04-25T10:04:10.181 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:04:10.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.291+0000 7fcfe6d57640 1 Processor -- start 2024-04-25T10:04:10.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.291+0000 7fcfe6d57640 1 -- start start 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe6d57640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe0144ca0 0x7fcfe006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe6d57640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfe006ec60 0x7fcfe0142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe6d57640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe00ccc70 con 0x7fcfe00ce510 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe6d57640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe00cc2b0 con 0x7fcfe006ec60 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe6d57640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe006e290 con 0x7fcfe0144ca0 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfe006ec60 0x7fcfe0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcfe006ec60 0x7fcfe0142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40512/0 (socket says 172.21.15.82:40512) 2024-04-25T10:04:10.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 -- 172.21.15.82:0/56976822 learned_addr learned my addr 172.21.15.82:0/56976822 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe4af3640 1 --1- 172.21.15.82:0/56976822 >> v1:172.21.15.82:6789/0 conn(0x7fcfe00ce510 0x7fcfe00ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49032/0 (socket says 172.21.15.82:49032) 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfdffff640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe0144ca0 0x7fcfe006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 -- 172.21.15.82:0/56976822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe0144ca0 msgr2=0x7fcfe006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe0144ca0 0x7fcfe006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 -- 172.21.15.82:0/56976822 >> v1:172.21.15.82:6789/0 conn(0x7fcfe00ce510 legacy=0x7fcfe00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 -- 172.21.15.82:0/56976822 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe006dc40 con 0x7fcfe006ec60 2024-04-25T10:04:10.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfdf7fe640 1 -- 172.21.15.82:0/56976822 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192917479 0 0) 0x7fcfe00ccc70 con 0x7fcfe00ce510 2024-04-25T10:04:10.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfe52f4640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe006ec60 0x7fcfe0142680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009e50 tx=0x7fcfd801f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4494b223ec6ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfdf7fe640 1 -- 172.21.15.82:0/56976822 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80b4070 con 0x7fcfe006ec60 2024-04-25T10:04:10.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfdf7fe640 1 -- 172.21.15.82:0/56976822 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80af400 con 0x7fcfe006ec60 2024-04-25T10:04:10.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.295+0000 7fcfdf7fe640 1 -- 172.21.15.82:0/56976822 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80af7d0 con 0x7fcfe006ec60 2024-04-25T10:04:10.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/56976822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe006ec60 msgr2=0x7fcfe0142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe006ec60 0x7fcfe0142680 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009e50 tx=0x7fcfd801f210 comp rx=0 tx=0).stop 2024-04-25T10:04:10.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/56976822 shutdown_connections 2024-04-25T10:04:10.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe006ec60 0x7fcfe0142680 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/56976822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe0144ca0 0x7fcfe006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/56976822 >> 172.21.15.82:0/56976822 conn(0x7fcfe00b11a0 msgr2=0x7fcfe00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/56976822 shutdown_connections 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/56976822 wait complete. 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 Processor -- start 2024-04-25T10:04:10.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- start start 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe006ec60 0x7fcfe0141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe00ce510 0x7fcfe00aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe01451f0 con 0x7fcfe0144ca0 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe00cc900 con 0x7fcfe00ce510 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe0145370 con 0x7fcfe006ec60 2024-04-25T10:04:10.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40516/0 (socket says 172.21.15.82:40516) 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 -- 172.21.15.82:0/931467625 learned_addr learned my addr 172.21.15.82:0/931467625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 -- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe006ec60 msgr2=0x7fcfe0141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe006ec60 0x7fcfe0141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 -- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe00ce510 msgr2=0x7fcfe00aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe00ce510 0x7fcfe00aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 -- 172.21.15.82:0/931467625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe01367f0 con 0x7fcfe0144ca0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe52f4640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80b2040 tx=0x7fcfd801bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80b4040 con 0x7fcfe0144ca0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd80c0070 con 0x7fcfe0144ca0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe0136ad0 con 0x7fcfe0144ca0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8009510 con 0x7fcfe0144ca0 2024-04-25T10:04:10.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.299+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe0137090 con 0x7fcfe0144ca0 2024-04-25T10:04:10.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe004efa0 con 0x7fcfe0144ca0 2024-04-25T10:04:10.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfd80a8d10 con 0x7fcfe0144ca0 2024-04-25T10:04:10.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfdd7fa640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:10.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfe4af3640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:10.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcfd8122f80 con 0x7fcfe0144ca0 2024-04-25T10:04:10.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.303+0000 7fcfe4af3640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 0x7fcfc4069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40011f0 tx=0x7fcfd4007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:10.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.307+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfd8018d70 con 0x7fcfe0144ca0 2024-04-25T10:04:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:10.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fcfe00b0f80 con 0x7fcfe0144ca0 2024-04-25T10:04:11.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.523+0000 7fcfdd7fa640 1 -- 172.21.15.82:0/931467625 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v133) v1 ==== 152+0+0 (secure 0 0 0) 0x7fcfd80c3360 con 0x7fcfe0144ca0 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 msgr2=0x7fcfc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 0x7fcfc4069420 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40011f0 tx=0x7fcfd4007930 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 msgr2=0x7fcfe00af2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fcfd80b2040 tx=0x7fcfd801bfb0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 shutdown_connections 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcfc4066f60 0x7fcfc4069420 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcfe0144ca0 0x7fcfe00af2e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcfe00ce510 0x7fcfe00aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 --2- 172.21.15.82:0/931467625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcfe006ec60 0x7fcfe0141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 >> 172.21.15.82:0/931467625 conn(0x7fcfe00b11a0 msgr2=0x7fcfe0071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 shutdown_connections 2024-04-25T10:04:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.527+0000 7fcfe6d57640 1 -- 172.21.15.82:0/931467625 wait complete. 2024-04-25T10:04:11.555 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:04:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.671+0000 7f67b46f1640 1 Processor -- start 2024-04-25T10:04:11.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.671+0000 7f67b46f1640 1 -- start start 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b46f1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f67ac15ecb0 0x7f67ac0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b46f1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f67ac0c9ef0 0x7f67ac0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b46f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ac0ca4c0 con 0x7f67ac0cf590 2024-04-25T10:04:11.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b46f1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac0ca780 con 0x7f67ac15ecb0 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b46f1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ac15f0f0 con 0x7f67ac0c9ef0 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b248d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f67ac0cf590 0x7f67ac0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49042/0 (socket says 172.21.15.82:49042) 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b248d640 1 -- 172.21.15.82:0/2729916403 learned_addr learned my addr 172.21.15.82:0/2729916403 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 --2- 172.21.15.82:0/2729916403 >> v2:172.21.15.82:3300/0 conn(0x7f67ac15ecb0 0x7f67ac0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b148b640 1 -- 172.21.15.82:0/2729916403 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116618560 0 0) 0x7f67ac0ca4c0 con 0x7f67ac0cf590 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b2c8e640 1 --2- 172.21.15.82:0/2729916403 >> v2:172.21.15.196:3300/0 conn(0x7f67ac0c9ef0 0x7f67ac0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b148b640 1 -- 172.21.15.82:0/2729916403 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6798003530 con 0x7f67ac0cf590 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 -- 172.21.15.82:0/2729916403 >> v2:172.21.15.196:3300/0 conn(0x7f67ac0c9ef0 msgr2=0x7f67ac0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 --2- 172.21.15.82:0/2729916403 >> v2:172.21.15.196:3300/0 conn(0x7f67ac0c9ef0 0x7f67ac0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 -- 172.21.15.82:0/2729916403 >> v1:172.21.15.82:6789/0 conn(0x7f67ac0cf590 legacy=0x7f67ac0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 -- 172.21.15.82:0/2729916403 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0cd690 con 0x7f67ac15ecb0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b1c8c640 1 --2- 172.21.15.82:0/2729916403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac0c99b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0b38c0 tx=0x7f679c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f7d030912ed20f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b148b640 1 -- 172.21.15.82:0/2729916403 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0b1070 con 0x7f67ac15ecb0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b148b640 1 -- 172.21.15.82:0/2729916403 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679c0ac5f0 con 0x7f67ac15ecb0 2024-04-25T10:04:11.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.675+0000 7f67b148b640 1 -- 172.21.15.82:0/2729916403 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679c0ac9a0 con 0x7f67ac15ecb0 2024-04-25T10:04:11.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/2729916403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 msgr2=0x7f67ac0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- 172.21.15.82:0/2729916403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac0c99b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0b38c0 tx=0x7f679c01c480 comp rx=0 tx=0).stop 2024-04-25T10:04:11.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/2729916403 shutdown_connections 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- 172.21.15.82:0/2729916403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac0c99b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- 172.21.15.82:0/2729916403 >> v2:172.21.15.196:3300/0 conn(0x7f67ac0c9ef0 0x7f67ac0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/2729916403 >> 172.21.15.82:0/2729916403 conn(0x7f67ac06ae60 msgr2=0x7f67ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/2729916403 shutdown_connections 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/2729916403 wait complete. 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 Processor -- start 2024-04-25T10:04:11.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- start start 2024-04-25T10:04:11.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ac0c9ef0 0x7f67ac098b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ac0cf590 0x7f67ac164090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac165860 con 0x7f67ac15ecb0 2024-04-25T10:04:11.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1656e0 con 0x7f67ac0cf590 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ac1659e0 con 0x7f67ac0c9ef0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40534/0 (socket says 172.21.15.82:40534) 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 -- 172.21.15.82:0/353553159 learned_addr learned my addr 172.21.15.82:0/353553159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 -- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ac0c9ef0 msgr2=0x7f67ac098b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ac0c9ef0 0x7f67ac098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 -- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ac0cf590 msgr2=0x7f67ac164090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ac0cf590 0x7f67ac164090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 -- 172.21.15.82:0/353553159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0aeb60 con 0x7f67ac15ecb0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b2c8e640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f67a8007e70 tx=0x7f67a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8014620 con 0x7f67ac15ecb0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a8019070 con 0x7f67ac15ecb0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac167c70 con 0x7f67ac15ecb0 2024-04-25T10:04:11.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a8014a20 con 0x7f67ac15ecb0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.679+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ac168230 con 0x7f67ac15ecb0 2024-04-25T10:04:11.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ac04efa0 con 0x7f67ac15ecb0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67a8014bc0 con 0x7f67ac15ecb0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67a2ffd640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 0x7f6790069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f67a8085740 con 0x7f67ac15ecb0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67b248d640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 0x7f6790069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:11.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.683+0000 7f67b248d640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 0x7f6790069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67a4007fd0 tx=0x7f67a4007900 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:11.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.687+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67a8011260 con 0x7f67ac15ecb0 2024-04-25T10:04:11.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.843+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f67ac0c7900 con 0x7f67ac15ecb0 2024-04-25T10:04:11.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.847+0000 7f67a2ffd640 1 -- 172.21.15.82:0/353553159 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67a8056950 con 0x7f67ac15ecb0 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 msgr2=0x7f6790069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 0x7f6790069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f67a4007fd0 tx=0x7f67a4007900 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 msgr2=0x7f67ac1645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f67a8007e70 tx=0x7f67a800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 shutdown_connections 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6790066f60 0x7f6790069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f67ac15ecb0 0x7f67ac1645d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f67ac0cf590 0x7f67ac164090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 --2- 172.21.15.82:0/353553159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f67ac0c9ef0 0x7f67ac098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:11.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 >> 172.21.15.82:0/353553159 conn(0x7f67ac06ae60 msgr2=0x7f67ac0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 shutdown_connections 2024-04-25T10:04:11.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:11.851+0000 7f67b46f1640 1 -- 172.21.15.82:0/353553159 wait complete. 2024-04-25T10:04:11.869 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:04:11.870 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:11.906 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:04:11.967 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:04:11.967 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:04:11.967 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:11.973 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:11.973 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:04:11.973 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:11.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:11.979 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:04:11.979 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:12.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:04:12.022 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:04:12.022 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:04:12.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:04:12.028 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:04:12.028 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:12.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.183+0000 7f42637ee640 1 Processor -- start 2024-04-25T10:04:12.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.183+0000 7f42637ee640 1 -- start start 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f425c15ecc0 0x7f425c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0b2090 con 0x7f425c15a700 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0b43e0 con 0x7f425c0b3f50 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c0b46c0 con 0x7f425c15ecc0 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f425c15ecc0 0x7f425c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f425c15ecc0 0x7f425c06e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40536/0 (socket says 172.21.15.82:40536) 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f426158a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f425c15a700 0x7f425c0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49056/0 (socket says 172.21.15.82:49056) 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 -- 172.21.15.82:0/860062391 learned_addr learned my addr 172.21.15.82:0/860062391 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 -- 172.21.15.82:0/860062391 >> v1:172.21.15.196:6789/0 conn(0x7f425c0b3f50 legacy=0x7f425c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 -- 172.21.15.82:0/860062391 >> v1:172.21.15.82:6789/0 conn(0x7f425c15a700 legacy=0x7f425c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 -- 172.21.15.82:0/860062391 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0cd6f0 con 0x7f425c15ecc0 2024-04-25T10:04:12.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4261d8b640 1 --2- 172.21.15.82:0/860062391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c15ecc0 0x7f425c06e1a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4254009e50 tx=0x7f425401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7557a11bc2c7a6b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4253fff640 1 -- 172.21.15.82:0/860062391 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540a5670 con 0x7f425c15ecc0 2024-04-25T10:04:12.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4253fff640 1 -- 172.21.15.82:0/860062391 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540a5810 con 0x7f425c15ecc0 2024-04-25T10:04:12.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f4253fff640 1 -- 172.21.15.82:0/860062391 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540a5be0 con 0x7f425c15ecc0 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 -- 172.21.15.82:0/860062391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c15ecc0 msgr2=0x7f425c06e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.187+0000 7f42637ee640 1 --2- 172.21.15.82:0/860062391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c15ecc0 0x7f425c06e1a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4254009e50 tx=0x7f425401c8a0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/860062391 shutdown_connections 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 --2- 172.21.15.82:0/860062391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c15ecc0 0x7f425c06e1a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/860062391 >> 172.21.15.82:0/860062391 conn(0x7f425c0b11a0 msgr2=0x7f425c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/860062391 shutdown_connections 2024-04-25T10:04:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/860062391 wait complete. 2024-04-25T10:04:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 Processor -- start 2024-04-25T10:04:12.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- start start 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f425c15a700 0x7f425c15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f425c15ecc0 0x7f425c141ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0cd310 con 0x7f425c14efd0 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0ce600 con 0x7f425c15a700 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0cd490 con 0x7f425c15ecc0 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40542/0 (socket says 172.21.15.82:40542) 2024-04-25T10:04:12.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 -- 172.21.15.82:0/2703548225 learned_addr learned my addr 172.21.15.82:0/2703548225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 -- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f425c15ecc0 msgr2=0x7f425c141ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f425c15ecc0 0x7f425c141ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 -- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f425c15a700 msgr2=0x7f425c15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f425c15a700 0x7f425c15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 -- 172.21.15.82:0/2703548225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c142420 con 0x7f425c14efd0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4261d8b640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4254009e50 tx=0x7f4254019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540bc020 con 0x7f425c14efd0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540b8670 con 0x7f425c14efd0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540b8a40 con 0x7f425c14efd0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c1568f0 con 0x7f425c14efd0 2024-04-25T10:04:12.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c156eb0 con 0x7f425c14efd0 2024-04-25T10:04:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.191+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c04efa0 con 0x7f425c14efd0 2024-04-25T10:04:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.195+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42540bf020 con 0x7f425c14efd0 2024-04-25T10:04:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.195+0000 7f4251ffb640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.195+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f425411ff80 con 0x7f425c14efd0 2024-04-25T10:04:12.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.195+0000 7f426158a640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.195+0000 7f426158a640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 0x7f4240069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f425c0b2260 tx=0x7f42580079b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.199+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f42540c6030 con 0x7f425c14efd0 2024-04-25T10:04:12.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.391+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f425c0b1790 con 0x7f425c14efd0 2024-04-25T10:04:12.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.391+0000 7f4251ffb640 1 -- 172.21.15.82:0/2703548225 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f42540165e0 con 0x7f425c14efd0 2024-04-25T10:04:12.396 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:12.396 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4996,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2859518619","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2859518619},{"type":"v1","addr":"172.21.15.196:6833","nonce":2859518619}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4999,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1622363756","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1622363756},{"type":"v1","addr":"172.21.15.196:6835","nonce":1622363756}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5436,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/192623549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":192623549},{"type":"v1","addr":"172.21.15.82:6835","nonce":192623549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5853,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/21060988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":21060988},{"type":"v1","addr":"172.21.15.82:6837","nonce":21060988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T10:04:12.396 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 19 2024-04-25T10:04:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 msgr2=0x7f4240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 0x7f4240069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f425c0b2260 tx=0x7f42580079b0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 msgr2=0x7f425c157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4254009e50 tx=0x7f4254019640 comp rx=0 tx=0).stop 2024-04-25T10:04:12.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 shutdown_connections 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4240066f60 0x7f4240069420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f425c15ecc0 0x7f425c141ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f425c14efd0 0x7f425c157990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 --2- 172.21.15.82:0/2703548225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f425c15a700 0x7f425c15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 >> 172.21.15.82:0/2703548225 conn(0x7f425c0b11a0 msgr2=0x7f425c155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 shutdown_connections 2024-04-25T10:04:12.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.395+0000 7f42637ee640 1 -- 172.21.15.82:0/2703548225 wait complete. 2024-04-25T10:04:12.417 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:04:12.417 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:04:12.417 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:04:12.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:12.418 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:04:12.418 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:04:12.418 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:04:12.418 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:12.418 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:04:12.418 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:04:12.419 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:04:12.419 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:12.419 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:04:12.419 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:04:12.419 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:04:12.419 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:04:12.429 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:04:12.430 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:04:12.430 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:04:12.431 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:04:12.438 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:04:12.438 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:04:12.439 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:04:12.439 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:04:12.474 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:04:12.475 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:12.475 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:04:12.475 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:12.483 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:04:12.483 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:12.483 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:04:12.483 DEBUG:teuthology.parallel:result is None 2024-04-25T10:04:12.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:04:12.538 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:04:12.542 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:04:12.555 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:04:12.556 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:04:12.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.591+0000 7f8c93140640 1 Processor -- start 2024-04-25T10:04:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.591+0000 7f8c93140640 1 -- start start 2024-04-25T10:04:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c93140640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c93140640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8c8c0d24f0 0x7f8c8c0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c93140640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c0ce6b0 con 0x7f8c8c15dc30 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c93140640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c0ce990 con 0x7f8c8c0d24f0 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c93140640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0cec70 con 0x7f8c8c15d850 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c916dd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8c8c0d24f0 0x7f8c8c0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34292/0 (socket says 172.21.15.82:34292) 2024-04-25T10:04:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 -- 172.21.15.82:0/862853036 learned_addr learned my addr 172.21.15.82:0/862853036 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 -- 172.21.15.82:0/862853036 >> v2:172.21.15.196:3300/0 conn(0x7f8c8c0d24f0 msgr2=0x7f8c8c0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 --2- 172.21.15.82:0/862853036 >> v2:172.21.15.196:3300/0 conn(0x7f8c8c0d24f0 0x7f8c8c0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 -- 172.21.15.82:0/862853036 >> v1:172.21.15.196:6789/0 conn(0x7f8c8c15dc30 legacy=0x7f8c8c0d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 -- 172.21.15.82:0/862853036 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c0cef50 con 0x7f8c8c15d850 2024-04-25T10:04:12.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.595+0000 7f8c90edc640 1 --2- 172.21.15.82:0/862853036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4039f9b33b8a0f38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8b7fe640 1 -- 172.21.15.82:0/862853036 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0aa5f0 con 0x7f8c8c15d850 2024-04-25T10:04:12.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8b7fe640 1 -- 172.21.15.82:0/862853036 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7c0af070 con 0x7f8c8c15d850 2024-04-25T10:04:12.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8b7fe640 1 -- 172.21.15.82:0/862853036 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0aa9b0 con 0x7f8c8c15d850 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/862853036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 msgr2=0x7f8c8c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- 172.21.15.82:0/862853036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009870 tx=0x7f8c7c01c560 comp rx=0 tx=0).stop 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/862853036 shutdown_connections 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- 172.21.15.82:0/862853036 >> v2:172.21.15.196:3300/0 conn(0x7f8c8c0d24f0 0x7f8c8c0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- 172.21.15.82:0/862853036 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c15d850 0x7f8c8c0d0160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/862853036 >> 172.21.15.82:0/862853036 conn(0x7f8c8c08b000 msgr2=0x7f8c8c0d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/862853036 shutdown_connections 2024-04-25T10:04:12.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/862853036 wait complete. 2024-04-25T10:04:12.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 Processor -- start 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- start start 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c0d24f0 0x7f8c8c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c8c15dc30 0x7f8c8c15ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c16a4c0 con 0x7f8c8c15d850 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c16a340 con 0x7f8c8c15dc30 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c16a640 con 0x7f8c8c0d24f0 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c90edc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c0d24f0 0x7f8c8c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40582/0 (socket says 172.21.15.82:40582) 2024-04-25T10:04:12.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 -- 172.21.15.82:0/1637524917 learned_addr learned my addr 172.21.15.82:0/1637524917 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c916dd640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c8c15dc30 0x7f8c8c15ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 -- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c0d24f0 msgr2=0x7f8c8c153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c0d24f0 0x7f8c8c153010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 -- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c8c15dc30 msgr2=0x7f8c8c15ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c8c15dc30 0x7f8c8c15ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 -- 172.21.15.82:0/1637524917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c154010 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c8bfff640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c450 tx=0x7f8c7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74016020 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c740052d0 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c1542f0 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74007260 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.599+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c1548b0 con 0x7f8c8c15d850 2024-04-25T10:04:12.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c04efa0 con 0x7f8c8c15d850 2024-04-25T10:04:12.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c74007400 con 0x7f8c8c15d850 2024-04-25T10:04:12.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c897fa640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 0x7f8c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:12.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8c740846a0 con 0x7f8c8c15d850 2024-04-25T10:04:12.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c90edc640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 0x7f8c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:12.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.603+0000 7f8c90edc640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 0x7f8c6c069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ae000 tx=0x7f8c7c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:12.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.607+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c74020020 con 0x7f8c8c15d850 2024-04-25T10:04:12.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:12.779+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c8c15a930 con 0x7f8c8c15d850 2024-04-25T10:04:13.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.547+0000 7f8c897fa640 1 -- 172.21.15.82:0/1637524917 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c74005480 con 0x7f8c8c15d850 2024-04-25T10:04:13.553 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:04:13.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.551+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 msgr2=0x7f8c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.551+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 0x7f8c6c069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c0ae000 tx=0x7f8c7c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:04:13.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.551+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 msgr2=0x7f8c8c1537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.551+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f8c7400c450 tx=0x7f8c7400c910 comp rx=0 tx=0).stop 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 shutdown_connections 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c6c066f60 0x7f8c6c069420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c8c15dc30 0x7f8c8c15ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c8c15d850 0x7f8c8c1537c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 --2- 172.21.15.82:0/1637524917 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c8c0d24f0 0x7f8c8c153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 >> 172.21.15.82:0/1637524917 conn(0x7f8c8c08b000 msgr2=0x7f8c8c15ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 shutdown_connections 2024-04-25T10:04:13.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.555+0000 7f8c93140640 1 -- 172.21.15.82:0/1637524917 wait complete. 2024-04-25T10:04:13.578 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:04:13.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.691+0000 7ff5249c2640 1 Processor -- start 2024-04-25T10:04:13.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.691+0000 7ff5249c2640 1 -- start start 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5200ce510 0x7ff5200ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 0x7ff52006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff52006ec60 0x7ff520142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cc2b0 con 0x7ff52006ec60 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff52006e290 con 0x7ff520144ca0 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cee30 con 0x7ff5200ce510 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff52006ec60 0x7ff520142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff52006ec60 0x7ff520142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40590/0 (socket says 172.21.15.82:40590) 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 -- 172.21.15.82:0/568260581 learned_addr learned my addr 172.21.15.82:0/568260581 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51dd9b640 1 --2- 172.21.15.82:0/568260581 >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 0x7ff52006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 -- 172.21.15.82:0/568260581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5200ce510 msgr2=0x7ff5200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 --2- 172.21.15.82:0/568260581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5200ce510 0x7ff5200ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 -- 172.21.15.82:0/568260581 >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 msgr2=0x7ff52006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 --2- 172.21.15.82:0/568260581 >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 0x7ff52006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 -- 172.21.15.82:0/568260581 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200ccc70 con 0x7ff52006ec60 2024-04-25T10:04:13.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51dd9b640 1 --2- 172.21.15.82:0/568260581 >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 0x7ff52006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51ed9d640 1 --2- 172.21.15.82:0/568260581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff52006ec60 0x7ff520142680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff52006cfe0 tx=0x7ff51001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab3d3e7dd6ebbf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51d59a640 1 -- 172.21.15.82:0/568260581 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100b5070 con 0x7ff52006ec60 2024-04-25T10:04:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51d59a640 1 -- 172.21.15.82:0/568260581 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5100a89d0 con 0x7ff52006ec60 2024-04-25T10:04:13.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff51d59a640 1 -- 172.21.15.82:0/568260581 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5100a8da0 con 0x7ff52006ec60 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- 172.21.15.82:0/568260581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff52006ec60 msgr2=0x7ff520142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- 172.21.15.82:0/568260581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff52006ec60 0x7ff520142680 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff52006cfe0 tx=0x7ff51001ed10 comp rx=0 tx=0).stop 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- 172.21.15.82:0/568260581 shutdown_connections 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- 172.21.15.82:0/568260581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff52006ec60 0x7ff520142680 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- 172.21.15.82:0/568260581 >> v2:172.21.15.196:3300/0 conn(0x7ff520144ca0 0x7ff52006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 --2- 172.21.15.82:0/568260581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5200ce510 0x7ff5200ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- 172.21.15.82:0/568260581 >> 172.21.15.82:0/568260581 conn(0x7ff5200b11a0 msgr2=0x7ff5200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- 172.21.15.82:0/568260581 shutdown_connections 2024-04-25T10:04:13.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.695+0000 7ff5249c2640 1 -- 172.21.15.82:0/568260581 wait complete. 2024-04-25T10:04:13.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 Processor -- start 2024-04-25T10:04:13.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- start start 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff52006ec60 0x7ff5201327b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff520144ca0 0x7ff5201f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200cc900 con 0x7ff5200ce510 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52006df20 con 0x7ff520144ca0 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201451f0 con 0x7ff52006ec60 2024-04-25T10:04:13.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40592/0 (socket says 172.21.15.82:40592) 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 -- 172.21.15.82:0/1377903800 learned_addr learned my addr 172.21.15.82:0/1377903800 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51ed9d640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff520144ca0 0x7ff5201f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51e59c640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff52006ec60 0x7ff5201327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 -- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff52006ec60 msgr2=0x7ff5201327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff52006ec60 0x7ff5201327b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 -- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff520144ca0 msgr2=0x7ff5201f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff520144ca0 0x7ff5201f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 -- 172.21.15.82:0/1377903800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff520141d60 con 0x7ff5200ce510 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff51dd9b640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff50800ce10 tx=0x7ff50800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800f890 con 0x7ff5200ce510 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50800fa30 con 0x7ff5200ce510 2024-04-25T10:04:13.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff520142020 con 0x7ff5200ce510 2024-04-25T10:04:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50800fe20 con 0x7ff5200ce510 2024-04-25T10:04:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5201425e0 con 0x7ff5200ce510 2024-04-25T10:04:13.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.699+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff52004efa0 con 0x7ff5200ce510 2024-04-25T10:04:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.703+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff508018400 con 0x7ff5200ce510 2024-04-25T10:04:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.703+0000 7ff516ffd640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 0x7ff500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.703+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff508085e10 con 0x7ff5200ce510 2024-04-25T10:04:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.703+0000 7ff51e59c640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 0x7ff500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:13.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.703+0000 7ff51e59c640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 0x7ff500069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:13.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.707+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff508014030 con 0x7ff5200ce510 2024-04-25T10:04:13.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff5200b1cc0 con 0x7ff5200ce510 2024-04-25T10:04:13.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff516ffd640 1 -- 172.21.15.82:0/1377903800 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff50800ae20 con 0x7ff5200ce510 2024-04-25T10:04:13.926 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.927 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.928 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:04:13.929 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.930 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:04:13.931 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:04:13.932 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:04:13.933 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.934 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:04:13.935 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.936 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:04:13.937 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:04:13.938 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.939 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:04:13.940 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:13.941 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:04:13.942 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 msgr2=0x7ff500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 0x7ff500069420 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff518006fd0 tx=0x7ff518008040 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 msgr2=0x7ff520141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff50800ce10 tx=0x7ff50800a650 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 shutdown_connections 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff500066f60 0x7ff500069420 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff520144ca0 0x7ff5201f3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5200ce510 0x7ff520141820 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 --2- 172.21.15.82:0/1377903800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff52006ec60 0x7ff5201327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.923+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 >> 172.21.15.82:0/1377903800 conn(0x7ff5200b11a0 msgr2=0x7ff5200b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.927+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 shutdown_connections 2024-04-25T10:04:13.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:13.927+0000 7ff5249c2640 1 -- 172.21.15.82:0/1377903800 wait complete. 2024-04-25T10:04:13.946 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:14.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.051+0000 7f491d147640 1 Processor -- start 2024-04-25T10:04:14.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.051+0000 7f491d147640 1 -- start start 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491d147640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 0x7f49180b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491d147640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180cbec0 con 0x7f491809a7b0 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491d147640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49180cc1a0 con 0x7f491809a180 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491d147640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49180cc480 con 0x7f49180b2c30 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f491809a7b0 0x7f4918099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49074/0 (socket says 172.21.15.82:49074) 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491659c640 1 -- 172.21.15.82:0/3064303586 learned_addr learned my addr 172.21.15.82:0/3064303586 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 0x7f49180b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260624441 0 0) 0x7f49180cbec0 con 0x7f491809a7b0 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4900003530 con 0x7f491809a7b0 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3354306679 0 0) 0x7f4900003530 con 0x7f491809a7b0 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 msgr2=0x7f49180b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 0x7f49180b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 >> v1:172.21.15.196:6789/0 conn(0x7f491809a180 legacy=0x7f49180ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180cc760 con 0x7f491809a7b0 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4908001680 con 0x7f491809a7b0 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 >> v1:172.21.15.82:6789/0 conn(0x7f491809a7b0 legacy=0x7f4918099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4900003a60 0x7f4900003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4900004810 0x7f4900004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 0x7f49000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49180cc480 con 0x7f4900003a60 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490000bad0 con 0x7f4900004810 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490000bcd0 con 0x7f49180b2c30 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4900003a60 0x7f4900003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 -- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 msgr2=0x7f49000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49180b2c30 0x7f49000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 -- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4900004810 msgr2=0x7f4900004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f491659c640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4900004810 0x7f4900004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4900004810 0x7f4900004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.055+0000 7f4916d9d640 1 -- 172.21.15.82:0/3064303586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900003530 con 0x7f4900003a60 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f4916d9d640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4900003a60 0x7f4900003e80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f490c009ae0 tx=0x7f490c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3e221e896c1aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0096b0 con 0x7f4900003a60 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f490c0a6460 con 0x7f4900003a60 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f4915d9b640 1 -- 172.21.15.82:0/3064303586 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f490c0a6850 con 0x7f4900003a60 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- 172.21.15.82:0/3064303586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4900003a60 msgr2=0x7f4900003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 --2- 172.21.15.82:0/3064303586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4900003a60 0x7f4900003e80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f490c009ae0 tx=0x7f490c01ca80 comp rx=0 tx=0).stop 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491759e640 1 -- 172.21.15.82:0/3064303586 reap_dead start 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- 172.21.15.82:0/3064303586 shutdown_connections 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- 172.21.15.82:0/3064303586 >> 172.21.15.82:0/3064303586 conn(0x7f49180b3010 msgr2=0x7f49180b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- 172.21.15.82:0/3064303586 shutdown_connections 2024-04-25T10:04:14.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- 172.21.15.82:0/3064303586 wait complete. 2024-04-25T10:04:14.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 Processor -- start 2024-04-25T10:04:14.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- start start 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4900003a60 0x7f49180c79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49181f7120 0x7f49181f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491806c380 con 0x7f49181fa240 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491806c200 con 0x7f49181f7120 2024-04-25T10:04:14.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491d147640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f491815d190 con 0x7f4900003a60 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491759e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40596/0 (socket says 172.21.15.82:40596) 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.059+0000 7f491759e640 1 -- 172.21.15.82:0/292293449 learned_addr learned my addr 172.21.15.82:0/292293449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491659c640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49181f7120 0x7f49181f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f4916d9d640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4900003a60 0x7f49180c79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 -- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4900003a60 msgr2=0x7f49180c79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4900003a60 0x7f49180c79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 -- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49181f7120 msgr2=0x7f49181f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49181f7120 0x7f49181f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 -- 172.21.15.82:0/292293449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49181fa730 con 0x7f49181fa240 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491759e640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4918071fe0 tx=0x7f491000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910017070 con 0x7f49181fa240 2024-04-25T10:04:14.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910005d50 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49181faa10 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491000a850 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49181f6c00 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49181f6710 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f491001a070 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 0x7f4904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f49100840e0 con 0x7f49181fa240 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f4916d9d640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 0x7f4904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.063+0000 7f4916d9d640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 0x7f4904069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ad000 tx=0x7f490c0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.067+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4910055340 con 0x7f49181fa240 2024-04-25T10:04:14.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f491804efa0 con 0x7f49181fa240 2024-04-25T10:04:14.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f48ff7fe640 1 -- 172.21.15.82:0/292293449 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13176 (secure 0 0 0) 0x7f4910054c40 con 0x7f49181fa240 2024-04-25T10:04:14.218 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:14.219 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":48,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:13.564412+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 msgr2=0x7f4904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 0x7f4904069420 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f490c0ad000 tx=0x7f490c0b2040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 msgr2=0x7f4918070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4918071fe0 tx=0x7f491000cb90 comp rx=0 tx=0).stop 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 shutdown_connections 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4904066f60 0x7f4904069420 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49181fa240 0x7f4918070490 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49181f7120 0x7f49181f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 --2- 172.21.15.82:0/292293449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4900003a60 0x7f49180c79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 >> 172.21.15.82:0/292293449 conn(0x7f49180b3010 msgr2=0x7f4918073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 shutdown_connections 2024-04-25T10:04:14.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.215+0000 7f491d147640 1 -- 172.21.15.82:0/292293449 wait complete. 2024-04-25T10:04:14.235 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:04:14.236 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:04:14.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 Processor -- start 2024-04-25T10:04:14.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 -- start start 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c7c0cf590 0x7f5c7c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c0ca4c0 con 0x7f5c7c0c9ef0 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c0ca780 con 0x7f5c7c0cf970 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c8185c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7c0cd440 con 0x7f5c7c0cf590 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40610/0 (socket says 172.21.15.82:40610) 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 -- 172.21.15.82:0/3252110042 learned_addr learned my addr 172.21.15.82:0/3252110042 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c7affd640 1 --2- 172.21.15.82:0/3252110042 >> v2:172.21.15.196:3300/0 conn(0x7f5c7c0cf590 0x7f5c7c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 -- 172.21.15.82:0/3252110042 >> v2:172.21.15.196:3300/0 conn(0x7f5c7c0cf590 msgr2=0x7f5c7c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 --2- 172.21.15.82:0/3252110042 >> v2:172.21.15.196:3300/0 conn(0x7f5c7c0cf590 0x7f5c7c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 -- 172.21.15.82:0/3252110042 >> v1:172.21.15.196:6789/0 conn(0x7f5c7c0c9ef0 legacy=0x7f5c7c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.355+0000 7f5c73fff640 1 -- 172.21.15.82:0/3252110042 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0cd690 con 0x7f5c7c0cf970 2024-04-25T10:04:14.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c73fff640 1 --2- 172.21.15.82:0/3252110042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0d0430 tx=0x7f5c6401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3f936890d0bfa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7a7fc640 1 -- 172.21.15.82:0/3252110042 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b0070 con 0x7f5c7c0cf970 2024-04-25T10:04:14.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7a7fc640 1 -- 172.21.15.82:0/3252110042 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640ab600 con 0x7f5c7c0cf970 2024-04-25T10:04:14.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7a7fc640 1 -- 172.21.15.82:0/3252110042 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640ab9d0 con 0x7f5c7c0cf970 2024-04-25T10:04:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/3252110042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 msgr2=0x7f5c7c0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- 172.21.15.82:0/3252110042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0d0430 tx=0x7f5c6401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/3252110042 shutdown_connections 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- 172.21.15.82:0/3252110042 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c0c99b0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- 172.21.15.82:0/3252110042 >> v2:172.21.15.196:3300/0 conn(0x7f5c7c0cf590 0x7f5c7c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/3252110042 >> 172.21.15.82:0/3252110042 conn(0x7f5c7c06ae60 msgr2=0x7f5c7c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/3252110042 shutdown_connections 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/3252110042 wait complete. 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 Processor -- start 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- start start 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 0x7f5c7c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 0x7f5c7c15c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c167660 con 0x7f5c7c0cf970 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1674e0 con 0x7f5c7c0cf590 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c1677e0 con 0x7f5c7c0c9ef0 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:40616/0 (socket says 172.21.15.82:40616) 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 -- 172.21.15.82:0/1691045664 learned_addr learned my addr 172.21.15.82:0/1691045664 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c73fff640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 0x7f5c7c15c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7affd640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 0x7f5c7c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 -- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 msgr2=0x7f5c7c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 0x7f5c7c159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 -- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 msgr2=0x7f5c7c15c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 0x7f5c7c15c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 -- 172.21.15.82:0/1691045664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c0b0c50 con 0x7f5c7c0cf970 2024-04-25T10:04:14.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7affd640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 0x7f5c7c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c73fff640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 0x7f5c7c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c7b7fe640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400b9a0 tx=0x7f5c7400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74019070 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400cba0 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c158d30 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c74014440 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.359+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1592f0 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c04efa0 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c7401c070 con 0x7f5c7c0cf970 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c78ff9640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 0x7f5c5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c7affd640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 0x7f5c5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5c74084a40 con 0x7f5c7c0cf970 2024-04-25T10:04:14.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.363+0000 7f5c7affd640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 0x7f5c5c069450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c15bf00 tx=0x7f5c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.367+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c74055c10 con 0x7f5c7c0cf970 2024-04-25T10:04:14.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.511+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c7c0c2730 con 0x7f5c7c0cf970 2024-04-25T10:04:14.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c78ff9640 1 -- 172.21.15.82:0/1691045664 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c74055510 con 0x7f5c7c0cf970 2024-04-25T10:04:14.517 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:04:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 msgr2=0x7f5c5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 0x7f5c5c069450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c15bf00 tx=0x7f5c6c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 msgr2=0x7f5c7c15c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400b9a0 tx=0x7f5c7400be60 comp rx=0 tx=0).stop 2024-04-25T10:04:14.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 shutdown_connections 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c5c066f90 0x7f5c5c069450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c7c0cf970 0x7f5c7c15c800 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c0cf590 0x7f5c7c15c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 --2- 172.21.15.82:0/1691045664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c0c9ef0 0x7f5c7c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 >> 172.21.15.82:0/1691045664 conn(0x7f5c7c06ae60 msgr2=0x7f5c7c0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 shutdown_connections 2024-04-25T10:04:14.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.515+0000 7f5c8185c640 1 -- 172.21.15.82:0/1691045664 wait complete. 2024-04-25T10:04:14.535 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:04:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 Processor -- start 2024-04-25T10:04:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 -- start start 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f989015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98900b3f50 0x7f98900cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98900b2090 con 0x7f98900ce430 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989015f200 con 0x7f98900b3f50 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9898379640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b43e0 con 0x7f989015a700 2024-04-25T10:04:14.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.635+0000 7f9896916640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f98900ce430 0x7f98900b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49086/0 (socket says 172.21.15.82:49086) 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 -- 172.21.15.82:0/1546431920 learned_addr learned my addr 172.21.15.82:0/1546431920 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896115640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f989015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895914640 1 --2- 172.21.15.82:0/1546431920 >> v2:172.21.15.196:3300/0 conn(0x7f98900b3f50 0x7f98900cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107474826 0 0) 0x7f98900b2090 con 0x7f98900ce430 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9878005740 con 0x7f98900ce430 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140045694 0 0) 0x7f9878005740 con 0x7f98900ce430 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 msgr2=0x7f989015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f989015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 >> v2:172.21.15.196:3300/0 conn(0x7f98900b3f50 msgr2=0x7f98900cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 --2- 172.21.15.82:0/1546431920 >> v2:172.21.15.196:3300/0 conn(0x7f98900b3f50 0x7f98900cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900b46c0 con 0x7f98900ce430 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9880001a00 con 0x7f98900ce430 2024-04-25T10:04:14.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 >> v1:172.21.15.82:6789/0 conn(0x7f98900ce430 legacy=0x7f98900b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f9878005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98780061c0 0x7f9878006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98780073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f989015f200 con 0x7f9878006f60 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b43e0 con 0x7f98780061c0 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987800ddf0 con 0x7f989015a700 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98780073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896115640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f9878005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895914640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98780061c0 0x7f9878006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 -- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 msgr2=0x7f9878005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f989015a700 0x7f9878005c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 -- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98780061c0 msgr2=0x7f9878006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98780061c0 0x7f9878006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 -- 172.21.15.82:0/1546431920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9878005740 con 0x7f9878006f60 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98780073e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f988000a920 tx=0x7f988001d410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d89ac949d1d9907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988000a4c0 con 0x7f9878006f60 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98800a5e50 con 0x7f9878006f60 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9895113640 1 -- 172.21.15.82:0/1546431920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98800ad890 con 0x7f9878006f60 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 -- 172.21.15.82:0/1546431920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 msgr2=0x7f98780073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 --2- 172.21.15.82:0/1546431920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98780073e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f988000a920 tx=0x7f988001d410 comp rx=0 tx=0).stop 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9896916640 1 -- 172.21.15.82:0/1546431920 reap_dead start 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 -- 172.21.15.82:0/1546431920 shutdown_connections 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 -- 172.21.15.82:0/1546431920 >> 172.21.15.82:0/1546431920 conn(0x7f98900b11a0 msgr2=0x7f98900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 -- 172.21.15.82:0/1546431920 shutdown_connections 2024-04-25T10:04:14.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.639+0000 7f9898379640 1 -- 172.21.15.82:0/1546431920 wait complete. 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 Processor -- start 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- start start 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9890155310 0x7f989015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f989015b2c0 0x7f989015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900cd490 con 0x7f9878006f60 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98900cd310 con 0x7f989015b2c0 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98900b2bb0 con 0x7f9890155310 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55330/0 (socket says 172.21.15.82:55330) 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 -- 172.21.15.82:0/4026631910 learned_addr learned my addr 172.21.15.82:0/4026631910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9895914640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9890155310 0x7f989015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896916640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f989015b2c0 0x7f989015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 -- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9890155310 msgr2=0x7f989015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9890155310 0x7f989015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 -- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f989015b2c0 msgr2=0x7f989015c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f989015b2c0 0x7f989015c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 -- 172.21.15.82:0/4026631910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98901fa6d0 con 0x7f9878006f60 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9896115640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f98880108c0 tx=0x7f9888010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800f040 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988800eb00 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988801c400 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98900ad8a0 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98900ac980 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f989004efa0 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.643+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f988801c5a0 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.647+0000 7f988effd640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 0x7f987c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.647+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9888085420 con 0x7f9878006f60 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.647+0000 7f9895914640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 0x7f987c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.647+0000 7f9895914640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 0x7f987c069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f98900b0cc0 tx=0x7f9884008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.651+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f98901fa6d0 con 0x7f9878006f60 2024-04-25T10:04:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.795+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f989015b6a0 con 0x7f9878006f60 2024-04-25T10:04:14.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.795+0000 7f988effd640 1 -- 172.21.15.82:0/4026631910 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v48) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9888018030 con 0x7f9878006f60 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:14.801 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:04:14.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 msgr2=0x7f987c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 0x7f987c069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f98900b0cc0 tx=0x7f9884008040 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 msgr2=0x7f98901fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f98880108c0 tx=0x7f9888010d80 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 shutdown_connections 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f987c066f60 0x7f987c069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9890155310 0x7f989015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f989015b2c0 0x7f989015c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 --2- 172.21.15.82:0/4026631910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9878006f60 0x7f98901fa190 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 >> 172.21.15.82:0/4026631910 conn(0x7f98900b11a0 msgr2=0x7f9890159010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 shutdown_connections 2024-04-25T10:04:14.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.799+0000 7f9898379640 1 -- 172.21.15.82:0/4026631910 wait complete. 2024-04-25T10:04:14.818 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:04:14.819 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:04:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.943+0000 7f328b3c2640 1 Processor -- start 2024-04-25T10:04:14.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.943+0000 7f328b3c2640 1 -- start start 2024-04-25T10:04:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328b3c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 0x7f328406d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328b3c2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f32840b36d0 0x7f32840b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328b3c2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840b3bd0 con 0x7f32840cd000 2024-04-25T10:04:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328b3c2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32840b3eb0 con 0x7f32840b36d0 2024-04-25T10:04:14.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328b3c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32840b4190 con 0x7f3284159f20 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f32840b36d0 0x7f32840b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f32840b36d0 0x7f32840b2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55338/0 (socket says 172.21.15.82:55338) 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 -- 172.21.15.82:0/2175905671 learned_addr learned my addr 172.21.15.82:0/2175905671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328915e640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 0x7f328406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 -- 172.21.15.82:0/2175905671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 msgr2=0x7f328406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 0x7f328406d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 -- 172.21.15.82:0/2175905671 >> v1:172.21.15.196:6789/0 conn(0x7f32840cd000 legacy=0x7f32840cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 -- 172.21.15.82:0/2175905671 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f328406cf80 con 0x7f32840b36d0 2024-04-25T10:04:14.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328915e640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 0x7f328406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:14.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.947+0000 7f328895d640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32840b36d0 0x7f32840b2190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f327400bb10 tx=0x7f327401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33306ff5954bb545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f327bfff640 1 -- 172.21.15.82:0/2175905671 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32740b4070 con 0x7f32840b36d0 2024-04-25T10:04:14.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f327bfff640 1 -- 172.21.15.82:0/2175905671 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32740a7c70 con 0x7f32840b36d0 2024-04-25T10:04:14.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f327bfff640 1 -- 172.21.15.82:0/2175905671 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32740af660 con 0x7f32840b36d0 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- 172.21.15.82:0/2175905671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32840b36d0 msgr2=0x7f32840b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32840b36d0 0x7f32840b2190 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f327400bb10 tx=0x7f327401ec00 comp rx=0 tx=0).stop 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- 172.21.15.82:0/2175905671 shutdown_connections 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f32840b36d0 0x7f32840b2190 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- 172.21.15.82:0/2175905671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3284159f20 0x7f328406d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- 172.21.15.82:0/2175905671 >> 172.21.15.82:0/2175905671 conn(0x7f328406ae60 msgr2=0x7f328406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- 172.21.15.82:0/2175905671 shutdown_connections 2024-04-25T10:04:14.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- 172.21.15.82:0/2175905671 wait complete. 2024-04-25T10:04:14.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 Processor -- start 2024-04-25T10:04:14.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- start start 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 0x7f32840c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32840cd000 0x7f32840cadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328406dcc0 con 0x7f3284159f20 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32840cc4d0 con 0x7f32840b36d0 2024-04-25T10:04:14.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328b3c2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328406de40 con 0x7f32840cd000 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55354/0 (socket says 172.21.15.82:55354) 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 -- 172.21.15.82:0/575257484 learned_addr learned my addr 172.21.15.82:0/575257484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328895d640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32840cd000 0x7f32840cadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328915e640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 0x7f32840c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 -- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32840cd000 msgr2=0x7f32840cadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32840cd000 0x7f32840cadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 -- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 msgr2=0x7f32840c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 0x7f32840c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 -- 172.21.15.82:0/575257484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284073de0 con 0x7f3284159f20 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328915e640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 0x7f32840c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f328995f640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c3c0 tx=0x7f327c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.951+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c0052d0 con 0x7f3284159f20 2024-04-25T10:04:14.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327c005470 con 0x7f3284159f20 2024-04-25T10:04:14.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f327c007260 con 0x7f3284159f20 2024-04-25T10:04:14.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32840c4d90 con 0x7f3284159f20 2024-04-25T10:04:14.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32840c5030 con 0x7f3284159f20 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284067c30 con 0x7f3284159f20 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f327c007400 con 0x7f3284159f20 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f3279ffb640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 0x7f3268069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f327c084200 con 0x7f3284159f20 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f328915e640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 0x7f3268069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:14.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.955+0000 7f328915e640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 0x7f3268069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3280009540 tx=0x7f328000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:14.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:14.959+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f327c054d80 con 0x7f3284159f20 2024-04-25T10:04:15.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.111+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f32840ac5a0 con 0x7f3284159f20 2024-04-25T10:04:15.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.555+0000 7f3279ffb640 1 -- 172.21.15.82:0/575257484 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v49) v1 ==== 138+0+0 (secure 0 0 0) 0x7f327c058e20 con 0x7f3284159f20 2024-04-25T10:04:15.558 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:04:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 msgr2=0x7f3268069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 0x7f3268069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3280009540 tx=0x7f328000b040 comp rx=0 tx=0).stop 2024-04-25T10:04:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 msgr2=0x7f3284158e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f327c00c3c0 tx=0x7f327c00c880 comp rx=0 tx=0).stop 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 shutdown_connections 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3268066f90 0x7f3268069450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32840cd000 0x7f32840cadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3284159f20 0x7f3284158e40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 --2- 172.21.15.82:0/575257484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f32840b36d0 0x7f32840c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 >> 172.21.15.82:0/575257484 conn(0x7f328406ae60 msgr2=0x7f3284070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 shutdown_connections 2024-04-25T10:04:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.559+0000 7f328b3c2640 1 -- 172.21.15.82:0/575257484 wait complete. 2024-04-25T10:04:15.585 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:04:15.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 Processor -- start 2024-04-25T10:04:15.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 -- start start 2024-04-25T10:04:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c06c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c0d1950 con 0x7f5c6c0cf270 2024-04-25T10:04:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0d2520 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c7381e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c0d1bf0 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c715ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c71dbb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c06c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c715ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:33922/0 (socket says 172.21.15.82:33922) 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c715ba640 1 -- 172.21.15.82:0/3884442788 learned_addr learned my addr 172.21.15.82:0/3884442788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c70db9640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f5c6c0cf270 0x7f5c6c0d36e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:44750/0 (socket says 172.21.15.82:44750) 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c71dbb640 1 -- 172.21.15.82:0/3884442788 >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 msgr2=0x7f5c6c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:15.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c71dbb640 1 --2- 172.21.15.82:0/3884442788 >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c71dbb640 1 -- 172.21.15.82:0/3884442788 >> v1:172.21.15.196:6789/0 conn(0x7f5c6c0cf270 legacy=0x7f5c6c0d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c71dbb640 1 -- 172.21.15.82:0/3884442788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0d1ed0 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.711+0000 7f5c715ba640 1 --2- 172.21.15.82:0/3884442788 >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:04:15.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c71dbb640 1 --2- 172.21.15.82:0/3884442788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c06c590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b573bd242d41215e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c63fff640 1 -- 172.21.15.82:0/3884442788 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b2070 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c63fff640 1 -- 172.21.15.82:0/3884442788 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640a59a0 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c63fff640 1 -- 172.21.15.82:0/3884442788 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640a5d70 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 -- 172.21.15.82:0/3884442788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 msgr2=0x7f5c6c06c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 --2- 172.21.15.82:0/3884442788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c06c590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5c64009d90 tx=0x7f5c6401c940 comp rx=0 tx=0).stop 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 -- 172.21.15.82:0/3884442788 shutdown_connections 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 --2- 172.21.15.82:0/3884442788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c06c590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 --2- 172.21.15.82:0/3884442788 >> v2:172.21.15.196:3300/0 conn(0x7f5c6c15d9c0 0x7f5c6c0d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 -- 172.21.15.82:0/3884442788 >> 172.21.15.82:0/3884442788 conn(0x7f5c6c09cad0 msgr2=0x7f5c6c08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 -- 172.21.15.82:0/3884442788 shutdown_connections 2024-04-25T10:04:15.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.715+0000 7f5c7381e640 1 -- 172.21.15.82:0/3884442788 wait complete. 2024-04-25T10:04:15.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 Processor -- start 2024-04-25T10:04:15.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- start start 2024-04-25T10:04:15.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c6c0cf270 0x7f5c6c154b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c06d460 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c06d2e0 con 0x7f5c6c0cf270 2024-04-25T10:04:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c06d5e0 con 0x7f5c6c0d3df0 2024-04-25T10:04:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55370/0 (socket says 172.21.15.82:55370) 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 -- 172.21.15.82:0/183987582 learned_addr learned my addr 172.21.15.82:0/183987582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c715ba640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c6c0cf270 0x7f5c6c154b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c70db9640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 -- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 msgr2=0x7f5c6c152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 -- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c6c0cf270 msgr2=0x7f5c6c154b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c6c0cf270 0x7f5c6c154b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 -- 172.21.15.82:0/183987582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c1555d0 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c71dbb640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5c640b0040 tx=0x7f5c640196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b2050 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c640be070 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c155890 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640ada90 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c155e50 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c04efa0 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c640adc30 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.719+0000 7f5c61ffb640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.723+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f5c64120230 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.723+0000 7f5c715ba640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:15.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.723+0000 7f5c715ba640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 0x7f5c50069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009a40 tx=0x7f5c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:15.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.727+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c64016560 con 0x7f5c6c15d9c0 2024-04-25T10:04:15.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:15.883+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5c6c0cb560 con 0x7f5c6c15d9c0 2024-04-25T10:04:16.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.559+0000 7f5c61ffb640 1 -- 172.21.15.82:0/183987582 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v50) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5c640ab340 con 0x7f5c6c15d9c0 2024-04-25T10:04:16.562 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 msgr2=0x7f5c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 0x7f5c50069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009a40 tx=0x7f5c68008040 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 msgr2=0x7f5c6c157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5c640b0040 tx=0x7f5c640196e0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 shutdown_connections 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c50066f60 0x7f5c50069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c6c0d3df0 0x7f5c6c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c6c15d9c0 0x7f5c6c157170 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 --2- 172.21.15.82:0/183987582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c6c0cf270 0x7f5c6c154b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 >> 172.21.15.82:0/183987582 conn(0x7f5c6c09cad0 msgr2=0x7f5c6c0ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 shutdown_connections 2024-04-25T10:04:16.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.563+0000 7f5c7381e640 1 -- 172.21.15.82:0/183987582 wait complete. 2024-04-25T10:04:16.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:04:16.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 Processor -- start 2024-04-25T10:04:16.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 -- start start 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d4815b420 0x7f9d480ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480cbd20 con 0x7f9d480cfad0 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480cff20 con 0x7f9d4815b420 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4f0b4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480cc930 con 0x7f9d4815b800 2024-04-25T10:04:16.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4ce50640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9d4815b420 0x7f9d480ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4d651640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9d480cfad0 0x7f9d4815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58316/0 (socket says 172.21.15.82:58316) 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4d651640 1 -- 172.21.15.82:0/3038693208 learned_addr learned my addr 172.21.15.82:0/3038693208 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d477fe640 1 -- 172.21.15.82:0/3038693208 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007308355 0 0) 0x7f9d480cbd20 con 0x7f9d480cfad0 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d477fe640 1 -- 172.21.15.82:0/3038693208 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d30003530 con 0x7f9d480cfad0 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d47fff640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4ce50640 1 -- 172.21.15.82:0/3038693208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 msgr2=0x7f9d480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4ce50640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4ce50640 1 -- 172.21.15.82:0/3038693208 >> v1:172.21.15.82:6789/0 conn(0x7f9d480cfad0 legacy=0x7f9d4815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d4ce50640 1 -- 172.21.15.82:0/3038693208 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480cc000 con 0x7f9d4815b420 2024-04-25T10:04:16.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.719+0000 7f9d47fff640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:16.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d4815b420 0x7f9d480ca3c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d380097c0 tx=0x7f9d3801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb15aaecaad1bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d477fe640 1 -- 172.21.15.82:0/3038693208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380a7070 con 0x7f9d4815b420 2024-04-25T10:04:16.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d477fe640 1 -- 172.21.15.82:0/3038693208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d380b0070 con 0x7f9d4815b420 2024-04-25T10:04:16.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d477fe640 1 -- 172.21.15.82:0/3038693208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380ab890 con 0x7f9d4815b420 2024-04-25T10:04:16.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/3038693208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d4815b420 msgr2=0x7f9d480ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d4815b420 0x7f9d480ca3c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9d380097c0 tx=0x7f9d3801c470 comp rx=0 tx=0).stop 2024-04-25T10:04:16.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/3038693208 shutdown_connections 2024-04-25T10:04:16.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/3038693208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d4815b420 0x7f9d480ca3c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/3038693208 >> 172.21.15.82:0/3038693208 conn(0x7f9d4806ae60 msgr2=0x7f9d4806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:16.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/3038693208 shutdown_connections 2024-04-25T10:04:16.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/3038693208 wait complete. 2024-04-25T10:04:16.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 Processor -- start 2024-04-25T10:04:16.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- start start 2024-04-25T10:04:16.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4815b420 0x7f9d481581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480caa80 con 0x7f9d480cfad0 2024-04-25T10:04:16.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480ca900 con 0x7f9d4815b420 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4f0b4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480cadb0 con 0x7f9d4815b800 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55398/0 (socket says 172.21.15.82:55398) 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 -- 172.21.15.82:0/463605548 learned_addr learned my addr 172.21.15.82:0/463605548 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4d651640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d47fff640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4815b420 0x7f9d481581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 -- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 msgr2=0x7f9d480989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 -- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4815b420 msgr2=0x7f9d481581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4815b420 0x7f9d481581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.723+0000 7f9d4ce50640 1 -- 172.21.15.82:0/463605548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d48098ef0 con 0x7f9d480cfad0 2024-04-25T10:04:16.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d4ce50640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801a0a0 tx=0x7f9d3801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380bf610 con 0x7f9d480cfad0 2024-04-25T10:04:16.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d380bf7d0 con 0x7f9d480cfad0 2024-04-25T10:04:16.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380bfbc0 con 0x7f9d480cfad0 2024-04-25T10:04:16.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480c31f0 con 0x7f9d480cfad0 2024-04-25T10:04:16.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d480c37b0 con 0x7f9d480cfad0 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d4804efa0 con 0x7f9d480cfad0 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9d380bd020 con 0x7f9d480cfad0 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d47fff640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9d3811fff0 con 0x7f9d480cfad0 2024-04-25T10:04:16.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.727+0000 7f9d47fff640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d480c5bd0 tx=0x7f9d3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:16.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.731+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9d380c6070 con 0x7f9d480cfad0 2024-04-25T10:04:16.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:16.911+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9d480cf7b0 con 0x7f9d480cfad0 2024-04-25T10:04:17.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.579+0000 7f9d457fa640 1 -- 172.21.15.82:0/463605548 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9d38016090 con 0x7f9d480cfad0 2024-04-25T10:04:17.583 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 msgr2=0x7f9d34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 0x7f9d34069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d480c5bd0 tx=0x7f9d3c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 msgr2=0x7f9d480c2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801a0a0 tx=0x7f9d3801acb0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 shutdown_connections 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9d34066f60 0x7f9d34069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9d4815b800 0x7f9d480989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9d4815b420 0x7f9d481581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 --2- 172.21.15.82:0/463605548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9d480cfad0 0x7f9d480c2c10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 >> 172.21.15.82:0/463605548 conn(0x7f9d4806ae60 msgr2=0x7f9d4806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 shutdown_connections 2024-04-25T10:04:17.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.583+0000 7f9d4f0b4640 1 -- 172.21.15.82:0/463605548 wait complete. 2024-04-25T10:04:17.619 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 Processor -- start 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 -- start start 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 0x7fb30006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb30006ec60 0x7fb300142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3000ccc70 con 0x7fb3000ce510 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3000cc2b0 con 0x7fb30006ec60 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb30508d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30006e290 con 0x7fb300144ca0 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2fed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb3000ce510 0x7fb3000ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58330/0 (socket says 172.21.15.82:58330) 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2fed9d640 1 -- 172.21.15.82:0/2207985704 learned_addr learned my addr 172.21.15.82:0/2207985704 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/2207985704 >> v2:172.21.15.82:3300/0 conn(0x7fb30006ec60 0x7fb300142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2fe59c640 1 -- 172.21.15.82:0/2207985704 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138543754 0 0) 0x7fb3000ccc70 con 0x7fb3000ce510 2024-04-25T10:04:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2fe59c640 1 -- 172.21.15.82:0/2207985704 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2ec003530 con 0x7fb3000ce510 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2f659c640 1 --2- 172.21.15.82:0/2207985704 >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 0x7fb30006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2ff59e640 1 -- 172.21.15.82:0/2207985704 >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 msgr2=0x7fb30006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/2207985704 >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 0x7fb30006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2ff59e640 1 -- 172.21.15.82:0/2207985704 >> v1:172.21.15.82:6789/0 conn(0x7fb3000ce510 legacy=0x7fb3000ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2ff59e640 1 -- 172.21.15.82:0/2207985704 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30006dc40 con 0x7fb30006ec60 2024-04-25T10:04:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.735+0000 7fb2f659c640 1 --2- 172.21.15.82:0/2207985704 >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 0x7fb30006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/2207985704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb30006ec60 0x7fb300142680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb300070fe0 tx=0x7fb2f801f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f255ff1010bd09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2fe59c640 1 -- 172.21.15.82:0/2207985704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80b4070 con 0x7fb30006ec60 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2fe59c640 1 -- 172.21.15.82:0/2207985704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f80af600 con 0x7fb30006ec60 2024-04-25T10:04:17.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2fe59c640 1 -- 172.21.15.82:0/2207985704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80af9d0 con 0x7fb30006ec60 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/2207985704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb30006ec60 msgr2=0x7fb300142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- 172.21.15.82:0/2207985704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb30006ec60 0x7fb300142680 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb300070fe0 tx=0x7fb2f801f110 comp rx=0 tx=0).stop 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/2207985704 shutdown_connections 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- 172.21.15.82:0/2207985704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb30006ec60 0x7fb300142680 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- 172.21.15.82:0/2207985704 >> v2:172.21.15.196:3300/0 conn(0x7fb300144ca0 0x7fb30006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/2207985704 >> 172.21.15.82:0/2207985704 conn(0x7fb3000b11a0 msgr2=0x7fb3000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/2207985704 shutdown_connections 2024-04-25T10:04:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/2207985704 wait complete. 2024-04-25T10:04:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 Processor -- start 2024-04-25T10:04:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- start start 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 0x7fb300143b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 0x7fb3001e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001451f0 con 0x7fb300144ca0 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3000cc900 con 0x7fb30006ec60 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb300145370 con 0x7fb3000ce510 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55428/0 (socket says 172.21.15.82:55428) 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 -- 172.21.15.82:0/3687001893 learned_addr learned my addr 172.21.15.82:0/3687001893 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2f659c640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 0x7fb3001e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2fed9d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 0x7fb300143b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 -- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 msgr2=0x7fb3001e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 0x7fb3001e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 -- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 msgr2=0x7fb300143b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 0x7fb300143b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 -- 172.21.15.82:0/3687001893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3001e1dd0 con 0x7fb300144ca0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2f659c640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 0x7fb3001e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2fed9d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 0x7fb300143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2ff59e640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2f801bd20 tx=0x7fb2f801beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f80b4040 con 0x7fb300144ca0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f80c1070 con 0x7fb300144ca0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f8009450 con 0x7fb300144ca0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3001e2070 con 0x7fb300144ca0 2024-04-25T10:04:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.739+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3001e63f0 con 0x7fb300144ca0 2024-04-25T10:04:17.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30004efa0 con 0x7fb300144ca0 2024-04-25T10:04:17.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2f80ba6b0 con 0x7fb300144ca0 2024-04-25T10:04:17.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb2f7fff640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 0x7fb2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:17.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2f8123ae0 con 0x7fb300144ca0 2024-04-25T10:04:17.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb2fed9d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 0x7fb2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:17.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.743+0000 7fb2fed9d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 0x7fb2e0069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0005890 tx=0x7fb2f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:17.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.747+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2f8018c70 con 0x7fb300144ca0 2024-04-25T10:04:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.907+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb30015b9c0 con 0x7fb300144ca0 2024-04-25T10:04:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.907+0000 7fb2f7fff640 1 -- 172.21.15.82:0/3687001893 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16967 (secure 0 0 0) 0x7fb2f80f49f0 con 0x7fb300144ca0 2024-04-25T10:04:17.912 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:17.912 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":51,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:16.914756+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:17.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 msgr2=0x7fb2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 0x7fb2e0069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb2f0005890 tx=0x7fb2f0009040 comp rx=0 tx=0).stop 2024-04-25T10:04:17.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 msgr2=0x7fb3001f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:17.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb2f801bd20 tx=0x7fb2f801beb0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 shutdown_connections 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2e0066f60 0x7fb2e0069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3000ce510 0x7fb3001e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb300144ca0 0x7fb3001f14f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 --2- 172.21.15.82:0/3687001893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb30006ec60 0x7fb300143b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 >> 172.21.15.82:0/3687001893 conn(0x7fb3000b11a0 msgr2=0x7fb30006d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 shutdown_connections 2024-04-25T10:04:17.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:17.911+0000 7fb30508d640 1 -- 172.21.15.82:0/3687001893 wait complete. 2024-04-25T10:04:17.931 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 Processor -- start 2024-04-25T10:04:18.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 -- start start 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc1451f0 0x7f5ffc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ffc0cee30 con 0x7f5ffc144ca0 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ffc0cc2b0 con 0x7f5ffc0ce510 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f60024bf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc06e290 con 0x7f5ffc1451f0 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55442/0 (socket says 172.21.15.82:55442) 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 -- 172.21.15.82:0/3361529747 learned_addr learned my addr 172.21.15.82:0/3361529747 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 -- 172.21.15.82:0/3361529747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc1451f0 msgr2=0x7f5ffc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f6000a5c640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc1451f0 0x7f5ffc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc1451f0 0x7f5ffc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 -- 172.21.15.82:0/3361529747 >> v1:172.21.15.82:6789/0 conn(0x7f5ffc144ca0 legacy=0x7f5ffc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.039+0000 7f5ffbfff640 1 -- 172.21.15.82:0/3361529747 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0ccc70 con 0x7f5ffc0ce510 2024-04-25T10:04:18.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f5ffbfff640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009e50 tx=0x7f5fec01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ba4a6883f0ce00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f5ffaffd640 1 -- 172.21.15.82:0/3361529747 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0b4070 con 0x7f5ffc0ce510 2024-04-25T10:04:18.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f5ffaffd640 1 -- 172.21.15.82:0/3361529747 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec0af400 con 0x7f5ffc0ce510 2024-04-25T10:04:18.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f5ffaffd640 1 -- 172.21.15.82:0/3361529747 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec0af7d0 con 0x7f5ffc0ce510 2024-04-25T10:04:18.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- 172.21.15.82:0/3361529747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc0ce510 msgr2=0x7f5ffc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5fec009e50 tx=0x7f5fec01f210 comp rx=0 tx=0).stop 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- 172.21.15.82:0/3361529747 shutdown_connections 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc1451f0 0x7f5ffc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 --2- 172.21.15.82:0/3361529747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc0ce510 0x7f5ffc0ce8f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- 172.21.15.82:0/3361529747 >> 172.21.15.82:0/3361529747 conn(0x7f5ffc0b11a0 msgr2=0x7f5ffc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- 172.21.15.82:0/3361529747 shutdown_connections 2024-04-25T10:04:18.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- 172.21.15.82:0/3361529747 wait complete. 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 Processor -- start 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.043+0000 7f60024bf640 1 -- start start 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc0ce510 0x7f5ffc0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ffc1451f0 0x7f5ffc1e17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc06ee30 con 0x7f5ffc144ca0 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc06df20 con 0x7f5ffc1451f0 2024-04-25T10:04:18.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc0cc900 con 0x7f5ffc0ce510 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55458/0 (socket says 172.21.15.82:55458) 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 -- 172.21.15.82:0/407412930 learned_addr learned my addr 172.21.15.82:0/407412930 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffbfff640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc0ce510 0x7f5ffc0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f6000a5c640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ffc1451f0 0x7f5ffc1e17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 -- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc0ce510 msgr2=0x7f5ffc0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc0ce510 0x7f5ffc0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 -- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ffc1451f0 msgr2=0x7f5ffc1e17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ffc1451f0 0x7f5ffc1e17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 -- 172.21.15.82:0/407412930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0b4190 con 0x7f5ffc144ca0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffb7fe640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c430 tx=0x7f5fe400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40052d0 con 0x7f5ffc144ca0 2024-04-25T10:04:18.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe4005470 con 0x7f5ffc144ca0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc0b4470 con 0x7f5ffc144ca0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007260 con 0x7f5ffc144ca0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc1e0fa0 con 0x7f5ffc144ca0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ffc04efa0 con 0x7f5ffc144ca0 2024-04-25T10:04:18.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fe4007400 con 0x7f5ffc144ca0 2024-04-25T10:04:18.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 0x7f5fdc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5fe4084730 con 0x7f5ffc144ca0 2024-04-25T10:04:18.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffbfff640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 0x7f5fdc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.047+0000 7f5ffbfff640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 0x7f5fdc069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0b1000 tx=0x7f5fec0aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.051+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5fe4055510 con 0x7f5ffc144ca0 2024-04-25T10:04:18.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.199+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ffc0b1cc0 con 0x7f5ffc144ca0 2024-04-25T10:04:18.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.199+0000 7f5ff8ff9640 1 -- 172.21.15.82:0/407412930 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+16967 (secure 0 0 0) 0x7f5fe4054e10 con 0x7f5ffc144ca0 2024-04-25T10:04:18.204 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:18.204 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":51,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:16.914756+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 msgr2=0x7f5fdc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 0x7f5fdc069450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0b1000 tx=0x7f5fec0aa000 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 msgr2=0x7f5ffc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c430 tx=0x7f5fe400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 shutdown_connections 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fdc066f90 0x7f5fdc069450 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ffc1451f0 0x7f5ffc1e17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ffc144ca0 0x7f5ffc138c30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 --2- 172.21.15.82:0/407412930 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ffc0ce510 0x7f5ffc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 >> 172.21.15.82:0/407412930 conn(0x7f5ffc0b11a0 msgr2=0x7f5ffc0c6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 shutdown_connections 2024-04-25T10:04:18.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.203+0000 7f60024bf640 1 -- 172.21.15.82:0/407412930 wait complete. 2024-04-25T10:04:18.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:04:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 Processor -- start 2024-04-25T10:04:18.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 -- start start 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 0x7f3aa40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa4073010 0x7f3aa40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa40cbd20 con 0x7f3aa40cf590 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa415ba90 con 0x7f3aa4073010 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa8880640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40cff30 con 0x7f3aa415b6b0 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa4073010 0x7f3aa40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa4073010 0x7f3aa40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55460/0 (socket says 172.21.15.82:55460) 2024-04-25T10:04:18.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 -- 172.21.15.82:0/1908320257 learned_addr learned my addr 172.21.15.82:0/1908320257 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 -- 172.21.15.82:0/1908320257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 msgr2=0x7f3aa40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa259c640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 0x7f3aa40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 0x7f3aa40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 -- 172.21.15.82:0/1908320257 >> v1:172.21.15.196:6789/0 conn(0x7f3aa40cf590 legacy=0x7f3aa415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa2d9d640 1 -- 172.21.15.82:0/1908320257 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40cc000 con 0x7f3aa4073010 2024-04-25T10:04:18.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.331+0000 7f3aa259c640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 0x7f3aa40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:18.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa2d9d640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa40cf9f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e50 tx=0x7f3a9c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e67b993de2bf14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa159a640 1 -- 172.21.15.82:0/1908320257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0a57e0 con 0x7f3aa4073010 2024-04-25T10:04:18.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa159a640 1 -- 172.21.15.82:0/1908320257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0a5980 con 0x7f3aa4073010 2024-04-25T10:04:18.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa159a640 1 -- 172.21.15.82:0/1908320257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0a5d50 con 0x7f3aa4073010 2024-04-25T10:04:18.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- 172.21.15.82:0/1908320257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 msgr2=0x7f3aa40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa40cf9f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e50 tx=0x7f3a9c01ca00 comp rx=0 tx=0).stop 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- 172.21.15.82:0/1908320257 shutdown_connections 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa40cf9f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- 172.21.15.82:0/1908320257 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa415b6b0 0x7f3aa40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- 172.21.15.82:0/1908320257 >> 172.21.15.82:0/1908320257 conn(0x7f3aa406ae60 msgr2=0x7f3aa406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- 172.21.15.82:0/1908320257 shutdown_connections 2024-04-25T10:04:18.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- 172.21.15.82:0/1908320257 wait complete. 2024-04-25T10:04:18.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 Processor -- start 2024-04-25T10:04:18.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- start start 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa40cf590 0x7f3aa4098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 0x7f3aa41574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ca900 con 0x7f3aa4073010 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40cba30 con 0x7f3aa415b6b0 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa8880640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40cadb0 con 0x7f3aa40cf590 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55462/0 (socket says 172.21.15.82:55462) 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.335+0000 7f3aa259c640 1 -- 172.21.15.82:0/135976736 learned_addr learned my addr 172.21.15.82:0/135976736 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa1d9b640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa40cf590 0x7f3aa4098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa2d9d640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 0x7f3aa41574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 -- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa40cf590 msgr2=0x7f3aa4098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa40cf590 0x7f3aa4098880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 -- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 msgr2=0x7f3aa41574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 0x7f3aa41574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 -- 172.21.15.82:0/135976736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa41540c0 con 0x7f3aa4073010 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa2d9d640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 0x7f3aa41574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa259c640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c7f0 tx=0x7f3a9400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94015070 con 0x7f3aa4073010 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a94007820 con 0x7f3aa4073010 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa41543a0 con 0x7f3aa4073010 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a94007c10 con 0x7f3aa4073010 2024-04-25T10:04:18.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa406d100 con 0x7f3aa4073010 2024-04-25T10:04:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a94018070 con 0x7f3aa4073010 2024-04-25T10:04:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa404efa0 con 0x7f3aa4073010 2024-04-25T10:04:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a940853a0 con 0x7f3aa4073010 2024-04-25T10:04:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa1d9b640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.339+0000 7f3aa1d9b640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a98007fd0 tx=0x7f3a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.343+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a940561a0 con 0x7f3aa4073010 2024-04-25T10:04:18.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.527+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3aa4165e30 con 0x7f3aa4073010 2024-04-25T10:04:18.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.627+0000 7f3a92ffd640 1 -- 172.21.15.82:0/135976736 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3a94055aa0 con 0x7f3aa4073010 2024-04-25T10:04:18.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 msgr2=0x7f3a84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3a98007fd0 tx=0x7f3a98009040 comp rx=0 tx=0).stop 2024-04-25T10:04:18.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 msgr2=0x7f3aa41584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3a9400c7f0 tx=0x7f3a9400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 shutdown_connections 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa40cf590 0x7f3aa4098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa415b6b0 0x7f3aa41574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 --2- 172.21.15.82:0/135976736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa4073010 0x7f3aa41584c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 >> 172.21.15.82:0/135976736 conn(0x7f3aa406ae60 msgr2=0x7f3aa40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 shutdown_connections 2024-04-25T10:04:18.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.631+0000 7f3aa8880640 1 -- 172.21.15.82:0/135976736 wait complete. 2024-04-25T10:04:18.651 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:04:18.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:04:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.759+0000 7f8136bcf640 1 Processor -- start 2024-04-25T10:04:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.759+0000 7f8136bcf640 1 -- start start 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f8136bcf640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f81300cfca0 0x7f813015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f8136bcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f81300caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f8136bcf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81300cbf10 con 0x7f81300b3cf0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f8136bcf640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81300cc1f0 con 0x7f81300cfca0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f8136bcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81300cc4d0 con 0x7f813015e4f0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813496b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f81300b3cf0 0x7f81300cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58346/0 (socket says 172.21.15.82:58346) 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813496b640 1 -- 172.21.15.82:0/4293716734 learned_addr learned my addr 172.21.15.82:0/4293716734 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617656285 0 0) 0x7f81300cbf10 con 0x7f81300b3cf0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8118003530 con 0x7f81300b3cf0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812ffff640 1 --2- 172.21.15.82:0/4293716734 >> v2:172.21.15.196:3300/0 conn(0x7f81300cfca0 0x7f813015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f81300caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3135073498 0 0) 0x7f8118003530 con 0x7f81300b3cf0 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 msgr2=0x7f81300caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f81300caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 >> v2:172.21.15.196:3300/0 conn(0x7f81300cfca0 msgr2=0x7f813015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 --2- 172.21.15.82:0/4293716734 >> v2:172.21.15.196:3300/0 conn(0x7f81300cfca0 0x7f813015dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300cc7b0 con 0x7f81300b3cf0 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8120001680 con 0x7f81300b3cf0 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 >> v1:172.21.15.82:6789/0 conn(0x7f81300b3cf0 legacy=0x7f81300cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8118003a30 0x7f8118003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f8118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8118004d50 0x7f81180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300cc1f0 con 0x7f8118004d50 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300cc4d0 con 0x7f8118003a30 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811800bb00 con 0x7f813015e4f0 2024-04-25T10:04:18.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8118004d50 0x7f81180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813496b640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8118003a30 0x7f8118003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f812ffff640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f8118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 -- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 msgr2=0x7f8118004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813015e4f0 0x7f8118004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 -- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8118003a30 msgr2=0x7f8118003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8118003a30 0x7f8118003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.763+0000 7f813516c640 1 -- 172.21.15.82:0/4293716734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118003530 con 0x7f8118004d50 2024-04-25T10:04:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f813516c640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8118004d50 0x7f81180051d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8128007330 tx=0x7f812801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1705bce61752a5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280b2070 con 0x7f8118004d50 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81280a5d20 con 0x7f8118004d50 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812f7fe640 1 -- 172.21.15.82:0/4293716734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128009be0 con 0x7f8118004d50 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/4293716734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8118004d50 msgr2=0x7f81180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 --2- 172.21.15.82:0/4293716734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8118004d50 0x7f81180051d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f8128007330 tx=0x7f812801ce20 comp rx=0 tx=0).stop 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f813516c640 1 -- 172.21.15.82:0/4293716734 reap_dead start 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/4293716734 shutdown_connections 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/4293716734 >> 172.21.15.82:0/4293716734 conn(0x7f813006ae60 msgr2=0x7f813006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/4293716734 shutdown_connections 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/4293716734 wait complete. 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 Processor -- start 2024-04-25T10:04:18.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- start start 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813015c430 0x7f81300b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813006e0d0 0x7f813006e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300b4250 con 0x7f81300c7750 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300b40d0 con 0x7f813015c430 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813015e930 con 0x7f813006e0d0 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55478/0 (socket says 172.21.15.82:55478) 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 -- 172.21.15.82:0/2546603142 learned_addr learned my addr 172.21.15.82:0/2546603142 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:18.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 -- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813006e0d0 msgr2=0x7f813006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813006e0d0 0x7f813006e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 -- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813015c430 msgr2=0x7f81300b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813015c430 0x7f81300b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 -- 172.21.15.82:0/2546603142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300ae500 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812ffff640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f812400f600 tx=0x7f812400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124019020 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124014870 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81300b1240 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8124014c60 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.767+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81300b0e20 con 0x7f81300c7750 2024-04-25T10:04:18.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813004efa0 con 0x7f81300c7750 2024-04-25T10:04:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f812401c020 con 0x7f81300c7750 2024-04-25T10:04:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f812d7fa640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81240858c0 con 0x7f81300c7750 2024-04-25T10:04:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f813496b640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:18.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.771+0000 7f813496b640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8120001cf0 tx=0x7f8120001c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:18.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.775+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8124023020 con 0x7f81300c7750 2024-04-25T10:04:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:18.963+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f81300ceee0 con 0x7f81300c7750 2024-04-25T10:04:19.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.635+0000 7f812d7fa640 1 -- 172.21.15.82:0/2546603142 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8124026350 con 0x7f81300c7750 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 msgr2=0x7f811c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8120001cf0 tx=0x7f8120001c80 comp rx=0 tx=0).stop 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 msgr2=0x7f81300c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f812400f600 tx=0x7f812400fac0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 shutdown_connections 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813006e0d0 0x7f813006e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f81300c7750 0x7f81300c7bd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 --2- 172.21.15.82:0/2546603142 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f813015c430 0x7f81300b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 >> 172.21.15.82:0/2546603142 conn(0x7f813006ae60 msgr2=0x7f813015d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 shutdown_connections 2024-04-25T10:04:19.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.639+0000 7f8136bcf640 1 -- 172.21.15.82:0/2546603142 wait complete. 2024-04-25T10:04:19.661 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:04:19.661 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:19.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 Processor -- start 2024-04-25T10:04:19.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 -- start start 2024-04-25T10:04:19.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f256815a700 0x7f256815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25680b24a0 0x7f25680b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25680b2090 con 0x7f25680b3f50 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256815f200 con 0x7f25680b24a0 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f256dc9c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25680b43e0 con 0x7f256815a700 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2566ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f25680b3f50 0x7f25680cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58352/0 (socket says 172.21.15.82:58352) 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2566ffd640 1 -- 172.21.15.82:0/1559594598 learned_addr learned my addr 172.21.15.82:0/1559594598 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2567fff640 1 --2- 172.21.15.82:0/1559594598 >> v2:172.21.15.82:3300/0 conn(0x7f25680b24a0 0x7f25680b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f25677fe640 1 --2- 172.21.15.82:0/1559594598 >> v2:172.21.15.196:3300/0 conn(0x7f256815a700 0x7f256815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f25667fc640 1 -- 172.21.15.82:0/1559594598 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415640501 0 0) 0x7f25680b2090 con 0x7f25680b3f50 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f25667fc640 1 -- 172.21.15.82:0/1559594598 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2550003530 con 0x7f25680b3f50 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2567fff640 1 -- 172.21.15.82:0/1559594598 >> v2:172.21.15.196:3300/0 conn(0x7f256815a700 msgr2=0x7f256815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2567fff640 1 --2- 172.21.15.82:0/1559594598 >> v2:172.21.15.196:3300/0 conn(0x7f256815a700 0x7f256815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2567fff640 1 -- 172.21.15.82:0/1559594598 >> v1:172.21.15.82:6789/0 conn(0x7f25680b3f50 legacy=0x7f25680cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.779+0000 7f2567fff640 1 -- 172.21.15.82:0/1559594598 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680b46c0 con 0x7f25680b24a0 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f2567fff640 1 --2- 172.21.15.82:0/1559594598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25680b24a0 0x7f25680b28f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f256806bd20 tx=0x7f256001c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7da250205fed9402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f25667fc640 1 -- 172.21.15.82:0/1559594598 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600b2070 con 0x7f25680b24a0 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f25667fc640 1 -- 172.21.15.82:0/1559594598 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25600a57c0 con 0x7f25680b24a0 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f25667fc640 1 -- 172.21.15.82:0/1559594598 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25600a5b70 con 0x7f25680b24a0 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- 172.21.15.82:0/1559594598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25680b24a0 msgr2=0x7f25680b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- 172.21.15.82:0/1559594598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25680b24a0 0x7f25680b28f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f256806bd20 tx=0x7f256001c810 comp rx=0 tx=0).stop 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- 172.21.15.82:0/1559594598 shutdown_connections 2024-04-25T10:04:19.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- 172.21.15.82:0/1559594598 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25680b24a0 0x7f25680b28f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- 172.21.15.82:0/1559594598 >> v2:172.21.15.196:3300/0 conn(0x7f256815a700 0x7f256815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- 172.21.15.82:0/1559594598 >> 172.21.15.82:0/1559594598 conn(0x7f25680b11a0 msgr2=0x7f25680b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- 172.21.15.82:0/1559594598 shutdown_connections 2024-04-25T10:04:19.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- 172.21.15.82:0/1559594598 wait complete. 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 Processor -- start 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- start start 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25680b24a0 0x7f256806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2568152f80 0x7f2568156ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680cd310 con 0x7f256815a700 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25680ce600 con 0x7f2568152f80 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.783+0000 7f256dc9c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25680cd490 con 0x7f25680b24a0 2024-04-25T10:04:19.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55492/0 (socket says 172.21.15.82:55492) 2024-04-25T10:04:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 -- 172.21.15.82:0/282888550 learned_addr learned my addr 172.21.15.82:0/282888550 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2567fff640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2568152f80 0x7f2568156ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 -- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25680b24a0 msgr2=0x7f256806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25680b24a0 0x7f256806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 -- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2568152f80 msgr2=0x7f2568156ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2568152f80 0x7f2568156ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 -- 172.21.15.82:0/282888550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2568161c20 con 0x7f256815a700 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f2566ffd640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c460 tx=0x7f255c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c017070 con 0x7f256815a700 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c005d10 con 0x7f256815a700 2024-04-25T10:04:19.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2568161000 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c0072b0 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2568161840 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256804efa0 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f255c01a070 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 0x7f2554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f255c0844a0 con 0x7f256815a700 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f25677fe640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 0x7f2554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:19.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.787+0000 7f25677fe640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 0x7f2554069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f25580098f0 tx=0x7f2558008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:19.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.791+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2568161000 con 0x7f256815a700 2024-04-25T10:04:19.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.987+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256806c030 con 0x7f256815a700 2024-04-25T10:04:19.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f254ffff640 1 -- 172.21.15.82:0/282888550 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f255c0553b0 con 0x7f256815a700 2024-04-25T10:04:19.993 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:19.994 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:19.994 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:04:19.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 msgr2=0x7f2554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 0x7f2554069420 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f25580098f0 tx=0x7f2558008040 comp rx=0 tx=0).stop 2024-04-25T10:04:19.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 msgr2=0x7f2568152a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f255c00c460 tx=0x7f255c00c920 comp rx=0 tx=0).stop 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 shutdown_connections 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2554066f60 0x7f2554069420 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2568152f80 0x7f2568156ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f256815a700 0x7f2568152a40 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 --2- 172.21.15.82:0/282888550 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25680b24a0 0x7f256806d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 >> 172.21.15.82:0/282888550 conn(0x7f25680b11a0 msgr2=0x7f25680b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 shutdown_connections 2024-04-25T10:04:19.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:19.991+0000 7f256dc9c640 1 -- 172.21.15.82:0/282888550 wait complete. 2024-04-25T10:04:20.012 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:04:20.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.115+0000 7f366f09e640 1 Processor -- start 2024-04-25T10:04:20.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.115+0000 7f366f09e640 1 -- start start 2024-04-25T10:04:20.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366f09e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366815da80 0x7f36680d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366f09e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f36680d07f0 0x7f366806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366f09e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366806be60 con 0x7f36680d1cd0 2024-04-25T10:04:20.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366f09e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366806c440 con 0x7f36680d07f0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366f09e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366815de60 con 0x7f366815da80 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f36680d1cd0 0x7f36680d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58356/0 (socket says 172.21.15.82:58356) 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 -- 172.21.15.82:0/3147638089 learned_addr learned my addr 172.21.15.82:0/3147638089 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366ce3a640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366815da80 0x7f36680d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366d63b640 1 --2- 172.21.15.82:0/3147638089 >> v2:172.21.15.196:3300/0 conn(0x7f36680d07f0 0x7f366806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1992034890 0 0) 0x7f366806be60 con 0x7f36680d1cd0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3650003530 con 0x7f36680d1cd0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3258647713 0 0) 0x7f3650003530 con 0x7f36680d1cd0 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366815da80 msgr2=0x7f36680d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366815da80 0x7f36680d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 >> v2:172.21.15.196:3300/0 conn(0x7f36680d07f0 msgr2=0x7f366806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 --2- 172.21.15.82:0/3147638089 >> v2:172.21.15.196:3300/0 conn(0x7f36680d07f0 0x7f366806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680d2450 con 0x7f36680d1cd0 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f365c001800 con 0x7f36680d1cd0 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 >> v1:172.21.15.82:6789/0 conn(0x7f36680d1cd0 legacy=0x7f36680d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3650003a30 0x7f3650003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004810 0x7f3650004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36500051d0 0x7f365000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366806c440 con 0x7f3650004810 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366815de60 con 0x7f36500051d0 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365000c070 con 0x7f3650003a30 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366ce3a640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f366815da80 0x7f36680d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f366d63b640 1 --2- 172.21.15.82:0/3147638089 >> v2:172.21.15.196:3300/0 conn(0x7f36680d07f0 0x7f366806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:04:20.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004810 0x7f3650004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 -- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3650003a30 msgr2=0x7f3650003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3650003a30 0x7f3650003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 -- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36500051d0 msgr2=0x7f365000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36500051d0 0x7f365000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.119+0000 7f3667fff640 1 -- 172.21.15.82:0/3147638089 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3650003530 con 0x7f3650004810 2024-04-25T10:04:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004810 0x7f3650004c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c240 tx=0x7f365c01f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb5e4b880a24bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c007d40 con 0x7f3650004810 2024-04-25T10:04:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c007270 con 0x7f3650004810 2024-04-25T10:04:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36677fe640 1 -- 172.21.15.82:0/3147638089 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0057b0 con 0x7f3650004810 2024-04-25T10:04:20.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/3147638089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004810 msgr2=0x7f3650004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 --2- 172.21.15.82:0/3147638089 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3650004810 0x7f3650004c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f365c00c240 tx=0x7f365c01f410 comp rx=0 tx=0).stop 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366d63b640 1 -- 172.21.15.82:0/3147638089 reap_dead start 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/3147638089 shutdown_connections 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/3147638089 >> 172.21.15.82:0/3147638089 conn(0x7f366808b000 msgr2=0x7f36680d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/3147638089 shutdown_connections 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/3147638089 wait complete. 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 Processor -- start 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- start start 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3668151b10 0x7f36680cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36680cc630 0x7f36680ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680d26f0 con 0x7f3668151ef0 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36680d20f0 con 0x7f36680cc630 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680d2870 con 0x7f3668151b10 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55512/0 (socket says 172.21.15.82:55512) 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 -- 172.21.15.82:0/221118711 learned_addr learned my addr 172.21.15.82:0/221118711 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366ce3a640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3668151b10 0x7f36680cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 -- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3668151b10 msgr2=0x7f36680cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3668151b10 0x7f36680cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 -- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36680cc630 msgr2=0x7f36680ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36680cc630 0x7f36680ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 -- 172.21.15.82:0/221118711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36680c8ad0 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f3667fff640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f365c001b00 tx=0x7f3650005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c007070 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c007420 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36680c7a30 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0057b0 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.123+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36680c7ff0 con 0x7f3668151ef0 2024-04-25T10:04:20.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366804efa0 con 0x7f3668151ef0 2024-04-25T10:04:20.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f365c005950 con 0x7f3668151ef0 2024-04-25T10:04:20.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f36657fa640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 0x7f3654069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f365c1222b0 con 0x7f3668151ef0 2024-04-25T10:04:20.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f366ce3a640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 0x7f3654069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.127+0000 7f366ce3a640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 0x7f3654069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3658006aa0 tx=0x7f3658008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.131+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f365c019120 con 0x7f3668151ef0 2024-04-25T10:04:20.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.279+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36680b2b20 con 0x7f3668151ef0 2024-04-25T10:04:20.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f36657fa640 1 -- 172.21.15.82:0/221118711 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+16954 (secure 0 0 0) 0x7f365c0f31c0 con 0x7f3668151ef0 2024-04-25T10:04:20.286 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:20.286 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":52,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:18.581501+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 msgr2=0x7f3654069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 0x7f3654069420 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3658006aa0 tx=0x7f3658008040 comp rx=0 tx=0).stop 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 msgr2=0x7f36680c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f365c001b00 tx=0x7f3650005840 comp rx=0 tx=0).stop 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 shutdown_connections 2024-04-25T10:04:20.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3654066f60 0x7f3654069420 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f36680cc630 0x7f36680ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3668151ef0 0x7f36680c71e0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 --2- 172.21.15.82:0/221118711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3668151b10 0x7f36680cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 >> 172.21.15.82:0/221118711 conn(0x7f366808b000 msgr2=0x7f3668160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 shutdown_connections 2024-04-25T10:04:20.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.283+0000 7f366f09e640 1 -- 172.21.15.82:0/221118711 wait complete. 2024-04-25T10:04:20.302 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:04:20.303 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 Processor -- start 2024-04-25T10:04:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 -- start start 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab80144ca0 0x7fab8006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800ccc70 con 0x7fab800ce510 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8006dc40 con 0x7fab8006ec60 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab86050640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800cc2b0 con 0x7fab80144ca0 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab80144ca0 0x7fab8006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fab800ce510 0x7fab800ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58366/0 (socket says 172.21.15.82:58366) 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7f7fe640 1 -- 172.21.15.82:0/175873312 learned_addr learned my addr 172.21.15.82:0/175873312 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7effd640 1 -- 172.21.15.82:0/175873312 >> v1:172.21.15.196:6789/0 conn(0x7fab8006ec60 legacy=0x7fab80142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7effd640 1 -- 172.21.15.82:0/175873312 >> v1:172.21.15.82:6789/0 conn(0x7fab800ce510 legacy=0x7fab800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7effd640 1 -- 172.21.15.82:0/175873312 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8006df20 con 0x7fab80144ca0 2024-04-25T10:04:20.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7e7fc640 1 -- 172.21.15.82:0/175873312 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960279104 0 0) 0x7fab800ccc70 con 0x7fab800ce510 2024-04-25T10:04:20.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.407+0000 7fab7effd640 1 --2- 172.21.15.82:0/175873312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab80144ca0 0x7fab8006e720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fab680097c0 tx=0x7fab6801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edb1fb8ac944ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7e7fc640 1 -- 172.21.15.82:0/175873312 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680b5070 con 0x7fab80144ca0 2024-04-25T10:04:20.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7e7fc640 1 -- 172.21.15.82:0/175873312 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab680a8950 con 0x7fab80144ca0 2024-04-25T10:04:20.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7e7fc640 1 -- 172.21.15.82:0/175873312 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680a8d20 con 0x7fab80144ca0 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/175873312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab80144ca0 msgr2=0x7fab8006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 --2- 172.21.15.82:0/175873312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab80144ca0 0x7fab8006e720 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fab680097c0 tx=0x7fab6801ec50 comp rx=0 tx=0).stop 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/175873312 shutdown_connections 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 --2- 172.21.15.82:0/175873312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab80144ca0 0x7fab8006e720 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/175873312 >> 172.21.15.82:0/175873312 conn(0x7fab800b11a0 msgr2=0x7fab800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/175873312 shutdown_connections 2024-04-25T10:04:20.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/175873312 wait complete. 2024-04-25T10:04:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 Processor -- start 2024-04-25T10:04:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- start start 2024-04-25T10:04:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8006ec60 0x7fab800acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab80144ca0 0x7fab800ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab80142d90 con 0x7fab800ce510 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab8006e290 con 0x7fab8006ec60 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab80142f10 con 0x7fab80144ca0 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55524/0 (socket says 172.21.15.82:55524) 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 -- 172.21.15.82:0/1661858942 learned_addr learned my addr 172.21.15.82:0/1661858942 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 -- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab80144ca0 msgr2=0x7fab800ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab80144ca0 0x7fab800ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 -- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8006ec60 msgr2=0x7fab800acac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7f7fe640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8006ec60 0x7fab800acac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8006ec60 0x7fab800acac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 -- 172.21.15.82:0/1661858942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800ae550 con 0x7fab800ce510 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab7effd640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fab680b3040 tx=0x7fab6801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680b5040 con 0x7fab800ce510 2024-04-25T10:04:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab680c0070 con 0x7fab800ce510 2024-04-25T10:04:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680b0ac0 con 0x7fab800ce510 2024-04-25T10:04:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab80140380 con 0x7fab800ce510 2024-04-25T10:04:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.411+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab800abfe0 con 0x7fab800ce510 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab8004efa0 con 0x7fab800ce510 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab680b0c60 con 0x7fab800ce510 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab77fff640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 0x7fab60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab68122780 con 0x7fab800ce510 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab7f7fe640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 0x7fab60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.415+0000 7fab7f7fe640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 0x7fab60069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab78000f70 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.419+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab680187b0 con 0x7fab800ce510 2024-04-25T10:04:20.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.559+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab800b1cc0 con 0x7fab60066f60 2024-04-25T10:04:20.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab77fff640 1 -- 172.21.15.82:0/1661858942 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237202 (secure 0 0 0) 0x7fab800b1cc0 con 0x7fab60066f60 2024-04-25T10:04:20.584 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:20.586 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 msgr2=0x7fab60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 0x7fab60069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fab78000f70 tx=0x7fab78009040 comp rx=0 tx=0).stop 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 msgr2=0x7fab8015b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fab680b3040 tx=0x7fab6801b9f0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 shutdown_connections 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab60066f60 0x7fab60069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab80144ca0 0x7fab800ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab800ce510 0x7fab8015b040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 --2- 172.21.15.82:0/1661858942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab8006ec60 0x7fab800acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 >> 172.21.15.82:0/1661858942 conn(0x7fab800b11a0 msgr2=0x7fab800af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 shutdown_connections 2024-04-25T10:04:20.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.579+0000 7fab86050640 1 -- 172.21.15.82:0/1661858942 wait complete. 2024-04-25T10:04:20.600 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-25T10:04:19.300848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238364,"kb_used_data":5536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736356,"statfs":{"total":773094113280,"available":772850028544,"internally_reserved":0,"allocated":5668864,"data_stored":3135252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001165"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.994312+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:04:13.994312+0000","last_peered":"2024-04-25T10:04:13.994312+0000","last_clean":"2024-04-25T10:04:13.994312+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:04:13.994312+0000","last_undegraded":"2024-04-25T10:04:13.994312+0000","last_fullsized":"2024-04-25T10:04:13.994312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":242961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":231993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:20.605 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.763+0000 7ff2a4265640 1 Processor -- start 2024-04-25T10:04:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.763+0000 7ff2a4265640 1 -- start start 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a4265640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff29c0b3f50 0x7ff29c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a4265640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff29c0b24a0 0x7ff29c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a4265640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29c0b2090 con 0x7ff29c15a700 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a4265640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c15f200 con 0x7ff29c0b24a0 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a4265640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff29c0b43e0 con 0x7ff29c0b3f50 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff29c0b24a0 0x7ff29c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff29c0b24a0 0x7ff29c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55534/0 (socket says 172.21.15.82:55534) 2024-04-25T10:04:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 -- 172.21.15.82:0/893381902 learned_addr learned my addr 172.21.15.82:0/893381902 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 -- 172.21.15.82:0/893381902 >> v2:172.21.15.196:3300/0 conn(0x7ff29c0b3f50 msgr2=0x7ff29c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 --2- 172.21.15.82:0/893381902 >> v2:172.21.15.196:3300/0 conn(0x7ff29c0b3f50 0x7ff29c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 -- 172.21.15.82:0/893381902 >> v1:172.21.15.196:6789/0 conn(0x7ff29c15a700 legacy=0x7ff29c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 -- 172.21.15.82:0/893381902 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c0b46c0 con 0x7ff29c0b24a0 2024-04-25T10:04:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a2802640 1 --2- 172.21.15.82:0/893381902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c0b24a0 0x7ff29c0b28f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff29c06c980 tx=0x7ff29401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ebd93906565435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a0fff640 1 -- 172.21.15.82:0/893381902 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940b0070 con 0x7ff29c0b24a0 2024-04-25T10:04:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a0fff640 1 -- 172.21.15.82:0/893381902 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940ab600 con 0x7ff29c0b24a0 2024-04-25T10:04:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.767+0000 7ff2a0fff640 1 -- 172.21.15.82:0/893381902 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940ab9d0 con 0x7ff29c0b24a0 2024-04-25T10:04:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/893381902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c0b24a0 msgr2=0x7ff29c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- 172.21.15.82:0/893381902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c0b24a0 0x7ff29c0b28f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff29c06c980 tx=0x7ff29401c3c0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/893381902 shutdown_connections 2024-04-25T10:04:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- 172.21.15.82:0/893381902 >> v2:172.21.15.196:3300/0 conn(0x7ff29c0b3f50 0x7ff29c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- 172.21.15.82:0/893381902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c0b24a0 0x7ff29c0b28f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/893381902 >> 172.21.15.82:0/893381902 conn(0x7ff29c0b11a0 msgr2=0x7ff29c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/893381902 shutdown_connections 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/893381902 wait complete. 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 Processor -- start 2024-04-25T10:04:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- start start 2024-04-25T10:04:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff29c0b24a0 0x7ff29c144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff29c15a700 0x7ff29c155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0cd310 con 0x7ff29c15c930 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0ce600 con 0x7ff29c0b24a0 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c0cd490 con 0x7ff29c15a700 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55544/0 (socket says 172.21.15.82:55544) 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 -- 172.21.15.82:0/747041654 learned_addr learned my addr 172.21.15.82:0/747041654 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2001640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff29c0b24a0 0x7ff29c144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 -- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff29c15a700 msgr2=0x7ff29c155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff29c15a700 0x7ff29c155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 -- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff29c0b24a0 msgr2=0x7ff29c144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff29c0b24a0 0x7ff29c144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 -- 172.21.15.82:0/747041654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c155aa0 con 0x7ff29c15c930 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a2802640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff2940ae040 tx=0x7ff29401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940ba020 con 0x7ff29c15c930 2024-04-25T10:04:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940b6620 con 0x7ff29c15c930 2024-04-25T10:04:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29c157d30 con 0x7ff29c15c930 2024-04-25T10:04:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2940b0040 con 0x7ff29c15c930 2024-04-25T10:04:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.771+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c1582f0 con 0x7ff29c15c930 2024-04-25T10:04:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c04efa0 con 0x7ff29c15c930 2024-04-25T10:04:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2940bd020 con 0x7ff29c15c930 2024-04-25T10:04:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2927fc640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 0x7ff280069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff29411f390 con 0x7ff29c15c930 2024-04-25T10:04:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2a2001640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 0x7ff280069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.775+0000 7ff2a2001640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 0x7ff280069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff28c001600 tx=0x7ff28c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.779+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2940c4030 con 0x7ff29c15c930 2024-04-25T10:04:20.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.919+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff29c145290 con 0x7ff280066f60 2024-04-25T10:04:20.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2927fc640 1 -- 172.21.15.82:0/747041654 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237202 (secure 0 0 0) 0x7ff29c145290 con 0x7ff280066f60 2024-04-25T10:04:20.935 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 msgr2=0x7ff280069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 0x7ff280069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff28c001600 tx=0x7ff28c009040 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 msgr2=0x7ff29c156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff2940ae040 tx=0x7ff29401ac00 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 shutdown_connections 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff280066f60 0x7ff280069420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff29c15a700 0x7ff29c155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff29c15c930 0x7ff29c156f60 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 --2- 172.21.15.82:0/747041654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff29c0b24a0 0x7ff29c144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 >> 172.21.15.82:0/747041654 conn(0x7ff29c0b11a0 msgr2=0x7ff29c0cf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:20.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 shutdown_connections 2024-04-25T10:04:20.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:20.931+0000 7ff2a4265640 1 -- 172.21.15.82:0/747041654 wait complete. 2024-04-25T10:04:20.951 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-25T10:04:19.300848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238364,"kb_used_data":5536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736356,"statfs":{"total":773094113280,"available":772850028544,"internally_reserved":0,"allocated":5668864,"data_stored":3135252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001165"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.994312+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:04:13.994312+0000","last_peered":"2024-04-25T10:04:13.994312+0000","last_clean":"2024-04-25T10:04:13.994312+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:04:13.994312+0000","last_undegraded":"2024-04-25T10:04:13.994312+0000","last_fullsized":"2024-04-25T10:04:13.994312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":242961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":231993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:20.955 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.103+0000 7f7ae4c64640 1 Processor -- start 2024-04-25T10:04:21.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.103+0000 7f7ae4c64640 1 -- start start 2024-04-25T10:04:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ae015a700 0x7f7ae015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ae00b3f50 0x7f7ae00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae00b2090 con 0x7f7ae00ce430 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae015f200 con 0x7f7ae015a700 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae00b43e0 con 0x7f7ae00b3f50 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ae015a700 0x7f7ae015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7aded9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7ae00ce430 0x7f7ae00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58378/0 (socket says 172.21.15.82:58378) 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7aded9d640 1 -- 172.21.15.82:0/4180781253 learned_addr learned my addr 172.21.15.82:0/4180781253 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ad5d9b640 1 --2- 172.21.15.82:0/4180781253 >> v2:172.21.15.196:3300/0 conn(0x7f7ae00b3f50 0x7f7ae00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 -- 172.21.15.82:0/4180781253 >> v2:172.21.15.196:3300/0 conn(0x7f7ae00b3f50 msgr2=0x7f7ae00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 --2- 172.21.15.82:0/4180781253 >> v2:172.21.15.196:3300/0 conn(0x7f7ae00b3f50 0x7f7ae00cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 -- 172.21.15.82:0/4180781253 >> v1:172.21.15.82:6789/0 conn(0x7f7ae00ce430 legacy=0x7f7ae00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7addd9b640 1 -- 172.21.15.82:0/4180781253 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128446374 0 0) 0x7f7ae00b2090 con 0x7f7ae00ce430 2024-04-25T10:04:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 -- 172.21.15.82:0/4180781253 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00b46c0 con 0x7f7ae015a700 2024-04-25T10:04:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ade59c640 1 --2- 172.21.15.82:0/4180781253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae015a700 0x7f7ae015ecc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00b3e20 tx=0x7f7ad001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cd481b913d0d1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7addd9b640 1 -- 172.21.15.82:0/4180781253 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00b0070 con 0x7f7ae015a700 2024-04-25T10:04:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7addd9b640 1 -- 172.21.15.82:0/4180781253 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad00ab600 con 0x7f7ae015a700 2024-04-25T10:04:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7addd9b640 1 -- 172.21.15.82:0/4180781253 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00ab9d0 con 0x7f7ae015a700 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- 172.21.15.82:0/4180781253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae015a700 msgr2=0x7f7ae015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/4180781253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae015a700 0x7f7ae015ecc0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00b3e20 tx=0x7f7ad001c330 comp rx=0 tx=0).stop 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- 172.21.15.82:0/4180781253 shutdown_connections 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/4180781253 >> v2:172.21.15.196:3300/0 conn(0x7f7ae00b3f50 0x7f7ae00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/4180781253 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae015a700 0x7f7ae015ecc0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- 172.21.15.82:0/4180781253 >> 172.21.15.82:0/4180781253 conn(0x7f7ae00b11a0 msgr2=0x7f7ae00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- 172.21.15.82:0/4180781253 shutdown_connections 2024-04-25T10:04:21.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.107+0000 7f7ae4c64640 1 -- 172.21.15.82:0/4180781253 wait complete. 2024-04-25T10:04:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 Processor -- start 2024-04-25T10:04:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- start start 2024-04-25T10:04:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ae015a700 0x7f7ae0157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ae014a670 0x7f7ae0149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cd490 con 0x7f7ae00ce430 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00cd310 con 0x7f7ae014a670 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae00b2bb0 con 0x7f7ae015a700 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55576/0 (socket says 172.21.15.82:55576) 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 -- 172.21.15.82:0/2993353308 learned_addr learned my addr 172.21.15.82:0/2993353308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 -- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ae015a700 msgr2=0x7f7ae0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ae015a700 0x7f7ae0157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 -- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ae014a670 msgr2=0x7f7ae0149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ae014a670 0x7f7ae0149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 -- 172.21.15.82:0/2993353308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0156470 con 0x7f7ae00ce430 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ade59c640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00ae040 tx=0x7f7ad001ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00b0050 con 0x7f7ae00ce430 2024-04-25T10:04:21.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad00bc070 con 0x7f7ae00ce430 2024-04-25T10:04:21.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad00b5a70 con 0x7f7ae00ce430 2024-04-25T10:04:21.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae0156750 con 0x7f7ae00ce430 2024-04-25T10:04:21.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae014f7d0 con 0x7f7ae00ce430 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.111+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae004efa0 con 0x7f7ae00ce430 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.115+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ad00ab4c0 con 0x7f7ae00ce430 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.115+0000 7f7ad77fe640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 0x7f7ac0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.115+0000 7f7ad5d9b640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 0x7f7ac0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.115+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ad011fbc0 con 0x7f7ae00ce430 2024-04-25T10:04:21.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.115+0000 7f7ad5d9b640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 0x7f7ac0069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0156060 tx=0x7f7ac8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.119+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ad0016070 con 0x7f7ae00ce430 2024-04-25T10:04:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.275+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ae006f260 con 0x7f7ac0066f60 2024-04-25T10:04:21.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ad77fe640 1 -- 172.21.15.82:0/2993353308 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237202 (secure 0 0 0) 0x7f7ae006f260 con 0x7f7ac0066f60 2024-04-25T10:04:21.291 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:21.293 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:04:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 msgr2=0x7f7ac0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 0x7f7ac0069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0156060 tx=0x7f7ac8008040 comp rx=0 tx=0).stop 2024-04-25T10:04:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 msgr2=0x7f7ae014b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00ae040 tx=0x7f7ad001ab70 comp rx=0 tx=0).stop 2024-04-25T10:04:21.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 shutdown_connections 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ac0066f60 0x7f7ac0069420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ae015a700 0x7f7ae0157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ae014a670 0x7f7ae0149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 --2- 172.21.15.82:0/2993353308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ae00ce430 0x7f7ae014b7a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.287+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 >> 172.21.15.82:0/2993353308 conn(0x7f7ae00b11a0 msgr2=0x7f7ae0158350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.291+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 shutdown_connections 2024-04-25T10:04:21.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.291+0000 7f7ae4c64640 1 -- 172.21.15.82:0/2993353308 wait complete. 2024-04-25T10:04:21.308 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-04-25T10:04:19.300848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":238364,"kb_used_data":5536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736356,"statfs":{"total":773094113280,"available":772850028544,"internally_reserved":0,"allocated":5668864,"data_stored":3135252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001165"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:16.555812+0000","last_change":"2024-04-25T10:04:16.555812+0000","last_active":"2024-04-25T10:04:16.555812+0000","last_peered":"2024-04-25T10:04:16.555812+0000","last_clean":"2024-04-25T10:04:16.555812+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:16.555812+0000","last_undegraded":"2024-04-25T10:04:16.555812+0000","last_fullsized":"2024-04-25T10:04:16.555812+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:04:15.545516+0000","last_change":"2024-04-25T10:04:15.545516+0000","last_active":"2024-04-25T10:04:15.545516+0000","last_peered":"2024-04-25T10:04:15.545516+0000","last_clean":"2024-04-25T10:04:15.545516+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:04:15.545516+0000","last_undegraded":"2024-04-25T10:04:15.545516+0000","last_fullsized":"2024-04-25T10:04:15.545516+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-25T10:04:13.994312+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:04:13.994312+0000","last_peered":"2024-04-25T10:04:13.994312+0000","last_clean":"2024-04-25T10:04:13.994312+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:04:13.994312+0000","last_undegraded":"2024-04-25T10:04:13.994312+0000","last_fullsized":"2024-04-25T10:04:13.994312+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27552,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":544768,"data_stored":237512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27584,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31724,"kb_used_data":544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340116,"statfs":{"total":96636764160,"available":96604278784,"internally_reserved":0,"allocated":557056,"data_stored":242961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31700,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340140,"statfs":{"total":96636764160,"available":96604303360,"internally_reserved":0,"allocated":532480,"data_stored":231993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27580,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":569344,"data_stored":248410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339520,"statfs":{"total":96636764160,"available":96603668480,"internally_reserved":0,"allocated":1163264,"data_stored":838778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:21.312 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-25T10:04:21.312 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-25T10:04:21.313 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-25T10:04:21.314 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.315 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-25T10:04:21.316 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.317 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.318 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-25T10:04:21.319 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.320 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.321 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-25T10:04:21.322 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2024-04-25T10:04:21.323 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T10:04:21.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2024-04-25T10:04:21.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T10:04:21.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:16.555812+0000', 'last_change': '2024-04-25T10:04:16.555812+0000', 'last_active': '2024-04-25T10:04:16.555812+0000', 'last_peered': '2024-04-25T10:04:16.555812+0000', 'last_clean': '2024-04-25T10:04:16.555812+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:16.555812+0000', 'last_undegraded': '2024-04-25T10:04:16.555812+0000', 'last_fullsized': '2024-04-25T10:04:16.555812+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:16.555812+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2024-04-25T10:04:21.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T10:04:15.545516+0000', 'last_change': '2024-04-25T10:04:15.545516+0000', 'last_active': '2024-04-25T10:04:15.545516+0000', 'last_peered': '2024-04-25T10:04:15.545516+0000', 'last_clean': '2024-04-25T10:04:15.545516+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:04:15.545516+0000', 'last_undegraded': '2024-04-25T10:04:15.545516+0000', 'last_fullsized': '2024-04-25T10:04:15.545516+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'last_clean_scrub_stamp': '2024-04-25T10:04:15.545516+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T10:04:21.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:04:21.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 Processor -- start 2024-04-25T10:04:21.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 -- start start 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61c015ee10 0x7f61c00cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61c00cc030 0x7f61c00cc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c00cff70 con 0x7f61c00b24c0 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c00cfd40 con 0x7f61c00cc030 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c6703640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c009a480 con 0x7f61c015ee10 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c4ca0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61c00cc030 0x7f61c00cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c4ca0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61c00cc030 0x7f61c00cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55590/0 (socket says 172.21.15.82:55590) 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.467+0000 7f61c4ca0640 1 -- 172.21.15.82:0/4037478958 learned_addr learned my addr 172.21.15.82:0/4037478958 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61bffff640 1 --2- 172.21.15.82:0/4037478958 >> v2:172.21.15.196:3300/0 conn(0x7f61c015ee10 0x7f61c00cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c4ca0640 1 -- 172.21.15.82:0/4037478958 >> v2:172.21.15.196:3300/0 conn(0x7f61c015ee10 msgr2=0x7f61c00cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c4ca0640 1 --2- 172.21.15.82:0/4037478958 >> v2:172.21.15.196:3300/0 conn(0x7f61c015ee10 0x7f61c00cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c4ca0640 1 -- 172.21.15.82:0/4037478958 >> v1:172.21.15.196:6789/0 conn(0x7f61c00b24c0 legacy=0x7f61c00cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c4ca0640 1 -- 172.21.15.82:0/4037478958 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c00d0250 con 0x7f61c00cc030 2024-04-25T10:04:21.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c4ca0640 1 --2- 172.21.15.82:0/4037478958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c00cc030 0x7f61c00cc440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f61c00743b0 tx=0x7f61b801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8f51d74e4f4cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61beffd640 1 -- 172.21.15.82:0/4037478958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a5780 con 0x7f61c00cc030 2024-04-25T10:04:21.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61beffd640 1 -- 172.21.15.82:0/4037478958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80a5920 con 0x7f61c00cc030 2024-04-25T10:04:21.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61beffd640 1 -- 172.21.15.82:0/4037478958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80a5cf0 con 0x7f61c00cc030 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 -- 172.21.15.82:0/4037478958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c00cc030 msgr2=0x7f61c00cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 --2- 172.21.15.82:0/4037478958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c00cc030 0x7f61c00cc440 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f61c00743b0 tx=0x7f61b801c9b0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 -- 172.21.15.82:0/4037478958 shutdown_connections 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 --2- 172.21.15.82:0/4037478958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c00cc030 0x7f61c00cc440 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 --2- 172.21.15.82:0/4037478958 >> v2:172.21.15.196:3300/0 conn(0x7f61c015ee10 0x7f61c00cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 -- 172.21.15.82:0/4037478958 >> 172.21.15.82:0/4037478958 conn(0x7f61c00b3010 msgr2=0x7f61c00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 -- 172.21.15.82:0/4037478958 shutdown_connections 2024-04-25T10:04:21.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.471+0000 7f61c6703640 1 -- 172.21.15.82:0/4037478958 wait complete. 2024-04-25T10:04:21.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 Processor -- start 2024-04-25T10:04:21.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- start start 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c00b24c0 0x7f61c015b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61c00cc030 0x7f61c0147320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00c6200 con 0x7f61c015ee10 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00c6080 con 0x7f61c00b24c0 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c00cd190 con 0x7f61c00cc030 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55600/0 (socket says 172.21.15.82:55600) 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 -- 172.21.15.82:0/3968535096 learned_addr learned my addr 172.21.15.82:0/3968535096 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bffff640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c00b24c0 0x7f61c015b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bf7fe640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61c00cc030 0x7f61c0147320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 -- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61c00cc030 msgr2=0x7f61c0147320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61c00cc030 0x7f61c0147320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 -- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c00b24c0 msgr2=0x7f61c015b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c00b24c0 0x7f61c015b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 -- 172.21.15.82:0/3968535096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0147860 con 0x7f61c015ee10 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c4ca0640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f61b80af040 tx=0x7f61b8019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80b8470 con 0x7f61c015ee10 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80b8630 con 0x7f61c015ee10 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c01f65e0 con 0x7f61c015ee10 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b80b8a20 con 0x7f61c015ee10 2024-04-25T10:04:21.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c01f6ba0 con 0x7f61c015ee10 2024-04-25T10:04:21.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c004efa0 con 0x7f61c015ee10 2024-04-25T10:04:21.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61b80c0030 con 0x7f61c015ee10 2024-04-25T10:04:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.475+0000 7f61bcff9640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.479+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61b8120ab0 con 0x7f61c015ee10 2024-04-25T10:04:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.479+0000 7f61bffff640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.479+0000 7f61bffff640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 0x7f61ac069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61c00c6050 tx=0x7f61b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.479+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61b80c7020 con 0x7f61c015ee10 2024-04-25T10:04:21.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f61c00d1300 con 0x7f61c015ee10 2024-04-25T10:04:21.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61bcff9640 1 -- 172.21.15.82:0/3968535096 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1432 (secure 0 0 0) 0x7f61c0147860 con 0x7f61c015ee10 2024-04-25T10:04:21.698 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:21.698 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":158,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039313,"num_in_osds":8,"osd_in_since":1714039304,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":244576256,"bytes_avail":772849537024,"bytes_total":773094113280,"write_bytes_sec":2495,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":24,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5946}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2024-04-25T10:04:05.297201+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:04:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 msgr2=0x7f61ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 0x7f61ac069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f61c00c6050 tx=0x7f61b4008040 comp rx=0 tx=0).stop 2024-04-25T10:04:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 msgr2=0x7f61c01649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f61b80af040 tx=0x7f61b8019750 comp rx=0 tx=0).stop 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 shutdown_connections 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61ac066f60 0x7f61ac069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61c00cc030 0x7f61c0147320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61c015ee10 0x7f61c01649d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 --2- 172.21.15.82:0/3968535096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61c00b24c0 0x7f61c015b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 >> 172.21.15.82:0/3968535096 conn(0x7f61c00b3010 msgr2=0x7f61c00c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 shutdown_connections 2024-04-25T10:04:21.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.695+0000 7f61c6703640 1 -- 172.21.15.82:0/3968535096 wait complete. 2024-04-25T10:04:21.715 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:04:21.715 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:21.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.815+0000 7fbb1cfe3640 1 Processor -- start 2024-04-25T10:04:21.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.815+0000 7fbb1cfe3640 1 -- start start 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1cfe3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb1806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1cfe3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180b2090 con 0x7fbb1815a700 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1cfe3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb180b43e0 con 0x7fbb180b3f50 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1cfe3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180b46c0 con 0x7fbb1815ecc0 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbb1815a700 0x7fbb180cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58390/0 (socket says 172.21.15.82:58390) 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 -- 172.21.15.82:0/2268965652 learned_addr learned my addr 172.21.15.82:0/2268965652 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb16d9d640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb1806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816357280 0 0) 0x7fbb180b2090 con 0x7fbb1815a700 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb00003530 con 0x7fbb1815a700 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920442295 0 0) 0x7fbb00003530 con 0x7fbb1815a700 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 msgr2=0x7fbb1806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb1806e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 >> v1:172.21.15.196:6789/0 conn(0x7fbb180b3f50 legacy=0x7fbb180b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb180cd6f0 con 0x7fbb1815a700 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fbb080016a0 con 0x7fbb1815a700 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 >> v1:172.21.15.82:6789/0 conn(0x7fbb1815a700 legacy=0x7fbb180cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb00003a60 0x7fbb00003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00004d50 0x7fbb000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180b46c0 con 0x7fbb00003a60 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000bc00 con 0x7fbb00004d50 2024-04-25T10:04:21.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0000be00 con 0x7fbb1815ecc0 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb00003a60 0x7fbb00003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb16d9d640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00004d50 0x7fbb000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb15d9b640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 -- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 msgr2=0x7fbb00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 -- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00004d50 msgr2=0x7fbb000051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00004d50 0x7fbb000051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb1659c640 1 -- 172.21.15.82:0/2268965652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00003530 con 0x7fbb00003a60 2024-04-25T10:04:21.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.819+0000 7fbb15d9b640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb1815ecc0 0x7fbb00004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:21.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1659c640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb00003a60 0x7fbb00003e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c290 tx=0x7fbb0801ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32790c5c08515daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08007d60 con 0x7fbb00003a60 2024-04-25T10:04:21.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb080a9070 con 0x7fbb00003a60 2024-04-25T10:04:21.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1559a640 1 -- 172.21.15.82:0/2268965652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08005d40 con 0x7fbb00003a60 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/2268965652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb00003a60 msgr2=0x7fbb00003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/2268965652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb00003a60 0x7fbb00003e80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800c290 tx=0x7fbb0801ccd0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb16d9d640 1 -- 172.21.15.82:0/2268965652 reap_dead start 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/2268965652 shutdown_connections 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/2268965652 >> 172.21.15.82:0/2268965652 conn(0x7fbb180b11a0 msgr2=0x7fbb180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/2268965652 shutdown_connections 2024-04-25T10:04:21.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/2268965652 wait complete. 2024-04-25T10:04:21.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 Processor -- start 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.823+0000 7fbb1cfe3640 1 -- start start 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00003a60 0x7fbb18162630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb18160790 0x7fbb1815d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180cd310 con 0x7fbb18162b70 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1815f0d0 con 0x7fbb00003a60 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb180cd490 con 0x7fbb18160790 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55618/0 (socket says 172.21.15.82:55618) 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 -- 172.21.15.82:0/562871943 learned_addr learned my addr 172.21.15.82:0/562871943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1659c640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00003a60 0x7fbb18162630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb15d9b640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb18160790 0x7fbb1815d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 -- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb18160790 msgr2=0x7fbb1815d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb18160790 0x7fbb1815d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 -- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00003a60 msgr2=0x7fbb18162630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00003a60 0x7fbb18162630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 -- 172.21.15.82:0/562871943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18160c20 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb16d9d640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbb18070fe0 tx=0x7fbb100115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb10011db0 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1001b070 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb100039a0 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb18160f00 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb181614c0 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1806f260 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb10003b40 con 0x7fbb18162b70 2024-04-25T10:04:21.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:21.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb10085860 con 0x7fbb18162b70 2024-04-25T10:04:21.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1659c640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:21.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.827+0000 7fbb1659c640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 0x7fbaf8069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb080ab000 tx=0x7fbb080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:21.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.831+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb10025020 con 0x7fbb18162b70 2024-04-25T10:04:21.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.971+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb1804ef80 con 0x7fbaf8066f60 2024-04-25T10:04:21.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.979+0000 7fbb06ffd640 1 -- 172.21.15.82:0/562871943 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251959 (secure 0 0 0) 0x7fbb1804ef80 con 0x7fbaf8066f60 2024-04-25T10:04:21.985 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:21.987 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 msgr2=0x7fbaf8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 0x7fbaf8069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbb080ab000 tx=0x7fbb080a7070 comp rx=0 tx=0).stop 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 msgr2=0x7fbb1815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbb18070fe0 tx=0x7fbb100115a0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 shutdown_connections 2024-04-25T10:04:21.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbaf8066f60 0x7fbaf8069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb18160790 0x7fbb1815d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb18162b70 0x7fbb1815d560 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 --2- 172.21.15.82:0/562871943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb00003a60 0x7fbb18162630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 >> 172.21.15.82:0/562871943 conn(0x7fbb180b11a0 msgr2=0x7fbb180716f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 shutdown_connections 2024-04-25T10:04:21.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:21.983+0000 7fbb1cfe3640 1 -- 172.21.15.82:0/562871943 wait complete. 2024-04-25T10:04:22.002 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-04-25T10:04:21.301442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238844,"kb_used_data":6008,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735876,"statfs":{"total":773094113280,"available":772849537024,"internally_reserved":0,"allocated":6152192,"data_stored":3359948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.744305"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590168+0000","last_change":"2024-04-25T10:04:17.620459+0000","last_active":"2024-04-25T10:04:18.590168+0000","last_peered":"2024-04-25T10:04:18.590168+0000","last_clean":"2024-04-25T10:04:18.590168+0000","last_became_active":"2024-04-25T10:04:17.620167+0000","last_became_peered":"2024-04-25T10:04:17.620167+0000","last_unstale":"2024-04-25T10:04:18.590168+0000","last_undegraded":"2024-04-25T10:04:18.590168+0000","last_fullsized":"2024-04-25T10:04:18.590168+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:54.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595314+0000","last_change":"2024-04-25T10:04:17.639196+0000","last_active":"2024-04-25T10:04:19.595314+0000","last_peered":"2024-04-25T10:04:19.595314+0000","last_clean":"2024-04-25T10:04:19.595314+0000","last_became_active":"2024-04-25T10:04:17.634300+0000","last_became_peered":"2024-04-25T10:04:17.634300+0000","last_unstale":"2024-04-25T10:04:19.595314+0000","last_undegraded":"2024-04-25T10:04:19.595314+0000","last_fullsized":"2024-04-25T10:04:19.595314+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591873+0000","last_change":"2024-04-25T10:04:17.617299+0000","last_active":"2024-04-25T10:04:19.591873+0000","last_peered":"2024-04-25T10:04:19.591873+0000","last_clean":"2024-04-25T10:04:19.591873+0000","last_became_active":"2024-04-25T10:04:17.617046+0000","last_became_peered":"2024-04-25T10:04:17.617046+0000","last_unstale":"2024-04-25T10:04:19.591873+0000","last_undegraded":"2024-04-25T10:04:19.591873+0000","last_fullsized":"2024-04-25T10:04:19.591873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:44.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595008+0000","last_change":"2024-04-25T10:04:17.637718+0000","last_active":"2024-04-25T10:04:19.595008+0000","last_peered":"2024-04-25T10:04:19.595008+0000","last_clean":"2024-04-25T10:04:19.595008+0000","last_became_active":"2024-04-25T10:04:17.634066+0000","last_became_peered":"2024-04-25T10:04:17.634066+0000","last_unstale":"2024-04-25T10:04:19.595008+0000","last_undegraded":"2024-04-25T10:04:19.595008+0000","last_fullsized":"2024-04-25T10:04:19.595008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:40.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587527+0000","last_change":"2024-04-25T10:04:17.640053+0000","last_active":"2024-04-25T10:04:18.587527+0000","last_peered":"2024-04-25T10:04:18.587527+0000","last_clean":"2024-04-25T10:04:18.587527+0000","last_became_active":"2024-04-25T10:04:17.639295+0000","last_became_peered":"2024-04-25T10:04:17.639295+0000","last_unstale":"2024-04-25T10:04:18.587527+0000","last_undegraded":"2024-04-25T10:04:18.587527+0000","last_fullsized":"2024-04-25T10:04:18.587527+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:42.777647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589381+0000","last_change":"2024-04-25T10:04:17.641548+0000","last_active":"2024-04-25T10:04:18.589381+0000","last_peered":"2024-04-25T10:04:18.589381+0000","last_clean":"2024-04-25T10:04:18.589381+0000","last_became_active":"2024-04-25T10:04:17.641247+0000","last_became_peered":"2024-04-25T10:04:17.641247+0000","last_unstale":"2024-04-25T10:04:18.589381+0000","last_undegraded":"2024-04-25T10:04:18.589381+0000","last_fullsized":"2024-04-25T10:04:18.589381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:58.756738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590700+0000","last_change":"2024-04-25T10:04:17.611089+0000","last_active":"2024-04-25T10:04:19.590700+0000","last_peered":"2024-04-25T10:04:19.590700+0000","last_clean":"2024-04-25T10:04:19.590700+0000","last_became_active":"2024-04-25T10:04:17.610807+0000","last_became_peered":"2024-04-25T10:04:17.610807+0000","last_unstale":"2024-04-25T10:04:19.590700+0000","last_undegraded":"2024-04-25T10:04:19.590700+0000","last_fullsized":"2024-04-25T10:04:19.590700+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625698+0000","last_change":"2024-04-25T10:04:17.613096+0000","last_active":"2024-04-25T10:04:19.625698+0000","last_peered":"2024-04-25T10:04:19.625698+0000","last_clean":"2024-04-25T10:04:19.625698+0000","last_became_active":"2024-04-25T10:04:17.612835+0000","last_became_peered":"2024-04-25T10:04:17.612835+0000","last_unstale":"2024-04-25T10:04:19.625698+0000","last_undegraded":"2024-04-25T10:04:19.625698+0000","last_fullsized":"2024-04-25T10:04:19.625698+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:45.110644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626244+0000","last_change":"2024-04-25T10:04:17.623258+0000","last_active":"2024-04-25T10:04:19.626244+0000","last_peered":"2024-04-25T10:04:19.626244+0000","last_clean":"2024-04-25T10:04:19.626244+0000","last_became_active":"2024-04-25T10:04:17.622796+0000","last_became_peered":"2024-04-25T10:04:17.622796+0000","last_unstale":"2024-04-25T10:04:19.626244+0000","last_undegraded":"2024-04-25T10:04:19.626244+0000","last_fullsized":"2024-04-25T10:04:19.626244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:17.227912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590063+0000","last_change":"2024-04-25T10:04:17.625940+0000","last_active":"2024-04-25T10:04:18.590063+0000","last_peered":"2024-04-25T10:04:18.590063+0000","last_clean":"2024-04-25T10:04:18.590063+0000","last_became_active":"2024-04-25T10:04:17.625665+0000","last_became_peered":"2024-04-25T10:04:17.625665+0000","last_unstale":"2024-04-25T10:04:18.590063+0000","last_undegraded":"2024-04-25T10:04:18.590063+0000","last_fullsized":"2024-04-25T10:04:18.590063+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:13.855322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624850+0000","last_change":"2024-04-25T10:04:17.644952+0000","last_active":"2024-04-25T10:04:19.624850+0000","last_peered":"2024-04-25T10:04:19.624850+0000","last_clean":"2024-04-25T10:04:19.624850+0000","last_became_active":"2024-04-25T10:04:17.644550+0000","last_became_peered":"2024-04-25T10:04:17.644550+0000","last_unstale":"2024-04-25T10:04:19.624850+0000","last_undegraded":"2024-04-25T10:04:19.624850+0000","last_fullsized":"2024-04-25T10:04:19.624850+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:12.443879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626031+0000","last_change":"2024-04-25T10:04:17.599974+0000","last_active":"2024-04-25T10:04:19.626031+0000","last_peered":"2024-04-25T10:04:19.626031+0000","last_clean":"2024-04-25T10:04:19.626031+0000","last_became_active":"2024-04-25T10:04:17.599650+0000","last_became_peered":"2024-04-25T10:04:17.599650+0000","last_unstale":"2024-04-25T10:04:19.626031+0000","last_undegraded":"2024-04-25T10:04:19.626031+0000","last_fullsized":"2024-04-25T10:04:19.626031+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:46.568980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589170+0000","last_change":"2024-04-25T10:04:17.623559+0000","last_active":"2024-04-25T10:04:18.589170+0000","last_peered":"2024-04-25T10:04:18.589170+0000","last_clean":"2024-04-25T10:04:18.589170+0000","last_became_active":"2024-04-25T10:04:17.623066+0000","last_became_peered":"2024-04-25T10:04:17.623066+0000","last_unstale":"2024-04-25T10:04:18.589170+0000","last_undegraded":"2024-04-25T10:04:18.589170+0000","last_fullsized":"2024-04-25T10:04:18.589170+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:50.351734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591036+0000","last_change":"2024-04-25T10:04:17.607521+0000","last_active":"2024-04-25T10:04:19.591036+0000","last_peered":"2024-04-25T10:04:19.591036+0000","last_clean":"2024-04-25T10:04:19.591036+0000","last_became_active":"2024-04-25T10:04:17.607204+0000","last_became_peered":"2024-04-25T10:04:17.607204+0000","last_unstale":"2024-04-25T10:04:19.591036+0000","last_undegraded":"2024-04-25T10:04:19.591036+0000","last_fullsized":"2024-04-25T10:04:19.591036+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:17.510586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594957+0000","last_change":"2024-04-25T10:04:17.638183+0000","last_active":"2024-04-25T10:04:19.594957+0000","last_peered":"2024-04-25T10:04:19.594957+0000","last_clean":"2024-04-25T10:04:19.594957+0000","last_became_active":"2024-04-25T10:04:17.637625+0000","last_became_peered":"2024-04-25T10:04:17.637625+0000","last_unstale":"2024-04-25T10:04:19.594957+0000","last_undegraded":"2024-04-25T10:04:19.594957+0000","last_fullsized":"2024-04-25T10:04:19.594957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.352440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589194+0000","last_change":"2024-04-25T10:04:17.623209+0000","last_active":"2024-04-25T10:04:18.589194+0000","last_peered":"2024-04-25T10:04:18.589194+0000","last_clean":"2024-04-25T10:04:18.589194+0000","last_became_active":"2024-04-25T10:04:17.622919+0000","last_became_peered":"2024-04-25T10:04:17.622919+0000","last_unstale":"2024-04-25T10:04:18.589194+0000","last_undegraded":"2024-04-25T10:04:18.589194+0000","last_fullsized":"2024-04-25T10:04:18.589194+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:13.323967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591521+0000","last_change":"2024-04-25T10:04:17.625508+0000","last_active":"2024-04-25T10:04:19.591521+0000","last_peered":"2024-04-25T10:04:19.591521+0000","last_clean":"2024-04-25T10:04:19.591521+0000","last_became_active":"2024-04-25T10:04:17.623814+0000","last_became_peered":"2024-04-25T10:04:17.623814+0000","last_unstale":"2024-04-25T10:04:19.591521+0000","last_undegraded":"2024-04-25T10:04:19.591521+0000","last_fullsized":"2024-04-25T10:04:19.591521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595438+0000","last_change":"2024-04-25T10:04:17.642432+0000","last_active":"2024-04-25T10:04:19.595438+0000","last_peered":"2024-04-25T10:04:19.595438+0000","last_clean":"2024-04-25T10:04:19.595438+0000","last_became_active":"2024-04-25T10:04:17.639977+0000","last_became_peered":"2024-04-25T10:04:17.639977+0000","last_unstale":"2024-04-25T10:04:19.595438+0000","last_undegraded":"2024-04-25T10:04:19.595438+0000","last_fullsized":"2024-04-25T10:04:19.595438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591694+0000","last_change":"2024-04-25T10:04:17.612339+0000","last_active":"2024-04-25T10:04:19.591694+0000","last_peered":"2024-04-25T10:04:19.591694+0000","last_clean":"2024-04-25T10:04:19.591694+0000","last_became_active":"2024-04-25T10:04:17.611228+0000","last_became_peered":"2024-04-25T10:04:17.611228+0000","last_unstale":"2024-04-25T10:04:19.591694+0000","last_undegraded":"2024-04-25T10:04:19.591694+0000","last_fullsized":"2024-04-25T10:04:19.591694+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:45.110644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588164+0000","last_change":"2024-04-25T10:04:17.603290+0000","last_active":"2024-04-25T10:04:18.588164+0000","last_peered":"2024-04-25T10:04:18.588164+0000","last_clean":"2024-04-25T10:04:18.588164+0000","last_became_active":"2024-04-25T10:04:17.602462+0000","last_became_peered":"2024-04-25T10:04:17.602462+0000","last_unstale":"2024-04-25T10:04:18.588164+0000","last_undegraded":"2024-04-25T10:04:18.588164+0000","last_fullsized":"2024-04-25T10:04:18.588164+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:30.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587673+0000","last_change":"2024-04-25T10:04:17.633966+0000","last_active":"2024-04-25T10:04:18.587673+0000","last_peered":"2024-04-25T10:04:18.587673+0000","last_clean":"2024-04-25T10:04:18.587673+0000","last_became_active":"2024-04-25T10:04:17.633238+0000","last_became_peered":"2024-04-25T10:04:17.633238+0000","last_unstale":"2024-04-25T10:04:18.587673+0000","last_undegraded":"2024-04-25T10:04:18.587673+0000","last_fullsized":"2024-04-25T10:04:18.587673+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:30.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625240+0000","last_change":"2024-04-25T10:04:17.623203+0000","last_active":"2024-04-25T10:04:19.625240+0000","last_peered":"2024-04-25T10:04:19.625240+0000","last_clean":"2024-04-25T10:04:19.625240+0000","last_became_active":"2024-04-25T10:04:17.622883+0000","last_became_peered":"2024-04-25T10:04:17.622883+0000","last_unstale":"2024-04-25T10:04:19.625240+0000","last_undegraded":"2024-04-25T10:04:19.625240+0000","last_fullsized":"2024-04-25T10:04:19.625240+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.859768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624302+0000","last_change":"2024-04-25T10:04:17.634620+0000","last_active":"2024-04-25T10:04:19.624302+0000","last_peered":"2024-04-25T10:04:19.624302+0000","last_clean":"2024-04-25T10:04:19.624302+0000","last_became_active":"2024-04-25T10:04:17.634335+0000","last_became_peered":"2024-04-25T10:04:17.634335+0000","last_unstale":"2024-04-25T10:04:19.624302+0000","last_undegraded":"2024-04-25T10:04:19.624302+0000","last_fullsized":"2024-04-25T10:04:19.624302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:31.640607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590753+0000","last_change":"2024-04-25T10:04:17.611672+0000","last_active":"2024-04-25T10:04:19.590753+0000","last_peered":"2024-04-25T10:04:19.590753+0000","last_clean":"2024-04-25T10:04:19.590753+0000","last_became_active":"2024-04-25T10:04:17.610536+0000","last_became_peered":"2024-04-25T10:04:17.610536+0000","last_unstale":"2024-04-25T10:04:19.590753+0000","last_undegraded":"2024-04-25T10:04:19.590753+0000","last_fullsized":"2024-04-25T10:04:19.590753+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591825+0000","last_change":"2024-04-25T10:04:17.608814+0000","last_active":"2024-04-25T10:04:19.591825+0000","last_peered":"2024-04-25T10:04:19.591825+0000","last_clean":"2024-04-25T10:04:19.591825+0000","last_became_active":"2024-04-25T10:04:17.608548+0000","last_became_peered":"2024-04-25T10:04:17.608548+0000","last_unstale":"2024-04-25T10:04:19.591825+0000","last_undegraded":"2024-04-25T10:04:19.591825+0000","last_fullsized":"2024-04-25T10:04:19.591825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:46.568980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626387+0000","last_change":"2024-04-25T10:04:17.623489+0000","last_active":"2024-04-25T10:04:19.626387+0000","last_peered":"2024-04-25T10:04:19.626387+0000","last_clean":"2024-04-25T10:04:19.626387+0000","last_became_active":"2024-04-25T10:04:17.623242+0000","last_became_peered":"2024-04-25T10:04:17.623242+0000","last_unstale":"2024-04-25T10:04:19.626387+0000","last_undegraded":"2024-04-25T10:04:19.626387+0000","last_fullsized":"2024-04-25T10:04:19.626387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.826321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590139+0000","last_change":"2024-04-25T10:04:17.618382+0000","last_active":"2024-04-25T10:04:18.590139+0000","last_peered":"2024-04-25T10:04:18.590139+0000","last_clean":"2024-04-25T10:04:18.590139+0000","last_became_active":"2024-04-25T10:04:17.618118+0000","last_became_peered":"2024-04-25T10:04:17.618118+0000","last_unstale":"2024-04-25T10:04:18.590139+0000","last_undegraded":"2024-04-25T10:04:18.590139+0000","last_fullsized":"2024-04-25T10:04:18.590139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:25.153889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588252+0000","last_change":"2024-04-25T10:04:17.608112+0000","last_active":"2024-04-25T10:04:18.588252+0000","last_peered":"2024-04-25T10:04:18.588252+0000","last_clean":"2024-04-25T10:04:18.588252+0000","last_became_active":"2024-04-25T10:04:17.607335+0000","last_became_peered":"2024-04-25T10:04:17.607335+0000","last_unstale":"2024-04-25T10:04:18.588252+0000","last_undegraded":"2024-04-25T10:04:18.588252+0000","last_fullsized":"2024-04-25T10:04:18.588252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:37.154221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591112+0000","last_change":"2024-04-25T10:04:17.616901+0000","last_active":"2024-04-25T10:04:19.591112+0000","last_peered":"2024-04-25T10:04:19.591112+0000","last_clean":"2024-04-25T10:04:19.591112+0000","last_became_active":"2024-04-25T10:04:17.616428+0000","last_became_peered":"2024-04-25T10:04:17.616428+0000","last_unstale":"2024-04-25T10:04:19.591112+0000","last_undegraded":"2024-04-25T10:04:19.591112+0000","last_fullsized":"2024-04-25T10:04:19.591112+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:44.107003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594905+0000","last_change":"2024-04-25T10:04:17.638860+0000","last_active":"2024-04-25T10:04:19.594905+0000","last_peered":"2024-04-25T10:04:19.594905+0000","last_clean":"2024-04-25T10:04:19.594905+0000","last_became_active":"2024-04-25T10:04:17.637531+0000","last_became_peered":"2024-04-25T10:04:17.637531+0000","last_unstale":"2024-04-25T10:04:19.594905+0000","last_undegraded":"2024-04-25T10:04:19.594905+0000","last_fullsized":"2024-04-25T10:04:19.594905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.560792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591657+0000","last_change":"2024-04-25T10:04:17.608971+0000","last_active":"2024-04-25T10:04:19.591657+0000","last_peered":"2024-04-25T10:04:19.591657+0000","last_clean":"2024-04-25T10:04:19.591657+0000","last_became_active":"2024-04-25T10:04:17.607554+0000","last_became_peered":"2024-04-25T10:04:17.607554+0000","last_unstale":"2024-04-25T10:04:19.591657+0000","last_undegraded":"2024-04-25T10:04:19.591657+0000","last_fullsized":"2024-04-25T10:04:19.591657+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.347166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122817+0000","last_change":"2024-04-25T10:04:17.616813+0000","last_active":"2024-04-25T10:04:19.122817+0000","last_peered":"2024-04-25T10:04:19.122817+0000","last_clean":"2024-04-25T10:04:19.122817+0000","last_became_active":"2024-04-25T10:04:17.615954+0000","last_became_peered":"2024-04-25T10:04:17.615954+0000","last_unstale":"2024-04-25T10:04:19.122817+0000","last_undegraded":"2024-04-25T10:04:19.122817+0000","last_fullsized":"2024-04-25T10:04:19.122817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594501+0000","last_change":"2024-04-25T10:04:17.638638+0000","last_active":"2024-04-25T10:04:19.594501+0000","last_peered":"2024-04-25T10:04:19.594501+0000","last_clean":"2024-04-25T10:04:19.594501+0000","last_became_active":"2024-04-25T10:04:17.638019+0000","last_became_peered":"2024-04-25T10:04:17.638019+0000","last_unstale":"2024-04-25T10:04:19.594501+0000","last_undegraded":"2024-04-25T10:04:19.594501+0000","last_fullsized":"2024-04-25T10:04:19.594501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:40.238684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589405+0000","last_change":"2024-04-25T10:04:16.582562+0000","last_active":"2024-04-25T10:04:18.589405+0000","last_peered":"2024-04-25T10:04:18.589405+0000","last_clean":"2024-04-25T10:04:18.589405+0000","last_became_active":"2024-04-25T10:04:16.581694+0000","last_became_peered":"2024-04-25T10:04:16.581694+0000","last_unstale":"2024-04-25T10:04:18.589405+0000","last_undegraded":"2024-04-25T10:04:18.589405+0000","last_fullsized":"2024-04-25T10:04:18.589405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:50.754619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122649+0000","last_change":"2024-04-25T10:04:17.638993+0000","last_active":"2024-04-25T10:04:19.122649+0000","last_peered":"2024-04-25T10:04:19.122649+0000","last_clean":"2024-04-25T10:04:19.122649+0000","last_became_active":"2024-04-25T10:04:17.638544+0000","last_became_peered":"2024-04-25T10:04:17.638544+0000","last_unstale":"2024-04-25T10:04:19.122649+0000","last_undegraded":"2024-04-25T10:04:19.122649+0000","last_fullsized":"2024-04-25T10:04:19.122649+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590208+0000","last_change":"2024-04-25T10:04:16.577803+0000","last_active":"2024-04-25T10:04:18.590208+0000","last_peered":"2024-04-25T10:04:18.590208+0000","last_clean":"2024-04-25T10:04:18.590208+0000","last_became_active":"2024-04-25T10:04:16.577566+0000","last_became_peered":"2024-04-25T10:04:16.577566+0000","last_unstale":"2024-04-25T10:04:18.590208+0000","last_undegraded":"2024-04-25T10:04:18.590208+0000","last_fullsized":"2024-04-25T10:04:18.590208+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587456+0000","last_change":"2024-04-25T10:04:17.633508+0000","last_active":"2024-04-25T10:04:18.587456+0000","last_peered":"2024-04-25T10:04:18.587456+0000","last_clean":"2024-04-25T10:04:18.587456+0000","last_became_active":"2024-04-25T10:04:17.633001+0000","last_became_peered":"2024-04-25T10:04:17.633001+0000","last_unstale":"2024-04-25T10:04:18.587456+0000","last_undegraded":"2024-04-25T10:04:18.587456+0000","last_fullsized":"2024-04-25T10:04:18.587456+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:48.673080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590225+0000","last_change":"2024-04-25T10:04:16.585813+0000","last_active":"2024-04-25T10:04:18.590225+0000","last_peered":"2024-04-25T10:04:18.590225+0000","last_clean":"2024-04-25T10:04:18.590225+0000","last_became_active":"2024-04-25T10:04:16.585428+0000","last_became_peered":"2024-04-25T10:04:16.585428+0000","last_unstale":"2024-04-25T10:04:18.590225+0000","last_undegraded":"2024-04-25T10:04:18.590225+0000","last_fullsized":"2024-04-25T10:04:18.590225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:45.171318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625239+0000","last_change":"2024-04-25T10:04:17.620072+0000","last_active":"2024-04-25T10:04:19.625239+0000","last_peered":"2024-04-25T10:04:19.625239+0000","last_clean":"2024-04-25T10:04:19.625239+0000","last_became_active":"2024-04-25T10:04:17.619809+0000","last_became_peered":"2024-04-25T10:04:17.619809+0000","last_unstale":"2024-04-25T10:04:19.625239+0000","last_undegraded":"2024-04-25T10:04:19.625239+0000","last_fullsized":"2024-04-25T10:04:19.625239+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123278+0000","last_change":"2024-04-25T10:04:16.596433+0000","last_active":"2024-04-25T10:04:19.123278+0000","last_peered":"2024-04-25T10:04:19.123278+0000","last_clean":"2024-04-25T10:04:19.123278+0000","last_became_active":"2024-04-25T10:04:16.594317+0000","last_became_peered":"2024-04-25T10:04:16.594317+0000","last_unstale":"2024-04-25T10:04:19.123278+0000","last_undegraded":"2024-04-25T10:04:19.123278+0000","last_fullsized":"2024-04-25T10:04:19.123278+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:46.908646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588624+0000","last_change":"2024-04-25T10:04:17.601284+0000","last_active":"2024-04-25T10:04:18.588624+0000","last_peered":"2024-04-25T10:04:18.588624+0000","last_clean":"2024-04-25T10:04:18.588624+0000","last_became_active":"2024-04-25T10:04:17.599129+0000","last_became_peered":"2024-04-25T10:04:17.599129+0000","last_unstale":"2024-04-25T10:04:18.588624+0000","last_undegraded":"2024-04-25T10:04:18.588624+0000","last_fullsized":"2024-04-25T10:04:18.588624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:30.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591205+0000","last_change":"2024-04-25T10:04:16.589640+0000","last_active":"2024-04-25T10:04:19.591205+0000","last_peered":"2024-04-25T10:04:19.591205+0000","last_clean":"2024-04-25T10:04:19.591205+0000","last_became_active":"2024-04-25T10:04:16.589365+0000","last_became_peered":"2024-04-25T10:04:16.589365+0000","last_unstale":"2024-04-25T10:04:19.591205+0000","last_undegraded":"2024-04-25T10:04:19.591205+0000","last_fullsized":"2024-04-25T10:04:19.591205+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:39.134229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595139+0000","last_change":"2024-04-25T10:04:17.643861+0000","last_active":"2024-04-25T10:04:19.595139+0000","last_peered":"2024-04-25T10:04:19.595139+0000","last_clean":"2024-04-25T10:04:19.595139+0000","last_became_active":"2024-04-25T10:04:17.642577+0000","last_became_peered":"2024-04-25T10:04:17.642577+0000","last_unstale":"2024-04-25T10:04:19.595139+0000","last_undegraded":"2024-04-25T10:04:19.595139+0000","last_fullsized":"2024-04-25T10:04:19.595139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.769875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587605+0000","last_change":"2024-04-25T10:04:16.586646+0000","last_active":"2024-04-25T10:04:18.587605+0000","last_peered":"2024-04-25T10:04:18.587605+0000","last_clean":"2024-04-25T10:04:18.587605+0000","last_became_active":"2024-04-25T10:04:16.582735+0000","last_became_peered":"2024-04-25T10:04:16.582735+0000","last_unstale":"2024-04-25T10:04:18.587605+0000","last_undegraded":"2024-04-25T10:04:18.587605+0000","last_fullsized":"2024-04-25T10:04:18.587605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:09.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626649+0000","last_change":"2024-04-25T10:04:17.620084+0000","last_active":"2024-04-25T10:04:19.626649+0000","last_peered":"2024-04-25T10:04:19.626649+0000","last_clean":"2024-04-25T10:04:19.626649+0000","last_became_active":"2024-04-25T10:04:17.619490+0000","last_became_peered":"2024-04-25T10:04:17.619490+0000","last_unstale":"2024-04-25T10:04:19.626649+0000","last_undegraded":"2024-04-25T10:04:19.626649+0000","last_fullsized":"2024-04-25T10:04:19.626649+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:35.185541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587498+0000","last_change":"2024-04-25T10:04:16.596633+0000","last_active":"2024-04-25T10:04:18.587498+0000","last_peered":"2024-04-25T10:04:18.587498+0000","last_clean":"2024-04-25T10:04:18.587498+0000","last_became_active":"2024-04-25T10:04:16.595945+0000","last_became_peered":"2024-04-25T10:04:16.595945+0000","last_unstale":"2024-04-25T10:04:18.587498+0000","last_undegraded":"2024-04-25T10:04:18.587498+0000","last_fullsized":"2024-04-25T10:04:18.587498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:45.171318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595279+0000","last_change":"2024-04-25T10:04:17.638402+0000","last_active":"2024-04-25T10:04:19.595279+0000","last_peered":"2024-04-25T10:04:19.595279+0000","last_clean":"2024-04-25T10:04:19.595279+0000","last_became_active":"2024-04-25T10:04:17.637821+0000","last_became_peered":"2024-04-25T10:04:17.637821+0000","last_unstale":"2024-04-25T10:04:19.595279+0000","last_undegraded":"2024-04-25T10:04:19.595279+0000","last_fullsized":"2024-04-25T10:04:19.595279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591445+0000","last_change":"2024-04-25T10:04:16.594413+0000","last_active":"2024-04-25T10:04:19.591445+0000","last_peered":"2024-04-25T10:04:19.591445+0000","last_clean":"2024-04-25T10:04:19.591445+0000","last_became_active":"2024-04-25T10:04:16.594157+0000","last_became_peered":"2024-04-25T10:04:16.594157+0000","last_unstale":"2024-04-25T10:04:19.591445+0000","last_undegraded":"2024-04-25T10:04:19.591445+0000","last_fullsized":"2024-04-25T10:04:19.591445+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:36.143924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587077+0000","last_change":"2024-04-25T10:04:17.602429+0000","last_active":"2024-04-25T10:04:18.587077+0000","last_peered":"2024-04-25T10:04:18.587077+0000","last_clean":"2024-04-25T10:04:18.587077+0000","last_became_active":"2024-04-25T10:04:17.601670+0000","last_became_peered":"2024-04-25T10:04:17.601670+0000","last_unstale":"2024-04-25T10:04:18.587077+0000","last_undegraded":"2024-04-25T10:04:18.587077+0000","last_fullsized":"2024-04-25T10:04:18.587077+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:54.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595367+0000","last_change":"2024-04-25T10:04:16.600592+0000","last_active":"2024-04-25T10:04:19.595367+0000","last_peered":"2024-04-25T10:04:19.595367+0000","last_clean":"2024-04-25T10:04:19.595367+0000","last_became_active":"2024-04-25T10:04:16.599063+0000","last_became_peered":"2024-04-25T10:04:16.599063+0000","last_unstale":"2024-04-25T10:04:19.595367+0000","last_undegraded":"2024-04-25T10:04:19.595367+0000","last_fullsized":"2024-04-25T10:04:19.595367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:05.963386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122591+0000","last_change":"2024-04-25T10:04:17.622606+0000","last_active":"2024-04-25T10:04:19.122591+0000","last_peered":"2024-04-25T10:04:19.122591+0000","last_clean":"2024-04-25T10:04:19.122591+0000","last_became_active":"2024-04-25T10:04:17.621889+0000","last_became_peered":"2024-04-25T10:04:17.621889+0000","last_unstale":"2024-04-25T10:04:19.122591+0000","last_undegraded":"2024-04-25T10:04:19.122591+0000","last_fullsized":"2024-04-25T10:04:19.122591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.769875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588530+0000","last_change":"2024-04-25T10:04:16.591093+0000","last_active":"2024-04-25T10:04:18.588530+0000","last_peered":"2024-04-25T10:04:18.588530+0000","last_clean":"2024-04-25T10:04:18.588530+0000","last_became_active":"2024-04-25T10:04:16.590343+0000","last_became_peered":"2024-04-25T10:04:16.590343+0000","last_unstale":"2024-04-25T10:04:18.588530+0000","last_undegraded":"2024-04-25T10:04:18.588530+0000","last_fullsized":"2024-04-25T10:04:18.588530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.077859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590868+0000","last_change":"2024-04-25T10:04:17.622400+0000","last_active":"2024-04-25T10:04:19.590868+0000","last_peered":"2024-04-25T10:04:19.590868+0000","last_clean":"2024-04-25T10:04:19.590868+0000","last_became_active":"2024-04-25T10:04:17.621956+0000","last_became_peered":"2024-04-25T10:04:17.621956+0000","last_unstale":"2024-04-25T10:04:19.590868+0000","last_undegraded":"2024-04-25T10:04:19.590868+0000","last_fullsized":"2024-04-25T10:04:19.590868+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595322+0000","last_change":"2024-04-25T10:04:16.600334+0000","last_active":"2024-04-25T10:04:19.595322+0000","last_peered":"2024-04-25T10:04:19.595322+0000","last_clean":"2024-04-25T10:04:19.595322+0000","last_became_active":"2024-04-25T10:04:16.599624+0000","last_became_peered":"2024-04-25T10:04:16.599624+0000","last_unstale":"2024-04-25T10:04:19.595322+0000","last_undegraded":"2024-04-25T10:04:19.595322+0000","last_fullsized":"2024-04-25T10:04:19.595322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:29.022281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587883+0000","last_change":"2024-04-25T10:04:17.595845+0000","last_active":"2024-04-25T10:04:18.587883+0000","last_peered":"2024-04-25T10:04:18.587883+0000","last_clean":"2024-04-25T10:04:18.587883+0000","last_became_active":"2024-04-25T10:04:17.595164+0000","last_became_peered":"2024-04-25T10:04:17.595164+0000","last_unstale":"2024-04-25T10:04:18.587883+0000","last_undegraded":"2024-04-25T10:04:18.587883+0000","last_fullsized":"2024-04-25T10:04:18.587883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:48.673080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626253+0000","last_change":"2024-04-25T10:04:16.586798+0000","last_active":"2024-04-25T10:04:19.626253+0000","last_peered":"2024-04-25T10:04:19.626253+0000","last_clean":"2024-04-25T10:04:19.626253+0000","last_became_active":"2024-04-25T10:04:16.586585+0000","last_became_peered":"2024-04-25T10:04:16.586585+0000","last_unstale":"2024-04-25T10:04:19.626253+0000","last_undegraded":"2024-04-25T10:04:19.626253+0000","last_fullsized":"2024-04-25T10:04:19.626253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:12.845025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123038+0000","last_change":"2024-04-25T10:04:17.645245+0000","last_active":"2024-04-25T10:04:19.123038+0000","last_peered":"2024-04-25T10:04:19.123038+0000","last_clean":"2024-04-25T10:04:19.123038+0000","last_became_active":"2024-04-25T10:04:17.643383+0000","last_became_peered":"2024-04-25T10:04:17.643383+0000","last_unstale":"2024-04-25T10:04:19.123038+0000","last_undegraded":"2024-04-25T10:04:19.123038+0000","last_fullsized":"2024-04-25T10:04:19.123038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591583+0000","last_change":"2024-04-25T10:04:16.593571+0000","last_active":"2024-04-25T10:04:19.591583+0000","last_peered":"2024-04-25T10:04:19.591583+0000","last_clean":"2024-04-25T10:04:19.591583+0000","last_became_active":"2024-04-25T10:04:16.593207+0000","last_became_peered":"2024-04-25T10:04:16.593207+0000","last_unstale":"2024-04-25T10:04:19.591583+0000","last_undegraded":"2024-04-25T10:04:19.591583+0000","last_fullsized":"2024-04-25T10:04:19.591583+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:41.767350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.592126+0000","last_change":"2024-04-25T10:04:17.610938+0000","last_active":"2024-04-25T10:04:19.592126+0000","last_peered":"2024-04-25T10:04:19.592126+0000","last_clean":"2024-04-25T10:04:19.592126+0000","last_became_active":"2024-04-25T10:04:17.608774+0000","last_became_peered":"2024-04-25T10:04:17.608774+0000","last_unstale":"2024-04-25T10:04:19.592126+0000","last_undegraded":"2024-04-25T10:04:19.592126+0000","last_fullsized":"2024-04-25T10:04:19.592126+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.826321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587716+0000","last_change":"2024-04-25T10:04:16.585541+0000","last_active":"2024-04-25T10:04:18.587716+0000","last_peered":"2024-04-25T10:04:18.587716+0000","last_clean":"2024-04-25T10:04:18.587716+0000","last_became_active":"2024-04-25T10:04:16.584645+0000","last_became_peered":"2024-04-25T10:04:16.584645+0000","last_unstale":"2024-04-25T10:04:18.587716+0000","last_undegraded":"2024-04-25T10:04:18.587716+0000","last_fullsized":"2024-04-25T10:04:18.587716+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:49.138416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626269+0000","last_change":"2024-04-25T10:04:17.636525+0000","last_active":"2024-04-25T10:04:19.626269+0000","last_peered":"2024-04-25T10:04:19.626269+0000","last_clean":"2024-04-25T10:04:19.626269+0000","last_became_active":"2024-04-25T10:04:17.636280+0000","last_became_peered":"2024-04-25T10:04:17.636280+0000","last_unstale":"2024-04-25T10:04:19.626269+0000","last_undegraded":"2024-04-25T10:04:19.626269+0000","last_fullsized":"2024-04-25T10:04:19.626269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:44.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594741+0000","last_change":"2024-04-25T10:04:16.606292+0000","last_active":"2024-04-25T10:04:19.594741+0000","last_peered":"2024-04-25T10:04:19.594741+0000","last_clean":"2024-04-25T10:04:19.594741+0000","last_became_active":"2024-04-25T10:04:16.605504+0000","last_became_peered":"2024-04-25T10:04:16.605504+0000","last_unstale":"2024-04-25T10:04:19.594741+0000","last_undegraded":"2024-04-25T10:04:19.594741+0000","last_fullsized":"2024-04-25T10:04:19.594741+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:49.341437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591598+0000","last_change":"2024-04-25T10:04:17.625236+0000","last_active":"2024-04-25T10:04:19.591598+0000","last_peered":"2024-04-25T10:04:19.591598+0000","last_clean":"2024-04-25T10:04:19.591598+0000","last_became_active":"2024-04-25T10:04:17.623409+0000","last_became_peered":"2024-04-25T10:04:17.623409+0000","last_unstale":"2024-04-25T10:04:19.591598+0000","last_undegraded":"2024-04-25T10:04:19.591598+0000","last_fullsized":"2024-04-25T10:04:19.591598+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.859768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626038+0000","last_change":"2024-04-25T10:04:16.589311+0000","last_active":"2024-04-25T10:04:19.626038+0000","last_peered":"2024-04-25T10:04:19.626038+0000","last_clean":"2024-04-25T10:04:19.626038+0000","last_became_active":"2024-04-25T10:04:16.588803+0000","last_became_peered":"2024-04-25T10:04:16.588803+0000","last_unstale":"2024-04-25T10:04:19.626038+0000","last_undegraded":"2024-04-25T10:04:19.626038+0000","last_fullsized":"2024-04-25T10:04:19.626038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:53.662409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625948+0000","last_change":"2024-04-25T10:04:17.620686+0000","last_active":"2024-04-25T10:04:19.625948+0000","last_peered":"2024-04-25T10:04:19.625948+0000","last_clean":"2024-04-25T10:04:19.625948+0000","last_became_active":"2024-04-25T10:04:17.620323+0000","last_became_peered":"2024-04-25T10:04:17.620323+0000","last_unstale":"2024-04-25T10:04:19.625948+0000","last_undegraded":"2024-04-25T10:04:19.625948+0000","last_fullsized":"2024-04-25T10:04:19.625948+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:30.448502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591958+0000","last_change":"2024-04-25T10:04:16.590540+0000","last_active":"2024-04-25T10:04:19.591958+0000","last_peered":"2024-04-25T10:04:19.591958+0000","last_clean":"2024-04-25T10:04:19.591958+0000","last_became_active":"2024-04-25T10:04:16.589517+0000","last_became_peered":"2024-04-25T10:04:16.589517+0000","last_unstale":"2024-04-25T10:04:19.591958+0000","last_undegraded":"2024-04-25T10:04:19.591958+0000","last_fullsized":"2024-04-25T10:04:19.591958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:29.864725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589193+0000","last_change":"2024-04-25T10:04:17.617502+0000","last_active":"2024-04-25T10:04:18.589193+0000","last_peered":"2024-04-25T10:04:18.589193+0000","last_clean":"2024-04-25T10:04:18.589193+0000","last_became_active":"2024-04-25T10:04:17.617241+0000","last_became_peered":"2024-04-25T10:04:17.617241+0000","last_unstale":"2024-04-25T10:04:18.589193+0000","last_undegraded":"2024-04-25T10:04:18.589193+0000","last_fullsized":"2024-04-25T10:04:18.589193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.380920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625077+0000","last_change":"2024-04-25T10:04:16.598220+0000","last_active":"2024-04-25T10:04:19.625077+0000","last_peered":"2024-04-25T10:04:19.625077+0000","last_clean":"2024-04-25T10:04:19.625077+0000","last_became_active":"2024-04-25T10:04:16.597975+0000","last_became_peered":"2024-04-25T10:04:16.597975+0000","last_unstale":"2024-04-25T10:04:19.625077+0000","last_undegraded":"2024-04-25T10:04:19.625077+0000","last_fullsized":"2024-04-25T10:04:19.625077+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:10.105919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587324+0000","last_change":"2024-04-25T10:04:17.622238+0000","last_active":"2024-04-25T10:04:18.587324+0000","last_peered":"2024-04-25T10:04:18.587324+0000","last_clean":"2024-04-25T10:04:18.587324+0000","last_became_active":"2024-04-25T10:04:17.621439+0000","last_became_peered":"2024-04-25T10:04:17.621439+0000","last_unstale":"2024-04-25T10:04:18.587324+0000","last_undegraded":"2024-04-25T10:04:18.587324+0000","last_fullsized":"2024-04-25T10:04:18.587324+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:40.144526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589152+0000","last_change":"2024-04-25T10:04:16.579948+0000","last_active":"2024-04-25T10:04:18.589152+0000","last_peered":"2024-04-25T10:04:18.589152+0000","last_clean":"2024-04-25T10:04:18.589152+0000","last_became_active":"2024-04-25T10:04:16.578658+0000","last_became_peered":"2024-04-25T10:04:16.578658+0000","last_unstale":"2024-04-25T10:04:18.589152+0000","last_undegraded":"2024-04-25T10:04:18.589152+0000","last_fullsized":"2024-04-25T10:04:18.589152+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.077859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589510+0000","last_change":"2024-04-25T10:04:17.621236+0000","last_active":"2024-04-25T10:04:18.589510+0000","last_peered":"2024-04-25T10:04:18.589510+0000","last_clean":"2024-04-25T10:04:18.589510+0000","last_became_active":"2024-04-25T10:04:17.620974+0000","last_became_peered":"2024-04-25T10:04:17.620974+0000","last_unstale":"2024-04-25T10:04:18.589510+0000","last_undegraded":"2024-04-25T10:04:18.589510+0000","last_fullsized":"2024-04-25T10:04:18.589510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:30.593485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588878+0000","last_change":"2024-04-25T10:04:16.578507+0000","last_active":"2024-04-25T10:04:18.588878+0000","last_peered":"2024-04-25T10:04:18.588878+0000","last_clean":"2024-04-25T10:04:18.588878+0000","last_became_active":"2024-04-25T10:04:16.577675+0000","last_became_peered":"2024-04-25T10:04:16.577675+0000","last_unstale":"2024-04-25T10:04:18.588878+0000","last_undegraded":"2024-04-25T10:04:18.588878+0000","last_fullsized":"2024-04-25T10:04:18.588878+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:15.792569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123079+0000","last_change":"2024-04-25T10:04:17.622875+0000","last_active":"2024-04-25T10:04:19.123079+0000","last_peered":"2024-04-25T10:04:19.123079+0000","last_clean":"2024-04-25T10:04:19.123079+0000","last_became_active":"2024-04-25T10:04:17.622135+0000","last_became_peered":"2024-04-25T10:04:17.622135+0000","last_unstale":"2024-04-25T10:04:19.123079+0000","last_undegraded":"2024-04-25T10:04:19.123079+0000","last_fullsized":"2024-04-25T10:04:19.123079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:17.510586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589819+0000","last_change":"2024-04-25T10:04:16.596941+0000","last_active":"2024-04-25T10:04:18.589819+0000","last_peered":"2024-04-25T10:04:18.589819+0000","last_clean":"2024-04-25T10:04:18.589819+0000","last_became_active":"2024-04-25T10:04:16.596643+0000","last_became_peered":"2024-04-25T10:04:16.596643+0000","last_unstale":"2024-04-25T10:04:18.589819+0000","last_undegraded":"2024-04-25T10:04:18.589819+0000","last_fullsized":"2024-04-25T10:04:18.589819+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:45.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589183+0000","last_change":"2024-04-25T10:04:17.599010+0000","last_active":"2024-04-25T10:04:18.589183+0000","last_peered":"2024-04-25T10:04:18.589183+0000","last_clean":"2024-04-25T10:04:18.589183+0000","last_became_active":"2024-04-25T10:04:17.598337+0000","last_became_peered":"2024-04-25T10:04:17.598337+0000","last_unstale":"2024-04-25T10:04:18.589183+0000","last_undegraded":"2024-04-25T10:04:18.589183+0000","last_fullsized":"2024-04-25T10:04:18.589183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:06.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625484+0000","last_change":"2024-04-25T10:04:16.586256+0000","last_active":"2024-04-25T10:04:19.625484+0000","last_peered":"2024-04-25T10:04:19.625484+0000","last_clean":"2024-04-25T10:04:19.625484+0000","last_became_active":"2024-04-25T10:04:16.585976+0000","last_became_peered":"2024-04-25T10:04:16.585976+0000","last_unstale":"2024-04-25T10:04:19.625484+0000","last_undegraded":"2024-04-25T10:04:19.625484+0000","last_fullsized":"2024-04-25T10:04:19.625484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:24.143592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591920+0000","last_change":"2024-04-25T10:04:17.616394+0000","last_active":"2024-04-25T10:04:19.591920+0000","last_peered":"2024-04-25T10:04:19.591920+0000","last_clean":"2024-04-25T10:04:19.591920+0000","last_became_active":"2024-04-25T10:04:17.612295+0000","last_became_peered":"2024-04-25T10:04:17.612295+0000","last_unstale":"2024-04-25T10:04:19.591920+0000","last_undegraded":"2024-04-25T10:04:19.591920+0000","last_fullsized":"2024-04-25T10:04:19.591920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.045726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591093+0000","last_change":"2024-04-25T10:04:16.591056+0000","last_active":"2024-04-25T10:04:19.591093+0000","last_peered":"2024-04-25T10:04:19.591093+0000","last_clean":"2024-04-25T10:04:19.591093+0000","last_became_active":"2024-04-25T10:04:16.590340+0000","last_became_peered":"2024-04-25T10:04:16.590340+0000","last_unstale":"2024-04-25T10:04:19.591093+0000","last_undegraded":"2024-04-25T10:04:19.591093+0000","last_fullsized":"2024-04-25T10:04:19.591093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:05.963386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590297+0000","last_change":"2024-04-25T10:04:17.609972+0000","last_active":"2024-04-25T10:04:18.590297+0000","last_peered":"2024-04-25T10:04:18.590297+0000","last_clean":"2024-04-25T10:04:18.590297+0000","last_became_active":"2024-04-25T10:04:17.609611+0000","last_became_peered":"2024-04-25T10:04:17.609611+0000","last_unstale":"2024-04-25T10:04:18.590297+0000","last_undegraded":"2024-04-25T10:04:18.590297+0000","last_fullsized":"2024-04-25T10:04:18.590297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:00.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624024+0000","last_change":"2024-04-25T10:04:16.594633+0000","last_active":"2024-04-25T10:04:19.624024+0000","last_peered":"2024-04-25T10:04:19.624024+0000","last_clean":"2024-04-25T10:04:19.624024+0000","last_became_active":"2024-04-25T10:04:16.594369+0000","last_became_peered":"2024-04-25T10:04:16.594369+0000","last_unstale":"2024-04-25T10:04:19.624024+0000","last_undegraded":"2024-04-25T10:04:19.624024+0000","last_fullsized":"2024-04-25T10:04:19.624024+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:30.644120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123349+0000","last_change":"2024-04-25T10:04:17.616522+0000","last_active":"2024-04-25T10:04:19.123349+0000","last_peered":"2024-04-25T10:04:19.123349+0000","last_clean":"2024-04-25T10:04:19.123349+0000","last_became_active":"2024-04-25T10:04:17.615685+0000","last_became_peered":"2024-04-25T10:04:17.615685+0000","last_unstale":"2024-04-25T10:04:19.123349+0000","last_undegraded":"2024-04-25T10:04:19.123349+0000","last_fullsized":"2024-04-25T10:04:19.123349+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589570+0000","last_change":"2024-04-25T10:04:16.591007+0000","last_active":"2024-04-25T10:04:18.589570+0000","last_peered":"2024-04-25T10:04:18.589570+0000","last_clean":"2024-04-25T10:04:18.589570+0000","last_became_active":"2024-04-25T10:04:16.590297+0000","last_became_peered":"2024-04-25T10:04:16.590297+0000","last_unstale":"2024-04-25T10:04:18.589570+0000","last_undegraded":"2024-04-25T10:04:18.589570+0000","last_fullsized":"2024-04-25T10:04:18.589570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:09.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589628+0000","last_change":"2024-04-25T10:04:17.604847+0000","last_active":"2024-04-25T10:04:18.589628+0000","last_peered":"2024-04-25T10:04:18.589628+0000","last_clean":"2024-04-25T10:04:18.589628+0000","last_became_active":"2024-04-25T10:04:17.604566+0000","last_became_peered":"2024-04-25T10:04:17.604566+0000","last_unstale":"2024-04-25T10:04:18.589628+0000","last_undegraded":"2024-04-25T10:04:18.589628+0000","last_fullsized":"2024-04-25T10:04:18.589628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:47.918943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595323+0000","last_change":"2024-04-25T10:04:16.600459+0000","last_active":"2024-04-25T10:04:19.595323+0000","last_peered":"2024-04-25T10:04:19.595323+0000","last_clean":"2024-04-25T10:04:19.595323+0000","last_became_active":"2024-04-25T10:04:16.599758+0000","last_became_peered":"2024-04-25T10:04:16.599758+0000","last_unstale":"2024-04-25T10:04:19.595323+0000","last_undegraded":"2024-04-25T10:04:19.595323+0000","last_fullsized":"2024-04-25T10:04:19.595323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:36.143924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625532+0000","last_change":"2024-04-25T10:04:17.623061+0000","last_active":"2024-04-25T10:04:19.625532+0000","last_peered":"2024-04-25T10:04:19.625532+0000","last_clean":"2024-04-25T10:04:19.625532+0000","last_became_active":"2024-04-25T10:04:17.622778+0000","last_became_peered":"2024-04-25T10:04:17.622778+0000","last_unstale":"2024-04-25T10:04:19.625532+0000","last_undegraded":"2024-04-25T10:04:19.625532+0000","last_fullsized":"2024-04-25T10:04:19.625532+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.347166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591434+0000","last_change":"2024-04-25T10:04:16.587720+0000","last_active":"2024-04-25T10:04:19.591434+0000","last_peered":"2024-04-25T10:04:19.591434+0000","last_clean":"2024-04-25T10:04:19.591434+0000","last_became_active":"2024-04-25T10:04:16.587136+0000","last_became_peered":"2024-04-25T10:04:16.587136+0000","last_unstale":"2024-04-25T10:04:19.591434+0000","last_undegraded":"2024-04-25T10:04:19.591434+0000","last_fullsized":"2024-04-25T10:04:19.591434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:47.662783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591838+0000","last_change":"2024-04-25T10:04:17.608003+0000","last_active":"2024-04-25T10:04:19.591838+0000","last_peered":"2024-04-25T10:04:19.591838+0000","last_clean":"2024-04-25T10:04:19.591838+0000","last_became_active":"2024-04-25T10:04:17.604950+0000","last_became_peered":"2024-04-25T10:04:17.604950+0000","last_unstale":"2024-04-25T10:04:19.591838+0000","last_undegraded":"2024-04-25T10:04:19.591838+0000","last_fullsized":"2024-04-25T10:04:19.591838+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:17.227912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625586+0000","last_change":"2024-04-25T10:04:16.586300+0000","last_active":"2024-04-25T10:04:19.625586+0000","last_peered":"2024-04-25T10:04:19.625586+0000","last_clean":"2024-04-25T10:04:19.625586+0000","last_became_active":"2024-04-25T10:04:16.586044+0000","last_became_peered":"2024-04-25T10:04:16.586044+0000","last_unstale":"2024-04-25T10:04:19.625586+0000","last_undegraded":"2024-04-25T10:04:19.625586+0000","last_fullsized":"2024-04-25T10:04:19.625586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:29.583189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123122+0000","last_change":"2024-04-25T10:04:17.644856+0000","last_active":"2024-04-25T10:04:19.123122+0000","last_peered":"2024-04-25T10:04:19.123122+0000","last_clean":"2024-04-25T10:04:19.123122+0000","last_became_active":"2024-04-25T10:04:17.644173+0000","last_became_peered":"2024-04-25T10:04:17.644173+0000","last_unstale":"2024-04-25T10:04:19.123122+0000","last_undegraded":"2024-04-25T10:04:19.123122+0000","last_fullsized":"2024-04-25T10:04:19.123122+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:44.107003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625399+0000","last_change":"2024-04-25T10:04:16.574522+0000","last_active":"2024-04-25T10:04:19.625399+0000","last_peered":"2024-04-25T10:04:19.625399+0000","last_clean":"2024-04-25T10:04:19.625399+0000","last_became_active":"2024-04-25T10:04:16.573325+0000","last_became_peered":"2024-04-25T10:04:16.573325+0000","last_unstale":"2024-04-25T10:04:19.625399+0000","last_undegraded":"2024-04-25T10:04:19.625399+0000","last_fullsized":"2024-04-25T10:04:19.625399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:57.746441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591715+0000","last_change":"2024-04-25T10:04:17.605043+0000","last_active":"2024-04-25T10:04:19.591715+0000","last_peered":"2024-04-25T10:04:19.591715+0000","last_clean":"2024-04-25T10:04:19.591715+0000","last_became_active":"2024-04-25T10:04:17.604782+0000","last_became_peered":"2024-04-25T10:04:17.604782+0000","last_unstale":"2024-04-25T10:04:19.591715+0000","last_undegraded":"2024-04-25T10:04:19.591715+0000","last_fullsized":"2024-04-25T10:04:19.591715+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:30.448502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.586800+0000","last_change":"2024-04-25T10:04:16.588417+0000","last_active":"2024-04-25T10:04:18.586800+0000","last_peered":"2024-04-25T10:04:18.586800+0000","last_clean":"2024-04-25T10:04:18.586800+0000","last_became_active":"2024-04-25T10:04:16.587702+0000","last_became_peered":"2024-04-25T10:04:16.587702+0000","last_unstale":"2024-04-25T10:04:18.586800+0000","last_undegraded":"2024-04-25T10:04:18.586800+0000","last_fullsized":"2024-04-25T10:04:18.586800+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:49.138416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626366+0000","last_change":"2024-04-25T10:04:17.622566+0000","last_active":"2024-04-25T10:04:19.626366+0000","last_peered":"2024-04-25T10:04:19.626366+0000","last_clean":"2024-04-25T10:04:19.626366+0000","last_became_active":"2024-04-25T10:04:17.621738+0000","last_became_peered":"2024-04-25T10:04:17.621738+0000","last_unstale":"2024-04-25T10:04:19.626366+0000","last_undegraded":"2024-04-25T10:04:19.626366+0000","last_fullsized":"2024-04-25T10:04:19.626366+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.045726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123109+0000","last_change":"2024-04-25T10:04:16.594813+0000","last_active":"2024-04-25T10:04:19.123109+0000","last_peered":"2024-04-25T10:04:19.123109+0000","last_clean":"2024-04-25T10:04:19.123109+0000","last_became_active":"2024-04-25T10:04:16.593686+0000","last_became_peered":"2024-04-25T10:04:16.593686+0000","last_unstale":"2024-04-25T10:04:19.123109+0000","last_undegraded":"2024-04-25T10:04:19.123109+0000","last_fullsized":"2024-04-25T10:04:19.123109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:29.583189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587324+0000","last_change":"2024-04-25T10:04:17.619694+0000","last_active":"2024-04-25T10:04:18.587324+0000","last_peered":"2024-04-25T10:04:18.587324+0000","last_clean":"2024-04-25T10:04:18.587324+0000","last_became_active":"2024-04-25T10:04:17.618888+0000","last_became_peered":"2024-04-25T10:04:17.618888+0000","last_unstale":"2024-04-25T10:04:18.587324+0000","last_undegraded":"2024-04-25T10:04:18.587324+0000","last_fullsized":"2024-04-25T10:04:18.587324+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:42.777647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594830+0000","last_change":"2024-04-25T10:04:16.607364+0000","last_active":"2024-04-25T10:04:19.594830+0000","last_peered":"2024-04-25T10:04:19.594830+0000","last_clean":"2024-04-25T10:04:19.594830+0000","last_became_active":"2024-04-25T10:04:16.607155+0000","last_became_peered":"2024-04-25T10:04:16.607155+0000","last_unstale":"2024-04-25T10:04:19.594830+0000","last_undegraded":"2024-04-25T10:04:19.594830+0000","last_fullsized":"2024-04-25T10:04:19.594830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:29.022281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594355+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:04:19.594355+0000","last_peered":"2024-04-25T10:04:19.594355+0000","last_clean":"2024-04-25T10:04:19.594355+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:04:19.594355+0000","last_undegraded":"2024-04-25T10:04:19.594355+0000","last_fullsized":"2024-04-25T10:04:19.594355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":865637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27616,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344224,"statfs":{"total":96636764160,"available":96608485376,"internally_reserved":0,"allocated":606208,"data_stored":268453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":622592,"data_stored":275727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31792,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340048,"statfs":{"total":96636764160,"available":96604209152,"internally_reserved":0,"allocated":622592,"data_stored":270087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":618496,"data_stored":270065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1208320,"data_stored":858877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:23.557 INFO:tasks.ceph.mds.a.smithi082.stderr:2024-04-25T10:04:23.555+0000 7fa41c76d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:04:25.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:04:25.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.131+0000 7f3e1fc53640 1 Processor -- start 2024-04-25T10:04:25.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.131+0000 7f3e1fc53640 1 -- start start 2024-04-25T10:04:25.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1fc53640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e18144ca0 0x7f3e1806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1fc53640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e180ccc70 con 0x7f3e1806ec60 2024-04-25T10:04:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1fc53640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e1806dc40 con 0x7f3e180ce510 2024-04-25T10:04:25.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1fc53640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180cc2b0 con 0x7f3e18144ca0 2024-04-25T10:04:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1e1f0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3e1806ec60 0x7f3e18142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42874/0 (socket says 172.21.15.82:42874) 2024-04-25T10:04:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1e1f0640 1 -- 172.21.15.82:0/3490869716 learned_addr learned my addr 172.21.15.82:0/3490869716 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/3490869716 >> v2:172.21.15.196:3300/0 conn(0x7f3e18144ca0 0x7f3e1806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176328145 0 0) 0x7f3e180ccc70 con 0x7f3e1806ec60 2024-04-25T10:04:25.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e00003530 con 0x7f3e1806ec60 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3915882342 0 0) 0x7f3e00003530 con 0x7f3e1806ec60 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 >> v2:172.21.15.196:3300/0 conn(0x7f3e18144ca0 msgr2=0x7f3e1806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 --2- 172.21.15.82:0/3490869716 >> v2:172.21.15.196:3300/0 conn(0x7f3e18144ca0 0x7f3e1806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 >> v1:172.21.15.196:6789/0 conn(0x7f3e180ce510 legacy=0x7f3e180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1806df20 con 0x7f3e1806ec60 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3e10001a00 con 0x7f3e1806ec60 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 >> v1:172.21.15.82:6789/0 conn(0x7f3e1806ec60 legacy=0x7f3e18142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e00003a30 0x7f3e00003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 0x7f3e00004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e000051d0 0x7f3e0000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180cc2b0 con 0x7f3e00004810 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0000c0e0 con 0x7f3e00003a30 2024-04-25T10:04:25.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0000c2e0 con 0x7f3e000051d0 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 0x7f3e00004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d9ef640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e00003a30 0x7f3e00003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1e1f0640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e000051d0 0x7f3e0000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e000051d0 msgr2=0x7f3e0000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e000051d0 0x7f3e0000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e00003a30 msgr2=0x7f3e00003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e00003a30 0x7f3e00003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/3490869716 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e00003530 con 0x7f3e00004810 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1e1f0640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e000051d0 0x7f3e0000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.135+0000 7f3e1e1f0640 1 -- 172.21.15.82:0/3490869716 reap_dead start 2024-04-25T10:04:25.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 0x7f3e00004c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ab0 tx=0x7f3e0c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807de59510e1eb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b2070 con 0x7f3e00004810 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0a56d0 con 0x7f3e00004810 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1c9ed640 1 -- 172.21.15.82:0/3490869716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0096b0 con 0x7f3e00004810 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 msgr2=0x7f3e00004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 0x7f3e00004c90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ab0 tx=0x7f3e0c01ca50 comp rx=0 tx=0).stop 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- 172.21.15.82:0/3490869716 shutdown_connections 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/3490869716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e00004810 0x7f3e00004c90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- 172.21.15.82:0/3490869716 >> 172.21.15.82:0/3490869716 conn(0x7f3e180b11a0 msgr2=0x7f3e180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- 172.21.15.82:0/3490869716 shutdown_connections 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- 172.21.15.82:0/3490869716 wait complete. 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 Processor -- start 2024-04-25T10:04:25.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- start start 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e00004810 0x7f3e18150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e18151240 0x7f3e180abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e181451f0 con 0x7f3e1814f780 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1806e290 con 0x7f3e18151240 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.139+0000 7f3e1fc53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18145370 con 0x7f3e00004810 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36368/0 (socket says 172.21.15.82:36368) 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/1348737405 learned_addr learned my addr 172.21.15.82:0/1348737405 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1e1f0640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e18151240 0x7f3e180abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d9ef640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e00004810 0x7f3e18150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e00004810 msgr2=0x7f3e18150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e00004810 0x7f3e18150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e18151240 msgr2=0x7f3e180abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e18151240 0x7f3e180abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 -- 172.21.15.82:0/1348737405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e1814fc60 con 0x7f3e1814f780 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d1ee640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ab0 tx=0x7f3e0c0197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0bd070 con 0x7f3e1814f780 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0ad640 con 0x7f3e1814f780 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1814ff40 con 0x7f3e1814f780 2024-04-25T10:04:25.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0b2040 con 0x7f3e1814f780 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e18150500 con 0x7f3e1814f780 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1804efa0 con 0x7f3e1814f780 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e0c0c0070 con 0x7f3e1814f780 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 0x7f3e04069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e0c11eef0 con 0x7f3e1814f780 2024-04-25T10:04:25.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d9ef640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 0x7f3e04069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.143+0000 7f3e1d9ef640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 0x7f3e04069450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e08000c10 tx=0x7f3e08009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.147+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e0c016760 con 0x7f3e1814f780 2024-04-25T10:04:25.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.303+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e180a8f30 con 0x7f3e04066f90 2024-04-25T10:04:25.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.311+0000 7f3e167fc640 1 -- 172.21.15.82:0/1348737405 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251845 (secure 0 0 0) 0x7f3e180a8f30 con 0x7f3e04066f90 2024-04-25T10:04:25.316 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:25.318 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:04:25.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 msgr2=0x7f3e04069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 0x7f3e04069450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3e08000c10 tx=0x7f3e08009040 comp rx=0 tx=0).stop 2024-04-25T10:04:25.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 msgr2=0x7f3e180ab7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c009ab0 tx=0x7f3e0c0197f0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 shutdown_connections 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e04066f90 0x7f3e04069450 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e18151240 0x7f3e180abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e1814f780 0x7f3e180ab7f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 --2- 172.21.15.82:0/1348737405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e00004810 0x7f3e18150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 >> 172.21.15.82:0/1348737405 conn(0x7f3e180b11a0 msgr2=0x7f3e180af0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 shutdown_connections 2024-04-25T10:04:25.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.315+0000 7f3e1fc53640 1 -- 172.21.15.82:0/1348737405 wait complete. 2024-04-25T10:04:25.333 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-25T10:04:25.302602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238892,"kb_used_data":6056,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735828,"statfs":{"total":773094113280,"available":772849487872,"internally_reserved":0,"allocated":6201344,"data_stored":3386716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002365"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590168+0000","last_change":"2024-04-25T10:04:17.620459+0000","last_active":"2024-04-25T10:04:18.590168+0000","last_peered":"2024-04-25T10:04:18.590168+0000","last_clean":"2024-04-25T10:04:18.590168+0000","last_became_active":"2024-04-25T10:04:17.620167+0000","last_became_peered":"2024-04-25T10:04:17.620167+0000","last_unstale":"2024-04-25T10:04:18.590168+0000","last_undegraded":"2024-04-25T10:04:18.590168+0000","last_fullsized":"2024-04-25T10:04:18.590168+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:54.672705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595314+0000","last_change":"2024-04-25T10:04:17.639196+0000","last_active":"2024-04-25T10:04:19.595314+0000","last_peered":"2024-04-25T10:04:19.595314+0000","last_clean":"2024-04-25T10:04:19.595314+0000","last_became_active":"2024-04-25T10:04:17.634300+0000","last_became_peered":"2024-04-25T10:04:17.634300+0000","last_unstale":"2024-04-25T10:04:19.595314+0000","last_undegraded":"2024-04-25T10:04:19.595314+0000","last_fullsized":"2024-04-25T10:04:19.595314+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591873+0000","last_change":"2024-04-25T10:04:17.617299+0000","last_active":"2024-04-25T10:04:19.591873+0000","last_peered":"2024-04-25T10:04:19.591873+0000","last_clean":"2024-04-25T10:04:19.591873+0000","last_became_active":"2024-04-25T10:04:17.617046+0000","last_became_peered":"2024-04-25T10:04:17.617046+0000","last_unstale":"2024-04-25T10:04:19.591873+0000","last_undegraded":"2024-04-25T10:04:19.591873+0000","last_fullsized":"2024-04-25T10:04:19.591873+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:44.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595008+0000","last_change":"2024-04-25T10:04:17.637718+0000","last_active":"2024-04-25T10:04:19.595008+0000","last_peered":"2024-04-25T10:04:19.595008+0000","last_clean":"2024-04-25T10:04:19.595008+0000","last_became_active":"2024-04-25T10:04:17.634066+0000","last_became_peered":"2024-04-25T10:04:17.634066+0000","last_unstale":"2024-04-25T10:04:19.595008+0000","last_undegraded":"2024-04-25T10:04:19.595008+0000","last_fullsized":"2024-04-25T10:04:19.595008+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:40.485737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587527+0000","last_change":"2024-04-25T10:04:17.640053+0000","last_active":"2024-04-25T10:04:18.587527+0000","last_peered":"2024-04-25T10:04:18.587527+0000","last_clean":"2024-04-25T10:04:18.587527+0000","last_became_active":"2024-04-25T10:04:17.639295+0000","last_became_peered":"2024-04-25T10:04:17.639295+0000","last_unstale":"2024-04-25T10:04:18.587527+0000","last_undegraded":"2024-04-25T10:04:18.587527+0000","last_fullsized":"2024-04-25T10:04:18.587527+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:42.777647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589381+0000","last_change":"2024-04-25T10:04:17.641548+0000","last_active":"2024-04-25T10:04:18.589381+0000","last_peered":"2024-04-25T10:04:18.589381+0000","last_clean":"2024-04-25T10:04:18.589381+0000","last_became_active":"2024-04-25T10:04:17.641247+0000","last_became_peered":"2024-04-25T10:04:17.641247+0000","last_unstale":"2024-04-25T10:04:18.589381+0000","last_undegraded":"2024-04-25T10:04:18.589381+0000","last_fullsized":"2024-04-25T10:04:18.589381+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:58.756738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590700+0000","last_change":"2024-04-25T10:04:17.611089+0000","last_active":"2024-04-25T10:04:19.590700+0000","last_peered":"2024-04-25T10:04:19.590700+0000","last_clean":"2024-04-25T10:04:19.590700+0000","last_became_active":"2024-04-25T10:04:17.610807+0000","last_became_peered":"2024-04-25T10:04:17.610807+0000","last_unstale":"2024-04-25T10:04:19.590700+0000","last_undegraded":"2024-04-25T10:04:19.590700+0000","last_fullsized":"2024-04-25T10:04:19.590700+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625698+0000","last_change":"2024-04-25T10:04:17.613096+0000","last_active":"2024-04-25T10:04:19.625698+0000","last_peered":"2024-04-25T10:04:19.625698+0000","last_clean":"2024-04-25T10:04:19.625698+0000","last_became_active":"2024-04-25T10:04:17.612835+0000","last_became_peered":"2024-04-25T10:04:17.612835+0000","last_unstale":"2024-04-25T10:04:19.625698+0000","last_undegraded":"2024-04-25T10:04:19.625698+0000","last_fullsized":"2024-04-25T10:04:19.625698+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:45.110644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626244+0000","last_change":"2024-04-25T10:04:17.623258+0000","last_active":"2024-04-25T10:04:19.626244+0000","last_peered":"2024-04-25T10:04:19.626244+0000","last_clean":"2024-04-25T10:04:19.626244+0000","last_became_active":"2024-04-25T10:04:17.622796+0000","last_became_peered":"2024-04-25T10:04:17.622796+0000","last_unstale":"2024-04-25T10:04:19.626244+0000","last_undegraded":"2024-04-25T10:04:19.626244+0000","last_fullsized":"2024-04-25T10:04:19.626244+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:17.227912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590063+0000","last_change":"2024-04-25T10:04:17.625940+0000","last_active":"2024-04-25T10:04:18.590063+0000","last_peered":"2024-04-25T10:04:18.590063+0000","last_clean":"2024-04-25T10:04:18.590063+0000","last_became_active":"2024-04-25T10:04:17.625665+0000","last_became_peered":"2024-04-25T10:04:17.625665+0000","last_unstale":"2024-04-25T10:04:18.590063+0000","last_undegraded":"2024-04-25T10:04:18.590063+0000","last_fullsized":"2024-04-25T10:04:18.590063+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:13.855322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624850+0000","last_change":"2024-04-25T10:04:17.644952+0000","last_active":"2024-04-25T10:04:19.624850+0000","last_peered":"2024-04-25T10:04:19.624850+0000","last_clean":"2024-04-25T10:04:19.624850+0000","last_became_active":"2024-04-25T10:04:17.644550+0000","last_became_peered":"2024-04-25T10:04:17.644550+0000","last_unstale":"2024-04-25T10:04:19.624850+0000","last_undegraded":"2024-04-25T10:04:19.624850+0000","last_fullsized":"2024-04-25T10:04:19.624850+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:12.443879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626031+0000","last_change":"2024-04-25T10:04:17.599974+0000","last_active":"2024-04-25T10:04:19.626031+0000","last_peered":"2024-04-25T10:04:19.626031+0000","last_clean":"2024-04-25T10:04:19.626031+0000","last_became_active":"2024-04-25T10:04:17.599650+0000","last_became_peered":"2024-04-25T10:04:17.599650+0000","last_unstale":"2024-04-25T10:04:19.626031+0000","last_undegraded":"2024-04-25T10:04:19.626031+0000","last_fullsized":"2024-04-25T10:04:19.626031+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:46.568980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589170+0000","last_change":"2024-04-25T10:04:17.623559+0000","last_active":"2024-04-25T10:04:18.589170+0000","last_peered":"2024-04-25T10:04:18.589170+0000","last_clean":"2024-04-25T10:04:18.589170+0000","last_became_active":"2024-04-25T10:04:17.623066+0000","last_became_peered":"2024-04-25T10:04:17.623066+0000","last_unstale":"2024-04-25T10:04:18.589170+0000","last_undegraded":"2024-04-25T10:04:18.589170+0000","last_fullsized":"2024-04-25T10:04:18.589170+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:50.351734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591036+0000","last_change":"2024-04-25T10:04:17.607521+0000","last_active":"2024-04-25T10:04:19.591036+0000","last_peered":"2024-04-25T10:04:19.591036+0000","last_clean":"2024-04-25T10:04:19.591036+0000","last_became_active":"2024-04-25T10:04:17.607204+0000","last_became_peered":"2024-04-25T10:04:17.607204+0000","last_unstale":"2024-04-25T10:04:19.591036+0000","last_undegraded":"2024-04-25T10:04:19.591036+0000","last_fullsized":"2024-04-25T10:04:19.591036+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:17.510586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594957+0000","last_change":"2024-04-25T10:04:17.638183+0000","last_active":"2024-04-25T10:04:19.594957+0000","last_peered":"2024-04-25T10:04:19.594957+0000","last_clean":"2024-04-25T10:04:19.594957+0000","last_became_active":"2024-04-25T10:04:17.637625+0000","last_became_peered":"2024-04-25T10:04:17.637625+0000","last_unstale":"2024-04-25T10:04:19.594957+0000","last_undegraded":"2024-04-25T10:04:19.594957+0000","last_fullsized":"2024-04-25T10:04:19.594957+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:13.352440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589194+0000","last_change":"2024-04-25T10:04:17.623209+0000","last_active":"2024-04-25T10:04:18.589194+0000","last_peered":"2024-04-25T10:04:18.589194+0000","last_clean":"2024-04-25T10:04:18.589194+0000","last_became_active":"2024-04-25T10:04:17.622919+0000","last_became_peered":"2024-04-25T10:04:17.622919+0000","last_unstale":"2024-04-25T10:04:18.589194+0000","last_undegraded":"2024-04-25T10:04:18.589194+0000","last_fullsized":"2024-04-25T10:04:18.589194+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:13.323967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591521+0000","last_change":"2024-04-25T10:04:17.625508+0000","last_active":"2024-04-25T10:04:19.591521+0000","last_peered":"2024-04-25T10:04:19.591521+0000","last_clean":"2024-04-25T10:04:19.591521+0000","last_became_active":"2024-04-25T10:04:17.623814+0000","last_became_peered":"2024-04-25T10:04:17.623814+0000","last_unstale":"2024-04-25T10:04:19.591521+0000","last_undegraded":"2024-04-25T10:04:19.591521+0000","last_fullsized":"2024-04-25T10:04:19.591521+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595438+0000","last_change":"2024-04-25T10:04:17.642432+0000","last_active":"2024-04-25T10:04:19.595438+0000","last_peered":"2024-04-25T10:04:19.595438+0000","last_clean":"2024-04-25T10:04:19.595438+0000","last_became_active":"2024-04-25T10:04:17.639977+0000","last_became_peered":"2024-04-25T10:04:17.639977+0000","last_unstale":"2024-04-25T10:04:19.595438+0000","last_undegraded":"2024-04-25T10:04:19.595438+0000","last_fullsized":"2024-04-25T10:04:19.595438+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591694+0000","last_change":"2024-04-25T10:04:17.612339+0000","last_active":"2024-04-25T10:04:19.591694+0000","last_peered":"2024-04-25T10:04:19.591694+0000","last_clean":"2024-04-25T10:04:19.591694+0000","last_became_active":"2024-04-25T10:04:17.611228+0000","last_became_peered":"2024-04-25T10:04:17.611228+0000","last_unstale":"2024-04-25T10:04:19.591694+0000","last_undegraded":"2024-04-25T10:04:19.591694+0000","last_fullsized":"2024-04-25T10:04:19.591694+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:45.110644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588164+0000","last_change":"2024-04-25T10:04:17.603290+0000","last_active":"2024-04-25T10:04:18.588164+0000","last_peered":"2024-04-25T10:04:18.588164+0000","last_clean":"2024-04-25T10:04:18.588164+0000","last_became_active":"2024-04-25T10:04:17.602462+0000","last_became_peered":"2024-04-25T10:04:17.602462+0000","last_unstale":"2024-04-25T10:04:18.588164+0000","last_undegraded":"2024-04-25T10:04:18.588164+0000","last_fullsized":"2024-04-25T10:04:18.588164+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:30.032578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587673+0000","last_change":"2024-04-25T10:04:17.633966+0000","last_active":"2024-04-25T10:04:18.587673+0000","last_peered":"2024-04-25T10:04:18.587673+0000","last_clean":"2024-04-25T10:04:18.587673+0000","last_became_active":"2024-04-25T10:04:17.633238+0000","last_became_peered":"2024-04-25T10:04:17.633238+0000","last_unstale":"2024-04-25T10:04:18.587673+0000","last_undegraded":"2024-04-25T10:04:18.587673+0000","last_fullsized":"2024-04-25T10:04:18.587673+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:30.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625240+0000","last_change":"2024-04-25T10:04:17.623203+0000","last_active":"2024-04-25T10:04:19.625240+0000","last_peered":"2024-04-25T10:04:19.625240+0000","last_clean":"2024-04-25T10:04:19.625240+0000","last_became_active":"2024-04-25T10:04:17.622883+0000","last_became_peered":"2024-04-25T10:04:17.622883+0000","last_unstale":"2024-04-25T10:04:19.625240+0000","last_undegraded":"2024-04-25T10:04:19.625240+0000","last_fullsized":"2024-04-25T10:04:19.625240+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.859768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624302+0000","last_change":"2024-04-25T10:04:17.634620+0000","last_active":"2024-04-25T10:04:19.624302+0000","last_peered":"2024-04-25T10:04:19.624302+0000","last_clean":"2024-04-25T10:04:19.624302+0000","last_became_active":"2024-04-25T10:04:17.634335+0000","last_became_peered":"2024-04-25T10:04:17.634335+0000","last_unstale":"2024-04-25T10:04:19.624302+0000","last_undegraded":"2024-04-25T10:04:19.624302+0000","last_fullsized":"2024-04-25T10:04:19.624302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:31.640607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590753+0000","last_change":"2024-04-25T10:04:17.611672+0000","last_active":"2024-04-25T10:04:19.590753+0000","last_peered":"2024-04-25T10:04:19.590753+0000","last_clean":"2024-04-25T10:04:19.590753+0000","last_became_active":"2024-04-25T10:04:17.610536+0000","last_became_peered":"2024-04-25T10:04:17.610536+0000","last_unstale":"2024-04-25T10:04:19.590753+0000","last_undegraded":"2024-04-25T10:04:19.590753+0000","last_fullsized":"2024-04-25T10:04:19.590753+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591825+0000","last_change":"2024-04-25T10:04:17.608814+0000","last_active":"2024-04-25T10:04:19.591825+0000","last_peered":"2024-04-25T10:04:19.591825+0000","last_clean":"2024-04-25T10:04:19.591825+0000","last_became_active":"2024-04-25T10:04:17.608548+0000","last_became_peered":"2024-04-25T10:04:17.608548+0000","last_unstale":"2024-04-25T10:04:19.591825+0000","last_undegraded":"2024-04-25T10:04:19.591825+0000","last_fullsized":"2024-04-25T10:04:19.591825+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:46.568980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626387+0000","last_change":"2024-04-25T10:04:17.623489+0000","last_active":"2024-04-25T10:04:19.626387+0000","last_peered":"2024-04-25T10:04:19.626387+0000","last_clean":"2024-04-25T10:04:19.626387+0000","last_became_active":"2024-04-25T10:04:17.623242+0000","last_became_peered":"2024-04-25T10:04:17.623242+0000","last_unstale":"2024-04-25T10:04:19.626387+0000","last_undegraded":"2024-04-25T10:04:19.626387+0000","last_fullsized":"2024-04-25T10:04:19.626387+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.826321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590139+0000","last_change":"2024-04-25T10:04:17.618382+0000","last_active":"2024-04-25T10:04:18.590139+0000","last_peered":"2024-04-25T10:04:18.590139+0000","last_clean":"2024-04-25T10:04:18.590139+0000","last_became_active":"2024-04-25T10:04:17.618118+0000","last_became_peered":"2024-04-25T10:04:17.618118+0000","last_unstale":"2024-04-25T10:04:18.590139+0000","last_undegraded":"2024-04-25T10:04:18.590139+0000","last_fullsized":"2024-04-25T10:04:18.590139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:25.153889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588252+0000","last_change":"2024-04-25T10:04:17.608112+0000","last_active":"2024-04-25T10:04:18.588252+0000","last_peered":"2024-04-25T10:04:18.588252+0000","last_clean":"2024-04-25T10:04:18.588252+0000","last_became_active":"2024-04-25T10:04:17.607335+0000","last_became_peered":"2024-04-25T10:04:17.607335+0000","last_unstale":"2024-04-25T10:04:18.588252+0000","last_undegraded":"2024-04-25T10:04:18.588252+0000","last_fullsized":"2024-04-25T10:04:18.588252+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:37.154221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591112+0000","last_change":"2024-04-25T10:04:17.616901+0000","last_active":"2024-04-25T10:04:19.591112+0000","last_peered":"2024-04-25T10:04:19.591112+0000","last_clean":"2024-04-25T10:04:19.591112+0000","last_became_active":"2024-04-25T10:04:17.616428+0000","last_became_peered":"2024-04-25T10:04:17.616428+0000","last_unstale":"2024-04-25T10:04:19.591112+0000","last_undegraded":"2024-04-25T10:04:19.591112+0000","last_fullsized":"2024-04-25T10:04:19.591112+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:44.107003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594905+0000","last_change":"2024-04-25T10:04:17.638860+0000","last_active":"2024-04-25T10:04:19.594905+0000","last_peered":"2024-04-25T10:04:19.594905+0000","last_clean":"2024-04-25T10:04:19.594905+0000","last_became_active":"2024-04-25T10:04:17.637531+0000","last_became_peered":"2024-04-25T10:04:17.637531+0000","last_unstale":"2024-04-25T10:04:19.594905+0000","last_undegraded":"2024-04-25T10:04:19.594905+0000","last_fullsized":"2024-04-25T10:04:19.594905+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.560792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591657+0000","last_change":"2024-04-25T10:04:17.608971+0000","last_active":"2024-04-25T10:04:19.591657+0000","last_peered":"2024-04-25T10:04:19.591657+0000","last_clean":"2024-04-25T10:04:19.591657+0000","last_became_active":"2024-04-25T10:04:17.607554+0000","last_became_peered":"2024-04-25T10:04:17.607554+0000","last_unstale":"2024-04-25T10:04:19.591657+0000","last_undegraded":"2024-04-25T10:04:19.591657+0000","last_fullsized":"2024-04-25T10:04:19.591657+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.347166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122817+0000","last_change":"2024-04-25T10:04:17.616813+0000","last_active":"2024-04-25T10:04:19.122817+0000","last_peered":"2024-04-25T10:04:19.122817+0000","last_clean":"2024-04-25T10:04:19.122817+0000","last_became_active":"2024-04-25T10:04:17.615954+0000","last_became_peered":"2024-04-25T10:04:17.615954+0000","last_unstale":"2024-04-25T10:04:19.122817+0000","last_undegraded":"2024-04-25T10:04:19.122817+0000","last_fullsized":"2024-04-25T10:04:19.122817+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:14.078617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594501+0000","last_change":"2024-04-25T10:04:17.638638+0000","last_active":"2024-04-25T10:04:19.594501+0000","last_peered":"2024-04-25T10:04:19.594501+0000","last_clean":"2024-04-25T10:04:19.594501+0000","last_became_active":"2024-04-25T10:04:17.638019+0000","last_became_peered":"2024-04-25T10:04:17.638019+0000","last_unstale":"2024-04-25T10:04:19.594501+0000","last_undegraded":"2024-04-25T10:04:19.594501+0000","last_fullsized":"2024-04-25T10:04:19.594501+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:40.238684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"51'4","reported_seq":18,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589405+0000","last_change":"2024-04-25T10:04:16.582562+0000","last_active":"2024-04-25T10:04:18.589405+0000","last_peered":"2024-04-25T10:04:18.589405+0000","last_clean":"2024-04-25T10:04:18.589405+0000","last_became_active":"2024-04-25T10:04:16.581694+0000","last_became_peered":"2024-04-25T10:04:16.581694+0000","last_unstale":"2024-04-25T10:04:18.589405+0000","last_undegraded":"2024-04-25T10:04:18.589405+0000","last_fullsized":"2024-04-25T10:04:18.589405+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:50.754619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122649+0000","last_change":"2024-04-25T10:04:17.638993+0000","last_active":"2024-04-25T10:04:19.122649+0000","last_peered":"2024-04-25T10:04:19.122649+0000","last_clean":"2024-04-25T10:04:19.122649+0000","last_became_active":"2024-04-25T10:04:17.638544+0000","last_became_peered":"2024-04-25T10:04:17.638544+0000","last_unstale":"2024-04-25T10:04:19.122649+0000","last_undegraded":"2024-04-25T10:04:19.122649+0000","last_fullsized":"2024-04-25T10:04:19.122649+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590208+0000","last_change":"2024-04-25T10:04:16.577803+0000","last_active":"2024-04-25T10:04:18.590208+0000","last_peered":"2024-04-25T10:04:18.590208+0000","last_clean":"2024-04-25T10:04:18.590208+0000","last_became_active":"2024-04-25T10:04:16.577566+0000","last_became_peered":"2024-04-25T10:04:16.577566+0000","last_unstale":"2024-04-25T10:04:18.590208+0000","last_undegraded":"2024-04-25T10:04:18.590208+0000","last_fullsized":"2024-04-25T10:04:18.590208+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:03.931920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587456+0000","last_change":"2024-04-25T10:04:17.633508+0000","last_active":"2024-04-25T10:04:18.587456+0000","last_peered":"2024-04-25T10:04:18.587456+0000","last_clean":"2024-04-25T10:04:18.587456+0000","last_became_active":"2024-04-25T10:04:17.633001+0000","last_became_peered":"2024-04-25T10:04:17.633001+0000","last_unstale":"2024-04-25T10:04:18.587456+0000","last_undegraded":"2024-04-25T10:04:18.587456+0000","last_fullsized":"2024-04-25T10:04:18.587456+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:48.673080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590225+0000","last_change":"2024-04-25T10:04:16.585813+0000","last_active":"2024-04-25T10:04:18.590225+0000","last_peered":"2024-04-25T10:04:18.590225+0000","last_clean":"2024-04-25T10:04:18.590225+0000","last_became_active":"2024-04-25T10:04:16.585428+0000","last_became_peered":"2024-04-25T10:04:16.585428+0000","last_unstale":"2024-04-25T10:04:18.590225+0000","last_undegraded":"2024-04-25T10:04:18.590225+0000","last_fullsized":"2024-04-25T10:04:18.590225+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:45.171318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625239+0000","last_change":"2024-04-25T10:04:17.620072+0000","last_active":"2024-04-25T10:04:19.625239+0000","last_peered":"2024-04-25T10:04:19.625239+0000","last_clean":"2024-04-25T10:04:19.625239+0000","last_became_active":"2024-04-25T10:04:17.619809+0000","last_became_peered":"2024-04-25T10:04:17.619809+0000","last_unstale":"2024-04-25T10:04:19.625239+0000","last_undegraded":"2024-04-25T10:04:19.625239+0000","last_fullsized":"2024-04-25T10:04:19.625239+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123278+0000","last_change":"2024-04-25T10:04:16.596433+0000","last_active":"2024-04-25T10:04:19.123278+0000","last_peered":"2024-04-25T10:04:19.123278+0000","last_clean":"2024-04-25T10:04:19.123278+0000","last_became_active":"2024-04-25T10:04:16.594317+0000","last_became_peered":"2024-04-25T10:04:16.594317+0000","last_unstale":"2024-04-25T10:04:19.123278+0000","last_undegraded":"2024-04-25T10:04:19.123278+0000","last_fullsized":"2024-04-25T10:04:19.123278+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:46.908646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588624+0000","last_change":"2024-04-25T10:04:17.601284+0000","last_active":"2024-04-25T10:04:18.588624+0000","last_peered":"2024-04-25T10:04:18.588624+0000","last_clean":"2024-04-25T10:04:18.588624+0000","last_became_active":"2024-04-25T10:04:17.599129+0000","last_became_peered":"2024-04-25T10:04:17.599129+0000","last_unstale":"2024-04-25T10:04:18.588624+0000","last_undegraded":"2024-04-25T10:04:18.588624+0000","last_fullsized":"2024-04-25T10:04:18.588624+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:30.875022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591205+0000","last_change":"2024-04-25T10:04:16.589640+0000","last_active":"2024-04-25T10:04:19.591205+0000","last_peered":"2024-04-25T10:04:19.591205+0000","last_clean":"2024-04-25T10:04:19.591205+0000","last_became_active":"2024-04-25T10:04:16.589365+0000","last_became_peered":"2024-04-25T10:04:16.589365+0000","last_unstale":"2024-04-25T10:04:19.591205+0000","last_undegraded":"2024-04-25T10:04:19.591205+0000","last_fullsized":"2024-04-25T10:04:19.591205+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:39.134229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595139+0000","last_change":"2024-04-25T10:04:17.643861+0000","last_active":"2024-04-25T10:04:19.595139+0000","last_peered":"2024-04-25T10:04:19.595139+0000","last_clean":"2024-04-25T10:04:19.595139+0000","last_became_active":"2024-04-25T10:04:17.642577+0000","last_became_peered":"2024-04-25T10:04:17.642577+0000","last_unstale":"2024-04-25T10:04:19.595139+0000","last_undegraded":"2024-04-25T10:04:19.595139+0000","last_fullsized":"2024-04-25T10:04:19.595139+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.769875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587605+0000","last_change":"2024-04-25T10:04:16.586646+0000","last_active":"2024-04-25T10:04:18.587605+0000","last_peered":"2024-04-25T10:04:18.587605+0000","last_clean":"2024-04-25T10:04:18.587605+0000","last_became_active":"2024-04-25T10:04:16.582735+0000","last_became_peered":"2024-04-25T10:04:16.582735+0000","last_unstale":"2024-04-25T10:04:18.587605+0000","last_undegraded":"2024-04-25T10:04:18.587605+0000","last_fullsized":"2024-04-25T10:04:18.587605+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:09.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626649+0000","last_change":"2024-04-25T10:04:17.620084+0000","last_active":"2024-04-25T10:04:19.626649+0000","last_peered":"2024-04-25T10:04:19.626649+0000","last_clean":"2024-04-25T10:04:19.626649+0000","last_became_active":"2024-04-25T10:04:17.619490+0000","last_became_peered":"2024-04-25T10:04:17.619490+0000","last_unstale":"2024-04-25T10:04:19.626649+0000","last_undegraded":"2024-04-25T10:04:19.626649+0000","last_fullsized":"2024-04-25T10:04:19.626649+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:35.185541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587498+0000","last_change":"2024-04-25T10:04:16.596633+0000","last_active":"2024-04-25T10:04:18.587498+0000","last_peered":"2024-04-25T10:04:18.587498+0000","last_clean":"2024-04-25T10:04:18.587498+0000","last_became_active":"2024-04-25T10:04:16.595945+0000","last_became_peered":"2024-04-25T10:04:16.595945+0000","last_unstale":"2024-04-25T10:04:18.587498+0000","last_undegraded":"2024-04-25T10:04:18.587498+0000","last_fullsized":"2024-04-25T10:04:18.587498+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:45.171318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595279+0000","last_change":"2024-04-25T10:04:17.638402+0000","last_active":"2024-04-25T10:04:19.595279+0000","last_peered":"2024-04-25T10:04:19.595279+0000","last_clean":"2024-04-25T10:04:19.595279+0000","last_became_active":"2024-04-25T10:04:17.637821+0000","last_became_peered":"2024-04-25T10:04:17.637821+0000","last_unstale":"2024-04-25T10:04:19.595279+0000","last_undegraded":"2024-04-25T10:04:19.595279+0000","last_fullsized":"2024-04-25T10:04:19.595279+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:47.884783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591445+0000","last_change":"2024-04-25T10:04:16.594413+0000","last_active":"2024-04-25T10:04:19.591445+0000","last_peered":"2024-04-25T10:04:19.591445+0000","last_clean":"2024-04-25T10:04:19.591445+0000","last_became_active":"2024-04-25T10:04:16.594157+0000","last_became_peered":"2024-04-25T10:04:16.594157+0000","last_unstale":"2024-04-25T10:04:19.591445+0000","last_undegraded":"2024-04-25T10:04:19.591445+0000","last_fullsized":"2024-04-25T10:04:19.591445+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:36.143924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587077+0000","last_change":"2024-04-25T10:04:17.602429+0000","last_active":"2024-04-25T10:04:18.587077+0000","last_peered":"2024-04-25T10:04:18.587077+0000","last_clean":"2024-04-25T10:04:18.587077+0000","last_became_active":"2024-04-25T10:04:17.601670+0000","last_became_peered":"2024-04-25T10:04:17.601670+0000","last_unstale":"2024-04-25T10:04:18.587077+0000","last_undegraded":"2024-04-25T10:04:18.587077+0000","last_fullsized":"2024-04-25T10:04:18.587077+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:54.048873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595367+0000","last_change":"2024-04-25T10:04:16.600592+0000","last_active":"2024-04-25T10:04:19.595367+0000","last_peered":"2024-04-25T10:04:19.595367+0000","last_clean":"2024-04-25T10:04:19.595367+0000","last_became_active":"2024-04-25T10:04:16.599063+0000","last_became_peered":"2024-04-25T10:04:16.599063+0000","last_unstale":"2024-04-25T10:04:19.595367+0000","last_undegraded":"2024-04-25T10:04:19.595367+0000","last_fullsized":"2024-04-25T10:04:19.595367+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:05.963386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.122591+0000","last_change":"2024-04-25T10:04:17.622606+0000","last_active":"2024-04-25T10:04:19.122591+0000","last_peered":"2024-04-25T10:04:19.122591+0000","last_clean":"2024-04-25T10:04:19.122591+0000","last_became_active":"2024-04-25T10:04:17.621889+0000","last_became_peered":"2024-04-25T10:04:17.621889+0000","last_unstale":"2024-04-25T10:04:19.122591+0000","last_undegraded":"2024-04-25T10:04:19.122591+0000","last_fullsized":"2024-04-25T10:04:19.122591+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:25.769875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588530+0000","last_change":"2024-04-25T10:04:16.591093+0000","last_active":"2024-04-25T10:04:18.588530+0000","last_peered":"2024-04-25T10:04:18.588530+0000","last_clean":"2024-04-25T10:04:18.588530+0000","last_became_active":"2024-04-25T10:04:16.590343+0000","last_became_peered":"2024-04-25T10:04:16.590343+0000","last_unstale":"2024-04-25T10:04:18.588530+0000","last_undegraded":"2024-04-25T10:04:18.588530+0000","last_fullsized":"2024-04-25T10:04:18.588530+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.077859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.590868+0000","last_change":"2024-04-25T10:04:17.622400+0000","last_active":"2024-04-25T10:04:19.590868+0000","last_peered":"2024-04-25T10:04:19.590868+0000","last_clean":"2024-04-25T10:04:19.590868+0000","last_became_active":"2024-04-25T10:04:17.621956+0000","last_became_peered":"2024-04-25T10:04:17.621956+0000","last_unstale":"2024-04-25T10:04:19.590868+0000","last_undegraded":"2024-04-25T10:04:19.590868+0000","last_fullsized":"2024-04-25T10:04:19.590868+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"51'3","reported_seq":17,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595322+0000","last_change":"2024-04-25T10:04:16.600334+0000","last_active":"2024-04-25T10:04:19.595322+0000","last_peered":"2024-04-25T10:04:19.595322+0000","last_clean":"2024-04-25T10:04:19.595322+0000","last_became_active":"2024-04-25T10:04:16.599624+0000","last_became_peered":"2024-04-25T10:04:16.599624+0000","last_unstale":"2024-04-25T10:04:19.595322+0000","last_undegraded":"2024-04-25T10:04:19.595322+0000","last_fullsized":"2024-04-25T10:04:19.595322+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:29.022281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587883+0000","last_change":"2024-04-25T10:04:17.595845+0000","last_active":"2024-04-25T10:04:18.587883+0000","last_peered":"2024-04-25T10:04:18.587883+0000","last_clean":"2024-04-25T10:04:18.587883+0000","last_became_active":"2024-04-25T10:04:17.595164+0000","last_became_peered":"2024-04-25T10:04:17.595164+0000","last_unstale":"2024-04-25T10:04:18.587883+0000","last_undegraded":"2024-04-25T10:04:18.587883+0000","last_fullsized":"2024-04-25T10:04:18.587883+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:48.673080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"51'6","reported_seq":20,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626253+0000","last_change":"2024-04-25T10:04:16.586798+0000","last_active":"2024-04-25T10:04:19.626253+0000","last_peered":"2024-04-25T10:04:19.626253+0000","last_clean":"2024-04-25T10:04:19.626253+0000","last_became_active":"2024-04-25T10:04:16.586585+0000","last_became_peered":"2024-04-25T10:04:16.586585+0000","last_unstale":"2024-04-25T10:04:19.626253+0000","last_undegraded":"2024-04-25T10:04:19.626253+0000","last_fullsized":"2024-04-25T10:04:19.626253+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:12.845025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123038+0000","last_change":"2024-04-25T10:04:17.645245+0000","last_active":"2024-04-25T10:04:19.123038+0000","last_peered":"2024-04-25T10:04:19.123038+0000","last_clean":"2024-04-25T10:04:19.123038+0000","last_became_active":"2024-04-25T10:04:17.643383+0000","last_became_peered":"2024-04-25T10:04:17.643383+0000","last_unstale":"2024-04-25T10:04:19.123038+0000","last_undegraded":"2024-04-25T10:04:19.123038+0000","last_fullsized":"2024-04-25T10:04:19.123038+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:19.231200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591583+0000","last_change":"2024-04-25T10:04:16.593571+0000","last_active":"2024-04-25T10:04:19.591583+0000","last_peered":"2024-04-25T10:04:19.591583+0000","last_clean":"2024-04-25T10:04:19.591583+0000","last_became_active":"2024-04-25T10:04:16.593207+0000","last_became_peered":"2024-04-25T10:04:16.593207+0000","last_unstale":"2024-04-25T10:04:19.591583+0000","last_undegraded":"2024-04-25T10:04:19.591583+0000","last_fullsized":"2024-04-25T10:04:19.591583+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:41.767350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.592126+0000","last_change":"2024-04-25T10:04:17.610938+0000","last_active":"2024-04-25T10:04:19.592126+0000","last_peered":"2024-04-25T10:04:19.592126+0000","last_clean":"2024-04-25T10:04:19.592126+0000","last_became_active":"2024-04-25T10:04:17.608774+0000","last_became_peered":"2024-04-25T10:04:17.608774+0000","last_unstale":"2024-04-25T10:04:19.592126+0000","last_undegraded":"2024-04-25T10:04:19.592126+0000","last_fullsized":"2024-04-25T10:04:19.592126+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:37.826321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587716+0000","last_change":"2024-04-25T10:04:16.585541+0000","last_active":"2024-04-25T10:04:18.587716+0000","last_peered":"2024-04-25T10:04:18.587716+0000","last_clean":"2024-04-25T10:04:18.587716+0000","last_became_active":"2024-04-25T10:04:16.584645+0000","last_became_peered":"2024-04-25T10:04:16.584645+0000","last_unstale":"2024-04-25T10:04:18.587716+0000","last_undegraded":"2024-04-25T10:04:18.587716+0000","last_fullsized":"2024-04-25T10:04:18.587716+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:49.138416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626269+0000","last_change":"2024-04-25T10:04:17.636525+0000","last_active":"2024-04-25T10:04:19.626269+0000","last_peered":"2024-04-25T10:04:19.626269+0000","last_clean":"2024-04-25T10:04:19.626269+0000","last_became_active":"2024-04-25T10:04:17.636280+0000","last_became_peered":"2024-04-25T10:04:17.636280+0000","last_unstale":"2024-04-25T10:04:19.626269+0000","last_undegraded":"2024-04-25T10:04:19.626269+0000","last_fullsized":"2024-04-25T10:04:19.626269+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:44.767691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594741+0000","last_change":"2024-04-25T10:04:16.606292+0000","last_active":"2024-04-25T10:04:19.594741+0000","last_peered":"2024-04-25T10:04:19.594741+0000","last_clean":"2024-04-25T10:04:19.594741+0000","last_became_active":"2024-04-25T10:04:16.605504+0000","last_became_peered":"2024-04-25T10:04:16.605504+0000","last_unstale":"2024-04-25T10:04:19.594741+0000","last_undegraded":"2024-04-25T10:04:19.594741+0000","last_fullsized":"2024-04-25T10:04:19.594741+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:49.341437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591598+0000","last_change":"2024-04-25T10:04:17.625236+0000","last_active":"2024-04-25T10:04:19.591598+0000","last_peered":"2024-04-25T10:04:19.591598+0000","last_clean":"2024-04-25T10:04:19.591598+0000","last_became_active":"2024-04-25T10:04:17.623409+0000","last_became_peered":"2024-04-25T10:04:17.623409+0000","last_unstale":"2024-04-25T10:04:19.591598+0000","last_undegraded":"2024-04-25T10:04:19.591598+0000","last_fullsized":"2024-04-25T10:04:19.591598+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:06.859768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"51'8","reported_seq":22,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626038+0000","last_change":"2024-04-25T10:04:16.589311+0000","last_active":"2024-04-25T10:04:19.626038+0000","last_peered":"2024-04-25T10:04:19.626038+0000","last_clean":"2024-04-25T10:04:19.626038+0000","last_became_active":"2024-04-25T10:04:16.588803+0000","last_became_peered":"2024-04-25T10:04:16.588803+0000","last_unstale":"2024-04-25T10:04:19.626038+0000","last_undegraded":"2024-04-25T10:04:19.626038+0000","last_fullsized":"2024-04-25T10:04:19.626038+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:53.662409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625948+0000","last_change":"2024-04-25T10:04:17.620686+0000","last_active":"2024-04-25T10:04:19.625948+0000","last_peered":"2024-04-25T10:04:19.625948+0000","last_clean":"2024-04-25T10:04:19.625948+0000","last_became_active":"2024-04-25T10:04:17.620323+0000","last_became_peered":"2024-04-25T10:04:17.620323+0000","last_unstale":"2024-04-25T10:04:19.625948+0000","last_undegraded":"2024-04-25T10:04:19.625948+0000","last_fullsized":"2024-04-25T10:04:19.625948+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:30.448502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591958+0000","last_change":"2024-04-25T10:04:16.590540+0000","last_active":"2024-04-25T10:04:19.591958+0000","last_peered":"2024-04-25T10:04:19.591958+0000","last_clean":"2024-04-25T10:04:19.591958+0000","last_became_active":"2024-04-25T10:04:16.589517+0000","last_became_peered":"2024-04-25T10:04:16.589517+0000","last_unstale":"2024-04-25T10:04:19.591958+0000","last_undegraded":"2024-04-25T10:04:19.591958+0000","last_fullsized":"2024-04-25T10:04:19.591958+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:29.864725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589193+0000","last_change":"2024-04-25T10:04:17.617502+0000","last_active":"2024-04-25T10:04:18.589193+0000","last_peered":"2024-04-25T10:04:18.589193+0000","last_clean":"2024-04-25T10:04:18.589193+0000","last_became_active":"2024-04-25T10:04:17.617241+0000","last_became_peered":"2024-04-25T10:04:17.617241+0000","last_unstale":"2024-04-25T10:04:18.589193+0000","last_undegraded":"2024-04-25T10:04:18.589193+0000","last_fullsized":"2024-04-25T10:04:18.589193+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.380920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625077+0000","last_change":"2024-04-25T10:04:16.598220+0000","last_active":"2024-04-25T10:04:19.625077+0000","last_peered":"2024-04-25T10:04:19.625077+0000","last_clean":"2024-04-25T10:04:19.625077+0000","last_became_active":"2024-04-25T10:04:16.597975+0000","last_became_peered":"2024-04-25T10:04:16.597975+0000","last_unstale":"2024-04-25T10:04:19.625077+0000","last_undegraded":"2024-04-25T10:04:19.625077+0000","last_fullsized":"2024-04-25T10:04:19.625077+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:10.105919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587324+0000","last_change":"2024-04-25T10:04:17.622238+0000","last_active":"2024-04-25T10:04:18.587324+0000","last_peered":"2024-04-25T10:04:18.587324+0000","last_clean":"2024-04-25T10:04:18.587324+0000","last_became_active":"2024-04-25T10:04:17.621439+0000","last_became_peered":"2024-04-25T10:04:17.621439+0000","last_unstale":"2024-04-25T10:04:18.587324+0000","last_undegraded":"2024-04-25T10:04:18.587324+0000","last_fullsized":"2024-04-25T10:04:18.587324+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:40.144526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"51'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589152+0000","last_change":"2024-04-25T10:04:16.579948+0000","last_active":"2024-04-25T10:04:18.589152+0000","last_peered":"2024-04-25T10:04:18.589152+0000","last_clean":"2024-04-25T10:04:18.589152+0000","last_became_active":"2024-04-25T10:04:16.578658+0000","last_became_peered":"2024-04-25T10:04:16.578658+0000","last_unstale":"2024-04-25T10:04:18.589152+0000","last_undegraded":"2024-04-25T10:04:18.589152+0000","last_fullsized":"2024-04-25T10:04:18.589152+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:50.077859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589510+0000","last_change":"2024-04-25T10:04:17.621236+0000","last_active":"2024-04-25T10:04:18.589510+0000","last_peered":"2024-04-25T10:04:18.589510+0000","last_clean":"2024-04-25T10:04:18.589510+0000","last_became_active":"2024-04-25T10:04:17.620974+0000","last_became_peered":"2024-04-25T10:04:17.620974+0000","last_unstale":"2024-04-25T10:04:18.589510+0000","last_undegraded":"2024-04-25T10:04:18.589510+0000","last_fullsized":"2024-04-25T10:04:18.589510+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:30.593485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.588878+0000","last_change":"2024-04-25T10:04:16.578507+0000","last_active":"2024-04-25T10:04:18.588878+0000","last_peered":"2024-04-25T10:04:18.588878+0000","last_clean":"2024-04-25T10:04:18.588878+0000","last_became_active":"2024-04-25T10:04:16.577675+0000","last_became_peered":"2024-04-25T10:04:16.577675+0000","last_unstale":"2024-04-25T10:04:18.588878+0000","last_undegraded":"2024-04-25T10:04:18.588878+0000","last_fullsized":"2024-04-25T10:04:18.588878+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:15.792569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123079+0000","last_change":"2024-04-25T10:04:17.622875+0000","last_active":"2024-04-25T10:04:19.123079+0000","last_peered":"2024-04-25T10:04:19.123079+0000","last_clean":"2024-04-25T10:04:19.123079+0000","last_became_active":"2024-04-25T10:04:17.622135+0000","last_became_peered":"2024-04-25T10:04:17.622135+0000","last_unstale":"2024-04-25T10:04:19.123079+0000","last_undegraded":"2024-04-25T10:04:19.123079+0000","last_fullsized":"2024-04-25T10:04:19.123079+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:17.510586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589819+0000","last_change":"2024-04-25T10:04:16.596941+0000","last_active":"2024-04-25T10:04:18.589819+0000","last_peered":"2024-04-25T10:04:18.589819+0000","last_clean":"2024-04-25T10:04:18.589819+0000","last_became_active":"2024-04-25T10:04:16.596643+0000","last_became_peered":"2024-04-25T10:04:16.596643+0000","last_unstale":"2024-04-25T10:04:18.589819+0000","last_undegraded":"2024-04-25T10:04:18.589819+0000","last_fullsized":"2024-04-25T10:04:18.589819+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:45.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589183+0000","last_change":"2024-04-25T10:04:17.599010+0000","last_active":"2024-04-25T10:04:18.589183+0000","last_peered":"2024-04-25T10:04:18.589183+0000","last_clean":"2024-04-25T10:04:18.589183+0000","last_became_active":"2024-04-25T10:04:17.598337+0000","last_became_peered":"2024-04-25T10:04:17.598337+0000","last_unstale":"2024-04-25T10:04:18.589183+0000","last_undegraded":"2024-04-25T10:04:18.589183+0000","last_fullsized":"2024-04-25T10:04:18.589183+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:06.973683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625484+0000","last_change":"2024-04-25T10:04:16.586256+0000","last_active":"2024-04-25T10:04:19.625484+0000","last_peered":"2024-04-25T10:04:19.625484+0000","last_clean":"2024-04-25T10:04:19.625484+0000","last_became_active":"2024-04-25T10:04:16.585976+0000","last_became_peered":"2024-04-25T10:04:16.585976+0000","last_unstale":"2024-04-25T10:04:19.625484+0000","last_undegraded":"2024-04-25T10:04:19.625484+0000","last_fullsized":"2024-04-25T10:04:19.625484+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:24.143592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591920+0000","last_change":"2024-04-25T10:04:17.616394+0000","last_active":"2024-04-25T10:04:19.591920+0000","last_peered":"2024-04-25T10:04:19.591920+0000","last_clean":"2024-04-25T10:04:19.591920+0000","last_became_active":"2024-04-25T10:04:17.612295+0000","last_became_peered":"2024-04-25T10:04:17.612295+0000","last_unstale":"2024-04-25T10:04:19.591920+0000","last_undegraded":"2024-04-25T10:04:19.591920+0000","last_fullsized":"2024-04-25T10:04:19.591920+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.045726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591093+0000","last_change":"2024-04-25T10:04:16.591056+0000","last_active":"2024-04-25T10:04:19.591093+0000","last_peered":"2024-04-25T10:04:19.591093+0000","last_clean":"2024-04-25T10:04:19.591093+0000","last_became_active":"2024-04-25T10:04:16.590340+0000","last_became_peered":"2024-04-25T10:04:16.590340+0000","last_unstale":"2024-04-25T10:04:19.591093+0000","last_undegraded":"2024-04-25T10:04:19.591093+0000","last_fullsized":"2024-04-25T10:04:19.591093+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:05.963386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.590297+0000","last_change":"2024-04-25T10:04:17.609972+0000","last_active":"2024-04-25T10:04:18.590297+0000","last_peered":"2024-04-25T10:04:18.590297+0000","last_clean":"2024-04-25T10:04:18.590297+0000","last_became_active":"2024-04-25T10:04:17.609611+0000","last_became_peered":"2024-04-25T10:04:17.609611+0000","last_unstale":"2024-04-25T10:04:18.590297+0000","last_undegraded":"2024-04-25T10:04:18.590297+0000","last_fullsized":"2024-04-25T10:04:18.590297+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:00.110743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.624024+0000","last_change":"2024-04-25T10:04:16.594633+0000","last_active":"2024-04-25T10:04:19.624024+0000","last_peered":"2024-04-25T10:04:19.624024+0000","last_clean":"2024-04-25T10:04:19.624024+0000","last_became_active":"2024-04-25T10:04:16.594369+0000","last_became_peered":"2024-04-25T10:04:16.594369+0000","last_unstale":"2024-04-25T10:04:19.624024+0000","last_undegraded":"2024-04-25T10:04:19.624024+0000","last_fullsized":"2024-04-25T10:04:19.624024+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:30.644120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123349+0000","last_change":"2024-04-25T10:04:17.616522+0000","last_active":"2024-04-25T10:04:19.123349+0000","last_peered":"2024-04-25T10:04:19.123349+0000","last_clean":"2024-04-25T10:04:19.123349+0000","last_became_active":"2024-04-25T10:04:17.615685+0000","last_became_peered":"2024-04-25T10:04:17.615685+0000","last_unstale":"2024-04-25T10:04:19.123349+0000","last_undegraded":"2024-04-25T10:04:19.123349+0000","last_fullsized":"2024-04-25T10:04:19.123349+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:07.617584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"51'2","reported_seq":16,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589570+0000","last_change":"2024-04-25T10:04:16.591007+0000","last_active":"2024-04-25T10:04:18.589570+0000","last_peered":"2024-04-25T10:04:18.589570+0000","last_clean":"2024-04-25T10:04:18.589570+0000","last_became_active":"2024-04-25T10:04:16.590297+0000","last_became_peered":"2024-04-25T10:04:16.590297+0000","last_unstale":"2024-04-25T10:04:18.589570+0000","last_undegraded":"2024-04-25T10:04:18.589570+0000","last_fullsized":"2024-04-25T10:04:18.589570+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:09.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.589628+0000","last_change":"2024-04-25T10:04:17.604847+0000","last_active":"2024-04-25T10:04:18.589628+0000","last_peered":"2024-04-25T10:04:18.589628+0000","last_clean":"2024-04-25T10:04:18.589628+0000","last_became_active":"2024-04-25T10:04:17.604566+0000","last_became_peered":"2024-04-25T10:04:17.604566+0000","last_unstale":"2024-04-25T10:04:18.589628+0000","last_undegraded":"2024-04-25T10:04:18.589628+0000","last_fullsized":"2024-04-25T10:04:18.589628+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:47.918943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.595323+0000","last_change":"2024-04-25T10:04:16.600459+0000","last_active":"2024-04-25T10:04:19.595323+0000","last_peered":"2024-04-25T10:04:19.595323+0000","last_clean":"2024-04-25T10:04:19.595323+0000","last_became_active":"2024-04-25T10:04:16.599758+0000","last_became_peered":"2024-04-25T10:04:16.599758+0000","last_unstale":"2024-04-25T10:04:19.595323+0000","last_undegraded":"2024-04-25T10:04:19.595323+0000","last_fullsized":"2024-04-25T10:04:19.595323+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:36.143924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625532+0000","last_change":"2024-04-25T10:04:17.623061+0000","last_active":"2024-04-25T10:04:19.625532+0000","last_peered":"2024-04-25T10:04:19.625532+0000","last_clean":"2024-04-25T10:04:19.625532+0000","last_became_active":"2024-04-25T10:04:17.622778+0000","last_became_peered":"2024-04-25T10:04:17.622778+0000","last_unstale":"2024-04-25T10:04:19.625532+0000","last_undegraded":"2024-04-25T10:04:19.625532+0000","last_fullsized":"2024-04-25T10:04:19.625532+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:20.347166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591434+0000","last_change":"2024-04-25T10:04:16.587720+0000","last_active":"2024-04-25T10:04:19.591434+0000","last_peered":"2024-04-25T10:04:19.591434+0000","last_clean":"2024-04-25T10:04:19.591434+0000","last_became_active":"2024-04-25T10:04:16.587136+0000","last_became_peered":"2024-04-25T10:04:16.587136+0000","last_unstale":"2024-04-25T10:04:19.591434+0000","last_undegraded":"2024-04-25T10:04:19.591434+0000","last_fullsized":"2024-04-25T10:04:19.591434+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:47.662783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591838+0000","last_change":"2024-04-25T10:04:17.608003+0000","last_active":"2024-04-25T10:04:19.591838+0000","last_peered":"2024-04-25T10:04:19.591838+0000","last_clean":"2024-04-25T10:04:19.591838+0000","last_became_active":"2024-04-25T10:04:17.604950+0000","last_became_peered":"2024-04-25T10:04:17.604950+0000","last_unstale":"2024-04-25T10:04:19.591838+0000","last_undegraded":"2024-04-25T10:04:19.591838+0000","last_fullsized":"2024-04-25T10:04:19.591838+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:17.227912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625586+0000","last_change":"2024-04-25T10:04:16.586300+0000","last_active":"2024-04-25T10:04:19.625586+0000","last_peered":"2024-04-25T10:04:19.625586+0000","last_clean":"2024-04-25T10:04:19.625586+0000","last_became_active":"2024-04-25T10:04:16.586044+0000","last_became_peered":"2024-04-25T10:04:16.586044+0000","last_unstale":"2024-04-25T10:04:19.625586+0000","last_undegraded":"2024-04-25T10:04:19.625586+0000","last_fullsized":"2024-04-25T10:04:19.625586+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:29.583189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123122+0000","last_change":"2024-04-25T10:04:17.644856+0000","last_active":"2024-04-25T10:04:19.123122+0000","last_peered":"2024-04-25T10:04:19.123122+0000","last_clean":"2024-04-25T10:04:19.123122+0000","last_became_active":"2024-04-25T10:04:17.644173+0000","last_became_peered":"2024-04-25T10:04:17.644173+0000","last_unstale":"2024-04-25T10:04:19.123122+0000","last_undegraded":"2024-04-25T10:04:19.123122+0000","last_fullsized":"2024-04-25T10:04:19.123122+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:44.107003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.625399+0000","last_change":"2024-04-25T10:04:16.574522+0000","last_active":"2024-04-25T10:04:19.625399+0000","last_peered":"2024-04-25T10:04:19.625399+0000","last_clean":"2024-04-25T10:04:19.625399+0000","last_became_active":"2024-04-25T10:04:16.573325+0000","last_became_peered":"2024-04-25T10:04:16.573325+0000","last_unstale":"2024-04-25T10:04:19.625399+0000","last_undegraded":"2024-04-25T10:04:19.625399+0000","last_fullsized":"2024-04-25T10:04:19.625399+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:57.746441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.591715+0000","last_change":"2024-04-25T10:04:17.605043+0000","last_active":"2024-04-25T10:04:19.591715+0000","last_peered":"2024-04-25T10:04:19.591715+0000","last_clean":"2024-04-25T10:04:19.591715+0000","last_became_active":"2024-04-25T10:04:17.604782+0000","last_became_peered":"2024-04-25T10:04:17.604782+0000","last_unstale":"2024-04-25T10:04:19.591715+0000","last_undegraded":"2024-04-25T10:04:19.591715+0000","last_fullsized":"2024-04-25T10:04:19.591715+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:30.448502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.586800+0000","last_change":"2024-04-25T10:04:16.588417+0000","last_active":"2024-04-25T10:04:18.586800+0000","last_peered":"2024-04-25T10:04:18.586800+0000","last_clean":"2024-04-25T10:04:18.586800+0000","last_became_active":"2024-04-25T10:04:16.587702+0000","last_became_peered":"2024-04-25T10:04:16.587702+0000","last_unstale":"2024-04-25T10:04:18.586800+0000","last_undegraded":"2024-04-25T10:04:18.586800+0000","last_fullsized":"2024-04-25T10:04:18.586800+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:49.138416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.626366+0000","last_change":"2024-04-25T10:04:17.622566+0000","last_active":"2024-04-25T10:04:19.626366+0000","last_peered":"2024-04-25T10:04:19.626366+0000","last_clean":"2024-04-25T10:04:19.626366+0000","last_became_active":"2024-04-25T10:04:17.621738+0000","last_became_peered":"2024-04-25T10:04:17.621738+0000","last_unstale":"2024-04-25T10:04:19.626366+0000","last_undegraded":"2024-04-25T10:04:19.626366+0000","last_fullsized":"2024-04-25T10:04:19.626366+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:00.045726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.123109+0000","last_change":"2024-04-25T10:04:16.594813+0000","last_active":"2024-04-25T10:04:19.123109+0000","last_peered":"2024-04-25T10:04:19.123109+0000","last_clean":"2024-04-25T10:04:19.123109+0000","last_became_active":"2024-04-25T10:04:16.593686+0000","last_became_peered":"2024-04-25T10:04:16.593686+0000","last_unstale":"2024-04-25T10:04:19.123109+0000","last_undegraded":"2024-04-25T10:04:19.123109+0000","last_fullsized":"2024-04-25T10:04:19.123109+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:29.583189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:18.587324+0000","last_change":"2024-04-25T10:04:17.619694+0000","last_active":"2024-04-25T10:04:18.587324+0000","last_peered":"2024-04-25T10:04:18.587324+0000","last_clean":"2024-04-25T10:04:18.587324+0000","last_became_active":"2024-04-25T10:04:17.618888+0000","last_became_peered":"2024-04-25T10:04:17.618888+0000","last_unstale":"2024-04-25T10:04:18.587324+0000","last_undegraded":"2024-04-25T10:04:18.587324+0000","last_fullsized":"2024-04-25T10:04:18.587324+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:16.555812+0000","last_clean_scrub_stamp":"2024-04-25T10:04:16.555812+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:42.777647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"52'1","reported_seq":15,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594830+0000","last_change":"2024-04-25T10:04:16.607364+0000","last_active":"2024-04-25T10:04:19.594830+0000","last_peered":"2024-04-25T10:04:19.594830+0000","last_clean":"2024-04-25T10:04:19.594830+0000","last_became_active":"2024-04-25T10:04:16.607155+0000","last_became_peered":"2024-04-25T10:04:16.607155+0000","last_unstale":"2024-04-25T10:04:19.594830+0000","last_undegraded":"2024-04-25T10:04:19.594830+0000","last_fullsized":"2024-04-25T10:04:19.594830+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":49,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:04:15.545516+0000","last_clean_scrub_stamp":"2024-04-25T10:04:15.545516+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:29.022281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-25T10:04:19.594355+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:04:19.594355+0000","last_peered":"2024-04-25T10:04:19.594355+0000","last_clean":"2024-04-25T10:04:19.594355+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:04:19.594355+0000","last_undegraded":"2024-04-25T10:04:19.594355+0000","last_fullsized":"2024-04-25T10:04:19.594355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":865637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574883,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27624,"kb_used_data":604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344216,"statfs":{"total":96636764160,"available":96608477184,"internally_reserved":0,"allocated":618496,"data_stored":275145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":622592,"data_stored":275727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":634880,"data_stored":276779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31800,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340040,"statfs":{"total":96636764160,"available":96604200960,"internally_reserved":0,"allocated":630784,"data_stored":276757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":275305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1220608,"data_stored":865569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:04:25.338 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:04:25.338 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:04:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 Processor -- start 2024-04-25T10:04:25.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 -- start start 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c0815b6b0 0x7f5c080ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c080cf590 0x7f5c080cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c0815f130 0x7f5c0815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0815ba90 con 0x7f5c080cf590 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c080cbd20 con 0x7f5c0815b6b0 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0ffc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cc000 con 0x7f5c0815f130 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c080cf590 0x7f5c080cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5c080cf590 0x7f5c080cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36380/0 (socket says 172.21.15.82:36380) 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 -- 172.21.15.82:0/3811804725 learned_addr learned my addr 172.21.15.82:0/3811804725 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0dd65640 1 --2- 172.21.15.82:0/3811804725 >> v2:172.21.15.196:3300/0 conn(0x7f5c0815b6b0 0x7f5c080ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 -- 172.21.15.82:0/3811804725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c0815f130 msgr2=0x7f5c0815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 --2- 172.21.15.82:0/3811804725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c0815f130 0x7f5c0815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 -- 172.21.15.82:0/3811804725 >> v2:172.21.15.196:3300/0 conn(0x7f5c0815b6b0 msgr2=0x7f5c080ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 --2- 172.21.15.82:0/3811804725 >> v2:172.21.15.196:3300/0 conn(0x7f5c0815b6b0 0x7f5c080ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.507+0000 7f5c0d564640 1 -- 172.21.15.82:0/3811804725 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080cc2e0 con 0x7f5c080cf590 2024-04-25T10:04:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0d564640 1 --2- 172.21.15.82:0/3811804725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c080cf590 0x7f5c080cfa20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5c080d0180 tx=0x7f5bf801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427515e4200d4aac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0cd63640 1 -- 172.21.15.82:0/3811804725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80b1070 con 0x7f5c080cf590 2024-04-25T10:04:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0cd63640 1 -- 172.21.15.82:0/3811804725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf80ac630 con 0x7f5c080cf590 2024-04-25T10:04:25.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0cd63640 1 -- 172.21.15.82:0/3811804725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80a8050 con 0x7f5c080cf590 2024-04-25T10:04:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/3811804725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c080cf590 msgr2=0x7f5c080cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/3811804725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c080cf590 0x7f5c080cfa20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5c080d0180 tx=0x7f5bf801c520 comp rx=0 tx=0).stop 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/3811804725 shutdown_connections 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/3811804725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c0815f130 0x7f5c0815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/3811804725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c080cf590 0x7f5c080cfa20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/3811804725 >> v2:172.21.15.196:3300/0 conn(0x7f5c0815b6b0 0x7f5c080ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/3811804725 >> 172.21.15.82:0/3811804725 conn(0x7f5c0806ae60 msgr2=0x7f5c0806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/3811804725 shutdown_connections 2024-04-25T10:04:25.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/3811804725 wait complete. 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 Processor -- start 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.511+0000 7f5c0ffc9640 1 -- start start 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c080cf590 0x7f5c08153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c0815f130 0x7f5c08164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080ca900 con 0x7f5c0815b6b0 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0815fef0 con 0x7f5c0815f130 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c080cadb0 con 0x7f5c080cf590 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36386/0 (socket says 172.21.15.82:36386) 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 -- 172.21.15.82:0/715825986 learned_addr learned my addr 172.21.15.82:0/715825986 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:25.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0e566640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c0815f130 0x7f5c08164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 -- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c080cf590 msgr2=0x7f5c08153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c080cf590 0x7f5c08153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 -- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c0815f130 msgr2=0x7f5c08164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c0815f130 0x7f5c08164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 -- 172.21.15.82:0/715825986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c080ae280 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0d564640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80af040 tx=0x7f5bf80192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80b1070 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf80ab5e0 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c080ace60 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf80a8040 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c08165920 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0804efa0 con 0x7f5c0815b6b0 2024-04-25T10:04:25.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bf80be060 con 0x7f5c0815b6b0 2024-04-25T10:04:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 0x7f5bec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5bf811e670 con 0x7f5c0815b6b0 2024-04-25T10:04:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0dd65640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 0x7f5bec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.515+0000 7f5c0dd65640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 0x7f5bec069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c040070b0 tx=0x7f5c04007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.519+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bf8016110 con 0x7f5c0815b6b0 2024-04-25T10:04:25.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.727+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5c080ad4b0 con 0x7f5c0815b6b0 2024-04-25T10:04:25.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5bfe7fc640 1 -- 172.21.15.82:0/715825986 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5bf80ef580 con 0x7f5c0815b6b0 2024-04-25T10:04:25.733 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:04:25.733 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:04:25.733 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:04:25.734 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:04:25.734 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:04:25.734 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:04:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 msgr2=0x7f5bec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 0x7f5bec069450 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c040070b0 tx=0x7f5c04007040 comp rx=0 tx=0).stop 2024-04-25T10:04:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 msgr2=0x7f5c08164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5bf80af040 tx=0x7f5bf80192c0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 shutdown_connections 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5bec066f90 0x7f5bec069450 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c0815f130 0x7f5c08164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c0815b6b0 0x7f5c08164670 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 --2- 172.21.15.82:0/715825986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c080cf590 0x7f5c08153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 >> 172.21.15.82:0/715825986 conn(0x7f5c0806ae60 msgr2=0x7f5c08156f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 shutdown_connections 2024-04-25T10:04:25.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.731+0000 7f5c0ffc9640 1 -- 172.21.15.82:0/715825986 wait complete. 2024-04-25T10:04:25.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:25.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.867+0000 7f11da24b640 1 Processor -- start 2024-04-25T10:04:25.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.867+0000 7f11da24b640 1 -- start start 2024-04-25T10:04:25.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d40d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f11d415dc30 0x7f11d40d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d40ce6b0 con 0x7f11d415d850 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d40ce990 con 0x7f11d415dc30 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d40cec70 con 0x7f11d40d0870 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f11d415d850 0x7f11d40d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42886/0 (socket says 172.21.15.82:42886) 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d37fe640 1 -- 172.21.15.82:0/2019020418 learned_addr learned my addr 172.21.15.82:0/2019020418 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 --2- 172.21.15.82:0/2019020418 >> v2:172.21.15.82:3300/0 conn(0x7f11d415dc30 0x7f11d40d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11caffd640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d40d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 -- 172.21.15.82:0/2019020418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 msgr2=0x7f11d40d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d40d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 -- 172.21.15.82:0/2019020418 >> v1:172.21.15.82:6789/0 conn(0x7f11d415d850 legacy=0x7f11d40d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 -- 172.21.15.82:0/2019020418 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d40cef50 con 0x7f11d415dc30 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d2ffd640 1 -- 172.21.15.82:0/2019020418 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696230235 0 0) 0x7f11d40ce6b0 con 0x7f11d415d850 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11caffd640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d3fff640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415dc30 0x7f11d40d2320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009d90 tx=0x7f11cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeecbb03b424a0b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d2ffd640 1 -- 172.21.15.82:0/2019020418 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b2070 con 0x7f11d415dc30 2024-04-25T10:04:25.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d2ffd640 1 -- 172.21.15.82:0/2019020418 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc0a5940 con 0x7f11d415dc30 2024-04-25T10:04:25.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11d2ffd640 1 -- 172.21.15.82:0/2019020418 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0a5d10 con 0x7f11d415dc30 2024-04-25T10:04:25.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- 172.21.15.82:0/2019020418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415dc30 msgr2=0x7f11d40d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415dc30 0x7f11d40d2320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009d90 tx=0x7f11cc01c970 comp rx=0 tx=0).stop 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- 172.21.15.82:0/2019020418 shutdown_connections 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415dc30 0x7f11d40d2320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 --2- 172.21.15.82:0/2019020418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d40d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- 172.21.15.82:0/2019020418 >> 172.21.15.82:0/2019020418 conn(0x7f11d408b000 msgr2=0x7f11d40d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- 172.21.15.82:0/2019020418 shutdown_connections 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.871+0000 7f11da24b640 1 -- 172.21.15.82:0/2019020418 wait complete. 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 Processor -- start 2024-04-25T10:04:25.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- start start 2024-04-25T10:04:25.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d4155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11d415dc30 0x7f11d414bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d416a4c0 con 0x7f11d415d850 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d416a340 con 0x7f11d415dc30 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d416a640 con 0x7f11d40d0870 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36390/0 (socket says 172.21.15.82:36390) 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 -- 172.21.15.82:0/4187657590 learned_addr learned my addr 172.21.15.82:0/4187657590 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 -- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 msgr2=0x7f11d4155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d4155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11d3fff640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11d415dc30 0x7f11d414bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 -- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11d415dc30 msgr2=0x7f11d414bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11d415dc30 0x7f11d414bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 -- 172.21.15.82:0/4187657590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4160bd0 con 0x7f11d415d850 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11caffd640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c820 tx=0x7f11bc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc015070 con 0x7f11d415d850 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc007a30 con 0x7f11d415d850 2024-04-25T10:04:25.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d40cc9d0 con 0x7f11d415d850 2024-04-25T10:04:25.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc007e40 con 0x7f11d415d850 2024-04-25T10:04:25.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d4160050 con 0x7f11d415d850 2024-04-25T10:04:25.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.875+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d415ac30 con 0x7f11d415d850 2024-04-25T10:04:25.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.879+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11bc018070 con 0x7f11d415d850 2024-04-25T10:04:25.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.879+0000 7f11d0ff9640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 0x7f11b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:25.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.879+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11bc085080 con 0x7f11d415d850 2024-04-25T10:04:25.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.879+0000 7f11d37fe640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 0x7f11b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:25.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.879+0000 7f11d37fe640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 0x7f11b4069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011f0 tx=0x7f11c4007930 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:25.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:25.883+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11d415ac30 con 0x7f11d415d850 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.075+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d40d0e60 con 0x7f11d415d850 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11d0ff9640 1 -- 172.21.15.82:0/4187657590 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f11bc055f00 con 0x7f11d415d850 2024-04-25T10:04:26.081 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:26.082 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 msgr2=0x7f11b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 0x7f11b4069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f11c40011f0 tx=0x7f11c4007930 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 msgr2=0x7f11d414b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f11bc00c820 tx=0x7f11bc00cce0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 shutdown_connections 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f11b4066f60 0x7f11b4069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11d415dc30 0x7f11d414bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11d415d850 0x7f11d414b980 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 --2- 172.21.15.82:0/4187657590 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f11d40d0870 0x7f11d4155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 >> 172.21.15.82:0/4187657590 conn(0x7f11d408b000 msgr2=0x7f11d4156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 shutdown_connections 2024-04-25T10:04:26.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.079+0000 7f11da24b640 1 -- 172.21.15.82:0/4187657590 wait complete. 2024-04-25T10:04:26.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:04:16.915215+0000', 'modified': '2024-04-25T10:04:18.967239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5946}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5946': {'gid': 5946, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/238219074', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 238219074}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 238219074}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:04:26.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:04:26.504 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:04:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 Processor -- start 2024-04-25T10:04:26.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 -- start start 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006fc50 0x7f83f006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f006b1d0 0x7f83f00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f006b710 con 0x7f83f00cfd80 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f0161560 con 0x7f83f006b1d0 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f84e0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01617b0 con 0x7f83f006fc50 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f006b1d0 0x7f83f00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f83f006b1d0 0x7f83f00cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36400/0 (socket says 172.21.15.82:36400) 2024-04-25T10:04:26.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 -- 172.21.15.82:0/2100795192 learned_addr learned my addr 172.21.15.82:0/2100795192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f627c640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006fc50 0x7f83f006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 -- 172.21.15.82:0/2100795192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006fc50 msgr2=0x7f83f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006fc50 0x7f83f006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 -- 172.21.15.82:0/2100795192 >> v1:172.21.15.196:6789/0 conn(0x7f83f00cfd80 legacy=0x7f83f015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.635+0000 7f83f5a7b640 1 -- 172.21.15.82:0/2100795192 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f0161a90 con 0x7f83f006b1d0 2024-04-25T10:04:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f006b1d0 0x7f83f00cf840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47570a4da96ff2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f527a640 1 -- 172.21.15.82:0/2100795192 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b0070 con 0x7f83f006b1d0 2024-04-25T10:04:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f527a640 1 -- 172.21.15.82:0/2100795192 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4004230 con 0x7f83f006b1d0 2024-04-25T10:04:26.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f527a640 1 -- 172.21.15.82:0/2100795192 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003620 con 0x7f83f006b1d0 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 -- 172.21.15.82:0/2100795192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f006b1d0 msgr2=0x7f83f00cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f006b1d0 0x7f83f00cf840 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 -- 172.21.15.82:0/2100795192 shutdown_connections 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f006b1d0 0x7f83f00cf840 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 --2- 172.21.15.82:0/2100795192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006fc50 0x7f83f006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 -- 172.21.15.82:0/2100795192 >> 172.21.15.82:0/2100795192 conn(0x7f83f0099a70 msgr2=0x7f83f00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 -- 172.21.15.82:0/2100795192 shutdown_connections 2024-04-25T10:04:26.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.639+0000 7f83f84e0640 1 -- 172.21.15.82:0/2100795192 wait complete. 2024-04-25T10:04:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 Processor -- start 2024-04-25T10:04:26.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- start start 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006b1d0 0x7f83f00b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f006fc50 0x7f83f015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00d0e30 con 0x7f83f00cfd80 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00d0210 con 0x7f83f006fc50 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f0070030 con 0x7f83f006b1d0 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36402/0 (socket says 172.21.15.82:36402) 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 -- 172.21.15.82:0/1396123625 learned_addr learned my addr 172.21.15.82:0/1396123625 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 -- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006b1d0 msgr2=0x7f83f00b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006b1d0 0x7f83f00b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 -- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f006fc50 msgr2=0x7f83f015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f006fc50 0x7f83f015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f006fc50 0x7f83f015c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 -- 172.21.15.82:0/1396123625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f015dcd0 con 0x7f83f00cfd80 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009df0 tx=0x7f83e000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83eeffd640 1 -- 172.21.15.82:0/1396123625 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000cd50 con 0x7f83f00cfd80 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83eeffd640 1 -- 172.21.15.82:0/1396123625 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0004020 con 0x7f83f00cfd80 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f015dfb0 con 0x7f83f00cfd80 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83eeffd640 1 -- 172.21.15.82:0/1396123625 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004450 con 0x7f83f00cfd80 2024-04-25T10:04:26.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00b1280 con 0x7f83f00cfd80 2024-04-25T10:04:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.643+0000 7f83eeffd640 1 -- 172.21.15.82:0/1396123625 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f83e001b020 con 0x7f83f00cfd80 2024-04-25T10:04:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83eeffd640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83eeffd640 1 -- 172.21.15.82:0/1396123625 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e0087290 con 0x7f83f00cfd80 2024-04-25T10:04:26.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f627c640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f627c640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83f00d1270 tx=0x7f83e800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 Processor -- start 2024-04-25T10:04:26.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- start start 2024-04-25T10:04:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x5645527bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645526595f0 con 0x564552771f30 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645525c0da0 con 0x5645527bc240 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564552549150 con 0x5645527bb920 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36416/0 (socket says 172.21.15.82:36416) 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 -- 172.21.15.82:0/3949138269 learned_addr learned my addr 172.21.15.82:0/3949138269 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f627c640 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x5645527bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 -- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 msgr2=0x5645527bbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x5645527bbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 -- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 msgr2=0x5645527719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 -- 172.21.15.82:0/3949138269 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564552562ca0 con 0x564552771f30 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f83e005fbe0 tx=0x7f83e0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d09b7829db7d6c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83d7fff640 1 -- 172.21.15.82:0/3949138269 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e006f020 con 0x564552771f30 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83d7fff640 1 -- 172.21.15.82:0/3949138269 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e005bb50 con 0x564552771f30 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83d7fff640 1 -- 172.21.15.82:0/3949138269 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0060110 con 0x564552771f30 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- 172.21.15.82:0/3949138269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 msgr2=0x564552772310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f83e005fbe0 tx=0x7f83e0000bd0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- 172.21.15.82:0/3949138269 shutdown_connections 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564552771f30 0x564552772310 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3949138269 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x5645527bbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- 172.21.15.82:0/3949138269 >> 172.21.15.82:0/3949138269 conn(0x5645527782a0 msgr2=0x5645527702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- 172.21.15.82:0/3949138269 shutdown_connections 2024-04-25T10:04:26.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.647+0000 7f83fb64d000 1 -- 172.21.15.82:0/3949138269 wait complete. 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- start start 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 Processor -- start 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bb920 0x5645527c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455257b990 con 0x5645527bc240 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645525bce40 con 0x5645527bb920 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56455254c900 con 0x564552771f30 2024-04-25T10:04:26.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 10 client.0 _ll_register_callbacks cb 0x5645526741c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36422/0 (socket says 172.21.15.82:36422) 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 -- 172.21.15.82:0/2401014518 learned_addr learned my addr 172.21.15.82:0/2401014518 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bb920 0x5645527c31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 -- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 msgr2=0x5645527c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 -- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bb920 msgr2=0x5645527c31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bb920 0x5645527c31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 -- 172.21.15.82:0/2401014518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564552579630 con 0x5645527bc240 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f83e0053e10 tx=0x7f83e0000f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532c0d2795a4be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0085250 con 0x5645527bc240 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e005bb50 con 0x5645527bc240 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e0085050 con 0x5645527bc240 2024-04-25T10:04:26.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e007c050 con 0x5645527bc240 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 10 client.6123 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5645525493a0 con 0x5645527bc240 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e005bb50 con 0x5645527bc240 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83e0054070 con 0x5645527bc240 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 10 client.6123 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 10 client.6123 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 20 client.6123 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56455277caf0 0x5645527764b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 4 client.6123 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56455255ca50 con 0x56455277caf0 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f627c640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56455277caf0 0x5645527764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83f627c640 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56455277caf0 0x5645527764b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30a5413ea04fc90 server_cookie=d607f24d6f4d1eef in_seq=0 out_seq=0 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 10 client.6123 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:26.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 1 -- 172.21.15.82:0/2401014518 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56455255ca50 con 0x56455277caf0 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 10 client.6123 handle_command_reply: tid=0 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 client.6123 shutdown 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 msgr2=0x564552775aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f83e0053e10 tx=0x7f83e0000f80 comp rx=0 tx=0).stop 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 shutdown_connections 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56455277caf0 0x5645527764b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552775aa0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83d57fa640 0 client.6123 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bb920 0x5645527c31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 --2- 172.21.15.82:0/2401014518 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 >> 172.21.15.82:0/2401014518 conn(0x5645527782a0 msgr2=0x5645527709e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 shutdown_connections 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.651+0000 7f83fb64d000 1 -- 172.21.15.82:0/2401014518 wait complete. 2024-04-25T10:04:26.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.655+0000 7f83fb64d000 20 client.6123 trim_cache size 0 max 16384 2024-04-25T10:04:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.667+0000 7f83fb64d000 1 Processor -- start 2024-04-25T10:04:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.667+0000 7f83fb64d000 1 -- start start 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 0x564552774320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x564552774860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f02004d0 con 0x5645527bc240 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f01ff6b0 con 0x564552771f30 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00c93b0 con 0x5645527bb920 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36432/0 (socket says 172.21.15.82:36432) 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 -- 172.21.15.82:0/432774747 learned_addr learned my addr 172.21.15.82:0/432774747 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x564552774860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 -- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 msgr2=0x564552774860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x564552774860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f627c640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 0x564552774320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 -- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 msgr2=0x564552774320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 0x564552774320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 -- 172.21.15.82:0/432774747 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5645527bc070 con 0x5645527bc240 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f627c640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 0x564552774320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f83e0057700 tx=0x7f83e0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e11f9e5043a5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83d77fe640 1 -- 172.21.15.82:0/432774747 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e006f040 con 0x5645527bc240 2024-04-25T10:04:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83d77fe640 1 -- 172.21.15.82:0/432774747 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0054070 con 0x5645527bc240 2024-04-25T10:04:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83d77fe640 1 -- 172.21.15.82:0/432774747 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0053750 con 0x5645527bc240 2024-04-25T10:04:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- 172.21.15.82:0/432774747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 msgr2=0x564552774da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f83e0057700 tx=0x7f83e0020000 comp rx=0 tx=0).stop 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- 172.21.15.82:0/432774747 shutdown_connections 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5645527bb920 0x564552774860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bc240 0x564552774da0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 --2- 172.21.15.82:0/432774747 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564552771f30 0x564552774320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- 172.21.15.82:0/432774747 >> 172.21.15.82:0/432774747 conn(0x5645527782a0 msgr2=0x5645527a7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- 172.21.15.82:0/432774747 shutdown_connections 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- 172.21.15.82:0/432774747 wait complete. 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 -- start start 2024-04-25T10:04:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.671+0000 7f83fb64d000 1 Processor -- start 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527d7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645525c53d0 con 0x5645527bb920 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56455267cd50 con 0x5645527bc240 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56455255b980 con 0x564552771f30 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 10 client.0 _ll_register_callbacks cb 0x7f83e0053ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36434/0 (socket says 172.21.15.82:36434) 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 -- 172.21.15.82:0/3363677694 learned_addr learned my addr 172.21.15.82:0/3363677694 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527d7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f627c640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 -- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 msgr2=0x5645527c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 -- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 msgr2=0x5645527d7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527d7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 -- 172.21.15.82:0/3363677694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564552642600 con 0x5645527bb920 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f627c640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f6a7d640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f5a7b640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f83e4008010 tx=0x7f83e401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da439d5ab1147dae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40b0040 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e4003a30 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f83e0054070 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83e40c5050 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 10 client.6129 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5645525559b0 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0054070 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f83e40c8020 con 0x5645527bb920 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 10 client.6129 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 10 client.6129 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 20 client.6129 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x5645527d7db0 0x56455281c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 4 client.6129 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5645525ce2e0 con 0x5645527d7db0 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f627c640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x5645527d7db0 0x56455281c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f627c640 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x5645527d7db0 0x56455281c090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc3f45e5eb12487b server_cookie=8e3e6c0b0f5739a1 in_seq=0 out_seq=0 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 10 client.6129 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 1 -- 172.21.15.82:0/3363677694 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5645525ce2e0 con 0x5645527d7db0 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 10 client.6129 handle_command_reply: tid=0 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 client.6129 shutdown 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 msgr2=0x5645527d7330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f83e4008010 tx=0x7f83e401ac30 comp rx=0 tx=0).stop 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 shutdown_connections 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x5645527d7db0 0x56455281c090 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5645527bc240 0x5645527d7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5645527bb920 0x5645527d7330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 --2- 172.21.15.82:0/3363677694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564552771f30 0x5645527c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83d57fa640 0 client.6129 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 >> 172.21.15.82:0/3363677694 conn(0x5645527782a0 msgr2=0x5645527764b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 shutdown_connections 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 1 -- 172.21.15.82:0/3363677694 wait complete. 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83fb64d000 20 client.6129 trim_cache size 0 max 16384 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 5946, 2024-04-25T10:04:26.685 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 9.0952649819999998, 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 24, 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 52, 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 52, 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 14.135508616999999 2024-04-25T10:04:26.686 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 msgr2=0x7f83dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f83f00d1270 tx=0x7f83e800b040 comp rx=0 tx=0).stop 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 msgr2=0x7f83f00cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009df0 tx=0x7f83e000c430 comp rx=0 tx=0).stop 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 shutdown_connections 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f83dc066f60 0x7f83dc069420 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f83f00cfd80 0x7f83f00cae30 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f83f006fc50 0x7f83f015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 --2- 172.21.15.82:0/1396123625 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f83f006b1d0 0x7f83f00b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 >> 172.21.15.82:0/1396123625 conn(0x7f83f0099a70 msgr2=0x7f83f00adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 shutdown_connections 2024-04-25T10:04:26.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.675+0000 7f83f84e0640 1 -- 172.21.15.82:0/1396123625 wait complete. 2024-04-25T10:04:26.696 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:04:26.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.795+0000 7f74947a8640 1 Processor -- start 2024-04-25T10:04:26.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.795+0000 7f74947a8640 1 -- start start 2024-04-25T10:04:26.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 0x7f748c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f748c0718c0 0x7f748c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0b2990 con 0x7f748c0718c0 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c15ae30 con 0x7f748c15a280 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c15b110 con 0x7f748c159ea0 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f748c0718c0 0x7f748c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f748c0718c0 0x7f748c071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36436/0 (socket says 172.21.15.82:36436) 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 -- 172.21.15.82:0/2706390276 learned_addr learned my addr 172.21.15.82:0/2706390276 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7491d43640 1 --2- 172.21.15.82:0/2706390276 >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 0x7f748c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 -- 172.21.15.82:0/2706390276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 msgr2=0x7f748c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 --2- 172.21.15.82:0/2706390276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 -- 172.21.15.82:0/2706390276 >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 msgr2=0x7f748c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:26.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 --2- 172.21.15.82:0/2706390276 >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 0x7f748c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 -- 172.21.15.82:0/2706390276 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0b2620 con 0x7f748c0718c0 2024-04-25T10:04:26.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7491d43640 1 --2- 172.21.15.82:0/2706390276 >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 0x7f748c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7492d45640 1 --2- 172.21.15.82:0/2706390276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c071d50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f748c06cea0 tx=0x7f748401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8ad1621d46b16d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7491542640 1 -- 172.21.15.82:0/2706390276 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840b0070 con 0x7f748c0718c0 2024-04-25T10:04:26.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7491542640 1 -- 172.21.15.82:0/2706390276 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74840ab600 con 0x7f748c0718c0 2024-04-25T10:04:26.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f7491542640 1 -- 172.21.15.82:0/2706390276 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840ab9d0 con 0x7f748c0718c0 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 -- 172.21.15.82:0/2706390276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 msgr2=0x7f748c071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.799+0000 7f74947a8640 1 --2- 172.21.15.82:0/2706390276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c071d50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f748c06cea0 tx=0x7f748401c520 comp rx=0 tx=0).stop 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2706390276 shutdown_connections 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- 172.21.15.82:0/2706390276 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c071d50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- 172.21.15.82:0/2706390276 >> v2:172.21.15.196:3300/0 conn(0x7f748c15a280 0x7f748c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- 172.21.15.82:0/2706390276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2706390276 >> 172.21.15.82:0/2706390276 conn(0x7f748c0b11a0 msgr2=0x7f748c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2706390276 shutdown_connections 2024-04-25T10:04:26.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2706390276 wait complete. 2024-04-25T10:04:26.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 Processor -- start 2024-04-25T10:04:26.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- start start 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 0x7f748c147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06d420 con 0x7f748c0718c0 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06d2a0 con 0x7f748c15a280 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06d5a0 con 0x7f748c159ea0 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36442/0 (socket says 172.21.15.82:36442) 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 -- 172.21.15.82:0/2514330464 learned_addr learned my addr 172.21.15.82:0/2514330464 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492d45640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 0x7f748c147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 -- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 msgr2=0x7f748c146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7491d43640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c146e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 -- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 msgr2=0x7f748c147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 0x7f748c147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 -- 172.21.15.82:0/2514330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c14c1e0 con 0x7f748c0718c0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492d45640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 0x7f748c147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7492544640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f748800ba50 tx=0x7f748800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488019070 con 0x7f748c0718c0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748800cc00 con 0x7f748c0718c0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c158c30 con 0x7f748c0718c0 2024-04-25T10:04:26.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7488014440 con 0x7f748c0718c0 2024-04-25T10:04:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.803+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0ccf40 con 0x7f748c0718c0 2024-04-25T10:04:26.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c0cca70 con 0x7f748c0718c0 2024-04-25T10:04:26.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f748801c070 con 0x7f748c0718c0 2024-04-25T10:04:26.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f7482ffd640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:26.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7488084f00 con 0x7f748c0718c0 2024-04-25T10:04:26.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f7491d43640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:26.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.807+0000 7f7491d43640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 0x7f7470069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f747c006fd0 tx=0x7f747c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:26.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.811+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7488055e10 con 0x7f748c0718c0 2024-04-25T10:04:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.987+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f748c06e6b0 con 0x7f748c0718c0 2024-04-25T10:04:26.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.987+0000 7f7482ffd640 1 -- 172.21.15.82:0/2514330464 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7488055710 con 0x7f748c0718c0 2024-04-25T10:04:26.992 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 msgr2=0x7f7470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 0x7f7470069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f747c006fd0 tx=0x7f747c008040 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 msgr2=0x7f748c14bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f748800ba50 tx=0x7f748800bf10 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 shutdown_connections 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f748c159ea0 0x7f748c146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f748c15a280 0x7f748c147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 --2- 172.21.15.82:0/2514330464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f748c0718c0 0x7f748c14bca0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 >> 172.21.15.82:0/2514330464 conn(0x7f748c0b11a0 msgr2=0x7f748c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:26.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 shutdown_connections 2024-04-25T10:04:26.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:26.991+0000 7f74947a8640 1 -- 172.21.15.82:0/2514330464 wait complete. 2024-04-25T10:04:27.010 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:04:27.010 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:04:27.010 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:04:27.010 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:04:27.010 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:27.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:04:27.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:04:27.011 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:04:27.011 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:27.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.011 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.017 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42403sec preferred_lft 42403sec 2024-04-25T10:04:27.018 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:27.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:04:27.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.020 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:04:27.070 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:04:27.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.072 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:04:27.121 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:04:27.121 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:04:27.121 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:04:27.121 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:04:27.121 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:04:27.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.122 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:04:27.122 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:04:27.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:04:27.123 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:04:27.123 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:04:27.123 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:04:27.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.199 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:04:27.245 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:04:27.246 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:04:27.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:27.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:04:27.299 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:04:27.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:27.300 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:27.346 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:04:27.346 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:04:27.347 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:27.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:04:27.455 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:04:27.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:04:27.508 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:04:27.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:04:27.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:04:27.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:04:27.509 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:04:27.509 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:04:27.509 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:04:27.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:04:27.510 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:04:27.510 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:04:27.510 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:04:27.510 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:04:27.511 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:04:27.512 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:27.513 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:27.514 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:27.514 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:27.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.515 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:04:27.562 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:04:27.562 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:04:27.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:04:27.621 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:04:27.670 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:04:27.670 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:04:27.670 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:04:27.670 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:04:27.670 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:04:27.671 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:27.672 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:27.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:27.673 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:04:27.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:04:27.683+0000 7f3f61b73580 -1 init, newargv = 0x5574e9f93d00 newargc=15 2024-04-25T10:04:27.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[41429]: starting ceph client 2024-04-25T10:04:27.690 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[41429]: starting fuse 2024-04-25T10:04:28.792 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:04:28.807 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:04:28.858 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:04:28.858 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:04:28.858 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:04:28.859 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:04:28.860 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:28.861 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:04:28.861 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:28.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:28.862 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:04:28.909 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:04:28.909 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:04:28.910 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:04:28.911 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:04:28.912 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:04:29.036 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.41429.asok 2024-04-25T10:04:29.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:29.048 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41429.asok 2024-04-25T10:04:29.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:29.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41429.asok status 2024-04-25T10:04:29.210 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:29.210 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "41429", 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6141, 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:04:29.211 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 6141 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3314588836 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3314588836 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:29.212 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.6141 192.168.0.1:0/3314588836", 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/3314588836", 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 24, 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 52, 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:04:29.213 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:29.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:29.224 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:04:29.224 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:04:29.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:29.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:29.230 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:04:29.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:29.286 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T10:04:29.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:29.342 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T10:04:29.396 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:04:29.499 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:04:29.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:29.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:04:29.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 Processor -- start 2024-04-25T10:04:29.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 -- start start 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c4c1451f0 0x7f5c4c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c0ccc70 con 0x7f5c4c0ce510 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4c06dc40 con 0x7f5c4c144ca0 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c52e8d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4c0cc2b0 con 0x7f5c4c1451f0 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c50c29640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5c4c0ce510 0x7f5c4c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42900/0 (socket says 172.21.15.82:42900) 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c50c29640 1 -- 172.21.15.82:0/3440354489 learned_addr learned my addr 172.21.15.82:0/3440354489 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 --2- 172.21.15.82:0/3440354489 >> v2:172.21.15.196:3300/0 conn(0x7f5c4c1451f0 0x7f5c4c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742523813 0 0) 0x7f5c4c0ccc70 con 0x7f5c4c0ce510 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c34003530 con 0x7f5c4c0ce510 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3266985779 0 0) 0x7f5c34003530 con 0x7f5c4c0ce510 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 >> v2:172.21.15.196:3300/0 conn(0x7f5c4c1451f0 msgr2=0x7f5c4c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 --2- 172.21.15.82:0/3440354489 >> v2:172.21.15.196:3300/0 conn(0x7f5c4c1451f0 0x7f5c4c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 >> v1:172.21.15.196:6789/0 conn(0x7f5c4c144ca0 legacy=0x7f5c4c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c06df20 con 0x7f5c4c0ce510 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5c40001680 con 0x7f5c4c0ce510 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 >> v1:172.21.15.82:6789/0 conn(0x7f5c4c0ce510 legacy=0x7f5c4c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34003ab0 0x7f5c34003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c34004810 0x7f5c34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 0x7f5c3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0cc2b0 con 0x7f5c34005220 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3400c160 con 0x7f5c34003ab0 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3400c360 con 0x7f5c34004810 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 0x7f5c3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c50c29640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34003ab0 0x7f5c34003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 -- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c34004810 msgr2=0x7f5c34004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c34004810 0x7f5c34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 -- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34003ab0 msgr2=0x7f5c34003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:29.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34003ab0 0x7f5c34003ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c5142a640 1 -- 172.21.15.82:0/3440354489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34003530 con 0x7f5c34005220 2024-04-25T10:04:29.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.675+0000 7f5c50c29640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34003ab0 0x7f5c34003ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c5142a640 1 -- 172.21.15.82:0/3440354489 reap_dead start 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c5142a640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 0x7f5c3400b9d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c1456a0 tx=0x7f5c4401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f2dde12dbe9913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c440b2070 con 0x7f5c34005220 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c440a5cf0 con 0x7f5c34005220 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4b7fe640 1 -- 172.21.15.82:0/3440354489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c44009bf0 con 0x7f5c34005220 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 msgr2=0x7f5c3400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 0x7f5c3400b9d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c1456a0 tx=0x7f5c4401cc20 comp rx=0 tx=0).stop 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- 172.21.15.82:0/3440354489 shutdown_connections 2024-04-25T10:04:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/3440354489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c34005220 0x7f5c3400b9d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- 172.21.15.82:0/3440354489 >> 172.21.15.82:0/3440354489 conn(0x7f5c4c0b11a0 msgr2=0x7f5c4c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- 172.21.15.82:0/3440354489 shutdown_connections 2024-04-25T10:04:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- 172.21.15.82:0/3440354489 wait complete. 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 Processor -- start 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- start start 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 0x7f5c4c0ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 0x7f5c4c0c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0ce8f0 con 0x7f5c4c15b250 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c06e290 con 0x7f5c34005220 2024-04-25T10:04:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c52e8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4c0cea70 con 0x7f5c4c0c75f0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36456/0 (socket says 172.21.15.82:36456) 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.679+0000 7f5c4bfff640 1 -- 172.21.15.82:0/126078844 learned_addr learned my addr 172.21.15.82:0/126078844 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c50c29640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 0x7f5c4c0ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c5142a640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 0x7f5c4c0c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 -- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 msgr2=0x7f5c4c0c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 0x7f5c4c0c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 -- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 msgr2=0x7f5c4c0ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 0x7f5c4c0ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 -- 172.21.15.82:0/126078844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c138b00 con 0x7f5c4c15b250 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c50c29640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 0x7f5c4c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c5142a640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 0x7f5c4c0c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c4bfff640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c3a0 tx=0x7f5c3c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c017070 con 0x7f5c4c15b250 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c0052d0 con 0x7f5c4c15b250 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c003840 con 0x7f5c4c15b250 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c0ad220 con 0x7f5c4c15b250 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c0ad7e0 con 0x7f5c4c15b250 2024-04-25T10:04:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c4c04efa0 con 0x7f5c4c15b250 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c3c01a070 con 0x7f5c4c15b250 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 0x7f5c38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c50c29640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 0x7f5c38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c3c0844a0 con 0x7f5c4c15b250 2024-04-25T10:04:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.683+0000 7f5c50c29640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 0x7f5c38069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001950 tx=0x7f5c40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.687+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c4c138b00 con 0x7f5c4c15b250 2024-04-25T10:04:29.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.883+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c4c0c5370 con 0x7f5c4c15b250 2024-04-25T10:04:29.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.887+0000 7f5c497fa640 1 -- 172.21.15.82:0/126078844 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f5c3c005480 con 0x7f5c4c15b250 2024-04-25T10:04:29.889 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:29.890 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:04:29.890 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:04:29.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 msgr2=0x7f5c38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 0x7f5c38069450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5c40001950 tx=0x7f5c40009040 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 msgr2=0x7f5c4c1385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c3a0 tx=0x7f5c3c00c860 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 shutdown_connections 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c38066f90 0x7f5c38069450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c4c0c75f0 0x7f5c4c0c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c4c15b250 0x7f5c4c1385c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 --2- 172.21.15.82:0/126078844 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c34005220 0x7f5c4c0ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 >> 172.21.15.82:0/126078844 conn(0x7f5c4c0b11a0 msgr2=0x7f5c4c0b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 shutdown_connections 2024-04-25T10:04:29.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:29.891+0000 7f5c52e8d640 1 -- 172.21.15.82:0/126078844 wait complete. 2024-04-25T10:04:29.909 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:04:30.041 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:04:30.041 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:04:30.042 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:04:30.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 Processor -- start 2024-04-25T10:04:30.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- start start 2024-04-25T10:04:30.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a029380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a0af880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62a013090 con 0x55f62a02af20 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62a012df0 con 0x55f62a0b7770 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f62a013330 con 0x55f62a0b6e50 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36486/0 (socket says 172.21.15.82:36486) 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 -- 172.21.15.82:0/3702101404 learned_addr learned my addr 172.21.15.82:0/3702101404 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 -- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 msgr2=0x55f62a029380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a029380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 -- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 msgr2=0x55f62a0af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a0af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 -- 172.21.15.82:0/3702101404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f62a0b02e0 con 0x55f62a02af20 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009640 tx=0x7f51a4009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8dc4856c8ee3df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b151d640 1 -- 172.21.15.82:0/3702101404 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40066c0 con 0x55f62a02af20 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b151d640 1 -- 172.21.15.82:0/3702101404 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4006860 con 0x55f62a02af20 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b151d640 1 -- 172.21.15.82:0/3702101404 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4006c50 con 0x55f62a02af20 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- 172.21.15.82:0/3702101404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 msgr2=0x55f62a0b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009640 tx=0x7f51a4009b00 comp rx=0 tx=0).stop 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- 172.21.15.82:0/3702101404 shutdown_connections 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a029380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a0af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- 172.21.15.82:0/3702101404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0b1200 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- 172.21.15.82:0/3702101404 >> 172.21.15.82:0/3702101404 conn(0x55f62a00dab0 msgr2=0x55f629fe6540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- 172.21.15.82:0/3702101404 shutdown_connections 2024-04-25T10:04:30.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- 172.21.15.82:0/3702101404 wait complete. 2024-04-25T10:04:30.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 Processor -- start 2024-04-25T10:04:30.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- start start 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0253a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a0230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a023630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62a0b0820 con 0x55f62a02af20 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f62a0b0580 con 0x55f62a0b7770 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b5202080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f62a0b0ac0 con 0x55f62a0b6e50 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0253a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36498/0 (socket says 172.21.15.82:36498) 2024-04-25T10:04:30.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b2d20640 1 -- 172.21.15.82:0/3938538783 learned_addr learned my addr 172.21.15.82:0/3938538783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.079+0000 7f51b251f640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a0230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a023630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 -- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 msgr2=0x55f62a0230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55f62a0b6e50 0x55f62a0230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 -- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 msgr2=0x55f62a023630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a023630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 -- 172.21.15.82:0/3938538783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f62a024480 con 0x55f62a02af20 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55f62a0b7770 0x55f62a023630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b2d20640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0253a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005e40 tx=0x7f51a4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 -- 172.21.15.82:0/3938538783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a4021020 con 0x55f62a02af20 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 -- 172.21.15.82:0/3938538783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a4012a80 con 0x55f62a02af20 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f62a024720 con 0x55f62a02af20 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 -- 172.21.15.82:0/3938538783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a40066c0 con 0x55f62a02af20 2024-04-25T10:04:30.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f62a024c60 con 0x55f62a02af20 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 -- 172.21.15.82:0/3938538783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51a4024020 con 0x55f62a02af20 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f51a0065970 0x7f51a0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f519bfff640 1 -- 172.21.15.82:0/3938538783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51a40855d0 con 0x55f62a02af20 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b251f640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f51a0065970 0x7f51a0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55f62a0aacc0 0x55f62a0ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e52) v9 -- 0x55f62a0ab5e0 con 0x55f62a0aacc0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55f62a0aacc0 0x55f62a0ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b251f640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f51a0065970 0x7f51a0067e30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51a80030b0 tx=0x7f51a8005820 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55f62a0aacc0 0x55f62a0ab0a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 -- 172.21.15.82:0/3938538783 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f51ac0aa960 con 0x55f62a0aacc0 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55f62a0aacc0 msgr2=0x55f62a0ab0a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55f62a0aacc0 0x55f62a0ab0a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f51a0065970 msgr2=0x7f51a0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f51a0065970 0x7f51a0067e30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f51a80030b0 tx=0x7f51a8005820 comp rx=0 tx=0).stop 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 msgr2=0x55f62a0253a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 --2- 172.21.15.82:0/3938538783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55f62a02af20 0x55f62a0253a0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005e40 tx=0x7f51a4003030 comp rx=0 tx=0).stop 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b3521640 1 -- 172.21.15.82:0/3938538783 reap_dead start 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 shutdown_connections 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 >> 172.21.15.82:0/3938538783 conn(0x55f62a00dab0 msgr2=0x55f62a0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 shutdown_connections 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.083+0000 7f51b5202080 1 -- 172.21.15.82:0/3938538783 wait complete. 2024-04-25T10:04:30.093 INFO:teuthology.orchestra.run.smithi082.stdout:Yalphaparent_a 2024-04-25T10:04:30.094 DEBUG:teuthology.orchestra.run.smithi082:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: "ancestors": [ 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1099511627776, 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "alpha", 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 2 2024-04-25T10:04:30.262 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1, 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "parent_a", 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 4 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: "pool": 8, 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: "old_pools": [] 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:30.263 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.265 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T10:04:30.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 Processor -- start 2024-04-25T10:04:30.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- start start 2024-04-25T10:04:30.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e2f570 0x55dbf3f04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 0x55dbf3f9be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e61b70 con 0x55dbf3f071f0 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e619f0 con 0x55dbf3e7d360 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e61cf0 con 0x55dbf3e2f570 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36512/0 (socket says 172.21.15.82:36512) 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 -- 172.21.15.82:0/4010512030 learned_addr learned my addr 172.21.15.82:0/4010512030 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 -- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e2f570 msgr2=0x55dbf3f04a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5183d8b640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 0x55dbf3f9be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f518458c640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e2f570 0x55dbf3f04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e2f570 0x55dbf3f04a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 -- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 msgr2=0x55dbf3f9be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 0x55dbf3f9be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 -- 172.21.15.82:0/4010512030 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbf3f06100 con 0x55dbf3f071f0 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5183d8b640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 0x55dbf3f9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5184d8d640 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5174009870 tx=0x7f5174005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a50a562bff288cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5182d89640 1 -- 172.21.15.82:0/4010512030 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174019070 con 0x55dbf3f071f0 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5182d89640 1 -- 172.21.15.82:0/4010512030 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174008d40 con 0x55dbf3f071f0 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5182d89640 1 -- 172.21.15.82:0/4010512030 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740146c0 con 0x55dbf3f071f0 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- 172.21.15.82:0/4010512030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 msgr2=0x55dbf3f9c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f5174009870 tx=0x7f5174005a10 comp rx=0 tx=0).stop 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- 172.21.15.82:0/4010512030 shutdown_connections 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3f9c3b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e7d360 0x55dbf3f9be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 --2- 172.21.15.82:0/4010512030 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e2f570 0x55dbf3f04a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- 172.21.15.82:0/4010512030 >> 172.21.15.82:0/4010512030 conn(0x55dbf3e35c90 msgr2=0x55dbf3e5f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- 172.21.15.82:0/4010512030 shutdown_connections 2024-04-25T10:04:30.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- 172.21.15.82:0/4010512030 wait complete. 2024-04-25T10:04:30.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 Processor -- start 2024-04-25T10:04:30.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.303+0000 7f5186a6e080 1 -- start start 2024-04-25T10:04:30.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e2f570 0x55dbf3eefcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e7d360 0x55dbf3f9fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3eec540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3f04590 con 0x55dbf3f071f0 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3e732c0 con 0x55dbf3e2f570 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dbf3fa02a0 con 0x55dbf3e7d360 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3eec540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3eec540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36520/0 (socket says 172.21.15.82:36520) 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 -- 172.21.15.82:0/1133748741 learned_addr learned my addr 172.21.15.82:0/1133748741 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5183d8b640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e7d360 0x55dbf3f9fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f518458c640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e2f570 0x55dbf3eefcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 -- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e7d360 msgr2=0x55dbf3f9fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55dbf3e7d360 0x55dbf3f9fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 -- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e2f570 msgr2=0x55dbf3eefcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55dbf3e2f570 0x55dbf3eefcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 -- 172.21.15.82:0/1133748741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dbf3fa0510 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5184d8d640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3eec540 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f51740146e0 tx=0x7f5174003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 -- 172.21.15.82:0/1133748741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174023020 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 -- 172.21.15.82:0/1133748741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517401a040 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dbf3fa07b0 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 -- 172.21.15.82:0/1133748741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174019070 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dbf3fa0cf0 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 -- 172.21.15.82:0/1133748741 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5174026020 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5164065970 0x7f5164067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5181d87640 1 -- 172.21.15.82:0/1133748741 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5174085800 con 0x55dbf3f071f0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f518458c640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5164065970 0x7f5164067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55dbf3fa2440 0x55dbf3fac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52) v9 -- 0x55dbf3fac860 con 0x55dbf3fa2440 2024-04-25T10:04:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5183d8b640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55dbf3fa2440 0x55dbf3fac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f518458c640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5164065970 0x7f5164067e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f517c005fd0 tx=0x7f517c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.307+0000 7f5183d8b640 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55dbf3fa2440 0x55dbf3fac320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5183d8b640 1 -- 172.21.15.82:0/1133748741 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v52'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f516c0ae030 con 0x55dbf3fa2440 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55dbf3fa2440 msgr2=0x55dbf3fac320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55dbf3fa2440 0x55dbf3fac320 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5164065970 msgr2=0x7f5164067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5164065970 0x7f5164067e30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f517c005fd0 tx=0x7f517c007040 comp rx=0 tx=0).stop 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 msgr2=0x55dbf3eec540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 --2- 172.21.15.82:0/1133748741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55dbf3f071f0 0x55dbf3eec540 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f51740146e0 tx=0x7f5174003030 comp rx=0 tx=0).stop 2024-04-25T10:04:30.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5184d8d640 1 -- 172.21.15.82:0/1133748741 reap_dead start 2024-04-25T10:04:30.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 shutdown_connections 2024-04-25T10:04:30.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 >> 172.21.15.82:0/1133748741 conn(0x55dbf3e35c90 msgr2=0x55dbf3f01760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 shutdown_connections 2024-04-25T10:04:30.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.311+0000 7f5186a6e080 1 -- 172.21.15.82:0/1133748741 wait complete. 2024-04-25T10:04:30.318 DEBUG:teuthology.orchestra.run.smithi082:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:04:30.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:30.362 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T10:04:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 Processor -- start 2024-04-25T10:04:30.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 -- start start 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f394406ac90 0x7f394406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394415cb80 con 0x7f39440b0310 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394406b5b0 con 0x7f39440b2ac0 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f394b30b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440afe10 con 0x7f394406ac90 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f394406ac90 0x7f394406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f394406ac90 0x7f394406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36534/0 (socket says 172.21.15.82:36534) 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39498a8640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f39440b0310 0x7f39440b0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42914/0 (socket says 172.21.15.82:42914) 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 -- 172.21.15.82:0/986605331 learned_addr learned my addr 172.21.15.82:0/986605331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 -- 172.21.15.82:0/986605331 >> v1:172.21.15.196:6789/0 conn(0x7f39440b2ac0 legacy=0x7f39440ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 -- 172.21.15.82:0/986605331 >> v1:172.21.15.82:6789/0 conn(0x7f39440b0310 legacy=0x7f39440b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f393bfff640 1 -- 172.21.15.82:0/986605331 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967731989 0 0) 0x7f394415cb80 con 0x7f39440b0310 2024-04-25T10:04:30.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.475+0000 7f39490a7640 1 -- 172.21.15.82:0/986605331 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440cd300 con 0x7f394406ac90 2024-04-25T10:04:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 --2- 172.21.15.82:0/986605331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394406ac90 0x7f394406b070 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3934009870 tx=0x7f393401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10096d7b761134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f393bfff640 1 -- 172.21.15.82:0/986605331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340a95f0 con 0x7f394406ac90 2024-04-25T10:04:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f393bfff640 1 -- 172.21.15.82:0/986605331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340a9790 con 0x7f394406ac90 2024-04-25T10:04:30.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f393bfff640 1 -- 172.21.15.82:0/986605331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340a9b60 con 0x7f394406ac90 2024-04-25T10:04:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- 172.21.15.82:0/986605331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394406ac90 msgr2=0x7f394406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 --2- 172.21.15.82:0/986605331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394406ac90 0x7f394406b070 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3934009870 tx=0x7f393401c530 comp rx=0 tx=0).stop 2024-04-25T10:04:30.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- 172.21.15.82:0/986605331 shutdown_connections 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 --2- 172.21.15.82:0/986605331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f394406ac90 0x7f394406b070 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- 172.21.15.82:0/986605331 >> 172.21.15.82:0/986605331 conn(0x7f39440b1fa0 msgr2=0x7f39440b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- 172.21.15.82:0/986605331 shutdown_connections 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- 172.21.15.82:0/986605331 wait complete. 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 Processor -- start 2024-04-25T10:04:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- start start 2024-04-25T10:04:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39440b2ac0 0x7f39440c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f394415e380 0x7f3944152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944169480 con 0x7f39440b0310 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944169300 con 0x7f39440b2ac0 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f394b30b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3944169600 con 0x7f394415e380 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36540/0 (socket says 172.21.15.82:36540) 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 -- 172.21.15.82:0/3323132863 learned_addr learned my addr 172.21.15.82:0/3323132863 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39488a6640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39440b2ac0 0x7f39440c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39498a8640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f394415e380 0x7f3944152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 -- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f394415e380 msgr2=0x7f3944152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f394415e380 0x7f3944152410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 -- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39440b2ac0 msgr2=0x7f39440c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.479+0000 7f39490a7640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39440b2ac0 0x7f39440c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f39490a7640 1 -- 172.21.15.82:0/3323132863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39440c95b0 con 0x7f39440b0310 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f39490a7640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f393401a160 tx=0x7f393401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 -- 172.21.15.82:0/3323132863 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340b1800 con 0x7f39440b0310 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 -- 172.21.15.82:0/3323132863 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340b19a0 con 0x7f39440b0310 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39440c9880 con 0x7f39440b0310 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 -- 172.21.15.82:0/3323132863 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340b1d70 con 0x7f39440b0310 2024-04-25T10:04:30.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39440c9e40 con 0x7f39440b0310 2024-04-25T10:04:30.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 -- 172.21.15.82:0/3323132863 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39340bc450 con 0x7f39440b0310 2024-04-25T10:04:30.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 0x7f3928069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f39488a6640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 0x7f3928069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f3939ffb640 1 -- 172.21.15.82:0/3323132863 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f393411fc70 con 0x7f39440b0310 2024-04-25T10:04:30.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.483+0000 7f39488a6640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 0x7f3928069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f394415bc20 tx=0x7f3940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 Processor -- start 2024-04-25T10:04:30.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- start start 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e65b0 0x563fdc09bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc09c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbf82f40 con 0x563fdc0e5c90 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbeea070 con 0x563fdc09c290 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdbe721a0 con 0x563fdc0e65b0 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36550/0 (socket says 172.21.15.82:36550) 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 -- 172.21.15.82:0/3075864831 learned_addr learned my addr 172.21.15.82:0/3075864831 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39498a8640 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc09c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 -- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e65b0 msgr2=0x563fdc09bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e65b0 0x563fdc09bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 -- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 msgr2=0x563fdc09c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc09c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 -- 172.21.15.82:0/3075864831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fdbe70510 con 0x563fdc0e5c90 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39498a8640 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc09c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f39490a7640 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f39340f0410 tx=0x7f39340192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd37415e186f6229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f392effd640 1 -- 172.21.15.82:0/3075864831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340f0b80 con 0x563fdc0e5c90 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f392effd640 1 -- 172.21.15.82:0/3075864831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340f0480 con 0x563fdc0e5c90 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f392effd640 1 -- 172.21.15.82:0/3075864831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39340f45d0 con 0x563fdc0e5c90 2024-04-25T10:04:30.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- 172.21.15.82:0/3075864831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 msgr2=0x563fdc0e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f39340f0410 tx=0x7f39340192d0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- 172.21.15.82:0/3075864831 shutdown_connections 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e65b0 0x563fdc09bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc09c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 --2- 172.21.15.82:0/3075864831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc0e6070 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- 172.21.15.82:0/3075864831 >> 172.21.15.82:0/3075864831 conn(0x563fdc098240 msgr2=0x563fdc09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- 172.21.15.82:0/3075864831 shutdown_connections 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- 172.21.15.82:0/3075864831 wait complete. 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 -- start start 2024-04-25T10:04:30.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.487+0000 7f394e478000 1 Processor -- start 2024-04-25T10:04:30.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc0a1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 0x563fdc0d2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbea46c0 con 0x563fdc0e65b0 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbee6110 con 0x563fdc09c290 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdbf82f40 con 0x563fdc0e5c90 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 10 client.0 _ll_register_callbacks cb 0x563fdc087570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36564/0 (socket says 172.21.15.82:36564) 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 -- 172.21.15.82:0/787888505 learned_addr learned my addr 172.21.15.82:0/787888505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39488a6640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 0x563fdc0d2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39490a7640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc0a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 -- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 msgr2=0x563fdc0d2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 0x563fdc0d2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 -- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 msgr2=0x563fdc0a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc0a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 -- 172.21.15.82:0/787888505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fdbe723b0 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39488a6640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 0x563fdc0d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39498a8640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e4f0 tx=0x7f393c00e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1183d352853ea0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c019060 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f393c014c80 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39340f0480 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f393c0070d0 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 10 client.6180 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x563fdbf6be00 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340f0480 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f393c007270 con 0x563fdc0e65b0 2024-04-25T10:04:30.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 10 client.6180 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 10 client.6180 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 20 client.6180 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc0ec4e0 0x563fdc0ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 4 client.6180 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563fdbe85d60 con 0x563fdc0ec4e0 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39490a7640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc0ec4e0 0x563fdc0ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 10 client.6180 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f39490a7640 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc0ec4e0 0x563fdc0ec8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc99dc0d6017e6c3 server_cookie=3baf9a2c1ca816e3 in_seq=0 out_seq=0 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 1 -- 172.21.15.82:0/787888505 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563fdbe85d60 con 0x563fdc0ec4e0 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 10 client.6180 handle_command_reply: tid=0 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 client.6180 shutdown 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 msgr2=0x563fdc0d3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f393c00e4f0 tx=0x7f393c00e9b0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 shutdown_connections 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc0ec4e0 0x563fdc0ec8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc0e5c90 0x563fdc0d2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e65b0 0x563fdc0d3450 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f3923fff640 0 client.6180 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 --2- 172.21.15.82:0/787888505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc09c290 0x563fdc0a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 >> 172.21.15.82:0/787888505 conn(0x563fdc098240 msgr2=0x563fdc09aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 shutdown_connections 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 1 -- 172.21.15.82:0/787888505 wait complete. 2024-04-25T10:04:30.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.491+0000 7f394e478000 20 client.6180 trim_cache size 0 max 16384 2024-04-25T10:04:30.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 Processor -- start 2024-04-25T10:04:30.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 -- start start 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e400af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400afbe0 con 0x7f3e400ae210 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e400ae680 con 0x7f3e4015a7e0 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e486c5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400aeed0 con 0x7f3e4015abc0 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46c62640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3e400ae210 0x7f3e400b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42926/0 (socket says 172.21.15.82:42926) 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46c62640 1 -- 172.21.15.82:0/3907248815 learned_addr learned my addr 172.21.15.82:0/3907248815 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453003131 0 0) 0x7f3e400afbe0 con 0x7f3e400ae210 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e2c005740 con 0x7f3e400ae210 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e400af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 667035074 0 0) 0x7f3e2c005740 con 0x7f3e400ae210 2024-04-25T10:04:30.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 msgr2=0x7f3e400af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e400af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 >> v1:172.21.15.196:6789/0 conn(0x7f3e4015a7e0 legacy=0x7f3e400adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4006ff20 con 0x7f3e400ae210 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3e38001a00 con 0x7f3e400ae210 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 >> v1:172.21.15.82:6789/0 conn(0x7f3e400ae210 legacy=0x7f3e400b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e2c005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e2c006230 0x7f3e2c006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2c006f60 0x7f3e2c0073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c00dd70 con 0x7f3e2c006230 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c00df20 con 0x7f3e2c006f60 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c00e120 con 0x7f3e4015abc0 2024-04-25T10:04:30.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e2c006230 0x7f3e2c006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46461640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e2c005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46c62640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2c006f60 0x7f3e2c0073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 -- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 msgr2=0x7f3e2c005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e2c005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 -- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2c006f60 msgr2=0x7f3e2c0073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2c006f60 0x7f3e2c0073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e45c60640 1 -- 172.21.15.82:0/3907248815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c005740 con 0x7f3e2c006230 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46461640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e4015abc0 0x7f3e2c005c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.499+0000 7f3e46c62640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e2c006f60 0x7f3e2c0073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e45c60640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e2c006230 0x7f3e2c006a20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ba30 tx=0x7f3e3001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17339f9c9f999230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300ae070 con 0x7f3e2c006230 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e300a9680 con 0x7f3e2c006230 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e4545f640 1 -- 172.21.15.82:0/3907248815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000b600 con 0x7f3e2c006230 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- 172.21.15.82:0/3907248815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e2c006230 msgr2=0x7f3e2c006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 --2- 172.21.15.82:0/3907248815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e2c006230 0x7f3e2c006a20 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000ba30 tx=0x7f3e3001c6a0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e46c62640 1 -- 172.21.15.82:0/3907248815 reap_dead start 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- 172.21.15.82:0/3907248815 shutdown_connections 2024-04-25T10:04:30.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- 172.21.15.82:0/3907248815 >> 172.21.15.82:0/3907248815 conn(0x7f3e400b5830 msgr2=0x7f3e400b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- 172.21.15.82:0/3907248815 shutdown_connections 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- 172.21.15.82:0/3907248815 wait complete. 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 Processor -- start 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.503+0000 7f3e486c5640 1 -- start start 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e40146f40 0x7f3e4015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4015d560 0x7f3e4014eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e4014f3f0 0x7f3e40148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400ae910 con 0x7f3e4014f3f0 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400b4900 con 0x7f3e4015d560 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400aebf0 con 0x7f3e40146f40 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e4014f3f0 0x7f3e40148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e4014f3f0 0x7f3e40148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36568/0 (socket says 172.21.15.82:36568) 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 -- 172.21.15.82:0/2272066247 learned_addr learned my addr 172.21.15.82:0/2272066247 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46461640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e40146f40 0x7f3e4015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e45c60640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4015d560 0x7f3e4014eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 -- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e40146f40 msgr2=0x7f3e4015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e40146f40 0x7f3e4015cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 -- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4015d560 msgr2=0x7f3e4014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e4015d560 0x7f3e4014eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 -- 172.21.15.82:0/2272066247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400aff00 con 0x7f3e4014f3f0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46c62640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e4014f3f0 0x7f3e40148fb0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f3e400d0f60 tx=0x7f3e38010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3801a070 con 0x7f3e4014f3f0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e38011040 con 0x7f3e4014f3f0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- 172.21.15.82:0/2272066247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e400b01a0 con 0x7f3e4014f3f0 2024-04-25T10:04:30.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38010ea0 con 0x7f3e4014f3f0 2024-04-25T10:04:30.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- 172.21.15.82:0/2272066247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e400b0760 con 0x7f3e4014f3f0 2024-04-25T10:04:30.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- 172.21.15.82:0/2272066247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f3e40068340 con 0x7f3e4014f3f0 2024-04-25T10:04:30.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 Processor -- start 2024-04-25T10:04:30.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 -- start start 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e3801d070 con 0x7f3e4014f3f0 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc1468f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc147370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944201100 con 0x563fdc0e5c90 2024-04-25T10:04:30.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3944200300 con 0x563fdc0e65b0 2024-04-25T10:04:30.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394415d2e0 con 0x563fdc09c290 2024-04-25T10:04:30.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e24066f60 0x7f3e24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36578/0 (socket says 172.21.15.82:36578) 2024-04-25T10:04:30.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 -- 172.21.15.82:0/253907356 learned_addr learned my addr 172.21.15.82:0/253907356 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46461640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e24066f60 0x7f3e24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e38088330 con 0x7f3e4014f3f0 2024-04-25T10:04:30.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39498a8640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc147370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39490a7640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc1468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f3e380887e0 con 0x7f3e4014f3f0 2024-04-25T10:04:30.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 -- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 msgr2=0x563fdc1468f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc1468f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 -- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 msgr2=0x563fdc147370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc147370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 -- 172.21.15.82:0/253907356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563fdc0e63e0 con 0x563fdc0e5c90 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39490a7640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc1468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f39488a6640 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3940005350 tx=0x7f3940011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5534ea76b5480966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e46461640 1 --2- 172.21.15.82:0/2272066247 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e24066f60 0x7f3e24069420 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e4015c340 tx=0x7f3e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f3e486c5640 1 -- 172.21.15.82:0/2272066247 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f3e400aeed0 con 0x7f3e4014f3f0 2024-04-25T10:04:30.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f392e7fc640 1 -- 172.21.15.82:0/253907356 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940005440 con 0x563fdc0e5c90 2024-04-25T10:04:30.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f392e7fc640 1 -- 172.21.15.82:0/253907356 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000e6b0 con 0x563fdc0e5c90 2024-04-25T10:04:30.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.507+0000 7f392e7fc640 1 -- 172.21.15.82:0/253907356 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000ea60 con 0x563fdc0e5c90 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/253907356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 msgr2=0x563fdc146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3940005350 tx=0x7f3940011000 comp rx=0 tx=0).stop 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/253907356 shutdown_connections 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc147370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc146e30 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- 172.21.15.82:0/253907356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc1468f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/253907356 >> 172.21.15.82:0/253907356 conn(0x563fdc098240 msgr2=0x563fdc09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/253907356 shutdown_connections 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/253907356 wait complete. 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- start start 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 Processor -- start 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc0eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc107410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbeabe10 con 0x563fdc0e5c90 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563fdbef0c80 con 0x563fdc0e65b0 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563fdbe8cbd0 con 0x563fdc09c290 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 10 client.0 _ll_register_callbacks cb 0x7f39340094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36592/0 (socket says 172.21.15.82:36592) 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 -- 172.21.15.82:0/3827811609 learned_addr learned my addr 172.21.15.82:0/3827811609 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39490a7640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc0eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 -- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 msgr2=0x563fdc0eb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc0eb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 -- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 msgr2=0x563fdc107410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 -- 172.21.15.82:0/3827811609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563fdbf41c50 con 0x563fdc0e5c90 2024-04-25T10:04:30.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39488a6640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3940024040 tx=0x7f3940000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e805befa026d4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394000eea0 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3940010040 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f394000e6b0 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3940005440 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 10 client.6192 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x563fdbeb2120 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394000e6b0 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3940021450 con 0x563fdc0e5c90 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 10 client.6192 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 10 client.6192 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:04:30.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 20 client.6192 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc14a510 0x563fdc14a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 4 client.6192 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x563fdbef75b0 con 0x563fdc14a510 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39490a7640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc14a510 0x563fdc14a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f3923fff640 10 client.6192 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.511+0000 7f39490a7640 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc14a510 0x563fdc14a920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1a76f8b68b4f99 server_cookie=cee4ca4761a873e1 in_seq=0 out_seq=0 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f3923fff640 1 -- 172.21.15.82:0/3827811609 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563fdbef75b0 con 0x563fdc14a510 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f3923fff640 10 client.6192 handle_command_reply: tid=0 2024-04-25T10:04:30.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 client.6192 shutdown 2024-04-25T10:04:30.528 INFO:tasks.ceph.mds.a.smithi082.stderr:2024-04-25T10:04:30.515+0000 7fa418765640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6141=0-4194304@1} caps={6141=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6141 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x556b6d765700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 msgr2=0x563fdc106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3940024040 tx=0x7f3940000ee0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 shutdown_connections 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x563fdc14a510 0x563fdc14a920 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563fdc0e65b0 0x563fdc107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563fdc0e5c90 0x563fdc106ed0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f3923fff640 0 client.6192 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 --2- 172.21.15.82:0/3827811609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563fdc09c290 0x563fdc0eb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 >> 172.21.15.82:0/3827811609 conn(0x563fdc098240 msgr2=0x563fdc09aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 shutdown_connections 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 1 -- 172.21.15.82:0/3827811609 wait complete. 2024-04-25T10:04:30.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394e478000 20 client.6192 trim_cache size 0 max 16384 2024-04-25T10:04:30.530 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:30.530 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:04:30.530 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "24c246ab-77a4-4777-abbe-a927f0dd9eb8", 2024-04-25T10:04:30.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:04:30.530 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 msgr2=0x7f3928069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 0x7f3928069450 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f394415bc20 tx=0x7f3940009040 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 msgr2=0x7f394415a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f393401a160 tx=0x7f393401ad70 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 shutdown_connections 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3928066f90 0x7f3928069450 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f394415e380 0x7f3944152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39440b2ac0 0x7f39440c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 --2- 172.21.15.82:0/3323132863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39440b0310 0x7f394415a550 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 >> 172.21.15.82:0/3323132863 conn(0x7f39440b1fa0 msgr2=0x7f39440aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 shutdown_connections 2024-04-25T10:04:30.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.515+0000 7f394b30b640 1 -- 172.21.15.82:0/3323132863 wait complete. 2024-04-25T10:04:30.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:30.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:04:30.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 Processor -- start 2024-04-25T10:04:30.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 -- start start 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f873c15b950 0x7f873c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f873c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873c15bd30 con 0x7f873c0cf420 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873c0d0510 con 0x7f873c15b950 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8743063640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0cc0d0 con 0x7f873c0cfb80 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f873c0cf420 0x7f873c0cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42932/0 (socket says 172.21.15.82:42932) 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873bfff640 1 -- 172.21.15.82:0/2371230690 learned_addr learned my addr 172.21.15.82:0/2371230690 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:04:30.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 --2- 172.21.15.82:0/2371230690 >> v2:172.21.15.196:3300/0 conn(0x7f873c15b950 0x7f873c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031622272 0 0) 0x7f873c15bd30 con 0x7f873c0cf420 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8741600640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f873c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f872c003530 con 0x7f873c0cf420 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3364492954 0 0) 0x7f872c003530 con 0x7f873c0cf420 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 msgr2=0x7f873c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f873c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 >> v2:172.21.15.196:3300/0 conn(0x7f873c15b950 msgr2=0x7f873c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 --2- 172.21.15.82:0/2371230690 >> v2:172.21.15.196:3300/0 conn(0x7f873c15b950 0x7f873c09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c09ab10 con 0x7f873c0cf420 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8728001680 con 0x7f873c0cf420 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 >> v1:172.21.15.82:6789/0 conn(0x7f873c0cf420 legacy=0x7f873c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f872c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f872c004810 0x7f872c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f872c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0d0510 con 0x7f872c003a30 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0cc0d0 con 0x7f872c004810 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872c00b9d0 con 0x7f873c0cfb80 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f872c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873bfff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f872c004810 0x7f872c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8741600640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f872c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 -- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 msgr2=0x7f872c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f872c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 -- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f872c004810 msgr2=0x7f872c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f872c004810 0x7f872c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8740dff640 1 -- 172.21.15.82:0/2371230690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872c003530 con 0x7f872c003a30 2024-04-25T10:04:30.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f8741600640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0cfb80 0x7f872c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.639+0000 7f873bfff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f872c004810 0x7f872c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8740dff640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f872c003e50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8730009b10 tx=0x7f873001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26288efa48e3ecb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300096b0 con 0x7f872c003a30 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87300a8410 con 0x7f872c003a30 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f873b7fe640 1 -- 172.21.15.82:0/2371230690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87300a8800 con 0x7f872c003a30 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- 172.21.15.82:0/2371230690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 msgr2=0x7f872c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 --2- 172.21.15.82:0/2371230690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f872c003e50 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8730009b10 tx=0x7f873001cab0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8741600640 1 -- 172.21.15.82:0/2371230690 reap_dead start 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- 172.21.15.82:0/2371230690 shutdown_connections 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- 172.21.15.82:0/2371230690 >> 172.21.15.82:0/2371230690 conn(0x7f873c0b30b0 msgr2=0x7f873c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- 172.21.15.82:0/2371230690 shutdown_connections 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- 172.21.15.82:0/2371230690 wait complete. 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 Processor -- start 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- start start 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 0x7f873c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0af640 0x7f873c0b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c06faa0 con 0x7f872c003a30 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0cf850 con 0x7f873c0c8690 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8743063640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c06fc20 con 0x7f873c0af640 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8740dff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8740dff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36602/0 (socket says 172.21.15.82:36602) 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.643+0000 7f8740dff640 1 -- 172.21.15.82:0/2061901567 learned_addr learned my addr 172.21.15.82:0/2061901567 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 -- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0af640 msgr2=0x7f873c0b5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0af640 0x7f873c0b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 -- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 msgr2=0x7f873c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f873bfff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 0x7f873c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 0x7f873c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 -- 172.21.15.82:0/2061901567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873c0c5a60 con 0x7f872c003a30 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f873bfff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 0x7f873c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8740dff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f873001d020 tx=0x7f8730019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 -- 172.21.15.82:0/2061901567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000f000 con 0x7f872c003a30 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 -- 172.21.15.82:0/2061901567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87300a8aa0 con 0x7f872c003a30 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873c0c5d40 con 0x7f872c003a30 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 -- 172.21.15.82:0/2061901567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730004020 con 0x7f872c003a30 2024-04-25T10:04:30.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873c0c6300 con 0x7f872c003a30 2024-04-25T10:04:30.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 -- 172.21.15.82:0/2061901567 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87300041c0 con 0x7f872c003a30 2024-04-25T10:04:30.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 0x7f8720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87397fa640 1 -- 172.21.15.82:0/2061901567 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f873011f340 con 0x7f872c003a30 2024-04-25T10:04:30.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f873bfff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 0x7f8720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f873bfff640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 0x7f8720069450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8728001830 tx=0x7f8728009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87461d0000 1 Processor -- start 2024-04-25T10:04:30.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.647+0000 7f87461d0000 1 -- start start 2024-04-25T10:04:30.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb19d0 0x55e280eb1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280e68480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e280cb4ff0 con 0x55e280eb22f0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e280c3d150 con 0x55e280e680a0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e280cb1090 con 0x55e280eb19d0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36604/0 (socket says 172.21.15.82:36604) 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1828954289 learned_addr learned my addr 172.21.15.82:0/1828954289 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f8741600640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280e68480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f8740dff640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb19d0 0x55e280eb1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb19d0 msgr2=0x55e280eb1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb19d0 0x55e280eb1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 msgr2=0x55e280e68480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280e68480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1828954289 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e280e4b0d0 con 0x55e280eb22f0 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f8741600640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280e68480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f873c0b1770 tx=0x7f8728011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3153ec6234393039 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87267fc640 1 -- 172.21.15.82:0/1828954289 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872800ea10 con 0x55e280eb22f0 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87267fc640 1 -- 172.21.15.82:0/1828954289 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872800ebb0 con 0x55e280eb22f0 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87267fc640 1 -- 172.21.15.82:0/1828954289 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728013480 con 0x55e280eb22f0 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- 172.21.15.82:0/1828954289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 msgr2=0x55e280e67b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f873c0b1770 tx=0x7f8728011000 comp rx=0 tx=0).stop 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- 172.21.15.82:0/1828954289 shutdown_connections 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280e68480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb22f0 0x55e280e67b60 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- 172.21.15.82:0/1828954289 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb19d0 0x55e280eb1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- 172.21.15.82:0/1828954289 >> 172.21.15.82:0/1828954289 conn(0x55e280e64020 msgr2=0x55e280e66440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- 172.21.15.82:0/1828954289 shutdown_connections 2024-04-25T10:04:30.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- 172.21.15.82:0/1828954289 wait complete. 2024-04-25T10:04:30.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- start start 2024-04-25T10:04:30.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 Processor -- start 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280e680a0 0x55e280e967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280eb22f0 0x55e280e97270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e280c79250 con 0x55e280eb19d0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e280c6f690 con 0x55e280eb22f0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e280c3da40 con 0x55e280e680a0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f87461d0000 10 client.0 _ll_register_callbacks cb 0x55e280e52e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36610/0 (socket says 172.21.15.82:36610) 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1187594864 learned_addr learned my addr 172.21.15.82:0/1187594864 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f8741600640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280eb22f0 0x55e280e97270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f8740dff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280e680a0 0x55e280e967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280e680a0 msgr2=0x55e280e967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280e680a0 0x55e280e967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280eb22f0 msgr2=0x55e280e97270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280eb22f0 0x55e280e97270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 -- 172.21.15.82:0/1187594864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e280d621c0 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f873bfff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8728009010 tx=0x7f8728007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9fbfb45946ef4bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f871b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728013cc0 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.651+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f872802ec70 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f872800ebb0 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8728013e60 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 10 client.6204 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e280d5e170 con 0x55e280eb19d0 2024-04-25T10:04:30.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872800ebb0 con 0x55e280eb19d0 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8728021450 con 0x55e280eb19d0 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 10 client.6204 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 10 client.6204 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 20 client.6204 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec1c80 0x55e280ec4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 4 client.6204 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e280c50c60 con 0x55e280ec1c80 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f8740dff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec1c80 0x55e280ec4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 10 client.6204 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f8740dff640 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec1c80 0x55e280ec4070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb337a03747a126 server_cookie=80dedb0a4a83be9f in_seq=0 out_seq=0 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 1 -- 172.21.15.82:0/1187594864 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e280c50c60 con 0x55e280ec1c80 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 10 client.6204 handle_command_reply: tid=0 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 client.6204 shutdown 2024-04-25T10:04:30.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 msgr2=0x55e280e96d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8728009010 tx=0x7f8728007000 comp rx=0 tx=0).stop 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 shutdown_connections 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec1c80 0x55e280ec4070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280eb22f0 0x55e280e97270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f871b7fe640 0 client.6204 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280e96d30 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 --2- 172.21.15.82:0/1187594864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280e680a0 0x55e280e967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 >> 172.21.15.82:0/1187594864 conn(0x55e280e64020 msgr2=0x55e280e66ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 shutdown_connections 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 1 -- 172.21.15.82:0/1187594864 wait complete. 2024-04-25T10:04:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.655+0000 7f87461d0000 20 client.6204 trim_cache size 0 max 16384 2024-04-25T10:04:30.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 Processor -- start 2024-04-25T10:04:30.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- start start 2024-04-25T10:04:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f0dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280f0e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c201940 con 0x55e280eb19d0 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c200b40 con 0x55e280e680a0 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873c0c7870 con 0x55e280eb22f0 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36612/0 (socket says 172.21.15.82:36612) 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 -- 172.21.15.82:0/2778486180 learned_addr learned my addr 172.21.15.82:0/2778486180 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 -- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 msgr2=0x55e280f0e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280f0e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f8740dff640 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f0dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 -- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 msgr2=0x55e280f0dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f0dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 -- 172.21.15.82:0/2778486180 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e280cfbde0 con 0x55e280eb19d0 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f873bfff640 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f872800eef0 tx=0x7f8728012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec7f34e3750f533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f8725ffb640 1 -- 172.21.15.82:0/2778486180 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872803b030 con 0x55e280eb19d0 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f8725ffb640 1 -- 172.21.15.82:0/2778486180 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728021a20 con 0x55e280eb19d0 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f8725ffb640 1 -- 172.21.15.82:0/2778486180 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872800eb20 con 0x55e280eb19d0 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- 172.21.15.82:0/2778486180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 msgr2=0x55e280f0e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f872800eef0 tx=0x7f8728012000 comp rx=0 tx=0).stop 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- 172.21.15.82:0/2778486180 shutdown_connections 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280f0e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f0e420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 --2- 172.21.15.82:0/2778486180 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f0dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- 172.21.15.82:0/2778486180 >> 172.21.15.82:0/2778486180 conn(0x55e280e64020 msgr2=0x55e280ebefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- 172.21.15.82:0/2778486180 shutdown_connections 2024-04-25T10:04:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- 172.21.15.82:0/2778486180 wait complete. 2024-04-25T10:04:30.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 -- start start 2024-04-25T10:04:30.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.671+0000 7f87461d0000 1 Processor -- start 2024-04-25T10:04:30.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f11b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280ec3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e280c4fe00 con 0x55e280eb19d0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873c200540 con 0x55e280e680a0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e280cb4300 con 0x55e280eb22f0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 10 client.0 _ll_register_callbacks cb 0x55e280e33040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36620/0 (socket says 172.21.15.82:36620) 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 -- 172.21.15.82:0/1974111397 learned_addr learned my addr 172.21.15.82:0/1974111397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8741600640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280ec3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8740dff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f11b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 -- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 msgr2=0x55e280ec3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280ec3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 -- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 msgr2=0x55e280f11b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f11b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 -- 172.21.15.82:0/1974111397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e280c568e0 con 0x55e280eb19d0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8741600640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280ec3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8740dff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f11b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f873bfff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c10 tx=0x7f8728001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1827df9de2f4d81a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728044070 con 0x55e280eb19d0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(52..52 src has 1..52) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8728013ac0 con 0x55e280eb19d0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8728021a20 con 0x55e280eb19d0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f872804c050 con 0x55e280eb19d0 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 10 client.6210 fetch_fsmap learned FSMap version 24 2024-04-25T10:04:30.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55e280d0e8f0 con 0x55e280eb19d0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8728021a20 con 0x55e280eb19d0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f872804f020 con 0x55e280eb19d0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 10 client.6210 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 10 client.6210 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 20 client.6210 populate_metadata read hostname 'smithi082' 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec48f0 0x55e280ec4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 4 client.6210 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e280cc2530 con 0x55e280ec48f0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8740dff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec48f0 0x55e280ec4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 10 client.6210 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8740dff640 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec48f0 0x55e280ec4d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=202e30397007bfc8 server_cookie=401b515d0289643d in_seq=0 out_seq=0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 1 -- 172.21.15.82:0/1974111397 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55e280cc2530 con 0x55e280ec48f0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 10 client.6210 handle_command_reply: tid=0 2024-04-25T10:04:30.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 client.6210 shutdown 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 msgr2=0x55e280f12080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8728000c10 tx=0x7f8728001a50 comp rx=0 tx=0).stop 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 shutdown_connections 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x55e280ec48f0 0x55e280ec4d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e280eb22f0 0x55e280ec3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f871b7fe640 0 client.6210 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e280eb19d0 0x55e280f12080 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 --2- 172.21.15.82:0/1974111397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e280e680a0 0x55e280f11b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 >> 172.21.15.82:0/1974111397 conn(0x55e280e64020 msgr2=0x55e280eb00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 shutdown_connections 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 1 -- 172.21.15.82:0/1974111397 wait complete. 2024-04-25T10:04:30.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f87461d0000 20 client.6210 trim_cache size 0 max 16384 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "24c246ab-77a4-4777-abbe-a927f0dd9eb8": { 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "24c246ab-77a4-4777-abbe-a927f0dd9eb8", 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:30.686 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:04:30.687 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:04:30.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 msgr2=0x7f8720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 0x7f8720069450 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8728001830 tx=0x7f8728009040 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 msgr2=0x7f873c151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f873001d020 tx=0x7f8730019850 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 shutdown_connections 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8720066f90 0x7f8720069450 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f873c0af640 0x7f873c0b5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f873c0c8690 0x7f873c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 --2- 172.21.15.82:0/2061901567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f872c003a30 0x7f873c151d60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 >> 172.21.15.82:0/2061901567 conn(0x7f873c0b30b0 msgr2=0x7f873c0b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 shutdown_connections 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.675+0000 7f8743063640 1 -- 172.21.15.82:0/2061901567 wait complete. 2024-04-25T10:04:30.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:30.683+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T10:04:30.515603+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f3e380867e0 con 0x7f3e4014f3f0 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: cluster: 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: id: 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: health: HEALTH_OK 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: services: 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T10:04:30.689 INFO:teuthology.orchestra.run.smithi082.stdout: mgr: y(active, since 2m), standbys: x 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: data: 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: volumes: 1/1 healthy 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: objects: 24 objects, 579 KiB 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: usage: 233 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: pgs: 97 active+clean 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: io: 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: client: 1.3 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-04-25T10:04:30.690 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.515603+0000 mds.a [INF] scrub queued for path: / 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.515606+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.515610+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.516322+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.516336+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T10:04:30.691 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:30.517345+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:04:30.695 INFO:tasks.cephfs.filesystem:scrub status for tag:24c246ab-77a4-4777-abbe-a927f0dd9eb8 - {'path': '/', 'tag': '24c246ab-77a4-4777-abbe-a927f0dd9eb8', 'options': 'recursive,repair'} 2024-04-25T10:04:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:32.687+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T10:04:32.544900+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f3e38056c90 con 0x7f3e4014f3f0 2024-04-25T10:04:32.692 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:04:32.544900+0000 mds.a [INF] scrub summary: idle 2024-04-25T10:04:36.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:36.610 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:48.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:48.731+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 9 ==== log(2 entries from seq 614 at 2024-04-25T10:04:48.475944+0000) v1 ==== 603+0+0 (secure 0 0 0) 0x7f3e380880c0 con 0x7f3e4014f3f0 2024-04-25T10:04:49.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:04:49.743+0000 7f3e36ffd640 1 -- 172.21.15.82:0/2272066247 <== mon.0 v2:172.21.15.82:3300/0 10 ==== log(2 entries from seq 616 at 2024-04-25T10:04:48.723146+0000) v1 ==== 607+0+0 (secure 0 0 0) 0x7f3e38056ef0 con 0x7f3e4014f3f0 2024-04-25T10:05:00.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:00.697 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:05:00.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 Processor -- start 2024-04-25T10:05:00.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 -- start start 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8415b6b0 0x7f4b840ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b84073010 0x7f4b840cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b840cbd20 con 0x7f4b840cf590 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b8415ba90 con 0x7f4b84073010 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b8a9a3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840cff30 con 0x7f4b8415b6b0 2024-04-25T10:05:00.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b84073010 0x7f4b840cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4b84073010 0x7f4b840cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48722/0 (socket says 172.21.15.82:48722) 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 -- 172.21.15.82:0/3797392752 learned_addr learned my addr 172.21.15.82:0/3797392752 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b83fff640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8415b6b0 0x7f4b840ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 -- 172.21.15.82:0/3797392752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8415b6b0 msgr2=0x7f4b840ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8415b6b0 0x7f4b840ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 -- 172.21.15.82:0/3797392752 >> v1:172.21.15.196:6789/0 conn(0x7f4b840cf590 legacy=0x7f4b8415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.823+0000 7f4b88f40640 1 -- 172.21.15.82:0/3797392752 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840cc000 con 0x7f4b84073010 2024-04-25T10:05:00.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b84073010 0x7f4b840cf9f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009e50 tx=0x7f4b7c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79107b726e29f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b837fe640 1 -- 172.21.15.82:0/3797392752 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0a57e0 con 0x7f4b84073010 2024-04-25T10:05:00.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b837fe640 1 -- 172.21.15.82:0/3797392752 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0a5980 con 0x7f4b84073010 2024-04-25T10:05:00.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b837fe640 1 -- 172.21.15.82:0/3797392752 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0a5d50 con 0x7f4b84073010 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/3797392752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b84073010 msgr2=0x7f4b840cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b84073010 0x7f4b840cf9f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c009e50 tx=0x7f4b7c01ca00 comp rx=0 tx=0).stop 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/3797392752 shutdown_connections 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b84073010 0x7f4b840cf9f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/3797392752 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b8415b6b0 0x7f4b840ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/3797392752 >> 172.21.15.82:0/3797392752 conn(0x7f4b8406ae60 msgr2=0x7f4b8406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/3797392752 shutdown_connections 2024-04-25T10:05:00.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/3797392752 wait complete. 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 Processor -- start 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- start start 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b84073010 0x7f4b84098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b840cf590 0x7f4b84150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840ca900 con 0x7f4b8415b6b0 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840cba30 con 0x7f4b840cf590 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840cadb0 con 0x7f4b84073010 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48738/0 (socket says 172.21.15.82:48738) 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 -- 172.21.15.82:0/783473480 learned_addr learned my addr 172.21.15.82:0/783473480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 -- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b84073010 msgr2=0x7f4b84098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b83fff640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b84073010 0x7f4b84098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b840cf590 0x7f4b84150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b84073010 0x7f4b84098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 -- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b840cf590 msgr2=0x7f4b84150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b840cf590 0x7f4b84150f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 -- 172.21.15.82:0/783473480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b840ac170 con 0x7f4b8415b6b0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b88f40640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01a630 tx=0x7f4b7c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b817fa640 1 -- 172.21.15.82:0/783473480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0b8670 con 0x7f4b8415b6b0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b817fa640 1 -- 172.21.15.82:0/783473480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0be070 con 0x7f4b8415b6b0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b817fa640 1 -- 172.21.15.82:0/783473480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0b8a70 con 0x7f4b8415b6b0 2024-04-25T10:05:00.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.827+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b84159c80 con 0x7f4b8415b6b0 2024-04-25T10:05:00.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b8415a240 con 0x7f4b8415b6b0 2024-04-25T10:05:00.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b817fa640 1 -- 172.21.15.82:0/783473480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b7c0b8c10 con 0x7f4b8415b6b0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b817fa640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 0x7f4b64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b817fa640 1 -- 172.21.15.82:0/783473480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4b7c1203a0 con 0x7f4b8415b6b0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b83fff640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 0x7f4b64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b83fff640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 0x7f4b64069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b740096f0 tx=0x7f4b74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 Processor -- start 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 -- start start 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a45d9d0 0x56190a45ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a413b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a260ff0 con 0x56190a4140a0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a1e9150 con 0x56190a45e2f0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a25d090 con 0x56190a45d9d0 2024-04-25T10:05:00.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48748/0 (socket says 172.21.15.82:48748) 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.831+0000 7f4b88f40640 1 -- 172.21.15.82:0/2526679107 learned_addr learned my addr 172.21.15.82:0/2526679107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a45d9d0 msgr2=0x56190a45ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b83fff640 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a45d9d0 0x56190a45ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a413b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a45d9d0 0x56190a45ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 msgr2=0x56190a413b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a413b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/2526679107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190a3f70d0 con 0x56190a4140a0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0ed0c0 tx=0x7f4b7c0f0680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7235e9dc1a42eafd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b79ffb640 1 -- 172.21.15.82:0/2526679107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c0f4d00 con 0x56190a4140a0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b79ffb640 1 -- 172.21.15.82:0/2526679107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0f92c0 con 0x56190a4140a0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b79ffb640 1 -- 172.21.15.82:0/2526679107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c1200b0 con 0x56190a4140a0 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/2526679107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 msgr2=0x56190a414480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c0ed0c0 tx=0x7f4b7c0f0680 comp rx=0 tx=0).stop 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/2526679107 shutdown_connections 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a4140a0 0x56190a414480 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a413b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- 172.21.15.82:0/2526679107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a45d9d0 0x56190a45ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/2526679107 >> 172.21.15.82:0/2526679107 conn(0x56190a410020 msgr2=0x56190a412440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/2526679107 shutdown_connections 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/2526679107 wait complete. 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- start start 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 Processor -- start 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a4427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a442d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a225250 con 0x56190a45e2f0 2024-04-25T10:05:00.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a21b690 con 0x56190a45d9d0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a1e9a40 con 0x56190a4140a0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 10 client.0 _ll_register_callbacks cb 0x56190a3fee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48752/0 (socket says 172.21.15.82:48752) 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/990801441 learned_addr learned my addr 172.21.15.82:0/990801441 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b83fff640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a4427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a442d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 msgr2=0x56190a4427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a4427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 msgr2=0x56190a442d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a442d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 -- 172.21.15.82:0/990801441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56190a30e1c0 con 0x56190a45e2f0 2024-04-25T10:05:00.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b88f40640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01cee0 tx=0x7f4b7c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7f17b56fe117a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c11c040 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4b7c1200b0 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b7c0f92c0 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b7c11e2f0 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 10 client.6222 fetch_fsmap learned FSMap version 24 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56190a30a170 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b7c0f92c0 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b7c008040 con 0x56190a45e2f0 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 10 client.6222 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 10 client.6222 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 20 client.6222 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a46dc80 0x56190a470070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 4 client.6222 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:00.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.835+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56190a1fcc60 con 0x56190a46dc80 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b83fff640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a46dc80 0x56190a470070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b62ffd640 10 client.6222 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b83fff640 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a46dc80 0x56190a470070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b14eedb8aa828 server_cookie=2d2d482d3b0695a5 in_seq=0 out_seq=0 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b62ffd640 1 -- 172.21.15.82:0/990801441 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56190a1fcc60 con 0x56190a46dc80 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b62ffd640 10 client.6222 handle_command_reply: tid=0 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 client.6222 shutdown 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 msgr2=0x56190a443270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01cee0 tx=0x7f4b7c009810 comp rx=0 tx=0).stop 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 shutdown_connections 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a46dc80 0x56190a470070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b62ffd640 0 client.6222 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a443270 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a442d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 --2- 172.21.15.82:0/990801441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a4427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 >> 172.21.15.82:0/990801441 conn(0x56190a410020 msgr2=0x56190a412ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 shutdown_connections 2024-04-25T10:05:00.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 1 -- 172.21.15.82:0/990801441 wait complete. 2024-04-25T10:05:00.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.839+0000 7f4b8db10000 20 client.6222 trim_cache size 0 max 16384 2024-04-25T10:05:00.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 Processor -- start 2024-04-25T10:05:00.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- start start 2024-04-25T10:05:00.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a413570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a412950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b84200da0 con 0x56190a45d9d0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b841fff70 con 0x56190a45e2f0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b840ad3c0 con 0x56190a4140a0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48758/0 (socket says 172.21.15.82:48758) 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 -- 172.21.15.82:0/1554839090 learned_addr learned my addr 172.21.15.82:0/1554839090 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 -- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 msgr2=0x56190a413570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b88f40640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a412950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b83fff640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a413570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a413570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 -- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 msgr2=0x56190a412950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a412950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 -- 172.21.15.82:0/1554839090 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56190a2a7de0 con 0x56190a45d9d0 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b88f40640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a412950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:00.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b7b7fe640 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0013d0 tx=0x7f4b6c001890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a595026720125a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b797fa640 1 -- 172.21.15.82:0/1554839090 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003600 con 0x56190a45d9d0 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b797fa640 1 -- 172.21.15.82:0/1554839090 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c018070 con 0x56190a45d9d0 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b797fa640 1 -- 172.21.15.82:0/1554839090 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c0039e0 con 0x56190a45d9d0 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- 172.21.15.82:0/1554839090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 msgr2=0x56190a412410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c0013d0 tx=0x7f4b6c001890 comp rx=0 tx=0).stop 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- 172.21.15.82:0/1554839090 shutdown_connections 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45e2f0 0x56190a412950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45d9d0 0x56190a412410 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 --2- 172.21.15.82:0/1554839090 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a413570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- 172.21.15.82:0/1554839090 >> 172.21.15.82:0/1554839090 conn(0x56190a410020 msgr2=0x56190a45c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- 172.21.15.82:0/1554839090 shutdown_connections 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- 172.21.15.82:0/1554839090 wait complete. 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 -- start start 2024-04-25T10:05:00.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.855+0000 7f4b8db10000 1 Processor -- start 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a470770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a470cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56190a1fbe00 con 0x56190a45e2f0 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b841ffaf0 con 0x56190a45d9d0 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56190a260300 con 0x56190a4140a0 2024-04-25T10:05:00.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 10 client.0 _ll_register_callbacks cb 0x56190a3df040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48762/0 (socket says 172.21.15.82:48762) 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 -- 172.21.15.82:0/106669003 learned_addr learned my addr 172.21.15.82:0/106669003 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b83fff640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a470770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 -- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 msgr2=0x56190a470770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a470770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 -- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 msgr2=0x56190a470cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a470cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 -- 172.21.15.82:0/106669003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56190a2028e0 con 0x56190a45e2f0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b88f40640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01c830 tx=0x7f4b7c0c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15e7e453ff125c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c11c040 con 0x56190a45e2f0 2024-04-25T10:05:00.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4b7c0f8e30 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6c018070 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b7c0ecf00 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 10 client.6228 fetch_fsmap learned FSMap version 24 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x56190a2ba8f0 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6c018070 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b7c11eb30 con 0x56190a45e2f0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 10 client.6228 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 20 client.6228 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a4cc760 0x56190a4ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 4 client.6228 mds_command: new command op to 5946 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 --> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56190a26e530 con 0x56190a4cc760 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b83fff640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a4cc760 0x56190a4ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:00.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b83fff640 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a4cc760 0x56190a4ceb50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55fd7c7991fbd55 server_cookie=32e5e0ddd88551ae in_seq=0 out_seq=0 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 10 client.6228 ms_handle_connect on v2:172.21.15.82:6836/238219074 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 1 -- 172.21.15.82:0/106669003 <== mds.0 v2:172.21.15.82:6836/238219074 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56190a26e530 con 0x56190a4cc760 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 10 client.6228 handle_command_reply: tid=0 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 client.6228 shutdown 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 msgr2=0x56190a4711f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01c830 tx=0x7f4b7c0c5000 comp rx=0 tx=0).stop 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 shutdown_connections 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:6836/238219074,v1:172.21.15.82:6837/238219074] conn(0x56190a4cc760 0x56190a4ceb50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56190a45e2f0 0x56190a4711f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b62ffd640 0 client.6228 ms_handle_reset on v2:172.21.15.82:6836/238219074 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56190a45d9d0 0x56190a470cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 --2- 172.21.15.82:0/106669003 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56190a4140a0 0x56190a470770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 >> 172.21.15.82:0/106669003 conn(0x56190a410020 msgr2=0x56190a45c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 shutdown_connections 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 1 -- 172.21.15.82:0/106669003 wait complete. 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8db10000 20 client.6228 trim_cache size 0 max 16384 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:05:00.869 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 msgr2=0x7f4b64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 0x7f4b64069420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4b740096f0 tx=0x7f4b74008040 comp rx=0 tx=0).stop 2024-04-25T10:05:00.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 msgr2=0x7f4b84151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c01a630 tx=0x7f4b7c0197a0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 shutdown_connections 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4b64066f60 0x7f4b64069420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4b8415b6b0 0x7f4b84151440 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4b840cf590 0x7f4b84150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 --2- 172.21.15.82:0/783473480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4b84073010 0x7f4b84098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 >> 172.21.15.82:0/783473480 conn(0x7f4b8406ae60 msgr2=0x7f4b840b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 shutdown_connections 2024-04-25T10:05:00.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:00.859+0000 7f4b8a9a3640 1 -- 172.21.15.82:0/783473480 wait complete. 2024-04-25T10:05:00.879 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:00.981 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:01.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.099+0000 7f113eb67640 1 Processor -- start 2024-04-25T10:05:01.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.099+0000 7f113eb67640 1 -- start start 2024-04-25T10:05:01.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113eb67640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f113815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113eb67640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113815ae30 con 0x7f1138159ea0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113eb67640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113815b110 con 0x7f113806d090 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113eb67640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11380b2990 con 0x7f113815a280 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1138159ea0 0x7f11380b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44928/0 (socket says 172.21.15.82:44928) 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 -- 172.21.15.82:0/581996986 learned_addr learned my addr 172.21.15.82:0/581996986 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f1137fff640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f113815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803308298 0 0) 0x7f113815ae30 con 0x7f1138159ea0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1124003530 con 0x7f1138159ea0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 586035790 0 0) 0x7f1124003530 con 0x7f1138159ea0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 msgr2=0x7f113815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f113815a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 >> v1:172.21.15.196:6789/0 conn(0x7f113806d090 legacy=0x7f113806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11380b2620 con 0x7f1138159ea0 2024-04-25T10:05:01.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1128001a00 con 0x7f1138159ea0 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 >> v1:172.21.15.82:6789/0 conn(0x7f1138159ea0 legacy=0x7f11380b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f1124003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1124004810 0x7f1124004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f1124005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380b2990 con 0x7f1124003a60 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112400bb90 con 0x7f1124004810 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112400dfa0 con 0x7f113815a280 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f1124003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f1137fff640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1124004810 0x7f1124004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113d104640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f1124005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 -- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 msgr2=0x7f1124005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f113815a280 0x7f1124005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 -- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1124004810 msgr2=0x7f1124004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1124004810 0x7f1124004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 -- 172.21.15.82:0/581996986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124003530 con 0x7f1124003a60 2024-04-25T10:05:01.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f1137fff640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1124004810 0x7f1124004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f113c903640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f1124003e80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f112800a770 tx=0x7f112801d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8891d9919c80b030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800a390 con 0x7f1124003a60 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11280ad630 con 0x7f1124003a60 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.103+0000 7f11377fe640 1 -- 172.21.15.82:0/581996986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11280ada20 con 0x7f1124003a60 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/581996986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 msgr2=0x7f1124003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 --2- 172.21.15.82:0/581996986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f1124003e80 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f112800a770 tx=0x7f112801d2a0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113d104640 1 -- 172.21.15.82:0/581996986 reap_dead start 2024-04-25T10:05:01.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/581996986 shutdown_connections 2024-04-25T10:05:01.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/581996986 >> 172.21.15.82:0/581996986 conn(0x7f11380b11a0 msgr2=0x7f11380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/581996986 shutdown_connections 2024-04-25T10:05:01.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/581996986 wait complete. 2024-04-25T10:05:01.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 Processor -- start 2024-04-25T10:05:01.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- start start 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11380a84d0 0x7f1138141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 0x7f113815f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11381631a0 con 0x7f1124003a60 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11380b3530 con 0x7f11380a84d0 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1138163320 con 0x7f1138142370 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48786/0 (socket says 172.21.15.82:48786) 2024-04-25T10:05:01.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 -- 172.21.15.82:0/1169506138 learned_addr learned my addr 172.21.15.82:0/1169506138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f1137fff640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11380a84d0 0x7f1138141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113d104640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 0x7f113815f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 -- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 msgr2=0x7f113815f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 0x7f113815f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 -- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11380a84d0 msgr2=0x7f1138141e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11380a84d0 0x7f1138141e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 -- 172.21.15.82:0/1169506138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113815e920 con 0x7f1124003a60 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113d104640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 0x7f113815f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113c903640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f112800a770 tx=0x7f1124005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128004b70 con 0x7f1124003a60 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128004d10 con 0x7f1124003a60 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128001940 con 0x7f1124003a60 2024-04-25T10:05:01.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113815ec00 con 0x7f1124003a60 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.107+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113815f1c0 con 0x7f1124003a60 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113804efa0 con 0x7f1124003a60 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f11280be060 con 0x7f1124003a60 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f11357fa640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 0x7f1118069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f112811f330 con 0x7f1124003a60 2024-04-25T10:05:01.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f1137fff640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 0x7f1118069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.111+0000 7f1137fff640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 0x7f1118069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11380ae4d0 tx=0x7f1120008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.115+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1128016fb0 con 0x7f1124003a60 2024-04-25T10:05:01.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.299+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11381593b0 con 0x7f1124003a60 2024-04-25T10:05:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f11357fa640 1 -- 172.21.15.82:0/1169506138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f11280f0210 con 0x7f1124003a60 2024-04-25T10:05:01.305 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:01.305 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:01.305 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:05:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 msgr2=0x7f1118069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 0x7f1118069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f11380ae4d0 tx=0x7f1120008040 comp rx=0 tx=0).stop 2024-04-25T10:05:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 msgr2=0x7f11381428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f112800a770 tx=0x7f1124005840 comp rx=0 tx=0).stop 2024-04-25T10:05:01.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 shutdown_connections 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1118066f60 0x7f1118069420 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1138142370 0x7f113815f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11380a84d0 0x7f1138141e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 --2- 172.21.15.82:0/1169506138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1124003a60 0x7f11381428b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 >> 172.21.15.82:0/1169506138 conn(0x7f11380b11a0 msgr2=0x7f113806d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 shutdown_connections 2024-04-25T10:05:01.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.303+0000 7f113eb67640 1 -- 172.21.15.82:0/1169506138 wait complete. 2024-04-25T10:05:01.323 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:05:01.471 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:05:01.471 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:01.471 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:05:01.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 Processor -- start 2024-04-25T10:05:01.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- start start 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98feb2570 0x55a98fee48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98ff8a1f0 0x55a98ff87ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98fee67c0 con 0x55a98ff00360 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98fe599f0 con 0x55a98ff8a1f0 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a98fee6940 con 0x55a98feb2570 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48804/0 (socket says 172.21.15.82:48804) 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 -- 172.21.15.82:0/1058620677 learned_addr learned my addr 172.21.15.82:0/1058620677 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 -- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98feb2570 msgr2=0x55a98fee48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98feb2570 0x55a98fee48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 -- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98ff8a1f0 msgr2=0x55a98ff87ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd992cbd640 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98ff8a1f0 0x55a98ff87ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98ff8a1f0 0x55a98ff87ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 -- 172.21.15.82:0/1058620677 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a98ff84da0 con 0x55a98ff00360 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd991cbb640 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd988019040 tx=0x7fd988003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9c5e53bf7c1482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd990cb9640 1 -- 172.21.15.82:0/1058620677 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880aa930 con 0x55a98ff00360 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd990cb9640 1 -- 172.21.15.82:0/1058620677 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9880aaaf0 con 0x55a98ff00360 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd990cb9640 1 -- 172.21.15.82:0/1058620677 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880bc400 con 0x55a98ff00360 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- 172.21.15.82:0/1058620677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 msgr2=0x55a98ff877a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd988019040 tx=0x7fd988003030 comp rx=0 tx=0).stop 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- 172.21.15.82:0/1058620677 shutdown_connections 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98ff8a1f0 0x55a98ff87ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98ff877a0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 --2- 172.21.15.82:0/1058620677 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98feb2570 0x55a98fee48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- 172.21.15.82:0/1058620677 >> 172.21.15.82:0/1058620677 conn(0x55a98feb8c90 msgr2=0x55a98ff89290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- 172.21.15.82:0/1058620677 shutdown_connections 2024-04-25T10:05:01.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.507+0000 7fd99499e080 1 -- 172.21.15.82:0/1058620677 wait complete. 2024-04-25T10:05:01.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 Processor -- start 2024-04-25T10:05:01.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- start start 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98feb2570 0x55a98ff6f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98fefb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98ff8a1f0 0x55a98fefbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98ff85040 con 0x55a98ff00360 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a98ff852e0 con 0x55a98feb2570 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a98ff85580 con 0x55a98ff8a1f0 2024-04-25T10:05:01.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98fefb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98fefb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48810/0 (socket says 172.21.15.82:48810) 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 -- 172.21.15.82:0/4137044157 learned_addr learned my addr 172.21.15.82:0/4137044157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9924bc640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98feb2570 0x55a98ff6f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 -- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98ff8a1f0 msgr2=0x55a98fefbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd992cbd640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98ff8a1f0 0x55a98fefbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a98ff8a1f0 0x55a98fefbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 -- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98feb2570 msgr2=0x55a98ff6f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98feb2570 0x55a98ff6f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 -- 172.21.15.82:0/4137044157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a98ff6de40 con 0x55a98ff00360 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9924bc640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a98feb2570 0x55a98ff6f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd991cbb640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98fefb990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd9880be040 tx=0x7fd9880181b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 -- 172.21.15.82:0/4137044157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880cc070 con 0x55a98ff00360 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 -- 172.21.15.82:0/4137044157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9880c4620 con 0x55a98ff00360 2024-04-25T10:05:01.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a98ff6e0e0 con 0x55a98ff00360 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 -- 172.21.15.82:0/4137044157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880c49d0 con 0x55a98ff00360 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a98ff6e620 con 0x55a98ff00360 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 -- 172.21.15.82:0/4137044157 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9880cf070 con 0x55a98ff00360 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd974065940 0x7fd974067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9837fe640 1 -- 172.21.15.82:0/4137044157 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd988120770 con 0x55a98ff00360 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.511+0000 7fd9924bc640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd974065940 0x7fd974067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a98ff6f800 0x55a98ff819c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v9 -- 0x55a98ff81f00 con 0x55a98ff6f800 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd992cbd640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a98ff6f800 0x55a98ff819c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd9924bc640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd974065940 0x7fd974067e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd992cbd640 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a98ff6f800 0x55a98ff819c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd992cbd640 1 -- 172.21.15.82:0/4137044157 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fd984007560 con 0x55a98ff6f800 2024-04-25T10:05:01.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a98ff6f800 msgr2=0x55a98ff819c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a98ff6f800 0x55a98ff819c0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd974065940 msgr2=0x7fd974067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd974065940 0x7fd974067e00 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd98c005fd0 tx=0x7fd98c007040 comp rx=0 tx=0).stop 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 msgr2=0x55a98fefb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 --2- 172.21.15.82:0/4137044157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a98ff00360 0x55a98fefb990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd9880be040 tx=0x7fd9880181b0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd992cbd640 1 -- 172.21.15.82:0/4137044157 reap_dead start 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 shutdown_connections 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 >> 172.21.15.82:0/4137044157 conn(0x55a98feb8c90 msgr2=0x55a98fef5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 shutdown_connections 2024-04-25T10:05:01.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.515+0000 7fd99499e080 1 -- 172.21.15.82:0/4137044157 wait complete. 2024-04-25T10:05:01.523 INFO:teuthology.orchestra.run.smithi082.stdout:Yalphaparent_a 2024-04-25T10:05:01.523 DEBUG:teuthology.orchestra.run.smithi082:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "ancestors": [ 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1099511627776, 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "alpha", 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 4 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1, 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "parent_a", 2024-04-25T10:05:01.613 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 7 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout: "pool": 8, 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout: "old_pools": [] 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:01.614 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:01.616 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:01.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:01.686 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:01.735 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:01.735 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:05:01.735 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:01.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.907+0000 7f54792db640 1 Processor -- start 2024-04-25T10:05:01.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.907+0000 7f54792db640 1 -- start start 2024-04-25T10:05:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54792db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547415a280 0x7f547415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54792db640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54740718c0 0x7f5474071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54792db640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f547415ae30 con 0x7f5474159ea0 2024-04-25T10:05:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54792db640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54740b2990 con 0x7f54740718c0 2024-04-25T10:05:01.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54792db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54740b3530 con 0x7f547415a280 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54740718c0 0x7f5474071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54740718c0 0x7f5474071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48824/0 (socket says 172.21.15.82:48824) 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 -- 172.21.15.82:0/3781740226 learned_addr learned my addr 172.21.15.82:0/3781740226 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f546a7fc640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547415a280 0x7f547415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 -- 172.21.15.82:0/3781740226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547415a280 msgr2=0x7f547415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547415a280 0x7f547415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 -- 172.21.15.82:0/3781740226 >> v1:172.21.15.82:6789/0 conn(0x7f5474159ea0 legacy=0x7f54740b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 -- 172.21.15.82:0/3781740226 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547415b110 con 0x7f54740718c0 2024-04-25T10:05:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.911+0000 7f54737fe640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54740718c0 0x7f5474071d50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f546c00d150 tx=0x7f546c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fae695c0856c6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54727fc640 1 -- 172.21.15.82:0/3781740226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0ad070 con 0x7f54740718c0 2024-04-25T10:05:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54727fc640 1 -- 172.21.15.82:0/3781740226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0a7b90 con 0x7f54740718c0 2024-04-25T10:05:01.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54727fc640 1 -- 172.21.15.82:0/3781740226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b1400 con 0x7f54740718c0 2024-04-25T10:05:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- 172.21.15.82:0/3781740226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54740718c0 msgr2=0x7f5474071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54740718c0 0x7f5474071d50 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f546c00d150 tx=0x7f546c01ec30 comp rx=0 tx=0).stop 2024-04-25T10:05:01.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- 172.21.15.82:0/3781740226 shutdown_connections 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54740718c0 0x7f5474071d50 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- 172.21.15.82:0/3781740226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f547415a280 0x7f547415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- 172.21.15.82:0/3781740226 >> 172.21.15.82:0/3781740226 conn(0x7f54740b11a0 msgr2=0x7f54740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- 172.21.15.82:0/3781740226 shutdown_connections 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- 172.21.15.82:0/3781740226 wait complete. 2024-04-25T10:05:01.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 Processor -- start 2024-04-25T10:05:01.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- start start 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 0x7f5474155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5474159ea0 0x7f54740ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547406d420 con 0x7f547415a280 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547406d2a0 con 0x7f54740718c0 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54792db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547406d5a0 con 0x7f5474159ea0 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48826/0 (socket says 172.21.15.82:48826) 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 -- 172.21.15.82:0/1268235006 learned_addr learned my addr 172.21.15.82:0/1268235006 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f546a7fc640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5474159ea0 0x7f54740ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f5472ffd640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 0x7f5474155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 -- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5474159ea0 msgr2=0x7f54740ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5474159ea0 0x7f54740ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 -- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 msgr2=0x7f5474155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 0x7f5474155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f54737fe640 1 -- 172.21.15.82:0/1268235006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5474155ca0 con 0x7f547415a280 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.915+0000 7f5472ffd640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 0x7f5474155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f54737fe640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f546c01d2e0 tx=0x7f546c01b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c0ad040 con 0x7f547415a280 2024-04-25T10:05:01.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c0b1e40 con 0x7f547415a280 2024-04-25T10:05:01.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54740ac480 con 0x7f547415a280 2024-04-25T10:05:01.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00b720 con 0x7f547415a280 2024-04-25T10:05:01.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54740ab9a0 con 0x7f547415a280 2024-04-25T10:05:01.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547404efa0 con 0x7f547415a280 2024-04-25T10:05:01.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f546c0c4020 con 0x7f547415a280 2024-04-25T10:05:01.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 0x7f5454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:01.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f5472ffd640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 0x7f5454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:01.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f546c121f10 con 0x7f547415a280 2024-04-25T10:05:01.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.919+0000 7f5472ffd640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 0x7f5454069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54640016f0 tx=0x7f5464009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:01.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:01.923+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f546c0cb020 con 0x7f547415a280 2024-04-25T10:05:02.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.083+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f547406e6b0 con 0x7f547415a280 2024-04-25T10:05:02.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.087+0000 7f546bfff640 1 -- 172.21.15.82:0/1268235006 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f546c018820 con 0x7f547415a280 2024-04-25T10:05:02.090 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:02.090 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":54,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:49.723093+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.087+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 msgr2=0x7f5454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.087+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 0x7f5454069420 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f54640016f0 tx=0x7f5464009040 comp rx=0 tx=0).stop 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.087+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 msgr2=0x7f547406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.087+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f546c01d2e0 tx=0x7f546c01b9d0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 shutdown_connections 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5454066f60 0x7f5454069420 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5474159ea0 0x7f54740ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f547415a280 0x7f547406d960 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 --2- 172.21.15.82:0/1268235006 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54740718c0 0x7f5474155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 >> 172.21.15.82:0/1268235006 conn(0x7f54740b11a0 msgr2=0x7f54740cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 shutdown_connections 2024-04-25T10:05:02.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.091+0000 7f54792db640 1 -- 172.21.15.82:0/1268235006 wait complete. 2024-04-25T10:05:02.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:15.148506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:15.924015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:02.109 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 Processor -- start 2024-04-25T10:05:02.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 -- start start 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcca015da80 0x7fcca00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 0x7fcca006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcca006be60 con 0x7fcca00d1cd0 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcca006c440 con 0x7fcca015da80 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca866d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca015de60 con 0x7fcca00d07f0 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca5c08640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcca00d1cd0 0x7fcca00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:45546/0 (socket says 172.21.15.82:45546) 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca5c08640 1 -- 172.21.15.82:0/1358552473 learned_addr learned my addr 172.21.15.82:0/1358552473 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:02.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6c0a640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 0x7fcca006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6409640 1 --2- 172.21.15.82:0/1358552473 >> v2:172.21.15.196:3300/0 conn(0x7fcca015da80 0x7fcca00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6409640 1 -- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 msgr2=0x7fcca006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6409640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 0x7fcca006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6409640 1 -- 172.21.15.82:0/1358552473 >> v1:172.21.15.196:6789/0 conn(0x7fcca00d1cd0 legacy=0x7fcca00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6409640 1 -- 172.21.15.82:0/1358552473 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca00d2450 con 0x7fcca015da80 2024-04-25T10:05:02.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.227+0000 7fcca6c0a640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 0x7fcca006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:02.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca6409640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca015da80 0x7fcca00d4230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097f0 tx=0x7fcc9c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510290083a731401 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca5407640 1 -- 172.21.15.82:0/1358552473 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0b1070 con 0x7fcca015da80 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca5407640 1 -- 172.21.15.82:0/1358552473 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c0ac630 con 0x7fcca015da80 2024-04-25T10:05:02.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca5407640 1 -- 172.21.15.82:0/1358552473 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c0a8050 con 0x7fcca015da80 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca015da80 msgr2=0x7fcca00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca015da80 0x7fcca00d4230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0097f0 tx=0x7fcc9c01c470 comp rx=0 tx=0).stop 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- 172.21.15.82:0/1358552473 shutdown_connections 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca00d07f0 0x7fcca006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 --2- 172.21.15.82:0/1358552473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca015da80 0x7fcca00d4230 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- 172.21.15.82:0/1358552473 >> 172.21.15.82:0/1358552473 conn(0x7fcca008b000 msgr2=0x7fcca00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- 172.21.15.82:0/1358552473 shutdown_connections 2024-04-25T10:05:02.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- 172.21.15.82:0/1358552473 wait complete. 2024-04-25T10:05:02.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 Processor -- start 2024-04-25T10:05:02.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.231+0000 7fcca866d640 1 -- start start 2024-04-25T10:05:02.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca00d07f0 0x7fcca0153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca015da80 0x7fcca00b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca006d7a0 con 0x7fcca00d1cd0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca006d620 con 0x7fcca00d07f0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcca006d920 con 0x7fcca015da80 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48840/0 (socket says 172.21.15.82:48840) 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 -- 172.21.15.82:0/549421740 learned_addr learned my addr 172.21.15.82:0/549421740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca6409640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca00d07f0 0x7fcca0153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca6c0a640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca015da80 0x7fcca00b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 -- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca015da80 msgr2=0x7fcca00b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca015da80 0x7fcca00b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 -- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca00d07f0 msgr2=0x7fcca0153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca00d07f0 0x7fcca0153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 -- 172.21.15.82:0/549421740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca0154d40 con 0x7fcca00d1cd0 2024-04-25T10:05:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca5c08640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c450 tx=0x7fcc9400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc94017070 con 0x7fcca00d1cd0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc94005d40 con 0x7fcca00d1cd0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca0155020 con 0x7fcca00d1cd0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc940072b0 con 0x7fcca00d1cd0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca01555e0 con 0x7fcca00d1cd0 2024-04-25T10:05:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca004efa0 con 0x7fcca00d1cd0 2024-04-25T10:05:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc9401a070 con 0x7fcca00d1cd0 2024-04-25T10:05:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 0x7fcc84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcc94084530 con 0x7fcca00d1cd0 2024-04-25T10:05:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.235+0000 7fcca6409640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 0x7fcc84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.239+0000 7fcca6409640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 0x7fcc84069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0ae000 tx=0x7fcc9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.239+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc94055300 con 0x7fcca00d1cd0 2024-04-25T10:05:02.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.387+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcca006ad40 con 0x7fcca00d1cd0 2024-04-25T10:05:02.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.387+0000 7fcc92ffd640 1 -- 172.21.15.82:0/549421740 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc94054c00 con 0x7fcca00d1cd0 2024-04-25T10:05:02.392 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 msgr2=0x7fcc84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 0x7fcc84069420 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c0ae000 tx=0x7fcc9c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 msgr2=0x7fcca0154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400c450 tx=0x7fcc9400c910 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 shutdown_connections 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcc84066f60 0x7fcc84069420 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcca015da80 0x7fcca00b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcca00d1cd0 0x7fcca0154440 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 --2- 172.21.15.82:0/549421740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcca00d07f0 0x7fcca0153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 >> 172.21.15.82:0/549421740 conn(0x7fcca008b000 msgr2=0x7fcca00ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 shutdown_connections 2024-04-25T10:05:02.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.391+0000 7fcca866d640 1 -- 172.21.15.82:0/549421740 wait complete. 2024-04-25T10:05:02.409 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:02.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 Processor -- start 2024-04-25T10:05:02.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 -- start start 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6de415b6b0 0x7f6de40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de40cf590 0x7f6de40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de40cbd20 con 0x7f6de415f130 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de415ba90 con 0x7f6de415b6b0 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6dea2c8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40cff30 con 0x7f6de40cf590 2024-04-25T10:05:02.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6de415b6b0 0x7f6de40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6de415b6b0 0x7f6de40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48846/0 (socket says 172.21.15.82:48846) 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 -- 172.21.15.82:0/2169305928 learned_addr learned my addr 172.21.15.82:0/2169305928 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6ddbfff640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de40cf590 0x7f6de40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 -- 172.21.15.82:0/2169305928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de40cf590 msgr2=0x7f6de40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de40cf590 0x7f6de40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 -- 172.21.15.82:0/2169305928 >> v1:172.21.15.196:6789/0 conn(0x7f6de415f130 legacy=0x7f6de415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 -- 172.21.15.82:0/2169305928 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40cc000 con 0x7f6de415b6b0 2024-04-25T10:05:02.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.531+0000 7f6de3fff640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de415b6b0 0x7f6de40ca3c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30326531b88f5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de37fe640 1 -- 172.21.15.82:0/2169305928 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40aa610 con 0x7f6de415b6b0 2024-04-25T10:05:02.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de37fe640 1 -- 172.21.15.82:0/2169305928 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd40af070 con 0x7f6de415b6b0 2024-04-25T10:05:02.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de37fe640 1 -- 172.21.15.82:0/2169305928 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40aa9d0 con 0x7f6de415b6b0 2024-04-25T10:05:02.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/2169305928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de415b6b0 msgr2=0x7f6de40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de415b6b0 0x7f6de40ca3c0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009870 tx=0x7f6dd401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/2169305928 shutdown_connections 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de40cf590 0x7f6de40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/2169305928 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de415b6b0 0x7f6de40ca3c0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/2169305928 >> 172.21.15.82:0/2169305928 conn(0x7f6de406ae60 msgr2=0x7f6de406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/2169305928 shutdown_connections 2024-04-25T10:05:02.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/2169305928 wait complete. 2024-04-25T10:05:02.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 Processor -- start 2024-04-25T10:05:02.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- start start 2024-04-25T10:05:02.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 0x7f6de41fa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 0x7f6de4164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40ca900 con 0x7f6de40cf590 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40cba30 con 0x7f6de415b6b0 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40cadb0 con 0x7f6de415f130 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48854/0 (socket says 172.21.15.82:48854) 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 -- 172.21.15.82:0/188035914 learned_addr learned my addr 172.21.15.82:0/188035914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de8865640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 0x7f6de4164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6ddbfff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 0x7f6de41fa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 -- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 msgr2=0x7f6de4164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 0x7f6de4164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 -- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 msgr2=0x7f6de41fa7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 0x7f6de41fa7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 -- 172.21.15.82:0/188035914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de40c3810 con 0x7f6de40cf590 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de8865640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 0x7f6de4164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6ddbfff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 0x7f6de41fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de3fff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40ac040 tx=0x7f6dd401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40bd5f0 con 0x7f6de40cf590 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd40bd790 con 0x7f6de40cf590 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40bdb80 con 0x7f6de40cf590 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de40cd590 con 0x7f6de40cf590 2024-04-25T10:05:02.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.535+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de41fb300 con 0x7f6de40cf590 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de404efa0 con 0x7f6de40cf590 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6dd40b33f0 con 0x7f6de40cf590 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6de17fa640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 0x7f6dc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6dd411fff0 con 0x7f6de40cf590 2024-04-25T10:05:02.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6ddbfff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 0x7f6dc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.539+0000 7f6ddbfff640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 0x7f6dc4069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6de4165f30 tx=0x7f6dcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.543+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6dd40c3060 con 0x7f6de40cf590 2024-04-25T10:05:02.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.699+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6de4098b30 con 0x7f6de40cf590 2024-04-25T10:05:02.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.699+0000 7f6de17fa640 1 -- 172.21.15.82:0/188035914 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6de40c3810 con 0x7f6de40cf590 2024-04-25T10:05:02.703 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 msgr2=0x7f6dc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 0x7f6dc4069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6de4165f30 tx=0x7f6dcc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 msgr2=0x7f6de40985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40ac040 tx=0x7f6dd401ad30 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 shutdown_connections 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dc4066f60 0x7f6dc4069420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6de415f130 0x7f6de4164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6de415b6b0 0x7f6de41fa7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 --2- 172.21.15.82:0/188035914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6de40cf590 0x7f6de40985f0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 >> 172.21.15.82:0/188035914 conn(0x7f6de406ae60 msgr2=0x7f6de4154830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 shutdown_connections 2024-04-25T10:05:02.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.703+0000 7f6dea2c8640 1 -- 172.21.15.82:0/188035914 wait complete. 2024-04-25T10:05:02.721 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:02.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 Processor -- start 2024-04-25T10:05:02.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 -- start start 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09f015b6b0 0x7f09f00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f00cf590 0x7f09f00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f00cbd20 con 0x7f09f015f130 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f015ba90 con 0x7f09f015b6b0 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.831+0000 7f09f5392640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cff30 con 0x7f09f00cf590 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09f015b6b0 0x7f09f00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f09f015b6b0 0x7f09f00ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48862/0 (socket says 172.21.15.82:48862) 2024-04-25T10:05:02.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 -- 172.21.15.82:0/4241616364 learned_addr learned my addr 172.21.15.82:0/4241616364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:02.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09ee7fc640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f00cf590 0x7f09f00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 -- 172.21.15.82:0/4241616364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f00cf590 msgr2=0x7f09f00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f00cf590 0x7f09f00cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 -- 172.21.15.82:0/4241616364 >> v1:172.21.15.196:6789/0 conn(0x7f09f015f130 legacy=0x7f09f015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:02.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 -- 172.21.15.82:0/4241616364 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00cc000 con 0x7f09f015b6b0 2024-04-25T10:05:02.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09eeffd640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015b6b0 0x7f09f00ca3c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009870 tx=0x7f09e001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3111ce82cbd37080 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09edffb640 1 -- 172.21.15.82:0/4241616364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00aa610 con 0x7f09f015b6b0 2024-04-25T10:05:02.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09edffb640 1 -- 172.21.15.82:0/4241616364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e00af070 con 0x7f09f015b6b0 2024-04-25T10:05:02.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09edffb640 1 -- 172.21.15.82:0/4241616364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e00aa9d0 con 0x7f09f015b6b0 2024-04-25T10:05:02.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 -- 172.21.15.82:0/4241616364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015b6b0 msgr2=0x7f09f00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015b6b0 0x7f09f00ca3c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f09e0009870 tx=0x7f09e001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 -- 172.21.15.82:0/4241616364 shutdown_connections 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f00cf590 0x7f09f00cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 --2- 172.21.15.82:0/4241616364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015b6b0 0x7f09f00ca3c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 -- 172.21.15.82:0/4241616364 >> 172.21.15.82:0/4241616364 conn(0x7f09f006ae60 msgr2=0x7f09f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 -- 172.21.15.82:0/4241616364 shutdown_connections 2024-04-25T10:05:02.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.835+0000 7f09f5392640 1 -- 172.21.15.82:0/4241616364 wait complete. 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 Processor -- start 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- start start 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09f00cf590 0x7f09f0098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 0x7f09f0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00ca900 con 0x7f09f015f130 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cba30 con 0x7f09f00cf590 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00cadb0 con 0x7f09f015b6b0 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48868/0 (socket says 172.21.15.82:48868) 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 -- 172.21.15.82:0/2273558301 learned_addr learned my addr 172.21.15.82:0/2273558301 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ee7fc640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 0x7f09f0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09eeffd640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09f00cf590 0x7f09f0098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 -- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 msgr2=0x7f09f0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 0x7f09f0152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 -- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09f00cf590 msgr2=0x7f09f0098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09f00cf590 0x7f09f0098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:02.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 -- 172.21.15.82:0/2273558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f015c8d0 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ee7fc640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 0x7f09f0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09ef7fe640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b440 tx=0x7f09e800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8015020 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e8011850 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f015cbb0 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8011c40 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f015d170 con 0x7f09f015f130 2024-04-25T10:05:02.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f015d370 con 0x7f09f015f130 2024-04-25T10:05:02.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.839+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09e8018020 con 0x7f09f015f130 2024-04-25T10:05:02.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.843+0000 7f09d77fe640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:02.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.843+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f09e8085010 con 0x7f09f015f130 2024-04-25T10:05:02.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.843+0000 7f09eeffd640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:02.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.843+0000 7f09eeffd640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09e00ae000 tx=0x7f09e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:02.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:02.847+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09f015d370 con 0x7f09f015f130 2024-04-25T10:05:03.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.003+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09f004efa0 con 0x7f09f015f130 2024-04-25T10:05:03.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.003+0000 7f09d77fe640 1 -- 172.21.15.82:0/2273558301 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f09e801f020 con 0x7f09f015f130 2024-04-25T10:05:03.008 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 msgr2=0x7f09dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 0x7f09dc069450 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f09e00ae000 tx=0x7f09e0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 msgr2=0x7f09f0153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f09e800b440 tx=0x7f09e800b900 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 shutdown_connections 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f09dc066f90 0x7f09dc069450 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f09f015b6b0 0x7f09f0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f09f015f130 0x7f09f0153460 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 --2- 172.21.15.82:0/2273558301 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f09f00cf590 0x7f09f0098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 >> 172.21.15.82:0/2273558301 conn(0x7f09f006ae60 msgr2=0x7f09f0154830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 shutdown_connections 2024-04-25T10:05:03.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.007+0000 7f09f5392640 1 -- 172.21.15.82:0/2273558301 wait complete. 2024-04-25T10:05:03.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:05:03.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 Processor -- start 2024-04-25T10:05:03.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 -- start start 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f200af280 0x7f5f2015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f200b2c30 0x7f5f2006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f20066c30 con 0x7f5f2006c1e0 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2006daa0 con 0x7f5f200af280 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f27797640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f200af660 con 0x7f5f200b2c30 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f25533640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f200af280 0x7f5f2015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f25533640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f200af280 0x7f5f2015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48874/0 (socket says 172.21.15.82:48874) 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.131+0000 7f5f25533640 1 -- 172.21.15.82:0/2193076908 learned_addr learned my addr 172.21.15.82:0/2193076908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:03.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f24d32640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f200b2c30 0x7f5f2006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f25533640 1 -- 172.21.15.82:0/2193076908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f200b2c30 msgr2=0x7f5f2006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:03.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f25533640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f200b2c30 0x7f5f2006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f25533640 1 -- 172.21.15.82:0/2193076908 >> v1:172.21.15.196:6789/0 conn(0x7f5f2006c1e0 legacy=0x7f5f20161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:03.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f25533640 1 -- 172.21.15.82:0/2193076908 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2015e640 con 0x7f5f200af280 2024-04-25T10:05:03.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f25533640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200af280 0x7f5f2015df10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009870 tx=0x7f5f1c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da831ea9429b0a20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f17fff640 1 -- 172.21.15.82:0/2193076908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0b2070 con 0x7f5f200af280 2024-04-25T10:05:03.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f17fff640 1 -- 172.21.15.82:0/2193076908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c0a9070 con 0x7f5f200af280 2024-04-25T10:05:03.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f17fff640 1 -- 172.21.15.82:0/2193076908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c0ad880 con 0x7f5f200af280 2024-04-25T10:05:03.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 -- 172.21.15.82:0/2193076908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200af280 msgr2=0x7f5f2015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200af280 0x7f5f2015df10 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009870 tx=0x7f5f1c01c560 comp rx=0 tx=0).stop 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 -- 172.21.15.82:0/2193076908 shutdown_connections 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f200b2c30 0x7f5f2006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 --2- 172.21.15.82:0/2193076908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200af280 0x7f5f2015df10 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 -- 172.21.15.82:0/2193076908 >> 172.21.15.82:0/2193076908 conn(0x7f5f20071810 msgr2=0x7f5f20071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 -- 172.21.15.82:0/2193076908 shutdown_connections 2024-04-25T10:05:03.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.135+0000 7f5f27797640 1 -- 172.21.15.82:0/2193076908 wait complete. 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 Processor -- start 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- start start 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 0x7f5f2014e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f200af280 0x7f5f20162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2016aab0 con 0x7f5f200b2c30 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2006c5e0 con 0x7f5f200af280 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2016ac30 con 0x7f5f2006c1e0 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48880/0 (socket says 172.21.15.82:48880) 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 -- 172.21.15.82:0/191883541 learned_addr learned my addr 172.21.15.82:0/191883541 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25533640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 0x7f5f2014e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f24d32640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f200af280 0x7f5f20162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 -- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 msgr2=0x7f5f2014e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 0x7f5f2014e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 -- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f200af280 msgr2=0x7f5f20162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f200af280 0x7f5f20162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 -- 172.21.15.82:0/191883541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20150850 con 0x7f5f200b2c30 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25533640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 0x7f5f2014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:03.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f25d34640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c7a0 tx=0x7f5f1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18003b80 con 0x7f5f200b2c30 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f18003d20 con 0x7f5f200b2c30 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2014eb70 con 0x7f5f200b2c30 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1800aa90 con 0x7f5f200b2c30 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2014ffb0 con 0x7f5f200b2c30 2024-04-25T10:05:03.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.139+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2004efa0 con 0x7f5f200b2c30 2024-04-25T10:05:03.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.143+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f1800ac30 con 0x7f5f200b2c30 2024-04-25T10:05:03.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.143+0000 7f5f15ffb640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.143+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5f18085140 con 0x7f5f200b2c30 2024-04-25T10:05:03.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.143+0000 7f5f25533640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.143+0000 7f5f25533640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 0x7f5f04069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0af000 tx=0x7f5f1c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.147+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f18055f90 con 0x7f5f200b2c30 2024-04-25T10:05:03.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.303+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f200727c0 con 0x7f5f200b2c30 2024-04-25T10:05:03.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.303+0000 7f5f15ffb640 1 -- 172.21.15.82:0/191883541 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f18055890 con 0x7f5f200b2c30 2024-04-25T10:05:03.307 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 msgr2=0x7f5f04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 0x7f5f04069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0af000 tx=0x7f5f1c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 msgr2=0x7f5f20162e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800c7a0 tx=0x7f5f1800cc60 comp rx=0 tx=0).stop 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 shutdown_connections 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f04066f60 0x7f5f04069420 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f200b2c30 0x7f5f20162e90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f200af280 0x7f5f20162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 --2- 172.21.15.82:0/191883541 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f2006c1e0 0x7f5f2014e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 >> 172.21.15.82:0/191883541 conn(0x7f5f20071810 msgr2=0x7f5f200b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 shutdown_connections 2024-04-25T10:05:03.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.307+0000 7f5f27797640 1 -- 172.21.15.82:0/191883541 wait complete. 2024-04-25T10:05:03.325 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:05:03.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:03.325 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:03.334 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[41429]: fuse finished with error 0 and tester_r 0 2024-04-25T10:05:03.336 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:05:03.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:03.336 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:05:03.500 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:05:03.500 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:03.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:03.506 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:05:03.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:03.507 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:05:03.553 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:05:03.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:03.554 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:05:03.555 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:03.560 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:03.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:05:03.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:03.560 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:05:03.608 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:05:03.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:03.609 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:05:03.609 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:03.615 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:03.615 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:05:03.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:03.615 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:05:03.661 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:05:03.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:05:03.663 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:03.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.779+0000 7f0d964be640 1 Processor -- start 2024-04-25T10:05:03.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.779+0000 7f0d964be640 1 -- start start 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d964be640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0d9015a700 0x7f0d9015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d964be640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900b2090 con 0x7f0d900b3f50 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d964be640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d900b43e0 con 0x7f0d900b24a0 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d964be640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d9015f200 con 0x7f0d9015a700 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0d900b3f50 0x7f0d900cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44940/0 (socket says 172.21.15.82:44940) 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8f7fe640 1 -- 172.21.15.82:0/2868432447 learned_addr learned my addr 172.21.15.82:0/2868432447 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:03.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8ffff640 1 --2- 172.21.15.82:0/2868432447 >> v2:172.21.15.196:3300/0 conn(0x7f0d9015a700 0x7f0d9015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8ffff640 1 -- 172.21.15.82:0/2868432447 >> v1:172.21.15.196:6789/0 conn(0x7f0d900b24a0 legacy=0x7f0d9006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8ffff640 1 -- 172.21.15.82:0/2868432447 >> v1:172.21.15.82:6789/0 conn(0x7f0d900b3f50 legacy=0x7f0d900cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8ffff640 1 -- 172.21.15.82:0/2868432447 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900b46c0 con 0x7f0d9015a700 2024-04-25T10:05:03.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.783+0000 7f0d8ffff640 1 --2- 172.21.15.82:0/2868432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d9015a700 0x7f0d9015ecc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097c0 tx=0x7f0d8801c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573b86c52d2d65ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d8effd640 1 -- 172.21.15.82:0/2868432447 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d880b0070 con 0x7f0d9015a700 2024-04-25T10:05:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d8effd640 1 -- 172.21.15.82:0/2868432447 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d88004230 con 0x7f0d9015a700 2024-04-25T10:05:03.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d8effd640 1 -- 172.21.15.82:0/2868432447 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88003620 con 0x7f0d9015a700 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 -- 172.21.15.82:0/2868432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d9015a700 msgr2=0x7f0d9015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 --2- 172.21.15.82:0/2868432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d9015a700 0x7f0d9015ecc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0d880097c0 tx=0x7f0d8801c350 comp rx=0 tx=0).stop 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 -- 172.21.15.82:0/2868432447 shutdown_connections 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 --2- 172.21.15.82:0/2868432447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d9015a700 0x7f0d9015ecc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 -- 172.21.15.82:0/2868432447 >> 172.21.15.82:0/2868432447 conn(0x7f0d900b11a0 msgr2=0x7f0d900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 -- 172.21.15.82:0/2868432447 shutdown_connections 2024-04-25T10:05:03.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.787+0000 7f0d964be640 1 -- 172.21.15.82:0/2868432447 wait complete. 2024-04-25T10:05:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 Processor -- start 2024-04-25T10:05:03.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- start start 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d900b24a0 0x7f0d9014a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 0x7f0d901568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900cd310 con 0x7f0d9015a700 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900ce600 con 0x7f0d900b24a0 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d900cd490 con 0x7f0d90157f10 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48884/0 (socket says 172.21.15.82:48884) 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 -- 172.21.15.82:0/4243519478 learned_addr learned my addr 172.21.15.82:0/4243519478 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8ffff640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d900b24a0 0x7f0d9014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d94a5b640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 0x7f0d901568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 -- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 msgr2=0x7f0d901568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 0x7f0d901568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 -- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d900b24a0 msgr2=0x7f0d9014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d900b24a0 0x7f0d9014a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 -- 172.21.15.82:0/4243519478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9015a0d0 con 0x7f0d9015a700 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d94a5b640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 0x7f0d901568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8f7fe640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cb90 tx=0x7f0d8000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80017070 con 0x7f0d9015a700 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8000ace0 con 0x7f0d9015a700 2024-04-25T10:05:03.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9015a3b0 con 0x7f0d9015a700 2024-04-25T10:05:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d80012620 con 0x7f0d9015a700 2024-04-25T10:05:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d901583a0 con 0x7f0d9015a700 2024-04-25T10:05:03.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9004efa0 con 0x7f0d9015a700 2024-04-25T10:05:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d8001a070 con 0x7f0d9015a700 2024-04-25T10:05:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.791+0000 7f0d8cff9640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 0x7f0d70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.795+0000 7f0d8ffff640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 0x7f0d70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.795+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0d80085010 con 0x7f0d9015a700 2024-04-25T10:05:03.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.795+0000 7f0d8ffff640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 0x7f0d70069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ad000 tx=0x7f0d88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:03.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.799+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d80055de0 con 0x7f0d9015a700 2024-04-25T10:05:03.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.991+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d9006f260 con 0x7f0d9015a700 2024-04-25T10:05:03.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.991+0000 7f0d8cff9640 1 -- 172.21.15.82:0/4243519478 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0d800556e0 con 0x7f0d9015a700 2024-04-25T10:05:03.995 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:03.996 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:03.996 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:05:03.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 msgr2=0x7f0d70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 0x7f0d70069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0d880ad000 tx=0x7f0d88008040 comp rx=0 tx=0).stop 2024-04-25T10:05:03.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 msgr2=0x7f0d901599f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0d8000cb90 tx=0x7f0d8000a430 comp rx=0 tx=0).stop 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 shutdown_connections 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d70066f90 0x7f0d70069450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d90157f10 0x7f0d901568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d9015a700 0x7f0d901599f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 --2- 172.21.15.82:0/4243519478 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d900b24a0 0x7f0d9014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 >> 172.21.15.82:0/4243519478 conn(0x7f0d900b11a0 msgr2=0x7f0d900cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 shutdown_connections 2024-04-25T10:05:03.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:03.995+0000 7f0d964be640 1 -- 172.21.15.82:0/4243519478 wait complete. 2024-04-25T10:05:04.015 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:04.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 Processor -- start 2024-04-25T10:05:04.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 -- start start 2024-04-25T10:05:04.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c7815da10 0x7f0c78071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c78071e90 0x7f0c78071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c780adbd0 con 0x7f0c780b24a0 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c780ae860 con 0x7f0c78071e90 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7c868640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7815e210 con 0x7f0c7815da10 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c78071e90 0x7f0c78071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c78071e90 0x7f0c78071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48890/0 (socket says 172.21.15.82:48890) 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 -- 172.21.15.82:0/3562191338 learned_addr learned my addr 172.21.15.82:0/3562191338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c7659c640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c7815da10 0x7f0c78071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 -- 172.21.15.82:0/3562191338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c7815da10 msgr2=0x7f0c78071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c7815da10 0x7f0c78071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 -- 172.21.15.82:0/3562191338 >> v1:172.21.15.196:6789/0 conn(0x7f0c780b24a0 legacy=0x7f0c781614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.127+0000 7f0c76d9d640 1 -- 172.21.15.82:0/3562191338 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7815e4f0 con 0x7f0c78071e90 2024-04-25T10:05:04.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78071e90 0x7f0c78071570 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009e50 tx=0x7f0c7001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c5dc2448d848a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7559a640 1 -- 172.21.15.82:0/3562191338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700a5600 con 0x7f0c78071e90 2024-04-25T10:05:04.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7559a640 1 -- 172.21.15.82:0/3562191338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700a57a0 con 0x7f0c78071e90 2024-04-25T10:05:04.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7559a640 1 -- 172.21.15.82:0/3562191338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700a5b70 con 0x7f0c78071e90 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- 172.21.15.82:0/3562191338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78071e90 msgr2=0x7f0c78071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78071e90 0x7f0c78071570 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009e50 tx=0x7f0c7001c910 comp rx=0 tx=0).stop 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- 172.21.15.82:0/3562191338 shutdown_connections 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c78071e90 0x7f0c78071570 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- 172.21.15.82:0/3562191338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c7815da10 0x7f0c78071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- 172.21.15.82:0/3562191338 >> 172.21.15.82:0/3562191338 conn(0x7f0c780b11a0 msgr2=0x7f0c780b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- 172.21.15.82:0/3562191338 shutdown_connections 2024-04-25T10:05:04.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- 172.21.15.82:0/3562191338 wait complete. 2024-04-25T10:05:04.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 Processor -- start 2024-04-25T10:05:04.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- start start 2024-04-25T10:05:04.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c78071e90 0x7f0c78151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 0x7f0c78151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7816aa10 con 0x7f0c7815da10 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7816a890 con 0x7f0c780b24a0 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c7c868640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7816ab90 con 0x7f0c78071e90 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48896/0 (socket says 172.21.15.82:48896) 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 -- 172.21.15.82:0/769202810 learned_addr learned my addr 172.21.15.82:0/769202810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 -- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c78071e90 msgr2=0x7f0c78151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c78071e90 0x7f0c78151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 -- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 msgr2=0x7f0c78151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c75d9b640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 0x7f0c78151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 0x7f0c78151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c76d9d640 1 -- 172.21.15.82:0/769202810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c780b3720 con 0x7f0c7815da10 2024-04-25T10:05:04.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.131+0000 7f0c75d9b640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 0x7f0c78151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c76d9d640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c7001a540 tx=0x7f0c700196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c700bc020 con 0x7f0c7815da10 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c700b8670 con 0x7f0c7815da10 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7815bbc0 con 0x7f0c7815da10 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c70009510 con 0x7f0c7815da10 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c78152780 con 0x7f0c7815da10 2024-04-25T10:05:04.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7804efa0 con 0x7f0c7815da10 2024-04-25T10:05:04.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c700bf020 con 0x7f0c7815da10 2024-04-25T10:05:04.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 0x7f0c58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0c7011fcb0 con 0x7f0c7815da10 2024-04-25T10:05:04.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c7659c640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 0x7f0c58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.135+0000 7f0c7659c640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 0x7f0c58069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0c781fa060 tx=0x7f0c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.139+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c700c6070 con 0x7f0c7815da10 2024-04-25T10:05:04.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.287+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c780b1dc0 con 0x7f0c7815da10 2024-04-25T10:05:04.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c66ffd640 1 -- 172.21.15.82:0/769202810 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f0c70016620 con 0x7f0c7815da10 2024-04-25T10:05:04.294 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:04.294 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":54,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:49.723093+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:04.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 msgr2=0x7f0c58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 0x7f0c58069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f0c781fa060 tx=0x7f0c6c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 msgr2=0x7f0c78146800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0c7001a540 tx=0x7f0c700196b0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 shutdown_connections 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c58066f60 0x7f0c58069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c7815da10 0x7f0c78146800 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c780b24a0 0x7f0c78151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 --2- 172.21.15.82:0/769202810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c78071e90 0x7f0c78151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 >> 172.21.15.82:0/769202810 conn(0x7f0c780b11a0 msgr2=0x7f0c78150970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 shutdown_connections 2024-04-25T10:05:04.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.291+0000 7f0c7c868640 1 -- 172.21.15.82:0/769202810 wait complete. 2024-04-25T10:05:04.312 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:04:15.148506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:04:15.924015+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:04.313 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:04.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 Processor -- start 2024-04-25T10:05:04.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 -- start start 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2280af370 0x7fd228166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22806ec30 con 0x7fd228066e30 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22806f230 con 0x7fd22806bca0 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22f806640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280669a0 con 0x7fd2280af370 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22cda1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd228066e30 0x7fd22806c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:44944/0 (socket says 172.21.15.82:44944) 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22cda1640 1 -- 172.21.15.82:0/3901282413 learned_addr learned my addr 172.21.15.82:0/3901282413 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:04.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22dda3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2280af370 0x7fd228166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd227fff640 1 -- 172.21.15.82:0/3901282413 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151741997 0 0) 0x7fd22806ec30 con 0x7fd228066e30 2024-04-25T10:05:04.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd227fff640 1 -- 172.21.15.82:0/3901282413 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003530 con 0x7fd228066e30 2024-04-25T10:05:04.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22dda3640 1 -- 172.21.15.82:0/3901282413 >> v1:172.21.15.196:6789/0 conn(0x7fd22806bca0 legacy=0x7fd2280cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22dda3640 1 -- 172.21.15.82:0/3901282413 >> v1:172.21.15.82:6789/0 conn(0x7fd228066e30 legacy=0x7fd22806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.419+0000 7fd22dda3640 1 -- 172.21.15.82:0/3901282413 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22806f510 con 0x7fd2280af370 2024-04-25T10:05:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22dda3640 1 --2- 172.21.15.82:0/3901282413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2280af370 0x7fd228166620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097c0 tx=0x7fd22001c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50640d386fec51b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd227fff640 1 -- 172.21.15.82:0/3901282413 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200b1070 con 0x7fd2280af370 2024-04-25T10:05:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd227fff640 1 -- 172.21.15.82:0/3901282413 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2200a48a0 con 0x7fd2280af370 2024-04-25T10:05:04.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd227fff640 1 -- 172.21.15.82:0/3901282413 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200a4c50 con 0x7fd2280af370 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/3901282413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2280af370 msgr2=0x7fd228166620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 --2- 172.21.15.82:0/3901282413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2280af370 0x7fd228166620 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd2200097c0 tx=0x7fd22001c060 comp rx=0 tx=0).stop 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/3901282413 shutdown_connections 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 --2- 172.21.15.82:0/3901282413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2280af370 0x7fd228166620 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/3901282413 >> 172.21.15.82:0/3901282413 conn(0x7fd228071810 msgr2=0x7fd228071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/3901282413 shutdown_connections 2024-04-25T10:05:04.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/3901282413 wait complete. 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 Processor -- start 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- start start 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd22806bca0 0x7fd228149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2280af370 0x7fd22815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228169f30 con 0x7fd228066e30 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd228169db0 con 0x7fd22806bca0 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22816a0b0 con 0x7fd2280af370 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48906/0 (socket says 172.21.15.82:48906) 2024-04-25T10:05:04.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 -- 172.21.15.82:0/2967534040 learned_addr learned my addr 172.21.15.82:0/2967534040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 -- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2280af370 msgr2=0x7fd22815b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2280af370 0x7fd22815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 -- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd22806bca0 msgr2=0x7fd228149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd22806bca0 0x7fd228149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 -- 172.21.15.82:0/2967534040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22814d250 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22d5a2640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c760 tx=0x7fd21800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218016020 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd218006d60 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd218003bc0 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.423+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd22814c9b0 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd228155240 con 0x7fd228066e30 2024-04-25T10:05:04.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd228155440 con 0x7fd228066e30 2024-04-25T10:05:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd21800a850 con 0x7fd228066e30 2024-04-25T10:05:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd225ffb640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 0x7fd214069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd218084e60 con 0x7fd228066e30 2024-04-25T10:05:04.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd22cda1640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 0x7fd214069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.427+0000 7fd22cda1640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 0x7fd214069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd21c007fd0 tx=0x7fd21c007940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.431+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd228155440 con 0x7fd228066e30 2024-04-25T10:05:04.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd22804efa0 con 0x7fd228066e30 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd225ffb640 1 -- 172.21.15.82:0/2967534040 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd218020020 con 0x7fd228066e30 2024-04-25T10:05:04.582 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 msgr2=0x7fd214069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 0x7fd214069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd21c007fd0 tx=0x7fd21c007940 comp rx=0 tx=0).stop 2024-04-25T10:05:04.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 msgr2=0x7fd2280ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd21800c760 tx=0x7fd21800cc20 comp rx=0 tx=0).stop 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 shutdown_connections 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd214066f60 0x7fd214069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2280af370 0x7fd22815b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd22806bca0 0x7fd228149f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 --2- 172.21.15.82:0/2967534040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd228066e30 0x7fd2280ad530 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 >> 172.21.15.82:0/2967534040 conn(0x7fd228071810 msgr2=0x7fd22806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 shutdown_connections 2024-04-25T10:05:04.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.579+0000 7fd22f806640 1 -- 172.21.15.82:0/2967534040 wait complete. 2024-04-25T10:05:04.600 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:05:04.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.707+0000 7f901fdb7640 1 Processor -- start 2024-04-25T10:05:04.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.707+0000 7f901fdb7640 1 -- start start 2024-04-25T10:05:04.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815b6b0 0x7f90180ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f90180cf590 0x7f90180cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90180cbd20 con 0x7f901815f130 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f901815ba90 con 0x7f90180cf590 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cff30 con 0x7f901815b6b0 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f90180cf590 0x7f90180cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f90180cf590 0x7f90180cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43870/0 (socket says 172.21.15.82:43870) 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 -- 172.21.15.82:0/3973579720 learned_addr learned my addr 172.21.15.82:0/3973579720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901db53640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815b6b0 0x7f90180ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 -- 172.21.15.82:0/3973579720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815b6b0 msgr2=0x7f90180ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815b6b0 0x7f90180ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 -- 172.21.15.82:0/3973579720 >> v1:172.21.15.196:6789/0 conn(0x7f901815f130 legacy=0x7f901815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:04.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 -- 172.21.15.82:0/3973579720 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90180cc000 con 0x7f90180cf590 2024-04-25T10:05:04.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901d352640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f90180cf9f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f900c009870 tx=0x7f900c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d15483ccc40d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901cb51640 1 -- 172.21.15.82:0/3973579720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c0aa610 con 0x7f90180cf590 2024-04-25T10:05:04.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901cb51640 1 -- 172.21.15.82:0/3973579720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900c0af070 con 0x7f90180cf590 2024-04-25T10:05:04.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901cb51640 1 -- 172.21.15.82:0/3973579720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900c0aa9d0 con 0x7f90180cf590 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- 172.21.15.82:0/3973579720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 msgr2=0x7f90180cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f90180cf9f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f900c009870 tx=0x7f900c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- 172.21.15.82:0/3973579720 shutdown_connections 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f90180cf9f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 --2- 172.21.15.82:0/3973579720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815b6b0 0x7f90180ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- 172.21.15.82:0/3973579720 >> 172.21.15.82:0/3973579720 conn(0x7f901806ae60 msgr2=0x7f901806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- 172.21.15.82:0/3973579720 shutdown_connections 2024-04-25T10:05:04.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.711+0000 7f901fdb7640 1 -- 172.21.15.82:0/3973579720 wait complete. 2024-04-25T10:05:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 Processor -- start 2024-04-25T10:05:04.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- start start 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f901815b6b0 0x7f9018157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815f130 0x7f90180c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180ca900 con 0x7f90180cf590 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cba30 con 0x7f901815b6b0 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90180cadb0 con 0x7f901815f130 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43876/0 (socket says 172.21.15.82:43876) 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 -- 172.21.15.82:0/1188433225 learned_addr learned my addr 172.21.15.82:0/1188433225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 -- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815f130 msgr2=0x7f90180c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815f130 0x7f90180c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 -- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f901815b6b0 msgr2=0x7f9018157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f901815b6b0 0x7f9018157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 -- 172.21.15.82:0/1188433225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901815a670 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901db53640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f901000ce40 tx=0x7f901000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010003ba0 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010003d40 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901815a950 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010004260 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901815a240 con 0x7f90180cf590 2024-04-25T10:05:04.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90180c4b80 con 0x7f90180cf590 2024-04-25T10:05:04.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.715+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9010004400 con 0x7f90180cf590 2024-04-25T10:05:04.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.719+0000 7f90167fc640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 0x7f9004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:04.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.719+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9010085cf0 con 0x7f90180cf590 2024-04-25T10:05:04.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.719+0000 7f901d352640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 0x7f9004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:04.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.719+0000 7f901d352640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 0x7f9004069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ae000 tx=0x7f900c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:04.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.723+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f901001d340 con 0x7f90180cf590 2024-04-25T10:05:04.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.879+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90180c4540 con 0x7f90180cf590 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.879+0000 7f90167fc640 1 -- 172.21.15.82:0/1188433225 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9010056b30 con 0x7f90180cf590 2024-04-25T10:05:04.883 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 msgr2=0x7f9004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 0x7f9004069420 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ae000 tx=0x7f900c004740 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 msgr2=0x7f9018159d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f901000ce40 tx=0x7f901000a660 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 shutdown_connections 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9004066f60 0x7f9004069420 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f901815f130 0x7f90180c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f901815b6b0 0x7f9018157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 --2- 172.21.15.82:0/1188433225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f90180cf590 0x7f9018159d00 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 >> 172.21.15.82:0/1188433225 conn(0x7f901806ae60 msgr2=0x7f901806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 shutdown_connections 2024-04-25T10:05:04.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:04.883+0000 7f901fdb7640 1 -- 172.21.15.82:0/1188433225 wait complete. 2024-04-25T10:05:04.901 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 Processor -- start 2024-04-25T10:05:05.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 -- start start 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf580cf590 0x7faf5815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf970 0x7faf580c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf580ca4c0 con 0x7faf580c9ef0 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf580ca780 con 0x7faf580cf590 2024-04-25T10:05:05.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf5e013640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf580cd440 con 0x7faf580cf970 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf580cf590 0x7faf5815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf4ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf970 0x7faf580c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf580cf590 0x7faf5815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:36656/0 (socket says 172.21.15.82:36656) 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf57fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7faf580c9ef0 0x7faf580b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:45038/0 (socket says 172.21.15.82:45038) 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 -- 172.21.15.82:0/4033246315 learned_addr learned my addr 172.21.15.82:0/4033246315 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 -- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf970 msgr2=0x7faf580c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 --2- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf970 0x7faf580c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 -- 172.21.15.82:0/4033246315 >> v1:172.21.15.196:6789/0 conn(0x7faf580c9ef0 legacy=0x7faf580b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.027+0000 7faf577fe640 1 -- 172.21.15.82:0/4033246315 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580cd690 con 0x7faf580cf590 2024-04-25T10:05:05.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf577fe640 1 --2- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf590 0x7faf5815ecb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf48009870 tx=0x7faf4801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=581a447668a1ceb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf56ffd640 1 -- 172.21.15.82:0/4033246315 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480b2070 con 0x7faf580cf590 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf56ffd640 1 -- 172.21.15.82:0/4033246315 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480ad640 con 0x7faf580cf590 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf56ffd640 1 -- 172.21.15.82:0/4033246315 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480ada10 con 0x7faf580cf590 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf590 msgr2=0x7faf5815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf590 0x7faf5815ecb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7faf48009870 tx=0x7faf4801c560 comp rx=0 tx=0).stop 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- 172.21.15.82:0/4033246315 shutdown_connections 2024-04-25T10:05:05.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf970 0x7faf580c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- 172.21.15.82:0/4033246315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf590 0x7faf5815ecb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- 172.21.15.82:0/4033246315 >> 172.21.15.82:0/4033246315 conn(0x7faf5806ae60 msgr2=0x7faf5806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- 172.21.15.82:0/4033246315 shutdown_connections 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- 172.21.15.82:0/4033246315 wait complete. 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 Processor -- start 2024-04-25T10:05:05.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- start start 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf590 0x7faf580c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf970 0x7faf5814abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580cbf60 con 0x7faf580c9ef0 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580cbde0 con 0x7faf580cf970 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf5e013640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf580cc0e0 con 0x7faf580cf590 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43892/0 (socket says 172.21.15.82:43892) 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.031+0000 7faf577fe640 1 -- 172.21.15.82:0/3107537552 learned_addr learned my addr 172.21.15.82:0/3107537552 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf57fff640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf970 0x7faf5814abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf4ffff640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf590 0x7faf580c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 -- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf590 msgr2=0x7faf580c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf590 0x7faf580c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 -- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf970 msgr2=0x7faf5814abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf970 0x7faf5814abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 -- 172.21.15.82:0/3107537552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5814b450 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf577fe640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7faf480b0040 tx=0x7faf48019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480b2040 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf480ad440 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf5814b730 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf480b6650 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5814bcf0 con 0x7faf580c9ef0 2024-04-25T10:05:05.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5804efa0 con 0x7faf580c9ef0 2024-04-25T10:05:05.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf480bf030 con 0x7faf580c9ef0 2024-04-25T10:05:05.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 0x7faf38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7faf4811e6f0 con 0x7faf580c9ef0 2024-04-25T10:05:05.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf4ffff640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 0x7faf38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.035+0000 7faf4ffff640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 0x7faf38069450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf40009970 tx=0x7faf40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.039+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf480c6060 con 0x7faf580c9ef0 2024-04-25T10:05:05.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.195+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf580b4170 con 0x7faf580c9ef0 2024-04-25T10:05:05.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf54ff9640 1 -- 172.21.15.82:0/3107537552 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7faf48016150 con 0x7faf580c9ef0 2024-04-25T10:05:05.201 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 msgr2=0x7faf38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 0x7faf38069450 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7faf40009970 tx=0x7faf40008040 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 msgr2=0x7faf581fae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7faf480b0040 tx=0x7faf48019300 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 shutdown_connections 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf38066f90 0x7faf38069450 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf580cf590 0x7faf580c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf580cf970 0x7faf5814abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 --2- 172.21.15.82:0/3107537552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf580c9ef0 0x7faf581fae50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 >> 172.21.15.82:0/3107537552 conn(0x7faf5806ae60 msgr2=0x7faf58156710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 shutdown_connections 2024-04-25T10:05:05.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.199+0000 7faf5e013640 1 -- 172.21.15.82:0/3107537552 wait complete. 2024-04-25T10:05:05.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:05.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.335+0000 7f1474718640 1 Processor -- start 2024-04-25T10:05:05.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.335+0000 7f1474718640 1 -- start start 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 0x7f146c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f146c144ca0 0x7f146c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146c0cee30 con 0x7f146c06ec60 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146c0cc2b0 con 0x7f146c144ca0 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c06e290 con 0x7f146c0ce510 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f146c144ca0 0x7f146c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1472cb5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f146c06ec60 0x7f146c142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45684/0 (socket says 172.21.15.82:45684) 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1472cb5640 1 -- 172.21.15.82:0/3898437865 learned_addr learned my addr 172.21.15.82:0/3898437865 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:05.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14724b4640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 0x7f146c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 -- 172.21.15.82:0/3898437865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 msgr2=0x7f146c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 0x7f146c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 -- 172.21.15.82:0/3898437865 >> v1:172.21.15.82:6789/0 conn(0x7f146c06ec60 legacy=0x7f146c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 -- 172.21.15.82:0/3898437865 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0ccc70 con 0x7f146c144ca0 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14714b2640 1 -- 172.21.15.82:0/3898437865 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343642654 0 0) 0x7f146c0cee30 con 0x7f146c06ec60 2024-04-25T10:05:05.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14724b4640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 0x7f146c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1471cb3640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c144ca0 0x7f146c06e720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f14600097c0 tx=0x7f146001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c36c6df6fc6761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14714b2640 1 -- 172.21.15.82:0/3898437865 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14600b5070 con 0x7f146c144ca0 2024-04-25T10:05:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14714b2640 1 -- 172.21.15.82:0/3898437865 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14600a8990 con 0x7f146c144ca0 2024-04-25T10:05:05.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f14714b2640 1 -- 172.21.15.82:0/3898437865 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14600a8d60 con 0x7f146c144ca0 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 -- 172.21.15.82:0/3898437865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c144ca0 msgr2=0x7f146c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.339+0000 7f1474718640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c144ca0 0x7f146c06e720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f14600097c0 tx=0x7f146001ec90 comp rx=0 tx=0).stop 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3898437865 shutdown_connections 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c144ca0 0x7f146c06e720 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 --2- 172.21.15.82:0/3898437865 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c0ce510 0x7f146c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3898437865 >> 172.21.15.82:0/3898437865 conn(0x7f146c0b11a0 msgr2=0x7f146c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3898437865 shutdown_connections 2024-04-25T10:05:05.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3898437865 wait complete. 2024-04-25T10:05:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 Processor -- start 2024-04-25T10:05:05.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- start start 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c06ec60 0x7f146c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f146c144ca0 0x7f146c13fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c0cc900 con 0x7f146c0ce510 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c06df20 con 0x7f146c144ca0 2024-04-25T10:05:05.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146c142d90 con 0x7f146c06ec60 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43900/0 (socket says 172.21.15.82:43900) 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 -- 172.21.15.82:0/3139981741 learned_addr learned my addr 172.21.15.82:0/3139981741 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 -- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c06ec60 msgr2=0x7f146c15afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1472cb5640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f146c144ca0 0x7f146c13fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f14724b4640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c06ec60 0x7f146c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c06ec60 0x7f146c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 -- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f146c144ca0 msgr2=0x7f146c13fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f146c144ca0 0x7f146c13fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 -- 172.21.15.82:0/3139981741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c0c9680 con 0x7f146c0ce510 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1471cb3640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f146001c8c0 tx=0x7f146001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14600b5040 con 0x7f146c0ce510 2024-04-25T10:05:05.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14600bf070 con 0x7f146c0ce510 2024-04-25T10:05:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c0c9960 con 0x7f146c0ce510 2024-04-25T10:05:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14600b0a60 con 0x7f146c0ce510 2024-04-25T10:05:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c0c9f20 con 0x7f146c0ce510 2024-04-25T10:05:05.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.343+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c04efa0 con 0x7f146c0ce510 2024-04-25T10:05:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.347+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14600b0c00 con 0x7f146c0ce510 2024-04-25T10:05:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.347+0000 7f146affd640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 0x7f1458069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.347+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1460122770 con 0x7f146c0ce510 2024-04-25T10:05:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.347+0000 7f14724b4640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 0x7f1458069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.347+0000 7f14724b4640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 0x7f1458069450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f145c006fd0 tx=0x7f145c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.351+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14600187f0 con 0x7f146c0ce510 2024-04-25T10:05:05.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.543+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f146c0b1cc0 con 0x7f146c0ce510 2024-04-25T10:05:05.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.543+0000 7f146affd640 1 -- 172.21.15.82:0/3139981741 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f14600af3a0 con 0x7f146c0ce510 2024-04-25T10:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:05.547 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:05.547 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 msgr2=0x7f1458069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 0x7f1458069450 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f145c006fd0 tx=0x7f145c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 msgr2=0x7f146c1f2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f146001c8c0 tx=0x7f146001ba30 comp rx=0 tx=0).stop 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 shutdown_connections 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1458066f90 0x7f1458069450 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f146c144ca0 0x7f146c13fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f146c0ce510 0x7f146c1f2940 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 --2- 172.21.15.82:0/3139981741 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f146c06ec60 0x7f146c15afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 >> 172.21.15.82:0/3139981741 conn(0x7f146c0b11a0 msgr2=0x7f146c0c4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 shutdown_connections 2024-04-25T10:05:05.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.547+0000 7f1474718640 1 -- 172.21.15.82:0/3139981741 wait complete. 2024-04-25T10:05:05.565 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.675+0000 7f575df1a640 1 Processor -- start 2024-04-25T10:05:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.675+0000 7f575df1a640 1 -- start start 2024-04-25T10:05:05.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f575df1a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f57580d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f575df1a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57580ce6b0 con 0x7f575815d850 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f575df1a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57580ce990 con 0x7f575815dc30 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f575df1a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57580cec70 con 0x7f57580d0870 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f57577fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f575815d850 0x7f57580d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45698/0 (socket says 172.21.15.82:45698) 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f57577fe640 1 -- 172.21.15.82:0/1384610106 learned_addr learned my addr 172.21.15.82:0/1384610106 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f574ffff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f57580d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232556769 0 0) 0x7f57580ce6b0 con 0x7f575815d850 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5744003530 con 0x7f575815d850 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3941259414 0 0) 0x7f5744003530 con 0x7f575815d850 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 msgr2=0x7f57580d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f57580d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 >> v1:172.21.15.196:6789/0 conn(0x7f575815dc30 legacy=0x7f5758167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580cef50 con 0x7f575815d850 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5748001680 con 0x7f575815d850 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 >> v1:172.21.15.82:6789/0 conn(0x7f575815d850 legacy=0x7f57580d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f5744003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004090 0x7f5744004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5744004da0 0x7f5744005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57580cec70 con 0x7f5744004da0 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574400bbe0 con 0x7f5744004090 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574400bde0 con 0x7f57580d0870 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5744004da0 0x7f5744005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f57577fe640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f5744003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f574ffff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004090 0x7f5744004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 -- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 msgr2=0x7f5744003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580d0870 0x7f5744003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 -- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004090 msgr2=0x7f5744004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004090 0x7f5744004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 -- 172.21.15.82:0/1384610106 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744003530 con 0x7f5744004da0 2024-04-25T10:05:05.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5757fff640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5744004da0 0x7f5744005220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5750001e70 tx=0x7f575001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b75e13662898df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750001a90 con 0x7f5744004da0 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5750003b80 con 0x7f5744004da0 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.679+0000 7f5756ffd640 1 -- 172.21.15.82:0/1384610106 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750005080 con 0x7f5744004da0 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/1384610106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5744004da0 msgr2=0x7f5744005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 --2- 172.21.15.82:0/1384610106 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5744004da0 0x7f5744005220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f5750001e70 tx=0x7f575001c780 comp rx=0 tx=0).stop 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f5757fff640 1 -- 172.21.15.82:0/1384610106 reap_dead start 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/1384610106 shutdown_connections 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/1384610106 >> 172.21.15.82:0/1384610106 conn(0x7f575808b000 msgr2=0x7f57580d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/1384610106 shutdown_connections 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/1384610106 wait complete. 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 Processor -- start 2024-04-25T10:05:05.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- start start 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004da0 0x7f57581f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580b55c0 0x7f57580b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57580ce490 con 0x7f57580b30b0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57580ce310 con 0x7f5744004da0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57580d1de0 con 0x7f57580b55c0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43918/0 (socket says 172.21.15.82:43918) 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 -- 172.21.15.82:0/4181087461 learned_addr learned my addr 172.21.15.82:0/4181087461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f5757fff640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580b55c0 0x7f57580b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f57577fe640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004da0 0x7f57581f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 -- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580b55c0 msgr2=0x7f57580b5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580b55c0 0x7f57580b5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 -- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004da0 msgr2=0x7f57581f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004da0 0x7f57581f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 -- 172.21.15.82:0/4181087461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57580cfed0 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f574ffff640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f574000f650 tx=0x7f574000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5740019020 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f574000d900 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57580cd9d0 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f574000dcf0 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.683+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57580b3b00 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57580b4500 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f574001c020 con 0x7f57580b30b0 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f5754ff9640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 0x7f5738069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5740085ad0 con 0x7f57580b30b0 2024-04-25T10:05:05.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f57577fe640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 0x7f5738069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.687+0000 7f57577fe640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 0x7f5738069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f57580c9590 tx=0x7f5748009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.691+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5740023030 con 0x7f57580b30b0 2024-04-25T10:05:05.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.839+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f575804efa0 con 0x7f57580b30b0 2024-04-25T10:05:05.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.839+0000 7f5754ff9640 1 -- 172.21.15.82:0/4181087461 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16993 (secure 0 0 0) 0x7f5740026390 con 0x7f57580b30b0 2024-04-25T10:05:05.844 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:05.844 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":54,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:49.723093+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:05.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 msgr2=0x7f5738069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 0x7f5738069450 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f57580c9590 tx=0x7f5748009040 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 msgr2=0x7f57580cd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f574000f650 tx=0x7f574000fb10 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 shutdown_connections 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5738066f90 0x7f5738069450 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57580b55c0 0x7f57580b5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57580b30b0 0x7f57580cd490 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 --2- 172.21.15.82:0/4181087461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5744004da0 0x7f57581f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 >> 172.21.15.82:0/4181087461 conn(0x7f575808b000 msgr2=0x7f57580b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 shutdown_connections 2024-04-25T10:05:05.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.843+0000 7f575df1a640 1 -- 172.21.15.82:0/4181087461 wait complete. 2024-04-25T10:05:05.862 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:05:05.862 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:05.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 Processor -- start 2024-04-25T10:05:05.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 -- start start 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c06ac90 0x7fed2c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 0x7fed2c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed2c06b5b0 con 0x7fed2c0cc130 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c15ff70 con 0x7fed2c06ac90 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed33f89640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2c0d0300 con 0x7fed2c15ca90 2024-04-25T10:05:05.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c06ac90 0x7fed2c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed2c06ac90 0x7fed2c06b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43928/0 (socket says 172.21.15.82:43928) 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 -- 172.21.15.82:0/3622064456 learned_addr learned my addr 172.21.15.82:0/3622064456 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 -- 172.21.15.82:0/3622064456 >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 msgr2=0x7fed2c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31524640 1 --2- 172.21.15.82:0/3622064456 >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 0x7fed2c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 --2- 172.21.15.82:0/3622064456 >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 0x7fed2c0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 -- 172.21.15.82:0/3622064456 >> v1:172.21.15.196:6789/0 conn(0x7fed2c0cc130 legacy=0x7fed2c0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31d25640 1 -- 172.21.15.82:0/3622064456 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c15f360 con 0x7fed2c06ac90 2024-04-25T10:05:05.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.979+0000 7fed31524640 1 --2- 172.21.15.82:0/3622064456 >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 0x7fed2c0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:05.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed31d25640 1 --2- 172.21.15.82:0/3622064456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c06b070 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06dbb0 tx=0x7fed2801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a2569dc941a38e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed30d23640 1 -- 172.21.15.82:0/3622064456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280aa610 con 0x7fed2c06ac90 2024-04-25T10:05:05.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed30d23640 1 -- 172.21.15.82:0/3622064456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed280af070 con 0x7fed2c06ac90 2024-04-25T10:05:05.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed30d23640 1 -- 172.21.15.82:0/3622064456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280aa9d0 con 0x7fed2c06ac90 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- 172.21.15.82:0/3622064456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 msgr2=0x7fed2c06b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 --2- 172.21.15.82:0/3622064456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c06b070 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fed2c06dbb0 tx=0x7fed2801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- 172.21.15.82:0/3622064456 shutdown_connections 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 --2- 172.21.15.82:0/3622064456 >> v2:172.21.15.196:3300/0 conn(0x7fed2c15ca90 0x7fed2c0d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 --2- 172.21.15.82:0/3622064456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c06b070 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- 172.21.15.82:0/3622064456 >> 172.21.15.82:0/3622064456 conn(0x7fed2c099a70 msgr2=0x7fed2c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- 172.21.15.82:0/3622064456 shutdown_connections 2024-04-25T10:05:05.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- 172.21.15.82:0/3622064456 wait complete. 2024-04-25T10:05:05.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 Processor -- start 2024-04-25T10:05:05.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.983+0000 7fed33f89640 1 -- start start 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc130 0x7fed2c15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c15ca90 0x7fed2c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c159750 con 0x7fed2c06ac90 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c1595d0 con 0x7fed2c0cc130 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2c168660 con 0x7fed2c15ca90 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43934/0 (socket says 172.21.15.82:43934) 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 -- 172.21.15.82:0/3813602630 learned_addr learned my addr 172.21.15.82:0/3813602630 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 -- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c15ca90 msgr2=0x7fed2c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c15ca90 0x7fed2c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 -- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc130 msgr2=0x7fed2c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc130 0x7fed2c15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 -- 172.21.15.82:0/3813602630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2c1f72b0 con 0x7fed2c06ac90 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31d25640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed2801cd50 tx=0x7fed2801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280bd5f0 con 0x7fed2c06ac90 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed280bd790 con 0x7fed2c06ac90 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2c151d10 con 0x7fed2c06ac90 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280bdb60 con 0x7fed2c06ac90 2024-04-25T10:05:05.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2c1522d0 con 0x7fed2c06ac90 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed2c152510 con 0x7fed2c06ac90 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed280b33f0 con 0x7fed2c06ac90 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31524640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fed2811f9d0 con 0x7fed2c06ac90 2024-04-25T10:05:05.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.987+0000 7fed31524640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 0x7fed10069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed2c155af0 tx=0x7fed1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:05.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:05.991+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed280c3060 con 0x7fed2c06ac90 2024-04-25T10:05:06.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.179+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed2c04efa0 con 0x7fed2c06ac90 2024-04-25T10:05:06.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.179+0000 7fed227fc640 1 -- 172.21.15.82:0/3813602630 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fed280a7070 con 0x7fed2c06ac90 2024-04-25T10:05:06.184 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:06.185 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:06.185 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 msgr2=0x7fed10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 0x7fed10069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fed2c155af0 tx=0x7fed1c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 msgr2=0x7fed2c1f7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed2801cd50 tx=0x7fed2801acf0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 shutdown_connections 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed10066f60 0x7fed10069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed2c15ca90 0x7fed2c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed2c0cc130 0x7fed2c15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 --2- 172.21.15.82:0/3813602630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed2c06ac90 0x7fed2c1f7ca0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 >> 172.21.15.82:0/3813602630 conn(0x7fed2c099a70 msgr2=0x7fed2c0cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 shutdown_connections 2024-04-25T10:05:06.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.183+0000 7fed33f89640 1 -- 172.21.15.82:0/3813602630 wait complete. 2024-04-25T10:05:06.201 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 Processor -- start 2024-04-25T10:05:06.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 -- start start 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e8415a700 0x7f7e8415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e840b3f50 0x7f7e840cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e840b2090 con 0x7f7e840ce430 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e8415f200 con 0x7f7e8415a700 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8c763640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840b43e0 con 0x7f7e840b3f50 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e8415a700 0x7f7e8415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8ad00640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7e840ce430 0x7f7e840b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45700/0 (socket says 172.21.15.82:45700) 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e8415a700 0x7f7e8415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43950/0 (socket says 172.21.15.82:43950) 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8ad00640 1 -- 172.21.15.82:0/445182119 learned_addr learned my addr 172.21.15.82:0/445182119 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e89cfe640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e840b3f50 0x7f7e840cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/445182119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e840b3f50 msgr2=0x7f7e840cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e840b3f50 0x7f7e840cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/445182119 >> v1:172.21.15.82:6789/0 conn(0x7f7e840ce430 legacy=0x7f7e840b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e894fd640 1 -- 172.21.15.82:0/445182119 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070812160 0 0) 0x7f7e840b2090 con 0x7f7e840ce430 2024-04-25T10:05:06.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.319+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/445182119 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840b46c0 con 0x7f7e8415a700 2024-04-25T10:05:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8a4ff640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e8415a700 0x7f7e8415ecc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097c0 tx=0x7f7e8001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8119d72d8ddf68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e894fd640 1 -- 172.21.15.82:0/445182119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800b0070 con 0x7f7e8415a700 2024-04-25T10:05:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e894fd640 1 -- 172.21.15.82:0/445182119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e80004230 con 0x7f7e8415a700 2024-04-25T10:05:06.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e894fd640 1 -- 172.21.15.82:0/445182119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e80003620 con 0x7f7e8415a700 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- 172.21.15.82:0/445182119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e8415a700 msgr2=0x7f7e8415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e8415a700 0x7f7e8415ecc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097c0 tx=0x7f7e8001c350 comp rx=0 tx=0).stop 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- 172.21.15.82:0/445182119 shutdown_connections 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e840b3f50 0x7f7e840cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 --2- 172.21.15.82:0/445182119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e8415a700 0x7f7e8415ecc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- 172.21.15.82:0/445182119 >> 172.21.15.82:0/445182119 conn(0x7f7e840b11a0 msgr2=0x7f7e840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- 172.21.15.82:0/445182119 shutdown_connections 2024-04-25T10:05:06.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- 172.21.15.82:0/445182119 wait complete. 2024-04-25T10:05:06.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 Processor -- start 2024-04-25T10:05:06.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.323+0000 7f7e8c763640 1 -- start start 2024-04-25T10:05:06.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 0x7f7e84071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e84071950 0x7f7e84156a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840cd490 con 0x7f7e840ce430 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840cd310 con 0x7f7e8415a700 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e840b2bb0 con 0x7f7e84071950 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43964/0 (socket says 172.21.15.82:43964) 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/420148007 learned_addr learned my addr 172.21.15.82:0/420148007 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e89cfe640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 0x7f7e84071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8ad00640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e84071950 0x7f7e84156a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e84071950 msgr2=0x7f7e84156a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e84071950 0x7f7e84156a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 msgr2=0x7f7e84071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 0x7f7e84071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 -- 172.21.15.82:0/420148007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e84157700 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e89cfe640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 0x7f7e84071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8a4ff640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7e80019f80 tx=0x7f7e800190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800b9070 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e800043e0 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e841579e0 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800b0040 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84157fa0 con 0x7f7e840ce430 2024-04-25T10:05:06.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8404efa0 con 0x7f7e840ce430 2024-04-25T10:05:06.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e800bc070 con 0x7f7e840ce430 2024-04-25T10:05:06.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 0x7f7e68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7e8011f540 con 0x7f7e840ce430 2024-04-25T10:05:06.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e89cfe640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 0x7f7e68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.327+0000 7f7e89cfe640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 0x7f7e68069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e840723d0 tx=0x7f7e74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.331+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e80016090 con 0x7f7e840ce430 2024-04-25T10:05:06.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.523+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e8406f260 con 0x7f7e840ce430 2024-04-25T10:05:06.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e7affd640 1 -- 172.21.15.82:0/420148007 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f7e800f0420 con 0x7f7e840ce430 2024-04-25T10:05:06.529 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:06.529 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:04:16.915215+0000","modified":"2024-04-25T10:04:18.967239+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5946},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5946":{"gid":5946,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/238219074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":238219074},{"type":"v1","addr":"172.21.15.82:6837","nonce":238219074}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:05:06.529 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 24 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 msgr2=0x7f7e68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 0x7f7e68069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7e840723d0 tx=0x7f7e74008040 comp rx=0 tx=0).stop 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 msgr2=0x7f7e8414dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f7e80019f80 tx=0x7f7e800190f0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 shutdown_connections 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e68066f90 0x7f7e68069450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e84071950 0x7f7e84156a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e8415a700 0x7f7e84071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 --2- 172.21.15.82:0/420148007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e840ce430 0x7f7e8414dfd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 >> 172.21.15.82:0/420148007 conn(0x7f7e840b11a0 msgr2=0x7f7e840aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 shutdown_connections 2024-04-25T10:05:06.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.527+0000 7f7e8c763640 1 -- 172.21.15.82:0/420148007 wait complete. 2024-04-25T10:05:06.547 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:06.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:06.656 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:06.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.655+0000 7fad19aae640 1 Processor -- start 2024-04-25T10:05:06.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.655+0000 7fad19aae640 1 -- start start 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fad1409a180 0x7fad140cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad140afb90 0x7fad140cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1409a950 con 0x7fad1415b920 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1409ac30 con 0x7fad140afb90 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad140ae380 con 0x7fad1409a180 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad140afb90 0x7fad140cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fad140afb90 0x7fad140cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43968/0 (socket says 172.21.15.82:43968) 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 -- 172.21.15.82:0/938521120 learned_addr learned my addr 172.21.15.82:0/938521120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad12ffd640 1 --2- 172.21.15.82:0/938521120 >> v2:172.21.15.196:3300/0 conn(0x7fad1409a180 0x7fad140cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 -- 172.21.15.82:0/938521120 >> v2:172.21.15.196:3300/0 conn(0x7fad1409a180 msgr2=0x7fad140cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 --2- 172.21.15.82:0/938521120 >> v2:172.21.15.196:3300/0 conn(0x7fad1409a180 0x7fad140cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 -- 172.21.15.82:0/938521120 >> v1:172.21.15.196:6789/0 conn(0x7fad1415b920 legacy=0x7fad14099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 -- 172.21.15.82:0/938521120 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad140ae660 con 0x7fad140afb90 2024-04-25T10:05:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad13fff640 1 --2- 172.21.15.82:0/938521120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140afb90 0x7fad140cc670 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad14072a50 tx=0x7fad0c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0cb572fc869a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad127fc640 1 -- 172.21.15.82:0/938521120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0a5780 con 0x7fad140afb90 2024-04-25T10:05:06.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad127fc640 1 -- 172.21.15.82:0/938521120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0a5920 con 0x7fad140afb90 2024-04-25T10:05:06.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad127fc640 1 -- 172.21.15.82:0/938521120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0a5cf0 con 0x7fad140afb90 2024-04-25T10:05:06.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 -- 172.21.15.82:0/938521120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140afb90 msgr2=0x7fad140cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.659+0000 7fad19aae640 1 --2- 172.21.15.82:0/938521120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140afb90 0x7fad140cc670 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fad14072a50 tx=0x7fad0c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/938521120 shutdown_connections 2024-04-25T10:05:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 --2- 172.21.15.82:0/938521120 >> v2:172.21.15.196:3300/0 conn(0x7fad1409a180 0x7fad140cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 --2- 172.21.15.82:0/938521120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad140afb90 0x7fad140cc670 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/938521120 >> 172.21.15.82:0/938521120 conn(0x7fad140b3010 msgr2=0x7fad140b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/938521120 shutdown_connections 2024-04-25T10:05:06.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/938521120 wait complete. 2024-04-25T10:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 Processor -- start 2024-04-25T10:05:06.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- start start 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1409a180 0x7fad14165bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad140afb90 0x7fad14158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad141684c0 con 0x7fad1415b920 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad140cd5d0 con 0x7fad1409a180 2024-04-25T10:05:06.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad14168640 con 0x7fad140afb90 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43970/0 (socket says 172.21.15.82:43970) 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 -- 172.21.15.82:0/3095757153 learned_addr learned my addr 172.21.15.82:0/3095757153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad137fe640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1409a180 0x7fad14165bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad12ffd640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad140afb90 0x7fad14158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 -- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad140afb90 msgr2=0x7fad14158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad140afb90 0x7fad14158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 -- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1409a180 msgr2=0x7fad14165bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1409a180 0x7fad14165bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 -- 172.21.15.82:0/3095757153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad141594c0 con 0x7fad1415b920 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad13fff640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0af040 tx=0x7fad0c019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009510 con 0x7fad1415b920 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0be070 con 0x7fad1415b920 2024-04-25T10:05:06.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c0b8ae0 con 0x7fad1415b920 2024-04-25T10:05:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad14164150 con 0x7fad1415b920 2024-04-25T10:05:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.663+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad14164710 con 0x7fad1415b920 2024-04-25T10:05:06.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1404efa0 con 0x7fad1415b920 2024-04-25T10:05:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad0c0b8c80 con 0x7fad1415b920 2024-04-25T10:05:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7facfbfff640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fad0c120380 con 0x7fad1415b920 2024-04-25T10:05:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7fad137fe640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.667+0000 7fad137fe640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 0x7fad00069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fad04007fd0 tx=0x7fad04007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.671+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad0c0165a0 con 0x7fad1415b920 2024-04-25T10:05:06.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.819+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad14099f30 con 0x7fad1415b920 2024-04-25T10:05:06.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.819+0000 7facfbfff640 1 -- 172.21.15.82:0/3095757153 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16993 (secure 0 0 0) 0x7fad0c0a58f0 con 0x7fad1415b920 2024-04-25T10:05:06.824 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:06.824 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":54,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:04:49.723093+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:04:15.148506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:04:15.924015+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.23","mappings":[{"from":0,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 msgr2=0x7fad00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 0x7fad00069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fad04007fd0 tx=0x7fad04007e40 comp rx=0 tx=0).stop 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 msgr2=0x7fad1415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0af040 tx=0x7fad0c019750 comp rx=0 tx=0).stop 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 shutdown_connections 2024-04-25T10:05:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad00066f60 0x7fad00069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad140afb90 0x7fad14158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad1415b920 0x7fad1415ab00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 --2- 172.21.15.82:0/3095757153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad1409a180 0x7fad14165bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 >> 172.21.15.82:0/3095757153 conn(0x7fad140b3010 msgr2=0x7fad14070a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 shutdown_connections 2024-04-25T10:05:06.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.823+0000 7fad19aae640 1 -- 172.21.15.82:0/3095757153 wait complete. 2024-04-25T10:05:06.842 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:05:06.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.955+0000 7fc21414a640 1 Processor -- start 2024-04-25T10:05:06.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.955+0000 7fc21414a640 1 -- start start 2024-04-25T10:05:06.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc21414a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc20c15a700 0x7fc20c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc21414a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc20c0b24a0 0x7fc20c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc21414a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc20c0b2090 con 0x7fc20c0b3f50 2024-04-25T10:05:06.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc21414a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20c15f200 con 0x7fc20c15a700 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc21414a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc20c0b43e0 con 0x7fc20c0b24a0 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc20c15a700 0x7fc20c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc20c15a700 0x7fc20c15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43972/0 (socket says 172.21.15.82:43972) 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 -- 172.21.15.82:0/4003108638 learned_addr learned my addr 172.21.15.82:0/4003108638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc2126e7640 1 --2- 172.21.15.82:0/4003108638 >> v2:172.21.15.196:3300/0 conn(0x7fc20c0b24a0 0x7fc20c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 -- 172.21.15.82:0/4003108638 >> v2:172.21.15.196:3300/0 conn(0x7fc20c0b24a0 msgr2=0x7fc20c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 --2- 172.21.15.82:0/4003108638 >> v2:172.21.15.196:3300/0 conn(0x7fc20c0b24a0 0x7fc20c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 -- 172.21.15.82:0/4003108638 >> v1:172.21.15.196:6789/0 conn(0x7fc20c0b3f50 legacy=0x7fc20c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:06.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 -- 172.21.15.82:0/4003108638 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0b46c0 con 0x7fc20c15a700 2024-04-25T10:05:06.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc211ee6640 1 --2- 172.21.15.82:0/4003108638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c15ecc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06bd20 tx=0x7fc20801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdf4179970bc121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc210ee4640 1 -- 172.21.15.82:0/4003108638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b0070 con 0x7fc20c15a700 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc210ee4640 1 -- 172.21.15.82:0/4003108638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2080ab600 con 0x7fc20c15a700 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.959+0000 7fc210ee4640 1 -- 172.21.15.82:0/4003108638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ab9d0 con 0x7fc20c15a700 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/4003108638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 msgr2=0x7fc20c15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- 172.21.15.82:0/4003108638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c15ecc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc20c06bd20 tx=0x7fc20801c3a0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/4003108638 shutdown_connections 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- 172.21.15.82:0/4003108638 >> v2:172.21.15.196:3300/0 conn(0x7fc20c0b24a0 0x7fc20c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- 172.21.15.82:0/4003108638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c15ecc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/4003108638 >> 172.21.15.82:0/4003108638 conn(0x7fc20c0b11a0 msgr2=0x7fc20c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/4003108638 shutdown_connections 2024-04-25T10:05:06.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/4003108638 wait complete. 2024-04-25T10:05:06.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 Processor -- start 2024-04-25T10:05:06.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- start start 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20c0b24a0 0x7fc20c13e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c06d5d0 0x7fc20c0ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0cd310 con 0x7fc20c15a700 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0ce600 con 0x7fc20c06d5d0 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c0cd490 con 0x7fc20c0b24a0 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43976/0 (socket says 172.21.15.82:43976) 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 -- 172.21.15.82:0/1999081086 learned_addr learned my addr 172.21.15.82:0/1999081086 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc211ee6640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20c0b24a0 0x7fc20c13e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2126e7640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c06d5d0 0x7fc20c0ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 -- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20c0b24a0 msgr2=0x7fc20c13e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20c0b24a0 0x7fc20c13e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 -- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c06d5d0 msgr2=0x7fc20c0ad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c06d5d0 0x7fc20c0ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 -- 172.21.15.82:0/1999081086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c1533b0 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2116e5640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c480 tx=0x7fc1fc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc012830 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1fc017070 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c0add80 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc012c30 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c0ae340 con 0x7fc20c15a700 2024-04-25T10:05:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.963+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c06d9e0 con 0x7fc20c15a700 2024-04-25T10:05:06.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.967+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc1fc01c460 con 0x7fc20c15a700 2024-04-25T10:05:06.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.967+0000 7fc2027fc640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:06.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.967+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc1fc085ac0 con 0x7fc20c15a700 2024-04-25T10:05:06.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.967+0000 7fc211ee6640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:06.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.967+0000 7fc211ee6640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 0x7fc1f0069450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad000 tx=0x7fc208008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:06.971+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc20c06d9e0 con 0x7fc20c15a700 2024-04-25T10:05:07.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.143+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc20c04efa0 con 0x7fc20c15a700 2024-04-25T10:05:07.149 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:05:07.147+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:05:07.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.147+0000 7fc2027fc640 1 -- 172.21.15.82:0/1999081086 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc1fc0569f0 con 0x7fc20c15a700 2024-04-25T10:05:07.151 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 msgr2=0x7fc1f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 0x7fc1f0069450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad000 tx=0x7fc208008040 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 msgr2=0x7fc20c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc00c480 tx=0x7fc1fc00c940 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 shutdown_connections 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc1f0066f90 0x7fc1f0069450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc20c06d5d0 0x7fc20c0ad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc20c15a700 0x7fc20c06d090 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 --2- 172.21.15.82:0/1999081086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc20c0b24a0 0x7fc20c13e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 >> 172.21.15.82:0/1999081086 conn(0x7fc20c0b11a0 msgr2=0x7fc20c0cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 shutdown_connections 2024-04-25T10:05:07.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.151+0000 7fc21414a640 1 -- 172.21.15.82:0/1999081086 wait complete. 2024-04-25T10:05:07.161 INFO:tasks.ceph.mds.a.smithi082.stderr: -4893> 2024-04-25T10:04:23.555+0000 7fa41c76d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:07.166 INFO:tasks.ceph.mds.a.smithi082.stderr: -3223> 2024-04-25T10:04:30.515+0000 7fa418765640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6141=0-4194304@1} caps={6141=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6141 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x556b6d765700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:07.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:05:07.204 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:05:07.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 Processor -- start 2024-04-25T10:05:07.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 -- start start 2024-04-25T10:05:07.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7c80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cbd20 con 0x7fc7c80cfad0 2024-04-25T10:05:07.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cc000 con 0x7fc7c815b420 2024-04-25T10:05:07.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cf118640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cff20 con 0x7fc7c815b800 2024-04-25T10:05:07.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cd6b5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc7c80cfad0 0x7fc7c8072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45704/0 (socket says 172.21.15.82:45704) 2024-04-25T10:05:07.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cd6b5640 1 -- 172.21.15.82:0/1626398335 learned_addr learned my addr 172.21.15.82:0/1626398335 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:07.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7c80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972229963 0 0) 0x7fc7c80cbd20 con 0x7fc7c80cfad0 2024-04-25T10:05:07.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b8003530 con 0x7fc7c80cfad0 2024-04-25T10:05:07.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362278907 0 0) 0x7fc7b8003530 con 0x7fc7c80cfad0 2024-04-25T10:05:07.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 msgr2=0x7fc7c80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7c80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 >> v1:172.21.15.196:6789/0 conn(0x7fc7c815b420 legacy=0x7fc7c80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80cc2e0 con 0x7fc7c80cfad0 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc7c0001a00 con 0x7fc7c80cfad0 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 >> v1:172.21.15.82:6789/0 conn(0x7fc7c80cfad0 legacy=0x7fc7c8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7b8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b8004050 0x7fc7b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7b8004d50 0x7fc7b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cff20 con 0x7fc7b8004050 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b800bb60 con 0x7fc7b8004d50 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b800bd60 con 0x7fc7c815b800 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b8004050 0x7fc7b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cceb4640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7b8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.279+0000 7fc7cd6b5640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7b8004d50 0x7fc7b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 -- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 msgr2=0x7fc7b8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7b8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 -- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7b8004d50 msgr2=0x7fc7b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:07.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7b8004d50 0x7fc7b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 -- 172.21.15.82:0/1626398335 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8003530 con 0x7fc7b8004050 2024-04-25T10:05:07.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cceb4640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7c815b800 0x7fc7b8003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b8004050 0x7fc7b8004810 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815bbe0 tx=0x7fc7b401c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7de8e116911283 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b400b600 con 0x7fc7b8004050 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40b0070 con 0x7fc7b8004050 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7c77fe640 1 -- 172.21.15.82:0/1626398335 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40ab890 con 0x7fc7b8004050 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 -- 172.21.15.82:0/1626398335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b8004050 msgr2=0x7fc7b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 --2- 172.21.15.82:0/1626398335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7b8004050 0x7fc7b8004810 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc7c815bbe0 tx=0x7fc7b401c640 comp rx=0 tx=0).stop 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cd6b5640 1 -- 172.21.15.82:0/1626398335 reap_dead start 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 -- 172.21.15.82:0/1626398335 shutdown_connections 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 -- 172.21.15.82:0/1626398335 >> 172.21.15.82:0/1626398335 conn(0x7fc7c806ae60 msgr2=0x7fc7c806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 -- 172.21.15.82:0/1626398335 shutdown_connections 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 -- 172.21.15.82:0/1626398335 wait complete. 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.283+0000 7fc7cf118640 1 Processor -- start 2024-04-25T10:05:07.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- start start 2024-04-25T10:05:07.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7b8004050 0x7fc7c815e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7c8159630 0x7fc7c8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cb9f0 con 0x7fc7c81591d0 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cc930 con 0x7fc7c8159630 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cadb0 con 0x7fc7b8004050 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44016/0 (socket says 172.21.15.82:44016) 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 -- 172.21.15.82:0/3009716925 learned_addr learned my addr 172.21.15.82:0/3009716925 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 -- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7b8004050 msgr2=0x7fc7c815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7b8004050 0x7fc7c815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 -- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7c8159630 msgr2=0x7fc7c8155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7c8159630 0x7fc7c8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 -- 172.21.15.82:0/3009716925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c815d370 con 0x7fc7c81591d0 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c7fff640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c10 tx=0x7fc7b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40ba070 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b40abc10 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4004020 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c815d650 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c815dc10 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c804efa0 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7b40041c0 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 0x7fc7ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fc7b411fd90 con 0x7fc7c81591d0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cceb4640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 0x7fc7ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:07.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.287+0000 7fc7cceb4640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 0x7fc7ac069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ba10 tx=0x7fc7bc00b0c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:07.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.291+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7b40c4060 con 0x7fc7c81591d0 2024-04-25T10:05:07.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:07.475+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc7c815fc80 con 0x7fc7c81591d0 2024-04-25T10:05:08.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.155+0000 7fc7c57fa640 1 -- 172.21.15.82:0/3009716925 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc7b4016350 con 0x7fc7c81591d0 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 msgr2=0x7fc7ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 0x7fc7ac069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00ba10 tx=0x7fc7bc00b0c0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 msgr2=0x7fc7c8155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc7b4000c10 tx=0x7fc7b8005840 comp rx=0 tx=0).stop 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 shutdown_connections 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc7ac066f60 0x7fc7ac069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7c8159630 0x7fc7c8155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7c81591d0 0x7fc7c8155720 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 --2- 172.21.15.82:0/3009716925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7b8004050 0x7fc7c815e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 >> 172.21.15.82:0/3009716925 conn(0x7fc7c806ae60 msgr2=0x7fc7c80ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 shutdown_connections 2024-04-25T10:05:08.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.159+0000 7fc7cf118640 1 -- 172.21.15.82:0/3009716925 wait complete. 2024-04-25T10:05:08.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:05:08.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.351+0000 7fa6fddc9640 1 Processor -- start 2024-04-25T10:05:08.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.351+0000 7fa6fddc9640 1 -- start start 2024-04-25T10:05:08.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.351+0000 7fa6fddc9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6f815b840 0x7fa6f809a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.351+0000 7fa6fddc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6f8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.351+0000 7fa6fddc9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80cd400 con 0x7fa6f8099fb0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6fddc9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80b08e0 con 0x7fa6f815b840 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6fddc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b1180 con 0x7fa6f80ccbf0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa6f8099fb0 0x7fa6f80cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45712/0 (socket says 172.21.15.82:45712) 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f7fff640 1 -- 172.21.15.82:0/3447220859 learned_addr learned my addr 172.21.15.82:0/3447220859 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3447220859 >> v2:172.21.15.196:3300/0 conn(0x7fa6f815b840 0x7fa6f809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6f8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614251438 0 0) 0x7fa6f80cd400 con 0x7fa6f8099fb0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e0003530 con 0x7fa6f8099fb0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053810047 0 0) 0x7fa6e0003530 con 0x7fa6f8099fb0 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 msgr2=0x7fa6f8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6f8099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 >> v2:172.21.15.196:3300/0 conn(0x7fa6f815b840 msgr2=0x7fa6f809a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 --2- 172.21.15.82:0/3447220859 >> v2:172.21.15.196:3300/0 conn(0x7fa6f815b840 0x7fa6f809a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80b0b30 con 0x7fa6f8099fb0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa6f0001a00 con 0x7fa6f8099fb0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6f8099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 >> v1:172.21.15.82:6789/0 conn(0x7fa6f8099fb0 legacy=0x7fa6f80cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e00039c0 0x7fa6e0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0004810 0x7fa6e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b08e0 con 0x7fa6e00039c0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80b1180 con 0x7fa6e0004810 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e000ba50 con 0x7fa6f80ccbf0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e00039c0 0x7fa6e0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0004810 0x7fa6e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f7fff640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 -- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 msgr2=0x7fa6e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6f80ccbf0 0x7fa6e00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 -- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0004810 msgr2=0x7fa6e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0004810 0x7fa6e0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 -- 172.21.15.82:0/3447220859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0003530 con 0x7fa6e00039c0 2024-04-25T10:05:08.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6e0004810 0x7fa6e0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:08.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e00039c0 0x7fa6e0003de0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009ae0 tx=0x7fa6e801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8bb622b2969e0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80b2070 con 0x7fa6e00039c0 2024-04-25T10:05:08.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e80a5510 con 0x7fa6e00039c0 2024-04-25T10:05:08.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.355+0000 7fa6f67fc640 1 -- 172.21.15.82:0/3447220859 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e80096b0 con 0x7fa6e00039c0 2024-04-25T10:05:08.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3447220859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e00039c0 msgr2=0x7fa6e0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3447220859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6e00039c0 0x7fa6e0003de0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa6e8009ae0 tx=0x7fa6e801cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f7fff640 1 -- 172.21.15.82:0/3447220859 reap_dead start 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3447220859 shutdown_connections 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3447220859 >> 172.21.15.82:0/3447220859 conn(0x7fa6f80b3030 msgr2=0x7fa6f80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3447220859 shutdown_connections 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3447220859 wait complete. 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 Processor -- start 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- start start 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 0x7fa6f80cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f806bda0 0x7fa6f806c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f81663f0 con 0x7fa6f80c8610 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80cd1e0 con 0x7fa6f806bda0 2024-04-25T10:05:08.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f8166570 con 0x7fa6e00039c0 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44040/0 (socket says 172.21.15.82:44040) 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 -- 172.21.15.82:0/3238522110 learned_addr learned my addr 172.21.15.82:0/3238522110 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 0x7fa6f80cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f7fff640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f806bda0 0x7fa6f806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 -- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 msgr2=0x7fa6f80cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 0x7fa6f80cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 -- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f806bda0 msgr2=0x7fa6f806c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f806bda0 0x7fa6f806c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 -- 172.21.15.82:0/3238522110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f806c760 con 0x7fa6f80c8610 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 0x7fa6f80cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6f6ffd640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0108c0 tx=0x7fa6ec010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec01a020 con 0x7fa6f80c8610 2024-04-25T10:05:08.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00eb60 con 0x7fa6f80c8610 2024-04-25T10:05:08.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f806c9f0 con 0x7fa6f80c8610 2024-04-25T10:05:08.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00f040 con 0x7fa6f80c8610 2024-04-25T10:05:08.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f806cfb0 con 0x7fa6f80c8610 2024-04-25T10:05:08.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.359+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f8068340 con 0x7fa6f80c8610 2024-04-25T10:05:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.363+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6ec01d020 con 0x7fa6f80c8610 2024-04-25T10:05:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.363+0000 7fa6dffff640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 0x7fa6e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.363+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa6ec085ae0 con 0x7fa6f80c8610 2024-04-25T10:05:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.363+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 0x7fa6e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:08.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.363+0000 7fa6f77fe640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 0x7fa6e4069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80af000 tx=0x7fa6e80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:08.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.367+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6ec024020 con 0x7fa6f80c8610 2024-04-25T10:05:08.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:08.527+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa6f80cdb00 con 0x7fa6f80c8610 2024-04-25T10:05:09.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.159+0000 7fa6dffff640 1 -- 172.21.15.82:0/3238522110 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa6ec0568d0 con 0x7fa6f80c8610 2024-04-25T10:05:09.165 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:05:09.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.163+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 msgr2=0x7fa6e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.163+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 0x7fa6e4069420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80af000 tx=0x7fa6e80046d0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.163+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 msgr2=0x7fa6f80c8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.163+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec0108c0 tx=0x7fa6ec010d80 comp rx=0 tx=0).stop 2024-04-25T10:05:09.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 shutdown_connections 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6e4066f60 0x7fa6e4069420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f806bda0 0x7fa6f806c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f80c8610 0x7fa6f80c8a90 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 --2- 172.21.15.82:0/3238522110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6e00039c0 0x7fa6f80cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 >> 172.21.15.82:0/3238522110 conn(0x7fa6f80b3030 msgr2=0x7fa6f80cefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 shutdown_connections 2024-04-25T10:05:09.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.167+0000 7fa6fddc9640 1 -- 172.21.15.82:0/3238522110 wait complete. 2024-04-25T10:05:09.190 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:05:09.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 Processor -- start 2024-04-25T10:05:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 -- start start 2024-04-25T10:05:09.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad940ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad940b0a50 con 0x7fad9415e7e0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9415f420 con 0x7fad940b1040 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad990b6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b1a60 con 0x7fad9415e0c0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fad9415e7e0 0x7fad9415ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45714/0 (socket says 172.21.15.82:45714) 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 -- 172.21.15.82:0/3763482169 learned_addr learned my addr 172.21.15.82:0/3763482169 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692674634 0 0) 0x7fad940b0a50 con 0x7fad9415e7e0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad80003530 con 0x7fad9415e7e0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9359e640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad940ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2110964072 0 0) 0x7fad80003530 con 0x7fad9415e7e0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 msgr2=0x7fad940ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad940ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 >> v1:172.21.15.196:6789/0 conn(0x7fad940b1040 legacy=0x7fad9415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940b0cf0 con 0x7fad9415e7e0 2024-04-25T10:05:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fad84001680 con 0x7fad9415e7e0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 >> v1:172.21.15.82:6789/0 conn(0x7fad9415e7e0 legacy=0x7fad9415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad800039f0 0x7fad80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad80004e70 0x7fad8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad940b1a60 con 0x7fad800039f0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad80005630 con 0x7fad80004e70 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8000bf40 con 0x7fad9415e0c0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad800039f0 0x7fad80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad8a59c640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9359e640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad80004e70 0x7fad8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 -- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 msgr2=0x7fad80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad9415e0c0 0x7fad80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 -- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad80004e70 msgr2=0x7fad8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad80004e70 0x7fad8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad92d9d640 1 -- 172.21.15.82:0/3763482169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad80003530 con 0x7fad800039f0 2024-04-25T10:05:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.319+0000 7fad9359e640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad80004e70 0x7fad8000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad92d9d640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad800039f0 0x7fad80003e10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad8400c1c0 tx=0x7fad8401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f679e95221a636a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84007d00 con 0x7fad800039f0 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad840a9020 con 0x7fad800039f0 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad9259c640 1 -- 172.21.15.82:0/3763482169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840a5b90 con 0x7fad800039f0 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- 172.21.15.82:0/3763482169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad800039f0 msgr2=0x7fad80003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 --2- 172.21.15.82:0/3763482169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad800039f0 0x7fad80003e10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fad8400c1c0 tx=0x7fad8401cc10 comp rx=0 tx=0).stop 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad9359e640 1 -- 172.21.15.82:0/3763482169 reap_dead start 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- 172.21.15.82:0/3763482169 shutdown_connections 2024-04-25T10:05:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- 172.21.15.82:0/3763482169 >> 172.21.15.82:0/3763482169 conn(0x7fad940b1fa0 msgr2=0x7fad940b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- 172.21.15.82:0/3763482169 shutdown_connections 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- 172.21.15.82:0/3763482169 wait complete. 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 Processor -- start 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.323+0000 7fad990b6640 1 -- start start 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad800039f0 0x7fad941f8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad940b4480 0x7fad940af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad94166450 con 0x7fad940b5ce0 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad941662d0 con 0x7fad800039f0 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad941665d0 con 0x7fad940b4480 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44058/0 (socket says 172.21.15.82:44058) 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 -- 172.21.15.82:0/1322187483 learned_addr learned my addr 172.21.15.82:0/1322187483 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:09.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad92d9d640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad800039f0 0x7fad941f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad9359e640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad940b4480 0x7fad940af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 -- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad940b4480 msgr2=0x7fad940af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad940b4480 0x7fad940af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 -- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad800039f0 msgr2=0x7fad941f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad800039f0 0x7fad941f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 -- 172.21.15.82:0/1322187483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940b4c40 con 0x7fad940b5ce0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8a59c640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e4d0 tx=0x7fad7c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0053a0 con 0x7fad940b5ce0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c018070 con 0x7fad940b5ce0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad940b4f20 con 0x7fad940b5ce0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c003820 con 0x7fad940b5ce0 2024-04-25T10:05:09.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad940b54e0 con 0x7fad940b5ce0 2024-04-25T10:05:09.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad94068340 con 0x7fad940b5ce0 2024-04-25T10:05:09.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fad7c0039c0 con 0x7fad940b5ce0 2024-04-25T10:05:09.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 0x7fad74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:09.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5807+0+0 (secure 0 0 0) 0x7fad7c0849a0 con 0x7fad940b5ce0 2024-04-25T10:05:09.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad92d9d640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 0x7fad74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:09.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.327+0000 7fad92d9d640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 0x7fad74069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad840aa000 tx=0x7fad940b2ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:09.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.331+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fad7c055950 con 0x7fad940b5ce0 2024-04-25T10:05:09.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:09.491+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fad940b6410 con 0x7fad940b5ce0 2024-04-25T10:05:10.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.170+0000 7fad8bfff640 1 -- 172.21.15.82:0/1322187483 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fad7c055250 con 0x7fad940b5ce0 2024-04-25T10:05:10.174 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:05:10.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 msgr2=0x7fad74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 0x7fad74069450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fad840aa000 tx=0x7fad940b2ff0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 msgr2=0x7fad94156990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00e4d0 tx=0x7fad7c00e990 comp rx=0 tx=0).stop 2024-04-25T10:05:10.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 shutdown_connections 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fad74066f90 0x7fad74069450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fad940b4480 0x7fad940af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fad940b5ce0 0x7fad94156990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 --2- 172.21.15.82:0/1322187483 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fad800039f0 0x7fad941f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 >> 172.21.15.82:0/1322187483 conn(0x7fad940b1fa0 msgr2=0x7fad940cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 shutdown_connections 2024-04-25T10:05:10.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.174+0000 7fad990b6640 1 -- 172.21.15.82:0/1322187483 wait complete. 2024-04-25T10:05:10.204 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:05:10.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.326+0000 7fc626c11640 1 Processor -- start 2024-04-25T10:05:10.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.326+0000 7fc626c11640 1 -- start start 2024-04-25T10:05:10.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc626c11640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6201634c0 0x7fc6201658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc626c11640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6200ca4c0 con 0x7fc6200cf590 2024-04-25T10:05:10.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc626c11640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6200cd690 con 0x7fc62015ecb0 2024-04-25T10:05:10.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc626c11640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6200ca780 con 0x7fc6201634c0 2024-04-25T10:05:10.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc6200cf590 0x7fc6200cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45728/0 (socket says 172.21.15.82:45728) 2024-04-25T10:05:10.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 -- 172.21.15.82:0/1614014974 learned_addr learned my addr 172.21.15.82:0/1614014974 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:10.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976813522 0 0) 0x7fc6200ca4c0 con 0x7fc6200cf590 2024-04-25T10:05:10.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc60c005740 con 0x7fc6200cf590 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 289242388 0 0) 0x7fc60c005740 con 0x7fc6200cf590 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 >> v2:172.21.15.196:3300/0 conn(0x7fc6201634c0 msgr2=0x7fc6201658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 --2- 172.21.15.82:0/1614014974 >> v2:172.21.15.196:3300/0 conn(0x7fc6201634c0 0x7fc6201658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 >> v1:172.21.15.196:6789/0 conn(0x7fc62015ecb0 legacy=0x7fc6200c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6200cd970 con 0x7fc6200cf590 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc618001680 con 0x7fc6200cf590 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 >> v1:172.21.15.82:6789/0 conn(0x7fc6200cf590 legacy=0x7fc6200cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 0x7fc60c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60c006a20 0x7fc60c006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c0075e0 0x7fc60c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c00e420 con 0x7fc60c005c40 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c00e5d0 con 0x7fc60c006a20 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c00e7d0 con 0x7fc60c0075e0 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 0x7fc60c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 -- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c0075e0 msgr2=0x7fc60c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c0075e0 0x7fc60c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 -- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60c006a20 msgr2=0x7fc60c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc60c006a20 0x7fc60c006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 -- 172.21.15.82:0/1614014974 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c005740 con 0x7fc60c005c40 2024-04-25T10:05:10.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6251ae640 1 -- 172.21.15.82:0/1614014974 reap_dead start 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc6249ad640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 0x7fc60c006040 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c240 tx=0x7fc61801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946a0722a84f1b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618007d40 con 0x7fc60c005c40 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6180a9070 con 0x7fc60c005c40 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.330+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1614014974 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc618005d80 con 0x7fc60c005c40 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 msgr2=0x7fc60c006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 0x7fc60c006040 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fc61800c240 tx=0x7fc61801cca0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/1614014974 shutdown_connections 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 --2- 172.21.15.82:0/1614014974 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc60c005c40 0x7fc60c006040 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/1614014974 >> 172.21.15.82:0/1614014974 conn(0x7fc62006ae60 msgr2=0x7fc62006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/1614014974 shutdown_connections 2024-04-25T10:05:10.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/1614014974 wait complete. 2024-04-25T10:05:10.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 Processor -- start 2024-04-25T10:05:10.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- start start 2024-04-25T10:05:10.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c005c40 0x7fc620162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6200b1580 0x7fc6201f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200b2540 con 0x7fc620098c00 2024-04-25T10:05:10.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200b23c0 con 0x7fc6200b1580 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6200b26c0 con 0x7fc60c005c40 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44066/0 (socket says 172.21.15.82:44066) 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 -- 172.21.15.82:0/2459340444 learned_addr learned my addr 172.21.15.82:0/2459340444 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc6249ad640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c005c40 0x7fc620162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 -- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c005c40 msgr2=0x7fc620162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c005c40 0x7fc620162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 -- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6200b1580 msgr2=0x7fc6201f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6200b1580 0x7fc6201f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 -- 172.21.15.82:0/2459340444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6201f72b0 con 0x7fc620098c00 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61ffff640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c7f0 tx=0x7fc60800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc608017070 con 0x7fc620098c00 2024-04-25T10:05:10.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc608003db0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6200cc2c0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080052b0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.334+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6200986b0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62004efa0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc60801a070 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc61d7fa640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc6080844c0 con 0x7fc620098c00 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc6249ad640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.338+0000 7fc6249ad640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 0x7fc600069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ab000 tx=0x7fc6180a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.342+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc6201f72b0 con 0x7fc620098c00 2024-04-25T10:05:10.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.518+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc6201568e0 con 0x7fc620098c00 2024-04-25T10:05:10.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.522+0000 7fc61d7fa640 1 -- 172.21.15.82:0/2459340444 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc6080556d0 con 0x7fc620098c00 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 msgr2=0x7fc600069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 0x7fc600069450 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc6180ab000 tx=0x7fc6180a7070 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 msgr2=0x7fc6201f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc60800c7f0 tx=0x7fc60800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 shutdown_connections 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc600066f90 0x7fc600069450 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc6200b1580 0x7fc6201f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc620098c00 0x7fc6201f6830 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 --2- 172.21.15.82:0/2459340444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc60c005c40 0x7fc620162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 >> 172.21.15.82:0/2459340444 conn(0x7fc62006ae60 msgr2=0x7fc6200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 shutdown_connections 2024-04-25T10:05:10.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.526+0000 7fc626c11640 1 -- 172.21.15.82:0/2459340444 wait complete. 2024-04-25T10:05:10.545 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T10:05:10.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 Processor -- start 2024-04-25T10:05:10.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 -- start start 2024-04-25T10:05:10.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f17b806ec60 0x7f17b8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b80ccc70 con 0x7f17b80ce510 2024-04-25T10:05:10.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b80cc2b0 con 0x7f17b806ec60 2024-04-25T10:05:10.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17be578640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b806e290 con 0x7f17b8144ca0 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17b77fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17bcb15640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f17b806ec60 0x7f17b8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17b7fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f17b80ce510 0x7f17b80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:45152/0 (socket says 172.21.15.82:45152) 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17b7fff640 1 -- 172.21.15.82:0/2079904718 learned_addr learned my addr 172.21.15.82:0/2079904718 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.650+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2079904718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:43874/0 (socket says 172.21.15.82:43874) 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b6ffd640 1 -- 172.21.15.82:0/2079904718 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543402308 0 0) 0x7f17b80ccc70 con 0x7f17b80ce510 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b6ffd640 1 -- 172.21.15.82:0/2079904718 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17a0003530 con 0x7f17b80ce510 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b77fe640 1 -- 172.21.15.82:0/2079904718 >> v2:172.21.15.196:3300/0 conn(0x7f17b806ec60 msgr2=0x7f17b8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2079904718 >> v2:172.21.15.196:3300/0 conn(0x7f17b806ec60 0x7f17b8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b77fe640 1 -- 172.21.15.82:0/2079904718 >> v1:172.21.15.196:6789/0 conn(0x7f17b80ce510 legacy=0x7f17b80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b77fe640 1 -- 172.21.15.82:0/2079904718 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b806dc40 con 0x7f17b8144ca0 2024-04-25T10:05:10.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2079904718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0097c0 tx=0x7f17ac01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb73ed9e8257cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b6ffd640 1 -- 172.21.15.82:0/2079904718 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0b5070 con 0x7f17b8144ca0 2024-04-25T10:05:10.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b6ffd640 1 -- 172.21.15.82:0/2079904718 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac0a8990 con 0x7f17b8144ca0 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17b6ffd640 1 -- 172.21.15.82:0/2079904718 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0a8d40 con 0x7f17b8144ca0 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 -- 172.21.15.82:0/2079904718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 msgr2=0x7f17b806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 --2- 172.21.15.82:0/2079904718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0097c0 tx=0x7f17ac01ec90 comp rx=0 tx=0).stop 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 -- 172.21.15.82:0/2079904718 shutdown_connections 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 --2- 172.21.15.82:0/2079904718 >> v2:172.21.15.196:3300/0 conn(0x7f17b806ec60 0x7f17b8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 --2- 172.21.15.82:0/2079904718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b8144ca0 0x7f17b806e720 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 -- 172.21.15.82:0/2079904718 >> 172.21.15.82:0/2079904718 conn(0x7f17b80b11a0 msgr2=0x7f17b80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 -- 172.21.15.82:0/2079904718 shutdown_connections 2024-04-25T10:05:10.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.654+0000 7f17be578640 1 -- 172.21.15.82:0/2079904718 wait complete. 2024-04-25T10:05:10.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 Processor -- start 2024-04-25T10:05:10.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- start start 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b80ce510 0x7f17b80b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b8144ca0 0x7f17b8132e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b81451f0 con 0x7f17b806ec60 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b80cc900 con 0x7f17b8144ca0 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b8145370 con 0x7f17b80ce510 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44070/0 (socket says 172.21.15.82:44070) 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 -- 172.21.15.82:0/2770516915 learned_addr learned my addr 172.21.15.82:0/2770516915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:10.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17bcb15640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b8144ca0 0x7f17b8132e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b80ce510 0x7f17b80b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 -- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b80ce510 msgr2=0x7f17b80b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b80ce510 0x7f17b80b0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 -- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b8144ca0 msgr2=0x7f17b8132e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b8144ca0 0x7f17b8132e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 -- 172.21.15.82:0/2770516915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b81e4bb0 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b7fff640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c450 tx=0x7f17a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8017070 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a8005cf0 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b8144220 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a8007290 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b81312a0 con 0x7f17b806ec60 2024-04-25T10:05:10.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b804efa0 con 0x7f17b806ec60 2024-04-25T10:05:10.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17a801a070 con 0x7f17b806ec60 2024-04-25T10:05:10.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.658+0000 7f17b4ff9640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 0x7f17a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:10.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.662+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f17a8084240 con 0x7f17b806ec60 2024-04-25T10:05:10.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.662+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 0x7f17a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:10.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.662+0000 7f17b77fe640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 0x7f17a4069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0b2000 tx=0x7f17ac0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:10.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.666+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17b81e4bb0 con 0x7f17b806ec60 2024-04-25T10:05:10.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:10.862+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f17b80b0f80 con 0x7f17b806ec60 2024-04-25T10:05:11.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.170+0000 7f17b4ff9640 1 -- 172.21.15.82:0/2770516915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v176) v1 ==== 149+0+0 (secure 0 0 0) 0x7f17a8055450 con 0x7f17b806ec60 2024-04-25T10:05:11.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 msgr2=0x7f17a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 0x7f17a4069420 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f17ac0b2000 tx=0x7f17ac0aa070 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 msgr2=0x7f17b81e4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f17a800c450 tx=0x7f17a800c910 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 shutdown_connections 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17a4066f60 0x7f17a4069420 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17b80ce510 0x7f17b80b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f17b8144ca0 0x7f17b8132e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 --2- 172.21.15.82:0/2770516915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f17b806ec60 0x7f17b81e4670 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 >> 172.21.15.82:0/2770516915 conn(0x7f17b80b11a0 msgr2=0x7f17b80c55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 shutdown_connections 2024-04-25T10:05:11.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.174+0000 7f17be578640 1 -- 172.21.15.82:0/2770516915 wait complete. 2024-04-25T10:05:11.199 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T10:05:11.199 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:05:11.200 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:05:11.200 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:11.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 Processor -- start 2024-04-25T10:05:11.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 -- start start 2024-04-25T10:05:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5a8144ca0 0x7fc5a806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5a806ec60 0x7fc5a8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5a80ccc70 con 0x7fc5a80ce510 2024-04-25T10:05:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a80cc2b0 con 0x7fc5a8144ca0 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc5adadb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5a806e290 con 0x7fc5a806ec60 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5a8144ca0 0x7fc5a806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5a8144ca0 0x7fc5a806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44082/0 (socket says 172.21.15.82:44082) 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 -- 172.21.15.82:0/340876874 learned_addr learned my addr 172.21.15.82:0/340876874 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 -- 172.21.15.82:0/340876874 >> v2:172.21.15.196:3300/0 conn(0x7fc5a806ec60 msgr2=0x7fc5a8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 --2- 172.21.15.82:0/340876874 >> v2:172.21.15.196:3300/0 conn(0x7fc5a806ec60 0x7fc5a8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 -- 172.21.15.82:0/340876874 >> v1:172.21.15.196:6789/0 conn(0x7fc5a80ce510 legacy=0x7fc5a80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.318+0000 7fc59effd640 1 -- 172.21.15.82:0/340876874 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a806dc40 con 0x7fc5a8144ca0 2024-04-25T10:05:11.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc59effd640 1 --2- 172.21.15.82:0/340876874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a8144ca0 0x7fc5a806e720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b48f0 tx=0x7fc59001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22439f5382146f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5a6ffd640 1 -- 172.21.15.82:0/340876874 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900b5070 con 0x7fc5a8144ca0 2024-04-25T10:05:11.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5a6ffd640 1 -- 172.21.15.82:0/340876874 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5900a89d0 con 0x7fc5a8144ca0 2024-04-25T10:05:11.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5a6ffd640 1 -- 172.21.15.82:0/340876874 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5900a8da0 con 0x7fc5a8144ca0 2024-04-25T10:05:11.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 -- 172.21.15.82:0/340876874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a8144ca0 msgr2=0x7fc5a806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 --2- 172.21.15.82:0/340876874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a8144ca0 0x7fc5a806e720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80b48f0 tx=0x7fc59001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 -- 172.21.15.82:0/340876874 shutdown_connections 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 --2- 172.21.15.82:0/340876874 >> v2:172.21.15.196:3300/0 conn(0x7fc5a806ec60 0x7fc5a8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 --2- 172.21.15.82:0/340876874 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a8144ca0 0x7fc5a806e720 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 -- 172.21.15.82:0/340876874 >> 172.21.15.82:0/340876874 conn(0x7fc5a80b11a0 msgr2=0x7fc5a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 -- 172.21.15.82:0/340876874 shutdown_connections 2024-04-25T10:05:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.322+0000 7fc5adadb640 1 -- 172.21.15.82:0/340876874 wait complete. 2024-04-25T10:05:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 Processor -- start 2024-04-25T10:05:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- start start 2024-04-25T10:05:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a80ce510 0x7fc5a81f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5a8144ca0 0x7fc5a81f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a81451f0 con 0x7fc5a806ec60 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a80cc900 con 0x7fc5a80ce510 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5a8145370 con 0x7fc5a8144ca0 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44096/0 (socket says 172.21.15.82:44096) 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 -- 172.21.15.82:0/1351049039 learned_addr learned my addr 172.21.15.82:0/1351049039 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 -- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5a8144ca0 msgr2=0x7fc5a81f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5a8144ca0 0x7fc5a81f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 -- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a80ce510 msgr2=0x7fc5a81f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a80ce510 0x7fc5a81f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 -- 172.21.15.82:0/1351049039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a8134790 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a77fe640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c490 tx=0x7fc598001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598017070 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598003600 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5a80c8d30 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598003a30 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a81e0d80 con 0x7fc5a806ec60 2024-04-25T10:05:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5a804efa0 con 0x7fc5a806ec60 2024-04-25T10:05:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc59801a070 con 0x7fc5a806ec60 2024-04-25T10:05:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc598084320 con 0x7fc5a806ec60 2024-04-25T10:05:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.326+0000 7fc59effd640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.330+0000 7fc59effd640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 0x7fc588069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc5900b2000 tx=0x7fc590004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.330+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5980554a0 con 0x7fc5a806ec60 2024-04-25T10:05:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5a80cc4b0 con 0x7fc5a806ec60 2024-04-25T10:05:11.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5a4ff9640 1 -- 172.21.15.82:0/1351049039 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+13303 (secure 0 0 0) 0x7fc598054da0 con 0x7fc5a806ec60 2024-04-25T10:05:11.498 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:11.499 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":57,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:10.162149+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/238219074":"2024-04-26T10:05:07.147956+0000","172.21.15.82:6836/238219074":"2024-04-26T10:05:07.147956+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 msgr2=0x7fc588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 0x7fc588069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc5900b2000 tx=0x7fc590004740 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 msgr2=0x7fc5a815b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.494+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fc59800c490 tx=0x7fc598001e80 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 shutdown_connections 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc588066f60 0x7fc588069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5a8144ca0 0x7fc5a81f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5a80ce510 0x7fc5a81f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 --2- 172.21.15.82:0/1351049039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5a806ec60 0x7fc5a815b050 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 >> 172.21.15.82:0/1351049039 conn(0x7fc5a80b11a0 msgr2=0x7fc5a80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 shutdown_connections 2024-04-25T10:05:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.498+0000 7fc5adadb640 1 -- 172.21.15.82:0/1351049039 wait complete. 2024-04-25T10:05:11.518 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:11.518 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.634+0000 7fe5a772b640 1 Processor -- start 2024-04-25T10:05:11.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.634+0000 7fe5a772b640 1 -- start start 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe5a00cf590 0x7fe5a00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 0x7fe5a015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a015ba90 con 0x7fe5a015b6b0 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a00cbd20 con 0x7fe5a00cf590 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a772b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00cc000 con 0x7fe5a015f130 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44112/0 (socket says 172.21.15.82:44112) 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 -- 172.21.15.82:0/1810854774 learned_addr learned my addr 172.21.15.82:0/1810854774 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a4cc6640 1 --2- 172.21.15.82:0/1810854774 >> v2:172.21.15.196:3300/0 conn(0x7fe5a00cf590 0x7fe5a00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a5cc8640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 0x7fe5a015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 -- 172.21.15.82:0/1810854774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 msgr2=0x7fe5a015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 0x7fe5a015f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 -- 172.21.15.82:0/1810854774 >> v2:172.21.15.196:3300/0 conn(0x7fe5a00cf590 msgr2=0x7fe5a00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/1810854774 >> v2:172.21.15.196:3300/0 conn(0x7fe5a00cf590 0x7fe5a00cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 -- 172.21.15.82:0/1810854774 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00cc2e0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a5cc8640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 0x7fe5a015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00b4430 tx=0x7fe59801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488c0f0da431936a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe597fff640 1 -- 172.21.15.82:0/1810854774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a5760 con 0x7fe5a015b6b0 2024-04-25T10:05:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe597fff640 1 -- 172.21.15.82:0/1810854774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5980a5900 con 0x7fe5a015b6b0 2024-04-25T10:05:11.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.638+0000 7fe597fff640 1 -- 172.21.15.82:0/1810854774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a5cd0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/1810854774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 msgr2=0x7fe5a00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe5a00b4430 tx=0x7fe59801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/1810854774 shutdown_connections 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a015f130 0x7fe5a015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- 172.21.15.82:0/1810854774 >> v2:172.21.15.196:3300/0 conn(0x7fe5a00cf590 0x7fe5a00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- 172.21.15.82:0/1810854774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a00ca3c0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/1810854774 >> 172.21.15.82:0/1810854774 conn(0x7fe5a006ae60 msgr2=0x7fe5a006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/1810854774 shutdown_connections 2024-04-25T10:05:11.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/1810854774 wait complete. 2024-04-25T10:05:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 Processor -- start 2024-04-25T10:05:11.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- start start 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a00cf590 0x7fe5a00c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 0x7fe5a00c3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00ca900 con 0x7fe5a015b6b0 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a015fef0 con 0x7fe5a015f130 2024-04-25T10:05:11.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a00cadb0 con 0x7fe5a00cf590 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44118/0 (socket says 172.21.15.82:44118) 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 -- 172.21.15.82:0/3104130578 learned_addr learned my addr 172.21.15.82:0/3104130578 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a5cc8640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 0x7fe5a00c3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a00cf590 0x7fe5a00c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 -- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a00cf590 msgr2=0x7fe5a00c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a00cf590 0x7fe5a00c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 -- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 msgr2=0x7fe5a00c3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 0x7fe5a00c3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 -- 172.21.15.82:0/3104130578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a00c5500 con 0x7fe5a015b6b0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a5cc8640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 0x7fe5a00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a4cc6640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d830 tx=0x7fe59000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000b6c0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590019070 con 0x7fe5a015b6b0 2024-04-25T10:05:11.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a00c5780 con 0x7fe5a015b6b0 2024-04-25T10:05:11.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000ba60 con 0x7fe5a015b6b0 2024-04-25T10:05:11.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.642+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a0158260 con 0x7fe5a015b6b0 2024-04-25T10:05:11.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a00c5ca0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe59001e460 con 0x7fe5a015b6b0 2024-04-25T10:05:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe595ffb640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe5900855b0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.646+0000 7fe5a54c7640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 0x7fe584069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ae000 tx=0x7fe5980a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.650+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5a00c5ca0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.794+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe5a004efa0 con 0x7fe5a015b6b0 2024-04-25T10:05:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe595ffb640 1 -- 172.21.15.82:0/3104130578 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5a00c5500 con 0x7fe5a015b6b0 2024-04-25T10:05:11.801 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:05:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 msgr2=0x7fe584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 0x7fe584069420 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe5980ae000 tx=0x7fe5980a7070 comp rx=0 tx=0).stop 2024-04-25T10:05:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 msgr2=0x7fe5a0156cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe59000d830 tx=0x7fe59000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 shutdown_connections 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe584066f60 0x7fe584069420 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe5a015f130 0x7fe5a00c3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe5a015b6b0 0x7fe5a0156cd0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 --2- 172.21.15.82:0/3104130578 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe5a00cf590 0x7fe5a00c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 >> 172.21.15.82:0/3104130578 conn(0x7fe5a006ae60 msgr2=0x7fe5a00aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 shutdown_connections 2024-04-25T10:05:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.798+0000 7fe5a772b640 1 -- 172.21.15.82:0/3104130578 wait complete. 2024-04-25T10:05:11.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:05:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 Processor -- start 2024-04-25T10:05:11.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 -- start start 2024-04-25T10:05:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1c0071740 0x7fa1c00acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c00b24a0 0x7fa1c00ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c00ae860 con 0x7fa1c015da10 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c015e210 con 0x7fa1c0071740 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1c508f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015e4f0 con 0x7fa1c00b24a0 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1c0071740 0x7fa1c00acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa1c0071740 0x7fa1c00acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44134/0 (socket says 172.21.15.82:44134) 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 -- 172.21.15.82:0/1273258900 learned_addr learned my addr 172.21.15.82:0/1273258900 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1bed9d640 1 --1- 172.21.15.82:0/1273258900 >> v1:172.21.15.82:6789/0 conn(0x7fa1c015da10 0x7fa1c0071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45730/0 (socket says 172.21.15.82:45730) 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1bf59e640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c00b24a0 0x7fa1c00ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 -- 172.21.15.82:0/1273258900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c00b24a0 msgr2=0x7fa1c00ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c00b24a0 0x7fa1c00ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 -- 172.21.15.82:0/1273258900 >> v1:172.21.15.82:6789/0 conn(0x7fa1c015da10 legacy=0x7fa1c0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.922+0000 7fa1be59c640 1 -- 172.21.15.82:0/1273258900 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00ae4c0 con 0x7fa1c0071740 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c0071740 0x7fa1c00acbc0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009870 tx=0x7fa1b001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be9d770bad777f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1bdd9b640 1 -- 172.21.15.82:0/1273258900 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00b0070 con 0x7fa1c0071740 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1bdd9b640 1 -- 172.21.15.82:0/1273258900 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b00ab600 con 0x7fa1c0071740 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1bdd9b640 1 -- 172.21.15.82:0/1273258900 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00ab9d0 con 0x7fa1c0071740 2024-04-25T10:05:11.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1273258900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c0071740 msgr2=0x7fa1c00acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c0071740 0x7fa1c00acbc0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fa1b0009870 tx=0x7fa1b001c440 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1273258900 shutdown_connections 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c00b24a0 0x7fa1c00ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1273258900 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c0071740 0x7fa1c00acbc0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1273258900 >> 172.21.15.82:0/1273258900 conn(0x7fa1c00b11a0 msgr2=0x7fa1c00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1273258900 shutdown_connections 2024-04-25T10:05:11.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1273258900 wait complete. 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 Processor -- start 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- start start 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c0071740 0x7fa1c00b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c015da10 0x7fa1c0153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c015ef00 con 0x7fa1c00b24a0 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c00adc60 con 0x7fa1c015da10 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c016b0a0 con 0x7fa1c0071740 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44136/0 (socket says 172.21.15.82:44136) 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 -- 172.21.15.82:0/1002561342 learned_addr learned my addr 172.21.15.82:0/1002561342 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 -- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c0071740 msgr2=0x7fa1c00b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c0071740 0x7fa1c00b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 -- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c015da10 msgr2=0x7fa1c0153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1bf59e640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c015da10 0x7fa1c0153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c015da10 0x7fa1c0153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 -- 172.21.15.82:0/1002561342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00aff50 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1be59c640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa1b001aaf0 tx=0x7fa1b00191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00b0040 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1b00b4640 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c00b0230 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1b00b4a30 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.926+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c0159cd0 con 0x7fa1c00b24a0 2024-04-25T10:05:11.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c004efa0 con 0x7fa1c00b24a0 2024-04-25T10:05:11.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa1b00ab4c0 con 0x7fa1c00b24a0 2024-04-25T10:05:11.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1a77fe640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 0x7fa1ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:11.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa1b011eb90 con 0x7fa1c00b24a0 2024-04-25T10:05:11.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1bed9d640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 0x7fa1ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:11.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.930+0000 7fa1bed9d640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 0x7fa1ac069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000f70 tx=0x7fa1b4007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:11.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:11.934+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1b0016150 con 0x7fa1c00b24a0 2024-04-25T10:05:12.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.134+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fa1c00b1dc0 con 0x7fa1c00b24a0 2024-04-25T10:05:12.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1a77fe640 1 -- 172.21.15.82:0/1002561342 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v177) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa1b00efda0 con 0x7fa1c00b24a0 2024-04-25T10:05:12.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 msgr2=0x7fa1ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 0x7fa1ac069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa1b4000f70 tx=0x7fa1b4007a10 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 msgr2=0x7fa1c01539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa1b001aaf0 tx=0x7fa1b00191e0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 shutdown_connections 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa1ac066f60 0x7fa1ac069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1c015da10 0x7fa1c0153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1c00b24a0 0x7fa1c01539b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 --2- 172.21.15.82:0/1002561342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa1c0071740 0x7fa1c00b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 >> 172.21.15.82:0/1002561342 conn(0x7fa1c00b11a0 msgr2=0x7fa1c006d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 shutdown_connections 2024-04-25T10:05:12.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.174+0000 7fa1c508f640 1 -- 172.21.15.82:0/1002561342 wait complete. 2024-04-25T10:05:12.196 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:05:12.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.314+0000 7fe6c45b0640 1 Processor -- start 2024-04-25T10:05:12.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.314+0000 7fe6c45b0640 1 -- start start 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c45b0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15b6b0 0x7fe6bc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c45b0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c45b0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0cbd20 con 0x7fe6bc15f130 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c45b0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc15ba90 con 0x7fe6bc0cf590 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c45b0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cff30 con 0x7fe6bc15b6b0 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c2b4d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe6bc15f130 0x7fe6bc15f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45738/0 (socket says 172.21.15.82:45738) 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44148/0 (socket says 172.21.15.82:44148) 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c2b4d640 1 -- 172.21.15.82:0/2570309070 learned_addr learned my addr 172.21.15.82:0/2570309070 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c234c640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15b6b0 0x7fe6bc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 -- 172.21.15.82:0/2570309070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15b6b0 msgr2=0x7fe6bc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15b6b0 0x7fe6bc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 -- 172.21.15.82:0/2570309070 >> v1:172.21.15.82:6789/0 conn(0x7fe6bc15f130 legacy=0x7fe6bc15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c134a640 1 -- 172.21.15.82:0/2570309070 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195048233 0 0) 0x7fe6bc0cbd20 con 0x7fe6bc15f130 2024-04-25T10:05:12.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 -- 172.21.15.82:0/2570309070 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0cc000 con 0x7fe6bc0cf590 2024-04-25T10:05:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c1b4b640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097c0 tx=0x7fe6ac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56225e827c9c33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c134a640 1 -- 172.21.15.82:0/2570309070 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0a7070 con 0x7fe6bc0cf590 2024-04-25T10:05:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c134a640 1 -- 172.21.15.82:0/2570309070 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0b0070 con 0x7fe6bc0cf590 2024-04-25T10:05:12.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.318+0000 7fe6c134a640 1 -- 172.21.15.82:0/2570309070 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0ab890 con 0x7fe6bc0cf590 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/2570309070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 msgr2=0x7fe6bc0cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0097c0 tx=0x7fe6ac01c470 comp rx=0 tx=0).stop 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/2570309070 shutdown_connections 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc0cf9f0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/2570309070 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15b6b0 0x7fe6bc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/2570309070 >> 172.21.15.82:0/2570309070 conn(0x7fe6bc06ae60 msgr2=0x7fe6bc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/2570309070 shutdown_connections 2024-04-25T10:05:12.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/2570309070 wait complete. 2024-04-25T10:05:12.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 Processor -- start 2024-04-25T10:05:12.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- start start 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc15b6b0 0x7fe6bc158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15f130 0x7fe6bc0980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0ca900 con 0x7fe6bc0cf590 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cba30 con 0x7fe6bc15b6b0 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cadb0 con 0x7fe6bc15f130 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44156/0 (socket says 172.21.15.82:44156) 2024-04-25T10:05:12.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 -- 172.21.15.82:0/3530648622 learned_addr learned my addr 172.21.15.82:0/3530648622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 -- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15f130 msgr2=0x7fe6bc0980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15f130 0x7fe6bc0980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 -- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc15b6b0 msgr2=0x7fe6bc158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c1b4b640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc15b6b0 0x7fe6bc158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc15b6b0 0x7fe6bc158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 -- 172.21.15.82:0/3530648622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0ac380 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c234c640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800d800 tx=0x7fe6b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800b6c0 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8019070 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc0ac660 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b800ba60 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0acc20 con 0x7fe6bc0cf590 2024-04-25T10:05:12.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.322+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc04efa0 con 0x7fe6bc0cf590 2024-04-25T10:05:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.326+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6b800bc00 con 0x7fe6bc0cf590 2024-04-25T10:05:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.326+0000 7fe6b2ffd640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.326+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe6b8085320 con 0x7fe6bc0cf590 2024-04-25T10:05:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.326+0000 7fe6c1b4b640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.326+0000 7fe6c1b4b640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0af000 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.330+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6b80564a0 con 0x7fe6bc0cf590 2024-04-25T10:05:12.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.490+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe6bc158a70 con 0x7fe6bc0cf590 2024-04-25T10:05:12.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6b2ffd640 1 -- 172.21.15.82:0/3530648622 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe6b8055da0 con 0x7fe6bc0cf590 2024-04-25T10:05:12.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 msgr2=0x7fe6a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 0x7fe6a0069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0af000 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-04-25T10:05:12.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 msgr2=0x7fe6bc150780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800d800 tx=0x7fe6b800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 shutdown_connections 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe6a0066f60 0x7fe6a0069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc15f130 0x7fe6bc0980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc15b6b0 0x7fe6bc158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 --2- 172.21.15.82:0/3530648622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc0cf590 0x7fe6bc150780 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 >> 172.21.15.82:0/3530648622 conn(0x7fe6bc06ae60 msgr2=0x7fe6bc0b3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 shutdown_connections 2024-04-25T10:05:12.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.494+0000 7fe6c45b0640 1 -- 172.21.15.82:0/3530648622 wait complete. 2024-04-25T10:05:12.516 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:05:12.516 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.553 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:05:12.615 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:05:12.615 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:05:12.615 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:12.621 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:05:12.621 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.627 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:12.627 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:05:12.627 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:05:12.669 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:05:12.670 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:05:12.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:05:12.675 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:05:12.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.822+0000 7f4de0164640 1 Processor -- start 2024-04-25T10:05:12.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.822+0000 7f4de0164640 1 -- start start 2024-04-25T10:05:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4de0164640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4de0164640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4dd806ec60 0x7f4dd8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4de0164640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dd80ccc70 con 0x7f4dd80ce510 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4de0164640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dd80cc2b0 con 0x7f4dd806ec60 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4de0164640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd806e290 con 0x7f4dd8144ca0 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4dde701640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4dd806ec60 0x7f4dd8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:43968/0 (socket says 172.21.15.82:43968) 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/196151914 learned_addr learned my addr 172.21.15.82:0/196151914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4dddf00640 1 --1- 172.21.15.82:0/196151914 >> v1:172.21.15.196:6789/0 conn(0x7f4dd80ce510 0x7f4dd80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:45166/0 (socket says 172.21.15.82:45166) 2024-04-25T10:05:12.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/196151914 >> v2:172.21.15.196:3300/0 conn(0x7f4dd806ec60 msgr2=0x7f4dd8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 --2- 172.21.15.82:0/196151914 >> v2:172.21.15.196:3300/0 conn(0x7f4dd806ec60 0x7f4dd8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/196151914 >> v1:172.21.15.196:6789/0 conn(0x7f4dd80ce510 legacy=0x7f4dd80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.826+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/196151914 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd806dc40 con 0x7f4dd8144ca0 2024-04-25T10:05:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- 172.21.15.82:0/196151914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009870 tx=0x7f4dc801ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0fda498548f14db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddcefe640 1 -- 172.21.15.82:0/196151914 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80b5070 con 0x7f4dd8144ca0 2024-04-25T10:05:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddcefe640 1 -- 172.21.15.82:0/196151914 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc80a8a50 con 0x7f4dd8144ca0 2024-04-25T10:05:12.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddcefe640 1 -- 172.21.15.82:0/196151914 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80a8e20 con 0x7f4dd8144ca0 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- 172.21.15.82:0/196151914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 msgr2=0x7f4dd806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- 172.21.15.82:0/196151914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4dc8009870 tx=0x7f4dc801ed50 comp rx=0 tx=0).stop 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- 172.21.15.82:0/196151914 shutdown_connections 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- 172.21.15.82:0/196151914 >> v2:172.21.15.196:3300/0 conn(0x7f4dd806ec60 0x7f4dd8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- 172.21.15.82:0/196151914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd8144ca0 0x7f4dd806e720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- 172.21.15.82:0/196151914 >> 172.21.15.82:0/196151914 conn(0x7f4dd80b11a0 msgr2=0x7f4dd80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- 172.21.15.82:0/196151914 shutdown_connections 2024-04-25T10:05:12.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- 172.21.15.82:0/196151914 wait complete. 2024-04-25T10:05:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 Processor -- start 2024-04-25T10:05:12.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- start start 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806ec60 0x7f4dd815b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 0x7f4dd81427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd81451f0 con 0x7f4dd80ce510 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd80cc900 con 0x7f4dd8144ca0 2024-04-25T10:05:12.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4de0164640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd8145370 con 0x7f4dd806ec60 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44162/0 (socket says 172.21.15.82:44162) 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/3577561311 learned_addr learned my addr 172.21.15.82:0/3577561311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4dde701640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 0x7f4dd81427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4dddf00640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806ec60 0x7f4dd815b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806ec60 msgr2=0x7f4dd815b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806ec60 0x7f4dd815b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 msgr2=0x7f4dd81427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 0x7f4dd81427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 -- 172.21.15.82:0/3577561311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80c7cf0 con 0x7f4dd80ce510 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4dde701640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 0x7f4dd81427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.830+0000 7f4ddd6ff640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80b3040 tx=0x7f4dc801baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80c0070 con 0x7f4dd80ce510 2024-04-25T10:05:12.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc80b08c0 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc80b5040 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd80c7fd0 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd80c8590 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dd804efa0 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4dc80b0cb0 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 0x7f4dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4dc8121fe0 con 0x7f4dd80ce510 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dddf00640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 0x7f4dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:12.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.834+0000 7f4dddf00640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 0x7f4dbc069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000f70 tx=0x7f4dd4007cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:12.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:12.838+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4dc80188b0 con 0x7f4dd80ce510 2024-04-25T10:05:13.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.018+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dd80b0cb0 con 0x7f4dd80ce510 2024-04-25T10:05:13.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.018+0000 7f4dce7fc640 1 -- 172.21.15.82:0/3577561311 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f4dc80f3160 con 0x7f4dd80ce510 2024-04-25T10:05:13.023 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:13.023 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4889,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/1550144255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1550144255},{"type":"v1","addr":"172.21.15.196:6833","nonce":1550144255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5236,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2900826283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2900826283},{"type":"v1","addr":"172.21.15.196:6835","nonce":2900826283}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5943,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/2062163522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2062163522},{"type":"v1","addr":"172.21.15.82:6835","nonce":2062163522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6348,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3179039763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3179039763},{"type":"v1","addr":"172.21.15.82:6837","nonce":3179039763}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26}],"filesystems":[]} 2024-04-25T10:05:13.023 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 26 2024-04-25T10:05:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 msgr2=0x7f4dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 0x7f4dbc069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4dd4000f70 tx=0x7f4dd4007cd0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 msgr2=0x7f4dd80c75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80b3040 tx=0x7f4dc801baf0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 shutdown_connections 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4dbc066f60 0x7f4dbc069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4dd8144ca0 0x7f4dd81427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4dd80ce510 0x7f4dd80c75f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 --2- 172.21.15.82:0/3577561311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4dd806ec60 0x7f4dd815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 >> 172.21.15.82:0/3577561311 conn(0x7f4dd80b11a0 msgr2=0x7f4dd80718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 shutdown_connections 2024-04-25T10:05:13.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.022+0000 7f4de0164640 1 -- 172.21.15.82:0/3577561311 wait complete. 2024-04-25T10:05:13.042 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:05:13.042 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:05:13.042 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:05:13.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:13.042 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:05:13.042 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:05:13.043 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:05:13.043 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:13.043 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:05:13.043 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:05:13.043 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:05:13.043 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:13.043 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:05:13.043 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:05:13.044 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:05:13.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:05:13.054 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:05:13.054 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:05:13.056 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:05:13.056 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:05:13.057 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:05:13.057 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:05:13.058 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:05:13.058 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:05:13.097 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:05:13.097 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:13.098 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:05:13.099 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:13.101 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:05:13.102 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:13.103 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:05:13.103 DEBUG:teuthology.parallel:result is None 2024-04-25T10:05:13.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:05:13.160 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:05:13.160 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:05:13.173 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:05:13.174 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:05:13.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 Processor -- start 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 -- start start 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38a015da80 0x7f38a00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38a00d1cd0 0x7f38a00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a00d0620 0x7f38a00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a006c440 con 0x7f38a015da80 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a006be60 con 0x7f38a00d1cd0 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a673b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015de60 con 0x7f38a00d0620 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f389ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38a015da80 0x7f38a00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f389ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f38a015da80 0x7f38a00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44198/0 (socket says 172.21.15.82:44198) 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f389ffff640 1 -- 172.21.15.82:0/3836525667 learned_addr learned my addr 172.21.15.82:0/3836525667 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:13.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f389f7fe640 1 --2- 172.21.15.82:0/3836525667 >> v2:172.21.15.196:3300/0 conn(0x7f38a00d1cd0 0x7f38a00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.210+0000 7f38a4cd8640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a00d0620 0x7f38a00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 -- 172.21.15.82:0/3836525667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a00d0620 msgr2=0x7f38a00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a00d0620 0x7f38a00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 -- 172.21.15.82:0/3836525667 >> v2:172.21.15.196:3300/0 conn(0x7f38a00d1cd0 msgr2=0x7f38a00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 --2- 172.21.15.82:0/3836525667 >> v2:172.21.15.196:3300/0 conn(0x7f38a00d1cd0 0x7f38a00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 -- 172.21.15.82:0/3836525667 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00d2450 con 0x7f38a015da80 2024-04-25T10:05:13.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389ffff640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a015da80 0x7f38a00d4230 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3894009870 tx=0x7f389401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294cc6223995597d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389effd640 1 -- 172.21.15.82:0/3836525667 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940b0070 con 0x7f38a015da80 2024-04-25T10:05:13.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389effd640 1 -- 172.21.15.82:0/3836525667 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38940ab600 con 0x7f38a015da80 2024-04-25T10:05:13.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f389effd640 1 -- 172.21.15.82:0/3836525667 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940ab9d0 con 0x7f38a015da80 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 -- 172.21.15.82:0/3836525667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a015da80 msgr2=0x7f38a00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a015da80 0x7f38a00d4230 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3894009870 tx=0x7f389401c520 comp rx=0 tx=0).stop 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 -- 172.21.15.82:0/3836525667 shutdown_connections 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a00d0620 0x7f38a00ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 --2- 172.21.15.82:0/3836525667 >> v2:172.21.15.196:3300/0 conn(0x7f38a00d1cd0 0x7f38a00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 --2- 172.21.15.82:0/3836525667 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a015da80 0x7f38a00d4230 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 -- 172.21.15.82:0/3836525667 >> 172.21.15.82:0/3836525667 conn(0x7f38a008b000 msgr2=0x7f38a00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 -- 172.21.15.82:0/3836525667 shutdown_connections 2024-04-25T10:05:13.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.214+0000 7f38a673b640 1 -- 172.21.15.82:0/3836525667 wait complete. 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 Processor -- start 2024-04-25T10:05:13.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- start start 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a00d1cd0 0x7f38a01436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a015da80 0x7f38a00b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006ca50 con 0x7f38a00d0620 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006c8d0 con 0x7f38a00d1cd0 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006cbd0 con 0x7f38a015da80 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44204/0 (socket says 172.21.15.82:44204) 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 -- 172.21.15.82:0/3555773133 learned_addr learned my addr 172.21.15.82:0/3555773133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389f7fe640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a00d1cd0 0x7f38a01436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a4cd8640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a015da80 0x7f38a00b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 -- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a015da80 msgr2=0x7f38a00b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a015da80 0x7f38a00b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 -- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a00d1cd0 msgr2=0x7f38a01436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a00d1cd0 0x7f38a01436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 -- 172.21.15.82:0/3555773133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00b8500 con 0x7f38a00d0620 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389ffff640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f389401abd0 tx=0x7f389401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940ba020 con 0x7f38a00d0620 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38940b6640 con 0x7f38a00d0620 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940b0040 con 0x7f38a00d0620 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00b87e0 con 0x7f38a00d0620 2024-04-25T10:05:13.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00b8da0 con 0x7f38a00d0620 2024-04-25T10:05:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a004efa0 con 0x7f38a00d0620 2024-04-25T10:05:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38940bd020 con 0x7f38a00d0620 2024-04-25T10:05:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:13.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.218+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f389411ee90 con 0x7f38a00d0620 2024-04-25T10:05:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.222+0000 7f389f7fe640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:13.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.222+0000 7f389f7fe640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 0x7f388c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f38a01561e0 tx=0x7f3890008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:13.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.226+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38940c4030 con 0x7f38a00d0620 2024-04-25T10:05:13.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:13.386+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f38a0067e40 con 0x7f38a00d0620 2024-04-25T10:05:14.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.186+0000 7f389cff9640 1 -- 172.21.15.82:0/3555773133 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3894016110 con 0x7f38a00d0620 2024-04-25T10:05:14.190 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 msgr2=0x7f388c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 0x7f388c069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f38a01561e0 tx=0x7f3890008040 comp rx=0 tx=0).stop 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 msgr2=0x7f38a0155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f389401abd0 tx=0x7f389401ad60 comp rx=0 tx=0).stop 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 shutdown_connections 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f388c066f60 0x7f388c069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38a015da80 0x7f38a00b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38a00d1cd0 0x7f38a01436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 --2- 172.21.15.82:0/3555773133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38a00d0620 0x7f38a0155ca0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 >> 172.21.15.82:0/3555773133 conn(0x7f38a008b000 msgr2=0x7f38a00b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 shutdown_connections 2024-04-25T10:05:14.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.190+0000 7f38a673b640 1 -- 172.21.15.82:0/3555773133 wait complete. 2024-04-25T10:05:14.215 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:05:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.330+0000 7f64cd0a0640 1 Processor -- start 2024-04-25T10:05:14.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.330+0000 7f64cd0a0640 1 -- start start 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64cd0a0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64c815da80 0x7f64c80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64cd0a0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f64c80d07f0 0x7f64c806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64cd0a0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c806be60 con 0x7f64c80d1cd0 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64cd0a0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c806c440 con 0x7f64c815da80 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64cd0a0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c815de60 con 0x7f64c80d07f0 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c6d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f64c815da80 0x7f64c80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f64c80d1cd0 0x7f64c80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:45760/0 (socket says 172.21.15.82:45760) 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c659c640 1 -- 172.21.15.82:0/2079934232 learned_addr learned my addr 172.21.15.82:0/2079934232 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:14.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c759e640 1 --2- 172.21.15.82:0/2079934232 >> v2:172.21.15.196:3300/0 conn(0x7f64c80d07f0 0x7f64c806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2079934232 >> v2:172.21.15.196:3300/0 conn(0x7f64c80d07f0 msgr2=0x7f64c806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c6d9d640 1 --2- 172.21.15.82:0/2079934232 >> v2:172.21.15.196:3300/0 conn(0x7f64c80d07f0 0x7f64c806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2079934232 >> v1:172.21.15.82:6789/0 conn(0x7f64c80d1cd0 legacy=0x7f64c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c5d9b640 1 -- 172.21.15.82:0/2079934232 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802670368 0 0) 0x7f64c806be60 con 0x7f64c80d1cd0 2024-04-25T10:05:14.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.334+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2079934232 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80d2450 con 0x7f64c815da80 2024-04-25T10:05:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- 172.21.15.82:0/2079934232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c815da80 0x7f64c80d4230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f64b00097c0 tx=0x7f64b001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f58a1384a1ed53a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c5d9b640 1 -- 172.21.15.82:0/2079934232 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b00b1070 con 0x7f64c815da80 2024-04-25T10:05:14.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c5d9b640 1 -- 172.21.15.82:0/2079934232 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b00ac5f0 con 0x7f64c815da80 2024-04-25T10:05:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c5d9b640 1 -- 172.21.15.82:0/2079934232 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b00ac9c0 con 0x7f64c815da80 2024-04-25T10:05:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2079934232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c815da80 msgr2=0x7f64c80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2079934232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c815da80 0x7f64c80d4230 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f64b00097c0 tx=0x7f64b001c430 comp rx=0 tx=0).stop 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2079934232 shutdown_connections 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2079934232 >> v2:172.21.15.196:3300/0 conn(0x7f64c80d07f0 0x7f64c806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2079934232 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c815da80 0x7f64c80d4230 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2079934232 >> 172.21.15.82:0/2079934232 conn(0x7f64c808b000 msgr2=0x7f64c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2079934232 shutdown_connections 2024-04-25T10:05:14.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2079934232 wait complete. 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 Processor -- start 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- start start 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64c80d1cd0 0x7f64c80b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64c815da80 0x7f64c80c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c806d7a0 con 0x7f64c80d07f0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c806d620 con 0x7f64c80d1cd0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c806d920 con 0x7f64c815da80 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44226/0 (socket says 172.21.15.82:44226) 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2535660996 learned_addr learned my addr 172.21.15.82:0/2535660996 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64c815da80 msgr2=0x7f64c80c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64c815da80 0x7f64c80c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64c80d1cd0 msgr2=0x7f64c80b8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64c80d1cd0 0x7f64c80b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 -- 172.21.15.82:0/2535660996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c80c76c0 con 0x7f64c80d07f0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64c6d9d640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64b00af040 tx=0x7f64b00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b00b1040 con 0x7f64c80d07f0 2024-04-25T10:05:14.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b00bc070 con 0x7f64c80d07f0 2024-04-25T10:05:14.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c80c79a0 con 0x7f64c80d07f0 2024-04-25T10:05:14.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b0006e20 con 0x7f64c80d07f0 2024-04-25T10:05:14.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.338+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c80c7f60 con 0x7f64c80d07f0 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c804efa0 con 0x7f64c80d07f0 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64b00b5650 con 0x7f64c80d07f0 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64bf7fe640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 0x7f64a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f64b011ebe0 con 0x7f64c80d07f0 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64c659c640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 0x7f64a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.342+0000 7f64c659c640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 0x7f64a8069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64c80cf930 tx=0x7f64b8007880 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.346+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64b00a9070 con 0x7f64c80d07f0 2024-04-25T10:05:14.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.550+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f64c81570f0 con 0x7f64c80d07f0 2024-04-25T10:05:14.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.550+0000 7f64bf7fe640 1 -- 172.21.15.82:0/2535660996 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f64b0016050 con 0x7f64c80d07f0 2024-04-25T10:05:14.554 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.555 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:05:14.556 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:05:14.557 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.558 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.559 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.560 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:05:14.561 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.562 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.563 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:05:14.564 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:05:14.565 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:05:14.566 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.567 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:05:14.568 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:05:14.569 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 msgr2=0x7f64a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 0x7f64a8069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f64c80cf930 tx=0x7f64b8007880 comp rx=0 tx=0).stop 2024-04-25T10:05:14.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 msgr2=0x7f64c8156bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f64b00af040 tx=0x7f64b00191d0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 shutdown_connections 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f64a8066f60 0x7f64a8069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f64c815da80 0x7f64c80c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f64c80d1cd0 0x7f64c80b8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 --2- 172.21.15.82:0/2535660996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f64c80d07f0 0x7f64c8156bb0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 >> 172.21.15.82:0/2535660996 conn(0x7f64c808b000 msgr2=0x7f64c80b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 shutdown_connections 2024-04-25T10:05:14.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.554+0000 7f64cd0a0640 1 -- 172.21.15.82:0/2535660996 wait complete. 2024-04-25T10:05:14.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:14.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.682+0000 7fcbe8306640 1 Processor -- start 2024-04-25T10:05:14.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.682+0000 7fcbe8306640 1 -- start start 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe8306640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbe015a700 0x7fcbe015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe8306640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00b2090 con 0x7fcbe00b3f50 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe8306640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe00b43e0 con 0x7fcbe00b24a0 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe8306640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe015f200 con 0x7fcbe015a700 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcbe00b3f50 0x7fcbe00cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48780/0 (socket says 172.21.15.82:48780) 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 -- 172.21.15.82:0/244871308 learned_addr learned my addr 172.21.15.82:0/244871308 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:14.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe60a2640 1 --2- 172.21.15.82:0/244871308 >> v2:172.21.15.196:3300/0 conn(0x7fcbe015a700 0x7fcbe015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052348600 0 0) 0x7fcbe00b2090 con 0x7fcbe00b3f50 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbcc003530 con 0x7fcbe00b3f50 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1500195960 0 0) 0x7fcbcc003530 con 0x7fcbe00b3f50 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 >> v2:172.21.15.196:3300/0 conn(0x7fcbe015a700 msgr2=0x7fcbe015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 --2- 172.21.15.82:0/244871308 >> v2:172.21.15.196:3300/0 conn(0x7fcbe015a700 0x7fcbe015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 >> v1:172.21.15.196:6789/0 conn(0x7fcbe00b24a0 legacy=0x7fcbe006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe00b46c0 con 0x7fcbe00b3f50 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcbdc0016a0 con 0x7fcbe00b3f50 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 >> v1:172.21.15.82:6789/0 conn(0x7fcbe00b3f50 legacy=0x7fcbe00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc003a30 0x7fcbcc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbcc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc0052d0 0x7fcbcc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe015f200 con 0x7fcbcc004810 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc00c210 con 0x7fcbcc0052d0 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc00c410 con 0x7fcbcc003a30 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbcc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe60a2640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc003a30 0x7fcbcc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 -- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc003a30 msgr2=0x7fcbcc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbcc003a30 0x7fcbcc003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 -- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc0052d0 msgr2=0x7fcbcc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbcc0052d0 0x7fcbcc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe58a1640 1 -- 172.21.15.82:0/244871308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc003530 con 0x7fcbcc004810 2024-04-25T10:05:14.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.686+0000 7fcbe68a3640 1 -- 172.21.15.82:0/244871308 reap_dead start 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbcc004c90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c210 tx=0x7fcbdc01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62329da4494eef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc007d20 con 0x7fcbcc004810 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc0a9020 con 0x7fcbcc004810 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe50a0640 1 -- 172.21.15.82:0/244871308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc005b20 con 0x7fcbcc004810 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 msgr2=0x7fcbcc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbcc004c90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c210 tx=0x7fcbdc01cc10 comp rx=0 tx=0).stop 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/244871308 shutdown_connections 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 --2- 172.21.15.82:0/244871308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbcc004c90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/244871308 >> 172.21.15.82:0/244871308 conn(0x7fcbe00b11a0 msgr2=0x7fcbe00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/244871308 shutdown_connections 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/244871308 wait complete. 2024-04-25T10:05:14.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 Processor -- start 2024-04-25T10:05:14.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- start start 2024-04-25T10:05:14.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 0x7fcbe01fa850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01fad90 0x7fcbe01fb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe00b2c90 con 0x7fcbcc004810 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe00ce700 con 0x7fcbe0168bf0 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe00cd310 con 0x7fcbe01fad90 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44122/0 (socket says 172.21.15.82:44122) 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 -- 172.21.15.82:0/3816532244 learned_addr learned my addr 172.21.15.82:0/3816532244 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 0x7fcbe01fa850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 -- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01fad90 msgr2=0x7fcbe01fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01fad90 0x7fcbe01fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 -- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 msgr2=0x7fcbe01fa850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 0x7fcbe01fa850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 -- 172.21.15.82:0/3816532244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe00ad6d0 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 0x7fcbe01fa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe60a2640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40107f0 tx=0x7fcbd4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd400f040 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd400eb60 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe00ad9b0 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.690+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd4010ea0 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe00adf70 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe004efa0 con 0x7fcbcc004810 2024-04-25T10:05:14.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbd401c400 con 0x7fcbcc004810 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbd2ffd640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 0x7fcbc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcbd4085400 con 0x7fcbcc004810 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 0x7fcbc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:14.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.694+0000 7fcbe58a1640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 0x7fcbc4069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0aa000 tx=0x7fcbe00b45b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:14.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.698+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbd4018030 con 0x7fcbcc004810 2024-04-25T10:05:14.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.858+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbe006f260 con 0x7fcbcc004810 2024-04-25T10:05:14.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.858+0000 7fcbd2ffd640 1 -- 172.21.15.82:0/3816532244 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13176 (secure 0 0 0) 0x7fcbd40565d0 con 0x7fcbcc004810 2024-04-25T10:05:14.863 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:14.863 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":61,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:14.193694+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:14.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 msgr2=0x7fcbc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 0x7fcbc4069420 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc0aa000 tx=0x7fcbe00b45b0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 msgr2=0x7fcbe01692a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:14.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcbd40107f0 tx=0x7fcbd4010cb0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 shutdown_connections 2024-04-25T10:05:14.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbc4066f60 0x7fcbc4069420 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbe01fad90 0x7fcbe01fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbe0168bf0 0x7fcbe01fa850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 --2- 172.21.15.82:0/3816532244 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbcc004810 0x7fcbe01692a0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:14.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 >> 172.21.15.82:0/3816532244 conn(0x7fcbe00b11a0 msgr2=0x7fcbe015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:14.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 shutdown_connections 2024-04-25T10:05:14.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:14.862+0000 7fcbe8306640 1 -- 172.21.15.82:0/3816532244 wait complete. 2024-04-25T10:05:14.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:05:14.883 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:05:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.002+0000 7ff89bc4c640 1 Processor -- start 2024-04-25T10:05:15.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 -- start start 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff89415b6b0 0x7ff8940ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff8940cf590 0x7ff8940cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8940cbd20 con 0x7ff89415f130 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89415ba90 con 0x7ff8940cf590 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89bc4c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8940cc000 con 0x7ff89415b6b0 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8991e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff8940cf590 0x7ff8940cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89a1e9640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff89415f130 0x7ff89415f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48788/0 (socket says 172.21.15.82:48788) 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff89a1e9640 1 -- 172.21.15.82:0/2311219109 learned_addr learned my addr 172.21.15.82:0/2311219109 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8999e8640 1 --2- 172.21.15.82:0/2311219109 >> v2:172.21.15.196:3300/0 conn(0x7ff89415b6b0 0x7ff8940ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8991e7640 1 -- 172.21.15.82:0/2311219109 >> v2:172.21.15.196:3300/0 conn(0x7ff89415b6b0 msgr2=0x7ff8940ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8991e7640 1 --2- 172.21.15.82:0/2311219109 >> v2:172.21.15.196:3300/0 conn(0x7ff89415b6b0 0x7ff8940ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8991e7640 1 -- 172.21.15.82:0/2311219109 >> v1:172.21.15.82:6789/0 conn(0x7ff89415f130 legacy=0x7ff89415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8991e7640 1 -- 172.21.15.82:0/2311219109 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8940cc2e0 con 0x7ff8940cf590 2024-04-25T10:05:15.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.006+0000 7ff8989e6640 1 -- 172.21.15.82:0/2311219109 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805578982 0 0) 0x7ff8940cbd20 con 0x7ff89415f130 2024-04-25T10:05:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8991e7640 1 --2- 172.21.15.82:0/2311219109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8940cf590 0x7ff8940cfa20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff8940d0180 tx=0x7ff88401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11abf00a17bde9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8989e6640 1 -- 172.21.15.82:0/2311219109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840b0070 con 0x7ff8940cf590 2024-04-25T10:05:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8989e6640 1 -- 172.21.15.82:0/2311219109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8840ab600 con 0x7ff8940cf590 2024-04-25T10:05:15.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8989e6640 1 -- 172.21.15.82:0/2311219109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8840ab9d0 con 0x7ff8940cf590 2024-04-25T10:05:15.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2311219109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8940cf590 msgr2=0x7ff8940cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2311219109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8940cf590 0x7ff8940cfa20 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7ff8940d0180 tx=0x7ff88401c480 comp rx=0 tx=0).stop 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2311219109 shutdown_connections 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2311219109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8940cf590 0x7ff8940cfa20 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2311219109 >> v2:172.21.15.196:3300/0 conn(0x7ff89415b6b0 0x7ff8940ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2311219109 >> 172.21.15.82:0/2311219109 conn(0x7ff89406ae60 msgr2=0x7ff89406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2311219109 shutdown_connections 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2311219109 wait complete. 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 Processor -- start 2024-04-25T10:05:15.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- start start 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8940cf590 0x7ff894098330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 0x7ff8940ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940cadb0 con 0x7ff89415f130 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940ca900 con 0x7ff89415b6b0 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89bc4c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8940caf30 con 0x7ff8940cf590 2024-04-25T10:05:15.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44150/0 (socket says 172.21.15.82:44150) 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 -- 172.21.15.82:0/2204981557 learned_addr learned my addr 172.21.15.82:0/2204981557 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8991e7640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 0x7ff8940ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8999e8640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8940cf590 0x7ff894098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 -- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8940cf590 msgr2=0x7ff894098330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8940cf590 0x7ff894098330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 -- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 msgr2=0x7ff8940ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 0x7ff8940ade20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff89a1e9640 1 -- 172.21.15.82:0/2204981557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894158ac0 con 0x7ff89415f130 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.010+0000 7ff8991e7640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 0x7ff8940ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff89a1e9640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c3f0 tx=0x7ff89000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8900074e0 con 0x7ff89415f130 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8900047f0 con 0x7ff89415f130 2024-04-25T10:05:15.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff894152550 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890007680 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8941554d0 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89404efa0 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff890011070 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 0x7ff878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff8900849b0 con 0x7ff89415f130 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff8999e8640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 0x7ff878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.014+0000 7ff8999e8640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 0x7ff878069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff89415a280 tx=0x7ff88c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.022+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff890055c10 con 0x7ff89415f130 2024-04-25T10:05:15.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.166+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff8940cb400 con 0x7ff89415f130 2024-04-25T10:05:15.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff88a7fc640 1 -- 172.21.15.82:0/2204981557 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff890055510 con 0x7ff89415f130 2024-04-25T10:05:15.173 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:05:15.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 msgr2=0x7ff878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 0x7ff878069420 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff89415a280 tx=0x7ff88c00b040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 msgr2=0x7ff89415a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7ff89000c3f0 tx=0x7ff89000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 shutdown_connections 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff878066f60 0x7ff878069420 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff89415f130 0x7ff89415a490 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff89415b6b0 0x7ff8940ade20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 --2- 172.21.15.82:0/2204981557 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8940cf590 0x7ff894098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 >> 172.21.15.82:0/2204981557 conn(0x7ff89406ae60 msgr2=0x7ff894165ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 shutdown_connections 2024-04-25T10:05:15.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.170+0000 7ff89bc4c640 1 -- 172.21.15.82:0/2204981557 wait complete. 2024-04-25T10:05:15.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:05:15.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 Processor -- start 2024-04-25T10:05:15.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 -- start start 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ea015da80 0x7f6ea00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ea00d1cd0 0x7f6ea00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea006be60 con 0x7f6ea00d0620 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea00d0a70 con 0x7f6ea015da80 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea68ab640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea006c440 con 0x7f6ea00d1cd0 2024-04-25T10:05:15.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ea015da80 0x7f6ea00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6ea015da80 0x7f6ea00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44152/0 (socket says 172.21.15.82:44152) 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6ea4e48640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6ea00d0620 0x7f6ea00ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48798/0 (socket says 172.21.15.82:48798) 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 -- 172.21.15.82:0/1098172730 learned_addr learned my addr 172.21.15.82:0/1098172730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e97fff640 1 --2- 172.21.15.82:0/1098172730 >> v2:172.21.15.196:3300/0 conn(0x7f6ea00d1cd0 0x7f6ea00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 -- 172.21.15.82:0/1098172730 >> v2:172.21.15.196:3300/0 conn(0x7f6ea00d1cd0 msgr2=0x7f6ea00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 --2- 172.21.15.82:0/1098172730 >> v2:172.21.15.196:3300/0 conn(0x7f6ea00d1cd0 0x7f6ea00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 -- 172.21.15.82:0/1098172730 >> v1:172.21.15.82:6789/0 conn(0x7f6ea00d0620 legacy=0x7f6ea00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9f7fe640 1 -- 172.21.15.82:0/1098172730 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510918259 0 0) 0x7f6ea006be60 con 0x7f6ea00d0620 2024-04-25T10:05:15.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.310+0000 7f6e9ffff640 1 -- 172.21.15.82:0/1098172730 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea015de60 con 0x7f6ea015da80 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9ffff640 1 --2- 172.21.15.82:0/1098172730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea015da80 0x7f6ea00d4230 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009870 tx=0x7f6e9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcfc8e1b1dd80a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9f7fe640 1 -- 172.21.15.82:0/1098172730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900b0070 con 0x7f6ea015da80 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9f7fe640 1 -- 172.21.15.82:0/1098172730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e900ab600 con 0x7f6ea015da80 2024-04-25T10:05:15.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9f7fe640 1 -- 172.21.15.82:0/1098172730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900ab9d0 con 0x7f6ea015da80 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- 172.21.15.82:0/1098172730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea015da80 msgr2=0x7f6ea00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/1098172730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea015da80 0x7f6ea00d4230 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009870 tx=0x7f6e9001c520 comp rx=0 tx=0).stop 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- 172.21.15.82:0/1098172730 shutdown_connections 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/1098172730 >> v2:172.21.15.196:3300/0 conn(0x7f6ea00d1cd0 0x7f6ea00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/1098172730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea015da80 0x7f6ea00d4230 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- 172.21.15.82:0/1098172730 >> 172.21.15.82:0/1098172730 conn(0x7f6ea008b000 msgr2=0x7f6ea00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- 172.21.15.82:0/1098172730 shutdown_connections 2024-04-25T10:05:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- 172.21.15.82:0/1098172730 wait complete. 2024-04-25T10:05:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 Processor -- start 2024-04-25T10:05:15.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- start start 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea00d1cd0 0x7f6ea0156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ea015da80 0x7f6ea015ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea016a730 con 0x7f6ea00d0620 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea016a5b0 con 0x7f6ea00d1cd0 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6ea68ab640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea016a8b0 con 0x7f6ea015da80 2024-04-25T10:05:15.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44164/0 (socket says 172.21.15.82:44164) 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.314+0000 7f6e9ffff640 1 -- 172.21.15.82:0/2026832221 learned_addr learned my addr 172.21.15.82:0/2026832221 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 -- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ea015da80 msgr2=0x7f6ea015ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ea015da80 0x7f6ea015ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 -- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea00d1cd0 msgr2=0x7f6ea0156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea00d1cd0 0x7f6ea0156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 -- 172.21.15.82:0/2026832221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea00b42d0 con 0x7f6ea00d0620 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9ffff640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6e90019130 tx=0x7f6e9001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900ba020 con 0x7f6ea00d0620 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e900b6620 con 0x7f6ea00d0620 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e900b0040 con 0x7f6ea00d0620 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea00c7720 con 0x7f6ea00d0620 2024-04-25T10:05:15.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea015f240 con 0x7f6ea00d0620 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea004efa0 con 0x7f6ea00d0620 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e900bd020 con 0x7f6ea00d0620 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 0x7f6e80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6e9011eff0 con 0x7f6ea00d0620 2024-04-25T10:05:15.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e97fff640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 0x7f6e80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.318+0000 7f6e97fff640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 0x7f6e80069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.322+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e900c4030 con 0x7f6ea00d0620 2024-04-25T10:05:15.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6ea00b02c0 con 0x7f6ea00d0620 2024-04-25T10:05:15.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6e9d7fa640 1 -- 172.21.15.82:0/2026832221 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6ea00c7720 con 0x7f6ea00d0620 2024-04-25T10:05:15.486 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:15.486 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:05:15.486 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:15.487 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:05:15.487 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:05:15.487 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:15.487 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:05:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 msgr2=0x7f6e80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 0x7f6e80069420 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e88006fd0 tx=0x7f6e88008040 comp rx=0 tx=0).stop 2024-04-25T10:05:15.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 msgr2=0x7f6ea00c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f6e90019130 tx=0x7f6e9001ad60 comp rx=0 tx=0).stop 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 shutdown_connections 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f60 0x7f6e80069420 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ea015da80 0x7f6ea015ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ea00d1cd0 0x7f6ea0156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.482+0000 7f6ea68ab640 1 --2- 172.21.15.82:0/2026832221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ea00d0620 0x7f6ea00c71e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.486+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 >> 172.21.15.82:0/2026832221 conn(0x7f6ea008b000 msgr2=0x7f6ea0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.486+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 shutdown_connections 2024-04-25T10:05:15.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.486+0000 7f6ea68ab640 1 -- 172.21.15.82:0/2026832221 wait complete. 2024-04-25T10:05:15.505 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:05:15.505 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:05:15.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 Processor -- start 2024-04-25T10:05:15.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 -- start start 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415a3c0 0x7f8ac40ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ac415a7a0 0x7f8ac40b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 0x7f8ac40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac415e0f0 con 0x7f8ac415a7a0 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40b24a0 con 0x7f8ac415ac00 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8acb00a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac415b0b0 con 0x7f8ac415a3c0 2024-04-25T10:05:15.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ac415a7a0 0x7f8ac40b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ac415a7a0 0x7f8ac40b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44174/0 (socket says 172.21.15.82:44174) 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 -- 172.21.15.82:0/357011819 learned_addr learned my addr 172.21.15.82:0/357011819 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8ac95a7640 1 --2- 172.21.15.82:0/357011819 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 0x7f8ac40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8ac8da6640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415a3c0 0x7f8ac40ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 -- 172.21.15.82:0/357011819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415a3c0 msgr2=0x7f8ac40ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415a3c0 0x7f8ac40ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 -- 172.21.15.82:0/357011819 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 msgr2=0x7f8ac40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 --2- 172.21.15.82:0/357011819 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 0x7f8ac40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8abbfff640 1 -- 172.21.15.82:0/357011819 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac415b390 con 0x7f8ac415a7a0 2024-04-25T10:05:15.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.626+0000 7f8ac95a7640 1 --2- 172.21.15.82:0/357011819 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 0x7f8ac40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8abbfff640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac40b27a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ac406e680 tx=0x7f8ab401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9e9713a32a598c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8abb7fe640 1 -- 172.21.15.82:0/357011819 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40b1070 con 0x7f8ac415a7a0 2024-04-25T10:05:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8abb7fe640 1 -- 172.21.15.82:0/357011819 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab40ac620 con 0x7f8ac415a7a0 2024-04-25T10:05:15.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8abb7fe640 1 -- 172.21.15.82:0/357011819 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40ac9f0 con 0x7f8ac415a7a0 2024-04-25T10:05:15.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- 172.21.15.82:0/357011819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 msgr2=0x7f8ac40b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac40b27a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8ac406e680 tx=0x7f8ab401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- 172.21.15.82:0/357011819 shutdown_connections 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 --2- 172.21.15.82:0/357011819 >> v2:172.21.15.196:3300/0 conn(0x7f8ac415ac00 0x7f8ac40cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac40b27a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 --2- 172.21.15.82:0/357011819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415a3c0 0x7f8ac40ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- 172.21.15.82:0/357011819 >> 172.21.15.82:0/357011819 conn(0x7f8ac406ae60 msgr2=0x7f8ac406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- 172.21.15.82:0/357011819 shutdown_connections 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- 172.21.15.82:0/357011819 wait complete. 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 Processor -- start 2024-04-25T10:05:15.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.630+0000 7f8acb00a640 1 -- start start 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 0x7f8ac40aca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415ac00 0x7f8ac4151f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac406cee0 con 0x7f8ac415a7a0 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40b2ef0 con 0x7f8ac415a3c0 2024-04-25T10:05:15.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac406d060 con 0x7f8ac415ac00 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44178/0 (socket says 172.21.15.82:44178) 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 -- 172.21.15.82:0/25166884 learned_addr learned my addr 172.21.15.82:0/25166884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ac95a7640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415ac00 0x7f8ac4151f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ac8da6640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 0x7f8ac40aca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 -- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415ac00 msgr2=0x7f8ac4151f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415ac00 0x7f8ac4151f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 -- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 msgr2=0x7f8ac40aca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 0x7f8ac40aca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 -- 172.21.15.82:0/25166884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40c3830 con 0x7f8ac415a7a0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ac8da6640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 0x7f8ac40aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8abbfff640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8ab401aba0 tx=0x7f8ab4019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40b1040 con 0x7f8ac415a7a0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab40ba070 con 0x7f8ac415a7a0 2024-04-25T10:05:15.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac40c3b10 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40b5660 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac40c40d0 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac4067c30 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ab40b5800 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 0x7f8ab0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8ab411eb60 con 0x7f8ac415a7a0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ac8da6640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 0x7f8ab0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:15.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.634+0000 7f8ac8da6640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 0x7f8ab0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000b9b0 tx=0x7f8ac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:15.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.638+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ab4016110 con 0x7f8ac415a7a0 2024-04-25T10:05:15.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:15.798+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8ac406edf0 con 0x7f8ac415a7a0 2024-04-25T10:05:16.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.194+0000 7f8ab97fa640 1 -- 172.21.15.82:0/25166884 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8ab40efdc0 con 0x7f8ac415a7a0 2024-04-25T10:05:16.200 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:05:16.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 msgr2=0x7f8ab0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 0x7f8ab0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000b9b0 tx=0x7f8ac0008040 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 msgr2=0x7f8ac415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8ab401aba0 tx=0x7f8ab4019290 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 shutdown_connections 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ab0066f90 0x7f8ab0069450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ac415ac00 0x7f8ac4151f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ac415a7a0 0x7f8ac415bff0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 --2- 172.21.15.82:0/25166884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ac415a3c0 0x7f8ac40aca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 >> 172.21.15.82:0/25166884 conn(0x7f8ac406ae60 msgr2=0x7f8ac40c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 shutdown_connections 2024-04-25T10:05:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.198+0000 7f8acb00a640 1 -- 172.21.15.82:0/25166884 wait complete. 2024-04-25T10:05:16.230 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:05:16.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.354+0000 7fa35b9c3640 1 Processor -- start 2024-04-25T10:05:16.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.354+0000 7fa35b9c3640 1 -- start start 2024-04-25T10:05:16.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35b9c3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa35415e6c0 0x7fa354168700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35b9c3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540cbf10 con 0x7fa3540cfca0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35b9c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3540cc1f0 con 0x7fa3540b3cf0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35b9c3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3540cc4d0 con 0x7fa35415e6c0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa358f5e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa3540cfca0 0x7fa35415dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48800/0 (socket says 172.21.15.82:48800) 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa358f5e640 1 -- 172.21.15.82:0/2398190958 learned_addr learned my addr 172.21.15.82:0/2398190958 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 --2- 172.21.15.82:0/2398190958 >> v2:172.21.15.196:3300/0 conn(0x7fa35415e6c0 0x7fa354168700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 940994516 0 0) 0x7fa3540cbf10 con 0x7fa3540cfca0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa340003530 con 0x7fa3540cfca0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1569955522 0 0) 0x7fa340003530 con 0x7fa3540cfca0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 >> v2:172.21.15.196:3300/0 conn(0x7fa35415e6c0 msgr2=0x7fa354168700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 --2- 172.21.15.82:0/2398190958 >> v2:172.21.15.196:3300/0 conn(0x7fa35415e6c0 0x7fa354168700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 >> v1:172.21.15.196:6789/0 conn(0x7fa3540b3cf0 legacy=0x7fa3540cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540cc7b0 con 0x7fa3540cfca0 2024-04-25T10:05:16.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa350001680 con 0x7fa3540cfca0 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 >> v1:172.21.15.82:6789/0 conn(0x7fa3540cfca0 legacy=0x7fa35415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa340003a30 0x7fa340003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa340004810 0x7fa340004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 0x7fa34000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540cc4d0 con 0x7fa3400051d0 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa34000c100 con 0x7fa340003a30 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa34000c300 con 0x7fa340004810 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 0x7fa34000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35975f640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa340003a30 0x7fa340003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa358f5e640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa340004810 0x7fa340004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 -- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa340004810 msgr2=0x7fa340004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa340004810 0x7fa340004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 -- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa340003a30 msgr2=0x7fa340003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:16.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa340003a30 0x7fa340003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 -- 172.21.15.82:0/2398190958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa340003530 con 0x7fa3400051d0 2024-04-25T10:05:16.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa35975f640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa340003a30 0x7fa340003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:16.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa358f5e640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa340004810 0x7fa340004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:16.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 -- 172.21.15.82:0/2398190958 reap_dead start 2024-04-25T10:05:16.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.358+0000 7fa359f60640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 0x7fa34000b9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009fc0 tx=0x7fa34c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b66bde59ee10b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009c10 con 0x7fa3400051d0 2024-04-25T10:05:16.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c0a5cd0 con 0x7fa3400051d0 2024-04-25T10:05:16.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa34bfff640 1 -- 172.21.15.82:0/2398190958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003be0 con 0x7fa3400051d0 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 msgr2=0x7fa34000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 0x7fa34000b9d0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009fc0 tx=0x7fa34c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/2398190958 shutdown_connections 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/2398190958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3400051d0 0x7fa34000b9d0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/2398190958 >> 172.21.15.82:0/2398190958 conn(0x7fa35406ae60 msgr2=0x7fa35406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/2398190958 shutdown_connections 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/2398190958 wait complete. 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 Processor -- start 2024-04-25T10:05:16.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- start start 2024-04-25T10:05:16.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3400051d0 0x7fa3540c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3540c9930 0x7fa3540c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b4250 con 0x7fa3540c94b0 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3540b40d0 con 0x7fa3400051d0 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35415eb30 con 0x7fa3540c9930 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44200/0 (socket says 172.21.15.82:44200) 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 -- 172.21.15.82:0/414779720 learned_addr learned my addr 172.21.15.82:0/414779720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35975f640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3400051d0 0x7fa3540c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa359f60640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3540c9930 0x7fa3540c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 -- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3540c9930 msgr2=0x7fa3540c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3540c9930 0x7fa3540c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 -- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3400051d0 msgr2=0x7fa3540c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3400051d0 0x7fa3540c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:16.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 -- 172.21.15.82:0/414779720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540c8410 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa358f5e640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa35000d780 tx=0x7fa35000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350019070 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3500074e0 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3540c86f0 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3500049f0 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.362+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3540c8cb0 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35404efa0 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa35001c070 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa349ffb640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 0x7fa338069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fa350084e30 con 0x7fa3540c94b0 2024-04-25T10:05:16.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa35975f640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 0x7fa338069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:16.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.366+0000 7fa35975f640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 0x7fa338069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c10 tx=0x7fa344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:16.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.370+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa350023020 con 0x7fa3540c94b0 2024-04-25T10:05:16.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:16.514+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa3540b3060 con 0x7fa3540c94b0 2024-04-25T10:05:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.210+0000 7fa349ffb640 1 -- 172.21.15.82:0/414779720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa350055f00 con 0x7fa3540c94b0 2024-04-25T10:05:17.214 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 msgr2=0x7fa338069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 0x7fa338069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa344000c10 tx=0x7fa344009040 comp rx=0 tx=0).stop 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 msgr2=0x7fa3540c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa35000d780 tx=0x7fa35000dc40 comp rx=0 tx=0).stop 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 shutdown_connections 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa338066f60 0x7fa338069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3540c9930 0x7fa3540c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3540c94b0 0x7fa3540c7870 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 --2- 172.21.15.82:0/414779720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3400051d0 0x7fa3540c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 >> 172.21.15.82:0/414779720 conn(0x7fa35406ae60 msgr2=0x7fa354073740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 shutdown_connections 2024-04-25T10:05:17.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.214+0000 7fa35b9c3640 1 -- 172.21.15.82:0/414779720 wait complete. 2024-04-25T10:05:17.249 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:05:17.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.370+0000 7fb36af4b640 1 Processor -- start 2024-04-25T10:05:17.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.370+0000 7fb36af4b640 1 -- start start 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb36af4b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb3640cf590 0x7fb3640cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb36af4b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640cbd20 con 0x7fb36415b6b0 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb36af4b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640cc000 con 0x7fb364072000 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb36af4b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36415ba90 con 0x7fb3640cf590 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb368ce7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb36415b6b0 0x7fb3640ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48812/0 (socket says 172.21.15.82:48812) 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb368ce7640 1 -- 172.21.15.82:0/3045415127 learned_addr learned my addr 172.21.15.82:0/3045415127 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb363fff640 1 --2- 172.21.15.82:0/3045415127 >> v2:172.21.15.196:3300/0 conn(0x7fb3640cf590 0x7fb3640cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585904056 0 0) 0x7fb3640cbd20 con 0x7fb36415b6b0 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb34c003530 con 0x7fb36415b6b0 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 805149227 0 0) 0x7fb34c003530 con 0x7fb36415b6b0 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 >> v2:172.21.15.196:3300/0 conn(0x7fb3640cf590 msgr2=0x7fb3640cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 --2- 172.21.15.82:0/3045415127 >> v2:172.21.15.196:3300/0 conn(0x7fb3640cf590 0x7fb3640cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 >> v1:172.21.15.196:6789/0 conn(0x7fb364072000 legacy=0x7fb364072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640cc2e0 con 0x7fb36415b6b0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb35c001680 con 0x7fb36415b6b0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 >> v1:172.21.15.82:6789/0 conn(0x7fb36415b6b0 legacy=0x7fb3640ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c003a30 0x7fb34c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb34c004810 0x7fb34c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 0x7fb34c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415ba90 con 0x7fb34c0051d0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c00c100 con 0x7fb34c004810 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c00c300 con 0x7fb34c003a30 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 0x7fb34c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb368ce7640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c003a30 0x7fb34c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb363fff640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb34c004810 0x7fb34c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 -- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c003a30 msgr2=0x7fb34c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c003a30 0x7fb34c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 -- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb34c004810 msgr2=0x7fb34c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb34c004810 0x7fb34c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.374+0000 7fb3694e8640 1 -- 172.21.15.82:0/3045415127 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c003530 con 0x7fb34c0051d0 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3694e8640 1 -- 172.21.15.82:0/3045415127 reap_dead start 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3694e8640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 0x7fb34c00b9d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb354001780 tx=0x7fb35401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71f99c63ad621f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3540013a0 con 0x7fb34c0051d0 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3540a9070 con 0x7fb34c0051d0 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3637fe640 1 -- 172.21.15.82:0/3045415127 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb354003600 con 0x7fb34c0051d0 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 msgr2=0x7fb34c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 0x7fb34c00b9d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb354001780 tx=0x7fb35401c420 comp rx=0 tx=0).stop 2024-04-25T10:05:17.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- 172.21.15.82:0/3045415127 shutdown_connections 2024-04-25T10:05:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 --2- 172.21.15.82:0/3045415127 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb34c0051d0 0x7fb34c00b9d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- 172.21.15.82:0/3045415127 >> 172.21.15.82:0/3045415127 conn(0x7fb36406ae60 msgr2=0x7fb36406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- 172.21.15.82:0/3045415127 shutdown_connections 2024-04-25T10:05:17.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- 172.21.15.82:0/3045415127 wait complete. 2024-04-25T10:05:17.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 Processor -- start 2024-04-25T10:05:17.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- start start 2024-04-25T10:05:17.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c0051d0 0x7fb3641f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 0x7fb364154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640cba30 con 0x7fb364158cb0 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640cc8a0 con 0x7fb3641f4e70 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb36af4b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640cadb0 con 0x7fb34c0051d0 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44216/0 (socket says 172.21.15.82:44216) 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 -- 172.21.15.82:0/4172140883 learned_addr learned my addr 172.21.15.82:0/4172140883 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb368ce7640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c0051d0 0x7fb3641f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3694e8640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 0x7fb364154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 -- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c0051d0 msgr2=0x7fb3641f1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c0051d0 0x7fb3641f1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 -- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 msgr2=0x7fb364154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 0x7fb364154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:17.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 -- 172.21.15.82:0/4172140883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364159a10 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb3694e8640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 0x7fb364154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.378+0000 7fb363fff640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb358009fb0 tx=0x7fb3580115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35801a020 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358016470 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3640c5c20 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358016840 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb364159450 con 0x7fb364158cb0 2024-04-25T10:05:17.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36404efa0 con 0x7fb364158cb0 2024-04-25T10:05:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb35801d020 con 0x7fb364158cb0 2024-04-25T10:05:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 0x7fb350069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb358085fb0 con 0x7fb364158cb0 2024-04-25T10:05:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb368ce7640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 0x7fb350069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:17.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.382+0000 7fb368ce7640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 0x7fb350069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35c001060 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:17.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.386+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb358024030 con 0x7fb364158cb0 2024-04-25T10:05:17.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:17.578+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb36415fa70 con 0x7fb364158cb0 2024-04-25T10:05:18.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.230+0000 7fb3617fa640 1 -- 172.21.15.82:0/4172140883 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7fb358056f20 con 0x7fb364158cb0 2024-04-25T10:05:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T10:05:18.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 msgr2=0x7fb350069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 0x7fb350069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb35c001060 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 msgr2=0x7fb364098860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb358009fb0 tx=0x7fb3580115a0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 shutdown_connections 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb350066f60 0x7fb350069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3641f4e70 0x7fb364154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb364158cb0 0x7fb364098860 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 --2- 172.21.15.82:0/4172140883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb34c0051d0 0x7fb3641f1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 >> 172.21.15.82:0/4172140883 conn(0x7fb36406ae60 msgr2=0x7fb36406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 shutdown_connections 2024-04-25T10:05:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.242+0000 7fb36af4b640 1 -- 172.21.15.82:0/4172140883 wait complete. 2024-04-25T10:05:18.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:18.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 Processor -- start 2024-04-25T10:05:18.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 -- start start 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 0x7f27380ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27380cee30 con 0x7f2738144ca0 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27380ccc70 con 0x7f27381451f0 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273c89e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27380cc2b0 con 0x7f27380ce510 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f2735d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2738144ca0 0x7f273806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48814/0 (socket says 172.21.15.82:48814) 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f2735d9b640 1 -- 172.21.15.82:0/988573586 learned_addr learned my addr 172.21.15.82:0/988573586 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 0x7f27380ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2106133793 0 0) 0x7f27380cee30 con 0x7f2738144ca0 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2724003530 con 0x7f2738144ca0 2024-04-25T10:05:18.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2702399239 0 0) 0x7f2724003530 con 0x7f2738144ca0 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 msgr2=0x7f27380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 0x7f27380ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 >> v1:172.21.15.196:6789/0 conn(0x7f27381451f0 legacy=0x7f2738142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273806dc40 con 0x7f2738144ca0 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2720001a00 con 0x7f2738144ca0 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 >> v1:172.21.15.82:6789/0 conn(0x7f2738144ca0 legacy=0x7f273806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2724003a60 0x7f2724003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2724004810 0x7f2724004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 0x7f27240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27380cc2b0 con 0x7f2724003a60 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272400bb60 con 0x7f2724004810 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272400bd60 con 0x7f27380ce510 2024-04-25T10:05:18.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2724003a60 0x7f2724003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 -- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 msgr2=0x7f27240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380ce510 0x7f27240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 -- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2724004810 msgr2=0x7f2724004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2724004810 0x7f2724004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.390+0000 7f273659c640 1 -- 172.21.15.82:0/988573586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724003530 con 0x7f2724003a60 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2724003a60 0x7f2724003e80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2728009ae0 tx=0x7f272801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b09a3151f3f3d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280b2070 con 0x7f2724003a60 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27280a55c0 con 0x7f2724003a60 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273559a640 1 -- 172.21.15.82:0/988573586 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280096b0 con 0x7f2724003a60 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- 172.21.15.82:0/988573586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2724003a60 msgr2=0x7f2724003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 --2- 172.21.15.82:0/988573586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2724003a60 0x7f2724003e80 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2728009ae0 tx=0x7f272801cab0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f2736d9d640 1 -- 172.21.15.82:0/988573586 reap_dead start 2024-04-25T10:05:18.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- 172.21.15.82:0/988573586 shutdown_connections 2024-04-25T10:05:18.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- 172.21.15.82:0/988573586 >> 172.21.15.82:0/988573586 conn(0x7f27380b11a0 msgr2=0x7f27380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- 172.21.15.82:0/988573586 shutdown_connections 2024-04-25T10:05:18.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- 172.21.15.82:0/988573586 wait complete. 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 Processor -- start 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- start start 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27381e5b10 0x7f27380a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 0x7f27381f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273806ee30 con 0x7f27380a8730 2024-04-25T10:05:18.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273806e290 con 0x7f27381e5b10 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273c89e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27380cc900 con 0x7f27380a9050 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44230/0 (socket says 172.21.15.82:44230) 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 -- 172.21.15.82:0/873974666 learned_addr learned my addr 172.21.15.82:0/873974666 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f2736d9d640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 0x7f27381f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f2735d9b640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27381e5b10 0x7f27380a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 -- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 msgr2=0x7f27381f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 0x7f27381f3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 -- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27381e5b10 msgr2=0x7f27380a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.394+0000 7f273659c640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27381e5b10 0x7f27380a8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f273659c640 1 -- 172.21.15.82:0/873974666 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27380b2fb0 con 0x7f27380a8730 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f2736d9d640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 0x7f27381f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:18.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f273659c640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f27280b0040 tx=0x7f2728019850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280b2040 con 0x7f27380a8730 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272800f000 con 0x7f27380a8730 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27380b3290 con 0x7f27380a8730 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728004020 con 0x7f27380a8730 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27380b3850 con 0x7f27380a8730 2024-04-25T10:05:18.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273804efa0 con 0x7f27380a8730 2024-04-25T10:05:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27280bf020 con 0x7f27380a8730 2024-04-25T10:05:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 0x7f2718069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f272811edd0 con 0x7f27380a8730 2024-04-25T10:05:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f2735d9b640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 0x7f2718069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.398+0000 7f2735d9b640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 0x7f2718069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2720001b70 tx=0x7f272000c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.402+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27280c6020 con 0x7f27380a8730 2024-04-25T10:05:18.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.562+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27381e6280 con 0x7f27380a8730 2024-04-25T10:05:18.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.562+0000 7f272effd640 1 -- 172.21.15.82:0/873974666 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f2728120020 con 0x7f27380a8730 2024-04-25T10:05:18.568 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:18.568 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":64,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:17.584119+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 msgr2=0x7f2718069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 0x7f2718069450 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2720001b70 tx=0x7f272000c040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 msgr2=0x7f27380b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f27280b0040 tx=0x7f2728019850 comp rx=0 tx=0).stop 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 shutdown_connections 2024-04-25T10:05:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2718066f90 0x7f2718069450 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f27380a9050 0x7f27381f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27381e5b10 0x7f27380a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 --2- 172.21.15.82:0/873974666 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27380a8730 0x7f27380b0a20 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 >> 172.21.15.82:0/873974666 conn(0x7f27380b11a0 msgr2=0x7f27380c6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 shutdown_connections 2024-04-25T10:05:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.566+0000 7f273c89e640 1 -- 172.21.15.82:0/873974666 wait complete. 2024-04-25T10:05:18.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:18.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.698+0000 7fb358d5c640 1 Processor -- start 2024-04-25T10:05:18.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.698+0000 7fb358d5c640 1 -- start start 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb3540ce510 0x7fb3540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb354144ca0 0x7fb35406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35406ec60 0x7fb354142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3540cc2b0 con 0x7fb3540ce510 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35406e290 con 0x7fb354144ca0 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb358d5c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540cee30 con 0x7fb35406ec60 2024-04-25T10:05:18.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb3540ce510 0x7fb3540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb3540ce510 0x7fb3540ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44232/0 (socket says 172.21.15.82:44232) 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 -- 172.21.15.82:0/3220428393 learned_addr learned my addr 172.21.15.82:0/3220428393 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb349d9b640 1 --2- 172.21.15.82:0/3220428393 >> v2:172.21.15.196:3300/0 conn(0x7fb354144ca0 0x7fb35406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb352d9d640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35406ec60 0x7fb354142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 -- 172.21.15.82:0/3220428393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35406ec60 msgr2=0x7fb354142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35406ec60 0x7fb354142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 -- 172.21.15.82:0/3220428393 >> v2:172.21.15.196:3300/0 conn(0x7fb354144ca0 msgr2=0x7fb35406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 --2- 172.21.15.82:0/3220428393 >> v2:172.21.15.196:3300/0 conn(0x7fb354144ca0 0x7fb35406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.702+0000 7fb35259c640 1 -- 172.21.15.82:0/3220428393 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3540ccc70 con 0x7fb3540ce510 2024-04-25T10:05:18.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb35259c640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3540ce510 0x7fb3540ce8f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb354070fe0 tx=0x7fb34401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebb41e774db4710 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb351d9b640 1 -- 172.21.15.82:0/3220428393 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3440b5070 con 0x7fb3540ce510 2024-04-25T10:05:18.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb351d9b640 1 -- 172.21.15.82:0/3220428393 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3440a89d0 con 0x7fb3540ce510 2024-04-25T10:05:18.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb351d9b640 1 -- 172.21.15.82:0/3220428393 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3440a8da0 con 0x7fb3540ce510 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- 172.21.15.82:0/3220428393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3540ce510 msgr2=0x7fb3540ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3540ce510 0x7fb3540ce8f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fb354070fe0 tx=0x7fb34401ed10 comp rx=0 tx=0).stop 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- 172.21.15.82:0/3220428393 shutdown_connections 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb35406ec60 0x7fb354142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 --2- 172.21.15.82:0/3220428393 >> v2:172.21.15.196:3300/0 conn(0x7fb354144ca0 0x7fb35406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 --2- 172.21.15.82:0/3220428393 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3540ce510 0x7fb3540ce8f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- 172.21.15.82:0/3220428393 >> 172.21.15.82:0/3220428393 conn(0x7fb3540b11a0 msgr2=0x7fb3540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- 172.21.15.82:0/3220428393 shutdown_connections 2024-04-25T10:05:18.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- 172.21.15.82:0/3220428393 wait complete. 2024-04-25T10:05:18.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 Processor -- start 2024-04-25T10:05:18.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.706+0000 7fb358d5c640 1 -- start start 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35406ec60 0x7fb3541438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3540ce510 0x7fb3541e2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3540cc900 con 0x7fb354144ca0 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35406df20 con 0x7fb35406ec60 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3541451f0 con 0x7fb3540ce510 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44234/0 (socket says 172.21.15.82:44234) 2024-04-25T10:05:18.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 -- 172.21.15.82:0/2597512120 learned_addr learned my addr 172.21.15.82:0/2597512120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb35259c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35406ec60 0x7fb3541438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb349d9b640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3540ce510 0x7fb3541e2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 -- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3540ce510 msgr2=0x7fb3541e2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3540ce510 0x7fb3541e2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 -- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35406ec60 msgr2=0x7fb3541438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35406ec60 0x7fb3541438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 -- 172.21.15.82:0/2597512120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3541e4850 con 0x7fb354144ca0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb352d9d640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb34c007e30 tx=0x7fb34c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c014420 con 0x7fb354144ca0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c019070 con 0x7fb354144ca0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3541e37b0 con 0x7fb354144ca0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c014800 con 0x7fb354144ca0 2024-04-25T10:05:18.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3541e3d70 con 0x7fb354144ca0 2024-04-25T10:05:18.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35404efa0 con 0x7fb354144ca0 2024-04-25T10:05:18.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb34c0149a0 con 0x7fb354144ca0 2024-04-25T10:05:18.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 0x7fb334069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:18.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb34c085920 con 0x7fb354144ca0 2024-04-25T10:05:18.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.710+0000 7fb35259c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 0x7fb334069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:18.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.714+0000 7fb35259c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 0x7fb334069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3440b2000 tx=0x7fb344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:18.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.718+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb34c056830 con 0x7fb354144ca0 2024-04-25T10:05:18.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.878+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3540b1cc0 con 0x7fb354144ca0 2024-04-25T10:05:18.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb34b7fe640 1 -- 172.21.15.82:0/2597512120 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16909 (secure 0 0 0) 0x7fb34c056130 con 0x7fb354144ca0 2024-04-25T10:05:18.887 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:18.887 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":64,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:17.584119+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 msgr2=0x7fb334069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 0x7fb334069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb3440b2000 tx=0x7fb344008040 comp rx=0 tx=0).stop 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 msgr2=0x7fb3540c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fb34c007e30 tx=0x7fb34c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 shutdown_connections 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb334066f90 0x7fb334069450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb3540ce510 0x7fb3541e2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb354144ca0 0x7fb3540c9e20 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 --2- 172.21.15.82:0/2597512120 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb35406ec60 0x7fb3541438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:18.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 >> 172.21.15.82:0/2597512120 conn(0x7fb3540b11a0 msgr2=0x7fb3540b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:18.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 shutdown_connections 2024-04-25T10:05:18.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:18.882+0000 7fb358d5c640 1 -- 172.21.15.82:0/2597512120 wait complete. 2024-04-25T10:05:18.905 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:05:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.022+0000 7fc107a24640 1 Processor -- start 2024-04-25T10:05:19.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 -- start start 2024-04-25T10:05:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1000cff50 0x7fc10015d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc10006b710 con 0x7fc10006b1d0 2024-04-25T10:05:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc100161560 con 0x7fc1000cff50 2024-04-25T10:05:19.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc107a24640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1001617b0 con 0x7fc10006fc50 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1000cff50 0x7fc10015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc104fbf640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc10006b1d0 0x7fc1000cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48826/0 (socket says 172.21.15.82:48826) 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc1000cff50 0x7fc10015d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44236/0 (socket says 172.21.15.82:44236) 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc104fbf640 1 -- 172.21.15.82:0/1631385882 learned_addr learned my addr 172.21.15.82:0/1631385882 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc1057c0640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 -- 172.21.15.82:0/1631385882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 msgr2=0x7fc10006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 -- 172.21.15.82:0/1631385882 >> v1:172.21.15.82:6789/0 conn(0x7fc10006b1d0 legacy=0x7fc1000cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 -- 172.21.15.82:0/1631385882 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100161a90 con 0x7fc1000cff50 2024-04-25T10:05:19.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc0f7fff640 1 -- 172.21.15.82:0/1631385882 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22663056 0 0) 0x7fc10006b710 con 0x7fc10006b1d0 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc105fc1640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015d180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0097c0 tx=0x7fc0fc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2740593949d8af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.026+0000 7fc0f7fff640 1 -- 172.21.15.82:0/1631385882 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0ab600 con 0x7fc1000cff50 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc0f7fff640 1 -- 172.21.15.82:0/1631385882 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0b0070 con 0x7fc1000cff50 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc0f7fff640 1 -- 172.21.15.82:0/1631385882 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0ab9c0 con 0x7fc1000cff50 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/1631385882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 msgr2=0x7fc10015d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015d180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0097c0 tx=0x7fc0fc01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/1631385882 shutdown_connections 2024-04-25T10:05:19.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015d180 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- 172.21.15.82:0/1631385882 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/1631385882 >> 172.21.15.82:0/1631385882 conn(0x7fc100099a70 msgr2=0x7fc1000d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:19.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/1631385882 shutdown_connections 2024-04-25T10:05:19.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/1631385882 wait complete. 2024-04-25T10:05:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 Processor -- start 2024-04-25T10:05:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- start start 2024-04-25T10:05:19.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc10006b1d0 0x7fc100152880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10015ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100162290 con 0x7fc1000cff50 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc100070240 con 0x7fc10006b1d0 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc100162410 con 0x7fc10006fc50 2024-04-25T10:05:19.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44250/0 (socket says 172.21.15.82:44250) 2024-04-25T10:05:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 -- 172.21.15.82:0/3610735879 learned_addr learned my addr 172.21.15.82:0/3610735879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc1057c0640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc10006b1d0 0x7fc100152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc104fbf640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10015ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 -- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 msgr2=0x7fc10015ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10015ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 -- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc10006b1d0 msgr2=0x7fc100152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc10006b1d0 0x7fc100152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 -- 172.21.15.82:0/3610735879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100153ca0 con 0x7fc1000cff50 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc105fc1640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0ad040 tx=0x7fc0fc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0bf610 con 0x7fc1000cff50 2024-04-25T10:05:19.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0fc0bf7b0 con 0x7fc1000cff50 2024-04-25T10:05:19.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc100153f60 con 0x7fc1000cff50 2024-04-25T10:05:19.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0bfba0 con 0x7fc1000cff50 2024-04-25T10:05:19.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.030+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc100154520 con 0x7fc1000cff50 2024-04-25T10:05:19.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc10004efa0 con 0x7fc1000cff50 2024-04-25T10:05:19.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0fc0bd020 con 0x7fc1000cff50 2024-04-25T10:05:19.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc0f5ffb640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 0x7fc0e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:19.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0fc11f810 con 0x7fc1000cff50 2024-04-25T10:05:19.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc1057c0640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 0x7fc0e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:19.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.034+0000 7fc1057c0640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 0x7fc0e4069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8003fd0 tx=0x7fc0f80039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:19.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.038+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0fc016090 con 0x7fc1000cff50 2024-04-25T10:05:19.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:19.230+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc1001563b0 con 0x7fc1000cff50 2024-04-25T10:05:20.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.246+0000 7fc0f5ffb640 1 -- 172.21.15.82:0/3610735879 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc0fc0f07a0 con 0x7fc1000cff50 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 msgr2=0x7fc0e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 0x7fc0e4069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8003fd0 tx=0x7fc0f80039a0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 msgr2=0x7fc10015f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc0ad040 tx=0x7fc0fc01ac20 comp rx=0 tx=0).stop 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 shutdown_connections 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0e4066f60 0x7fc0e4069420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc10006fc50 0x7fc10015ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc1000cff50 0x7fc10015f170 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 --2- 172.21.15.82:0/3610735879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc10006b1d0 0x7fc100152880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 >> 172.21.15.82:0/3610735879 conn(0x7fc100099a70 msgr2=0x7fc1000aee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 shutdown_connections 2024-04-25T10:05:20.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.250+0000 7fc107a24640 1 -- 172.21.15.82:0/3610735879 wait complete. 2024-04-25T10:05:20.276 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:05:20.276 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:05:20.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 Processor -- start 2024-04-25T10:05:20.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 -- start start 2024-04-25T10:05:20.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24440b1ac0 0x7f24440cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24440afb20 con 0x7f244406b820 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24440b06c0 con 0x7f24440b1ac0 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2449fad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440afd90 con 0x7f244406b440 2024-04-25T10:05:20.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24440b1ac0 0x7f24440cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f24440b1ac0 0x7f24440cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44252/0 (socket says 172.21.15.82:44252) 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 -- 172.21.15.82:0/4062375514 learned_addr learned my addr 172.21.15.82:0/4062375514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 -- 172.21.15.82:0/4062375514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 msgr2=0x7f24440b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 --2- 172.21.15.82:0/4062375514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 -- 172.21.15.82:0/4062375514 >> v1:172.21.15.196:6789/0 conn(0x7f244406b820 legacy=0x7f24440b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:20.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.402+0000 7f2443fff640 1 -- 172.21.15.82:0/4062375514 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440b0070 con 0x7f24440b1ac0 2024-04-25T10:05:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2443fff640 1 --2- 172.21.15.82:0/4062375514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24440b1ac0 0x7f24440cf650 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f243c009e50 tx=0x7f243c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82435b461f57c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f24427fc640 1 -- 172.21.15.82:0/4062375514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0a5780 con 0x7f24440b1ac0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f24427fc640 1 -- 172.21.15.82:0/4062375514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c0a5920 con 0x7f24440b1ac0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f24427fc640 1 -- 172.21.15.82:0/4062375514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0a5cf0 con 0x7f24440b1ac0 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- 172.21.15.82:0/4062375514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24440b1ac0 msgr2=0x7f24440cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- 172.21.15.82:0/4062375514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24440b1ac0 0x7f24440cf650 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f243c009e50 tx=0x7f243c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- 172.21.15.82:0/4062375514 shutdown_connections 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- 172.21.15.82:0/4062375514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24440b1ac0 0x7f24440cf650 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- 172.21.15.82:0/4062375514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- 172.21.15.82:0/4062375514 >> 172.21.15.82:0/4062375514 conn(0x7f24440b1f10 msgr2=0x7f24440b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- 172.21.15.82:0/4062375514 shutdown_connections 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- 172.21.15.82:0/4062375514 wait complete. 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 Processor -- start 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- start start 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 0x7f2444147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cd170 con 0x7f244406b820 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440ccff0 con 0x7f24440b1ac0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2449fad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244415d080 con 0x7f244406b440 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44262/0 (socket says 172.21.15.82:44262) 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 -- 172.21.15.82:0/1860510586 learned_addr learned my addr 172.21.15.82:0/1860510586 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2443fff640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 0x7f2444147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f24437fe640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 -- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 msgr2=0x7f24440ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 -- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 msgr2=0x7f2444147940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 0x7f2444147940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2442ffd640 1 -- 172.21.15.82:0/1860510586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444157b20 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f2443fff640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 0x7f2444147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.406+0000 7f24437fe640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f2442ffd640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f243800c410 tx=0x7f243800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438011790 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438017070 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438011bb0 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444156fa0 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2444157560 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244404efa0 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f243801c460 con 0x7f244406b820 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 0x7f2430069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:20.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f24437fe640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 0x7f2430069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:20.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2438084de0 con 0x7f244406b820 2024-04-25T10:05:20.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.410+0000 7f24437fe640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 0x7f2430069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f243400a8b0 tx=0x7f2434008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:20.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.414+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2438021020 con 0x7f244406b820 2024-04-25T10:05:20.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:20.610+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f24440cc160 con 0x7f244406b820 2024-04-25T10:05:21.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.250+0000 7f242bfff640 1 -- 172.21.15.82:0/1860510586 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2438055c60 con 0x7f244406b820 2024-04-25T10:05:21.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 msgr2=0x7f2430069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 0x7f2430069420 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f243400a8b0 tx=0x7f2434008040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 msgr2=0x7f2444147400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f243800c410 tx=0x7f243800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 shutdown_connections 2024-04-25T10:05:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2430066f60 0x7f2430069420 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24440b1ac0 0x7f2444147940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244406b820 0x7f2444147400 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 --2- 172.21.15.82:0/1860510586 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f244406b440 0x7f24440ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 >> 172.21.15.82:0/1860510586 conn(0x7f24440b1f10 msgr2=0x7f24440ad9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 shutdown_connections 2024-04-25T10:05:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.254+0000 7f2449fad640 1 -- 172.21.15.82:0/1860510586 wait complete. 2024-04-25T10:05:21.278 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:05:21.278 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.394+0000 7f15353aa640 1 Processor -- start 2024-04-25T10:05:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.394+0000 7f15353aa640 1 -- start start 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f15353aa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530159ea0 0x7f15300b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f15353aa640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1530071fd0 0x7f153015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f15353aa640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f153015ae30 con 0x7f153015a280 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f15353aa640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15300b2990 con 0x7f1530071fd0 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f15353aa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153015b110 con 0x7f1530159ea0 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1530071fd0 0x7f153015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1530071fd0 0x7f153015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44266/0 (socket says 172.21.15.82:44266) 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 -- 172.21.15.82:0/3034663877 learned_addr learned my addr 172.21.15.82:0/3034663877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152effd640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530159ea0 0x7f15300b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 -- 172.21.15.82:0/3034663877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530159ea0 msgr2=0x7f15300b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530159ea0 0x7f15300b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 -- 172.21.15.82:0/3034663877 >> v1:172.21.15.196:6789/0 conn(0x7f153015a280 legacy=0x7f15300718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.398+0000 7f152f7fe640 1 -- 172.21.15.82:0/3034663877 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15300b2620 con 0x7f1530071fd0 2024-04-25T10:05:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152f7fe640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1530071fd0 0x7f153015a700 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395fe61a2878d150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152e7fc640 1 -- 172.21.15.82:0/3034663877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280a7070 con 0x7f1530071fd0 2024-04-25T10:05:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152e7fc640 1 -- 172.21.15.82:0/3034663877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15280b0070 con 0x7f1530071fd0 2024-04-25T10:05:21.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152e7fc640 1 -- 172.21.15.82:0/3034663877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280ab890 con 0x7f1530071fd0 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- 172.21.15.82:0/3034663877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1530071fd0 msgr2=0x7f153015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1530071fd0 0x7f153015a700 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f1528009870 tx=0x7f152801c560 comp rx=0 tx=0).stop 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- 172.21.15.82:0/3034663877 shutdown_connections 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1530071fd0 0x7f153015a700 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- 172.21.15.82:0/3034663877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530159ea0 0x7f15300b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- 172.21.15.82:0/3034663877 >> 172.21.15.82:0/3034663877 conn(0x7f15300b11a0 msgr2=0x7f15300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- 172.21.15.82:0/3034663877 shutdown_connections 2024-04-25T10:05:21.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- 172.21.15.82:0/3034663877 wait complete. 2024-04-25T10:05:21.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 Processor -- start 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- start start 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530071fd0 0x7f15301f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1530159ea0 0x7f1530155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1530166d40 con 0x7f153015a280 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1530166bc0 con 0x7f1530159ea0 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15353aa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1530166ec0 con 0x7f1530071fd0 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44272/0 (socket says 172.21.15.82:44272) 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152f7fe640 1 -- 172.21.15.82:0/1050916453 learned_addr learned my addr 172.21.15.82:0/1050916453 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f152effd640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530071fd0 0x7f15301f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.402+0000 7f15267fc640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1530159ea0 0x7f1530155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 -- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530071fd0 msgr2=0x7f15301f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530071fd0 0x7f15301f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 -- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1530159ea0 msgr2=0x7f1530155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1530159ea0 0x7f1530155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 -- 172.21.15.82:0/1050916453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153014c0f0 con 0x7f153015a280 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152f7fe640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f152801a190 tx=0x7f152801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280be5f0 con 0x7f153015a280 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15280be790 con 0x7f153015a280 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1530148960 con 0x7f153015a280 2024-04-25T10:05:21.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15280beb40 con 0x7f153015a280 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1530148f20 con 0x7f153015a280 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f153004efa0 con 0x7f153015a280 2024-04-25T10:05:21.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15280b43f0 con 0x7f153015a280 2024-04-25T10:05:21.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 0x7f1510069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152effd640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 0x7f1510069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f152811f9d0 con 0x7f153015a280 2024-04-25T10:05:21.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.406+0000 7f152effd640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 0x7f1510069450 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1520007330 tx=0x7f152000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.410+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1528016150 con 0x7f153015a280 2024-04-25T10:05:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.602+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f153006e6b0 con 0x7f153015a280 2024-04-25T10:05:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.602+0000 7f1527fff640 1 -- 172.21.15.82:0/1050916453 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f15280f0a00 con 0x7f153015a280 2024-04-25T10:05:21.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:21.608 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:05:20.613208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:21.609 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-25T10:05:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 msgr2=0x7f1510069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 0x7f1510069450 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1520007330 tx=0x7f152000b040 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 msgr2=0x7f1530147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f152801a190 tx=0x7f152801ada0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 shutdown_connections 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1510066f90 0x7f1510069450 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f153015a280 0x7f1530147800 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1530159ea0 0x7f1530155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 --2- 172.21.15.82:0/1050916453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1530071fd0 0x7f15301f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 >> 172.21.15.82:0/1050916453 conn(0x7f15300b11a0 msgr2=0x7f153014ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 shutdown_connections 2024-04-25T10:05:21.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.606+0000 7f15353aa640 1 -- 172.21.15.82:0/1050916453 wait complete. 2024-04-25T10:05:21.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:05:21.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 Processor -- start 2024-04-25T10:05:21.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 -- start start 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4da009a180 0x7f4da00b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00cbec0 con 0x7f4da00b2c30 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00cc1a0 con 0x7f4da009a7b0 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4da4b4d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00cc480 con 0x7f4da009a180 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4da00b2c30 0x7f4da0099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48834/0 (socket says 172.21.15.82:48834) 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 -- 172.21.15.82:0/1821148826 learned_addr learned my addr 172.21.15.82:0/1821148826 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/1821148826 >> v2:172.21.15.196:3300/0 conn(0x7f4da009a180 0x7f4da00b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036585292 0 0) 0x7f4da00cbec0 con 0x7f4da00b2c30 2024-04-25T10:05:21.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d88003530 con 0x7f4da00b2c30 2024-04-25T10:05:21.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653281620 0 0) 0x7f4d88003530 con 0x7f4da00b2c30 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 >> v2:172.21.15.196:3300/0 conn(0x7f4da009a180 msgr2=0x7f4da00b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 --2- 172.21.15.82:0/1821148826 >> v2:172.21.15.196:3300/0 conn(0x7f4da009a180 0x7f4da00b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 >> v1:172.21.15.196:6789/0 conn(0x7f4da009a7b0 legacy=0x7f4da00ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00cc760 con 0x7f4da00b2c30 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4d90001a00 con 0x7f4da00b2c30 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 >> v1:172.21.15.82:6789/0 conn(0x7f4da00b2c30 legacy=0x7f4da0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 0x7f4d88003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88005220 0x7f4d8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da00cc480 con 0x7f4d88003a80 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8800c150 con 0x7f4d88005220 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8800c350 con 0x7f4d88004810 2024-04-25T10:05:21.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 0x7f4d88003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 -- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d88004810 msgr2=0x7f4d88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9ed9d640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88005220 0x7f4d8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 -- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88005220 msgr2=0x7f4d8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88005220 0x7f4d8800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9e59c640 1 -- 172.21.15.82:0/1821148826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d88003530 con 0x7f4d88003a80 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d88004810 0x7f4d88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9ed9d640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88005220 0x7f4d8800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:21.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.738+0000 7f4d9ed9d640 1 -- 172.21.15.82:0/1821148826 reap_dead start 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 0x7f4d88003ea0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000a7b0 tx=0x7f4d9001d2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fab21f839dc6aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9000a3d0 con 0x7f4d88003a80 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900ad650 con 0x7f4d88003a80 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4d9d59a640 1 -- 172.21.15.82:0/1821148826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900adac0 con 0x7f4d88003a80 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 msgr2=0x7f4d88003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 0x7f4d88003ea0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000a7b0 tx=0x7f4d9001d2b0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- 172.21.15.82:0/1821148826 shutdown_connections 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/1821148826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d88003a80 0x7f4d88003ea0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- 172.21.15.82:0/1821148826 >> 172.21.15.82:0/1821148826 conn(0x7f4da00b3010 msgr2=0x7f4da00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- 172.21.15.82:0/1821148826 shutdown_connections 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- 172.21.15.82:0/1821148826 wait complete. 2024-04-25T10:05:21.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 Processor -- start 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 -- start start 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88003a80 0x7f4da00c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.742+0000 7f4da4b4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4da014c2e0 0x7f4da015ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da009a000 con 0x7f4da00c7c90 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0099e80 con 0x7f4d88003a80 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da006c200 con 0x7f4da014c2e0 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44296/0 (socket says 172.21.15.82:44296) 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 -- 172.21.15.82:0/2906423477 learned_addr learned my addr 172.21.15.82:0/2906423477 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88003a80 0x7f4da00c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9ed9d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4da014c2e0 0x7f4da015ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 -- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4da014c2e0 msgr2=0x7f4da015ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4da014c2e0 0x7f4da015ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 -- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88003a80 msgr2=0x7f4da00c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88003a80 0x7f4da00c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 -- 172.21.15.82:0/2906423477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00c6ed0 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9dd9b640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4d940107c0 tx=0x7f4d94010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400f040 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9400eb60 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da00c71b0 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401c400 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da00c7770 con 0x7f4da00c7c90 2024-04-25T10:05:21.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da004efa0 con 0x7f4da00c7c90 2024-04-25T10:05:21.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d9401c5a0 con 0x7f4da00c7c90 2024-04-25T10:05:21.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:21.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:21.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d940856f0 con 0x7f4da00c7c90 2024-04-25T10:05:21.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.746+0000 7f4d9e59c640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 0x7f4d80069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000a780 tx=0x7f4d900a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:21.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.750+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d94018030 con 0x7f4da00c7c90 2024-04-25T10:05:21.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.910+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4da006ba80 con 0x7f4da00c7c90 2024-04-25T10:05:21.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.910+0000 7f4d8effd640 1 -- 172.21.15.82:0/2906423477 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f4d94056600 con 0x7f4da00c7c90 2024-04-25T10:05:21.915 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:21.915 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":65,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:19.230242+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 msgr2=0x7f4d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 0x7f4d80069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000a780 tx=0x7f4d900a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 msgr2=0x7f4da014f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4d940107c0 tx=0x7f4d94010c80 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 shutdown_connections 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d80066f60 0x7f4d80069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4da014c2e0 0x7f4da015ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4da00c7c90 0x7f4da014f6f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 --2- 172.21.15.82:0/2906423477 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d88003a80 0x7f4da00c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 >> 172.21.15.82:0/2906423477 conn(0x7f4da00b3010 msgr2=0x7f4da006fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 shutdown_connections 2024-04-25T10:05:21.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:21.914+0000 7f4da4b4d640 1 -- 172.21.15.82:0/2906423477 wait complete. 2024-04-25T10:05:21.932 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:05:21.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.042+0000 7f0e1808f640 1 Processor -- start 2024-04-25T10:05:22.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.042+0000 7f0e1808f640 1 -- start start 2024-04-25T10:05:22.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1808f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1808f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0e100d1cd0 0x7f0e100d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1808f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1006be60 con 0x7f0e100d0620 2024-04-25T10:05:22.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1808f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1006c440 con 0x7f0e100d1cd0 2024-04-25T10:05:22.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1808f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1015de60 con 0x7f0e1015da80 2024-04-25T10:05:22.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e15e2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1562a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0e100d1cd0 0x7f0e100d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e15e2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:43262/0 (socket says 172.21.15.82:43262) 2024-04-25T10:05:22.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e15e2b640 1 -- 172.21.15.82:0/3246435103 learned_addr learned my addr 172.21.15.82:0/3246435103 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:22.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1562a640 1 -- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 msgr2=0x7f0e100d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1562a640 1 --2- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1562a640 1 -- 172.21.15.82:0/3246435103 >> v1:172.21.15.196:6789/0 conn(0x7f0e100d0620 legacy=0x7f0e100ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.046+0000 7f0e1562a640 1 -- 172.21.15.82:0/3246435103 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100d2450 con 0x7f0e100d1cd0 2024-04-25T10:05:22.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d1cd0 0x7f0e100d00e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097a0 tx=0x7f0e0001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e21bde5d1edb97f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e14e29640 1 -- 172.21.15.82:0/3246435103 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b2070 con 0x7f0e100d1cd0 2024-04-25T10:05:22.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e14e29640 1 -- 172.21.15.82:0/3246435103 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000ad640 con 0x7f0e100d1cd0 2024-04-25T10:05:22.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e14e29640 1 -- 172.21.15.82:0/3246435103 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000ada10 con 0x7f0e100d1cd0 2024-04-25T10:05:22.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d1cd0 msgr2=0x7f0e100d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d1cd0 0x7f0e100d00e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e000097a0 tx=0x7f0e0001c480 comp rx=0 tx=0).stop 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- 172.21.15.82:0/3246435103 shutdown_connections 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d1cd0 0x7f0e100d00e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3246435103 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- 172.21.15.82:0/3246435103 >> 172.21.15.82:0/3246435103 conn(0x7f0e1008b000 msgr2=0x7f0e100d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- 172.21.15.82:0/3246435103 shutdown_connections 2024-04-25T10:05:22.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- 172.21.15.82:0/3246435103 wait complete. 2024-04-25T10:05:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 Processor -- start 2024-04-25T10:05:22.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- start start 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 0x7f0e1015acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1016a730 con 0x7f0e100d1cd0 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1016a5b0 con 0x7f0e100d0620 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1808f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1016a8b0 con 0x7f0e1015da80 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44304/0 (socket says 172.21.15.82:44304) 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 -- 172.21.15.82:0/3170480946 learned_addr learned my addr 172.21.15.82:0/3170480946 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:22.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e15e2b640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 0x7f0e1015acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1662c640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 -- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 msgr2=0x7f0e100b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100b0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 -- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 msgr2=0x7f0e1015acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 0x7f0e1015acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 -- 172.21.15.82:0/3170480946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100c8ee0 con 0x7f0e100d1cd0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e15e2b640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 0x7f0e1015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.050+0000 7f0e1562a640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0e000b0040 tx=0x7f0e00019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b2040 con 0x7f0e100d1cd0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e000ad440 con 0x7f0e100d1cd0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100c82d0 con 0x7f0e100d1cd0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e000b6690 con 0x7f0e100d1cd0 2024-04-25T10:05:22.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e100c8890 con 0x7f0e100d1cd0 2024-04-25T10:05:22.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1004efa0 con 0x7f0e100d1cd0 2024-04-25T10:05:22.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e000bd020 con 0x7f0e100d1cd0 2024-04-25T10:05:22.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 0x7f0df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0e0011e6f0 con 0x7f0e100d1cd0 2024-04-25T10:05:22.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e15e2b640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 0x7f0df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.054+0000 7f0e15e2b640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 0x7f0df4069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b00 tx=0x7f0e0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.058+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e00016070 con 0x7f0e100d1cd0 2024-04-25T10:05:22.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.206+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e100d1580 con 0x7f0df4066f60 2024-04-25T10:05:22.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.222+0000 7f0e067fc640 1 -- 172.21.15.82:0/3170480946 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252022 (secure 0 0 0) 0x7f0e100d1580 con 0x7f0df4066f60 2024-04-25T10:05:22.230 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:22.232 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:05:22.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 msgr2=0x7f0df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 0x7f0df4069420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c009b00 tx=0x7f0e0c008040 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 msgr2=0x7f0e100c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0e000b0040 tx=0x7f0e00019220 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 shutdown_connections 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0df4066f60 0x7f0df4069420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0e1015da80 0x7f0e100b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0e100d1cd0 0x7f0e100c7980 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 --2- 172.21.15.82:0/3170480946 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0e100d0620 0x7f0e1015acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 >> 172.21.15.82:0/3170480946 conn(0x7f0e1008b000 msgr2=0x7f0e10160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 shutdown_connections 2024-04-25T10:05:22.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.226+0000 7f0e1808f640 1 -- 172.21.15.82:0/3170480946 wait complete. 2024-04-25T10:05:22.247 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-25T10:05:21.320085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252608,"kb_used_data":7296,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722112,"statfs":{"total":773094113280,"available":772835442688,"internally_reserved":0,"allocated":7471104,"data_stored":3990756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.118249"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738473+0000","last_change":"2024-04-25T10:05:18.266073+0000","last_active":"2024-04-25T10:05:19.738473+0000","last_peered":"2024-04-25T10:05:19.738473+0000","last_clean":"2024-04-25T10:05:19.738473+0000","last_became_active":"2024-04-25T10:05:18.265847+0000","last_became_peered":"2024-04-25T10:05:18.265847+0000","last_unstale":"2024-04-25T10:05:19.738473+0000","last_undegraded":"2024-04-25T10:05:19.738473+0000","last_fullsized":"2024-04-25T10:05:19.738473+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:12.029114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738850+0000","last_change":"2024-04-25T10:05:18.249987+0000","last_active":"2024-04-25T10:05:19.738850+0000","last_peered":"2024-04-25T10:05:19.738850+0000","last_clean":"2024-04-25T10:05:19.738850+0000","last_became_active":"2024-04-25T10:05:18.249680+0000","last_became_peered":"2024-04-25T10:05:18.249680+0000","last_unstale":"2024-04-25T10:05:19.738850+0000","last_undegraded":"2024-04-25T10:05:19.738850+0000","last_fullsized":"2024-04-25T10:05:19.738850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:10.560967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734665+0000","last_change":"2024-04-25T10:05:18.254504+0000","last_active":"2024-04-25T10:05:19.734665+0000","last_peered":"2024-04-25T10:05:19.734665+0000","last_clean":"2024-04-25T10:05:19.734665+0000","last_became_active":"2024-04-25T10:05:18.253594+0000","last_became_peered":"2024-04-25T10:05:18.253594+0000","last_unstale":"2024-04-25T10:05:19.734665+0000","last_undegraded":"2024-04-25T10:05:19.734665+0000","last_fullsized":"2024-04-25T10:05:19.734665+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:11.626947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239827+0000","last_change":"2024-04-25T10:05:18.253911+0000","last_active":"2024-04-25T10:05:19.239827+0000","last_peered":"2024-04-25T10:05:19.239827+0000","last_clean":"2024-04-25T10:05:19.239827+0000","last_became_active":"2024-04-25T10:05:18.242018+0000","last_became_peered":"2024-04-25T10:05:18.242018+0000","last_unstale":"2024-04-25T10:05:19.239827+0000","last_undegraded":"2024-04-25T10:05:19.239827+0000","last_fullsized":"2024-04-25T10:05:19.239827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.986096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236781+0000","last_change":"2024-04-25T10:05:18.258181+0000","last_active":"2024-04-25T10:05:19.236781+0000","last_peered":"2024-04-25T10:05:19.236781+0000","last_clean":"2024-04-25T10:05:19.236781+0000","last_became_active":"2024-04-25T10:05:18.256680+0000","last_became_peered":"2024-04-25T10:05:18.256680+0000","last_unstale":"2024-04-25T10:05:19.236781+0000","last_undegraded":"2024-04-25T10:05:19.236781+0000","last_fullsized":"2024-04-25T10:05:19.236781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.986096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237285+0000","last_change":"2024-04-25T10:05:18.254498+0000","last_active":"2024-04-25T10:05:19.237285+0000","last_peered":"2024-04-25T10:05:19.237285+0000","last_clean":"2024-04-25T10:05:19.237285+0000","last_became_active":"2024-04-25T10:05:18.254254+0000","last_became_peered":"2024-04-25T10:05:18.254254+0000","last_unstale":"2024-04-25T10:05:19.237285+0000","last_undegraded":"2024-04-25T10:05:19.237285+0000","last_fullsized":"2024-04-25T10:05:19.237285+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:24.377634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736334+0000","last_change":"2024-04-25T10:05:18.259499+0000","last_active":"2024-04-25T10:05:19.736334+0000","last_peered":"2024-04-25T10:05:19.736334+0000","last_clean":"2024-04-25T10:05:19.736334+0000","last_became_active":"2024-04-25T10:05:18.259218+0000","last_became_peered":"2024-04-25T10:05:18.259218+0000","last_unstale":"2024-04-25T10:05:19.736334+0000","last_undegraded":"2024-04-25T10:05:19.736334+0000","last_fullsized":"2024-04-25T10:05:19.736334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:59.400771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632463+0000","last_change":"2024-04-25T10:05:18.263468+0000","last_active":"2024-04-25T10:05:19.632463+0000","last_peered":"2024-04-25T10:05:19.632463+0000","last_clean":"2024-04-25T10:05:19.632463+0000","last_became_active":"2024-04-25T10:05:18.262198+0000","last_became_peered":"2024-04-25T10:05:18.262198+0000","last_unstale":"2024-04-25T10:05:19.632463+0000","last_undegraded":"2024-04-25T10:05:19.632463+0000","last_fullsized":"2024-04-25T10:05:19.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:17.114446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735112+0000","last_change":"2024-04-25T10:05:18.286456+0000","last_active":"2024-04-25T10:05:19.735112+0000","last_peered":"2024-04-25T10:05:19.735112+0000","last_clean":"2024-04-25T10:05:19.735112+0000","last_became_active":"2024-04-25T10:05:18.285639+0000","last_became_peered":"2024-04-25T10:05:18.285639+0000","last_unstale":"2024-04-25T10:05:19.735112+0000","last_undegraded":"2024-04-25T10:05:19.735112+0000","last_fullsized":"2024-04-25T10:05:19.735112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734778+0000","last_change":"2024-04-25T10:05:18.261697+0000","last_active":"2024-04-25T10:05:19.734778+0000","last_peered":"2024-04-25T10:05:19.734778+0000","last_clean":"2024-04-25T10:05:19.734778+0000","last_became_active":"2024-04-25T10:05:18.260953+0000","last_became_peered":"2024-04-25T10:05:18.260953+0000","last_unstale":"2024-04-25T10:05:19.734778+0000","last_undegraded":"2024-04-25T10:05:19.734778+0000","last_fullsized":"2024-04-25T10:05:19.734778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:17.114446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239466+0000","last_change":"2024-04-25T10:05:18.257006+0000","last_active":"2024-04-25T10:05:19.239466+0000","last_peered":"2024-04-25T10:05:19.239466+0000","last_clean":"2024-04-25T10:05:19.239466+0000","last_became_active":"2024-04-25T10:05:18.253864+0000","last_became_peered":"2024-04-25T10:05:18.253864+0000","last_unstale":"2024-04-25T10:05:19.239466+0000","last_undegraded":"2024-04-25T10:05:19.239466+0000","last_fullsized":"2024-04-25T10:05:19.239466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:16.686586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632396+0000","last_change":"2024-04-25T10:05:18.263264+0000","last_active":"2024-04-25T10:05:19.632396+0000","last_peered":"2024-04-25T10:05:19.632396+0000","last_clean":"2024-04-25T10:05:19.632396+0000","last_became_active":"2024-04-25T10:05:18.261547+0000","last_became_peered":"2024-04-25T10:05:18.261547+0000","last_unstale":"2024-04-25T10:05:19.632396+0000","last_undegraded":"2024-04-25T10:05:19.632396+0000","last_fullsized":"2024-04-25T10:05:19.632396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:11.626947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632581+0000","last_change":"2024-04-25T10:05:18.286006+0000","last_active":"2024-04-25T10:05:19.632581+0000","last_peered":"2024-04-25T10:05:19.632581+0000","last_clean":"2024-04-25T10:05:19.632581+0000","last_became_active":"2024-04-25T10:05:18.285439+0000","last_became_peered":"2024-04-25T10:05:18.285439+0000","last_unstale":"2024-04-25T10:05:19.632581+0000","last_undegraded":"2024-04-25T10:05:19.632581+0000","last_fullsized":"2024-04-25T10:05:19.632581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:47.932432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734126+0000","last_change":"2024-04-25T10:05:18.254578+0000","last_active":"2024-04-25T10:05:19.734126+0000","last_peered":"2024-04-25T10:05:19.734126+0000","last_clean":"2024-04-25T10:05:19.734126+0000","last_became_active":"2024-04-25T10:05:18.253882+0000","last_became_peered":"2024-04-25T10:05:18.253882+0000","last_unstale":"2024-04-25T10:05:19.734126+0000","last_undegraded":"2024-04-25T10:05:19.734126+0000","last_fullsized":"2024-04-25T10:05:19.734126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:53.588305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733948+0000","last_change":"2024-04-25T10:05:18.254412+0000","last_active":"2024-04-25T10:05:19.733948+0000","last_peered":"2024-04-25T10:05:19.733948+0000","last_clean":"2024-04-25T10:05:19.733948+0000","last_became_active":"2024-04-25T10:05:18.253709+0000","last_became_peered":"2024-04-25T10:05:18.253709+0000","last_unstale":"2024-04-25T10:05:19.733948+0000","last_undegraded":"2024-04-25T10:05:19.733948+0000","last_fullsized":"2024-04-25T10:05:19.733948+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:27.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737179+0000","last_change":"2024-04-25T10:05:18.279250+0000","last_active":"2024-04-25T10:05:19.737179+0000","last_peered":"2024-04-25T10:05:19.737179+0000","last_clean":"2024-04-25T10:05:19.737179+0000","last_became_active":"2024-04-25T10:05:18.278909+0000","last_became_peered":"2024-04-25T10:05:18.278909+0000","last_unstale":"2024-04-25T10:05:19.737179+0000","last_undegraded":"2024-04-25T10:05:19.737179+0000","last_fullsized":"2024-04-25T10:05:19.737179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:48.562976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738859+0000","last_change":"2024-04-25T10:05:18.280248+0000","last_active":"2024-04-25T10:05:19.738859+0000","last_peered":"2024-04-25T10:05:19.738859+0000","last_clean":"2024-04-25T10:05:19.738859+0000","last_became_active":"2024-04-25T10:05:18.279941+0000","last_became_peered":"2024-04-25T10:05:18.279941+0000","last_unstale":"2024-04-25T10:05:19.738859+0000","last_undegraded":"2024-04-25T10:05:19.738859+0000","last_fullsized":"2024-04-25T10:05:19.738859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:03.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239692+0000","last_change":"2024-04-25T10:05:18.253411+0000","last_active":"2024-04-25T10:05:19.239692+0000","last_peered":"2024-04-25T10:05:19.239692+0000","last_clean":"2024-04-25T10:05:19.239692+0000","last_became_active":"2024-04-25T10:05:18.242696+0000","last_became_peered":"2024-04-25T10:05:18.242696+0000","last_unstale":"2024-04-25T10:05:19.239692+0000","last_undegraded":"2024-04-25T10:05:19.239692+0000","last_fullsized":"2024-04-25T10:05:19.239692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.017832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238179+0000","last_change":"2024-04-25T10:05:18.244207+0000","last_active":"2024-04-25T10:05:19.238179+0000","last_peered":"2024-04-25T10:05:19.238179+0000","last_clean":"2024-04-25T10:05:19.238179+0000","last_became_active":"2024-04-25T10:05:18.238573+0000","last_became_peered":"2024-04-25T10:05:18.238573+0000","last_unstale":"2024-04-25T10:05:19.238179+0000","last_undegraded":"2024-04-25T10:05:19.238179+0000","last_fullsized":"2024-04-25T10:05:19.238179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:45.487013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.739153+0000","last_change":"2024-04-25T10:05:18.239096+0000","last_active":"2024-04-25T10:05:19.739153+0000","last_peered":"2024-04-25T10:05:19.739153+0000","last_clean":"2024-04-25T10:05:19.739153+0000","last_became_active":"2024-04-25T10:05:18.238342+0000","last_became_peered":"2024-04-25T10:05:18.238342+0000","last_unstale":"2024-04-25T10:05:19.739153+0000","last_undegraded":"2024-04-25T10:05:19.739153+0000","last_fullsized":"2024-04-25T10:05:19.739153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:58.893177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632826+0000","last_change":"2024-04-25T10:05:18.262558+0000","last_active":"2024-04-25T10:05:19.632826+0000","last_peered":"2024-04-25T10:05:19.632826+0000","last_clean":"2024-04-25T10:05:19.632826+0000","last_became_active":"2024-04-25T10:05:18.261268+0000","last_became_peered":"2024-04-25T10:05:18.261268+0000","last_unstale":"2024-04-25T10:05:19.632826+0000","last_undegraded":"2024-04-25T10:05:19.632826+0000","last_fullsized":"2024-04-25T10:05:19.632826+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:41.414208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737376+0000","last_change":"2024-04-25T10:05:18.257533+0000","last_active":"2024-04-25T10:05:19.737376+0000","last_peered":"2024-04-25T10:05:19.737376+0000","last_clean":"2024-04-25T10:05:19.737376+0000","last_became_active":"2024-04-25T10:05:18.257184+0000","last_became_peered":"2024-04-25T10:05:18.257184+0000","last_unstale":"2024-04-25T10:05:19.737376+0000","last_undegraded":"2024-04-25T10:05:19.737376+0000","last_fullsized":"2024-04-25T10:05:19.737376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:21.755389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736580+0000","last_change":"2024-04-25T10:05:18.257479+0000","last_active":"2024-04-25T10:05:19.736580+0000","last_peered":"2024-04-25T10:05:19.736580+0000","last_clean":"2024-04-25T10:05:19.736580+0000","last_became_active":"2024-04-25T10:05:18.257121+0000","last_became_peered":"2024-04-25T10:05:18.257121+0000","last_unstale":"2024-04-25T10:05:19.736580+0000","last_undegraded":"2024-04-25T10:05:19.736580+0000","last_fullsized":"2024-04-25T10:05:19.736580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:56.867311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733721+0000","last_change":"2024-04-25T10:05:18.258238+0000","last_active":"2024-04-25T10:05:19.733721+0000","last_peered":"2024-04-25T10:05:19.733721+0000","last_clean":"2024-04-25T10:05:19.733721+0000","last_became_active":"2024-04-25T10:05:18.257372+0000","last_became_peered":"2024-04-25T10:05:18.257372+0000","last_unstale":"2024-04-25T10:05:19.733721+0000","last_undegraded":"2024-04-25T10:05:19.733721+0000","last_fullsized":"2024-04-25T10:05:19.733721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:59.029882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632988+0000","last_change":"2024-04-25T10:05:18.269501+0000","last_active":"2024-04-25T10:05:19.632988+0000","last_peered":"2024-04-25T10:05:19.632988+0000","last_clean":"2024-04-25T10:05:19.632988+0000","last_became_active":"2024-04-25T10:05:18.268762+0000","last_became_peered":"2024-04-25T10:05:18.268762+0000","last_unstale":"2024-04-25T10:05:19.632988+0000","last_undegraded":"2024-04-25T10:05:19.632988+0000","last_fullsized":"2024-04-25T10:05:19.632988+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:24.377634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632373+0000","last_change":"2024-04-25T10:05:18.272435+0000","last_active":"2024-04-25T10:05:19.632373+0000","last_peered":"2024-04-25T10:05:19.632373+0000","last_clean":"2024-04-25T10:05:19.632373+0000","last_became_active":"2024-04-25T10:05:18.271638+0000","last_became_peered":"2024-04-25T10:05:18.271638+0000","last_unstale":"2024-04-25T10:05:19.632373+0000","last_undegraded":"2024-04-25T10:05:19.632373+0000","last_fullsized":"2024-04-25T10:05:19.632373+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:45.141237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239633+0000","last_change":"2024-04-25T10:05:18.282530+0000","last_active":"2024-04-25T10:05:19.239633+0000","last_peered":"2024-04-25T10:05:19.239633+0000","last_clean":"2024-04-25T10:05:19.239633+0000","last_became_active":"2024-04-25T10:05:18.280335+0000","last_became_peered":"2024-04-25T10:05:18.280335+0000","last_unstale":"2024-04-25T10:05:19.239633+0000","last_undegraded":"2024-04-25T10:05:19.239633+0000","last_fullsized":"2024-04-25T10:05:19.239633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:49.276161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736478+0000","last_change":"2024-04-25T10:05:18.265097+0000","last_active":"2024-04-25T10:05:19.736478+0000","last_peered":"2024-04-25T10:05:19.736478+0000","last_clean":"2024-04-25T10:05:19.736478+0000","last_became_active":"2024-04-25T10:05:18.264848+0000","last_became_peered":"2024-04-25T10:05:18.264848+0000","last_unstale":"2024-04-25T10:05:19.736478+0000","last_undegraded":"2024-04-25T10:05:19.736478+0000","last_fullsized":"2024-04-25T10:05:19.736478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:09.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734403+0000","last_change":"2024-04-25T10:05:18.286213+0000","last_active":"2024-04-25T10:05:19.734403+0000","last_peered":"2024-04-25T10:05:19.734403+0000","last_clean":"2024-04-25T10:05:19.734403+0000","last_became_active":"2024-04-25T10:05:18.285562+0000","last_became_peered":"2024-04-25T10:05:18.285562+0000","last_unstale":"2024-04-25T10:05:19.734403+0000","last_undegraded":"2024-04-25T10:05:19.734403+0000","last_fullsized":"2024-04-25T10:05:19.734403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:17.659611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632064+0000","last_change":"2024-04-25T10:05:18.262843+0000","last_active":"2024-04-25T10:05:19.632064+0000","last_peered":"2024-04-25T10:05:19.632064+0000","last_clean":"2024-04-25T10:05:19.632064+0000","last_became_active":"2024-04-25T10:05:18.261646+0000","last_became_peered":"2024-04-25T10:05:18.261646+0000","last_unstale":"2024-04-25T10:05:19.632064+0000","last_undegraded":"2024-04-25T10:05:19.632064+0000","last_fullsized":"2024-04-25T10:05:19.632064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:53.588305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237301+0000","last_change":"2024-04-25T10:05:18.283172+0000","last_active":"2024-04-25T10:05:19.237301+0000","last_peered":"2024-04-25T10:05:19.237301+0000","last_clean":"2024-04-25T10:05:19.237301+0000","last_became_active":"2024-04-25T10:05:18.282668+0000","last_became_peered":"2024-04-25T10:05:18.282668+0000","last_unstale":"2024-04-25T10:05:19.237301+0000","last_undegraded":"2024-04-25T10:05:19.237301+0000","last_fullsized":"2024-04-25T10:05:19.237301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:34.485681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236895+0000","last_change":"2024-04-25T10:05:18.243140+0000","last_active":"2024-04-25T10:05:19.236895+0000","last_peered":"2024-04-25T10:05:19.236895+0000","last_clean":"2024-04-25T10:05:19.236895+0000","last_became_active":"2024-04-25T10:05:18.242534+0000","last_became_peered":"2024-04-25T10:05:18.242534+0000","last_unstale":"2024-04-25T10:05:19.236895+0000","last_undegraded":"2024-04-25T10:05:19.236895+0000","last_fullsized":"2024-04-25T10:05:19.236895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:47.932432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237863+0000","last_change":"2024-04-25T10:05:18.247126+0000","last_active":"2024-04-25T10:05:19.237863+0000","last_peered":"2024-04-25T10:05:19.237863+0000","last_clean":"2024-04-25T10:05:19.237863+0000","last_became_active":"2024-04-25T10:05:18.246550+0000","last_became_peered":"2024-04-25T10:05:18.246550+0000","last_unstale":"2024-04-25T10:05:19.237863+0000","last_undegraded":"2024-04-25T10:05:19.237863+0000","last_fullsized":"2024-04-25T10:05:19.237863+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738858+0000","last_change":"2024-04-25T10:05:17.259528+0000","last_active":"2024-04-25T10:05:19.738858+0000","last_peered":"2024-04-25T10:05:19.738858+0000","last_clean":"2024-04-25T10:05:19.738858+0000","last_became_active":"2024-04-25T10:05:17.259246+0000","last_became_peered":"2024-04-25T10:05:17.259246+0000","last_unstale":"2024-04-25T10:05:19.738858+0000","last_undegraded":"2024-04-25T10:05:19.738858+0000","last_fullsized":"2024-04-25T10:05:19.738858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:48.265234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239931+0000","last_change":"2024-04-25T10:05:18.243350+0000","last_active":"2024-04-25T10:05:19.239931+0000","last_peered":"2024-04-25T10:05:19.239931+0000","last_clean":"2024-04-25T10:05:19.239931+0000","last_became_active":"2024-04-25T10:05:18.242523+0000","last_became_peered":"2024-04-25T10:05:18.242523+0000","last_unstale":"2024-04-25T10:05:19.239931+0000","last_undegraded":"2024-04-25T10:05:19.239931+0000","last_fullsized":"2024-04-25T10:05:19.239931+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:26.134704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237466+0000","last_change":"2024-04-25T10:05:17.255885+0000","last_active":"2024-04-25T10:05:19.237466+0000","last_peered":"2024-04-25T10:05:19.237466+0000","last_clean":"2024-04-25T10:05:19.237466+0000","last_became_active":"2024-04-25T10:05:17.255484+0000","last_became_peered":"2024-04-25T10:05:17.255484+0000","last_unstale":"2024-04-25T10:05:19.237466+0000","last_undegraded":"2024-04-25T10:05:19.237466+0000","last_fullsized":"2024-04-25T10:05:19.237466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:52.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237368+0000","last_change":"2024-04-25T10:05:18.243963+0000","last_active":"2024-04-25T10:05:19.237368+0000","last_peered":"2024-04-25T10:05:19.237368+0000","last_clean":"2024-04-25T10:05:19.237368+0000","last_became_active":"2024-04-25T10:05:18.243474+0000","last_became_peered":"2024-04-25T10:05:18.243474+0000","last_unstale":"2024-04-25T10:05:19.237368+0000","last_undegraded":"2024-04-25T10:05:19.237368+0000","last_fullsized":"2024-04-25T10:05:19.237368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:56.867311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237310+0000","last_change":"2024-04-25T10:05:17.247531+0000","last_active":"2024-04-25T10:05:19.237310+0000","last_peered":"2024-04-25T10:05:19.237310+0000","last_clean":"2024-04-25T10:05:19.237310+0000","last_became_active":"2024-04-25T10:05:17.247230+0000","last_became_peered":"2024-04-25T10:05:17.247230+0000","last_unstale":"2024-04-25T10:05:19.237310+0000","last_undegraded":"2024-04-25T10:05:19.237310+0000","last_fullsized":"2024-04-25T10:05:19.237310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:43.354561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734932+0000","last_change":"2024-04-25T10:05:18.260143+0000","last_active":"2024-04-25T10:05:19.734932+0000","last_peered":"2024-04-25T10:05:19.734932+0000","last_clean":"2024-04-25T10:05:19.734932+0000","last_became_active":"2024-04-25T10:05:18.259329+0000","last_became_peered":"2024-04-25T10:05:18.259329+0000","last_unstale":"2024-04-25T10:05:19.734932+0000","last_undegraded":"2024-04-25T10:05:19.734932+0000","last_fullsized":"2024-04-25T10:05:19.734932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:44.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237452+0000","last_change":"2024-04-25T10:05:17.231646+0000","last_active":"2024-04-25T10:05:19.237452+0000","last_peered":"2024-04-25T10:05:19.237452+0000","last_clean":"2024-04-25T10:05:19.237452+0000","last_became_active":"2024-04-25T10:05:17.230987+0000","last_became_peered":"2024-04-25T10:05:17.230987+0000","last_unstale":"2024-04-25T10:05:19.237452+0000","last_undegraded":"2024-04-25T10:05:19.237452+0000","last_fullsized":"2024-04-25T10:05:19.237452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:58.018955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737564+0000","last_change":"2024-04-25T10:05:18.253214+0000","last_active":"2024-04-25T10:05:19.737564+0000","last_peered":"2024-04-25T10:05:19.737564+0000","last_clean":"2024-04-25T10:05:19.737564+0000","last_became_active":"2024-04-25T10:05:18.252886+0000","last_became_peered":"2024-04-25T10:05:18.252886+0000","last_unstale":"2024-04-25T10:05:19.737564+0000","last_undegraded":"2024-04-25T10:05:19.737564+0000","last_fullsized":"2024-04-25T10:05:19.737564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:25.797922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737948+0000","last_change":"2024-04-25T10:05:17.235866+0000","last_active":"2024-04-25T10:05:19.737948+0000","last_peered":"2024-04-25T10:05:19.737948+0000","last_clean":"2024-04-25T10:05:19.737948+0000","last_became_active":"2024-04-25T10:05:17.235469+0000","last_became_peered":"2024-04-25T10:05:17.235469+0000","last_unstale":"2024-04-25T10:05:19.737948+0000","last_undegraded":"2024-04-25T10:05:19.737948+0000","last_fullsized":"2024-04-25T10:05:19.737948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:10.616020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738752+0000","last_change":"2024-04-25T10:05:18.280259+0000","last_active":"2024-04-25T10:05:19.738752+0000","last_peered":"2024-04-25T10:05:19.738752+0000","last_clean":"2024-04-25T10:05:19.738752+0000","last_became_active":"2024-04-25T10:05:18.279957+0000","last_became_peered":"2024-04-25T10:05:18.279957+0000","last_unstale":"2024-04-25T10:05:19.738752+0000","last_undegraded":"2024-04-25T10:05:19.738752+0000","last_fullsized":"2024-04-25T10:05:19.738752+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.621379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737652+0000","last_change":"2024-04-25T10:05:17.238014+0000","last_active":"2024-04-25T10:05:19.737652+0000","last_peered":"2024-04-25T10:05:19.737652+0000","last_clean":"2024-04-25T10:05:19.737652+0000","last_became_active":"2024-04-25T10:05:17.236728+0000","last_became_peered":"2024-04-25T10:05:17.236728+0000","last_unstale":"2024-04-25T10:05:19.737652+0000","last_undegraded":"2024-04-25T10:05:19.737652+0000","last_fullsized":"2024-04-25T10:05:19.737652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.844495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236560+0000","last_change":"2024-04-25T10:05:18.250458+0000","last_active":"2024-04-25T10:05:19.236560+0000","last_peered":"2024-04-25T10:05:19.236560+0000","last_clean":"2024-04-25T10:05:19.236560+0000","last_became_active":"2024-04-25T10:05:18.249706+0000","last_became_peered":"2024-04-25T10:05:18.249706+0000","last_unstale":"2024-04-25T10:05:19.236560+0000","last_undegraded":"2024-04-25T10:05:19.236560+0000","last_fullsized":"2024-04-25T10:05:19.236560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.017832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736821+0000","last_change":"2024-04-25T10:05:17.245582+0000","last_active":"2024-04-25T10:05:19.736821+0000","last_peered":"2024-04-25T10:05:19.736821+0000","last_clean":"2024-04-25T10:05:19.736821+0000","last_became_active":"2024-04-25T10:05:17.245239+0000","last_became_peered":"2024-04-25T10:05:17.245239+0000","last_unstale":"2024-04-25T10:05:19.736821+0000","last_undegraded":"2024-04-25T10:05:19.736821+0000","last_fullsized":"2024-04-25T10:05:19.736821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:58.018955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737808+0000","last_change":"2024-04-25T10:05:18.233488+0000","last_active":"2024-04-25T10:05:19.737808+0000","last_peered":"2024-04-25T10:05:19.737808+0000","last_clean":"2024-04-25T10:05:19.737808+0000","last_became_active":"2024-04-25T10:05:18.232718+0000","last_became_peered":"2024-04-25T10:05:18.232718+0000","last_unstale":"2024-04-25T10:05:19.737808+0000","last_undegraded":"2024-04-25T10:05:19.737808+0000","last_fullsized":"2024-04-25T10:05:19.737808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.734876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236609+0000","last_change":"2024-04-25T10:05:17.252617+0000","last_active":"2024-04-25T10:05:19.236609+0000","last_peered":"2024-04-25T10:05:19.236609+0000","last_clean":"2024-04-25T10:05:19.236609+0000","last_became_active":"2024-04-25T10:05:17.251875+0000","last_became_peered":"2024-04-25T10:05:17.251875+0000","last_unstale":"2024-04-25T10:05:19.236609+0000","last_undegraded":"2024-04-25T10:05:19.236609+0000","last_fullsized":"2024-04-25T10:05:19.236609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:06.492874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736861+0000","last_change":"2024-04-25T10:05:18.244742+0000","last_active":"2024-04-25T10:05:19.736861+0000","last_peered":"2024-04-25T10:05:19.736861+0000","last_clean":"2024-04-25T10:05:19.736861+0000","last_became_active":"2024-04-25T10:05:18.244049+0000","last_became_peered":"2024-04-25T10:05:18.244049+0000","last_unstale":"2024-04-25T10:05:19.736861+0000","last_undegraded":"2024-04-25T10:05:19.736861+0000","last_fullsized":"2024-04-25T10:05:19.736861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:43.009797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735154+0000","last_change":"2024-04-25T10:05:17.262111+0000","last_active":"2024-04-25T10:05:19.735154+0000","last_peered":"2024-04-25T10:05:19.735154+0000","last_clean":"2024-04-25T10:05:19.735154+0000","last_became_active":"2024-04-25T10:05:17.261418+0000","last_became_peered":"2024-04-25T10:05:17.261418+0000","last_unstale":"2024-04-25T10:05:19.735154+0000","last_undegraded":"2024-04-25T10:05:19.735154+0000","last_fullsized":"2024-04-25T10:05:19.735154+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:11.018187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738325+0000","last_change":"2024-04-25T10:05:18.251978+0000","last_active":"2024-04-25T10:05:19.738325+0000","last_peered":"2024-04-25T10:05:19.738325+0000","last_clean":"2024-04-25T10:05:19.738325+0000","last_became_active":"2024-04-25T10:05:18.251649+0000","last_became_peered":"2024-04-25T10:05:18.251649+0000","last_unstale":"2024-04-25T10:05:19.738325+0000","last_undegraded":"2024-04-25T10:05:19.738325+0000","last_fullsized":"2024-04-25T10:05:19.738325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:39.807129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.240000+0000","last_change":"2024-04-25T10:05:17.235813+0000","last_active":"2024-04-25T10:05:19.240000+0000","last_peered":"2024-04-25T10:05:19.240000+0000","last_clean":"2024-04-25T10:05:19.240000+0000","last_became_active":"2024-04-25T10:05:17.234668+0000","last_became_peered":"2024-04-25T10:05:17.234668+0000","last_unstale":"2024-04-25T10:05:19.240000+0000","last_undegraded":"2024-04-25T10:05:19.240000+0000","last_fullsized":"2024-04-25T10:05:19.240000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.250690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237726+0000","last_change":"2024-04-25T10:05:18.237283+0000","last_active":"2024-04-25T10:05:19.237726+0000","last_peered":"2024-04-25T10:05:19.237726+0000","last_clean":"2024-04-25T10:05:19.237726+0000","last_became_active":"2024-04-25T10:05:18.236564+0000","last_became_peered":"2024-04-25T10:05:18.236564+0000","last_unstale":"2024-04-25T10:05:19.237726+0000","last_undegraded":"2024-04-25T10:05:19.237726+0000","last_fullsized":"2024-04-25T10:05:19.237726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:57.541006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734972+0000","last_change":"2024-04-25T10:05:17.253324+0000","last_active":"2024-04-25T10:05:19.734972+0000","last_peered":"2024-04-25T10:05:19.734972+0000","last_clean":"2024-04-25T10:05:19.734972+0000","last_became_active":"2024-04-25T10:05:17.252849+0000","last_became_peered":"2024-04-25T10:05:17.252849+0000","last_unstale":"2024-04-25T10:05:19.734972+0000","last_undegraded":"2024-04-25T10:05:19.734972+0000","last_fullsized":"2024-04-25T10:05:19.734972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.476085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.240140+0000","last_change":"2024-04-25T10:05:18.254997+0000","last_active":"2024-04-25T10:05:19.240140+0000","last_peered":"2024-04-25T10:05:19.240140+0000","last_clean":"2024-04-25T10:05:19.240140+0000","last_became_active":"2024-04-25T10:05:18.242307+0000","last_became_peered":"2024-04-25T10:05:18.242307+0000","last_unstale":"2024-04-25T10:05:19.240140+0000","last_undegraded":"2024-04-25T10:05:19.240140+0000","last_fullsized":"2024-04-25T10:05:19.240140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:57.541006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737160+0000","last_change":"2024-04-25T10:05:17.246675+0000","last_active":"2024-04-25T10:05:19.737160+0000","last_peered":"2024-04-25T10:05:19.737160+0000","last_clean":"2024-04-25T10:05:19.737160+0000","last_became_active":"2024-04-25T10:05:17.246363+0000","last_became_peered":"2024-04-25T10:05:17.246363+0000","last_unstale":"2024-04-25T10:05:19.737160+0000","last_undegraded":"2024-04-25T10:05:19.737160+0000","last_fullsized":"2024-04-25T10:05:19.737160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:15.643774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239256+0000","last_change":"2024-04-25T10:05:18.283608+0000","last_active":"2024-04-25T10:05:19.239256+0000","last_peered":"2024-04-25T10:05:19.239256+0000","last_clean":"2024-04-25T10:05:19.239256+0000","last_became_active":"2024-04-25T10:05:18.283077+0000","last_became_peered":"2024-04-25T10:05:18.283077+0000","last_unstale":"2024-04-25T10:05:19.239256+0000","last_undegraded":"2024-04-25T10:05:19.239256+0000","last_fullsized":"2024-04-25T10:05:19.239256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:16.654701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734317+0000","last_change":"2024-04-25T10:05:17.257105+0000","last_active":"2024-04-25T10:05:19.734317+0000","last_peered":"2024-04-25T10:05:19.734317+0000","last_clean":"2024-04-25T10:05:19.734317+0000","last_became_active":"2024-04-25T10:05:17.256415+0000","last_became_peered":"2024-04-25T10:05:17.256415+0000","last_unstale":"2024-04-25T10:05:19.734317+0000","last_undegraded":"2024-04-25T10:05:19.734317+0000","last_fullsized":"2024-04-25T10:05:19.734317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:59.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632564+0000","last_change":"2024-04-25T10:05:18.248968+0000","last_active":"2024-04-25T10:05:19.632564+0000","last_peered":"2024-04-25T10:05:19.632564+0000","last_clean":"2024-04-25T10:05:19.632564+0000","last_became_active":"2024-04-25T10:05:18.248218+0000","last_became_peered":"2024-04-25T10:05:18.248218+0000","last_unstale":"2024-04-25T10:05:19.632564+0000","last_undegraded":"2024-04-25T10:05:19.632564+0000","last_fullsized":"2024-04-25T10:05:19.632564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.631996+0000","last_change":"2024-04-25T10:05:17.247771+0000","last_active":"2024-04-25T10:05:19.631996+0000","last_peered":"2024-04-25T10:05:19.631996+0000","last_clean":"2024-04-25T10:05:19.631996+0000","last_became_active":"2024-04-25T10:05:17.245928+0000","last_became_peered":"2024-04-25T10:05:17.245928+0000","last_unstale":"2024-04-25T10:05:19.631996+0000","last_undegraded":"2024-04-25T10:05:19.631996+0000","last_fullsized":"2024-04-25T10:05:19.631996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.476085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236942+0000","last_change":"2024-04-25T10:05:18.244705+0000","last_active":"2024-04-25T10:05:19.236942+0000","last_peered":"2024-04-25T10:05:19.236942+0000","last_clean":"2024-04-25T10:05:19.236942+0000","last_became_active":"2024-04-25T10:05:18.238952+0000","last_became_peered":"2024-04-25T10:05:18.238952+0000","last_unstale":"2024-04-25T10:05:19.236942+0000","last_undegraded":"2024-04-25T10:05:19.236942+0000","last_fullsized":"2024-04-25T10:05:19.236942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.874237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734242+0000","last_change":"2024-04-25T10:05:17.255176+0000","last_active":"2024-04-25T10:05:19.734242+0000","last_peered":"2024-04-25T10:05:19.734242+0000","last_clean":"2024-04-25T10:05:19.734242+0000","last_became_active":"2024-04-25T10:05:17.254374+0000","last_became_peered":"2024-04-25T10:05:17.254374+0000","last_unstale":"2024-04-25T10:05:19.734242+0000","last_undegraded":"2024-04-25T10:05:19.734242+0000","last_fullsized":"2024-04-25T10:05:19.734242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:24.655678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736296+0000","last_change":"2024-04-25T10:05:18.260223+0000","last_active":"2024-04-25T10:05:19.736296+0000","last_peered":"2024-04-25T10:05:19.736296+0000","last_clean":"2024-04-25T10:05:19.736296+0000","last_became_active":"2024-04-25T10:05:18.257535+0000","last_became_peered":"2024-04-25T10:05:18.257535+0000","last_unstale":"2024-04-25T10:05:19.736296+0000","last_undegraded":"2024-04-25T10:05:19.736296+0000","last_fullsized":"2024-04-25T10:05:19.736296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:45.487013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238996+0000","last_change":"2024-04-25T10:05:17.254350+0000","last_active":"2024-04-25T10:05:19.238996+0000","last_peered":"2024-04-25T10:05:19.238996+0000","last_clean":"2024-04-25T10:05:19.238996+0000","last_became_active":"2024-04-25T10:05:17.253928+0000","last_became_peered":"2024-04-25T10:05:17.253928+0000","last_unstale":"2024-04-25T10:05:19.238996+0000","last_undegraded":"2024-04-25T10:05:19.238996+0000","last_fullsized":"2024-04-25T10:05:19.238996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.740109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734546+0000","last_change":"2024-04-25T10:05:18.286163+0000","last_active":"2024-04-25T10:05:19.734546+0000","last_peered":"2024-04-25T10:05:19.734546+0000","last_clean":"2024-04-25T10:05:19.734546+0000","last_became_active":"2024-04-25T10:05:18.285584+0000","last_became_peered":"2024-04-25T10:05:18.285584+0000","last_unstale":"2024-04-25T10:05:19.734546+0000","last_undegraded":"2024-04-25T10:05:19.734546+0000","last_fullsized":"2024-04-25T10:05:19.734546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:45.141237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632737+0000","last_change":"2024-04-25T10:05:17.243578+0000","last_active":"2024-04-25T10:05:19.632737+0000","last_peered":"2024-04-25T10:05:19.632737+0000","last_clean":"2024-04-25T10:05:19.632737+0000","last_became_active":"2024-04-25T10:05:17.242848+0000","last_became_peered":"2024-04-25T10:05:17.242848+0000","last_unstale":"2024-04-25T10:05:19.632737+0000","last_undegraded":"2024-04-25T10:05:19.632737+0000","last_fullsized":"2024-04-25T10:05:19.632737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:24.655678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238082+0000","last_change":"2024-04-25T10:05:18.259011+0000","last_active":"2024-04-25T10:05:19.238082+0000","last_peered":"2024-04-25T10:05:19.238082+0000","last_clean":"2024-04-25T10:05:19.238082+0000","last_became_active":"2024-04-25T10:05:18.258199+0000","last_became_peered":"2024-04-25T10:05:18.258199+0000","last_unstale":"2024-04-25T10:05:19.238082+0000","last_undegraded":"2024-04-25T10:05:19.238082+0000","last_fullsized":"2024-04-25T10:05:19.238082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:54.337634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734922+0000","last_change":"2024-04-25T10:05:17.255726+0000","last_active":"2024-04-25T10:05:19.734922+0000","last_peered":"2024-04-25T10:05:19.734922+0000","last_clean":"2024-04-25T10:05:19.734922+0000","last_became_active":"2024-04-25T10:05:17.255193+0000","last_became_peered":"2024-04-25T10:05:17.255193+0000","last_unstale":"2024-04-25T10:05:19.734922+0000","last_undegraded":"2024-04-25T10:05:19.734922+0000","last_fullsized":"2024-04-25T10:05:19.734922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.998869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239910+0000","last_change":"2024-04-25T10:05:18.255531+0000","last_active":"2024-04-25T10:05:19.239910+0000","last_peered":"2024-04-25T10:05:19.239910+0000","last_clean":"2024-04-25T10:05:19.239910+0000","last_became_active":"2024-04-25T10:05:18.243046+0000","last_became_peered":"2024-04-25T10:05:18.243046+0000","last_unstale":"2024-04-25T10:05:19.239910+0000","last_undegraded":"2024-04-25T10:05:19.239910+0000","last_fullsized":"2024-04-25T10:05:19.239910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.874237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735044+0000","last_change":"2024-04-25T10:05:17.255681+0000","last_active":"2024-04-25T10:05:19.735044+0000","last_peered":"2024-04-25T10:05:19.735044+0000","last_clean":"2024-04-25T10:05:19.735044+0000","last_became_active":"2024-04-25T10:05:17.254926+0000","last_became_peered":"2024-04-25T10:05:17.254926+0000","last_unstale":"2024-04-25T10:05:19.735044+0000","last_undegraded":"2024-04-25T10:05:19.735044+0000","last_fullsized":"2024-04-25T10:05:19.735044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:08.342799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237849+0000","last_change":"2024-04-25T10:05:18.235488+0000","last_active":"2024-04-25T10:05:19.237849+0000","last_peered":"2024-04-25T10:05:19.237849+0000","last_clean":"2024-04-25T10:05:19.237849+0000","last_became_active":"2024-04-25T10:05:18.234806+0000","last_became_peered":"2024-04-25T10:05:18.234806+0000","last_unstale":"2024-04-25T10:05:19.237849+0000","last_undegraded":"2024-04-25T10:05:19.237849+0000","last_fullsized":"2024-04-25T10:05:19.237849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.557108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.739217+0000","last_change":"2024-04-25T10:05:17.247259+0000","last_active":"2024-04-25T10:05:19.739217+0000","last_peered":"2024-04-25T10:05:19.739217+0000","last_clean":"2024-04-25T10:05:19.739217+0000","last_became_active":"2024-04-25T10:05:17.246554+0000","last_became_peered":"2024-04-25T10:05:17.246554+0000","last_unstale":"2024-04-25T10:05:19.739217+0000","last_undegraded":"2024-04-25T10:05:19.739217+0000","last_fullsized":"2024-04-25T10:05:19.739217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:52.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236887+0000","last_change":"2024-04-25T10:05:18.242056+0000","last_active":"2024-04-25T10:05:19.236887+0000","last_peered":"2024-04-25T10:05:19.236887+0000","last_clean":"2024-04-25T10:05:19.236887+0000","last_became_active":"2024-04-25T10:05:18.241680+0000","last_became_peered":"2024-04-25T10:05:18.241680+0000","last_unstale":"2024-04-25T10:05:19.236887+0000","last_undegraded":"2024-04-25T10:05:19.236887+0000","last_fullsized":"2024-04-25T10:05:19.236887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237253+0000","last_change":"2024-04-25T10:05:17.241034+0000","last_active":"2024-04-25T10:05:19.237253+0000","last_peered":"2024-04-25T10:05:19.237253+0000","last_clean":"2024-04-25T10:05:19.237253+0000","last_became_active":"2024-04-25T10:05:17.240667+0000","last_became_peered":"2024-04-25T10:05:17.240667+0000","last_unstale":"2024-04-25T10:05:19.237253+0000","last_undegraded":"2024-04-25T10:05:19.237253+0000","last_fullsized":"2024-04-25T10:05:19.237253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:29.807096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632884+0000","last_change":"2024-04-25T10:05:18.277623+0000","last_active":"2024-04-25T10:05:19.632884+0000","last_peered":"2024-04-25T10:05:19.632884+0000","last_clean":"2024-04-25T10:05:19.632884+0000","last_became_active":"2024-04-25T10:05:18.276877+0000","last_became_peered":"2024-04-25T10:05:18.276877+0000","last_unstale":"2024-04-25T10:05:19.632884+0000","last_undegraded":"2024-04-25T10:05:19.632884+0000","last_fullsized":"2024-04-25T10:05:19.632884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:17.659611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632258+0000","last_change":"2024-04-25T10:05:17.248585+0000","last_active":"2024-04-25T10:05:19.632258+0000","last_peered":"2024-04-25T10:05:19.632258+0000","last_clean":"2024-04-25T10:05:19.632258+0000","last_became_active":"2024-04-25T10:05:17.247080+0000","last_became_peered":"2024-04-25T10:05:17.247080+0000","last_unstale":"2024-04-25T10:05:19.632258+0000","last_undegraded":"2024-04-25T10:05:19.632258+0000","last_fullsized":"2024-04-25T10:05:19.632258+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:08.342799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237014+0000","last_change":"2024-04-25T10:05:18.251129+0000","last_active":"2024-04-25T10:05:19.237014+0000","last_peered":"2024-04-25T10:05:19.237014+0000","last_clean":"2024-04-25T10:05:19.237014+0000","last_became_active":"2024-04-25T10:05:18.249949+0000","last_became_peered":"2024-04-25T10:05:18.249949+0000","last_unstale":"2024-04-25T10:05:19.237014+0000","last_undegraded":"2024-04-25T10:05:19.237014+0000","last_fullsized":"2024-04-25T10:05:19.237014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:26.134704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236931+0000","last_change":"2024-04-25T10:05:17.249980+0000","last_active":"2024-04-25T10:05:19.236931+0000","last_peered":"2024-04-25T10:05:19.236931+0000","last_clean":"2024-04-25T10:05:19.236931+0000","last_became_active":"2024-04-25T10:05:17.249129+0000","last_became_peered":"2024-04-25T10:05:17.249129+0000","last_unstale":"2024-04-25T10:05:19.236931+0000","last_undegraded":"2024-04-25T10:05:19.236931+0000","last_fullsized":"2024-04-25T10:05:19.236931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:31.306889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237173+0000","last_change":"2024-04-25T10:05:18.258349+0000","last_active":"2024-04-25T10:05:19.237173+0000","last_peered":"2024-04-25T10:05:19.237173+0000","last_clean":"2024-04-25T10:05:19.237173+0000","last_became_active":"2024-04-25T10:05:18.258058+0000","last_became_peered":"2024-04-25T10:05:18.258058+0000","last_unstale":"2024-04-25T10:05:19.237173+0000","last_undegraded":"2024-04-25T10:05:19.237173+0000","last_fullsized":"2024-04-25T10:05:19.237173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:41.414208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.633007+0000","last_change":"2024-04-25T10:05:17.256923+0000","last_active":"2024-04-25T10:05:19.633007+0000","last_peered":"2024-04-25T10:05:19.633007+0000","last_clean":"2024-04-25T10:05:19.633007+0000","last_became_active":"2024-04-25T10:05:17.256526+0000","last_became_peered":"2024-04-25T10:05:17.256526+0000","last_unstale":"2024-04-25T10:05:19.633007+0000","last_undegraded":"2024-04-25T10:05:19.633007+0000","last_fullsized":"2024-04-25T10:05:19.633007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.998869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737283+0000","last_change":"2024-04-25T10:05:18.243267+0000","last_active":"2024-04-25T10:05:19.737283+0000","last_peered":"2024-04-25T10:05:19.737283+0000","last_clean":"2024-04-25T10:05:19.737283+0000","last_became_active":"2024-04-25T10:05:18.242959+0000","last_became_peered":"2024-04-25T10:05:18.242959+0000","last_unstale":"2024-04-25T10:05:19.737283+0000","last_undegraded":"2024-04-25T10:05:19.737283+0000","last_fullsized":"2024-04-25T10:05:19.737283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.499355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632806+0000","last_change":"2024-04-25T10:05:17.253283+0000","last_active":"2024-04-25T10:05:19.632806+0000","last_peered":"2024-04-25T10:05:19.632806+0000","last_clean":"2024-04-25T10:05:19.632806+0000","last_became_active":"2024-04-25T10:05:17.252104+0000","last_became_peered":"2024-04-25T10:05:17.252104+0000","last_unstale":"2024-04-25T10:05:19.632806+0000","last_undegraded":"2024-04-25T10:05:19.632806+0000","last_fullsized":"2024-04-25T10:05:19.632806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:11.018187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632579+0000","last_change":"2024-04-25T10:05:18.249132+0000","last_active":"2024-04-25T10:05:19.632579+0000","last_peered":"2024-04-25T10:05:19.632579+0000","last_clean":"2024-04-25T10:05:19.632579+0000","last_became_active":"2024-04-25T10:05:18.248355+0000","last_became_peered":"2024-04-25T10:05:18.248355+0000","last_unstale":"2024-04-25T10:05:19.632579+0000","last_undegraded":"2024-04-25T10:05:19.632579+0000","last_fullsized":"2024-04-25T10:05:19.632579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:44.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237383+0000","last_change":"2024-04-25T10:05:17.250923+0000","last_active":"2024-04-25T10:05:19.237383+0000","last_peered":"2024-04-25T10:05:19.237383+0000","last_clean":"2024-04-25T10:05:19.237383+0000","last_became_active":"2024-04-25T10:05:17.250529+0000","last_became_peered":"2024-04-25T10:05:17.250529+0000","last_unstale":"2024-04-25T10:05:19.237383+0000","last_undegraded":"2024-04-25T10:05:19.237383+0000","last_fullsized":"2024-04-25T10:05:19.237383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.844495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238274+0000","last_change":"2024-04-25T10:05:18.236065+0000","last_active":"2024-04-25T10:05:19.238274+0000","last_peered":"2024-04-25T10:05:19.238274+0000","last_clean":"2024-04-25T10:05:19.238274+0000","last_became_active":"2024-04-25T10:05:18.233432+0000","last_became_peered":"2024-04-25T10:05:18.233432+0000","last_unstale":"2024-04-25T10:05:19.238274+0000","last_undegraded":"2024-04-25T10:05:19.238274+0000","last_fullsized":"2024-04-25T10:05:19.238274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:09.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737264+0000","last_change":"2024-04-25T10:05:17.247680+0000","last_active":"2024-04-25T10:05:19.737264+0000","last_peered":"2024-04-25T10:05:19.737264+0000","last_clean":"2024-04-25T10:05:19.737264+0000","last_became_active":"2024-04-25T10:05:17.246905+0000","last_became_peered":"2024-04-25T10:05:17.246905+0000","last_unstale":"2024-04-25T10:05:19.737264+0000","last_undegraded":"2024-04-25T10:05:19.737264+0000","last_fullsized":"2024-04-25T10:05:19.737264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:15.675659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236344+0000","last_change":"2024-04-25T10:05:18.258330+0000","last_active":"2024-04-25T10:05:19.236344+0000","last_peered":"2024-04-25T10:05:19.236344+0000","last_clean":"2024-04-25T10:05:19.236344+0000","last_became_active":"2024-04-25T10:05:18.256736+0000","last_became_peered":"2024-04-25T10:05:18.256736+0000","last_unstale":"2024-04-25T10:05:19.236344+0000","last_undegraded":"2024-04-25T10:05:19.236344+0000","last_fullsized":"2024-04-25T10:05:19.236344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:49.276161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734636+0000","last_change":"2024-04-25T10:05:17.261364+0000","last_active":"2024-04-25T10:05:19.734636+0000","last_peered":"2024-04-25T10:05:19.734636+0000","last_clean":"2024-04-25T10:05:19.734636+0000","last_became_active":"2024-04-25T10:05:17.257835+0000","last_became_peered":"2024-04-25T10:05:17.257835+0000","last_unstale":"2024-04-25T10:05:19.734636+0000","last_undegraded":"2024-04-25T10:05:19.734636+0000","last_fullsized":"2024-04-25T10:05:19.734636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:53.326707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237695+0000","last_change":"2024-04-25T10:05:18.270656+0000","last_active":"2024-04-25T10:05:19.237695+0000","last_peered":"2024-04-25T10:05:19.237695+0000","last_clean":"2024-04-25T10:05:19.237695+0000","last_became_active":"2024-04-25T10:05:18.269902+0000","last_became_peered":"2024-04-25T10:05:18.269902+0000","last_unstale":"2024-04-25T10:05:19.237695+0000","last_undegraded":"2024-04-25T10:05:19.237695+0000","last_fullsized":"2024-04-25T10:05:19.237695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.238488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737675+0000","last_change":"2024-04-25T10:05:17.237915+0000","last_active":"2024-04-25T10:05:19.737675+0000","last_peered":"2024-04-25T10:05:19.737675+0000","last_clean":"2024-04-25T10:05:19.737675+0000","last_became_active":"2024-04-25T10:05:17.237528+0000","last_became_peered":"2024-04-25T10:05:17.237528+0000","last_unstale":"2024-04-25T10:05:19.737675+0000","last_undegraded":"2024-04-25T10:05:19.737675+0000","last_fullsized":"2024-04-25T10:05:19.737675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:44.130310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736836+0000","last_change":"2024-04-25T10:05:18.280432+0000","last_active":"2024-04-25T10:05:19.736836+0000","last_peered":"2024-04-25T10:05:19.736836+0000","last_clean":"2024-04-25T10:05:19.736836+0000","last_became_active":"2024-04-25T10:05:18.280122+0000","last_became_peered":"2024-04-25T10:05:18.280122+0000","last_unstale":"2024-04-25T10:05:19.736836+0000","last_undegraded":"2024-04-25T10:05:19.736836+0000","last_fullsized":"2024-04-25T10:05:19.736836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.666605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737057+0000","last_change":"2024-04-25T10:05:17.245484+0000","last_active":"2024-04-25T10:05:19.737057+0000","last_peered":"2024-04-25T10:05:19.737057+0000","last_clean":"2024-04-25T10:05:19.737057+0000","last_became_active":"2024-04-25T10:05:17.245096+0000","last_became_peered":"2024-04-25T10:05:17.245096+0000","last_unstale":"2024-04-25T10:05:19.737057+0000","last_undegraded":"2024-04-25T10:05:19.737057+0000","last_fullsized":"2024-04-25T10:05:19.737057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:16.648683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736442+0000","last_change":"2024-04-25T10:05:18.253458+0000","last_active":"2024-04-25T10:05:19.736442+0000","last_peered":"2024-04-25T10:05:19.736442+0000","last_clean":"2024-04-25T10:05:19.736442+0000","last_became_active":"2024-04-25T10:05:18.253127+0000","last_became_peered":"2024-04-25T10:05:18.253127+0000","last_unstale":"2024-04-25T10:05:19.736442+0000","last_undegraded":"2024-04-25T10:05:19.736442+0000","last_fullsized":"2024-04-25T10:05:19.736442+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:31.237519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735655+0000","last_change":"2024-04-25T10:05:17.245484+0000","last_active":"2024-04-25T10:05:19.735655+0000","last_peered":"2024-04-25T10:05:19.735655+0000","last_clean":"2024-04-25T10:05:19.735655+0000","last_became_active":"2024-04-25T10:05:17.245094+0000","last_became_peered":"2024-04-25T10:05:17.245094+0000","last_unstale":"2024-04-25T10:05:19.735655+0000","last_undegraded":"2024-04-25T10:05:19.735655+0000","last_fullsized":"2024-04-25T10:05:19.735655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.227561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238692+0000","last_change":"2024-04-25T10:05:18.257706+0000","last_active":"2024-04-25T10:05:19.238692+0000","last_peered":"2024-04-25T10:05:19.238692+0000","last_clean":"2024-04-25T10:05:19.238692+0000","last_became_active":"2024-04-25T10:05:18.256153+0000","last_became_peered":"2024-04-25T10:05:18.256153+0000","last_unstale":"2024-04-25T10:05:19.238692+0000","last_undegraded":"2024-04-25T10:05:19.238692+0000","last_fullsized":"2024-04-25T10:05:19.238692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.666605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737493+0000","last_change":"2024-04-25T10:05:17.235903+0000","last_active":"2024-04-25T10:05:19.737493+0000","last_peered":"2024-04-25T10:05:19.737493+0000","last_clean":"2024-04-25T10:05:19.737493+0000","last_became_active":"2024-04-25T10:05:17.235503+0000","last_became_peered":"2024-04-25T10:05:17.235503+0000","last_unstale":"2024-04-25T10:05:19.737493+0000","last_undegraded":"2024-04-25T10:05:19.737493+0000","last_fullsized":"2024-04-25T10:05:19.737493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:31.306889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733791+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:05:19.733791+0000","last_peered":"2024-04-25T10:05:19.733791+0000","last_clean":"2024-04-25T10:05:19.733791+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:05:19.733791+0000","last_undegraded":"2024-04-25T10:05:19.733791+0000","last_fullsized":"2024-04-25T10:05:19.733791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":940121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":355703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":348999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":357281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":348977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":348977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":351353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":939345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:22.251 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.394+0000 7fec8eff6640 1 Processor -- start 2024-04-25T10:05:22.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.394+0000 7fec8eff6640 1 -- start start 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec880ce510 0x7fec880ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880cee30 con 0x7fec881451f0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec880ccc70 con 0x7fec88144ca0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec880cc2b0 con 0x7fec880ce510 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8cd92640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec880ce510 0x7fec880ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8d593640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fec881451f0 0x7fec88142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48842/0 (socket says 172.21.15.82:48842) 2024-04-25T10:05:22.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8d593640 1 -- 172.21.15.82:0/3791519202 learned_addr learned my addr 172.21.15.82:0/3791519202 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8cd92640 1 -- 172.21.15.82:0/3791519202 >> v1:172.21.15.196:6789/0 conn(0x7fec88144ca0 legacy=0x7fec8806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8cd92640 1 -- 172.21.15.82:0/3791519202 >> v1:172.21.15.82:6789/0 conn(0x7fec881451f0 legacy=0x7fec88142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8cd92640 1 -- 172.21.15.82:0/3791519202 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8806dc40 con 0x7fec880ce510 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8cd92640 1 --2- 172.21.15.82:0/3791519202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec880ce510 0x7fec880ce8f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a00aef6fdbba6c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec877fe640 1 -- 172.21.15.82:0/3791519202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec800b4070 con 0x7fec880ce510 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec877fe640 1 -- 172.21.15.82:0/3791519202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec800af600 con 0x7fec880ce510 2024-04-25T10:05:22.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec877fe640 1 -- 172.21.15.82:0/3791519202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec800af9d0 con 0x7fec880ce510 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- 172.21.15.82:0/3791519202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec880ce510 msgr2=0x7fec880ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 --2- 172.21.15.82:0/3791519202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec880ce510 0x7fec880ce8f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fec80009d90 tx=0x7fec8001f120 comp rx=0 tx=0).stop 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- 172.21.15.82:0/3791519202 shutdown_connections 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 --2- 172.21.15.82:0/3791519202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec880ce510 0x7fec880ce8f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- 172.21.15.82:0/3791519202 >> 172.21.15.82:0/3791519202 conn(0x7fec880b11a0 msgr2=0x7fec880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- 172.21.15.82:0/3791519202 shutdown_connections 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.398+0000 7fec8eff6640 1 -- 172.21.15.82:0/3791519202 wait complete. 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 Processor -- start 2024-04-25T10:05:22.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- start start 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec880ce510 0x7fec880b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec88144ca0 0x7fec881e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806ee30 con 0x7fec881451f0 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8806e290 con 0x7fec880ce510 2024-04-25T10:05:22.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec880cc900 con 0x7fec88144ca0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44312/0 (socket says 172.21.15.82:44312) 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 -- 172.21.15.82:0/2081017534 learned_addr learned my addr 172.21.15.82:0/2081017534 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8cd92640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec880ce510 0x7fec880b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec87fff640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec88144ca0 0x7fec881e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 -- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec88144ca0 msgr2=0x7fec881e71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec88144ca0 0x7fec881e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 -- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec880ce510 msgr2=0x7fec880b25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec880ce510 0x7fec880b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 -- 172.21.15.82:0/2081017534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec880b2f10 con 0x7fec881451f0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8d593640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fec7800c3c0 tx=0x7fec7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78017070 con 0x7fec881451f0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec78005cf0 con 0x7fec881451f0 2024-04-25T10:05:22.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec780072f0 con 0x7fec881451f0 2024-04-25T10:05:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec880b31f0 con 0x7fec881451f0 2024-04-25T10:05:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec881e10f0 con 0x7fec881451f0 2024-04-25T10:05:22.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.402+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8804efa0 con 0x7fec881451f0 2024-04-25T10:05:22.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.406+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec7801a070 con 0x7fec881451f0 2024-04-25T10:05:22.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.406+0000 7fec857fa640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 0x7fec68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.406+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec78084400 con 0x7fec881451f0 2024-04-25T10:05:22.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.406+0000 7fec8cd92640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 0x7fec68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.406+0000 7fec8cd92640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 0x7fec68069450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fec800b1000 tx=0x7fec800aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.410+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec78055310 con 0x7fec881451f0 2024-04-25T10:05:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.554+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec880b1cc0 con 0x7fec68066f90 2024-04-25T10:05:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.558+0000 7fec857fa640 1 -- 172.21.15.82:0/2081017534 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252022 (secure 0 0 0) 0x7fec880b1cc0 con 0x7fec68066f90 2024-04-25T10:05:22.565 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:22.567 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:05:22.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 msgr2=0x7fec68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 0x7fec68069450 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fec800b1000 tx=0x7fec800aa000 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 msgr2=0x7fec881f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fec7800c3c0 tx=0x7fec7800c880 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 shutdown_connections 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec68066f90 0x7fec68069450 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec88144ca0 0x7fec881e71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec881451f0 0x7fec881f7360 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 --2- 172.21.15.82:0/2081017534 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec880ce510 0x7fec880b25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 >> 172.21.15.82:0/2081017534 conn(0x7fec880b11a0 msgr2=0x7fec880c6670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 shutdown_connections 2024-04-25T10:05:22.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.562+0000 7fec8eff6640 1 -- 172.21.15.82:0/2081017534 wait complete. 2024-04-25T10:05:22.583 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-04-25T10:05:21.320085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":148,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252608,"kb_used_data":7296,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722112,"statfs":{"total":773094113280,"available":772835442688,"internally_reserved":0,"allocated":7471104,"data_stored":3990756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.118249"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738473+0000","last_change":"2024-04-25T10:05:18.266073+0000","last_active":"2024-04-25T10:05:19.738473+0000","last_peered":"2024-04-25T10:05:19.738473+0000","last_clean":"2024-04-25T10:05:19.738473+0000","last_became_active":"2024-04-25T10:05:18.265847+0000","last_became_peered":"2024-04-25T10:05:18.265847+0000","last_unstale":"2024-04-25T10:05:19.738473+0000","last_undegraded":"2024-04-25T10:05:19.738473+0000","last_fullsized":"2024-04-25T10:05:19.738473+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:12.029114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738850+0000","last_change":"2024-04-25T10:05:18.249987+0000","last_active":"2024-04-25T10:05:19.738850+0000","last_peered":"2024-04-25T10:05:19.738850+0000","last_clean":"2024-04-25T10:05:19.738850+0000","last_became_active":"2024-04-25T10:05:18.249680+0000","last_became_peered":"2024-04-25T10:05:18.249680+0000","last_unstale":"2024-04-25T10:05:19.738850+0000","last_undegraded":"2024-04-25T10:05:19.738850+0000","last_fullsized":"2024-04-25T10:05:19.738850+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:10.560967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734665+0000","last_change":"2024-04-25T10:05:18.254504+0000","last_active":"2024-04-25T10:05:19.734665+0000","last_peered":"2024-04-25T10:05:19.734665+0000","last_clean":"2024-04-25T10:05:19.734665+0000","last_became_active":"2024-04-25T10:05:18.253594+0000","last_became_peered":"2024-04-25T10:05:18.253594+0000","last_unstale":"2024-04-25T10:05:19.734665+0000","last_undegraded":"2024-04-25T10:05:19.734665+0000","last_fullsized":"2024-04-25T10:05:19.734665+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:11.626947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239827+0000","last_change":"2024-04-25T10:05:18.253911+0000","last_active":"2024-04-25T10:05:19.239827+0000","last_peered":"2024-04-25T10:05:19.239827+0000","last_clean":"2024-04-25T10:05:19.239827+0000","last_became_active":"2024-04-25T10:05:18.242018+0000","last_became_peered":"2024-04-25T10:05:18.242018+0000","last_unstale":"2024-04-25T10:05:19.239827+0000","last_undegraded":"2024-04-25T10:05:19.239827+0000","last_fullsized":"2024-04-25T10:05:19.239827+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.986096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236781+0000","last_change":"2024-04-25T10:05:18.258181+0000","last_active":"2024-04-25T10:05:19.236781+0000","last_peered":"2024-04-25T10:05:19.236781+0000","last_clean":"2024-04-25T10:05:19.236781+0000","last_became_active":"2024-04-25T10:05:18.256680+0000","last_became_peered":"2024-04-25T10:05:18.256680+0000","last_unstale":"2024-04-25T10:05:19.236781+0000","last_undegraded":"2024-04-25T10:05:19.236781+0000","last_fullsized":"2024-04-25T10:05:19.236781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:13.986096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237285+0000","last_change":"2024-04-25T10:05:18.254498+0000","last_active":"2024-04-25T10:05:19.237285+0000","last_peered":"2024-04-25T10:05:19.237285+0000","last_clean":"2024-04-25T10:05:19.237285+0000","last_became_active":"2024-04-25T10:05:18.254254+0000","last_became_peered":"2024-04-25T10:05:18.254254+0000","last_unstale":"2024-04-25T10:05:19.237285+0000","last_undegraded":"2024-04-25T10:05:19.237285+0000","last_fullsized":"2024-04-25T10:05:19.237285+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:24.377634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736334+0000","last_change":"2024-04-25T10:05:18.259499+0000","last_active":"2024-04-25T10:05:19.736334+0000","last_peered":"2024-04-25T10:05:19.736334+0000","last_clean":"2024-04-25T10:05:19.736334+0000","last_became_active":"2024-04-25T10:05:18.259218+0000","last_became_peered":"2024-04-25T10:05:18.259218+0000","last_unstale":"2024-04-25T10:05:19.736334+0000","last_undegraded":"2024-04-25T10:05:19.736334+0000","last_fullsized":"2024-04-25T10:05:19.736334+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:59.400771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632463+0000","last_change":"2024-04-25T10:05:18.263468+0000","last_active":"2024-04-25T10:05:19.632463+0000","last_peered":"2024-04-25T10:05:19.632463+0000","last_clean":"2024-04-25T10:05:19.632463+0000","last_became_active":"2024-04-25T10:05:18.262198+0000","last_became_peered":"2024-04-25T10:05:18.262198+0000","last_unstale":"2024-04-25T10:05:19.632463+0000","last_undegraded":"2024-04-25T10:05:19.632463+0000","last_fullsized":"2024-04-25T10:05:19.632463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:17.114446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735112+0000","last_change":"2024-04-25T10:05:18.286456+0000","last_active":"2024-04-25T10:05:19.735112+0000","last_peered":"2024-04-25T10:05:19.735112+0000","last_clean":"2024-04-25T10:05:19.735112+0000","last_became_active":"2024-04-25T10:05:18.285639+0000","last_became_peered":"2024-04-25T10:05:18.285639+0000","last_unstale":"2024-04-25T10:05:19.735112+0000","last_undegraded":"2024-04-25T10:05:19.735112+0000","last_fullsized":"2024-04-25T10:05:19.735112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734778+0000","last_change":"2024-04-25T10:05:18.261697+0000","last_active":"2024-04-25T10:05:19.734778+0000","last_peered":"2024-04-25T10:05:19.734778+0000","last_clean":"2024-04-25T10:05:19.734778+0000","last_became_active":"2024-04-25T10:05:18.260953+0000","last_became_peered":"2024-04-25T10:05:18.260953+0000","last_unstale":"2024-04-25T10:05:19.734778+0000","last_undegraded":"2024-04-25T10:05:19.734778+0000","last_fullsized":"2024-04-25T10:05:19.734778+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:46:17.114446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239466+0000","last_change":"2024-04-25T10:05:18.257006+0000","last_active":"2024-04-25T10:05:19.239466+0000","last_peered":"2024-04-25T10:05:19.239466+0000","last_clean":"2024-04-25T10:05:19.239466+0000","last_became_active":"2024-04-25T10:05:18.253864+0000","last_became_peered":"2024-04-25T10:05:18.253864+0000","last_unstale":"2024-04-25T10:05:19.239466+0000","last_undegraded":"2024-04-25T10:05:19.239466+0000","last_fullsized":"2024-04-25T10:05:19.239466+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:16.686586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632396+0000","last_change":"2024-04-25T10:05:18.263264+0000","last_active":"2024-04-25T10:05:19.632396+0000","last_peered":"2024-04-25T10:05:19.632396+0000","last_clean":"2024-04-25T10:05:19.632396+0000","last_became_active":"2024-04-25T10:05:18.261547+0000","last_became_peered":"2024-04-25T10:05:18.261547+0000","last_unstale":"2024-04-25T10:05:19.632396+0000","last_undegraded":"2024-04-25T10:05:19.632396+0000","last_fullsized":"2024-04-25T10:05:19.632396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:11.626947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632581+0000","last_change":"2024-04-25T10:05:18.286006+0000","last_active":"2024-04-25T10:05:19.632581+0000","last_peered":"2024-04-25T10:05:19.632581+0000","last_clean":"2024-04-25T10:05:19.632581+0000","last_became_active":"2024-04-25T10:05:18.285439+0000","last_became_peered":"2024-04-25T10:05:18.285439+0000","last_unstale":"2024-04-25T10:05:19.632581+0000","last_undegraded":"2024-04-25T10:05:19.632581+0000","last_fullsized":"2024-04-25T10:05:19.632581+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:47.932432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734126+0000","last_change":"2024-04-25T10:05:18.254578+0000","last_active":"2024-04-25T10:05:19.734126+0000","last_peered":"2024-04-25T10:05:19.734126+0000","last_clean":"2024-04-25T10:05:19.734126+0000","last_became_active":"2024-04-25T10:05:18.253882+0000","last_became_peered":"2024-04-25T10:05:18.253882+0000","last_unstale":"2024-04-25T10:05:19.734126+0000","last_undegraded":"2024-04-25T10:05:19.734126+0000","last_fullsized":"2024-04-25T10:05:19.734126+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:53.588305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733948+0000","last_change":"2024-04-25T10:05:18.254412+0000","last_active":"2024-04-25T10:05:19.733948+0000","last_peered":"2024-04-25T10:05:19.733948+0000","last_clean":"2024-04-25T10:05:19.733948+0000","last_became_active":"2024-04-25T10:05:18.253709+0000","last_became_peered":"2024-04-25T10:05:18.253709+0000","last_unstale":"2024-04-25T10:05:19.733948+0000","last_undegraded":"2024-04-25T10:05:19.733948+0000","last_fullsized":"2024-04-25T10:05:19.733948+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:27.855422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737179+0000","last_change":"2024-04-25T10:05:18.279250+0000","last_active":"2024-04-25T10:05:19.737179+0000","last_peered":"2024-04-25T10:05:19.737179+0000","last_clean":"2024-04-25T10:05:19.737179+0000","last_became_active":"2024-04-25T10:05:18.278909+0000","last_became_peered":"2024-04-25T10:05:18.278909+0000","last_unstale":"2024-04-25T10:05:19.737179+0000","last_undegraded":"2024-04-25T10:05:19.737179+0000","last_fullsized":"2024-04-25T10:05:19.737179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:48.562976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738859+0000","last_change":"2024-04-25T10:05:18.280248+0000","last_active":"2024-04-25T10:05:19.738859+0000","last_peered":"2024-04-25T10:05:19.738859+0000","last_clean":"2024-04-25T10:05:19.738859+0000","last_became_active":"2024-04-25T10:05:18.279941+0000","last_became_peered":"2024-04-25T10:05:18.279941+0000","last_unstale":"2024-04-25T10:05:19.738859+0000","last_undegraded":"2024-04-25T10:05:19.738859+0000","last_fullsized":"2024-04-25T10:05:19.738859+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:03.090885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239692+0000","last_change":"2024-04-25T10:05:18.253411+0000","last_active":"2024-04-25T10:05:19.239692+0000","last_peered":"2024-04-25T10:05:19.239692+0000","last_clean":"2024-04-25T10:05:19.239692+0000","last_became_active":"2024-04-25T10:05:18.242696+0000","last_became_peered":"2024-04-25T10:05:18.242696+0000","last_unstale":"2024-04-25T10:05:19.239692+0000","last_undegraded":"2024-04-25T10:05:19.239692+0000","last_fullsized":"2024-04-25T10:05:19.239692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.017832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238179+0000","last_change":"2024-04-25T10:05:18.244207+0000","last_active":"2024-04-25T10:05:19.238179+0000","last_peered":"2024-04-25T10:05:19.238179+0000","last_clean":"2024-04-25T10:05:19.238179+0000","last_became_active":"2024-04-25T10:05:18.238573+0000","last_became_peered":"2024-04-25T10:05:18.238573+0000","last_unstale":"2024-04-25T10:05:19.238179+0000","last_undegraded":"2024-04-25T10:05:19.238179+0000","last_fullsized":"2024-04-25T10:05:19.238179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:45.487013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.739153+0000","last_change":"2024-04-25T10:05:18.239096+0000","last_active":"2024-04-25T10:05:19.739153+0000","last_peered":"2024-04-25T10:05:19.739153+0000","last_clean":"2024-04-25T10:05:19.739153+0000","last_became_active":"2024-04-25T10:05:18.238342+0000","last_became_peered":"2024-04-25T10:05:18.238342+0000","last_unstale":"2024-04-25T10:05:19.739153+0000","last_undegraded":"2024-04-25T10:05:19.739153+0000","last_fullsized":"2024-04-25T10:05:19.739153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:58.893177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632826+0000","last_change":"2024-04-25T10:05:18.262558+0000","last_active":"2024-04-25T10:05:19.632826+0000","last_peered":"2024-04-25T10:05:19.632826+0000","last_clean":"2024-04-25T10:05:19.632826+0000","last_became_active":"2024-04-25T10:05:18.261268+0000","last_became_peered":"2024-04-25T10:05:18.261268+0000","last_unstale":"2024-04-25T10:05:19.632826+0000","last_undegraded":"2024-04-25T10:05:19.632826+0000","last_fullsized":"2024-04-25T10:05:19.632826+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:41.414208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737376+0000","last_change":"2024-04-25T10:05:18.257533+0000","last_active":"2024-04-25T10:05:19.737376+0000","last_peered":"2024-04-25T10:05:19.737376+0000","last_clean":"2024-04-25T10:05:19.737376+0000","last_became_active":"2024-04-25T10:05:18.257184+0000","last_became_peered":"2024-04-25T10:05:18.257184+0000","last_unstale":"2024-04-25T10:05:19.737376+0000","last_undegraded":"2024-04-25T10:05:19.737376+0000","last_fullsized":"2024-04-25T10:05:19.737376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:21.755389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736580+0000","last_change":"2024-04-25T10:05:18.257479+0000","last_active":"2024-04-25T10:05:19.736580+0000","last_peered":"2024-04-25T10:05:19.736580+0000","last_clean":"2024-04-25T10:05:19.736580+0000","last_became_active":"2024-04-25T10:05:18.257121+0000","last_became_peered":"2024-04-25T10:05:18.257121+0000","last_unstale":"2024-04-25T10:05:19.736580+0000","last_undegraded":"2024-04-25T10:05:19.736580+0000","last_fullsized":"2024-04-25T10:05:19.736580+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:56.867311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733721+0000","last_change":"2024-04-25T10:05:18.258238+0000","last_active":"2024-04-25T10:05:19.733721+0000","last_peered":"2024-04-25T10:05:19.733721+0000","last_clean":"2024-04-25T10:05:19.733721+0000","last_became_active":"2024-04-25T10:05:18.257372+0000","last_became_peered":"2024-04-25T10:05:18.257372+0000","last_unstale":"2024-04-25T10:05:19.733721+0000","last_undegraded":"2024-04-25T10:05:19.733721+0000","last_fullsized":"2024-04-25T10:05:19.733721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:59.029882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632988+0000","last_change":"2024-04-25T10:05:18.269501+0000","last_active":"2024-04-25T10:05:19.632988+0000","last_peered":"2024-04-25T10:05:19.632988+0000","last_clean":"2024-04-25T10:05:19.632988+0000","last_became_active":"2024-04-25T10:05:18.268762+0000","last_became_peered":"2024-04-25T10:05:18.268762+0000","last_unstale":"2024-04-25T10:05:19.632988+0000","last_undegraded":"2024-04-25T10:05:19.632988+0000","last_fullsized":"2024-04-25T10:05:19.632988+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:24.377634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632373+0000","last_change":"2024-04-25T10:05:18.272435+0000","last_active":"2024-04-25T10:05:19.632373+0000","last_peered":"2024-04-25T10:05:19.632373+0000","last_clean":"2024-04-25T10:05:19.632373+0000","last_became_active":"2024-04-25T10:05:18.271638+0000","last_became_peered":"2024-04-25T10:05:18.271638+0000","last_unstale":"2024-04-25T10:05:19.632373+0000","last_undegraded":"2024-04-25T10:05:19.632373+0000","last_fullsized":"2024-04-25T10:05:19.632373+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:45.141237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239633+0000","last_change":"2024-04-25T10:05:18.282530+0000","last_active":"2024-04-25T10:05:19.239633+0000","last_peered":"2024-04-25T10:05:19.239633+0000","last_clean":"2024-04-25T10:05:19.239633+0000","last_became_active":"2024-04-25T10:05:18.280335+0000","last_became_peered":"2024-04-25T10:05:18.280335+0000","last_unstale":"2024-04-25T10:05:19.239633+0000","last_undegraded":"2024-04-25T10:05:19.239633+0000","last_fullsized":"2024-04-25T10:05:19.239633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:49.276161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736478+0000","last_change":"2024-04-25T10:05:18.265097+0000","last_active":"2024-04-25T10:05:19.736478+0000","last_peered":"2024-04-25T10:05:19.736478+0000","last_clean":"2024-04-25T10:05:19.736478+0000","last_became_active":"2024-04-25T10:05:18.264848+0000","last_became_peered":"2024-04-25T10:05:18.264848+0000","last_unstale":"2024-04-25T10:05:19.736478+0000","last_undegraded":"2024-04-25T10:05:19.736478+0000","last_fullsized":"2024-04-25T10:05:19.736478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:09.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734403+0000","last_change":"2024-04-25T10:05:18.286213+0000","last_active":"2024-04-25T10:05:19.734403+0000","last_peered":"2024-04-25T10:05:19.734403+0000","last_clean":"2024-04-25T10:05:19.734403+0000","last_became_active":"2024-04-25T10:05:18.285562+0000","last_became_peered":"2024-04-25T10:05:18.285562+0000","last_unstale":"2024-04-25T10:05:19.734403+0000","last_undegraded":"2024-04-25T10:05:19.734403+0000","last_fullsized":"2024-04-25T10:05:19.734403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:17.659611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632064+0000","last_change":"2024-04-25T10:05:18.262843+0000","last_active":"2024-04-25T10:05:19.632064+0000","last_peered":"2024-04-25T10:05:19.632064+0000","last_clean":"2024-04-25T10:05:19.632064+0000","last_became_active":"2024-04-25T10:05:18.261646+0000","last_became_peered":"2024-04-25T10:05:18.261646+0000","last_unstale":"2024-04-25T10:05:19.632064+0000","last_undegraded":"2024-04-25T10:05:19.632064+0000","last_fullsized":"2024-04-25T10:05:19.632064+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:53.588305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237301+0000","last_change":"2024-04-25T10:05:18.283172+0000","last_active":"2024-04-25T10:05:19.237301+0000","last_peered":"2024-04-25T10:05:19.237301+0000","last_clean":"2024-04-25T10:05:19.237301+0000","last_became_active":"2024-04-25T10:05:18.282668+0000","last_became_peered":"2024-04-25T10:05:18.282668+0000","last_unstale":"2024-04-25T10:05:19.237301+0000","last_undegraded":"2024-04-25T10:05:19.237301+0000","last_fullsized":"2024-04-25T10:05:19.237301+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:34.485681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236895+0000","last_change":"2024-04-25T10:05:18.243140+0000","last_active":"2024-04-25T10:05:19.236895+0000","last_peered":"2024-04-25T10:05:19.236895+0000","last_clean":"2024-04-25T10:05:19.236895+0000","last_became_active":"2024-04-25T10:05:18.242534+0000","last_became_peered":"2024-04-25T10:05:18.242534+0000","last_unstale":"2024-04-25T10:05:19.236895+0000","last_undegraded":"2024-04-25T10:05:19.236895+0000","last_fullsized":"2024-04-25T10:05:19.236895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:47.932432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237863+0000","last_change":"2024-04-25T10:05:18.247126+0000","last_active":"2024-04-25T10:05:19.237863+0000","last_peered":"2024-04-25T10:05:19.237863+0000","last_clean":"2024-04-25T10:05:19.237863+0000","last_became_active":"2024-04-25T10:05:18.246550+0000","last_became_peered":"2024-04-25T10:05:18.246550+0000","last_unstale":"2024-04-25T10:05:19.237863+0000","last_undegraded":"2024-04-25T10:05:19.237863+0000","last_fullsized":"2024-04-25T10:05:19.237863+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:00.139684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"64'4","reported_seq":18,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738858+0000","last_change":"2024-04-25T10:05:17.259528+0000","last_active":"2024-04-25T10:05:19.738858+0000","last_peered":"2024-04-25T10:05:19.738858+0000","last_clean":"2024-04-25T10:05:19.738858+0000","last_became_active":"2024-04-25T10:05:17.259246+0000","last_became_peered":"2024-04-25T10:05:17.259246+0000","last_unstale":"2024-04-25T10:05:19.738858+0000","last_undegraded":"2024-04-25T10:05:19.738858+0000","last_fullsized":"2024-04-25T10:05:19.738858+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:48.265234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239931+0000","last_change":"2024-04-25T10:05:18.243350+0000","last_active":"2024-04-25T10:05:19.239931+0000","last_peered":"2024-04-25T10:05:19.239931+0000","last_clean":"2024-04-25T10:05:19.239931+0000","last_became_active":"2024-04-25T10:05:18.242523+0000","last_became_peered":"2024-04-25T10:05:18.242523+0000","last_unstale":"2024-04-25T10:05:19.239931+0000","last_undegraded":"2024-04-25T10:05:19.239931+0000","last_fullsized":"2024-04-25T10:05:19.239931+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:26.134704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237466+0000","last_change":"2024-04-25T10:05:17.255885+0000","last_active":"2024-04-25T10:05:19.237466+0000","last_peered":"2024-04-25T10:05:19.237466+0000","last_clean":"2024-04-25T10:05:19.237466+0000","last_became_active":"2024-04-25T10:05:17.255484+0000","last_became_peered":"2024-04-25T10:05:17.255484+0000","last_unstale":"2024-04-25T10:05:19.237466+0000","last_undegraded":"2024-04-25T10:05:19.237466+0000","last_fullsized":"2024-04-25T10:05:19.237466+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:52.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237368+0000","last_change":"2024-04-25T10:05:18.243963+0000","last_active":"2024-04-25T10:05:19.237368+0000","last_peered":"2024-04-25T10:05:19.237368+0000","last_clean":"2024-04-25T10:05:19.237368+0000","last_became_active":"2024-04-25T10:05:18.243474+0000","last_became_peered":"2024-04-25T10:05:18.243474+0000","last_unstale":"2024-04-25T10:05:19.237368+0000","last_undegraded":"2024-04-25T10:05:19.237368+0000","last_fullsized":"2024-04-25T10:05:19.237368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:56.867311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237310+0000","last_change":"2024-04-25T10:05:17.247531+0000","last_active":"2024-04-25T10:05:19.237310+0000","last_peered":"2024-04-25T10:05:19.237310+0000","last_clean":"2024-04-25T10:05:19.237310+0000","last_became_active":"2024-04-25T10:05:17.247230+0000","last_became_peered":"2024-04-25T10:05:17.247230+0000","last_unstale":"2024-04-25T10:05:19.237310+0000","last_undegraded":"2024-04-25T10:05:19.237310+0000","last_fullsized":"2024-04-25T10:05:19.237310+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:43.354561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734932+0000","last_change":"2024-04-25T10:05:18.260143+0000","last_active":"2024-04-25T10:05:19.734932+0000","last_peered":"2024-04-25T10:05:19.734932+0000","last_clean":"2024-04-25T10:05:19.734932+0000","last_became_active":"2024-04-25T10:05:18.259329+0000","last_became_peered":"2024-04-25T10:05:18.259329+0000","last_unstale":"2024-04-25T10:05:19.734932+0000","last_undegraded":"2024-04-25T10:05:19.734932+0000","last_fullsized":"2024-04-25T10:05:19.734932+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:44.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237452+0000","last_change":"2024-04-25T10:05:17.231646+0000","last_active":"2024-04-25T10:05:19.237452+0000","last_peered":"2024-04-25T10:05:19.237452+0000","last_clean":"2024-04-25T10:05:19.237452+0000","last_became_active":"2024-04-25T10:05:17.230987+0000","last_became_peered":"2024-04-25T10:05:17.230987+0000","last_unstale":"2024-04-25T10:05:19.237452+0000","last_undegraded":"2024-04-25T10:05:19.237452+0000","last_fullsized":"2024-04-25T10:05:19.237452+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:58.018955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737564+0000","last_change":"2024-04-25T10:05:18.253214+0000","last_active":"2024-04-25T10:05:19.737564+0000","last_peered":"2024-04-25T10:05:19.737564+0000","last_clean":"2024-04-25T10:05:19.737564+0000","last_became_active":"2024-04-25T10:05:18.252886+0000","last_became_peered":"2024-04-25T10:05:18.252886+0000","last_unstale":"2024-04-25T10:05:19.737564+0000","last_undegraded":"2024-04-25T10:05:19.737564+0000","last_fullsized":"2024-04-25T10:05:19.737564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:25.797922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737948+0000","last_change":"2024-04-25T10:05:17.235866+0000","last_active":"2024-04-25T10:05:19.737948+0000","last_peered":"2024-04-25T10:05:19.737948+0000","last_clean":"2024-04-25T10:05:19.737948+0000","last_became_active":"2024-04-25T10:05:17.235469+0000","last_became_peered":"2024-04-25T10:05:17.235469+0000","last_unstale":"2024-04-25T10:05:19.737948+0000","last_undegraded":"2024-04-25T10:05:19.737948+0000","last_fullsized":"2024-04-25T10:05:19.737948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:10.616020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738752+0000","last_change":"2024-04-25T10:05:18.280259+0000","last_active":"2024-04-25T10:05:19.738752+0000","last_peered":"2024-04-25T10:05:19.738752+0000","last_clean":"2024-04-25T10:05:19.738752+0000","last_became_active":"2024-04-25T10:05:18.279957+0000","last_became_peered":"2024-04-25T10:05:18.279957+0000","last_unstale":"2024-04-25T10:05:19.738752+0000","last_undegraded":"2024-04-25T10:05:19.738752+0000","last_fullsized":"2024-04-25T10:05:19.738752+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:26.621379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737652+0000","last_change":"2024-04-25T10:05:17.238014+0000","last_active":"2024-04-25T10:05:19.737652+0000","last_peered":"2024-04-25T10:05:19.737652+0000","last_clean":"2024-04-25T10:05:19.737652+0000","last_became_active":"2024-04-25T10:05:17.236728+0000","last_became_peered":"2024-04-25T10:05:17.236728+0000","last_unstale":"2024-04-25T10:05:19.737652+0000","last_undegraded":"2024-04-25T10:05:19.737652+0000","last_fullsized":"2024-04-25T10:05:19.737652+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.844495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236560+0000","last_change":"2024-04-25T10:05:18.250458+0000","last_active":"2024-04-25T10:05:19.236560+0000","last_peered":"2024-04-25T10:05:19.236560+0000","last_clean":"2024-04-25T10:05:19.236560+0000","last_became_active":"2024-04-25T10:05:18.249706+0000","last_became_peered":"2024-04-25T10:05:18.249706+0000","last_unstale":"2024-04-25T10:05:19.236560+0000","last_undegraded":"2024-04-25T10:05:19.236560+0000","last_fullsized":"2024-04-25T10:05:19.236560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.017832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736821+0000","last_change":"2024-04-25T10:05:17.245582+0000","last_active":"2024-04-25T10:05:19.736821+0000","last_peered":"2024-04-25T10:05:19.736821+0000","last_clean":"2024-04-25T10:05:19.736821+0000","last_became_active":"2024-04-25T10:05:17.245239+0000","last_became_peered":"2024-04-25T10:05:17.245239+0000","last_unstale":"2024-04-25T10:05:19.736821+0000","last_undegraded":"2024-04-25T10:05:19.736821+0000","last_fullsized":"2024-04-25T10:05:19.736821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:58.018955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737808+0000","last_change":"2024-04-25T10:05:18.233488+0000","last_active":"2024-04-25T10:05:19.737808+0000","last_peered":"2024-04-25T10:05:19.737808+0000","last_clean":"2024-04-25T10:05:19.737808+0000","last_became_active":"2024-04-25T10:05:18.232718+0000","last_became_peered":"2024-04-25T10:05:18.232718+0000","last_unstale":"2024-04-25T10:05:19.737808+0000","last_undegraded":"2024-04-25T10:05:19.737808+0000","last_fullsized":"2024-04-25T10:05:19.737808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:28.734876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236609+0000","last_change":"2024-04-25T10:05:17.252617+0000","last_active":"2024-04-25T10:05:19.236609+0000","last_peered":"2024-04-25T10:05:19.236609+0000","last_clean":"2024-04-25T10:05:19.236609+0000","last_became_active":"2024-04-25T10:05:17.251875+0000","last_became_peered":"2024-04-25T10:05:17.251875+0000","last_unstale":"2024-04-25T10:05:19.236609+0000","last_undegraded":"2024-04-25T10:05:19.236609+0000","last_fullsized":"2024-04-25T10:05:19.236609+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:29:06.492874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736861+0000","last_change":"2024-04-25T10:05:18.244742+0000","last_active":"2024-04-25T10:05:19.736861+0000","last_peered":"2024-04-25T10:05:19.736861+0000","last_clean":"2024-04-25T10:05:19.736861+0000","last_became_active":"2024-04-25T10:05:18.244049+0000","last_became_peered":"2024-04-25T10:05:18.244049+0000","last_unstale":"2024-04-25T10:05:19.736861+0000","last_undegraded":"2024-04-25T10:05:19.736861+0000","last_fullsized":"2024-04-25T10:05:19.736861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:43.009797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735154+0000","last_change":"2024-04-25T10:05:17.262111+0000","last_active":"2024-04-25T10:05:19.735154+0000","last_peered":"2024-04-25T10:05:19.735154+0000","last_clean":"2024-04-25T10:05:19.735154+0000","last_became_active":"2024-04-25T10:05:17.261418+0000","last_became_peered":"2024-04-25T10:05:17.261418+0000","last_unstale":"2024-04-25T10:05:19.735154+0000","last_undegraded":"2024-04-25T10:05:19.735154+0000","last_fullsized":"2024-04-25T10:05:19.735154+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:11.018187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.738325+0000","last_change":"2024-04-25T10:05:18.251978+0000","last_active":"2024-04-25T10:05:19.738325+0000","last_peered":"2024-04-25T10:05:19.738325+0000","last_clean":"2024-04-25T10:05:19.738325+0000","last_became_active":"2024-04-25T10:05:18.251649+0000","last_became_peered":"2024-04-25T10:05:18.251649+0000","last_unstale":"2024-04-25T10:05:19.738325+0000","last_undegraded":"2024-04-25T10:05:19.738325+0000","last_fullsized":"2024-04-25T10:05:19.738325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:39.807129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.240000+0000","last_change":"2024-04-25T10:05:17.235813+0000","last_active":"2024-04-25T10:05:19.240000+0000","last_peered":"2024-04-25T10:05:19.240000+0000","last_clean":"2024-04-25T10:05:19.240000+0000","last_became_active":"2024-04-25T10:05:17.234668+0000","last_became_peered":"2024-04-25T10:05:17.234668+0000","last_unstale":"2024-04-25T10:05:19.240000+0000","last_undegraded":"2024-04-25T10:05:19.240000+0000","last_fullsized":"2024-04-25T10:05:19.240000+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.250690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237726+0000","last_change":"2024-04-25T10:05:18.237283+0000","last_active":"2024-04-25T10:05:19.237726+0000","last_peered":"2024-04-25T10:05:19.237726+0000","last_clean":"2024-04-25T10:05:19.237726+0000","last_became_active":"2024-04-25T10:05:18.236564+0000","last_became_peered":"2024-04-25T10:05:18.236564+0000","last_unstale":"2024-04-25T10:05:19.237726+0000","last_undegraded":"2024-04-25T10:05:19.237726+0000","last_fullsized":"2024-04-25T10:05:19.237726+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:57.541006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"64'3","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734972+0000","last_change":"2024-04-25T10:05:17.253324+0000","last_active":"2024-04-25T10:05:19.734972+0000","last_peered":"2024-04-25T10:05:19.734972+0000","last_clean":"2024-04-25T10:05:19.734972+0000","last_became_active":"2024-04-25T10:05:17.252849+0000","last_became_peered":"2024-04-25T10:05:17.252849+0000","last_unstale":"2024-04-25T10:05:19.734972+0000","last_undegraded":"2024-04-25T10:05:19.734972+0000","last_fullsized":"2024-04-25T10:05:19.734972+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.476085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.240140+0000","last_change":"2024-04-25T10:05:18.254997+0000","last_active":"2024-04-25T10:05:19.240140+0000","last_peered":"2024-04-25T10:05:19.240140+0000","last_clean":"2024-04-25T10:05:19.240140+0000","last_became_active":"2024-04-25T10:05:18.242307+0000","last_became_peered":"2024-04-25T10:05:18.242307+0000","last_unstale":"2024-04-25T10:05:19.240140+0000","last_undegraded":"2024-04-25T10:05:19.240140+0000","last_fullsized":"2024-04-25T10:05:19.240140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:57.541006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"64'6","reported_seq":20,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737160+0000","last_change":"2024-04-25T10:05:17.246675+0000","last_active":"2024-04-25T10:05:19.737160+0000","last_peered":"2024-04-25T10:05:19.737160+0000","last_clean":"2024-04-25T10:05:19.737160+0000","last_became_active":"2024-04-25T10:05:17.246363+0000","last_became_peered":"2024-04-25T10:05:17.246363+0000","last_unstale":"2024-04-25T10:05:19.737160+0000","last_undegraded":"2024-04-25T10:05:19.737160+0000","last_fullsized":"2024-04-25T10:05:19.737160+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:15.643774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239256+0000","last_change":"2024-04-25T10:05:18.283608+0000","last_active":"2024-04-25T10:05:19.239256+0000","last_peered":"2024-04-25T10:05:19.239256+0000","last_clean":"2024-04-25T10:05:19.239256+0000","last_became_active":"2024-04-25T10:05:18.283077+0000","last_became_peered":"2024-04-25T10:05:18.283077+0000","last_unstale":"2024-04-25T10:05:19.239256+0000","last_undegraded":"2024-04-25T10:05:19.239256+0000","last_fullsized":"2024-04-25T10:05:19.239256+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:16.654701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734317+0000","last_change":"2024-04-25T10:05:17.257105+0000","last_active":"2024-04-25T10:05:19.734317+0000","last_peered":"2024-04-25T10:05:19.734317+0000","last_clean":"2024-04-25T10:05:19.734317+0000","last_became_active":"2024-04-25T10:05:17.256415+0000","last_became_peered":"2024-04-25T10:05:17.256415+0000","last_unstale":"2024-04-25T10:05:19.734317+0000","last_undegraded":"2024-04-25T10:05:19.734317+0000","last_fullsized":"2024-04-25T10:05:19.734317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:59.128757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632564+0000","last_change":"2024-04-25T10:05:18.248968+0000","last_active":"2024-04-25T10:05:19.632564+0000","last_peered":"2024-04-25T10:05:19.632564+0000","last_clean":"2024-04-25T10:05:19.632564+0000","last_became_active":"2024-04-25T10:05:18.248218+0000","last_became_peered":"2024-04-25T10:05:18.248218+0000","last_unstale":"2024-04-25T10:05:19.632564+0000","last_undegraded":"2024-04-25T10:05:19.632564+0000","last_fullsized":"2024-04-25T10:05:19.632564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.631996+0000","last_change":"2024-04-25T10:05:17.247771+0000","last_active":"2024-04-25T10:05:19.631996+0000","last_peered":"2024-04-25T10:05:19.631996+0000","last_clean":"2024-04-25T10:05:19.631996+0000","last_became_active":"2024-04-25T10:05:17.245928+0000","last_became_peered":"2024-04-25T10:05:17.245928+0000","last_unstale":"2024-04-25T10:05:19.631996+0000","last_undegraded":"2024-04-25T10:05:19.631996+0000","last_fullsized":"2024-04-25T10:05:19.631996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:44.476085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236942+0000","last_change":"2024-04-25T10:05:18.244705+0000","last_active":"2024-04-25T10:05:19.236942+0000","last_peered":"2024-04-25T10:05:19.236942+0000","last_clean":"2024-04-25T10:05:19.236942+0000","last_became_active":"2024-04-25T10:05:18.238952+0000","last_became_peered":"2024-04-25T10:05:18.238952+0000","last_unstale":"2024-04-25T10:05:19.236942+0000","last_undegraded":"2024-04-25T10:05:19.236942+0000","last_fullsized":"2024-04-25T10:05:19.236942+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.874237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734242+0000","last_change":"2024-04-25T10:05:17.255176+0000","last_active":"2024-04-25T10:05:19.734242+0000","last_peered":"2024-04-25T10:05:19.734242+0000","last_clean":"2024-04-25T10:05:19.734242+0000","last_became_active":"2024-04-25T10:05:17.254374+0000","last_became_peered":"2024-04-25T10:05:17.254374+0000","last_unstale":"2024-04-25T10:05:19.734242+0000","last_undegraded":"2024-04-25T10:05:19.734242+0000","last_fullsized":"2024-04-25T10:05:19.734242+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:24.655678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736296+0000","last_change":"2024-04-25T10:05:18.260223+0000","last_active":"2024-04-25T10:05:19.736296+0000","last_peered":"2024-04-25T10:05:19.736296+0000","last_clean":"2024-04-25T10:05:19.736296+0000","last_became_active":"2024-04-25T10:05:18.257535+0000","last_became_peered":"2024-04-25T10:05:18.257535+0000","last_unstale":"2024-04-25T10:05:19.736296+0000","last_undegraded":"2024-04-25T10:05:19.736296+0000","last_fullsized":"2024-04-25T10:05:19.736296+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:45.487013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"64'8","reported_seq":22,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238996+0000","last_change":"2024-04-25T10:05:17.254350+0000","last_active":"2024-04-25T10:05:19.238996+0000","last_peered":"2024-04-25T10:05:19.238996+0000","last_clean":"2024-04-25T10:05:19.238996+0000","last_became_active":"2024-04-25T10:05:17.253928+0000","last_became_peered":"2024-04-25T10:05:17.253928+0000","last_unstale":"2024-04-25T10:05:19.238996+0000","last_undegraded":"2024-04-25T10:05:19.238996+0000","last_fullsized":"2024-04-25T10:05:19.238996+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:43.740109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734546+0000","last_change":"2024-04-25T10:05:18.286163+0000","last_active":"2024-04-25T10:05:19.734546+0000","last_peered":"2024-04-25T10:05:19.734546+0000","last_clean":"2024-04-25T10:05:19.734546+0000","last_became_active":"2024-04-25T10:05:18.285584+0000","last_became_peered":"2024-04-25T10:05:18.285584+0000","last_unstale":"2024-04-25T10:05:19.734546+0000","last_undegraded":"2024-04-25T10:05:19.734546+0000","last_fullsized":"2024-04-25T10:05:19.734546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:45.141237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632737+0000","last_change":"2024-04-25T10:05:17.243578+0000","last_active":"2024-04-25T10:05:19.632737+0000","last_peered":"2024-04-25T10:05:19.632737+0000","last_clean":"2024-04-25T10:05:19.632737+0000","last_became_active":"2024-04-25T10:05:17.242848+0000","last_became_peered":"2024-04-25T10:05:17.242848+0000","last_unstale":"2024-04-25T10:05:19.632737+0000","last_undegraded":"2024-04-25T10:05:19.632737+0000","last_fullsized":"2024-04-25T10:05:19.632737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:24.655678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238082+0000","last_change":"2024-04-25T10:05:18.259011+0000","last_active":"2024-04-25T10:05:19.238082+0000","last_peered":"2024-04-25T10:05:19.238082+0000","last_clean":"2024-04-25T10:05:19.238082+0000","last_became_active":"2024-04-25T10:05:18.258199+0000","last_became_peered":"2024-04-25T10:05:18.258199+0000","last_unstale":"2024-04-25T10:05:19.238082+0000","last_undegraded":"2024-04-25T10:05:19.238082+0000","last_fullsized":"2024-04-25T10:05:19.238082+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:54.337634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734922+0000","last_change":"2024-04-25T10:05:17.255726+0000","last_active":"2024-04-25T10:05:19.734922+0000","last_peered":"2024-04-25T10:05:19.734922+0000","last_clean":"2024-04-25T10:05:19.734922+0000","last_became_active":"2024-04-25T10:05:17.255193+0000","last_became_peered":"2024-04-25T10:05:17.255193+0000","last_unstale":"2024-04-25T10:05:19.734922+0000","last_undegraded":"2024-04-25T10:05:19.734922+0000","last_fullsized":"2024-04-25T10:05:19.734922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.998869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.239910+0000","last_change":"2024-04-25T10:05:18.255531+0000","last_active":"2024-04-25T10:05:19.239910+0000","last_peered":"2024-04-25T10:05:19.239910+0000","last_clean":"2024-04-25T10:05:19.239910+0000","last_became_active":"2024-04-25T10:05:18.243046+0000","last_became_peered":"2024-04-25T10:05:18.243046+0000","last_unstale":"2024-04-25T10:05:19.239910+0000","last_undegraded":"2024-04-25T10:05:19.239910+0000","last_fullsized":"2024-04-25T10:05:19.239910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:57.874237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"64'1","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735044+0000","last_change":"2024-04-25T10:05:17.255681+0000","last_active":"2024-04-25T10:05:19.735044+0000","last_peered":"2024-04-25T10:05:19.735044+0000","last_clean":"2024-04-25T10:05:19.735044+0000","last_became_active":"2024-04-25T10:05:17.254926+0000","last_became_peered":"2024-04-25T10:05:17.254926+0000","last_unstale":"2024-04-25T10:05:19.735044+0000","last_undegraded":"2024-04-25T10:05:19.735044+0000","last_fullsized":"2024-04-25T10:05:19.735044+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:08.342799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237849+0000","last_change":"2024-04-25T10:05:18.235488+0000","last_active":"2024-04-25T10:05:19.237849+0000","last_peered":"2024-04-25T10:05:19.237849+0000","last_clean":"2024-04-25T10:05:19.237849+0000","last_became_active":"2024-04-25T10:05:18.234806+0000","last_became_peered":"2024-04-25T10:05:18.234806+0000","last_unstale":"2024-04-25T10:05:19.237849+0000","last_undegraded":"2024-04-25T10:05:19.237849+0000","last_fullsized":"2024-04-25T10:05:19.237849+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:27:21.557108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.739217+0000","last_change":"2024-04-25T10:05:17.247259+0000","last_active":"2024-04-25T10:05:19.739217+0000","last_peered":"2024-04-25T10:05:19.739217+0000","last_clean":"2024-04-25T10:05:19.739217+0000","last_became_active":"2024-04-25T10:05:17.246554+0000","last_became_peered":"2024-04-25T10:05:17.246554+0000","last_unstale":"2024-04-25T10:05:19.739217+0000","last_undegraded":"2024-04-25T10:05:19.739217+0000","last_fullsized":"2024-04-25T10:05:19.739217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:52.577377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236887+0000","last_change":"2024-04-25T10:05:18.242056+0000","last_active":"2024-04-25T10:05:19.236887+0000","last_peered":"2024-04-25T10:05:19.236887+0000","last_clean":"2024-04-25T10:05:19.236887+0000","last_became_active":"2024-04-25T10:05:18.241680+0000","last_became_peered":"2024-04-25T10:05:18.241680+0000","last_unstale":"2024-04-25T10:05:19.236887+0000","last_undegraded":"2024-04-25T10:05:19.236887+0000","last_fullsized":"2024-04-25T10:05:19.236887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:43.703262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237253+0000","last_change":"2024-04-25T10:05:17.241034+0000","last_active":"2024-04-25T10:05:19.237253+0000","last_peered":"2024-04-25T10:05:19.237253+0000","last_clean":"2024-04-25T10:05:19.237253+0000","last_became_active":"2024-04-25T10:05:17.240667+0000","last_became_peered":"2024-04-25T10:05:17.240667+0000","last_unstale":"2024-04-25T10:05:19.237253+0000","last_undegraded":"2024-04-25T10:05:19.237253+0000","last_fullsized":"2024-04-25T10:05:19.237253+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:29.807096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632884+0000","last_change":"2024-04-25T10:05:18.277623+0000","last_active":"2024-04-25T10:05:19.632884+0000","last_peered":"2024-04-25T10:05:19.632884+0000","last_clean":"2024-04-25T10:05:19.632884+0000","last_became_active":"2024-04-25T10:05:18.276877+0000","last_became_peered":"2024-04-25T10:05:18.276877+0000","last_unstale":"2024-04-25T10:05:19.632884+0000","last_undegraded":"2024-04-25T10:05:19.632884+0000","last_fullsized":"2024-04-25T10:05:19.632884+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:17.659611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632258+0000","last_change":"2024-04-25T10:05:17.248585+0000","last_active":"2024-04-25T10:05:19.632258+0000","last_peered":"2024-04-25T10:05:19.632258+0000","last_clean":"2024-04-25T10:05:19.632258+0000","last_became_active":"2024-04-25T10:05:17.247080+0000","last_became_peered":"2024-04-25T10:05:17.247080+0000","last_unstale":"2024-04-25T10:05:19.632258+0000","last_undegraded":"2024-04-25T10:05:19.632258+0000","last_fullsized":"2024-04-25T10:05:19.632258+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:08.342799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237014+0000","last_change":"2024-04-25T10:05:18.251129+0000","last_active":"2024-04-25T10:05:19.237014+0000","last_peered":"2024-04-25T10:05:19.237014+0000","last_clean":"2024-04-25T10:05:19.237014+0000","last_became_active":"2024-04-25T10:05:18.249949+0000","last_became_peered":"2024-04-25T10:05:18.249949+0000","last_unstale":"2024-04-25T10:05:19.237014+0000","last_undegraded":"2024-04-25T10:05:19.237014+0000","last_fullsized":"2024-04-25T10:05:19.237014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:26.134704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236931+0000","last_change":"2024-04-25T10:05:17.249980+0000","last_active":"2024-04-25T10:05:19.236931+0000","last_peered":"2024-04-25T10:05:19.236931+0000","last_clean":"2024-04-25T10:05:19.236931+0000","last_became_active":"2024-04-25T10:05:17.249129+0000","last_became_peered":"2024-04-25T10:05:17.249129+0000","last_unstale":"2024-04-25T10:05:19.236931+0000","last_undegraded":"2024-04-25T10:05:19.236931+0000","last_fullsized":"2024-04-25T10:05:19.236931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:31.306889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237173+0000","last_change":"2024-04-25T10:05:18.258349+0000","last_active":"2024-04-25T10:05:19.237173+0000","last_peered":"2024-04-25T10:05:19.237173+0000","last_clean":"2024-04-25T10:05:19.237173+0000","last_became_active":"2024-04-25T10:05:18.258058+0000","last_became_peered":"2024-04-25T10:05:18.258058+0000","last_unstale":"2024-04-25T10:05:19.237173+0000","last_undegraded":"2024-04-25T10:05:19.237173+0000","last_fullsized":"2024-04-25T10:05:19.237173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:41.414208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"64'2","reported_seq":16,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.633007+0000","last_change":"2024-04-25T10:05:17.256923+0000","last_active":"2024-04-25T10:05:19.633007+0000","last_peered":"2024-04-25T10:05:19.633007+0000","last_clean":"2024-04-25T10:05:19.633007+0000","last_became_active":"2024-04-25T10:05:17.256526+0000","last_became_peered":"2024-04-25T10:05:17.256526+0000","last_unstale":"2024-04-25T10:05:19.633007+0000","last_undegraded":"2024-04-25T10:05:19.633007+0000","last_fullsized":"2024-04-25T10:05:19.633007+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:41.998869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737283+0000","last_change":"2024-04-25T10:05:18.243267+0000","last_active":"2024-04-25T10:05:19.737283+0000","last_peered":"2024-04-25T10:05:19.737283+0000","last_clean":"2024-04-25T10:05:19.737283+0000","last_became_active":"2024-04-25T10:05:18.242959+0000","last_became_peered":"2024-04-25T10:05:18.242959+0000","last_unstale":"2024-04-25T10:05:19.737283+0000","last_undegraded":"2024-04-25T10:05:19.737283+0000","last_fullsized":"2024-04-25T10:05:19.737283+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:14.499355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632806+0000","last_change":"2024-04-25T10:05:17.253283+0000","last_active":"2024-04-25T10:05:19.632806+0000","last_peered":"2024-04-25T10:05:19.632806+0000","last_clean":"2024-04-25T10:05:19.632806+0000","last_became_active":"2024-04-25T10:05:17.252104+0000","last_became_peered":"2024-04-25T10:05:17.252104+0000","last_unstale":"2024-04-25T10:05:19.632806+0000","last_undegraded":"2024-04-25T10:05:19.632806+0000","last_fullsized":"2024-04-25T10:05:19.632806+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:11.018187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.632579+0000","last_change":"2024-04-25T10:05:18.249132+0000","last_active":"2024-04-25T10:05:19.632579+0000","last_peered":"2024-04-25T10:05:19.632579+0000","last_clean":"2024-04-25T10:05:19.632579+0000","last_became_active":"2024-04-25T10:05:18.248355+0000","last_became_peered":"2024-04-25T10:05:18.248355+0000","last_unstale":"2024-04-25T10:05:19.632579+0000","last_undegraded":"2024-04-25T10:05:19.632579+0000","last_fullsized":"2024-04-25T10:05:19.632579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:44.627978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237383+0000","last_change":"2024-04-25T10:05:17.250923+0000","last_active":"2024-04-25T10:05:19.237383+0000","last_peered":"2024-04-25T10:05:19.237383+0000","last_clean":"2024-04-25T10:05:19.237383+0000","last_became_active":"2024-04-25T10:05:17.250529+0000","last_became_peered":"2024-04-25T10:05:17.250529+0000","last_unstale":"2024-04-25T10:05:19.237383+0000","last_undegraded":"2024-04-25T10:05:19.237383+0000","last_fullsized":"2024-04-25T10:05:19.237383+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:26.844495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238274+0000","last_change":"2024-04-25T10:05:18.236065+0000","last_active":"2024-04-25T10:05:19.238274+0000","last_peered":"2024-04-25T10:05:19.238274+0000","last_clean":"2024-04-25T10:05:19.238274+0000","last_became_active":"2024-04-25T10:05:18.233432+0000","last_became_peered":"2024-04-25T10:05:18.233432+0000","last_unstale":"2024-04-25T10:05:19.238274+0000","last_undegraded":"2024-04-25T10:05:19.238274+0000","last_fullsized":"2024-04-25T10:05:19.238274+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:09.353727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737264+0000","last_change":"2024-04-25T10:05:17.247680+0000","last_active":"2024-04-25T10:05:19.737264+0000","last_peered":"2024-04-25T10:05:19.737264+0000","last_clean":"2024-04-25T10:05:19.737264+0000","last_became_active":"2024-04-25T10:05:17.246905+0000","last_became_peered":"2024-04-25T10:05:17.246905+0000","last_unstale":"2024-04-25T10:05:19.737264+0000","last_undegraded":"2024-04-25T10:05:19.737264+0000","last_fullsized":"2024-04-25T10:05:19.737264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:15.675659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.236344+0000","last_change":"2024-04-25T10:05:18.258330+0000","last_active":"2024-04-25T10:05:19.236344+0000","last_peered":"2024-04-25T10:05:19.236344+0000","last_clean":"2024-04-25T10:05:19.236344+0000","last_became_active":"2024-04-25T10:05:18.256736+0000","last_became_peered":"2024-04-25T10:05:18.256736+0000","last_unstale":"2024-04-25T10:05:19.236344+0000","last_undegraded":"2024-04-25T10:05:19.236344+0000","last_fullsized":"2024-04-25T10:05:19.236344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:49.276161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.734636+0000","last_change":"2024-04-25T10:05:17.261364+0000","last_active":"2024-04-25T10:05:19.734636+0000","last_peered":"2024-04-25T10:05:19.734636+0000","last_clean":"2024-04-25T10:05:19.734636+0000","last_became_active":"2024-04-25T10:05:17.257835+0000","last_became_peered":"2024-04-25T10:05:17.257835+0000","last_unstale":"2024-04-25T10:05:19.734636+0000","last_undegraded":"2024-04-25T10:05:19.734636+0000","last_fullsized":"2024-04-25T10:05:19.734636+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:53.326707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.237695+0000","last_change":"2024-04-25T10:05:18.270656+0000","last_active":"2024-04-25T10:05:19.237695+0000","last_peered":"2024-04-25T10:05:19.237695+0000","last_clean":"2024-04-25T10:05:19.237695+0000","last_became_active":"2024-04-25T10:05:18.269902+0000","last_became_peered":"2024-04-25T10:05:18.269902+0000","last_unstale":"2024-04-25T10:05:19.237695+0000","last_undegraded":"2024-04-25T10:05:19.237695+0000","last_fullsized":"2024-04-25T10:05:19.237695+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:35.238488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737675+0000","last_change":"2024-04-25T10:05:17.237915+0000","last_active":"2024-04-25T10:05:19.737675+0000","last_peered":"2024-04-25T10:05:19.737675+0000","last_clean":"2024-04-25T10:05:19.737675+0000","last_became_active":"2024-04-25T10:05:17.237528+0000","last_became_peered":"2024-04-25T10:05:17.237528+0000","last_unstale":"2024-04-25T10:05:19.737675+0000","last_undegraded":"2024-04-25T10:05:19.737675+0000","last_fullsized":"2024-04-25T10:05:19.737675+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:44.130310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736836+0000","last_change":"2024-04-25T10:05:18.280432+0000","last_active":"2024-04-25T10:05:19.736836+0000","last_peered":"2024-04-25T10:05:19.736836+0000","last_clean":"2024-04-25T10:05:19.736836+0000","last_became_active":"2024-04-25T10:05:18.280122+0000","last_became_peered":"2024-04-25T10:05:18.280122+0000","last_unstale":"2024-04-25T10:05:19.736836+0000","last_undegraded":"2024-04-25T10:05:19.736836+0000","last_fullsized":"2024-04-25T10:05:19.736836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.666605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737057+0000","last_change":"2024-04-25T10:05:17.245484+0000","last_active":"2024-04-25T10:05:19.737057+0000","last_peered":"2024-04-25T10:05:19.737057+0000","last_clean":"2024-04-25T10:05:19.737057+0000","last_became_active":"2024-04-25T10:05:17.245096+0000","last_became_peered":"2024-04-25T10:05:17.245096+0000","last_unstale":"2024-04-25T10:05:19.737057+0000","last_undegraded":"2024-04-25T10:05:19.737057+0000","last_fullsized":"2024-04-25T10:05:19.737057+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:16.648683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.736442+0000","last_change":"2024-04-25T10:05:18.253458+0000","last_active":"2024-04-25T10:05:19.736442+0000","last_peered":"2024-04-25T10:05:19.736442+0000","last_clean":"2024-04-25T10:05:19.736442+0000","last_became_active":"2024-04-25T10:05:18.253127+0000","last_became_peered":"2024-04-25T10:05:18.253127+0000","last_unstale":"2024-04-25T10:05:19.736442+0000","last_undegraded":"2024-04-25T10:05:19.736442+0000","last_fullsized":"2024-04-25T10:05:19.736442+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:31.237519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.735655+0000","last_change":"2024-04-25T10:05:17.245484+0000","last_active":"2024-04-25T10:05:19.735655+0000","last_peered":"2024-04-25T10:05:19.735655+0000","last_clean":"2024-04-25T10:05:19.735655+0000","last_became_active":"2024-04-25T10:05:17.245094+0000","last_became_peered":"2024-04-25T10:05:17.245094+0000","last_unstale":"2024-04-25T10:05:19.735655+0000","last_undegraded":"2024-04-25T10:05:19.735655+0000","last_fullsized":"2024-04-25T10:05:19.735655+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:34.227561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.238692+0000","last_change":"2024-04-25T10:05:18.257706+0000","last_active":"2024-04-25T10:05:19.238692+0000","last_peered":"2024-04-25T10:05:19.238692+0000","last_clean":"2024-04-25T10:05:19.238692+0000","last_became_active":"2024-04-25T10:05:18.256153+0000","last_became_peered":"2024-04-25T10:05:18.256153+0000","last_unstale":"2024-04-25T10:05:19.238692+0000","last_undegraded":"2024-04-25T10:05:19.238692+0000","last_fullsized":"2024-04-25T10:05:19.238692+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:17.199846+0000","last_clean_scrub_stamp":"2024-04-25T10:05:17.199846+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:25.666605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.737493+0000","last_change":"2024-04-25T10:05:17.235903+0000","last_active":"2024-04-25T10:05:19.737493+0000","last_peered":"2024-04-25T10:05:19.737493+0000","last_clean":"2024-04-25T10:05:19.737493+0000","last_became_active":"2024-04-25T10:05:17.235503+0000","last_became_peered":"2024-04-25T10:05:17.235503+0000","last_unstale":"2024-04-25T10:05:19.737493+0000","last_undegraded":"2024-04-25T10:05:19.737493+0000","last_fullsized":"2024-04-25T10:05:19.737493+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:05:16.188919+0000","last_clean_scrub_stamp":"2024-04-25T10:05:16.188919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:31.306889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":254,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-04-25T10:05:19.733791+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:05:19.733791+0000","last_peered":"2024-04-25T10:05:19.733791+0000","last_clean":"2024-04-25T10:05:19.733791+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:05:19.733791+0000","last_undegraded":"2024-04-25T10:05:19.733791+0000","last_fullsized":"2024-04-25T10:05:19.733791+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574892,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1384448,"data_stored":940121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":355703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":778240,"data_stored":348999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27800,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344040,"statfs":{"total":96636764160,"available":96608296960,"internally_reserved":0,"allocated":798720,"data_stored":357281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31944,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339896,"statfs":{"total":96636764160,"available":96604053504,"internally_reserved":0,"allocated":774144,"data_stored":348977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31940,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339900,"statfs":{"total":96636764160,"available":96604057600,"internally_reserved":0,"allocated":774144,"data_stored":348977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31972,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339868,"statfs":{"total":96636764160,"available":96604024832,"internally_reserved":0,"allocated":802816,"data_stored":351353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574891,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1368064,"data_stored":939345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:05:22.587 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:05:22.587 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:05:22.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 Processor -- start 2024-04-25T10:05:22.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 -- start start 2024-04-25T10:05:22.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d3015ecb0 0x7f8d300c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d300c9ef0 0x7f8d300ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d300ca4c0 con 0x7f8d300cf590 2024-04-25T10:05:22.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d300ca780 con 0x7f8d3015ecb0 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d36609640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3015f0f0 con 0x7f8d300c9ef0 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d3015ecb0 0x7f8d300c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d3015ecb0 0x7f8d300c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:53666/0 (socket says 172.21.15.82:53666) 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/720877188 learned_addr learned my addr 172.21.15.82:0/720877188 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d34ba6640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d300c9ef0 0x7f8d300ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d300c9ef0 msgr2=0x7f8d300ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d300c9ef0 0x7f8d300ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/720877188 >> v1:172.21.15.196:6789/0 conn(0x7f8d300cf590 legacy=0x7f8d300cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.758+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/720877188 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d300cd690 con 0x7f8d3015ecb0 2024-04-25T10:05:22.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d2f7fe640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d3015ecb0 0x7f8d300c99b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000bb10 tx=0x7f8d2001ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9abbfa3b6b6531f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d2effd640 1 -- 172.21.15.82:0/720877188 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200b4070 con 0x7f8d3015ecb0 2024-04-25T10:05:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d2effd640 1 -- 172.21.15.82:0/720877188 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d200a7c10 con 0x7f8d3015ecb0 2024-04-25T10:05:22.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d2effd640 1 -- 172.21.15.82:0/720877188 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200af440 con 0x7f8d3015ecb0 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 -- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d3015ecb0 msgr2=0x7f8d300c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d3015ecb0 0x7f8d300c99b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000bb10 tx=0x7f8d2001ec30 comp rx=0 tx=0).stop 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 -- 172.21.15.82:0/720877188 shutdown_connections 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d300c9ef0 0x7f8d300ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 --2- 172.21.15.82:0/720877188 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d3015ecb0 0x7f8d300c99b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 -- 172.21.15.82:0/720877188 >> 172.21.15.82:0/720877188 conn(0x7f8d3006ae60 msgr2=0x7f8d3006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 -- 172.21.15.82:0/720877188 shutdown_connections 2024-04-25T10:05:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.762+0000 7f8d36609640 1 -- 172.21.15.82:0/720877188 wait complete. 2024-04-25T10:05:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 Processor -- start 2024-04-25T10:05:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- start start 2024-04-25T10:05:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d300c9ef0 0x7f8d3015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d3015ecb0 0x7f8d30153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300cc1b0 con 0x7f8d300cf590 2024-04-25T10:05:22.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300cc030 con 0x7f8d300c9ef0 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d300cc330 con 0x7f8d3015ecb0 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44326/0 (socket says 172.21.15.82:44326) 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/742463097 learned_addr learned my addr 172.21.15.82:0/742463097 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d3015ecb0 msgr2=0x7f8d30153300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d3015ecb0 0x7f8d30153300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d300c9ef0 msgr2=0x7f8d3015ab40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d300c9ef0 0x7f8d3015ab40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 -- 172.21.15.82:0/742463097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3015b250 con 0x7f8d300cf590 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2f7fe640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8d2001d2e0 tx=0x7f8d2001b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200b4070 con 0x7f8d300cf590 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d200b6070 con 0x7f8d300cf590 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3015b530 con 0x7f8d300cf590 2024-04-25T10:05:22.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d200bb470 con 0x7f8d300cf590 2024-04-25T10:05:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d3015baf0 con 0x7f8d300cf590 2024-04-25T10:05:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d3004efa0 con 0x7f8d300cf590 2024-04-25T10:05:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.766+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d200bb610 con 0x7f8d300cf590 2024-04-25T10:05:22.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.770+0000 7f8d2cff9640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.770+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d2011f6e0 con 0x7f8d300cf590 2024-04-25T10:05:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.770+0000 7f8d2ffff640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:22.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.770+0000 7f8d2ffff640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f70 tx=0x7f8d180079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:22.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.774+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d20018820 con 0x7f8d300cf590 2024-04-25T10:05:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.990+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8d300b4170 con 0x7f8d300cf590 2024-04-25T10:05:22.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.994+0000 7f8d2cff9640 1 -- 172.21.15.82:0/742463097 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8d200f0680 con 0x7f8d300cf590 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:05:22.999 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 msgr2=0x7f8d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 0x7f8d10069420 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8d18000f70 tx=0x7f8d180079c0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 msgr2=0x7f8d30098680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8d2001d2e0 tx=0x7f8d2001b9d0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 shutdown_connections 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d10066f60 0x7f8d10069420 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d3015ecb0 0x7f8d30153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d300cf590 0x7f8d30098680 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 --2- 172.21.15.82:0/742463097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d300c9ef0 0x7f8d3015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 >> 172.21.15.82:0/742463097 conn(0x7f8d3006ae60 msgr2=0x7f8d3015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 shutdown_connections 2024-04-25T10:05:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:22.998+0000 7f8d36609640 1 -- 172.21.15.82:0/742463097 wait complete. 2024-04-25T10:05:23.018 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:23.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.134+0000 7ff935369640 1 Processor -- start 2024-04-25T10:05:23.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.134+0000 7ff935369640 1 -- start start 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff9300d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9300d1cd0 0x7ff9300d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9300d0620 0x7ff9300ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93006c440 con 0x7ff9300d1cd0 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff93006be60 con 0x7ff9300d0620 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff935369640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93015de60 con 0x7ff93015da80 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9300d1cd0 0x7ff9300d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9300d1cd0 0x7ff9300d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44336/0 (socket says 172.21.15.82:44336) 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 -- 172.21.15.82:0/1507790899 learned_addr learned my addr 172.21.15.82:0/1507790899 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff92f7fe640 1 --2- 172.21.15.82:0/1507790899 >> v2:172.21.15.196:3300/0 conn(0x7ff9300d0620 0x7ff9300ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff92effd640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff9300d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 -- 172.21.15.82:0/1507790899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 msgr2=0x7ff9300d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff9300d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 -- 172.21.15.82:0/1507790899 >> v2:172.21.15.196:3300/0 conn(0x7ff9300d0620 msgr2=0x7ff9300ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 --2- 172.21.15.82:0/1507790899 >> v2:172.21.15.196:3300/0 conn(0x7ff9300d0620 0x7ff9300ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 -- 172.21.15.82:0/1507790899 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300d2450 con 0x7ff9300d1cd0 2024-04-25T10:05:23.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff927fff640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d1cd0 0x7ff9300d00e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff93006de60 tx=0x7ff91801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f4893dc4eabf1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff92e7fc640 1 -- 172.21.15.82:0/1507790899 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180ab5f0 con 0x7ff9300d1cd0 2024-04-25T10:05:23.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff92e7fc640 1 -- 172.21.15.82:0/1507790899 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9180b0070 con 0x7ff9300d1cd0 2024-04-25T10:05:23.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.138+0000 7ff92e7fc640 1 -- 172.21.15.82:0/1507790899 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9180ab9b0 con 0x7ff9300d1cd0 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/1507790899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d1cd0 msgr2=0x7ff9300d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d1cd0 0x7ff9300d00e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff93006de60 tx=0x7ff91801c520 comp rx=0 tx=0).stop 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/1507790899 shutdown_connections 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- 172.21.15.82:0/1507790899 >> v2:172.21.15.196:3300/0 conn(0x7ff9300d0620 0x7ff9300ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d1cd0 0x7ff9300d00e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- 172.21.15.82:0/1507790899 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff9300d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/1507790899 >> 172.21.15.82:0/1507790899 conn(0x7ff93008b000 msgr2=0x7ff9300d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/1507790899 shutdown_connections 2024-04-25T10:05:23.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/1507790899 wait complete. 2024-04-25T10:05:23.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 Processor -- start 2024-04-25T10:05:23.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- start start 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9300d1cd0 0x7ff930154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff930156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93006ca30 con 0x7ff9300d0620 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93006c8b0 con 0x7ff9300d1cd0 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93006cbb0 con 0x7ff93015da80 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44348/0 (socket says 172.21.15.82:44348) 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 -- 172.21.15.82:0/3715093589 learned_addr learned my addr 172.21.15.82:0/3715093589 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 -- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 msgr2=0x7ff930156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff930156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 -- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9300d1cd0 msgr2=0x7ff930154e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9300d1cd0 0x7ff930154e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 -- 172.21.15.82:0/3715093589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9301553c0 con 0x7ff9300d0620 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff92effd640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d620 tx=0x7ff92000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000b620 con 0x7ff9300d0620 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920019070 con 0x7ff9300d0620 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff930157360 con 0x7ff9300d0620 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92000b9e0 con 0x7ff9300d0620 2024-04-25T10:05:23.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.142+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff930157920 con 0x7ff9300d0620 2024-04-25T10:05:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93004efa0 con 0x7ff9300d0620 2024-04-25T10:05:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff92000bb80 con 0x7ff9300d0620 2024-04-25T10:05:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff9277fe640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 0x7ff910069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff927fff640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 0x7ff910069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff9200855c0 con 0x7ff9300d0620 2024-04-25T10:05:23.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.146+0000 7ff927fff640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 0x7ff910069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9180af000 tx=0x7ff9180a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.150+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9301553c0 con 0x7ff9300d0620 2024-04-25T10:05:23.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.334+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9300b4470 con 0x7ff9300d0620 2024-04-25T10:05:23.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.334+0000 7ff9277fe640 1 -- 172.21.15.82:0/3715093589 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4936 (secure 0 0 0) 0x7ff920056430 con 0x7ff9300d0620 2024-04-25T10:05:23.339 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:23.339 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:05:20.613208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:23.339 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-25T10:05:23.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 msgr2=0x7ff910069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 0x7ff910069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff9180af000 tx=0x7ff9180a7000 comp rx=0 tx=0).stop 2024-04-25T10:05:23.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 msgr2=0x7ff93015f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff92000d620 tx=0x7ff92000dae0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 shutdown_connections 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff910066f60 0x7ff910069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff93015da80 0x7ff930156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9300d1cd0 0x7ff930154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 --2- 172.21.15.82:0/3715093589 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9300d0620 0x7ff93015f070 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 >> 172.21.15.82:0/3715093589 conn(0x7ff93008b000 msgr2=0x7ff9300cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 shutdown_connections 2024-04-25T10:05:23.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.338+0000 7ff935369640 1 -- 172.21.15.82:0/3715093589 wait complete. 2024-04-25T10:05:23.512 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:05:23.506+0000 7feefa592640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:05:23.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:05:17.584870+0000', 'modified': '2024-04-25T10:05:20.613208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6420}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6420': {'gid': 6420, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/191077906', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 191077906}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 191077906}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:05:23.759 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:05:23.760 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 Processor -- start 2024-04-25T10:05:23.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 -- start start 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89e40b3b30 0x7f89e40c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e409a150 con 0x7f89e40cc0c0 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e40cbc30 con 0x7f89e40b3b30 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89ea9bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e409a430 con 0x7f89e40c44f0 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89e40b3b30 0x7f89e40c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f89e40b3b30 0x7f89e40c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44358/0 (socket says 172.21.15.82:44358) 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 -- 172.21.15.82:0/670252417 learned_addr learned my addr 172.21.15.82:0/670252417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e3fff640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 -- 172.21.15.82:0/670252417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 msgr2=0x7f89e4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e4099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 -- 172.21.15.82:0/670252417 >> v1:172.21.15.196:6789/0 conn(0x7f89e40cc0c0 legacy=0x7f89e40cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e37fe640 1 -- 172.21.15.82:0/670252417 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e409a710 con 0x7f89e40b3b30 2024-04-25T10:05:23.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.882+0000 7f89e3fff640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89e37fe640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40b3b30 0x7f89e40c48d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009870 tx=0x7f89cc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08572f390693b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89e2ffd640 1 -- 172.21.15.82:0/670252417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0b0070 con 0x7f89e40b3b30 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89e2ffd640 1 -- 172.21.15.82:0/670252417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc0ab600 con 0x7f89e40b3b30 2024-04-25T10:05:23.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89e2ffd640 1 -- 172.21.15.82:0/670252417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0ab9d0 con 0x7f89e40b3b30 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- 172.21.15.82:0/670252417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40b3b30 msgr2=0x7f89e40c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40b3b30 0x7f89e40c48d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f89cc009870 tx=0x7f89cc01c400 comp rx=0 tx=0).stop 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- 172.21.15.82:0/670252417 shutdown_connections 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40b3b30 0x7f89e40c48d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/670252417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- 172.21.15.82:0/670252417 >> 172.21.15.82:0/670252417 conn(0x7f89e40b3010 msgr2=0x7f89e40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- 172.21.15.82:0/670252417 shutdown_connections 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- 172.21.15.82:0/670252417 wait complete. 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 Processor -- start 2024-04-25T10:05:23.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.886+0000 7f89ea9bc640 1 -- start start 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40b3b30 0x7f89e415bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e40b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e416e9f0 con 0x7f89e40cc0c0 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e416e870 con 0x7f89e40b3b30 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e416eb70 con 0x7f89e40c44f0 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44368/0 (socket says 172.21.15.82:44368) 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 -- 172.21.15.82:0/4221207318 learned_addr learned my addr 172.21.15.82:0/4221207318 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e37fe640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e40b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e3fff640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40b3b30 0x7f89e415bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 -- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 msgr2=0x7f89e40b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e40b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 -- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40b3b30 msgr2=0x7f89e415bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40b3b30 0x7f89e415bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 -- 172.21.15.82:0/4221207318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4152a50 con 0x7f89e40cc0c0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e8f59640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f89d400b410 tx=0x7f89d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 -- 172.21.15.82:0/4221207318 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4015020 con 0x7f89e40cc0c0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 -- 172.21.15.82:0/4221207318 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4011870 con 0x7f89e40cc0c0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e40c5e80 con 0x7f89e40cc0c0 2024-04-25T10:05:23.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 -- 172.21.15.82:0/4221207318 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4011c60 con 0x7f89e40cc0c0 2024-04-25T10:05:23.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e40c6930 con 0x7f89e40cc0c0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 -- 172.21.15.82:0/4221207318 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89d4018020 con 0x7f89e40cc0c0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 0x7f89c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e3fff640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 0x7f89c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e0ff9640 1 -- 172.21.15.82:0/4221207318 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89d4084c30 con 0x7f89e40cc0c0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.890+0000 7f89e3fff640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 0x7f89c4069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0075b0 tx=0x7f89dc007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 Processor -- start 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- start start 2024-04-25T10:05:23.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e6391369f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e639137310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e63901e5f0 con 0x55e639180920 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e638f85da0 con 0x55e639136f30 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e638f0e150 con 0x55e639181240 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44382/0 (socket says 172.21.15.82:44382) 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 -- 172.21.15.82:0/2249017734 learned_addr learned my addr 172.21.15.82:0/2249017734 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e37fe640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e6391369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e8f59640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e639137310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 -- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 msgr2=0x55e6391369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e6391369f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 -- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 msgr2=0x55e639137310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e639137310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 -- 172.21.15.82:0/2249017734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e638f27ca0 con 0x55e639180920 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e8f59640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e639137310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c700 tx=0x7f89dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57341b0ddee95c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89d9ffb640 1 -- 172.21.15.82:0/2249017734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc027070 con 0x55e639180920 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89d9ffb640 1 -- 172.21.15.82:0/2249017734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc018430 con 0x55e639180920 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89d9ffb640 1 -- 172.21.15.82:0/2249017734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc0187e0 con 0x55e639180920 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- 172.21.15.82:0/2249017734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 msgr2=0x55e639180d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f89dc00c700 tx=0x7f89dc011000 comp rx=0 tx=0).stop 2024-04-25T10:05:23.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- 172.21.15.82:0/2249017734 shutdown_connections 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e6391369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e639137310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- 172.21.15.82:0/2249017734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639180920 0x55e639180d00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- 172.21.15.82:0/2249017734 >> 172.21.15.82:0/2249017734 conn(0x55e63913d2a0 msgr2=0x55e6391352d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- 172.21.15.82:0/2249017734 shutdown_connections 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- 172.21.15.82:0/2249017734 wait complete. 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- start start 2024-04-25T10:05:23.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 Processor -- start 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e63913aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e638f40990 con 0x55e639136f30 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e638f81e40 con 0x55e639180920 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e638f11900 con 0x55e639181240 2024-04-25T10:05:23.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89edb29000 10 client.0 _ll_register_callbacks cb 0x55e6390391c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.894+0000 7f89e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44392/0 (socket says 172.21.15.82:44392) 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 -- 172.21.15.82:0/2121244839 learned_addr learned my addr 172.21.15.82:0/2121244839 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 -- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 msgr2=0x55e63913aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e63913aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 -- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 msgr2=0x55e6391881c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 -- 172.21.15.82:0/2121244839 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e638f3e630 con 0x55e639136f30 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e3fff640 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f89dc007d30 tx=0x7f89dc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b538d5e477e488a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89dc027050 con 0x55e639136f30 2024-04-25T10:05:23.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89dc02e450 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89dc018430 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89dc02e690 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 10 client.6561 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e638f0e3a0 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89dc018430 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89dc021030 con 0x55e639136f30 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 10 client.6561 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 10 client.6561 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 20 client.6561 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e63913c160 0x55e639191c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 4 client.6561 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e638f21a50 con 0x55e63913c160 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e37fe640 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e63913c160 0x55e639191c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 10 client.6561 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89e37fe640 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e63913c160 0x55e639191c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2790393e3cfbf1cd server_cookie=80cbeb34144fe8b0 in_seq=0 out_seq=0 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 1 -- 172.21.15.82:0/2121244839 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e638f21a50 con 0x55e63913c160 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 10 client.6561 handle_command_reply: tid=0 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 client.6561 shutdown 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 msgr2=0x55e639187c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f89dc007d30 tx=0x7f89dc004ac0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 shutdown_connections 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e63913c160 0x55e639191c70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639181240 0x55e63913aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89c2ffd640 0 client.6561 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 --2- 172.21.15.82:0/2121244839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639136f30 0x55e639187c80 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 >> 172.21.15.82:0/2121244839 conn(0x55e63913d2a0 msgr2=0x55e6391359e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 shutdown_connections 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 1 -- 172.21.15.82:0/2121244839 wait complete. 2024-04-25T10:05:23.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.898+0000 7f89edb29000 20 client.6561 trim_cache size 0 max 16384 2024-04-25T10:05:23.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 Processor -- start 2024-04-25T10:05:23.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- start start 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e6391dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639180920 0x55e6391dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4206800 con 0x55e639181240 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4070370 con 0x55e639136f30 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e4070970 con 0x55e639180920 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44398/0 (socket says 172.21.15.82:44398) 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 -- 172.21.15.82:0/1632797145 learned_addr learned my addr 172.21.15.82:0/1632797145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 -- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639180920 msgr2=0x55e6391dc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639180920 0x55e6391dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e3fff640 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e6391dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 -- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 msgr2=0x55e6391dc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e6391dc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 -- 172.21.15.82:0/1632797145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e639181070 con 0x55e639181240 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89e8f59640 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f89d405d620 tx=0x7f89d4055c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9061dc4a50062cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89d97fa640 1 -- 172.21.15.82:0/1632797145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d405db50 con 0x55e639181240 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89d97fa640 1 -- 172.21.15.82:0/1632797145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4011e00 con 0x55e639181240 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89d97fa640 1 -- 172.21.15.82:0/1632797145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4053740 con 0x55e639181240 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- 172.21.15.82:0/1632797145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 msgr2=0x55e6391dcdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f89d405d620 tx=0x7f89d4055c00 comp rx=0 tx=0).stop 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- 172.21.15.82:0/1632797145 shutdown_connections 2024-04-25T10:05:23.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639180920 0x55e6391dc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391dcdb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 --2- 172.21.15.82:0/1632797145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639136f30 0x55e6391dc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- 172.21.15.82:0/1632797145 >> 172.21.15.82:0/1632797145 conn(0x55e63913d2a0 msgr2=0x55e63913a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- 172.21.15.82:0/1632797145 shutdown_connections 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- 172.21.15.82:0/1632797145 wait complete. 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 -- start start 2024-04-25T10:05:23.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.914+0000 7f89edb29000 1 Processor -- start 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639136f30 0x55e6391e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e638f8a3d0 con 0x55e639181240 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e639041d50 con 0x55e639180920 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e638f20980 con 0x55e639136f30 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 10 client.0 _ll_register_callbacks cb 0x7f89dc0077e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:23.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44414/0 (socket says 172.21.15.82:44414) 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 -- 172.21.15.82:0/629692585 learned_addr learned my addr 172.21.15.82:0/629692585 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e37fe640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e3fff640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639136f30 0x55e6391e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 -- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639136f30 msgr2=0x55e6391e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639136f30 0x55e6391e15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 -- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 msgr2=0x55e6391e1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391e1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 -- 172.21.15.82:0/629692585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e639007600 con 0x55e639181240 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e8f59640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f89d4051850 tx=0x7f89d401c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce706525307e0e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:23.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4080070 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f89d4051ab0 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89d4011e00 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d4059170 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 10 client.6567 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e638f1a9b0 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4011e00 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89d4054890 con 0x55e639181240 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 10 client.6567 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 10 client.6567 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 20 client.6567 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:23.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e6391de130 0x55e6391de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 4 client.6567 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e638f932e0 con 0x55e6391de130 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e3fff640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e6391de130 0x55e6391de510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 10 client.6567 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89e3fff640 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e6391de130 0x55e6391de510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef07b4760996536e server_cookie=84e1ede8060ed0e9 in_seq=0 out_seq=0 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 1 -- 172.21.15.82:0/629692585 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55e638f932e0 con 0x55e6391de130 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 10 client.6567 handle_command_reply: tid=0 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 client.6567 shutdown 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 msgr2=0x55e6391ddbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f89d4051850 tx=0x7f89d401c040 comp rx=0 tx=0).stop 2024-04-25T10:05:23.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 shutdown_connections 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e6391de130 0x55e6391de510 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e639181240 0x55e6391ddbf0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89c2ffd640 0 client.6567 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e639180920 0x55e6391e1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 --2- 172.21.15.82:0/629692585 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e639136f30 0x55e6391e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 >> 172.21.15.82:0/629692585 conn(0x55e63913d2a0 msgr2=0x55e639188870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 shutdown_connections 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 1 -- 172.21.15.82:0/629692585 wait complete. 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89edb29000 20 client.6567 trim_cache size 0 max 16384 2024-04-25T10:05:23.931 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6420, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 5.6858235309999996, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 32, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 65, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 65, 2024-04-25T10:05:23.932 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 10.760403602 2024-04-25T10:05:23.933 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:23.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 msgr2=0x7f89c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 0x7f89c4069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0075b0 tx=0x7f89dc007540 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 msgr2=0x7f89e40b2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f89d400b410 tx=0x7f89d400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 shutdown_connections 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f89c4066f90 0x7f89c4069450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f89e40c44f0 0x7f89e40b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f89e40cc0c0 0x7f89e40b2320 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 --2- 172.21.15.82:0/4221207318 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f89e40b3b30 0x7f89e415bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 >> 172.21.15.82:0/4221207318 conn(0x7f89e40b3010 msgr2=0x7f89e40ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 shutdown_connections 2024-04-25T10:05:23.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:23.918+0000 7f89ea9bc640 1 -- 172.21.15.82:0/4221207318 wait complete. 2024-04-25T10:05:23.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:05:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 Processor -- start 2024-04-25T10:05:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 -- start start 2024-04-25T10:05:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12cc0d07f0 0x7f12cc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0d1cd0 0x7f12cc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc06be60 con 0x7f12cc15da80 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc06c440 con 0x7f12cc0d07f0 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d3371640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc15de60 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d090c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12cc0d07f0 0x7f12cc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d090c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12cc0d07f0 0x7f12cc0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44426/0 (socket says 172.21.15.82:44426) 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d090c640 1 -- 172.21.15.82:0/2216667772 learned_addr learned my addr 172.21.15.82:0/2216667772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.038+0000 7f12d110d640 1 --1- 172.21.15.82:0/2216667772 >> v1:172.21.15.82:6789/0 conn(0x7f12cc15da80 0x7f12cc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48850/0 (socket says 172.21.15.82:48850) 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d090c640 1 -- 172.21.15.82:0/2216667772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0d1cd0 msgr2=0x7f12cc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d090c640 1 --2- 172.21.15.82:0/2216667772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0d1cd0 0x7f12cc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d090c640 1 -- 172.21.15.82:0/2216667772 >> v1:172.21.15.82:6789/0 conn(0x7f12cc15da80 legacy=0x7f12cc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d090c640 1 -- 172.21.15.82:0/2216667772 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0d2450 con 0x7f12cc0d07f0 2024-04-25T10:05:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12c3fff640 1 -- 172.21.15.82:0/2216667772 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63254486 0 0) 0x7f12cc06be60 con 0x7f12cc15da80 2024-04-25T10:05:24.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d090c640 1 --2- 172.21.15.82:0/2216667772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d07f0 0x7f12cc0d4230 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f12c8006fd0 tx=0x7f12c801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3f1a8168d151fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12c3fff640 1 -- 172.21.15.82:0/2216667772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80aa610 con 0x7f12cc0d07f0 2024-04-25T10:05:24.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12c3fff640 1 -- 172.21.15.82:0/2216667772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c80af070 con 0x7f12cc0d07f0 2024-04-25T10:05:24.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12c3fff640 1 -- 172.21.15.82:0/2216667772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80aa9d0 con 0x7f12cc0d07f0 2024-04-25T10:05:24.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 -- 172.21.15.82:0/2216667772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d07f0 msgr2=0x7f12cc0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 --2- 172.21.15.82:0/2216667772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d07f0 0x7f12cc0d4230 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f12c8006fd0 tx=0x7f12c801c510 comp rx=0 tx=0).stop 2024-04-25T10:05:24.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 -- 172.21.15.82:0/2216667772 shutdown_connections 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 --2- 172.21.15.82:0/2216667772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc0d1cd0 0x7f12cc06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 --2- 172.21.15.82:0/2216667772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d07f0 0x7f12cc0d4230 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 -- 172.21.15.82:0/2216667772 >> 172.21.15.82:0/2216667772 conn(0x7f12cc08b000 msgr2=0x7f12cc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 -- 172.21.15.82:0/2216667772 shutdown_connections 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.042+0000 7f12d3371640 1 -- 172.21.15.82:0/2216667772 wait complete. 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 Processor -- start 2024-04-25T10:05:24.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- start start 2024-04-25T10:05:24.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc0d07f0 0x7f12cc15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc15da80 0x7f12cc0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc06d7a0 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc06d620 con 0x7f12cc0d07f0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc06d920 con 0x7f12cc15da80 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44436/0 (socket says 172.21.15.82:44436) 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 -- 172.21.15.82:0/3329020165 learned_addr learned my addr 172.21.15.82:0/3329020165 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d110d640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc0d07f0 0x7f12cc15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 -- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc15da80 msgr2=0x7f12cc0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc15da80 0x7f12cc0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 -- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc0d07f0 msgr2=0x7f12cc15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc0d07f0 0x7f12cc15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 -- 172.21.15.82:0/3329020165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc15e640 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d090c640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f12c801a140 tx=0x7f12c80192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8005db0 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8003910 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc15e920 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8003ce0 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc15a8d0 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc04efa0 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12c8006030 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 0x7f12b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f12c811f450 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.046+0000 7f12d110d640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 0x7f12b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:24.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.050+0000 7f12d110d640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 0x7f12b0069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f70 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:24.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.050+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12c80a7070 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.230+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12cc0ceb80 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.230+0000 7f12c1ffb640 1 -- 172.21.15.82:0/3329020165 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f12c8016130 con 0x7f12cc0d1cd0 2024-04-25T10:05:24.235 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:05:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 msgr2=0x7f12b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 0x7f12b0069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f12bc000f70 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 msgr2=0x7f12cc0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:24.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f12c801a140 tx=0x7f12c80192b0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 shutdown_connections 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b0066f60 0x7f12b0069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12cc15da80 0x7f12cc0b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12cc0d1cd0 0x7f12cc0b41a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 --2- 172.21.15.82:0/3329020165 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12cc0d07f0 0x7f12cc15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 >> 172.21.15.82:0/3329020165 conn(0x7f12cc08b000 msgr2=0x7f12cc0d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 shutdown_connections 2024-04-25T10:05:24.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:24.234+0000 7f12d3371640 1 -- 172.21.15.82:0/3329020165 wait complete. 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:05:24.253 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:24.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.254 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.260 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42345sec preferred_lft 42345sec 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:05:24.261 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:05:24.262 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:05:24.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.263 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:05:24.317 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:05:24.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.318 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:05:24.369 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:05:24.370 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:24.370 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:24.370 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:05:24.370 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:05:24.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.370 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:05:24.370 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:05:24.370 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:05:24.371 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:05:24.371 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:05:24.371 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:05:24.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.445 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:05:24.493 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:05:24.493 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:05:24.494 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:24.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.494 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:05:24.549 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:05:24.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:24.549 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:24.597 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:05:24.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:05:24.597 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:24.644 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:05:24.702 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:05:24.750 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:24.751 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:24.752 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:24.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.753 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:05:24.801 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:05:24.801 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:05:24.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:05:24.860 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:05:24.913 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:05:24.914 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:05:24.915 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:24.917 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:24.918 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:24.918 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:24.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:24.919 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:05:24.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:05:24.922+0000 7fe78faf6580 -1 init, newargv = 0x5650a7bacb80 newargc=15 2024-04-25T10:05:24.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[43150]: starting ceph client 2024-04-25T10:05:24.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[43150]: starting fuse 2024-04-25T10:05:26.032 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:05:26.047 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:05:26.095 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:05:26.096 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:05:26.097 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:05:26.098 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:26.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.100 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:05:26.145 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:05:26.145 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:05:26.146 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:05:26.147 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:05:26.147 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:05:26.147 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:05:26.147 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:05:26.150 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:05:26.151 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:05:26.271 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.43150.asok 2024-04-25T10:05:26.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:26.283 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43150.asok 2024-04-25T10:05:26.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.283 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43150.asok status 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:05:26.438 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "43150", 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6585, 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 6585 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:05:26.439 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 249035533 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 249035533 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.6585 192.168.0.1:0/249035533", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/249035533", 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 32, 2024-04-25T10:05:26.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 65, 2024-04-25T10:05:26.441 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:05:26.441 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:05:26.441 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:05:26.441 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:26.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.452 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:05:26.452 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:26.457 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:05:26.458 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:26.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:05:26.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.517 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T10:05:26.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:26.578 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T10:05:26.632 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:05:26.632 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:05:26.632 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:05:26.632 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:05:26.632 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T10:05:26.633 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:05:26.743 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:05:26.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:05:26.752 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:26.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 Processor -- start 2024-04-25T10:05:26.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 -- start start 2024-04-25T10:05:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25240ce510 0x7f25240ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 0x7f2524142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25240cee30 con 0x7f2524144ca0 2024-04-25T10:05:26.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25240cc2b0 con 0x7f25240ce510 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2529184640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252406e290 con 0x7f25241451f0 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25240ce510 0x7f25240ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25240ce510 0x7f25240ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41376/0 (socket says 172.21.15.82:41376) 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 -- 172.21.15.82:0/126955838 learned_addr learned my addr 172.21.15.82:0/126955838 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f252359e640 1 --2- 172.21.15.82:0/126955838 >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 0x7f2524142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 -- 172.21.15.82:0/126955838 >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 msgr2=0x7f2524142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 --2- 172.21.15.82:0/126955838 >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 0x7f2524142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 -- 172.21.15.82:0/126955838 >> v1:172.21.15.196:6789/0 conn(0x7f2524144ca0 legacy=0x7f252406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f2522d9d640 1 -- 172.21.15.82:0/126955838 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25240ccc70 con 0x7f25240ce510 2024-04-25T10:05:26.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.918+0000 7f252359e640 1 --2- 172.21.15.82:0/126955838 >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 0x7f2524142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2522d9d640 1 --2- 172.21.15.82:0/126955838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240ce8f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25240b48f0 tx=0x7f251801ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78377e86a5cb6b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2521d9b640 1 -- 172.21.15.82:0/126955838 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180b5070 con 0x7f25240ce510 2024-04-25T10:05:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2521d9b640 1 -- 172.21.15.82:0/126955838 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180a89d0 con 0x7f25240ce510 2024-04-25T10:05:26.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2521d9b640 1 -- 172.21.15.82:0/126955838 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180a8da0 con 0x7f25240ce510 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- 172.21.15.82:0/126955838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 msgr2=0x7f25240ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- 172.21.15.82:0/126955838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240ce8f0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f25240b48f0 tx=0x7f251801ecd0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- 172.21.15.82:0/126955838 shutdown_connections 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- 172.21.15.82:0/126955838 >> v2:172.21.15.196:3300/0 conn(0x7f25241451f0 0x7f2524142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- 172.21.15.82:0/126955838 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240ce8f0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- 172.21.15.82:0/126955838 >> 172.21.15.82:0/126955838 conn(0x7f25240b11a0 msgr2=0x7f25240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- 172.21.15.82:0/126955838 shutdown_connections 2024-04-25T10:05:26.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- 172.21.15.82:0/126955838 wait complete. 2024-04-25T10:05:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 Processor -- start 2024-04-25T10:05:26.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- start start 2024-04-25T10:05:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 0x7f25241f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25241451f0 0x7f25241f65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252406ee30 con 0x7f25240ce510 2024-04-25T10:05:26.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252406df20 con 0x7f2524144ca0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2529184640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240cc900 con 0x7f25241451f0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2522d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41386/0 (socket says 172.21.15.82:41386) 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f2522d9d640 1 -- 172.21.15.82:0/2835703249 learned_addr learned my addr 172.21.15.82:0/2835703249 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.922+0000 7f252259c640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 0x7f25241f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f252359e640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25241451f0 0x7f25241f65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 -- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25241451f0 msgr2=0x7f25241f65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25241451f0 0x7f25241f65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 -- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 msgr2=0x7f25241f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 0x7f25241f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 -- 172.21.15.82:0/2835703249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25241f3860 con 0x7f25240ce510 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f252259c640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 0x7f25241f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2522d9d640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25180b3040 tx=0x7f251801ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180c0070 con 0x7f25240ce510 2024-04-25T10:05:26.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180b0b60 con 0x7f25240ce510 2024-04-25T10:05:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25241f3b40 con 0x7f25240ce510 2024-04-25T10:05:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180b5070 con 0x7f25240ce510 2024-04-25T10:05:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241f4100 con 0x7f25240ce510 2024-04-25T10:05:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252404efa0 con 0x7f25240ce510 2024-04-25T10:05:26.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25180c3070 con 0x7f25240ce510 2024-04-25T10:05:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 0x7f2510069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2518122400 con 0x7f25240ce510 2024-04-25T10:05:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f252259c640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 0x7f2510069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:26.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.926+0000 7f252259c640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 0x7f2510069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f25240ca4f0 tx=0x7f2514009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:26.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:26.930+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2518018830 con 0x7f25240ce510 2024-04-25T10:05:27.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.118+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25240b1cc0 con 0x7f25240ce510 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.118+0000 7f250b7fe640 1 -- 172.21.15.82:0/2835703249 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f25180f3280 con 0x7f25240ce510 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:27.124 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:05:20.613208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:27.125 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 32 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 msgr2=0x7f2510069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 0x7f2510069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f25240ca4f0 tx=0x7f2514009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 msgr2=0x7f25240c8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f25180b3040 tx=0x7f251801ba70 comp rx=0 tx=0).stop 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 shutdown_connections 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2510066f60 0x7f2510069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25241451f0 0x7f25241f65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2524144ca0 0x7f25241f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 --2- 172.21.15.82:0/2835703249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25240ce510 0x7f25240c8db0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 >> 172.21.15.82:0/2835703249 conn(0x7f25240b11a0 msgr2=0x7f25240b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 shutdown_connections 2024-04-25T10:05:27.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.122+0000 7f2529184640 1 -- 172.21.15.82:0/2835703249 wait complete. 2024-04-25T10:05:27.143 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:05:27.286 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:05:27.286 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:05:27.287 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:05:27.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.318+0000 7f4a111dc080 1 Processor -- start 2024-04-25T10:05:27.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- start start 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed214ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed214161f0 0x55ed213708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed213727c0 con 0x55ed2138c360 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed212e59f0 con 0x55ed2133e570 2024-04-25T10:05:27.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed21372940 con 0x55ed214161f0 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41404/0 (socket says 172.21.15.82:41404) 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 -- 172.21.15.82:0/864492315 learned_addr learned my addr 172.21.15.82:0/864492315 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 -- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed214161f0 msgr2=0x55ed213708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed214161f0 0x55ed213708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 -- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 msgr2=0x55ed214ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed214ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 -- 172.21.15.82:0/864492315 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed21413150 con 0x55ed2138c360 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4a040ae040 tx=0x7f4a04003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8324348d6f5da9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0d4f7640 1 -- 172.21.15.82:0/864492315 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a040b0040 con 0x55ed2138c360 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0d4f7640 1 -- 172.21.15.82:0/864492315 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a040ba070 con 0x55ed2138c360 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0d4f7640 1 -- 172.21.15.82:0/864492315 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a040abcd0 con 0x55ed2138c360 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- 172.21.15.82:0/864492315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 msgr2=0x55ed214a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f4a040ae040 tx=0x7f4a04003030 comp rx=0 tx=0).stop 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- 172.21.15.82:0/864492315 shutdown_connections 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed214161f0 0x55ed213708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed2138c360 0x55ed214a7bf0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- 172.21.15.82:0/864492315 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed214ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- 172.21.15.82:0/864492315 >> 172.21.15.82:0/864492315 conn(0x55ed21344c90 msgr2=0x55ed21415480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- 172.21.15.82:0/864492315 shutdown_connections 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- 172.21.15.82:0/864492315 wait complete. 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 Processor -- start 2024-04-25T10:05:27.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- start start 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed213f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed2138c360 0x55ed213f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 0x55ed213f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed214133f0 con 0x55ed214161f0 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed2136c9d0 con 0x55ed2133e570 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a111dc080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed214a8130 con 0x55ed2138c360 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0f4fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 0x55ed213f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0f4fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 0x55ed213f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41418/0 (socket says 172.21.15.82:41418) 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.322+0000 7f4a0f4fb640 1 -- 172.21.15.82:0/3076653755 learned_addr learned my addr 172.21.15.82:0/3076653755 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed2138c360 0x55ed213f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0ecfa640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed213f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 -- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed2138c360 msgr2=0x55ed213f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ed2138c360 0x55ed213f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 -- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 msgr2=0x55ed213f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed213f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 -- 172.21.15.82:0/3076653755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed213f6f70 con 0x55ed214161f0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0ecfa640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ed2133e570 0x55ed213f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0f4fb640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 0x55ed213f5400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4a08002170 tx=0x7f4a0800b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 -- 172.21.15.82:0/3076653755 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08016070 con 0x55ed214161f0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 -- 172.21.15.82:0/3076653755 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a080062f0 con 0x55ed214161f0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed213f5b10 con 0x55ed214161f0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 -- 172.21.15.82:0/3076653755 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a08004770 con 0x55ed214161f0 2024-04-25T10:05:27.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed213f6050 con 0x55ed214161f0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 -- 172.21.15.82:0/3076653755 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a08019070 con 0x55ed214161f0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49fc065970 0x7f49fc067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f49f7fff640 1 -- 172.21.15.82:0/3076653755 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a08083130 con 0x55ed214161f0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0ecfa640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49fc065970 0x7f49fc067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a111dc080 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55ed21384b80 0x55ed21384f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e65) v9 -- 0x55ed213854a0 con 0x55ed21384b80 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0ecfa640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49fc065970 0x7f49fc067e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005fd0 tx=0x7f4a00005c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.326+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55ed21384b80 0x55ed21384f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a0e4f9640 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55ed21384b80 0x55ed21384f60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a0e4f9640 1 -- 172.21.15.82:0/3076653755 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f4a040b3040 con 0x55ed21384b80 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55ed21384b80 msgr2=0x55ed21384f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55ed21384b80 0x55ed21384f60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49fc065970 msgr2=0x7f49fc067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49fc065970 0x7f49fc067e30 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005fd0 tx=0x7f4a00005c00 comp rx=0 tx=0).stop 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 msgr2=0x55ed213f5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 --2- 172.21.15.82:0/3076653755 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ed214161f0 0x55ed213f5400 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f4a08002170 tx=0x7f4a0800b4b0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a0f4fb640 1 -- 172.21.15.82:0/3076653755 reap_dead start 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 shutdown_connections 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 >> 172.21.15.82:0/3076653755 conn(0x55ed21344c90 msgr2=0x55ed21411f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 shutdown_connections 2024-04-25T10:05:27.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.330+0000 7f4a111dc080 1 -- 172.21.15.82:0/3076653755 wait complete. 2024-04-25T10:05:27.338 INFO:teuthology.orchestra.run.smithi082.stdout:Xfiledir_test 2024-04-25T10:05:27.338 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:27.339 DEBUG:teuthology.orchestra.run.smithi082:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: "ancestors": [ 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1099511627776, 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "file", 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 2 2024-04-25T10:05:27.433 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: "dirino": 1, 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "dir_test", 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 4 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: "pool": 10, 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: "old_pools": [] 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:27.434 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:27.436 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T10:05:27.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.510+0000 7fa3dde1b080 1 Processor -- start 2024-04-25T10:05:27.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.510+0000 7fa3dde1b080 1 -- start start 2024-04-25T10:05:27.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59df73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59df3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b59d587c0 con 0x556b59d24570 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b59ccb9f0 con 0x556b59d72360 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b59d58940 con 0x556b59dfc1f0 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41444/0 (socket says 172.21.15.82:41444) 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 -- 172.21.15.82:0/907164023 learned_addr learned my addr 172.21.15.82:0/907164023 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db138640 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59df73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dc13a640 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59df3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 -- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 msgr2=0x556b59df3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59df3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 -- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 msgr2=0x556b59df73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59df73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 -- 172.21.15.82:0/907164023 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b59df9250 con 0x556b59d24570 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4001600 tx=0x7fa3d4001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f2df2ca26ba0d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3da136640 1 -- 172.21.15.82:0/907164023 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400f510 con 0x556b59d24570 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3da136640 1 -- 172.21.15.82:0/907164023 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400f6d0 con 0x556b59d24570 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3da136640 1 -- 172.21.15.82:0/907164023 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400fac0 con 0x556b59d24570 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- 172.21.15.82:0/907164023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 msgr2=0x556b59d568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4001600 tx=0x7fa3d4001ac0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- 172.21.15.82:0/907164023 shutdown_connections 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59df3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59df73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/907164023 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59d568c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- 172.21.15.82:0/907164023 >> 172.21.15.82:0/907164023 conn(0x556b59d2ac90 msgr2=0x556b59dfb140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- 172.21.15.82:0/907164023 shutdown_connections 2024-04-25T10:05:27.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- 172.21.15.82:0/907164023 wait complete. 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 Processor -- start 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- start start 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59df5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59d6b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59d6da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b59df6cb0 con 0x556b59d24570 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b59df6f50 con 0x556b59d72360 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dde1b080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x556b59df94f0 con 0x556b59dfc1f0 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59df5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59df5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41454/0 (socket says 172.21.15.82:41454) 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db939640 1 -- 172.21.15.82:0/465609067 learned_addr learned my addr 172.21.15.82:0/465609067 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3db138640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59d6b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.514+0000 7fa3dc13a640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59d6da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 -- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 msgr2=0x556b59d6da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59d6da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 -- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 msgr2=0x556b59d6b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x556b59d72360 0x556b59d6b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 -- 172.21.15.82:0/465609067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b59df61a0 con 0x556b59d24570 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dc13a640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x556b59dfc1f0 0x556b59d6da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db939640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59df5b40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4003000 tx=0x7fa3d4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 -- 172.21.15.82:0/465609067 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4021020 con 0x556b59d24570 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 -- 172.21.15.82:0/465609067 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d4012a40 con 0x556b59d24570 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556b59d6aa70 con 0x556b59d24570 2024-04-25T10:05:27.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 -- 172.21.15.82:0/465609067 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4012e30 con 0x556b59d24570 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556b59d6afb0 con 0x556b59d24570 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 -- 172.21.15.82:0/465609067 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3d4024020 con 0x556b59d24570 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3c8065940 0x7fa3c8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3d9134640 1 -- 172.21.15.82:0/465609067 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa3d4085cb0 con 0x556b59d24570 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db138640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3c8065940 0x7fa3c8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x556b59df7a10 0x556b59de36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e65) v9 -- 0x556b59e950c0 con 0x556b59df7a10 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3db138640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3c8065940 0x7fa3c8067e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005390 tx=0x7fa3d0005320 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dc13a640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x556b59df7a10 0x556b59de36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.518+0000 7fa3dc13a640 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x556b59df7a10 0x556b59de36f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dc13a640 1 -- 172.21.15.82:0/465609067 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v65'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa3cc0aa580 con 0x556b59df7a10 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x556b59df7a10 msgr2=0x556b59de36f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x556b59df7a10 0x556b59de36f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3c8065940 msgr2=0x7fa3c8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3c8065940 0x7fa3c8067e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0005390 tx=0x7fa3d0005320 comp rx=0 tx=0).stop 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 msgr2=0x556b59df5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 --2- 172.21.15.82:0/465609067 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x556b59d24570 0x556b59df5b40 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fa3d4003000 tx=0x7fa3d4000f50 comp rx=0 tx=0).stop 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dc13a640 1 -- 172.21.15.82:0/465609067 reap_dead start 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 shutdown_connections 2024-04-25T10:05:27.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 >> 172.21.15.82:0/465609067 conn(0x556b59d2ac90 msgr2=0x556b59d6e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 shutdown_connections 2024-04-25T10:05:27.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.522+0000 7fa3dde1b080 1 -- 172.21.15.82:0/465609067 wait complete. 2024-04-25T10:05:27.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:27.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T10:05:27.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 Processor -- start 2024-04-25T10:05:27.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 -- start start 2024-04-25T10:05:27.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf0c0ccbd0 0x7faf0c0cd060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c0cf490 con 0x7faf0c15ed60 2024-04-25T10:05:27.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0c0cf770 con 0x7faf0c0b0e10 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf11584640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0c15f140 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faf0c0ccbd0 0x7faf0c0cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faf0c15ed60 0x7faf0c09a760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:34788/0 (socket says 172.21.15.82:34788) 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0affd640 1 -- 172.21.15.82:0/1407149610 learned_addr learned my addr 172.21.15.82:0/1407149610 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf09ffb640 1 -- 172.21.15.82:0/1407149610 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4024103015 0 0) 0x7faf0c0cf490 con 0x7faf0c15ed60 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf09ffb640 1 -- 172.21.15.82:0/1407149610 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef8003530 con 0x7faf0c15ed60 2024-04-25T10:05:27.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0a7fc640 1 -- 172.21.15.82:0/1407149610 >> v1:172.21.15.196:6789/0 conn(0x7faf0c0b0e10 legacy=0x7faf0c0cfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0a7fc640 1 -- 172.21.15.82:0/1407149610 >> v1:172.21.15.82:6789/0 conn(0x7faf0c15ed60 legacy=0x7faf0c09a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.690+0000 7faf0a7fc640 1 -- 172.21.15.82:0/1407149610 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0cc3d0 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1407149610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c0cd060 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097c0 tx=0x7faefc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e7f901877b72c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf09ffb640 1 -- 172.21.15.82:0/1407149610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0b0070 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf09ffb640 1 -- 172.21.15.82:0/1407149610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0ab600 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf09ffb640 1 -- 172.21.15.82:0/1407149610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0ab9d0 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- 172.21.15.82:0/1407149610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c0ccbd0 msgr2=0x7faf0c0cd060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 --2- 172.21.15.82:0/1407149610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c0cd060 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7faefc0097c0 tx=0x7faefc01c410 comp rx=0 tx=0).stop 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- 172.21.15.82:0/1407149610 shutdown_connections 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 --2- 172.21.15.82:0/1407149610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c0cd060 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- 172.21.15.82:0/1407149610 >> 172.21.15.82:0/1407149610 conn(0x7faf0c0b2fd0 msgr2=0x7faf0c0b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- 172.21.15.82:0/1407149610 shutdown_connections 2024-04-25T10:05:27.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- 172.21.15.82:0/1407149610 wait complete. 2024-04-25T10:05:27.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 Processor -- start 2024-04-25T10:05:27.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 -- start start 2024-04-25T10:05:27.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf0c0b0e10 0x7faf0c0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.694+0000 7faf11584640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf11584640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0d0330 con 0x7faf0c15ed60 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf11584640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0d01b0 con 0x7faf0c0ccbd0 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf11584640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0b11f0 con 0x7faf0c0b0e10 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41478/0 (socket says 172.21.15.82:41478) 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 -- 172.21.15.82:0/1374856514 learned_addr learned my addr 172.21.15.82:0/1374856514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0affd640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf0c0b0e10 0x7faf0c0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 -- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf0c0b0e10 msgr2=0x7faf0c0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf0c0b0e10 0x7faf0c0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 -- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 msgr2=0x7faf0c14f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c14f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 -- 172.21.15.82:0/1374856514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0c95b0 con 0x7faf0c15ed60 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c14f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0b7fe640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c770 tx=0x7faf0400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 -- 172.21.15.82:0/1374856514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf04003b80 con 0x7faf0c15ed60 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 -- 172.21.15.82:0/1374856514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf04003d20 con 0x7faf0c15ed60 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 -- 172.21.15.82:0/1374856514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0400aa90 con 0x7faf0c15ed60 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0c0c9890 con 0x7faf0c15ed60 2024-04-25T10:05:27.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0c0c91a0 con 0x7faf0c15ed60 2024-04-25T10:05:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 -- 172.21.15.82:0/1374856514 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf0400ac30 con 0x7faf0c15ed60 2024-04-25T10:05:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 0x7faeec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf037fe640 1 -- 172.21.15.82:0/1374856514 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf04085140 con 0x7faf0c15ed60 2024-04-25T10:05:27.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0affd640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 0x7faeec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf0affd640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 0x7faeec069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faef40011f0 tx=0x7faef4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf146f1000 1 Processor -- start 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.698+0000 7faf146f1000 1 -- start start 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c31d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c2d32b0 0x55e91c2d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c11cfd0 con 0x55e91c31dd20 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c0a5170 con 0x55e91c31d400 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91c119070 con 0x55e91c2d32b0 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41486/0 (socket says 172.21.15.82:41486) 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 -- 172.21.15.82:0/4073121670 learned_addr learned my addr 172.21.15.82:0/4073121670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 -- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c2d32b0 msgr2=0x55e91c2d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c2d32b0 0x55e91c2d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c31d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 -- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 msgr2=0x55e91c31d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c31d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 -- 172.21.15.82:0/4073121670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91c163d20 con 0x55e91c31dd20 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7faefc0ae040 tx=0x7faefc0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c584fc6e25a4340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf00ff9640 1 -- 172.21.15.82:0/4073121670 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0b0050 con 0x55e91c31dd20 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf00ff9640 1 -- 172.21.15.82:0/4073121670 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0b1040 con 0x55e91c31dd20 2024-04-25T10:05:27.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf00ff9640 1 -- 172.21.15.82:0/4073121670 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0ab8c0 con 0x55e91c31dd20 2024-04-25T10:05:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- 172.21.15.82:0/4073121670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 msgr2=0x55e91c2d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7faefc0ae040 tx=0x7faefc0a6000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- 172.21.15.82:0/4073121670 shutdown_connections 2024-04-25T10:05:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c2d32b0 0x55e91c2d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c2d2d70 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- 172.21.15.82:0/4073121670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c31d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- 172.21.15.82:0/4073121670 >> 172.21.15.82:0/4073121670 conn(0x55e91c2d5710 msgr2=0x55e91c2d5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- 172.21.15.82:0/4073121670 shutdown_connections 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- 172.21.15.82:0/4073121670 wait complete. 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- start start 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 Processor -- start 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c31dd20 0x55e91c30b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c1dac10 con 0x55e91c2d32b0 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c0d7680 con 0x55e91c31d400 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91c1d7eb0 con 0x55e91c31dd20 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf146f1000 10 client.0 _ll_register_callbacks cb 0x55e91c1d2080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:27.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41490/0 (socket says 172.21.15.82:41490) 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 -- 172.21.15.82:0/3196021024 learned_addr learned my addr 172.21.15.82:0/3196021024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0b7fe640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c31dd20 0x55e91c30b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 -- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c31dd20 msgr2=0x55e91c30b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c31dd20 0x55e91c30b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 -- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 msgr2=0x55e91c30b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.702+0000 7faf0affd640 1 -- 172.21.15.82:0/3196021024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e91c0f4d30 con 0x55e91c2d32b0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf0affd640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7faef4005ec0 tx=0x7faef4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f2f8d3c191ca18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4007560 con 0x55e91c2d32b0 2024-04-25T10:05:27.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faef4007700 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faefc0b1040 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef4022430 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 10 client.6627 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e91c0a53c0 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0b1040 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faef40225d0 con 0x55e91c2d32b0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 10 client.6627 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 10 client.6627 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 20 client.6627 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32c6a0 0x55e91c32ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 4 client.6627 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e91c0b8e00 con 0x55e91c32c6a0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32c6a0 0x55e91c32ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 10 client.6627 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32c6a0 0x55e91c32ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=897d1275be9bcf09 server_cookie=e894f14772a33f in_seq=0 out_seq=0 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 1 -- 172.21.15.82:0/3196021024 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e91c0b8e00 con 0x55e91c32c6a0 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 10 client.6627 handle_command_reply: tid=0 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 client.6627 shutdown 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 msgr2=0x55e91c30ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7faef4005ec0 tx=0x7faef4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 shutdown_connections 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32c6a0 0x55e91c32ea90 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c31dd20 0x55e91c30b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faee9ffb640 0 client.6627 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 --2- 172.21.15.82:0/3196021024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c2d32b0 0x55e91c30ad40 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 >> 172.21.15.82:0/3196021024 conn(0x55e91c2d5710 msgr2=0x55e91c2d3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 shutdown_connections 2024-04-25T10:05:27.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 1 -- 172.21.15.82:0/3196021024 wait complete. 2024-04-25T10:05:27.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.706+0000 7faf146f1000 20 client.6627 trim_cache size 0 max 16384 2024-04-25T10:05:27.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 Processor -- start 2024-04-25T10:05:27.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- start start 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c2daca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c06db30 con 0x55e91c31dd20 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c0b05c0 con 0x55e91c31d400 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0c1f8a70 con 0x55e91c331000 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41502/0 (socket says 172.21.15.82:41502) 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 -- 172.21.15.82:0/2727221509 learned_addr learned my addr 172.21.15.82:0/2727221509 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 -- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 msgr2=0x55e91c2daca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c2daca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 -- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 msgr2=0x55e91c30a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 -- 172.21.15.82:0/2727221509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e91c19f0f0 con 0x55e91c31dd20 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faefc008010 tx=0x7faefc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4623167e3ae4ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faeebfff640 1 -- 172.21.15.82:0/2727221509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0b0040 con 0x55e91c31dd20 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faeebfff640 1 -- 172.21.15.82:0/2727221509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0ab6c0 con 0x55e91c31dd20 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faeebfff640 1 -- 172.21.15.82:0/2727221509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc0abab0 con 0x55e91c31dd20 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- 172.21.15.82:0/2727221509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 msgr2=0x55e91c330ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7faefc008010 tx=0x7faefc004740 comp rx=0 tx=0).stop 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- 172.21.15.82:0/2727221509 shutdown_connections 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c2daca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31dd20 0x55e91c330ac0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 --2- 172.21.15.82:0/2727221509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31d400 0x55e91c30a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- 172.21.15.82:0/2727221509 >> 172.21.15.82:0/2727221509 conn(0x55e91c2d5710 msgr2=0x55e91c2da590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- 172.21.15.82:0/2727221509 shutdown_connections 2024-04-25T10:05:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- 172.21.15.82:0/2727221509 wait complete. 2024-04-25T10:05:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 -- start start 2024-04-25T10:05:27.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.722+0000 7faf146f1000 1 Processor -- start 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31dd20 0x55e91c32ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c32cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c2ce4f0 con 0x55e91c31d400 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e91c2d8d50 con 0x55e91c31dd20 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e91c0e0860 con 0x55e91c331000 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 10 client.0 _ll_register_callbacks cb 0x55e91c0eaf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41508/0 (socket says 172.21.15.82:41508) 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 -- 172.21.15.82:0/1060769029 learned_addr learned my addr 172.21.15.82:0/1060769029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 -- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 msgr2=0x55e91c32cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31dd20 0x55e91c32ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0b7fe640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c32cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c32cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 -- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31dd20 msgr2=0x55e91c32ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31dd20 0x55e91c32ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 -- 172.21.15.82:0/1060769029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e91c0b1d20 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0affd640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faef4009010 tx=0x7faef4001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dfe7f5486b9be67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4030070 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faef40049f0 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faefc0ab6c0 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef400eb20 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 10 client.6633 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e91c0c40c0 con 0x55e91c31d400 2024-04-25T10:05:27.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faefc0ab6c0 con 0x55e91c31d400 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7faef4004bf0 con 0x55e91c31d400 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 10 client.6633 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 10 client.6633 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 20 client.6633 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32de10 0x55e91c38b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 4 client.6633 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55e91c12a510 con 0x55e91c32de10 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32de10 0x55e91c38b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 10 client.6633 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf0a7fc640 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32de10 0x55e91c38b170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e604b464e5c9fc1 server_cookie=d28ac022dc70db92 in_seq=0 out_seq=0 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 1 -- 172.21.15.82:0/1060769029 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e91c12a510 con 0x55e91c32de10 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 10 client.6633 handle_command_reply: tid=0 2024-04-25T10:05:27.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 client.6633 shutdown 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 msgr2=0x55e91c3309d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7faef4009010 tx=0x7faef4001160 comp rx=0 tx=0).stop 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 shutdown_connections 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55e91c32de10 0x55e91c38b170 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e91c331000 0x55e91c32cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e91c31dd20 0x55e91c32ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 --2- 172.21.15.82:0/1060769029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e91c31d400 0x55e91c3309d0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faee9ffb640 0 client.6633 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 >> 172.21.15.82:0/1060769029 conn(0x55e91c2d5710 msgr2=0x55e91c2dae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 shutdown_connections 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 1 -- 172.21.15.82:0/1060769029 wait complete. 2024-04-25T10:05:27.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf146f1000 20 client.6633 trim_cache size 0 max 16384 2024-04-25T10:05:27.738 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:27.738 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:05:27.738 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "35cbd68a-00a5-4591-8708-3cc5cd30cc09", 2024-04-25T10:05:27.738 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:05:27.738 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:27.738 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:05:27.726+0000 7feef658a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6585=0-4194304@1} caps={6585=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6585 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56489997fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 msgr2=0x7faeec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 0x7faeec069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7faef40011f0 tx=0x7faef4009040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.726+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 msgr2=0x7faf0c15d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faf0400c770 tx=0x7faf0400cc30 comp rx=0 tx=0).stop 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 shutdown_connections 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faeec066f60 0x7faeec069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf0c15ed60 0x7faf0c15d3b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf0c0ccbd0 0x7faf0c14f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 --2- 172.21.15.82:0/1374856514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf0c0b0e10 0x7faf0c0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 >> 172.21.15.82:0/1374856514 conn(0x7faf0c0b2fd0 msgr2=0x7faf0c15f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 shutdown_connections 2024-04-25T10:05:27.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.730+0000 7faf11584640 1 -- 172.21.15.82:0/1374856514 wait complete. 2024-04-25T10:05:27.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:27.748 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:05:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 Processor -- start 2024-04-25T10:05:27.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 -- start start 2024-04-25T10:05:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6d700b5270 0x7f6d700b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d7006d090 0x7f6d7006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d7006bff0 con 0x7f6d700b5650 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7006cc00 con 0x7f6d7006d090 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d74f46640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d7006c2d0 con 0x7f6d700b5270 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d6ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d7006d090 0x7f6d7006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d6ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6d7006d090 0x7f6d7006d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41522/0 (socket says 172.21.15.82:41522) 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.858+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2371757665 learned_addr learned my addr 172.21.15.82:0/2371757665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/2371757665 >> v2:172.21.15.196:3300/0 conn(0x7f6d700b5270 0x7f6d700b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2371757665 >> v2:172.21.15.196:3300/0 conn(0x7f6d700b5270 msgr2=0x7f6d700b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/2371757665 >> v2:172.21.15.196:3300/0 conn(0x7f6d700b5270 0x7f6d700b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2371757665 >> v1:172.21.15.196:6789/0 conn(0x7f6d700b5650 legacy=0x7f6d700d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2371757665 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7006c5b0 con 0x7f6d7006d090 2024-04-25T10:05:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/2371757665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7006d090 0x7f6d7006d520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6d7008bb10 tx=0x7f6d6801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c7da47ba7b0b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6d59a640 1 -- 172.21.15.82:0/2371757665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680ab5f0 con 0x7f6d7006d090 2024-04-25T10:05:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6d59a640 1 -- 172.21.15.82:0/2371757665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d680b0070 con 0x7f6d7006d090 2024-04-25T10:05:27.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d6d59a640 1 -- 172.21.15.82:0/2371757665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680ab9b0 con 0x7f6d7006d090 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 -- 172.21.15.82:0/2371757665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7006d090 msgr2=0x7f6d7006d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 --2- 172.21.15.82:0/2371757665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7006d090 0x7f6d7006d520 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f6d7008bb10 tx=0x7f6d6801c4e0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 -- 172.21.15.82:0/2371757665 shutdown_connections 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 --2- 172.21.15.82:0/2371757665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d7006d090 0x7f6d7006d520 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 --2- 172.21.15.82:0/2371757665 >> v2:172.21.15.196:3300/0 conn(0x7f6d700b5270 0x7f6d700b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 -- 172.21.15.82:0/2371757665 >> 172.21.15.82:0/2371757665 conn(0x7f6d7008b000 msgr2=0x7f6d700d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 -- 172.21.15.82:0/2371757665 shutdown_connections 2024-04-25T10:05:27.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.862+0000 7f6d74f46640 1 -- 172.21.15.82:0/2371757665 wait complete. 2024-04-25T10:05:27.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 Processor -- start 2024-04-25T10:05:27.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- start start 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 0x7f6d70153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d700b5650 0x7f6d700b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7015de20 con 0x7f6d700b5270 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700b5ac0 con 0x7f6d7006d090 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7015dfa0 con 0x7f6d700b5650 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41536/0 (socket says 172.21.15.82:41536) 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/1162174192 learned_addr learned my addr 172.21.15.82:0/1162174192 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 0x7f6d70153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d700b5650 0x7f6d700b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d700b5650 msgr2=0x7f6d700b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d700b5650 0x7f6d700b0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 msgr2=0x7f6d70153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 0x7f6d70153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/1162174192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d701580b0 con 0x7f6d700b5270 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 0x7f6d70153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b3f0 tx=0x7f6d5800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d66ffd640 1 -- 172.21.15.82:0/1162174192 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58015020 con 0x7f6d700b5270 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d66ffd640 1 -- 172.21.15.82:0/1162174192 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58011850 con 0x7f6d700b5270 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d70153710 con 0x7f6d700b5270 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d66ffd640 1 -- 172.21.15.82:0/1162174192 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58011c40 con 0x7f6d700b5270 2024-04-25T10:05:27.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d70153cd0 con 0x7f6d700b5270 2024-04-25T10:05:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.866+0000 7f6d66ffd640 1 -- 172.21.15.82:0/1162174192 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d58018020 con 0x7f6d700b5270 2024-04-25T10:05:27.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d66ffd640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d66ffd640 1 -- 172.21.15.82:0/1162174192 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6d58084dc0 con 0x7f6d700b5270 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 0x7f6d50069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d7014d1d0 tx=0x7f6d6000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 Processor -- start 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- start start 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea9421adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 0x55ea941d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9401dff0 con 0x55ea9421b2f0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea93fa6150 con 0x55ea941d10a0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea9401a090 con 0x55ea9421a9d0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41546/0 (socket says 172.21.15.82:41546) 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/235718650 learned_addr learned my addr 172.21.15.82:0/235718650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 0x55ea941d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea9421adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 msgr2=0x55ea9421adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea9421adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 msgr2=0x55ea941d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 0x55ea941d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 -- 172.21.15.82:0/235718650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea941b40d0 con 0x55ea9421b2f0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 0x55ea941d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6d58051a30 tx=0x7f6d58055710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd7c47b49eac0ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d4ffff640 1 -- 172.21.15.82:0/235718650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58011de0 con 0x55ea9421b2f0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d4ffff640 1 -- 172.21.15.82:0/235718650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58053670 con 0x55ea9421b2f0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d4ffff640 1 -- 172.21.15.82:0/235718650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d58051c40 con 0x55ea9421b2f0 2024-04-25T10:05:27.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- 172.21.15.82:0/235718650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 msgr2=0x55ea941d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f6d58051a30 tx=0x7f6d58055710 comp rx=0 tx=0).stop 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- 172.21.15.82:0/235718650 shutdown_connections 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea941d10a0 0x55ea941d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9421b2f0 0x55ea941d0b60 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 --2- 172.21.15.82:0/235718650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea9421adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- 172.21.15.82:0/235718650 >> 172.21.15.82:0/235718650 conn(0x55ea941cd020 msgr2=0x55ea941cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- 172.21.15.82:0/235718650 shutdown_connections 2024-04-25T10:05:27.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- 172.21.15.82:0/235718650 wait complete. 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.870+0000 7f6d780b3000 1 -- start start 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 Processor -- start 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea941ffd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94200270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea93fe2250 con 0x55ea941d10a0 2024-04-25T10:05:27.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea93fd8690 con 0x55ea9421b2f0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea93fa6a40 con 0x55ea9421a9d0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 10 client.0 _ll_register_callbacks cb 0x55ea941bbe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41548/0 (socket says 172.21.15.82:41548) 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 -- 172.21.15.82:0/1985509805 learned_addr learned my addr 172.21.15.82:0/1985509805 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94200270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea941ffd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 -- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 msgr2=0x55ea941ffd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea941ffd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 -- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 msgr2=0x55ea94200270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94200270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 -- 172.21.15.82:0/1985509805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea940cb1c0 con 0x55ea941d10a0 2024-04-25T10:05:27.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6d600094c0 tx=0x7f6d60004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404c43571c8244b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60014df0 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6d60021410 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d58053670 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d6002e050 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 10 client.6645 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55ea940c7170 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d58053670 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d60031020 con 0x55ea941d10a0 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 10 client.6645 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 10 client.6645 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 20 client.6645 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:27.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422ac80 0x55ea9422d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 4 client.6645 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea93fb9c60 con 0x55ea9422ac80 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422ac80 0x55ea9422d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422ac80 0x55ea9422d070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a344dca1728d63 server_cookie=ac65f0afffa460e7 in_seq=0 out_seq=0 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 10 client.6645 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/1985509805 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea93fb9c60 con 0x55ea9422ac80 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 10 client.6645 handle_command_reply: tid=0 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 client.6645 shutdown 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 msgr2=0x55ea941ff7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f6d600094c0 tx=0x7f6d60004ac0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 shutdown_connections 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422ac80 0x55ea9422d070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea941ffd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94200270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 --2- 172.21.15.82:0/1985509805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea941ff7f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d4d7fa640 0 client.6645 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 >> 172.21.15.82:0/1985509805 conn(0x55ea941cd020 msgr2=0x55ea941cfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 shutdown_connections 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 1 -- 172.21.15.82:0/1985509805 wait complete. 2024-04-25T10:05:27.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.874+0000 7f6d780b3000 20 client.6645 trim_cache size 0 max 16384 2024-04-25T10:05:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 Processor -- start 2024-04-25T10:05:27.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 -- start start 2024-04-25T10:05:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421a9d0 0x55ea941d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421b2f0 0x55ea941d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70203500 con 0x55ea941d10a0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d70202720 con 0x55ea9421a9d0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7014cc50 con 0x55ea9421b2f0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41564/0 (socket says 172.21.15.82:41564) 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 -- 172.21.15.82:0/3471974364 learned_addr learned my addr 172.21.15.82:0/3471974364 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 -- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421b2f0 msgr2=0x55ea941d0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421b2f0 0x55ea941d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 -- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421a9d0 msgr2=0x55ea941d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421a9d0 0x55ea941d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 -- 172.21.15.82:0/3471974364 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea94064de0 con 0x55ea941d10a0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6d60007fd0 tx=0x7f6d6001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481af03f20baffc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.890+0000 7f6d4f7fe640 1 -- 172.21.15.82:0/3471974364 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60014df0 con 0x55ea941d10a0 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4f7fe640 1 -- 172.21.15.82:0/3471974364 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60026070 con 0x55ea941d10a0 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4f7fe640 1 -- 172.21.15.82:0/3471974364 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60021630 con 0x55ea941d10a0 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/3471974364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 msgr2=0x55ea94219e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f6d60007fd0 tx=0x7f6d6001f000 comp rx=0 tx=0).stop 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/3471974364 shutdown_connections 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421b2f0 0x55ea941d0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421a9d0 0x55ea941d0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- 172.21.15.82:0/3471974364 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea941d10a0 0x55ea94219e10 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/3471974364 >> 172.21.15.82:0/3471974364 conn(0x55ea941cd020 msgr2=0x55ea942190d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/3471974364 shutdown_connections 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/3471974364 wait complete. 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- start start 2024-04-25T10:05:27.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 Processor -- start 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea94283d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94284270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9401d300 con 0x55ea9422d380 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea93fb8e00 con 0x55ea9421b2f0 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea94022620 con 0x55ea9421a9d0 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 10 client.0 _ll_register_callbacks cb 0x55ea9419c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41574/0 (socket says 172.21.15.82:41574) 2024-04-25T10:05:27.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2020125594 learned_addr learned my addr 172.21.15.82:0/2020125594 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea94283d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6dd9b640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94284270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 msgr2=0x55ea94283d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea94283d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 msgr2=0x55ea94284270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94284270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 -- 172.21.15.82:0/2020125594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea93fbf8e0 con 0x55ea9422d380 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea94283d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6ed9d640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6d6801a110 tx=0x7f6d6801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63590da349ba0844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d680ba070 con 0x55ea9422d380 2024-04-25T10:05:27.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6d680bfc40 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d60026070 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d680bfe40 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 10 client.6651 fetch_fsmap learned FSMap version 32 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55ea940778f0 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60026070 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 32) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6d680abb10 con 0x55ea9422d380 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 10 client.6651 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 10 client.6651 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 20 client.6651 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422e630 0x55ea9428cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 4 client.6651 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea9402b530 con 0x55ea9422e630 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422e630 0x55ea9428cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:27.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d6e59c640 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422e630 0x55ea9428cba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd33ff256c9b5451 server_cookie=f311cf9fe068afb7 in_seq=0 out_seq=0 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 10 client.6651 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 1 -- 172.21.15.82:0/2020125594 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x55ea9402b530 con 0x55ea9422e630 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d4d7fa640 10 client.6651 handle_command_reply: tid=0 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.894+0000 7f6d780b3000 1 client.6651 shutdown 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 msgr2=0x55ea9422d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6d6801a110 tx=0x7f6d6801ad20 comp rx=0 tx=0).stop 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 shutdown_connections 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55ea9422e630 0x55ea9428cba0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55ea9422d380 0x55ea9422d760 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55ea9421b2f0 0x55ea94284270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 --2- 172.21.15.82:0/2020125594 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55ea9421a9d0 0x55ea94283d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 >> 172.21.15.82:0/2020125594 conn(0x55ea941cd020 msgr2=0x55ea94219960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d4d7fa640 0 client.6651 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 shutdown_connections 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 1 -- 172.21.15.82:0/2020125594 wait complete. 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d780b3000 20 client.6651 trim_cache size 0 max 16384 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "35cbd68a-00a5-4591-8708-3cc5cd30cc09": { 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "35cbd68a-00a5-4591-8708-3cc5cd30cc09", 2024-04-25T10:05:27.907 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive" 2024-04-25T10:05:27.908 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:27.908 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:27.908 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 msgr2=0x7f6d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 0x7f6d50069420 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6d7014d1d0 tx=0x7f6d6000b040 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 msgr2=0x7f6d70157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6d5800b3f0 tx=0x7f6d5800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 shutdown_connections 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6d50066f60 0x7f6d50069420 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6d700b5650 0x7f6d700b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6d700b5270 0x7f6d70157b70 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 --2- 172.21.15.82:0/1162174192 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6d7006d090 0x7f6d70153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 >> 172.21.15.82:0/1162174192 conn(0x7f6d7008b000 msgr2=0x7f6d700d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 shutdown_connections 2024-04-25T10:05:27.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:27.898+0000 7f6d74f46640 1 -- 172.21.15.82:0/1162174192 wait complete. 2024-04-25T10:05:27.916 INFO:tasks.cephfs.filesystem:scrub status for tag:35cbd68a-00a5-4591-8708-3cc5cd30cc09 - {'path': '/', 'tag': '35cbd68a-00a5-4591-8708-3cc5cd30cc09', 'options': 'recursive'} 2024-04-25T10:05:30.396 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:05:30.390+0000 7ff48c596640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:05:36.710 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:36.713 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:57.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:57.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:05:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 Processor -- start 2024-04-25T10:05:58.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 -- start start 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98f40d1320 0x7f98f40d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98f40d3c20 0x7f98f406c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f40d1950 con 0x7f98f415d9c0 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40d1720 con 0x7f98f40d3c20 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f8a13640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f40d2520 con 0x7f98f40d1320 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98f40d3c20 0x7f98f406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f98f40d3c20 0x7f98f406c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41060/0 (socket says 172.21.15.82:41060) 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 -- 172.21.15.82:0/1200265243 learned_addr learned my addr 172.21.15.82:0/1200265243 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/1200265243 >> v2:172.21.15.196:3300/0 conn(0x7f98f40d1320 0x7f98f40d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 -- 172.21.15.82:0/1200265243 >> v2:172.21.15.196:3300/0 conn(0x7f98f40d1320 msgr2=0x7f98f40d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/1200265243 >> v2:172.21.15.196:3300/0 conn(0x7f98f40d1320 0x7f98f40d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 -- 172.21.15.82:0/1200265243 >> v1:172.21.15.196:6789/0 conn(0x7f98f415d9c0 legacy=0x7f98f40cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.050+0000 7f98f2d9d640 1 -- 172.21.15.82:0/1200265243 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40d1bf0 con 0x7f98f40d3c20 2024-04-25T10:05:58.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/1200265243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f406c590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f98f40d26a0 tx=0x7f98ec01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2c66cac970fac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f159a640 1 -- 172.21.15.82:0/1200265243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0a5760 con 0x7f98f40d3c20 2024-04-25T10:05:58.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f159a640 1 -- 172.21.15.82:0/1200265243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec0a5900 con 0x7f98f40d3c20 2024-04-25T10:05:58.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f159a640 1 -- 172.21.15.82:0/1200265243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0a5cd0 con 0x7f98f40d3c20 2024-04-25T10:05:58.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 -- 172.21.15.82:0/1200265243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 msgr2=0x7f98f406c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1200265243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f406c590 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f98f40d26a0 tx=0x7f98ec01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 -- 172.21.15.82:0/1200265243 shutdown_connections 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1200265243 >> v2:172.21.15.196:3300/0 conn(0x7f98f40d1320 0x7f98f40d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1200265243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f406c590 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 -- 172.21.15.82:0/1200265243 >> 172.21.15.82:0/1200265243 conn(0x7f98f409cad0 msgr2=0x7f98f408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 -- 172.21.15.82:0/1200265243 shutdown_connections 2024-04-25T10:05:58.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.054+0000 7f98f8a13640 1 -- 172.21.15.82:0/1200265243 wait complete. 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 Processor -- start 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- start start 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98f40d1320 0x7f98f40b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98f415d9c0 0x7f98f4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f406d460 con 0x7f98f40d3c20 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f406d2e0 con 0x7f98f40d1320 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f406d5e0 con 0x7f98f415d9c0 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41070/0 (socket says 172.21.15.82:41070) 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 -- 172.21.15.82:0/1790240428 learned_addr learned my addr 172.21.15.82:0/1790240428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f259c640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98f40d1320 0x7f98f40b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98f415d9c0 0x7f98f4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 -- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98f415d9c0 msgr2=0x7f98f4154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98f415d9c0 0x7f98f4154d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 -- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98f40d1320 msgr2=0x7f98f40b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98f40d1320 0x7f98f40b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 -- 172.21.15.82:0/1790240428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f40b8bc0 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d800 tx=0x7f98e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98e2ffd640 1 -- 172.21.15.82:0/1790240428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e40059a0 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98e2ffd640 1 -- 172.21.15.82:0/1790240428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4005b40 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f40b8ea0 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98e2ffd640 1 -- 172.21.15.82:0/1790240428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4005d30 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f40b70f0 con 0x7f98f40d3c20 2024-04-25T10:05:58.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.058+0000 7f98e2ffd640 1 -- 172.21.15.82:0/1790240428 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98e401b020 con 0x7f98f40d3c20 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98e2ffd640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 0x7f98d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98e2ffd640 1 -- 172.21.15.82:0/1790240428 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f98e4084fd0 con 0x7f98f40d3c20 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f259c640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 0x7f98d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f259c640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 0x7f98d4069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98f40c6da0 tx=0x7f98e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 Processor -- start 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- start start 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9c0adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bc10a0 0x55acf9bc1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf9a0dff0 con 0x55acf9c0b2f0 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf9996150 con 0x55acf9c0a9d0 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf9a0a090 con 0x55acf9bc10a0 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41076/0 (socket says 172.21.15.82:41076) 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3491162848 learned_addr learned my addr 172.21.15.82:0/3491162848 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bc10a0 msgr2=0x55acf9bc1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bc10a0 0x55acf9bc1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 msgr2=0x55acf9c0adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9c0adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3491162848 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf9ba40d0 con 0x55acf9c0b2f0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f98e4083310 tx=0x7f98e4055170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175d9c1827f00234 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98d3fff640 1 -- 172.21.15.82:0/3491162848 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4081070 con 0x55acf9c0b2f0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98d3fff640 1 -- 172.21.15.82:0/3491162848 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e40598a0 con 0x55acf9c0b2f0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98d3fff640 1 -- 172.21.15.82:0/3491162848 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e406d070 con 0x55acf9c0b2f0 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- 172.21.15.82:0/3491162848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 msgr2=0x55acf9bc0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f98e4083310 tx=0x7f98e4055170 comp rx=0 tx=0).stop 2024-04-25T10:05:58.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- 172.21.15.82:0/3491162848 shutdown_connections 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bc10a0 0x55acf9bc1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9c0b2f0 0x55acf9bc0b60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3491162848 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9c0adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- 172.21.15.82:0/3491162848 >> 172.21.15.82:0/3491162848 conn(0x55acf9bbd020 msgr2=0x55acf9bbf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- 172.21.15.82:0/3491162848 shutdown_connections 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.062+0000 7f98fbb80000 1 -- 172.21.15.82:0/3491162848 wait complete. 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- start start 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 Processor -- start 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9befd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bf0270 0x55acf9bf0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf99d2250 con 0x55acf9bc10a0 2024-04-25T10:05:58.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf99c8690 con 0x55acf9bf0270 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf9996a40 con 0x55acf9c0a9d0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 10 client.0 _ll_register_callbacks cb 0x55acf9babe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41084/0 (socket says 172.21.15.82:41084) 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 -- 172.21.15.82:0/3288024988 learned_addr learned my addr 172.21.15.82:0/3288024988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9befd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bf0270 0x55acf9bf0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 -- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 msgr2=0x55acf9befd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9befd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 -- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bf0270 msgr2=0x55acf9bf0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bf0270 0x55acf9bf0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 -- 172.21.15.82:0/3288024988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acf9abb1c0 con 0x55acf9bc10a0 2024-04-25T10:05:58.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f259c640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f98e800dd60 tx=0x7f98e800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8527a3834623a0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8004020 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f98e80041c0 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98e40598a0 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e8004400 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 10 client.6663 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55acf9ab7170 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e40598a0 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98e802c080 con 0x55acf9bc10a0 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 10 client.6663 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 10 client.6663 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 20 client.6663 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:58.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9bf0f20 0x55acf9c1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 4 client.6663 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf99a9c60 con 0x55acf9bf0f20 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9bf0f20 0x55acf9c1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 10 client.6663 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9bf0f20 0x55acf9c1d160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac0d9a4ab1f1843 server_cookie=f894dcb287faf437 in_seq=0 out_seq=0 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 1 -- 172.21.15.82:0/3288024988 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acf99a9c60 con 0x55acf9bf0f20 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 10 client.6663 handle_command_reply: tid=0 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 client.6663 shutdown 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 msgr2=0x55acf9bef7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f98e800dd60 tx=0x7f98e800bf90 comp rx=0 tx=0).stop 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 shutdown_connections 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9bf0f20 0x55acf9c1d160 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9befd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bf0270 0x55acf9bf0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3288024988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bef7f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98d17fa640 0 client.6663 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 >> 172.21.15.82:0/3288024988 conn(0x55acf9bbd020 msgr2=0x55acf9bbfca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 shutdown_connections 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 1 -- 172.21.15.82:0/3288024988 wait complete. 2024-04-25T10:05:58.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.066+0000 7f98fbb80000 20 client.6663 trim_cache size 0 max 16384 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 Processor -- start 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 -- start start 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bf0270 0x55acf9beece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9bef220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f42031a0 con 0x55acf9bc10a0 2024-04-25T10:05:58.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98f259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41086/0 (socket says 172.21.15.82:41086) 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98f259c640 1 -- 172.21.15.82:0/4100466124 learned_addr learned my addr 172.21.15.82:0/4100466124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f42022c0 con 0x55acf9c0a9d0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.082+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4158210 con 0x55acf9bf0270 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9bef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bf0270 0x55acf9beece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 -- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bf0270 msgr2=0x55acf9beece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bf0270 0x55acf9beece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 -- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 msgr2=0x55acf9bef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9bef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 -- 172.21.15.82:0/4100466124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf9bf09c0 con 0x55acf9bc10a0 2024-04-25T10:05:58.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f98e80059a0 tx=0x7f98e801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3087174ba7f91366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d37fe640 1 -- 172.21.15.82:0/4100466124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8030070 con 0x55acf9bc10a0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d37fe640 1 -- 172.21.15.82:0/4100466124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e8034600 con 0x55acf9bc10a0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d37fe640 1 -- 172.21.15.82:0/4100466124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e80349b0 con 0x55acf9bc10a0 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 msgr2=0x55acf9bee7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f98e80059a0 tx=0x7f98e801d000 comp rx=0 tx=0).stop 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 shutdown_connections 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9bf0270 0x55acf9beece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9c0a9d0 0x55acf9bef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- 172.21.15.82:0/4100466124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bc10a0 0x55acf9bee7a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 >> 172.21.15.82:0/4100466124 conn(0x55acf9bbd020 msgr2=0x55acf9c17730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 shutdown_connections 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/4100466124 wait complete. 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- start start 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 Processor -- start 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 0x55acf9bc0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9c1e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf99a8e00 con 0x55acf9bf0270 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f4201cf0 con 0x55acf9bc10a0 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acf9a0d300 con 0x55acf9c0a9d0 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 10 client.0 _ll_register_callbacks cb 0x55acf9b8c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41088/0 (socket says 172.21.15.82:41088) 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3618840481 learned_addr learned my addr 172.21.15.82:0/3618840481 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f2d9d640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9c1e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 0x55acf9bc0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 msgr2=0x55acf9c1e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9c1e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 msgr2=0x55acf9bc0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 0x55acf9bc0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 -- 172.21.15.82:0/3618840481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acf9a54de0 con 0x55acf9bf0270 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f259c640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 0x55acf9bc0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98f1d9b640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f98e40833c0 tx=0x7f98e4000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec2babfa9dbb3b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e4078060 con 0x55acf9bf0270 2024-04-25T10:05:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f98e401ce20 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98e8034600 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e4073070 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 10 client.6669 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55acf99af8e0 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e4061030 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98e4013070 con 0x55acf9bf0270 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 10 client.6669 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 10 client.6669 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 20 client.6669 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9c1e6f0 0x55acf9c1ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 4 client.6669 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.086+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55acf9a1b530 con 0x55acf9c1e6f0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f259c640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9c1e6f0 0x55acf9c1ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98d17fa640 10 client.6669 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f259c640 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9c1e6f0 0x55acf9c1ead0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628499fa55211d90 server_cookie=bdd711feee6380a1 in_seq=0 out_seq=0 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98d17fa640 1 -- 172.21.15.82:0/3618840481 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55acf9a1b530 con 0x55acf9c1e6f0 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98d17fa640 10 client.6669 handle_command_reply: tid=0 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 client.6669 shutdown 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 msgr2=0x55acf9bc0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f98e40833c0 tx=0x7f98e4000bd0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 shutdown_connections 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55acf9c1e6f0 0x55acf9c1ead0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55acf9c0a9d0 0x55acf9c1e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98d17fa640 0 client.6669 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55acf9bf0270 0x55acf9bc0860 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 --2- 172.21.15.82:0/3618840481 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55acf9bc10a0 0x55acf9bc0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 >> 172.21.15.82:0/3618840481 conn(0x55acf9bbd020 msgr2=0x55acf9c17fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 shutdown_connections 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 1 -- 172.21.15.82:0/3618840481 wait complete. 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98fbb80000 20 client.6669 trim_cache size 0 max 16384 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:05:58.102 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 msgr2=0x7f98d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 0x7f98d4069420 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98f40c6da0 tx=0x7f98e8008040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 msgr2=0x7f98f40b6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f98e400d800 tx=0x7f98e400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 shutdown_connections 2024-04-25T10:05:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f98d4066f60 0x7f98d4069420 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98f415d9c0 0x7f98f4154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f98f40d3c20 0x7f98f40b6bb0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 --2- 172.21.15.82:0/1790240428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98f40d1320 0x7f98f40b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 >> 172.21.15.82:0/1790240428 conn(0x7f98f409cad0 msgr2=0x7f98f40ca820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 shutdown_connections 2024-04-25T10:05:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.090+0000 7f98f8a13640 1 -- 172.21.15.82:0/1790240428 wait complete. 2024-04-25T10:05:58.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:05:58.110 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:05:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 Processor -- start 2024-04-25T10:05:58.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 -- start start 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd640c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efd640cb1e0 0x7efd64161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd640cec00 con 0x7efd640cf9c0 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640c73a0 con 0x7efd640cb1e0 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd694a0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cf530 con 0x7efd6415d970 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd627fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efd640cf9c0 0x7efd640caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49690/0 (socket says 172.21.15.82:49690) 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd627fc640 1 -- 172.21.15.82:0/391700915 learned_addr learned my addr 172.21.15.82:0/391700915 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd62ffd640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd640c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 --2- 172.21.15.82:0/391700915 >> v2:172.21.15.196:3300/0 conn(0x7efd640cb1e0 0x7efd64161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3117823013 0 0) 0x7efd640cec00 con 0x7efd640cf9c0 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd4c003530 con 0x7efd640cf9c0 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208181582 0 0) 0x7efd4c003530 con 0x7efd640cf9c0 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 msgr2=0x7efd640c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd640c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 >> v2:172.21.15.196:3300/0 conn(0x7efd640cb1e0 msgr2=0x7efd64161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 --2- 172.21.15.82:0/391700915 >> v2:172.21.15.196:3300/0 conn(0x7efd640cb1e0 0x7efd64161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd640ceee0 con 0x7efd640cf9c0 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7efd58001680 con 0x7efd640cf9c0 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd62ffd640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd640c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 >> v1:172.21.15.82:6789/0 conn(0x7efd640cf9c0 legacy=0x7efd640caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4c004d50 0x7efd4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cf530 con 0x7efd4c004d50 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c00b9d0 con 0x7efd4c003a30 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4c00bb50 con 0x7efd6415d970 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4c004d50 0x7efd4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd627fc640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd62ffd640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 -- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 msgr2=0x7efd4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 -- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd4c003a30 msgr2=0x7efd4c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd637fe640 1 -- 172.21.15.82:0/391700915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c003530 con 0x7efd4c004d50 2024-04-25T10:05:58.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd627fc640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd6415d970 0x7efd4c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.218+0000 7efd62ffd640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd4c003a30 0x7efd4c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd637fe640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4c004d50 0x7efd4c0051d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00e170 tx=0x7efd5c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3e80df5a0451d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c0b2070 con 0x7efd4c004d50 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c0a5cb0 con 0x7efd4c004d50 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd61ffb640 1 -- 172.21.15.82:0/391700915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c009c10 con 0x7efd4c004d50 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- 172.21.15.82:0/391700915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4c004d50 msgr2=0x7efd4c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 --2- 172.21.15.82:0/391700915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd4c004d50 0x7efd4c0051d0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00e170 tx=0x7efd5c01cca0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd637fe640 1 -- 172.21.15.82:0/391700915 reap_dead start 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- 172.21.15.82:0/391700915 shutdown_connections 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- 172.21.15.82:0/391700915 >> 172.21.15.82:0/391700915 conn(0x7efd64099a70 msgr2=0x7efd640d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- 172.21.15.82:0/391700915 shutdown_connections 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- 172.21.15.82:0/391700915 wait complete. 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 Processor -- start 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.222+0000 7efd694a0640 1 -- start start 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640ca2b0 0x7efd640c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 0x7efd64153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cb670 con 0x7efd640c9ed0 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6415dd50 con 0x7efd640ca2b0 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640cb7f0 con 0x7efd64153090 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41110/0 (socket says 172.21.15.82:41110) 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 -- 172.21.15.82:0/3899504668 learned_addr learned my addr 172.21.15.82:0/3899504668 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd627fc640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640ca2b0 0x7efd640c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd637fe640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 0x7efd64153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 -- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 msgr2=0x7efd64153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 0x7efd64153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 -- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640ca2b0 msgr2=0x7efd640c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640ca2b0 0x7efd640c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 -- 172.21.15.82:0/3899504668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6409e070 con 0x7efd640c9ed0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd637fe640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 0x7efd64153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd62ffd640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efd54010860 tx=0x7efd54010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5400f040 con 0x7efd640c9ed0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5400eb60 con 0x7efd640c9ed0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5401c400 con 0x7efd640c9ed0 2024-04-25T10:05:58.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd640c8ab0 con 0x7efd640c9ed0 2024-04-25T10:05:58.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd640c9070 con 0x7efd640c9ed0 2024-04-25T10:05:58.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd6404efa0 con 0x7efd640c9ed0 2024-04-25T10:05:58.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd5401c5a0 con 0x7efd640c9ed0 2024-04-25T10:05:58.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 0x7efd50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7efd54085cb0 con 0x7efd640c9ed0 2024-04-25T10:05:58.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.226+0000 7efd627fc640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 0x7efd50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.230+0000 7efd627fc640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 0x7efd50069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efd580017f0 tx=0x7efd58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.230+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd54018030 con 0x7efd640c9ed0 2024-04-25T10:05:58.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.414+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd64099ff0 con 0x7efd640c9ed0 2024-04-25T10:05:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.414+0000 7efd4b7fe640 1 -- 172.21.15.82:0/3899504668 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4936 (secure 0 0 0) 0x7efd54056b10 con 0x7efd640c9ed0 2024-04-25T10:05:58.420 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:58.420 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:05:30.293038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:05:58.420 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 33 2024-04-25T10:05:58.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 msgr2=0x7efd50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 0x7efd50069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7efd580017f0 tx=0x7efd58009040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 msgr2=0x7efd640c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7efd54010860 tx=0x7efd54010d20 comp rx=0 tx=0).stop 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 shutdown_connections 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd50066f60 0x7efd50069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd64153090 0x7efd64153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efd640ca2b0 0x7efd640c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 --2- 172.21.15.82:0/3899504668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efd640c9ed0 0x7efd640c7f60 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 >> 172.21.15.82:0/3899504668 conn(0x7efd64099a70 msgr2=0x7efd640cba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 shutdown_connections 2024-04-25T10:05:58.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.418+0000 7efd694a0640 1 -- 172.21.15.82:0/3899504668 wait complete. 2024-04-25T10:05:58.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:05:58.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.550+0000 7f92d392b640 1 Processor -- start 2024-04-25T10:05:58.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.550+0000 7f92d392b640 1 -- start start 2024-04-25T10:05:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 0x7f92cc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 0x7f92cc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0ca780 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc0cd440 con 0x7f92cc0cf970 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d392b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0cdce0 con 0x7f92cc0cf590 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41120/0 (socket says 172.21.15.82:41120) 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3470781725 learned_addr learned my addr 172.21.15.82:0/3470781725 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/3470781725 >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 0x7f92cc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d16c7640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 0x7f92cc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3470781725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 msgr2=0x7f92cc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 0x7f92cc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3470781725 >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 msgr2=0x7f92cc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3470781725 >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 0x7f92cc0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3470781725 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0ca4c0 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d16c7640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 0x7f92cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.554+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/3470781725 >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 0x7f92cc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:05:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0d0110 tx=0x7f92c401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc1f22fceff537f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92c3fff640 1 -- 172.21.15.82:0/3470781725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40a5580 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92c3fff640 1 -- 172.21.15.82:0/3470781725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40a5720 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92c3fff640 1 -- 172.21.15.82:0/3470781725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40a5af0 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- 172.21.15.82:0/3470781725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0c9ef0 msgr2=0x7f92cc0b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f92cc0d0110 tx=0x7f92c401c700 comp rx=0 tx=0).stop 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- 172.21.15.82:0/3470781725 shutdown_connections 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0c9ef0 0x7f92cc0b21b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- 172.21.15.82:0/3470781725 >> v2:172.21.15.196:3300/0 conn(0x7f92cc0cf970 0x7f92cc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- 172.21.15.82:0/3470781725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0cf590 0x7f92cc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- 172.21.15.82:0/3470781725 >> 172.21.15.82:0/3470781725 conn(0x7f92cc06ae60 msgr2=0x7f92cc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- 172.21.15.82:0/3470781725 shutdown_connections 2024-04-25T10:05:58.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- 172.21.15.82:0/3470781725 wait complete. 2024-04-25T10:05:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 Processor -- start 2024-04-25T10:05:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- start start 2024-04-25T10:05:58.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0c9ef0 0x7f92cc06b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92cc0cf590 0x7f92cc06eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0ac2f0 con 0x7f92cc0cf970 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0ac170 con 0x7f92cc0cf590 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d392b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0ce150 con 0x7f92cc0c9ef0 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.558+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41128/0 (socket says 172.21.15.82:41128) 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 -- 172.21.15.82:0/2262645204 learned_addr learned my addr 172.21.15.82:0/2262645204 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 -- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0c9ef0 msgr2=0x7f92cc06b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92cc0cf590 0x7f92cc06eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0c9ef0 0x7f92cc06b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 -- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92cc0cf590 msgr2=0x7f92cc06eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92cc0cf590 0x7f92cc06eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 -- 172.21.15.82:0/2262645204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0cb340 con 0x7f92cc0cf970 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f92c40af040 tx=0x7f92c40194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 -- 172.21.15.82:0/2262645204 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4009510 con 0x7f92cc0cf970 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 -- 172.21.15.82:0/2262645204 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40be070 con 0x7f92cc0cf970 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc1652b0 con 0x7f92cc0cf970 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 -- 172.21.15.82:0/2262645204 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40b88b0 con 0x7f92cc0cf970 2024-04-25T10:05:58.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc164fd0 con 0x7f92cc0cf970 2024-04-25T10:05:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 -- 172.21.15.82:0/2262645204 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92c40b8a50 con 0x7f92cc0cf970 2024-04-25T10:05:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 0x7f92b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92c1ffb640 1 -- 172.21.15.82:0/2262645204 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f92c4120590 con 0x7f92cc0cf970 2024-04-25T10:05:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d16c7640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 0x7f92b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d16c7640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 0x7f92b0069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009710 tx=0x7f92bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d6a98000 1 Processor -- start 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.562+0000 7f92d6a98000 1 -- start start 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe689e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68543a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe673bba0 con 0x55cbe68548e0 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe66a1090 con 0x55cbe689ec00 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe6629160 con 0x55cbe689e2e0 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41132/0 (socket says 172.21.15.82:41132) 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3154660337 learned_addr learned my addr 172.21.15.82:0/3154660337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 msgr2=0x55cbe689e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe689e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 msgr2=0x55cbe68543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3154660337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe674ac20 con 0x55cbe68548e0 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f92c40ed0e0 tx=0x7f92c4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d561af0daad42f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92b6ffd640 1 -- 172.21.15.82:0/3154660337 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40f0c80 con 0x55cbe68548e0 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92b6ffd640 1 -- 172.21.15.82:0/3154660337 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40f4dd0 con 0x55cbe68548e0 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92b6ffd640 1 -- 172.21.15.82:0/3154660337 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40f9390 con 0x55cbe68548e0 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3154660337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 msgr2=0x55cbe6854cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f92c40ed0e0 tx=0x7f92c4009810 comp rx=0 tx=0).stop 2024-04-25T10:05:58.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3154660337 shutdown_connections 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe68548e0 0x55cbe6854cc0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3154660337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe689e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3154660337 >> 172.21.15.82:0/3154660337 conn(0x55cbe6850890 msgr2=0x55cbe6852c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3154660337 shutdown_connections 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3154660337 wait complete. 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- start start 2024-04-25T10:05:58.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 Processor -- start 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe68548e0 0x55cbe685f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe685f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe669d130 con 0x55cbe689ec00 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe6665130 con 0x55cbe68548e0 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe675f300 con 0x55cbe689e2e0 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 10 client.0 _ll_register_callbacks cb 0x55cbe683fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41140/0 (socket says 172.21.15.82:41140) 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3910879468 learned_addr learned my addr 172.21.15.82:0/3910879468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d16c7640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe68548e0 0x55cbe685f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe685f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 msgr2=0x55cbe685f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe685f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe68548e0 msgr2=0x55cbe685f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe68548e0 0x55cbe685f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3910879468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe667e060 con 0x55cbe689ec00 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f92c40f7230 tx=0x7f92c401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c091c06a3f9b32a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92abfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40f0c80 con 0x55cbe689ec00 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f92c40eef80 con 0x55cbe689ec00 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92c40f4dd0 con 0x55cbe689ec00 2024-04-25T10:05:58.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c411e3e0 con 0x55cbe689ec00 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 10 client.6687 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55cbe66427c0 con 0x55cbe689ec00 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.566+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40f4dd0 con 0x55cbe689ec00 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92c4111070 con 0x55cbe689ec00 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 10 client.6687 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 10 client.6687 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 20 client.6687 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe68607d0 0x55cbe68af850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 4 client.6687 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbe663ccd0 con 0x55cbe68607d0 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d16c7640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe68607d0 0x55cbe68af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92abfff640 10 client.6687 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d16c7640 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe68607d0 0x55cbe68af850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e4bcfcfedabbc server_cookie=3afeffe38aa82213 in_seq=0 out_seq=0 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92abfff640 1 -- 172.21.15.82:0/3910879468 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cbe663ccd0 con 0x55cbe68607d0 2024-04-25T10:05:58.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92abfff640 10 client.6687 handle_command_reply: tid=0 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 client.6687 shutdown 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 msgr2=0x55cbe685ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f92c40f7230 tx=0x7f92c401c700 comp rx=0 tx=0).stop 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 shutdown_connections 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe68607d0 0x55cbe68af850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe689e2e0 0x55cbe685f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe685ff00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92abfff640 0 client.6687 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3910879468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe68548e0 0x55cbe685f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 >> 172.21.15.82:0/3910879468 conn(0x55cbe6850890 msgr2=0x55cbe6853510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 shutdown_connections 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 1 -- 172.21.15.82:0/3910879468 wait complete. 2024-04-25T10:05:58.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.570+0000 7f92d6a98000 20 client.6687 trim_cache size 0 max 16384 2024-04-25T10:05:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 Processor -- start 2024-04-25T10:05:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- start start 2024-04-25T10:05:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe6859a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689e2e0 0x55cbe6853c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc202410 con 0x55cbe689ec00 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc164410 con 0x55cbe689e2e0 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc1531a0 con 0x55cbe68548e0 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41146/0 (socket says 172.21.15.82:41146) 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3049496252 learned_addr learned my addr 172.21.15.82:0/3049496252 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 msgr2=0x55cbe6859a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe6859a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689e2e0 msgr2=0x55cbe6853c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689e2e0 0x55cbe6853c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 -- 172.21.15.82:0/3049496252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbe665cfc0 con 0x55cbe689ec00 2024-04-25T10:05:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f92c40f0c10 tx=0x7f92c4107000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f2df7e4a946a63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92b67fc640 1 -- 172.21.15.82:0/3049496252 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40ecf20 con 0x55cbe689ec00 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92b67fc640 1 -- 172.21.15.82:0/3049496252 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40f4dd0 con 0x55cbe689ec00 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92b67fc640 1 -- 172.21.15.82:0/3049496252 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40f0c80 con 0x55cbe689ec00 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- 172.21.15.82:0/3049496252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 msgr2=0x55cbe68541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f92c40f0c10 tx=0x7f92c4107000 comp rx=0 tx=0).stop 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- 172.21.15.82:0/3049496252 shutdown_connections 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689ec00 0x55cbe68541d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689e2e0 0x55cbe6853c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 --2- 172.21.15.82:0/3049496252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe6859a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- 172.21.15.82:0/3049496252 >> 172.21.15.82:0/3049496252 conn(0x55cbe6850890 msgr2=0x55cbe6858d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- 172.21.15.82:0/3049496252 shutdown_connections 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- 172.21.15.82:0/3049496252 wait complete. 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 -- start start 2024-04-25T10:05:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.582+0000 7f92d6a98000 1 Processor -- start 2024-04-25T10:05:58.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe68fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68b92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe663be70 con 0x55cbe689e2e0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbe6629a50 con 0x55cbe689ec00 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cbe66a03a0 con 0x55cbe68548e0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 10 client.0 _ll_register_callbacks cb 0x55cbe664d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41162/0 (socket says 172.21.15.82:41162) 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 -- 172.21.15.82:0/1531902513 learned_addr learned my addr 172.21.15.82:0/1531902513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68b92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d16c7640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe68fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 -- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 msgr2=0x55cbe68fe6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe68fe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 -- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 msgr2=0x55cbe68b92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68b92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 -- 172.21.15.82:0/1531902513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbe66fa9e0 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d1ec8640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d0ec6640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f92c800dba0 tx=0x7f92c800f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4754414d3bc69705 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8018070 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f92c8014480 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92c40f4dd0 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92c8023050 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 10 client.6693 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55cbe6640630 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c40f4dd0 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f92c8026020 con 0x55cbe689e2e0 2024-04-25T10:05:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 10 client.6693 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 10 client.6693 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 20 client.6693 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe6905500 0x55cbe69078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 4 client.6693 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55cbe66ae5d0 con 0x55cbe6905500 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d16c7640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe6905500 0x55cbe69078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 10 client.6693 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d16c7640 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe6905500 0x55cbe69078f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=385ae5094d7ace5a server_cookie=1effd6d7e0e503e3 in_seq=0 out_seq=0 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 1 -- 172.21.15.82:0/1531902513 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+140 (crc 0 0 0) 0x55cbe66ae5d0 con 0x55cbe6905500 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 10 client.6693 handle_command_reply: tid=0 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 client.6693 shutdown 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 msgr2=0x55cbe68febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f92c800dba0 tx=0x7f92c800f5b0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 shutdown_connections 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55cbe6905500 0x55cbe69078f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cbe689ec00 0x55cbe68b92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92abfff640 0 client.6693 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cbe689e2e0 0x55cbe68febe0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 --2- 172.21.15.82:0/1531902513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cbe68548e0 0x55cbe68fe6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 >> 172.21.15.82:0/1531902513 conn(0x55cbe6850890 msgr2=0x55cbe689db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 shutdown_connections 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 1 -- 172.21.15.82:0/1531902513 wait complete. 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d6a98000 20 client.6693 trim_cache size 0 max 16384 2024-04-25T10:05:58.597 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 531481507, 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir_test/file" 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:58.598 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 msgr2=0x7f92b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 0x7f92b0069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009710 tx=0x7f92bc008040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 msgr2=0x7f92cc0c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f92c40af040 tx=0x7f92c40194a0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 shutdown_connections 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92b0066f60 0x7f92b0069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92cc0cf970 0x7f92cc0c5340 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92cc0cf590 0x7f92cc06eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 --2- 172.21.15.82:0/2262645204 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92cc0c9ef0 0x7f92cc06b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 >> 172.21.15.82:0/2262645204 conn(0x7f92cc06ae60 msgr2=0x7f92cc154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 shutdown_connections 2024-04-25T10:05:58.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.586+0000 7f92d392b640 1 -- 172.21.15.82:0/2262645204 wait complete. 2024-04-25T10:05:58.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:05:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.710+0000 7ff6e8cb6640 1 Processor -- start 2024-04-25T10:05:58.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.710+0000 7ff6e8cb6640 1 -- start start 2024-04-25T10:05:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 0x7ff6e40b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6e4070fa0 0x7ff6e40cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e415bbb0 con 0x7ff6e406e680 2024-04-25T10:05:58.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40ace70 con 0x7ff6e4070fa0 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40cfe80 con 0x7ff6e40cf560 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6e4070fa0 0x7ff6e40cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6e4070fa0 0x7ff6e40cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41164/0 (socket says 172.21.15.82:41164) 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 -- 172.21.15.82:0/4242773019 learned_addr learned my addr 172.21.15.82:0/4242773019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e259c640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 0x7ff6e40b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 -- 172.21.15.82:0/4242773019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 msgr2=0x7ff6e40b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 0x7ff6e40b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 -- 172.21.15.82:0/4242773019 >> v1:172.21.15.196:6789/0 conn(0x7ff6e406e680 legacy=0x7ff6e40ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 -- 172.21.15.82:0/4242773019 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40ac280 con 0x7ff6e4070fa0 2024-04-25T10:05:58.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e259c640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 0x7ff6e40b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e1d9b640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e4070fa0 0x7ff6e40cf940 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009870 tx=0x7ff6d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9745d4d87656281 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e159a640 1 -- 172.21.15.82:0/4242773019 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80b1070 con 0x7ff6e4070fa0 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e159a640 1 -- 172.21.15.82:0/4242773019 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80ac630 con 0x7ff6e4070fa0 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e159a640 1 -- 172.21.15.82:0/4242773019 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80a8050 con 0x7ff6e4070fa0 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/4242773019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e4070fa0 msgr2=0x7ff6e40cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e4070fa0 0x7ff6e40cf940 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff6d8009870 tx=0x7ff6d801c520 comp rx=0 tx=0).stop 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/4242773019 shutdown_connections 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e4070fa0 0x7ff6e40cf940 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/4242773019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e40cf560 0x7ff6e40b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/4242773019 >> 172.21.15.82:0/4242773019 conn(0x7ff6e406ae60 msgr2=0x7ff6e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/4242773019 shutdown_connections 2024-04-25T10:05:58.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.714+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/4242773019 wait complete. 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 Processor -- start 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- start start 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e406e680 0x7ff6e4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e4070fa0 0x7ff6e40ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4073190 con 0x7ff6e40cf560 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e406eb10 con 0x7ff6e406e680 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4073310 con 0x7ff6e4070fa0 2024-04-25T10:05:58.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41166/0 (socket says 172.21.15.82:41166) 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 -- 172.21.15.82:0/1505852564 learned_addr learned my addr 172.21.15.82:0/1505852564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e259c640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e406e680 0x7ff6e4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e1d9b640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e4070fa0 0x7ff6e40ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 -- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e4070fa0 msgr2=0x7ff6e40ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e4070fa0 0x7ff6e40ae330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 -- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e406e680 msgr2=0x7ff6e4155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e406e680 0x7ff6e4155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 -- 172.21.15.82:0/1505852564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40ce450 con 0x7ff6e40cf560 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e2d9d640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b990 tx=0x7ff6dc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc019070 con 0x7ff6e40cf560 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00cba0 con 0x7ff6e40cf560 2024-04-25T10:05:58.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40cea90 con 0x7ff6e40cf560 2024-04-25T10:05:58.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc014440 con 0x7ff6e40cf560 2024-04-25T10:05:58.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e415ae00 con 0x7ff6e40cf560 2024-04-25T10:05:58.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.718+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e404efa0 con 0x7ff6e40cf560 2024-04-25T10:05:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.722+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff6dc01c070 con 0x7ff6e40cf560 2024-04-25T10:05:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.722+0000 7ff6d2ffd640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 0x7ff6c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.722+0000 7ff6e259c640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 0x7ff6c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.722+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7ff6dc084f70 con 0x7ff6e40cf560 2024-04-25T10:05:58.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.722+0000 7ff6e259c640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 0x7ff6c4069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6e414b930 tx=0x7ff6d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:58.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.726+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff6dc055df0 con 0x7ff6e40cf560 2024-04-25T10:05:58.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.902+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff6e40c9530 con 0x7ff6e40cf560 2024-04-25T10:05:58.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.902+0000 7ff6d2ffd640 1 -- 172.21.15.82:0/1505852564 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7ff6dc0556f0 con 0x7ff6e40cf560 2024-04-25T10:05:58.907 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:05:58.907 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:05:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 msgr2=0x7ff6c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 0x7ff6c4069420 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff6e414b930 tx=0x7ff6d4008040 comp rx=0 tx=0).stop 2024-04-25T10:05:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 msgr2=0x7ff6e415a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b990 tx=0x7ff6dc00be50 comp rx=0 tx=0).stop 2024-04-25T10:05:58.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 shutdown_connections 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff6c4066f60 0x7ff6c4069420 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff6e4070fa0 0x7ff6e40ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6e40cf560 0x7ff6e415a8c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 --2- 172.21.15.82:0/1505852564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff6e406e680 0x7ff6e4155c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 >> 172.21.15.82:0/1505852564 conn(0x7ff6e406ae60 msgr2=0x7ff6e40af180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 shutdown_connections 2024-04-25T10:05:58.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:58.906+0000 7ff6e8cb6640 1 -- 172.21.15.82:0/1505852564 wait complete. 2024-04-25T10:05:58.924 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:05:58.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:58.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T10:05:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 Processor -- start 2024-04-25T10:05:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 -- start start 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb6200d12c0 0x7fb6200d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6200d21f0 0x7fb62006cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62015dd60 con 0x7fb62015d980 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62006c810 con 0x7fb6200d21f0 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb62619e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6200d45e0 con 0x7fb6200d12c0 2024-04-25T10:05:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6200d21f0 0x7fb62006cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6200d21f0 0x7fb62006cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41168/0 (socket says 172.21.15.82:41168) 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 -- 172.21.15.82:0/2413622170 learned_addr learned my addr 172.21.15.82:0/2413622170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61f7fe640 1 --1- 172.21.15.82:0/2413622170 >> v1:172.21.15.82:6789/0 conn(0x7fb62015d980 0x7fb6200d3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49698/0 (socket says 172.21.15.82:49698) 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61effd640 1 --2- 172.21.15.82:0/2413622170 >> v2:172.21.15.196:3300/0 conn(0x7fb6200d12c0 0x7fb6200d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 -- 172.21.15.82:0/2413622170 >> v2:172.21.15.196:3300/0 conn(0x7fb6200d12c0 msgr2=0x7fb6200d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 --2- 172.21.15.82:0/2413622170 >> v2:172.21.15.196:3300/0 conn(0x7fb6200d12c0 0x7fb6200d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 -- 172.21.15.82:0/2413622170 >> v1:172.21.15.82:6789/0 conn(0x7fb62015d980 legacy=0x7fb6200d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61ffff640 1 -- 172.21.15.82:0/2413622170 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6200b02c0 con 0x7fb6200d21f0 2024-04-25T10:05:59.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.042+0000 7fb61e7fc640 1 -- 172.21.15.82:0/2413622170 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616716200 0 0) 0x7fb62015dd60 con 0x7fb62015d980 2024-04-25T10:05:59.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb61ffff640 1 --2- 172.21.15.82:0/2413622170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d21f0 0x7fb62006cca0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb6200b61f0 tx=0x7fb60801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519aedc1de86d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb61e7fc640 1 -- 172.21.15.82:0/2413622170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080b0070 con 0x7fb6200d21f0 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb61e7fc640 1 -- 172.21.15.82:0/2413622170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6080ab600 con 0x7fb6200d21f0 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb61e7fc640 1 -- 172.21.15.82:0/2413622170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080ab9d0 con 0x7fb6200d21f0 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- 172.21.15.82:0/2413622170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d21f0 msgr2=0x7fb62006cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 --2- 172.21.15.82:0/2413622170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d21f0 0x7fb62006cca0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fb6200b61f0 tx=0x7fb60801c490 comp rx=0 tx=0).stop 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- 172.21.15.82:0/2413622170 shutdown_connections 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 --2- 172.21.15.82:0/2413622170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d21f0 0x7fb62006cca0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 --2- 172.21.15.82:0/2413622170 >> v2:172.21.15.196:3300/0 conn(0x7fb6200d12c0 0x7fb6200d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- 172.21.15.82:0/2413622170 >> 172.21.15.82:0/2413622170 conn(0x7fb62008b000 msgr2=0x7fb6200d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- 172.21.15.82:0/2413622170 shutdown_connections 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- 172.21.15.82:0/2413622170 wait complete. 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 Processor -- start 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.046+0000 7fb62619e640 1 -- start start 2024-04-25T10:05:59.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6200d21f0 0x7fb620154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb62015d980 0x7fb6200ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62006db70 con 0x7fb6200d12c0 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62006d9f0 con 0x7fb6200d21f0 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62006dcf0 con 0x7fb62015d980 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41170/0 (socket says 172.21.15.82:41170) 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2080387558 learned_addr learned my addr 172.21.15.82:0/2080387558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61effd640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6200d21f0 0x7fb620154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61ffff640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb62015d980 0x7fb6200ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb62015d980 msgr2=0x7fb6200ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb62015d980 0x7fb6200ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6200d21f0 msgr2=0x7fb620154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6200d21f0 0x7fb620154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2080387558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62015f080 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb610001570 tx=0x7fb61000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 -- 172.21.15.82:0/2080387558 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610018070 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 -- 172.21.15.82:0/2080387558 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61000a640 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb620155fe0 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 -- 172.21.15.82:0/2080387558 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61000aa30 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6201565a0 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 -- 172.21.15.82:0/2080387558 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb61001b070 con 0x7fb6200d12c0 2024-04-25T10:05:59.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 0x7fb600069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61effd640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 0x7fb600069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb617fff640 1 -- 172.21.15.82:0/2080387558 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb610084930 con 0x7fb6200d12c0 2024-04-25T10:05:59.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.050+0000 7fb61effd640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 0x7fb600069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb6201580c0 tx=0x7fb618008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 Processor -- start 2024-04-25T10:05:59.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- start start 2024-04-25T10:05:59.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4aa8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4aa8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e48f5d00 con 0x55a0e4af1520 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e487ddf0 con 0x55a0e4aa8970 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0e48f1da0 con 0x55a0e4af1e40 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41176/0 (socket says 172.21.15.82:41176) 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 -- 172.21.15.82:0/658847153 learned_addr learned my addr 172.21.15.82:0/658847153 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 -- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 msgr2=0x55a0e4aa8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4aa8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 -- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 msgr2=0x55a0e4aa8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4aa8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 -- 172.21.15.82:0/658847153 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e48b0580 con 0x55a0e4af1520 2024-04-25T10:05:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb6100014b0 tx=0x7fb610055150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f261dfd625ebb90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb6157fa640 1 -- 172.21.15.82:0/658847153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610053310 con 0x55a0e4af1520 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb6157fa640 1 -- 172.21.15.82:0/658847153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb610051680 con 0x55a0e4af1520 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb6157fa640 1 -- 172.21.15.82:0/658847153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61005d490 con 0x55a0e4af1520 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- 172.21.15.82:0/658847153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 msgr2=0x55a0e4af1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fb6100014b0 tx=0x7fb610055150 comp rx=0 tx=0).stop 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- 172.21.15.82:0/658847153 shutdown_connections 2024-04-25T10:05:59.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4aa8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4aa8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 --2- 172.21.15.82:0/658847153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af1900 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- 172.21.15.82:0/658847153 >> 172.21.15.82:0/658847153 conn(0x55a0e498fc10 msgr2=0x55a0e4aa6d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- 172.21.15.82:0/658847153 shutdown_connections 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- 172.21.15.82:0/658847153 wait complete. 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 -- start start 2024-04-25T10:05:59.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.054+0000 7fb62930b000 1 Processor -- start 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1520 0x55a0e4af8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4aacf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e488e6d0 con 0x55a0e4aa8970 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e48b0950 con 0x55a0e4af1e40 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0e48f5010 con 0x55a0e4af1520 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 10 client.0 _ll_register_callbacks cb 0x55a0e4a94030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:59.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41188/0 (socket says 172.21.15.82:41188) 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2261363397 learned_addr learned my addr 172.21.15.82:0/2261363397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61ffff640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4aacf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61effd640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1520 0x55a0e4af8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1520 msgr2=0x55a0e4af8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1520 0x55a0e4af8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 msgr2=0x55a0e4aacf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4aacf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 -- 172.21.15.82:0/2261363397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e4892d40 con 0x55a0e4aa8970 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb610007fd0 tx=0x7fb610001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f287b86f8ea8e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb610054670 con 0x55a0e4aa8970 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb61005b100 con 0x55a0e4aa8970 2024-04-25T10:05:59.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb610051680 con 0x55a0e4aa8970 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb610079050 con 0x55a0e4aa8970 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 10 client.6714 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55a0e48f0cf0 con 0x55a0e4aa8970 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb610051680 con 0x55a0e4aa8970 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb61007c020 con 0x55a0e4aa8970 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 10 client.6714 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:59.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 10 client.6714 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 20 client.6714 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4aadca0 0x55a0e4aae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 4 client.6714 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0e48919d0 con 0x55a0e4aadca0 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61effd640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4aadca0 0x55a0e4aae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 10 client.6714 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb61effd640 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4aadca0 0x55a0e4aae080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdfa38307519ecd server_cookie=76670480f11960e in_seq=0 out_seq=0 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/2261363397 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0e48919d0 con 0x55a0e4aadca0 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 10 client.6714 handle_command_reply: tid=0 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 client.6714 shutdown 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 msgr2=0x55a0e4af7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb610007fd0 tx=0x7fb610001260 comp rx=0 tx=0).stop 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 shutdown_connections 2024-04-25T10:05:59.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4aadca0 0x55a0e4aae080 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1520 0x55a0e4af8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4aacf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 --2- 172.21.15.82:0/2261363397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4aa8970 0x55a0e4af7bd0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 >> 172.21.15.82:0/2261363397 conn(0x55a0e498fc10 msgr2=0x55a0e4aa7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb5fe7fc640 0 client.6714 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 shutdown_connections 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 1 -- 172.21.15.82:0/2261363397 wait complete. 2024-04-25T10:05:59.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.058+0000 7fb62930b000 20 client.6714 trim_cache size 0 max 16384 2024-04-25T10:05:59.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 Processor -- start 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 -- start start 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4aa8970 0x55a0e4af3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4af3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb620203180 con 0x55a0e4af1520 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6202022a0 con 0x55a0e4af1e40 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb620160740 con 0x55a0e4aa8970 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41190/0 (socket says 172.21.15.82:41190) 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 -- 172.21.15.82:0/4034222614 learned_addr learned my addr 172.21.15.82:0/4034222614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61ffff640 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4af3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4aa8970 0x55a0e4af3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 -- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4aa8970 msgr2=0x55a0e4af3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4aa8970 0x55a0e4af3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 -- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 msgr2=0x55a0e4af3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4af3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 -- 172.21.15.82:0/4034222614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0e4af1c70 con 0x55a0e4af1520 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb61effd640 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fb618004420 tx=0x7fb618014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2847f17a7592bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb614ff9640 1 -- 172.21.15.82:0/4034222614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61800f040 con 0x55a0e4af1520 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb614ff9640 1 -- 172.21.15.82:0/4034222614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61802a070 con 0x55a0e4af1520 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.074+0000 7fb614ff9640 1 -- 172.21.15.82:0/4034222614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618003e60 con 0x55a0e4af1520 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4034222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 msgr2=0x55a0e4af3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fb618004420 tx=0x7fb618014000 comp rx=0 tx=0).stop 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4034222614 shutdown_connections 2024-04-25T10:05:59.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4af1e40 0x55a0e4af3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4af3770 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- 172.21.15.82:0/4034222614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4aa8970 0x55a0e4af3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4034222614 >> 172.21.15.82:0/4034222614 conn(0x55a0e498fc10 msgr2=0x55a0e4aefd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4034222614 shutdown_connections 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4034222614 wait complete. 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- start start 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 Processor -- start 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4af6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4b0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0e48f5010 con 0x55a0e4af1520 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb620201cb0 con 0x55a0e4aa8970 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0e487a560 con 0x55a0e4af1e40 2024-04-25T10:05:59.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 10 client.0 _ll_register_callbacks cb 0x7fb6100512b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41198/0 (socket says 172.21.15.82:41198) 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 -- 172.21.15.82:0/4274985612 learned_addr learned my addr 172.21.15.82:0/4274985612 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4af6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61ffff640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4b0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 -- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 msgr2=0x55a0e4b0c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4b0c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 -- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 msgr2=0x55a0e4af6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4af6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 -- 172.21.15.82:0/4274985612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0e4978d30 con 0x55a0e4af1520 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4af6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61effd640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb6180040d0 tx=0x7fb618004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e5655a6f2ca9ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:59.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6180248f0 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb61800e460 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb61802a070 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb61803e050 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 10 client.6720 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55a0e48c25a0 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61802a070 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb618041020 con 0x55a0e4af1520 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 10 client.6720 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 10 client.6720 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 20 client.6720 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4b0cd70 0x55a0e4b50fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 4 client.6720 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x55a0e48d2610 con 0x55a0e4b0cd70 2024-04-25T10:05:59.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4b0cd70 0x55a0e4b50fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb5fe7fc640 10 client.6720 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.078+0000 7fb61f7fe640 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4b0cd70 0x55a0e4b50fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa411410a41852c0 server_cookie=52b0d61ae4af1039 in_seq=0 out_seq=0 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb5fe7fc640 1 -- 172.21.15.82:0/4274985612 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a0e48d2610 con 0x55a0e4b0cd70 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb5fe7fc640 10 client.6720 handle_command_reply: tid=0 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 client.6720 shutdown 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 msgr2=0x55a0e4b0c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb6180040d0 tx=0x7fb618004740 comp rx=0 tx=0).stop 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 shutdown_connections 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55a0e4b0cd70 0x55a0e4b50fb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a0e4af1e40 0x55a0e4b0c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a0e4af1520 0x55a0e4b0c2f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 --2- 172.21.15.82:0/4274985612 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a0e4aa8970 0x55a0e4af6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 >> 172.21.15.82:0/4274985612 conn(0x55a0e498fc10 msgr2=0x55a0e4af0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb5fe7fc640 0 client.6720 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 shutdown_connections 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 1 -- 172.21.15.82:0/4274985612 wait complete. 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62930b000 20 client.6720 trim_cache size 0 max 16384 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "faeaf6d0-7bdc-45f9-bca0-1d42c610d232", 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:05:59.099 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:59.100 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:05:59.082+0000 7feef658a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:05:26.630910+0000 1=1+0) (iversion lock) caps={6585=pAsLsXsFs/-@2},l=6585(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56489997fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:05:59.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 msgr2=0x7fb600069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 0x7fb600069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb6201580c0 tx=0x7fb618008040 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 msgr2=0x7fb620158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fb610001570 tx=0x7fb61000cd20 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 shutdown_connections 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb600066f60 0x7fb600069420 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb62015d980 0x7fb6200ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6200d21f0 0x7fb620154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 --2- 172.21.15.82:0/2080387558 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6200d12c0 0x7fb620158d50 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 >> 172.21.15.82:0/2080387558 conn(0x7fb62008b000 msgr2=0x7fb6200b65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 shutdown_connections 2024-04-25T10:05:59.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.082+0000 7fb62619e640 1 -- 172.21.15.82:0/2080387558 wait complete. 2024-04-25T10:05:59.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:59.103 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:05:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.214+0000 7fc193f06640 1 Processor -- start 2024-04-25T10:05:59.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.214+0000 7fc193f06640 1 -- start start 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.214+0000 7fc193f06640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc18c15b6b0 0x7fc18c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc193f06640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc18c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc193f06640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c0cbd20 con 0x7fc18c15f130 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc193f06640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c15ba90 con 0x7fc18c15b6b0 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc193f06640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cff30 con 0x7fc18c0cf590 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1924a3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc18c15f130 0x7fc18c15f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:49712/0 (socket says 172.21.15.82:49712) 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1924a3640 1 -- 172.21.15.82:0/238176280 learned_addr learned my addr 172.21.15.82:0/238176280 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc191ca2640 1 --2- 172.21.15.82:0/238176280 >> v2:172.21.15.196:3300/0 conn(0x7fc18c15b6b0 0x7fc18c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc18c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026767848 0 0) 0x7fc18c0cbd20 con 0x7fc18c15f130 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc178003530 con 0x7fc18c15f130 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335755005 0 0) 0x7fc178003530 con 0x7fc18c15f130 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 msgr2=0x7fc18c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc18c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 >> v2:172.21.15.196:3300/0 conn(0x7fc18c15b6b0 msgr2=0x7fc18c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 --2- 172.21.15.82:0/238176280 >> v2:172.21.15.196:3300/0 conn(0x7fc18c15b6b0 0x7fc18c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c0cc000 con 0x7fc18c15f130 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc184001a00 con 0x7fc18c15f130 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 >> v1:172.21.15.82:6789/0 conn(0x7fc18c15f130 legacy=0x7fc18c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc178003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc178003fb0 0x7fc178004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc178004d50 0x7fc1780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c15ba90 con 0x7fc178003fb0 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800b9d0 con 0x7fc178004d50 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800bb80 con 0x7fc18c0cf590 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc178003fb0 0x7fc178004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 -- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 msgr2=0x7fc178003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc191ca2640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc178003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc18c0cf590 0x7fc178003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 -- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc178004d50 msgr2=0x7fc1780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc178004d50 0x7fc1780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 -- 172.21.15.82:0/238176280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178003530 con 0x7fc178003fb0 2024-04-25T10:05:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc1914a1640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc178003fb0 0x7fc178004810 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009ae0 tx=0x7fc17c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab71c6acc7a0985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0b2070 con 0x7fc178003fb0 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0a5660 con 0x7fc178003fb0 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.218+0000 7fc190ca0640 1 -- 172.21.15.82:0/238176280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0096b0 con 0x7fc178003fb0 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/238176280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc178003fb0 msgr2=0x7fc178004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 --2- 172.21.15.82:0/238176280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc178003fb0 0x7fc178004810 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc17c009ae0 tx=0x7fc17c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1924a3640 1 -- 172.21.15.82:0/238176280 reap_dead start 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/238176280 shutdown_connections 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/238176280 >> 172.21.15.82:0/238176280 conn(0x7fc18c06ae60 msgr2=0x7fc18c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/238176280 shutdown_connections 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/238176280 wait complete. 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 Processor -- start 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- start start 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 0x7fc18c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c0b0920 0x7fc18c0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cc8a0 con 0x7fc18c06b4a0 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cc5c0 con 0x7fc18c0b0920 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0cba30 con 0x7fc178003fb0 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41218/0 (socket says 172.21.15.82:41218) 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 -- 172.21.15.82:0/451272094 learned_addr learned my addr 172.21.15.82:0/451272094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc191ca2640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 0x7fc18c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 -- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 msgr2=0x7fc18c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 0x7fc18c156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 -- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c0b0920 msgr2=0x7fc18c0c8b40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c0b0920 0x7fc18c0c8b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 -- 172.21.15.82:0/451272094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c06e580 con 0x7fc18c06b4a0 2024-04-25T10:05:59.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc191ca2640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 0x7fc18c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1914a1640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc17c01c8a0 tx=0x7fc178005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1827fc640 1 -- 172.21.15.82:0/451272094 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0b2040 con 0x7fc18c06b4a0 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1827fc640 1 -- 172.21.15.82:0/451272094 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00f000 con 0x7fc18c06b4a0 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c0cb730 con 0x7fc18c06b4a0 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc1827fc640 1 -- 172.21.15.82:0/451272094 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c004220 con 0x7fc18c06b4a0 2024-04-25T10:05:59.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.222+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c0c4530 con 0x7fc18c06b4a0 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1827fc640 1 -- 172.21.15.82:0/451272094 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc17c0be030 con 0x7fc18c06b4a0 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1827fc640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 0x7fc170069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1827fc640 1 -- 172.21.15.82:0/451272094 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc17c11eaa0 con 0x7fc18c06b4a0 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc191ca2640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 0x7fc170069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc191ca2640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 0x7fc170069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc18c15bd30 tx=0x7fc188008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 Processor -- start 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 -- start start 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9a60a0 0x558c9b9a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9b7f2ff0 con 0x558c9b9f02f0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9b77b150 con 0x558c9b9a60a0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9b7ef090 con 0x558c9b9ef9d0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41234/0 (socket says 172.21.15.82:41234) 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 -- 172.21.15.82:0/2923783215 learned_addr learned my addr 172.21.15.82:0/2923783215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1924a3640 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9a60a0 0x558c9b9a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc191ca2640 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 -- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 msgr2=0x558c9b9efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 -- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9a60a0 msgr2=0x558c9b9a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9a60a0 0x558c9b9a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 -- 172.21.15.82:0/2923783215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9b9890d0 con 0x558c9b9f02f0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc1914a1640 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0eb680 tx=0x7fc17c0ef120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce2d6304d2f1794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc16f7fe640 1 -- 172.21.15.82:0/2923783215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0ef220 con 0x558c9b9f02f0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc16f7fe640 1 -- 172.21.15.82:0/2923783215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0f3370 con 0x558c9b9f02f0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc16f7fe640 1 -- 172.21.15.82:0/2923783215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c0f7930 con 0x558c9b9f02f0 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.226+0000 7fc197073000 1 -- 172.21.15.82:0/2923783215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 msgr2=0x558c9b9a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0eb680 tx=0x7fc17c0ef120 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/2923783215 shutdown_connections 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9a60a0 0x558c9b9a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9f02f0 0x558c9b9a5b60 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/2923783215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/2923783215 >> 172.21.15.82:0/2923783215 conn(0x558c9b9a2020 msgr2=0x558c9b9a4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/2923783215 shutdown_connections 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/2923783215 wait complete. 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- start start 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 Processor -- start 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9ef9d0 0x558c9b9d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9f02f0 0x558c9b9d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9b7b7250 con 0x558c9b9a60a0 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9b7ad690 con 0x558c9b9ef9d0 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9b77ba40 con 0x558c9b9f02f0 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 10 client.0 _ll_register_callbacks cb 0x558c9b990e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41250/0 (socket says 172.21.15.82:41250) 2024-04-25T10:05:59.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 -- 172.21.15.82:0/3478989226 learned_addr learned my addr 172.21.15.82:0/3478989226 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc1924a3640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9f02f0 0x558c9b9d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 -- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9f02f0 msgr2=0x558c9b9d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc1914a1640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9ef9d0 0x558c9b9d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9f02f0 0x558c9b9d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 -- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9ef9d0 msgr2=0x558c9b9d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9ef9d0 0x558c9b9d4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 -- 172.21.15.82:0/3478989226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c9b8a01c0 con 0x558c9b9a60a0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc191ca2640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc188006e30 tx=0x7fc188004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680107aa1739e811 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188025070 con 0x558c9b9a60a0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc188014730 con 0x558c9b9a60a0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc17c0f3370 con 0x558c9b9a60a0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc188030050 con 0x558c9b9a60a0 2024-04-25T10:05:59.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 10 client.6732 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x558c9b89c170 con 0x558c9b9a60a0 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c0f3370 con 0x558c9b9a60a0 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc188033020 con 0x558c9b9a60a0 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 10 client.6732 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 10 client.6732 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 20 client.6732 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9b9ffc80 0x558c9ba02070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 4 client.6732 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c9b78ec60 con 0x558c9b9ffc80 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc1914a1640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9b9ffc80 0x558c9ba02070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 10 client.6732 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc1914a1640 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9b9ffc80 0x558c9ba02070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd2cd42fdfc1264 server_cookie=14f743a6971818c9 in_seq=0 out_seq=0 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 1 -- 172.21.15.82:0/3478989226 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c9b78ec60 con 0x558c9b9ffc80 2024-04-25T10:05:59.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 10 client.6732 handle_command_reply: tid=0 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 client.6732 shutdown 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 msgr2=0x558c9b9d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc188006e30 tx=0x7fc188004740 comp rx=0 tx=0).stop 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 shutdown_connections 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9b9ffc80 0x558c9ba02070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9f02f0 0x558c9b9d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc16cff9640 0 client.6732 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9ef9d0 0x558c9b9d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 --2- 172.21.15.82:0/3478989226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9d47f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.230+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 >> 172.21.15.82:0/3478989226 conn(0x558c9b9a2020 msgr2=0x558c9b9a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.234+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 shutdown_connections 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.234+0000 7fc197073000 1 -- 172.21.15.82:0/3478989226 wait complete. 2024-04-25T10:05:59.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.234+0000 7fc197073000 20 client.6732 trim_cache size 0 max 16384 2024-04-25T10:05:59.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 Processor -- start 2024-04-25T10:05:59.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 -- start start 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9fd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9b9a4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1fe890 con 0x558c9b9a60a0 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1fd9b0 con 0x558c9b9f02f0 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c15dae0 con 0x558c9b9ef9d0 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41264/0 (socket says 172.21.15.82:41264) 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 -- 172.21.15.82:0/1483265753 learned_addr learned my addr 172.21.15.82:0/1483265753 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 -- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 msgr2=0x558c9b9fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 -- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 msgr2=0x558c9b9a4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9b9a4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 -- 172.21.15.82:0/1483265753 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c9b839de0 con 0x558c9b9a60a0 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.246+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc188008010 tx=0x7fc188021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4427af400227b502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16effd640 1 -- 172.21.15.82:0/1483265753 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188025040 con 0x558c9b9a60a0 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16effd640 1 -- 172.21.15.82:0/1483265753 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188039070 con 0x558c9b9a60a0 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16effd640 1 -- 172.21.15.82:0/1483265753 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188014b10 con 0x558c9b9a60a0 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1483265753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 msgr2=0x558c9b9fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc188008010 tx=0x7fc188021000 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1483265753 shutdown_connections 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9b9fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9b9a4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1483265753 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9b9fce40 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1483265753 >> 172.21.15.82:0/1483265753 conn(0x558c9b9a2020 msgr2=0x558c9b9fc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1483265753 shutdown_connections 2024-04-25T10:05:59.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1483265753 wait complete. 2024-04-25T10:05:59.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- start start 2024-04-25T10:05:59.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 Processor -- start 2024-04-25T10:05:59.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9ba54080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9ba0efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9b78de00 con 0x558c9b9a60a0 2024-04-25T10:05:59.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c1fd390 con 0x558c9b9f02f0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c9b7f2300 con 0x558c9b9ef9d0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 10 client.0 _ll_register_callbacks cb 0x558c9b971040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41266/0 (socket says 172.21.15.82:41266) 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 -- 172.21.15.82:0/1900018444 learned_addr learned my addr 172.21.15.82:0/1900018444 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 -- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 msgr2=0x558c9ba54080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc1914a1640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9ba54080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9ba54080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 -- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 msgr2=0x558c9ba0efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc1924a3640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9ba0efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9ba0efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 -- 172.21.15.82:0/1900018444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c9b7948e0 con 0x558c9b9a60a0 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc1914a1640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9ba54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:59.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc191ca2640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc188008010 tx=0x7fc188016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecb774394d84ca12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc188043020 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc188014730 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc188039070 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc18804d050 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 10 client.6738 fetch_fsmap learned FSMap version 33 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x558c9b84c8f0 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc188039070 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc188050020 con 0x558c9b9a60a0 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 10 client.6738 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 10 client.6738 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 20 client.6738 populate_metadata read hostname 'smithi082' 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9ba0f910 0x558c9ba0fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:59.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 4 client.6738 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c9b800530 con 0x558c9ba0f910 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc1914a1640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9ba0f910 0x558c9ba0fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 10 client.6738 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc1914a1640 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9ba0f910 0x558c9ba0fcf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fb0cae980f17c56 server_cookie=3345151dabc6eaea in_seq=0 out_seq=0 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 1 -- 172.21.15.82:0/1900018444 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x558c9b800530 con 0x558c9ba0f910 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc16cff9640 10 client.6738 handle_command_reply: tid=0 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 client.6738 shutdown 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 msgr2=0x558c9ba53b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.250+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc188008010 tx=0x7fc188016000 comp rx=0 tx=0).stop 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 shutdown_connections 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x558c9ba0f910 0x558c9ba0fcf0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558c9b9ef9d0 0x558c9ba54080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558c9b9f02f0 0x558c9ba0efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc16cff9640 0 client.6738 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 --2- 172.21.15.82:0/1900018444 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558c9b9a60a0 0x558c9ba53b40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 >> 172.21.15.82:0/1900018444 conn(0x558c9b9a2020 msgr2=0x558c9b9fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 shutdown_connections 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 1 -- 172.21.15.82:0/1900018444 wait complete. 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc197073000 20 client.6738 trim_cache size 0 max 16384 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:05:59.262 INFO:teuthology.orchestra.run.smithi082.stdout: "faeaf6d0-7bdc-45f9-bca0-1d42c610d232": { 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "faeaf6d0-7bdc-45f9-bca0-1d42c610d232", 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair,force" 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:05:59.263 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 msgr2=0x7fc170069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 0x7fc170069450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc18c15bd30 tx=0x7fc188008040 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 msgr2=0x7fc18c0c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc17c01c8a0 tx=0x7fc178005840 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 shutdown_connections 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc170066f90 0x7fc170069450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc18c0b0920 0x7fc18c0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc18c06b4a0 0x7fc18c0c8600 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 --2- 172.21.15.82:0/451272094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc178003fb0 0x7fc18c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 >> 172.21.15.82:0/451272094 conn(0x7fc18c06ae60 msgr2=0x7fc18c0b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:59.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 shutdown_connections 2024-04-25T10:05:59.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:05:59.254+0000 7fc193f06640 1 -- 172.21.15.82:0/451272094 wait complete. 2024-04-25T10:05:59.272 INFO:tasks.cephfs.filesystem:scrub status for tag:faeaf6d0-7bdc-45f9-bca0-1d42c610d232 - {'path': '/', 'tag': 'faeaf6d0-7bdc-45f9-bca0-1d42c610d232', 'options': 'recursive,repair,force'} 2024-04-25T10:06:06.996 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:06.999 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:29.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:29.273 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T10:06:29.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.402+0000 7fb6452ce640 1 Processor -- start 2024-04-25T10:06:29.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.402+0000 7fb6452ce640 1 -- start start 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb6400cf590 0x7fb6400cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400cbd20 con 0x7fb64015b6b0 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400cc000 con 0x7fb640072000 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64015ba90 con 0x7fb6400cf590 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb64015b6b0 0x7fb6400ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53980/0 (socket says 172.21.15.82:53980) 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63effd640 1 -- 172.21.15.82:0/3480681202 learned_addr learned my addr 172.21.15.82:0/3480681202 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:29.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3480681202 >> v2:172.21.15.82:3300/0 conn(0x7fb6400cf590 0x7fb6400cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3480681202 >> v1:172.21.15.196:6789/0 conn(0x7fb640072000 legacy=0x7fb640072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3480681202 >> v1:172.21.15.82:6789/0 conn(0x7fb64015b6b0 legacy=0x7fb6400ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63dffb640 1 -- 172.21.15.82:0/3480681202 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323854073 0 0) 0x7fb6400cbd20 con 0x7fb64015b6b0 2024-04-25T10:06:29.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3480681202 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400cc2e0 con 0x7fb6400cf590 2024-04-25T10:06:29.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3480681202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6400cf590 0x7fb6400cfa20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1c1d591728bcb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63dffb640 1 -- 172.21.15.82:0/3480681202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340b0070 con 0x7fb6400cf590 2024-04-25T10:06:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63dffb640 1 -- 172.21.15.82:0/3480681202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb634004230 con 0x7fb6400cf590 2024-04-25T10:06:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb63dffb640 1 -- 172.21.15.82:0/3480681202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634003620 con 0x7fb6400cf590 2024-04-25T10:06:29.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- 172.21.15.82:0/3480681202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6400cf590 msgr2=0x7fb6400cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 --2- 172.21.15.82:0/3480681202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6400cf590 0x7fb6400cfa20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fb6340097c0 tx=0x7fb63401c460 comp rx=0 tx=0).stop 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- 172.21.15.82:0/3480681202 shutdown_connections 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 --2- 172.21.15.82:0/3480681202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb6400cf590 0x7fb6400cfa20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- 172.21.15.82:0/3480681202 >> 172.21.15.82:0/3480681202 conn(0x7fb64006ae60 msgr2=0x7fb64006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- 172.21.15.82:0/3480681202 shutdown_connections 2024-04-25T10:06:29.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.406+0000 7fb6452ce640 1 -- 172.21.15.82:0/3480681202 wait complete. 2024-04-25T10:06:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 Processor -- start 2024-04-25T10:06:29.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- start start 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6400cf590 0x7fb6401535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb64015b6b0 0x7fb6400ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400caf30 con 0x7fb640072000 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400cadb0 con 0x7fb6400cf590 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6401692b0 con 0x7fb64015b6b0 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57778/0 (socket says 172.21.15.82:57778) 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 -- 172.21.15.82:0/1158250275 learned_addr learned my addr 172.21.15.82:0/1158250275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6400cf590 0x7fb6401535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 -- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb64015b6b0 msgr2=0x7fb6400ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb64015b6b0 0x7fb6400ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 -- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6400cf590 msgr2=0x7fb6401535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6400cf590 0x7fb6401535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 -- 172.21.15.82:0/1158250275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64015db40 con 0x7fb640072000 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb63effd640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c4b0 tx=0x7fb63000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6277fe640 1 -- 172.21.15.82:0/1158250275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630018070 con 0x7fb640072000 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6277fe640 1 -- 172.21.15.82:0/1158250275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630007500 con 0x7fb640072000 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64015de20 con 0x7fb640072000 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6277fe640 1 -- 172.21.15.82:0/1158250275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630004a30 con 0x7fb640072000 2024-04-25T10:06:29.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.410+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64015e3e0 con 0x7fb640072000 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb6277fe640 1 -- 172.21.15.82:0/1158250275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb63001b070 con 0x7fb640072000 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb6277fe640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb6277fe640 1 -- 172.21.15.82:0/1158250275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb630084590 con 0x7fb640072000 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 0x7fb62c069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ad000 tx=0x7fb634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 Processor -- start 2024-04-25T10:06:29.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- start start 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcb74db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb2b0a0 0x55b3dcb2b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc977ff0 con 0x55b3dcb752f0 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc900150 con 0x55b3dcb749d0 2024-04-25T10:06:29.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc974090 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57792/0 (socket says 172.21.15.82:57792) 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3811695855 learned_addr learned my addr 172.21.15.82:0/3811695855 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63effd640 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcb74db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63f7fe640 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb2b0a0 0x55b3dcb2b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb2b0a0 msgr2=0x55b3dcb2b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb2b0a0 0x55b3dcb2b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 msgr2=0x55b3dcb74db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcb74db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 -- 172.21.15.82:0/3811695855 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3dcb0e0d0 con 0x55b3dcb752f0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb63401a090 tx=0x7fb634004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd27631f1537959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb624ff9640 1 -- 172.21.15.82:0/3811695855 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340c8070 con 0x55b3dcb752f0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb624ff9640 1 -- 172.21.15.82:0/3811695855 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340b1040 con 0x55b3dcb752f0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb624ff9640 1 -- 172.21.15.82:0/3811695855 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6340b0050 con 0x55b3dcb752f0 2024-04-25T10:06:29.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- 172.21.15.82:0/3811695855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 msgr2=0x55b3dcb2ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb63401a090 tx=0x7fb634004740 comp rx=0 tx=0).stop 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- 172.21.15.82:0/3811695855 shutdown_connections 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb2b0a0 0x55b3dcb2b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb752f0 0x55b3dcb2ab60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 --2- 172.21.15.82:0/3811695855 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcb74db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- 172.21.15.82:0/3811695855 >> 172.21.15.82:0/3811695855 conn(0x55b3dcb27020 msgr2=0x55b3dcb29440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- 172.21.15.82:0/3811695855 shutdown_connections 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.414+0000 7fb64843b000 1 -- 172.21.15.82:0/3811695855 wait complete. 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- start start 2024-04-25T10:06:29.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 Processor -- start 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb59d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb5a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc93c250 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc932690 con 0x55b3dcb752f0 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc900a40 con 0x55b3dcb749d0 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 10 client.0 _ll_register_callbacks cb 0x55b3dcb15e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57802/0 (socket says 172.21.15.82:57802) 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 -- 172.21.15.82:0/3853435397 learned_addr learned my addr 172.21.15.82:0/3853435397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb59d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63f7fe640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb5a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 -- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 msgr2=0x55b3dcb59d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb59d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 -- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 msgr2=0x55b3dcb5a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb5a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 -- 172.21.15.82:0/3853435397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3dca251c0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63effd640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb63005ce60 tx=0x7fb630058f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87f0f319984275e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300512f0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb63005cf00 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb6340b1040 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb630051490 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 10 client.6753 fetch_fsmap learned FSMap version 34 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b3dca21170 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340b1040 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb6300840c0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 10 client.6753 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 10 client.6753 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 20 client.6753 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb84c80 0x55b3dcb87070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 4 client.6753 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3dc913c60 con 0x55b3dcb84c80 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb84c80 0x55b3dcb87070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb84c80 0x55b3dcb87070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ee84f8b0fddf04 server_cookie=f4a46b0eda55380 in_seq=0 out_seq=0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 10 client.6753 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 1 -- 172.21.15.82:0/3853435397 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b3dc913c60 con 0x55b3dcb84c80 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 10 client.6753 handle_command_reply: tid=0 2024-04-25T10:06:29.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 client.6753 shutdown 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 msgr2=0x55b3dcb597f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fb63005ce60 tx=0x7fb630058f10 comp rx=0 tx=0).stop 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 shutdown_connections 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb84c80 0x55b3dcb87070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb59d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb5a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb61dffb640 0 client.6753 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 --2- 172.21.15.82:0/3853435397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb597f0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 >> 172.21.15.82:0/3853435397 conn(0x55b3dcb27020 msgr2=0x55b3dcb29ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 shutdown_connections 2024-04-25T10:06:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.418+0000 7fb64843b000 1 -- 172.21.15.82:0/3853435397 wait complete. 2024-04-25T10:06:29.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.422+0000 7fb64843b000 20 client.6753 trim_cache size 0 max 16384 2024-04-25T10:06:29.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 Processor -- start 2024-04-25T10:06:29.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 -- start start 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcbd9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb752f0 0x55b3dcbda170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6402011f0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640200310 con 0x55b3dcb749d0 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb640159fc0 con 0x55b3dcb752f0 2024-04-25T10:06:29.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57814/0 (socket says 172.21.15.82:57814) 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.434+0000 7fb63effd640 1 -- 172.21.15.82:0/1706488409 learned_addr learned my addr 172.21.15.82:0/1706488409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb752f0 msgr2=0x55b3dcbda170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb752f0 0x55b3dcbda170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 msgr2=0x55b3dcbd9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcbd9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/1706488409 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3dc9bede0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb630054c10 tx=0x7fb630057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e6aee2aa2f1cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61ffff640 1 -- 172.21.15.82:0/1706488409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630070070 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61ffff640 1 -- 172.21.15.82:0/1706488409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63005cf00 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61ffff640 1 -- 172.21.15.82:0/1706488409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6300545e0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/1706488409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 msgr2=0x55b3dcbd96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fb630054c10 tx=0x7fb630057000 comp rx=0 tx=0).stop 2024-04-25T10:06:29.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/1706488409 shutdown_connections 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb752f0 0x55b3dcbda170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb749d0 0x55b3dcbd9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- 172.21.15.82:0/1706488409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcbd96f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/1706488409 >> 172.21.15.82:0/1706488409 conn(0x55b3dcb27020 msgr2=0x55b3dcb730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/1706488409 shutdown_connections 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/1706488409 wait complete. 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- start start 2024-04-25T10:06:29.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 Processor -- start 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb88260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc912e00 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64015a5c0 con 0x55b3dcb752f0 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b3dc977300 con 0x55b3dcb749d0 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 10 client.0 _ll_register_callbacks cb 0x55b3dcaf6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57820/0 (socket says 172.21.15.82:57820) 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/701170658 learned_addr learned my addr 172.21.15.82:0/701170658 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 msgr2=0x55b3dcb87d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb87d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 msgr2=0x55b3dcb88260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb88260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 -- 172.21.15.82:0/701170658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b3dc9198e0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63effd640 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c480 tx=0x7fb630000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a78132a18b5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630070070 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb63001ed10 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb63005cf00 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6300512f0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 10 client.6759 fetch_fsmap learned FSMap version 34 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55b3dc9d18f0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63005cf00 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb6300540d0 con 0x55b3dcb2b0a0 2024-04-25T10:06:29.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 10 client.6759 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 10 client.6759 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 20 client.6759 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb88bc0 0x55b3dcbe5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 4 client.6759 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b3dc985530 con 0x55b3dcb88bc0 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.438+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb88bc0 0x55b3dcbe5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb63e7fc640 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb88bc0 0x55b3dcbe5770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19eec3072cd192ec server_cookie=c1e5544b6670160e in_seq=0 out_seq=0 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb61dffb640 10 client.6759 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb61dffb640 1 -- 172.21.15.82:0/701170658 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55b3dc985530 con 0x55b3dcb88bc0 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb61dffb640 10 client.6759 handle_command_reply: tid=0 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 client.6759 shutdown 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 msgr2=0x55b3dcb877e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c480 tx=0x7fb630000ee0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 shutdown_connections 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x55b3dcb88bc0 0x55b3dcbe5770 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b3dcb749d0 0x55b3dcb87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b3dcb752f0 0x55b3dcb88260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb61dffb640 0 client.6759 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 --2- 172.21.15.82:0/701170658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b3dcb2b0a0 0x55b3dcb877e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 >> 172.21.15.82:0/701170658 conn(0x55b3dcb27020 msgr2=0x55b3dcb73960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 shutdown_connections 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 1 -- 172.21.15.82:0/701170658 wait complete. 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb64843b000 20 client.6759 trim_cache size 0 max 16384 2024-04-25T10:06:29.450 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:06:29.451 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:06:29.451 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:06:29.451 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 msgr2=0x7fb62c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 0x7fb62c069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb6340ad000 tx=0x7fb634008040 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 msgr2=0x7fb640098920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c4b0 tx=0x7fb63000c970 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 shutdown_connections 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb62c066f60 0x7fb62c069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb64015b6b0 0x7fb6400ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb6400cf590 0x7fb6401535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 --2- 172.21.15.82:0/1158250275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb640072000 0x7fb640098920 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 >> 172.21.15.82:0/1158250275 conn(0x7fb64006ae60 msgr2=0x7fb6400b1870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 shutdown_connections 2024-04-25T10:06:29.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.442+0000 7fb6452ce640 1 -- 172.21.15.82:0/1158250275 wait complete. 2024-04-25T10:06:29.461 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:29.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:29.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.566+0000 7fa711cfe640 1 Processor -- start 2024-04-25T10:06:29.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.566+0000 7fa711cfe640 1 -- start start 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa711cfe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa70c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa711cfe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa70c071e90 0x7fa70c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa711cfe640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c0adbd0 con 0x7fa70c0b24a0 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa711cfe640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70c0ae860 con 0x7fa70c071e90 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa711cfe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c15e210 con 0x7fa70c15da10 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa70c0b24a0 0x7fa70c1614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53982/0 (socket says 172.21.15.82:53982) 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 -- 172.21.15.82:0/1245703901 learned_addr learned my addr 172.21.15.82:0/1245703901 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70bfff640 1 --2- 172.21.15.82:0/1245703901 >> v2:172.21.15.196:3300/0 conn(0x7fa70c071e90 0x7fa70c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70b7fe640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa70c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3297497989 0 0) 0x7fa70c0adbd0 con 0x7fa70c0b24a0 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003530 con 0x7fa70c0b24a0 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1180459584 0 0) 0x7fa6f8003530 con 0x7fa70c0b24a0 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 msgr2=0x7fa70c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa70c071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 >> v2:172.21.15.196:3300/0 conn(0x7fa70c071e90 msgr2=0x7fa70c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 --2- 172.21.15.82:0/1245703901 >> v2:172.21.15.196:3300/0 conn(0x7fa70c071e90 0x7fa70c071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c15e4f0 con 0x7fa70c0b24a0 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa6f4001680 con 0x7fa70c0b24a0 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 >> v1:172.21.15.82:6789/0 conn(0x7fa70c0b24a0 legacy=0x7fa70c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa6f8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa6f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f8004d50 0x7fa6f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0ae860 con 0x7fa6f8003fb0 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c15e210 con 0x7fa6f8004d50 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f800bb30 con 0x7fa70c15da10 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa6f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70b7fe640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa6f8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa70bfff640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f8004d50 0x7fa6f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 -- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 msgr2=0x7fa6f8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15da10 0x7fa6f8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 -- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f8004d50 msgr2=0x7fa6f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6f8004d50 0x7fa6f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.570+0000 7fa702ffd640 1 -- 172.21.15.82:0/1245703901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f8003530 con 0x7fa6f8003fb0 2024-04-25T10:06:29.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa702ffd640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa6f8004810 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c210 tx=0x7fa6f401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ef882d57eeb3ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4007d60 con 0x7fa6f8003fb0 2024-04-25T10:06:29.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40a9020 con 0x7fa6f8003fb0 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70affd640 1 -- 172.21.15.82:0/1245703901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4005ae0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- 172.21.15.82:0/1245703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 msgr2=0x7fa6f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 --2- 172.21.15.82:0/1245703901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa6f8004810 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400c210 tx=0x7fa6f401cca0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70bfff640 1 -- 172.21.15.82:0/1245703901 reap_dead start 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- 172.21.15.82:0/1245703901 shutdown_connections 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- 172.21.15.82:0/1245703901 >> 172.21.15.82:0/1245703901 conn(0x7fa70c0b11a0 msgr2=0x7fa70c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- 172.21.15.82:0/1245703901 shutdown_connections 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- 172.21.15.82:0/1245703901 wait complete. 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 Processor -- start 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- start start 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15a870 0x7fa70c1f5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa70c166d60 0x7fa70c1671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c168a00 con 0x7fa6f8003fb0 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c168880 con 0x7fa70c166d60 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa711cfe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c168b80 con 0x7fa70c15a870 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57832/0 (socket says 172.21.15.82:57832) 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 -- 172.21.15.82:0/937516431 learned_addr learned my addr 172.21.15.82:0/937516431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70bfff640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa70c166d60 0x7fa70c1671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 -- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15a870 msgr2=0x7fa70c1f5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa702ffd640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15a870 0x7fa70c1f5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15a870 0x7fa70c1f5f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 -- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa70c166d60 msgr2=0x7fa70c1671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa70c166d60 0x7fa70c1671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.574+0000 7fa70b7fe640 1 -- 172.21.15.82:0/937516431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c15afe0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa70b7fe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0115e0 tx=0x7fa6fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00f8f0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6fc00fa90 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa70c15b2c0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc00fe40 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c15acf0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c04efa0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6fc015460 con 0x7fa6f8003fb0 2024-04-25T10:06:29.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 0x7fa6ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa6fc085c60 con 0x7fa6f8003fb0 2024-04-25T10:06:29.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa702ffd640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 0x7fa6ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.578+0000 7fa702ffd640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 0x7fa6ec069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40aa000 tx=0x7fa6f4001ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.582+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6fc019030 con 0x7fa6f8003fb0 2024-04-25T10:06:29.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.770+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa70c0b1dc0 con 0x7fa6f8003fb0 2024-04-25T10:06:29.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.770+0000 7fa708ff9640 1 -- 172.21.15.82:0/937516431 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fa6fc056b40 con 0x7fa6f8003fb0 2024-04-25T10:06:29.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:29.775 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:06:06.293306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:29.775 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 msgr2=0x7fa6ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 0x7fa6ec069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40aa000 tx=0x7fa6f4001ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 msgr2=0x7fa70c165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0115e0 tx=0x7fa6fc011aa0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 shutdown_connections 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6ec066f60 0x7fa6ec069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa70c15a870 0x7fa70c1f5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa70c166d60 0x7fa70c1671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 --2- 172.21.15.82:0/937516431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6f8003fb0 0x7fa70c165420 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 >> 172.21.15.82:0/937516431 conn(0x7fa70c0b11a0 msgr2=0x7fa70c06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 shutdown_connections 2024-04-25T10:06:29.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.774+0000 7fa711cfe640 1 -- 172.21.15.82:0/937516431 wait complete. 2024-04-25T10:06:29.792 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:06:29.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.906+0000 7f56a2947640 1 Processor -- start 2024-04-25T10:06:29.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.906+0000 7f56a2947640 1 -- start start 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f569c0d07f0 0x7f569c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 0x7f569c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569c06be60 con 0x7f569c15da80 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c06c440 con 0x7f569c0d07f0 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f569c15de60 con 0x7f569c0d1cd0 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f569c0d07f0 0x7f569c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f569c15da80 0x7f569c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53990/0 (socket says 172.21.15.82:53990) 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569bfff640 1 -- 172.21.15.82:0/408012255 learned_addr learned my addr 172.21.15.82:0/408012255 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/408012255 >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 0x7f569c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 -- 172.21.15.82:0/408012255 >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 msgr2=0x7f569c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 --2- 172.21.15.82:0/408012255 >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 0x7f569c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 -- 172.21.15.82:0/408012255 >> v1:172.21.15.82:6789/0 conn(0x7f569c15da80 legacy=0x7f569c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 -- 172.21.15.82:0/408012255 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c0d2450 con 0x7f569c0d07f0 2024-04-25T10:06:29.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569affd640 1 -- 172.21.15.82:0/408012255 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536569945 0 0) 0x7f569c06be60 con 0x7f569c15da80 2024-04-25T10:06:29.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/408012255 >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 0x7f569c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569b7fe640 1 --2- 172.21.15.82:0/408012255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c0d4230 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f569c06de30 tx=0x7f568401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b6e87873d63ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569affd640 1 -- 172.21.15.82:0/408012255 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840b2070 con 0x7f569c0d07f0 2024-04-25T10:06:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569affd640 1 -- 172.21.15.82:0/408012255 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56840a9070 con 0x7f569c0d07f0 2024-04-25T10:06:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f569affd640 1 -- 172.21.15.82:0/408012255 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840ad880 con 0x7f569c0d07f0 2024-04-25T10:06:29.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- 172.21.15.82:0/408012255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 msgr2=0x7f569c0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 --2- 172.21.15.82:0/408012255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c0d4230 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f569c06de30 tx=0x7f568401c430 comp rx=0 tx=0).stop 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- 172.21.15.82:0/408012255 shutdown_connections 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 --2- 172.21.15.82:0/408012255 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c0d4230 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 --2- 172.21.15.82:0/408012255 >> v2:172.21.15.196:3300/0 conn(0x7f569c0d1cd0 0x7f569c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- 172.21.15.82:0/408012255 >> 172.21.15.82:0/408012255 conn(0x7f569c08b000 msgr2=0x7f569c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- 172.21.15.82:0/408012255 shutdown_connections 2024-04-25T10:06:29.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.910+0000 7f56a2947640 1 -- 172.21.15.82:0/408012255 wait complete. 2024-04-25T10:06:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 Processor -- start 2024-04-25T10:06:29.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- start start 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 0x7f569c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c15da80 0x7f569c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c06d7a0 con 0x7f569c0d07f0 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c06d620 con 0x7f569c15da80 2024-04-25T10:06:29.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c06d920 con 0x7f569c0d1cd0 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57846/0 (socket says 172.21.15.82:57846) 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 -- 172.21.15.82:0/187128540 learned_addr learned my addr 172.21.15.82:0/187128540 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c15da80 0x7f569c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569b7fe640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 0x7f569c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 -- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 msgr2=0x7f569c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 0x7f569c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 -- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c15da80 msgr2=0x7f569c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c15da80 0x7f569c15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 -- 172.21.15.82:0/187128540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f569c1505d0 con 0x7f569c0d07f0 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569b7fe640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 0x7f569c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f569bfff640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f569400c7c0 tx=0x7f569400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f5698ff9640 1 -- 172.21.15.82:0/187128540 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5694018070 con 0x7f569c0d07f0 2024-04-25T10:06:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f5698ff9640 1 -- 172.21.15.82:0/187128540 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f569400a640 con 0x7f569c0d07f0 2024-04-25T10:06:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f569c15ea90 con 0x7f569c0d07f0 2024-04-25T10:06:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f5698ff9640 1 -- 172.21.15.82:0/187128540 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569400aa30 con 0x7f569c0d07f0 2024-04-25T10:06:29.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.914+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569c15f050 con 0x7f569c0d07f0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5698ff9640 1 -- 172.21.15.82:0/187128540 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f569401b070 con 0x7f569c0d07f0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5698ff9640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 0x7f567c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5698ff9640 1 -- 172.21.15.82:0/187128540 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f56940848b0 con 0x7f569c0d07f0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f569b7fe640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 0x7f567c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f569b7fe640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 0x7f567c069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f56840af000 tx=0x7f5684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 Processor -- start 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 -- start start 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562296296ba0 con 0x5622963af8e0 2024-04-25T10:06:29.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622961fc090 con 0x5622963f9c00 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x562296184160 con 0x5622963f92e0 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57862/0 (socket says 172.21.15.82:57862) 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 -- 172.21.15.82:0/956391178 learned_addr learned my addr 172.21.15.82:0/956391178 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f569bfff640 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f569b7fe640 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 -- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 msgr2=0x5622963f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 -- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 msgr2=0x5622963af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 -- 172.21.15.82:0/956391178 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622962a5c20 con 0x5622963af8e0 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f568c00caa0 tx=0x7f568c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80030de691045ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5691ffb640 1 -- 172.21.15.82:0/956391178 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c018020 con 0x5622963af8e0 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5691ffb640 1 -- 172.21.15.82:0/956391178 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c00eb70 con 0x5622963af8e0 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.918+0000 7f5691ffb640 1 -- 172.21.15.82:0/956391178 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c013450 con 0x5622963af8e0 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/956391178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 msgr2=0x5622963afcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f568c00caa0 tx=0x7f568c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/956391178 shutdown_connections 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963af8e0 0x5622963afcc0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/956391178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/956391178 >> 172.21.15.82:0/956391178 conn(0x5622963ab890 msgr2=0x5622963adc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/956391178 shutdown_connections 2024-04-25T10:06:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/956391178 wait complete. 2024-04-25T10:06:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- start start 2024-04-25T10:06:29.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 Processor -- start 2024-04-25T10:06:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x562296400640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x562296400b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622961f8130 con 0x5622963f9c00 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622961c0130 con 0x5622963af8e0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622962ba300 con 0x5622963f92e0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 10 client.0 _ll_register_callbacks cb 0x56229639abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57868/0 (socket says 172.21.15.82:57868) 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 -- 172.21.15.82:0/1997935922 learned_addr learned my addr 172.21.15.82:0/1997935922 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569bfff640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x562296400640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569b7fe640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x562296400b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 -- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 msgr2=0x562296400b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x562296400b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 -- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 msgr2=0x562296400640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x562296400640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 -- 172.21.15.82:0/1997935922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622961d9060 con 0x5622963f9c00 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569bfff640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x562296400640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569b7fe640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x562296400b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f568c00cfa0 tx=0x7f568c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74be8c944106606c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c003de0 con 0x5622963f9c00 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f568c00eb70 con 0x5622963f9c00 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f568c029480 con 0x5622963f9c00 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568c031050 con 0x5622963f9c00 2024-04-25T10:06:29.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 10 client.6780 fetch_fsmap learned FSMap version 34 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x56229619d7c0 con 0x5622963f9c00 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568c00eb70 con 0x5622963f9c00 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f568c034020 con 0x5622963f9c00 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 10 client.6780 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 10 client.6780 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 20 client.6780 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x5622963e7e30 0x5622963e8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 4 client.6780 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562296197cd0 con 0x5622963e7e30 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569bfff640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x5622963e7e30 0x5622963e8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f569bfff640 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x5622963e7e30 0x5622963e8210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb71a157746bcaf7 server_cookie=47bc72be2b31b51a in_seq=0 out_seq=0 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 10 client.6780 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 1 -- 172.21.15.82:0/1997935922 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562296197cd0 con 0x5622963e7e30 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f567affd640 10 client.6780 handle_command_reply: tid=0 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.922+0000 7f56a5ab4000 1 client.6780 shutdown 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 msgr2=0x5622963e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f568c00cfa0 tx=0x7f568c004740 comp rx=0 tx=0).stop 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 shutdown_connections 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x5622963e7e30 0x5622963e8210 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x562296400b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963e7090 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/1997935922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x562296400640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 >> 172.21.15.82:0/1997935922 conn(0x5622963ab890 msgr2=0x5622963ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f567affd640 0 client.6780 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 shutdown_connections 2024-04-25T10:06:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 1 -- 172.21.15.82:0/1997935922 wait complete. 2024-04-25T10:06:29.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.926+0000 7f56a5ab4000 20 client.6780 trim_cache size 0 max 16384 2024-04-25T10:06:29.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 Processor -- start 2024-04-25T10:06:29.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- start start 2024-04-25T10:06:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963af8e0 0x5622963e6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c2020f0 con 0x5622963f92e0 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569c0b7080 con 0x5622963f9c00 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569c160630 con 0x5622963af8e0 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57882/0 (socket says 172.21.15.82:57882) 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 -- 172.21.15.82:0/3198389711 learned_addr learned my addr 172.21.15.82:0/3198389711 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569bfff640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963af8e0 0x5622963e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 -- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963af8e0 msgr2=0x5622963e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963af8e0 0x5622963e6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 -- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 msgr2=0x5622963adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 -- 172.21.15.82:0/3198389711 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622961b7fc0 con 0x5622963f92e0 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f569b7fe640 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f56840b0040 tx=0x7f56840a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f1a60cf5d3605d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56917fa640 1 -- 172.21.15.82:0/3198389711 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840b9980 con 0x5622963f92e0 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56917fa640 1 -- 172.21.15.82:0/3198389711 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56840c9070 con 0x5622963f92e0 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56917fa640 1 -- 172.21.15.82:0/3198389711 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840b9d20 con 0x5622963f92e0 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3198389711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 msgr2=0x5622963e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f56840b0040 tx=0x7f56840a7000 comp rx=0 tx=0).stop 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3198389711 shutdown_connections 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963f9c00 0x5622963adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f92e0 0x5622963e6e40 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3198389711 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963af8e0 0x5622963e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3198389711 >> 172.21.15.82:0/3198389711 conn(0x5622963ab890 msgr2=0x5622963e61f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3198389711 shutdown_connections 2024-04-25T10:06:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3198389711 wait complete. 2024-04-25T10:06:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- start start 2024-04-25T10:06:29.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 Processor -- start 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x5622963fd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963fde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562296196e70 con 0x5622963f9c00 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x562296184a50 con 0x5622963af8e0 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622961fb3a0 con 0x5622963f92e0 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.938+0000 7f56a5ab4000 10 client.0 _ll_register_callbacks cb 0x5622961a8240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:29.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57884/0 (socket says 172.21.15.82:57884) 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 -- 172.21.15.82:0/3761004891 learned_addr learned my addr 172.21.15.82:0/3761004891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f569bfff640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x5622963fd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f569b7fe640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963fde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 -- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 msgr2=0x5622963fde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963fde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 -- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 msgr2=0x5622963fd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x5622963fd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 -- 172.21.15.82:0/3761004891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622962559e0 con 0x5622963f9c00 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f569bfff640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x5622963fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a0ee4640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c9f0 tx=0x7f568c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b05b317ba556aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568c018040 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f568c029c80 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56840c9070 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f568c042050 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 10 client.6786 fetch_fsmap learned FSMap version 34 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x56229619b630 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56840c9070 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f568c045020 con 0x5622963f9c00 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 10 client.6786 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 10 client.6786 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 20 client.6786 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:29.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x56229645f150 0x562296461540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 4 client.6786 mds_command: new command op to 6420 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 --> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5622962095d0 con 0x56229645f150 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f569bfff640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x56229645f150 0x562296461540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f569bfff640 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x56229645f150 0x562296461540 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e94959bff82c62c server_cookie=e94a8b1414bb3137 in_seq=0 out_seq=0 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 10 client.6786 ms_handle_connect on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 1 -- 172.21.15.82:0/3761004891 <== mds.0 v2:172.21.15.82:6835/191077906 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5622962095d0 con 0x56229645f150 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 10 client.6786 handle_command_reply: tid=0 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 client.6786 shutdown 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 msgr2=0x5622963fe3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f568c00c9f0 tx=0x7f568c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 shutdown_connections 2024-04-25T10:06:29.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:6835/191077906,v1:172.21.15.82:6837/191077906] conn(0x56229645f150 0x562296461540 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5622963f92e0 0x5622963fde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5622963f9c00 0x5622963fe3c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f567affd640 0 client.6786 ms_handle_reset on v2:172.21.15.82:6835/191077906 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 --2- 172.21.15.82:0/3761004891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5622963af8e0 0x5622963fd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 >> 172.21.15.82:0/3761004891 conn(0x5622963ab890 msgr2=0x5622963f8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 shutdown_connections 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 1 -- 172.21.15.82:0/3761004891 wait complete. 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a5ab4000 20 client.6786 trim_cache size 0 max 16384 2024-04-25T10:06:29.953 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:29.954 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-25T10:06:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 msgr2=0x7f567c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 0x7f567c069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f56840af000 tx=0x7f5684008040 comp rx=0 tx=0).stop 2024-04-25T10:06:29.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 msgr2=0x7f569c15a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f569400c7c0 tx=0x7f569400cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 shutdown_connections 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f567c066f60 0x7f567c069420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f569c0d1cd0 0x7f569c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f569c15da80 0x7f569c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 --2- 172.21.15.82:0/187128540 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f569c0d07f0 0x7f569c15a840 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 >> 172.21.15.82:0/187128540 conn(0x7f569c08b000 msgr2=0x7f569c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 shutdown_connections 2024-04-25T10:06:29.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:29.942+0000 7f56a2947640 1 -- 172.21.15.82:0/187128540 wait complete. 2024-04-25T10:06:29.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:06:30.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 Processor -- start 2024-04-25T10:06:30.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 -- start start 2024-04-25T10:06:30.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0d80161160 0x7f0d800d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d800b2780 0x7f0d800d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d800d1d40 con 0x7f0d800b3470 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800d2020 con 0x7f0d800b2780 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d86b5a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d800d2300 con 0x7f0d80161160 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d800b2780 0x7f0d800d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0d800b2780 0x7f0d800d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57900/0 (socket says 172.21.15.82:57900) 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 -- 172.21.15.82:0/519347931 learned_addr learned my addr 172.21.15.82:0/519347931 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d848f6640 1 --2- 172.21.15.82:0/519347931 >> v2:172.21.15.196:3300/0 conn(0x7f0d80161160 0x7f0d800d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 -- 172.21.15.82:0/519347931 >> v2:172.21.15.196:3300/0 conn(0x7f0d80161160 msgr2=0x7f0d800d00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 --2- 172.21.15.82:0/519347931 >> v2:172.21.15.196:3300/0 conn(0x7f0d80161160 0x7f0d800d00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 -- 172.21.15.82:0/519347931 >> v1:172.21.15.196:6789/0 conn(0x7f0d800b3470 legacy=0x7f0d800b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.066+0000 7f0d7ffff640 1 -- 172.21.15.82:0/519347931 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800d25e0 con 0x7f0d800b2780 2024-04-25T10:06:30.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d7ffff640 1 --2- 172.21.15.82:0/519347931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d800b2780 0x7f0d800d0600 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0d800d3610 tx=0x7f0d7401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568b7e3a65ed976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d7f7fe640 1 -- 172.21.15.82:0/519347931 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740a7070 con 0x7f0d800b2780 2024-04-25T10:06:30.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d7f7fe640 1 -- 172.21.15.82:0/519347931 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d740b0070 con 0x7f0d800b2780 2024-04-25T10:06:30.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d7f7fe640 1 -- 172.21.15.82:0/519347931 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740ab890 con 0x7f0d800b2780 2024-04-25T10:06:30.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- 172.21.15.82:0/519347931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d800b2780 msgr2=0x7f0d800d0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/519347931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d800b2780 0x7f0d800d0600 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0d800d3610 tx=0x7f0d7401c410 comp rx=0 tx=0).stop 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- 172.21.15.82:0/519347931 shutdown_connections 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/519347931 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d800b2780 0x7f0d800d0600 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/519347931 >> v2:172.21.15.196:3300/0 conn(0x7f0d80161160 0x7f0d800d00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- 172.21.15.82:0/519347931 >> 172.21.15.82:0/519347931 conn(0x7f0d8008b000 msgr2=0x7f0d800d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- 172.21.15.82:0/519347931 shutdown_connections 2024-04-25T10:06:30.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- 172.21.15.82:0/519347931 wait complete. 2024-04-25T10:06:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 Processor -- start 2024-04-25T10:06:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.070+0000 7f0d86b5a640 1 -- start start 2024-04-25T10:06:30.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d800b2780 0x7f0d800b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d800b3470 0x7f0d800b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800cf710 con 0x7f0d80161160 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800cf590 con 0x7f0d800b3470 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800cf890 con 0x7f0d800b2780 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57906/0 (socket says 172.21.15.82:57906) 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 -- 172.21.15.82:0/431530299 learned_addr learned my addr 172.21.15.82:0/431530299 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 -- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d800b2780 msgr2=0x7f0d800b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d800b2780 0x7f0d800b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 -- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d800b3470 msgr2=0x7f0d800b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d800b3470 0x7f0d800b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 -- 172.21.15.82:0/431530299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80153e30 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d850f7640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7b0 tx=0x7f0d7800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800ce60 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78004b20 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d80154110 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800a850 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d80154fa0 con 0x7f0d80161160 2024-04-25T10:06:30.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d80158ad0 con 0x7f0d80161160 2024-04-25T10:06:30.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d7800a9f0 con 0x7f0d80161160 2024-04-25T10:06:30.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f0d78085510 con 0x7f0d80161160 2024-04-25T10:06:30.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d848f6640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.074+0000 7f0d848f6640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 0x7f0d6c069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b850 tx=0x7f0d70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.078+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d78020070 con 0x7f0d80161160 2024-04-25T10:06:30.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.258+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0d8015fd60 con 0x7f0d80161160 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d7d7fa640 1 -- 172.21.15.82:0/431530299 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0d78056270 con 0x7f0d80161160 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:30.264 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 msgr2=0x7f0d6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 0x7f0d6c069420 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b850 tx=0x7f0d70008040 comp rx=0 tx=0).stop 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 msgr2=0x7f0d8014dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0d7800c7b0 tx=0x7f0d7800cc70 comp rx=0 tx=0).stop 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 shutdown_connections 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d6c066f60 0x7f0d6c069420 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0d80161160 0x7f0d8014dc30 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0d800b3470 0x7f0d800b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 --2- 172.21.15.82:0/431530299 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0d800b2780 0x7f0d800b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 >> 172.21.15.82:0/431530299 conn(0x7f0d8008b000 msgr2=0x7f0d8015ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 shutdown_connections 2024-04-25T10:06:30.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.262+0000 7f0d86b5a640 1 -- 172.21.15.82:0/431530299 wait complete. 2024-04-25T10:06:30.282 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:06:30.282 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:30.282 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:30.329 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:30.390 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:30.391 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:06:30.391 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:30.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:30.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.554+0000 7fa3d761d640 1 Processor -- start 2024-04-25T10:06:30.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.554+0000 7fa3d761d640 1 -- start start 2024-04-25T10:06:30.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3d0159ea0 0x7fa3d00b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d0071fd0 0x7fa3d015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d015ae30 con 0x7fa3d015a280 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d00b2990 con 0x7fa3d0159ea0 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d015b110 con 0x7fa3d0071fd0 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3d0159ea0 0x7fa3d00b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa3d0159ea0 0x7fa3d00b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57914/0 (socket says 172.21.15.82:57914) 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1290904199 learned_addr learned my addr 172.21.15.82:0/1290904199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1290904199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d0071fd0 msgr2=0x7fa3d015a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 --2- 172.21.15.82:0/1290904199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d0071fd0 0x7fa3d015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1290904199 >> v1:172.21.15.196:6789/0 conn(0x7fa3d015a280 legacy=0x7fa3d00718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1290904199 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d00b2620 con 0x7fa3d0159ea0 2024-04-25T10:06:30.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d53b9640 1 --2- 172.21.15.82:0/1290904199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00b2e20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009870 tx=0x7fa3cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4f0b2fcd969ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3c7fff640 1 -- 172.21.15.82:0/1290904199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b0070 con 0x7fa3d0159ea0 2024-04-25T10:06:30.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3c7fff640 1 -- 172.21.15.82:0/1290904199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0ab600 con 0x7fa3d0159ea0 2024-04-25T10:06:30.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3c7fff640 1 -- 172.21.15.82:0/1290904199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0ab9d0 con 0x7fa3d0159ea0 2024-04-25T10:06:30.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- 172.21.15.82:0/1290904199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0159ea0 msgr2=0x7fa3d00b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1290904199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00b2e20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc009870 tx=0x7fa3cc01c520 comp rx=0 tx=0).stop 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- 172.21.15.82:0/1290904199 shutdown_connections 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1290904199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d0071fd0 0x7fa3d015a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1290904199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00b2e20 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- 172.21.15.82:0/1290904199 >> 172.21.15.82:0/1290904199 conn(0x7fa3d00b11a0 msgr2=0x7fa3d00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- 172.21.15.82:0/1290904199 shutdown_connections 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.558+0000 7fa3d761d640 1 -- 172.21.15.82:0/1290904199 wait complete. 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 Processor -- start 2024-04-25T10:06:30.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- start start 2024-04-25T10:06:30.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d015a280 0x7fa3d00ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d006d460 con 0x7fa3d0071fd0 2024-04-25T10:06:30.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d006d2e0 con 0x7fa3d0159ea0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d006d5e0 con 0x7fa3d015a280 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57926/0 (socket says 172.21.15.82:57926) 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1497022977 learned_addr learned my addr 172.21.15.82:0/1497022977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d015a280 msgr2=0x7fa3d00ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d015a280 0x7fa3d00ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3d0159ea0 msgr2=0x7fa3d00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 -- 172.21.15.82:0/1497022977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0148fb0 con 0x7fa3d0071fd0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d53b9640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ae040 tx=0x7fa3cc01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0ba020 con 0x7fa3d0071fd0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc0b6640 con 0x7fa3d0071fd0 2024-04-25T10:06:30.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d00ccab0 con 0x7fa3d0071fd0 2024-04-25T10:06:30.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc0b0040 con 0x7fa3d0071fd0 2024-04-25T10:06:30.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d00ccd50 con 0x7fa3d0071fd0 2024-04-25T10:06:30.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d004efa0 con 0x7fa3d0071fd0 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.562+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa3cc0bd020 con 0x7fa3d0071fd0 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.566+0000 7fa3c5ffb640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 0x7fa3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.566+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fa3cc11f420 con 0x7fa3d0071fd0 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.566+0000 7fa3d4bb8640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 0x7fa3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.566+0000 7fa3d4bb8640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 0x7fa3b4069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009670 tx=0x7fa3c0006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.570+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa3cc0c4030 con 0x7fa3d0071fd0 2024-04-25T10:06:30.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.718+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3d00afc00 con 0x7fa3d0071fd0 2024-04-25T10:06:30.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.718+0000 7fa3c5ffb640 1 -- 172.21.15.82:0/1497022977 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17137 (secure 0 0 0) 0x7fa3cc016110 con 0x7fa3d0071fd0 2024-04-25T10:06:30.723 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:30.723 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":67,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:50.454515+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 msgr2=0x7fa3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 0x7fa3b4069420 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0009670 tx=0x7fa3c0006bd0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 msgr2=0x7fa3d0148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc0ae040 tx=0x7fa3cc01ad60 comp rx=0 tx=0).stop 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 shutdown_connections 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3b4066f60 0x7fa3b4069420 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa3d015a280 0x7fa3d00ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa3d0159ea0 0x7fa3d00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 --2- 172.21.15.82:0/1497022977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa3d0071fd0 0x7fa3d0148a70 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 >> 172.21.15.82:0/1497022977 conn(0x7fa3d00b11a0 msgr2=0x7fa3d015b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 shutdown_connections 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.722+0000 7fa3d761d640 1 -- 172.21.15.82:0/1497022977 wait complete. 2024-04-25T10:06:30.740 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:15.838157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:16.556491+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:30.740 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:30.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 Processor -- start 2024-04-25T10:06:30.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 -- start start 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f406415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40640b2c30 0x7f406406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4064066c30 con 0x7f406406c1e0 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406406daa0 con 0x7f40640b2c30 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f406a498640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406406c640 con 0x7f40640af280 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f406406c1e0 0x7f4064161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53992/0 (socket says 172.21.15.82:53992) 2024-04-25T10:06:30.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 -- 172.21.15.82:0/3174746614 learned_addr learned my addr 172.21.15.82:0/3174746614 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4063fff640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f406415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f40637fe640 1 --2- 172.21.15.82:0/3174746614 >> v2:172.21.15.196:3300/0 conn(0x7f40640b2c30 0x7f406406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605593481 0 0) 0x7f4064066c30 con 0x7f406406c1e0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4050003530 con 0x7f406406c1e0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3796365670 0 0) 0x7f4050003530 con 0x7f406406c1e0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 msgr2=0x7f406415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f406415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 >> v2:172.21.15.196:3300/0 conn(0x7f40640b2c30 msgr2=0x7f406406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 --2- 172.21.15.82:0/3174746614 >> v2:172.21.15.196:3300/0 conn(0x7f40640b2c30 0x7f406406bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40640af660 con 0x7f406406c1e0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f405c001680 con 0x7f406406c1e0 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 >> v1:172.21.15.82:6789/0 conn(0x7f406406c1e0 legacy=0x7f4064161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050003a30 0x7f4050003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f4050004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f40500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405000b9d0 con 0x7f4050004d50 2024-04-25T10:06:30.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405000bb50 con 0x7f4050003a30 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405000bcd0 con 0x7f40640af280 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f40500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f40637fe640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f4050004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4063fff640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050003a30 0x7f4050003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 -- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 msgr2=0x7f4050004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f4050004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 -- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050003a30 msgr2=0x7f4050003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4050003a30 0x7f4050003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f4068a35640 1 -- 172.21.15.82:0/3174746614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4050003530 con 0x7f4050004d50 2024-04-25T10:06:30.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.854+0000 7f40637fe640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40640af280 0x7f4050004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f4068a35640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f40500051d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c240 tx=0x7f405c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f93fdb93b9e5ab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c007d60 con 0x7f4050004d50 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c0a9020 con 0x7f4050004d50 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f4062ffd640 1 -- 172.21.15.82:0/3174746614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0a5be0 con 0x7f4050004d50 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- 172.21.15.82:0/3174746614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 msgr2=0x7f40500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 --2- 172.21.15.82:0/3174746614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f40500051d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c240 tx=0x7f405c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f4068a35640 1 -- 172.21.15.82:0/3174746614 reap_dead start 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- 172.21.15.82:0/3174746614 shutdown_connections 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- 172.21.15.82:0/3174746614 >> 172.21.15.82:0/3174746614 conn(0x7f4064071810 msgr2=0x7f4064071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- 172.21.15.82:0/3174746614 shutdown_connections 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- 172.21.15.82:0/3174746614 wait complete. 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 Processor -- start 2024-04-25T10:06:30.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.858+0000 7f406a498640 1 -- start start 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4064142ff0 0x7f4064155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 0x7f4064152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406415ec00 con 0x7f4050004d50 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406415e920 con 0x7f4064142ff0 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406406ef40 con 0x7f4064151dd0 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57932/0 (socket says 172.21.15.82:57932) 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 -- 172.21.15.82:0/3142653021 learned_addr learned my addr 172.21.15.82:0/3142653021 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f40637fe640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4064142ff0 0x7f4064155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4068a35640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 0x7f4064152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 -- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 msgr2=0x7f4064152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 0x7f4064152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 -- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4064142ff0 msgr2=0x7f4064155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4064142ff0 0x7f4064155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 -- 172.21.15.82:0/3142653021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4064149400 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4068a35640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 0x7f4064152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4063fff640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f405400f350 tx=0x7f405400f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405401c070 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4054017620 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40641529b0 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054017a10 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4064143640 con 0x7f4050004d50 2024-04-25T10:06:30.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406404efa0 con 0x7f4050004d50 2024-04-25T10:06:30.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f405401f070 con 0x7f4050004d50 2024-04-25T10:06:30.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 0x7f4044069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f4054085860 con 0x7f4050004d50 2024-04-25T10:06:30.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f40637fe640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 0x7f4044069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.862+0000 7f40637fe640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 0x7f4044069450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4064152ff0 tx=0x7f404c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:30.866+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4054012070 con 0x7f4050004d50 2024-04-25T10:06:31.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.014+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40640727c0 con 0x7f4050004d50 2024-04-25T10:06:31.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.014+0000 7f4060ff9640 1 -- 172.21.15.82:0/3142653021 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4054056710 con 0x7f4050004d50 2024-04-25T10:06:31.019 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 msgr2=0x7f4044069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 0x7f4044069450 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4064152ff0 tx=0x7f404c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 msgr2=0x7f4064155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f405400f350 tx=0x7f405400f810 comp rx=0 tx=0).stop 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 shutdown_connections 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4044066f90 0x7f4044069450 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4064151dd0 0x7f4064152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4064142ff0 0x7f4064155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 --2- 172.21.15.82:0/3142653021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4050004d50 0x7f4064155030 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 >> 172.21.15.82:0/3142653021 conn(0x7f4064071810 msgr2=0x7f406415b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 shutdown_connections 2024-04-25T10:06:31.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.018+0000 7f406a498640 1 -- 172.21.15.82:0/3142653021 wait complete. 2024-04-25T10:06:31.035 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:31.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 Processor -- start 2024-04-25T10:06:31.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 -- start start 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57140b3cf0 0x7f57140cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57140cfad0 0x7f571415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57140cbf10 con 0x7f571415e4f0 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57140cff10 con 0x7f57140cfad0 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f571a88e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57140cc1f0 con 0x7f57140b3cf0 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57140cfad0 0x7f571415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f57140cfad0 0x7f571415dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57942/0 (socket says 172.21.15.82:57942) 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 -- 172.21.15.82:0/2296599326 learned_addr learned my addr 172.21.15.82:0/2296599326 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f5713fff640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57140b3cf0 0x7f57140cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 -- 172.21.15.82:0/2296599326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57140b3cf0 msgr2=0x7f57140cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57140b3cf0 0x7f57140cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 -- 172.21.15.82:0/2296599326 >> v1:172.21.15.196:6789/0 conn(0x7f571415e4f0 legacy=0x7f5714161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 -- 172.21.15.82:0/2296599326 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140cc4d0 con 0x7f57140cfad0 2024-04-25T10:06:31.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.150+0000 7f57137fe640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415dfb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5708009870 tx=0x7f570801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec89f84ef9f7c43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5712ffd640 1 -- 172.21.15.82:0/2296599326 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57080b0070 con 0x7f57140cfad0 2024-04-25T10:06:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5712ffd640 1 -- 172.21.15.82:0/2296599326 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57080ab600 con 0x7f57140cfad0 2024-04-25T10:06:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5712ffd640 1 -- 172.21.15.82:0/2296599326 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57080ab9d0 con 0x7f57140cfad0 2024-04-25T10:06:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/2296599326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 msgr2=0x7f571415dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415dfb0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5708009870 tx=0x7f570801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/2296599326 shutdown_connections 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415dfb0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- 172.21.15.82:0/2296599326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f57140b3cf0 0x7f57140cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/2296599326 >> 172.21.15.82:0/2296599326 conn(0x7f571406ae60 msgr2=0x7f571406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/2296599326 shutdown_connections 2024-04-25T10:06:31.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/2296599326 wait complete. 2024-04-25T10:06:31.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 Processor -- start 2024-04-25T10:06:31.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- start start 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 0x7f5714155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 0x7f57140c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b4250 con 0x7f57140cfad0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57140b40d0 con 0x7f57140b3cf0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571415e900 con 0x7f571415e4f0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57950/0 (socket says 172.21.15.82:57950) 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 -- 172.21.15.82:0/623060915 learned_addr learned my addr 172.21.15.82:0/623060915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5713fff640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 0x7f5714155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5718e2b640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 0x7f57140c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 -- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 msgr2=0x7f57140c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 0x7f57140c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 -- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 msgr2=0x7f5714155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 0x7f5714155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 -- 172.21.15.82:0/623060915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140c9020 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5713fff640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 0x7f5714155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5718e2b640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 0x7f57140c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f57137fe640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f570801abd0 tx=0x7f57080192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57080b0040 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57080b4440 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571415d830 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57080b4830 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.154+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57140c8540 con 0x7f57140cfad0 2024-04-25T10:06:31.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571404efa0 con 0x7f57140cfad0 2024-04-25T10:06:31.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57080bb020 con 0x7f57140cfad0 2024-04-25T10:06:31.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f5710ff9640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 0x7f5700069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f570811e750 con 0x7f57140cfad0 2024-04-25T10:06:31.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f5713fff640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 0x7f5700069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.158+0000 7f5713fff640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 0x7f5700069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5704006fd0 tx=0x7f5704008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.162+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57080c2020 con 0x7f57140cfad0 2024-04-25T10:06:31.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57141567c0 con 0x7f57140cfad0 2024-04-25T10:06:31.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f5710ff9640 1 -- 172.21.15.82:0/623060915 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5708016110 con 0x7f57140cfad0 2024-04-25T10:06:31.314 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 msgr2=0x7f5700069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 0x7f5700069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5704006fd0 tx=0x7f5704008040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 msgr2=0x7f571415d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f570801abd0 tx=0x7f57080192c0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 shutdown_connections 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.310+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5700066f60 0x7f5700069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f571415e4f0 0x7f57140c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f57140cfad0 0x7f571415d2f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 --2- 172.21.15.82:0/623060915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f57140b3cf0 0x7f5714155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 >> 172.21.15.82:0/623060915 conn(0x7f571406ae60 msgr2=0x7f571415eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 shutdown_connections 2024-04-25T10:06:31.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.314+0000 7f571a88e640 1 -- 172.21.15.82:0/623060915 wait complete. 2024-04-25T10:06:31.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 Processor -- start 2024-04-25T10:06:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 -- start start 2024-04-25T10:06:31.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc52415ecb0 0x7fc5240c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5240c9ef0 0x7fc5240ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5240ca4c0 con 0x7fc5240cf590 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5240ca780 con 0x7fc52415ecb0 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc529dbe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc52415f0f0 con 0x7fc5240c9ef0 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc52415ecb0 0x7fc5240c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc52415ecb0 0x7fc5240c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57954/0 (socket says 172.21.15.82:57954) 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 -- 172.21.15.82:0/3381787935 learned_addr learned my addr 172.21.15.82:0/3381787935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 -- 172.21.15.82:0/3381787935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5240c9ef0 msgr2=0x7fc5240ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 --2- 172.21.15.82:0/3381787935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5240c9ef0 0x7fc5240ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 -- 172.21.15.82:0/3381787935 >> v1:172.21.15.196:6789/0 conn(0x7fc5240cf590 legacy=0x7fc5240cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 -- 172.21.15.82:0/3381787935 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5240cd690 con 0x7fc52415ecb0 2024-04-25T10:06:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.442+0000 7fc522ffd640 1 --2- 172.21.15.82:0/3381787935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc52415ecb0 0x7fc5240c99b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51801e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d5108d8d0f886c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5227fc640 1 -- 172.21.15.82:0/3381787935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180ac660 con 0x7fc52415ecb0 2024-04-25T10:06:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5227fc640 1 -- 172.21.15.82:0/3381787935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5180b1070 con 0x7fc52415ecb0 2024-04-25T10:06:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5227fc640 1 -- 172.21.15.82:0/3381787935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5180aca20 con 0x7fc52415ecb0 2024-04-25T10:06:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/3381787935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc52415ecb0 msgr2=0x7fc5240c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- 172.21.15.82:0/3381787935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc52415ecb0 0x7fc5240c99b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc518009870 tx=0x7fc51801e730 comp rx=0 tx=0).stop 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/3381787935 shutdown_connections 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- 172.21.15.82:0/3381787935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5240c9ef0 0x7fc5240ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- 172.21.15.82:0/3381787935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc52415ecb0 0x7fc5240c99b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/3381787935 >> 172.21.15.82:0/3381787935 conn(0x7fc52406ae60 msgr2=0x7fc52406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/3381787935 shutdown_connections 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/3381787935 wait complete. 2024-04-25T10:06:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 Processor -- start 2024-04-25T10:06:31.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- start start 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240cf590 0x7fc524096750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc52415ecb0 0x7fc524096c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240cc1b0 con 0x7fc5240c9ef0 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240cc030 con 0x7fc5240cf590 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5240cc330 con 0x7fc52415ecb0 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57970/0 (socket says 172.21.15.82:57970) 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 -- 172.21.15.82:0/2000840214 learned_addr learned my addr 172.21.15.82:0/2000840214 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc522ffd640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240cf590 0x7fc524096750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 -- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc52415ecb0 msgr2=0x7fc524096c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc52415ecb0 0x7fc524096c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 -- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240cf590 msgr2=0x7fc524096750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240cf590 0x7fc524096750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 -- 172.21.15.82:0/2000840214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc524150cb0 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc5237fe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c480 tx=0x7fc51400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514012830 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514017070 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5240c50c0 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514012c30 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.446+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5240c5680 con 0x7fc5240c9ef0 2024-04-25T10:06:31.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52404efa0 con 0x7fc5240c9ef0 2024-04-25T10:06:31.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc51401c460 con 0x7fc5240c9ef0 2024-04-25T10:06:31.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc50bfff640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 0x7fc510069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fc514085000 con 0x7fc5240c9ef0 2024-04-25T10:06:31.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc522ffd640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 0x7fc510069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.450+0000 7fc522ffd640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 0x7fc510069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc5180b0000 tx=0x7fc5180a9000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.454+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc514055e80 con 0x7fc5240c9ef0 2024-04-25T10:06:31.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc5240b3f20 con 0x7fc5240c9ef0 2024-04-25T10:06:31.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc50bfff640 1 -- 172.21.15.82:0/2000840214 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc514055780 con 0x7fc5240c9ef0 2024-04-25T10:06:31.605 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:06:31.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 msgr2=0x7fc510069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 0x7fc510069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc5180b0000 tx=0x7fc5180a9000 comp rx=0 tx=0).stop 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 msgr2=0x7fc5240c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc51400c480 tx=0x7fc51400c940 comp rx=0 tx=0).stop 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 shutdown_connections 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc510066f60 0x7fc510069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc52415ecb0 0x7fc524096c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5240cf590 0x7fc524096750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 --2- 172.21.15.82:0/2000840214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5240c9ef0 0x7fc5240c1ee0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 >> 172.21.15.82:0/2000840214 conn(0x7fc52406ae60 msgr2=0x7fc52406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 shutdown_connections 2024-04-25T10:06:31.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.602+0000 7fc529dbe640 1 -- 172.21.15.82:0/2000840214 wait complete. 2024-04-25T10:06:31.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:31.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 Processor -- start 2024-04-25T10:06:31.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 -- start start 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 0x7f8b6c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 0x7f8b6c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b6c06ec60 0x7f8b6c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cc2b0 con 0x7f8b6c06ec60 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b6c06e290 con 0x7f8b6c144ca0 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b71fe2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cee30 con 0x7f8b6c0ce510 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b6c06ec60 0x7f8b6c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b6c06ec60 0x7f8b6c142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57984/0 (socket says 172.21.15.82:57984) 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 -- 172.21.15.82:0/2355989815 learned_addr learned my addr 172.21.15.82:0/2355989815 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b63fff640 1 --2- 172.21.15.82:0/2355989815 >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 0x7f8b6c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6b7fe640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 0x7f8b6c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 -- 172.21.15.82:0/2355989815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 msgr2=0x7f8b6c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 0x7f8b6c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 -- 172.21.15.82:0/2355989815 >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 msgr2=0x7f8b6c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/2355989815 >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 0x7f8b6c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6bfff640 1 -- 172.21.15.82:0/2355989815 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c0ccc70 con 0x7f8b6c06ec60 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b63fff640 1 --2- 172.21.15.82:0/2355989815 >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 0x7f8b6c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.730+0000 7f8b6b7fe640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 0x7f8b6c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c06ec60 0x7f8b6c142680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c06cfe0 tx=0x7f8b6401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45164b0360d58554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6affd640 1 -- 172.21.15.82:0/2355989815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b4070 con 0x7f8b6c06ec60 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6affd640 1 -- 172.21.15.82:0/2355989815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b640af400 con 0x7f8b6c06ec60 2024-04-25T10:06:31.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6affd640 1 -- 172.21.15.82:0/2355989815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640af7d0 con 0x7f8b6c06ec60 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/2355989815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c06ec60 msgr2=0x7f8b6c142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c06ec60 0x7f8b6c142680 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c06cfe0 tx=0x7f8b6401f1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/2355989815 shutdown_connections 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c06ec60 0x7f8b6c142680 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/2355989815 >> v2:172.21.15.196:3300/0 conn(0x7f8b6c144ca0 0x7f8b6c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/2355989815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c0ce510 0x7f8b6c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/2355989815 >> 172.21.15.82:0/2355989815 conn(0x7f8b6c0b11a0 msgr2=0x7f8b6c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/2355989815 shutdown_connections 2024-04-25T10:06:31.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/2355989815 wait complete. 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 Processor -- start 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- start start 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c06ec60 0x7f8b6c1f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b6c0ce510 0x7f8b6c143470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c0cc900 con 0x7f8b6c144ca0 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c06df20 con 0x7f8b6c0ce510 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1451f0 con 0x7f8b6c06ec60 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57998/0 (socket says 172.21.15.82:57998) 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 -- 172.21.15.82:0/3688646334 learned_addr learned my addr 172.21.15.82:0/3688646334 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b63fff640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b6c0ce510 0x7f8b6c143470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6b7fe640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c06ec60 0x7f8b6c1f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 -- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c06ec60 msgr2=0x7f8b6c1f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c06ec60 0x7f8b6c1f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 -- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b6c0ce510 msgr2=0x7f8b6c143470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b6c0ce510 0x7f8b6c143470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 -- 172.21.15.82:0/3688646334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6c138990 con 0x7f8b6c144ca0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b6bfff640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8b640b2040 tx=0x7f8b6401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b640b4040 con 0x7f8b6c144ca0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b640c0070 con 0x7f8b6c144ca0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c138c70 con 0x7f8b6c144ca0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b64009510 con 0x7f8b6c144ca0 2024-04-25T10:06:31.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.734+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c139230 con 0x7f8b6c144ca0 2024-04-25T10:06:31.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6c04efa0 con 0x7f8b6c144ca0 2024-04-25T10:06:31.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b640c5460 con 0x7f8b6c144ca0 2024-04-25T10:06:31.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b68ff9640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 0x7f8b4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b6b7fe640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 0x7f8b4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f8b64123770 con 0x7f8b6c144ca0 2024-04-25T10:06:31.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.738+0000 7f8b6b7fe640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 0x7f8b4c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8b54006b20 tx=0x7f8b54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:31.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.742+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b64018d40 con 0x7f8b6c144ca0 2024-04-25T10:06:31.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.898+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b6c0b1cc0 con 0x7f8b6c144ca0 2024-04-25T10:06:31.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b68ff9640 1 -- 172.21.15.82:0/3688646334 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v67) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8b640f4580 con 0x7f8b6c144ca0 2024-04-25T10:06:31.904 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:06:31.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 msgr2=0x7f8b4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 0x7f8b4c069420 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8b54006b20 tx=0x7f8b54008040 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 msgr2=0x7f8b6c1439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8b640b2040 tx=0x7f8b6401bf80 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 shutdown_connections 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b4c066f60 0x7f8b4c069420 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b6c144ca0 0x7f8b6c1439b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b6c0ce510 0x7f8b6c143470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 --2- 172.21.15.82:0/3688646334 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b6c06ec60 0x7f8b6c1f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 >> 172.21.15.82:0/3688646334 conn(0x7f8b6c0b11a0 msgr2=0x7f8b6c0c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 shutdown_connections 2024-04-25T10:06:31.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:31.902+0000 7f8b71fe2640 1 -- 172.21.15.82:0/3688646334 wait complete. 2024-04-25T10:06:31.922 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:06:31.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.922 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:31.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[43150]: fuse finished with error 0 and tester_r 0 2024-04-25T10:06:31.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:06:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.935 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:32.004 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:06:32.004 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:32.021 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:32.022 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:06:32.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:32.022 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:06:32.069 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:06:32.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:32.070 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:06:32.070 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:32.075 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:32.075 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:06:32.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:32.076 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:06:32.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:32.125 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:06:32.125 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:06:32.125 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:32.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:32.131 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:06:32.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:32.131 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:06:32.176 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:06:32.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:06:32.178 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 Processor -- start 2024-04-25T10:06:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 -- start start 2024-04-25T10:06:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd2806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd2815ab30 con 0x7fdd2815a750 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd280cf560 con 0x7fdd28066bb0 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd2d6a2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28067100 con 0x7fdd2806c3b0 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd26ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdd2815a750 0x7fdd2806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54008/0 (socket says 172.21.15.82:54008) 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd26ffd640 1 -- 172.21.15.82:0/1985247602 learned_addr learned my addr 172.21.15.82:0/1985247602 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd2806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3071056282 0 0) 0x7fdd2815ab30 con 0x7fdd2815a750 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd10003530 con 0x7fdd2815a750 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 601193866 0 0) 0x7fdd10003530 con 0x7fdd2815a750 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 msgr2=0x7fdd2806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd2806ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 >> v1:172.21.15.196:6789/0 conn(0x7fdd28066bb0 legacy=0x7fdd280b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd280cf840 con 0x7fdd2815a750 2024-04-25T10:06:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fdd18001a00 con 0x7fdd2815a750 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 >> v1:172.21.15.82:6789/0 conn(0x7fdd2815a750 legacy=0x7fdd2806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd10003a60 0x7fdd10003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd28067100 con 0x7fdd10004810 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd10005510 con 0x7fdd10003a60 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1000bf10 con 0x7fdd2806c3b0 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd26ffd640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd10003a60 0x7fdd10003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd277fe640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 -- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 msgr2=0x7fdd1000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd1000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 -- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd10003a60 msgr2=0x7fdd10003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd10003a60 0x7fdd10003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd267fc640 1 -- 172.21.15.82:0/1985247602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd10003530 con 0x7fdd10004810 2024-04-25T10:06:32.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd26ffd640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd10003a60 0x7fdd10003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.298+0000 7fdd277fe640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd2806c3b0 0x7fdd1000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd267fc640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd10004c90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ae0 tx=0x7fdd1c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f5887285ce61a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0096b0 con 0x7fdd10004810 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1c0a6450 con 0x7fdd10004810 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd25ffb640 1 -- 172.21.15.82:0/1985247602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1c0a6640 con 0x7fdd10004810 2024-04-25T10:06:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/1985247602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 msgr2=0x7fdd10004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/1985247602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd10004c90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c009ae0 tx=0x7fdd1c01ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd277fe640 1 -- 172.21.15.82:0/1985247602 reap_dead start 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/1985247602 shutdown_connections 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/1985247602 >> 172.21.15.82:0/1985247602 conn(0x7fdd28071810 msgr2=0x7fdd28071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/1985247602 shutdown_connections 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/1985247602 wait complete. 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 Processor -- start 2024-04-25T10:06:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.302+0000 7fdd2d6a2640 1 -- start start 2024-04-25T10:06:32.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd28152920 0x7fdd28152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd28154ac0 0x7fdd28154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806dc00 con 0x7fdd10004810 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806da80 con 0x7fdd28154ac0 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2806dd80 con 0x7fdd28152920 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58016/0 (socket says 172.21.15.82:58016) 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 -- 172.21.15.82:0/933975965 learned_addr learned my addr 172.21.15.82:0/933975965 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd267fc640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd28152920 0x7fdd28152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd277fe640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd28154ac0 0x7fdd28154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 -- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd28152920 msgr2=0x7fdd28152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd28152920 0x7fdd28152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 -- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd28154ac0 msgr2=0x7fdd28154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd28154ac0 0x7fdd28154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 -- 172.21.15.82:0/933975965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd281558b0 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd26ffd640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fdd18007c50 tx=0x7fdd1800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18019070 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd18004d70 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd28155b90 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd18007310 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd281581d0 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2804efa0 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd1801c070 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 0x7fdd14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fdd180851a0 con 0x7fdd10004810 2024-04-25T10:06:32.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd267fc640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 0x7fdd14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.306+0000 7fdd267fc640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 0x7fdd14069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ad000 tx=0x7fdd1c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.310+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdd180559d0 con 0x7fdd10004810 2024-04-25T10:06:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.506+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd280b4c00 con 0x7fdd10004810 2024-04-25T10:06:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd0f7fe640 1 -- 172.21.15.82:0/933975965 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fdd18059a70 con 0x7fdd10004810 2024-04-25T10:06:32.512 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:32.512 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:06:06.293306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:32.512 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-04-25T10:06:32.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 msgr2=0x7fdd14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 0x7fdd14069420 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c0ad000 tx=0x7fdd1c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 msgr2=0x7fdd2813cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fdd18007c50 tx=0x7fdd1800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 shutdown_connections 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd14066f60 0x7fdd14069420 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd28152920 0x7fdd28152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd28154ac0 0x7fdd28154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 --2- 172.21.15.82:0/933975965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd10004810 0x7fdd2813cd30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 >> 172.21.15.82:0/933975965 conn(0x7fdd28071810 msgr2=0x7fdd28073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 shutdown_connections 2024-04-25T10:06:32.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.510+0000 7fdd2d6a2640 1 -- 172.21.15.82:0/933975965 wait complete. 2024-04-25T10:06:32.531 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:32.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 Processor -- start 2024-04-25T10:06:32.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 -- start start 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6e9c0b3f50 0x7f6e9c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e9c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c0b2090 con 0x7f6e9c15a700 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c15f200 con 0x7f6e9c0b3f50 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea413d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b43e0 con 0x7f6e9c0b24a0 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea1ed9640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6e9c15a700 0x7f6e9c0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54012/0 (socket says 172.21.15.82:54012) 2024-04-25T10:06:32.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea1ed9640 1 -- 172.21.15.82:0/943579958 learned_addr learned my addr 172.21.15.82:0/943579958 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:32.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203945314 0 0) 0x7f6e9c0b2090 con 0x7f6e9c15a700 2024-04-25T10:06:32.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/943579958 >> v2:172.21.15.196:3300/0 conn(0x7f6e9c0b3f50 0x7f6e9c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea26da640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e9c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e88005740 con 0x7f6e9c15a700 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867114501 0 0) 0x7f6e88005740 con 0x7f6e9c15a700 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 msgr2=0x7f6e9c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e9c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 >> v2:172.21.15.196:3300/0 conn(0x7f6e9c0b3f50 msgr2=0x7f6e9c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 --2- 172.21.15.82:0/943579958 >> v2:172.21.15.196:3300/0 conn(0x7f6e9c0b3f50 0x7f6e9c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0b46c0 con 0x7f6e9c15a700 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6e980016a0 con 0x7f6e9c15a700 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 >> v1:172.21.15.82:6789/0 conn(0x7f6e9c15a700 legacy=0x7f6e9c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88005c40 0x7f6e88006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e88006a20 0x7f6e88006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e880073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15f200 con 0x7f6e88006a20 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0b43e0 con 0x7f6e88005c40 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8800dce0 con 0x7f6e9c0b24a0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e88006a20 0x7f6e88006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea26da640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e880073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea1ed9640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88005c40 0x7f6e88006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 -- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 msgr2=0x7f6e880073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c0b24a0 0x7f6e880073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 -- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88005c40 msgr2=0x7f6e88006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88005c40 0x7f6e88006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.650+0000 7f6ea16d8640 1 -- 172.21.15.82:0/943579958 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e88005740 con 0x7f6e88006a20 2024-04-25T10:06:32.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e88006a20 0x7f6e88006ea0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009ab0 tx=0x7f6e8c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f816e8876fb8794 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0b2070 con 0x7f6e88006a20 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c0a5610 con 0x7f6e88006a20 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea0ed7640 1 -- 172.21.15.82:0/943579958 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c0096b0 con 0x7f6e88006a20 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- 172.21.15.82:0/943579958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e88006a20 msgr2=0x7f6e88006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 --2- 172.21.15.82:0/943579958 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e88006a20 0x7f6e88006ea0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c009ab0 tx=0x7f6e8c01ca40 comp rx=0 tx=0).stop 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea26da640 1 -- 172.21.15.82:0/943579958 reap_dead start 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- 172.21.15.82:0/943579958 shutdown_connections 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- 172.21.15.82:0/943579958 >> 172.21.15.82:0/943579958 conn(0x7f6e9c0b11a0 msgr2=0x7f6e9c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- 172.21.15.82:0/943579958 shutdown_connections 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- 172.21.15.82:0/943579958 wait complete. 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 Processor -- start 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- start start 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 0x7f6e9c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c15f5c0 0x7f6e9c15fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0ce600 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15aae0 con 0x7f6e88006a20 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea413d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c0cd310 con 0x7f6e9c15f5c0 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea26da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea26da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58022/0 (socket says 172.21.15.82:58022) 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea26da640 1 -- 172.21.15.82:0/769865875 learned_addr learned my addr 172.21.15.82:0/769865875 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea16d8640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c15f5c0 0x7f6e9c15fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea1ed9640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 0x7f6e9c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.654+0000 7f6ea26da640 1 -- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c15f5c0 msgr2=0x7f6e9c15fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea26da640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c15f5c0 0x7f6e9c15fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea26da640 1 -- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 msgr2=0x7f6e9c1f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea26da640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 0x7f6e9c1f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea26da640 1 -- 172.21.15.82:0/769865875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c1f5cc0 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea1ed9640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 0x7f6e9c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea26da640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c06bfc0 tx=0x7f6e940115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94011e10 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9401b070 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c1606c0 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940039a0 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c160c80 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9c04efa0 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e94003b60 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f6e94085850 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea1ed9640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.658+0000 7f6ea1ed9640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c1635c0 tx=0x7f6e98001050 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.662+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e94056080 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.826+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e9c06f260 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.826+0000 7f6e927fc640 1 -- 172.21.15.82:0/769865875 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17137 (secure 0 0 0) 0x7f6e9405a120 con 0x7f6e9c0b01f0 2024-04-25T10:06:32.831 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:32.832 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":67,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:50.454515+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:32.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 msgr2=0x7f6e80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c1635c0 tx=0x7f6e98001050 comp rx=0 tx=0).stop 2024-04-25T10:06:32.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 msgr2=0x7f6e9c0b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c06bfc0 tx=0x7f6e940115a0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 shutdown_connections 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9c15f5c0 0x7f6e9c15fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e9c0b01f0 0x7f6e9c0b06a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 --2- 172.21.15.82:0/769865875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e88006a20 0x7f6e9c1f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 >> 172.21.15.82:0/769865875 conn(0x7f6e9c0b11a0 msgr2=0x7f6e9c15dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 shutdown_connections 2024-04-25T10:06:32.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.830+0000 7f6ea413d640 1 -- 172.21.15.82:0/769865875 wait complete. 2024-04-25T10:06:32.851 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:05:15.838157+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:05:16.556491+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:32.852 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:32.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 Processor -- start 2024-04-25T10:06:32.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 -- start start 2024-04-25T10:06:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0bb0159bd0 0x7f0bb00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0bb00b28c0 0x7f0bb0163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bb00c9ef0 con 0x7f0bb0159fb0 2024-04-25T10:06:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb0072780 con 0x7f0bb0159bd0 2024-04-25T10:06:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0bb5820640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bb0071b70 con 0x7f0bb00b28c0 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0bb0159bd0 0x7f0bb00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0bb0159bd0 0x7f0bb00c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58026/0 (socket says 172.21.15.82:58026) 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 -- 172.21.15.82:0/3320594464 learned_addr learned my addr 172.21.15.82:0/3320594464 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baf7fe640 1 --2- 172.21.15.82:0/3320594464 >> v2:172.21.15.196:3300/0 conn(0x7f0bb00b28c0 0x7f0bb0163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 -- 172.21.15.82:0/3320594464 >> v2:172.21.15.196:3300/0 conn(0x7f0bb00b28c0 msgr2=0x7f0bb0163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 --2- 172.21.15.82:0/3320594464 >> v2:172.21.15.196:3300/0 conn(0x7f0bb00b28c0 0x7f0bb0163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 -- 172.21.15.82:0/3320594464 >> v1:172.21.15.196:6789/0 conn(0x7f0bb0159fb0 legacy=0x7f0bb00b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:32.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.974+0000 7f0baeffd640 1 -- 172.21.15.82:0/3320594464 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb0071e50 con 0x7f0bb0159bd0 2024-04-25T10:06:32.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baeffd640 1 --2- 172.21.15.82:0/3320594464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159bd0 0x7f0bb00c99b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072bb0 tx=0x7f0ba001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c9f57706fd90cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0badffb640 1 -- 172.21.15.82:0/3320594464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00b4070 con 0x7f0bb0159bd0 2024-04-25T10:06:32.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0badffb640 1 -- 172.21.15.82:0/3320594464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba00a7bd0 con 0x7f0bb0159bd0 2024-04-25T10:06:32.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0badffb640 1 -- 172.21.15.82:0/3320594464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00af440 con 0x7f0bb0159bd0 2024-04-25T10:06:32.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- 172.21.15.82:0/3320594464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159bd0 msgr2=0x7f0bb00c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- 172.21.15.82:0/3320594464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159bd0 0x7f0bb00c99b0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0072bb0 tx=0x7f0ba001eb80 comp rx=0 tx=0).stop 2024-04-25T10:06:32.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- 172.21.15.82:0/3320594464 shutdown_connections 2024-04-25T10:06:32.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- 172.21.15.82:0/3320594464 >> v2:172.21.15.196:3300/0 conn(0x7f0bb00b28c0 0x7f0bb0163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- 172.21.15.82:0/3320594464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159bd0 0x7f0bb00c99b0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- 172.21.15.82:0/3320594464 >> 172.21.15.82:0/3320594464 conn(0x7f0bb006ae60 msgr2=0x7f0bb006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:32.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- 172.21.15.82:0/3320594464 shutdown_connections 2024-04-25T10:06:32.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- 172.21.15.82:0/3320594464 wait complete. 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 Processor -- start 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- start start 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 0x7f0bb0158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bb0159bd0 0x7f0bb00c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0166030 con 0x7f0bb0159fb0 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb0165eb0 con 0x7f0bb00b28c0 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bb5820640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb01661b0 con 0x7f0bb0159bd0 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58030/0 (socket says 172.21.15.82:58030) 2024-04-25T10:06:32.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 -- 172.21.15.82:0/1251870650 learned_addr learned my addr 172.21.15.82:0/1251870650 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baeffd640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 0x7f0bb0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 -- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bb0159bd0 msgr2=0x7f0bb00c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0bae7fc640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bb0159bd0 0x7f0bb00c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bb0159bd0 0x7f0bb00c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 -- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 msgr2=0x7f0bb0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 0x7f0bb0158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baf7fe640 1 -- 172.21.15.82:0/1251870650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00ad5e0 con 0x7f0bb0159fb0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.978+0000 7f0baeffd640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 0x7f0bb0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0baf7fe640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800cc60 tx=0x7f0ba800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba80039a0 con 0x7f0bb0159fb0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba8003b40 con 0x7f0bb0159fb0 2024-04-25T10:06:32.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb00ad8c0 con 0x7f0bb0159fb0 2024-04-25T10:06:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8004020 con 0x7f0bb0159fb0 2024-04-25T10:06:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb00ade80 con 0x7f0bb0159fb0 2024-04-25T10:06:32.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb00c7210 con 0x7f0bb0159fb0 2024-04-25T10:06:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ba80041c0 con 0x7f0bb0159fb0 2024-04-25T10:06:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 0x7f0b9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f0ba8085720 con 0x7f0bb0159fb0 2024-04-25T10:06:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0baeffd640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 0x7f0b9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:32.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.982+0000 7f0baeffd640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 0x7f0b9c069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00b1000 tx=0x7f0ba00a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:32.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:32.986+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ba8014030 con 0x7f0bb0159fb0 2024-04-25T10:06:33.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0bb004efa0 con 0x7f0bb0159fb0 2024-04-25T10:06:33.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0b97fff640 1 -- 172.21.15.82:0/1251870650 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ba8056510 con 0x7f0bb0159fb0 2024-04-25T10:06:33.150 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 msgr2=0x7f0b9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 0x7f0b9c069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00b1000 tx=0x7f0ba00a9070 comp rx=0 tx=0).stop 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 msgr2=0x7f0bb00c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800cc60 tx=0x7f0ba800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.146+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 shutdown_connections 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b9c066f60 0x7f0b9c069420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0bb0159bd0 0x7f0bb00c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0bb0159fb0 0x7f0bb00c53f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 --2- 172.21.15.82:0/1251870650 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0bb00b28c0 0x7f0bb0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 >> 172.21.15.82:0/1251870650 conn(0x7f0bb006ae60 msgr2=0x7f0bb0070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 shutdown_connections 2024-04-25T10:06:33.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.150+0000 7f0bb5820640 1 -- 172.21.15.82:0/1251870650 wait complete. 2024-04-25T10:06:33.168 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.278+0000 7f7afa8ca640 1 Processor -- start 2024-04-25T10:06:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 -- start start 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7af4159bd0 0x7f7af40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af4163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af40c9ef0 con 0x7f7af4159fb0 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af4072780 con 0x7f7af4159bd0 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7afa8ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4071b70 con 0x7f7af40b28c0 2024-04-25T10:06:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7af4159bd0 0x7f7af40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7af4159bd0 0x7f7af40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58032/0 (socket says 172.21.15.82:58032) 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7af4159fb0 0x7f7af40b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54014/0 (socket says 172.21.15.82:54014) 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 -- 172.21.15.82:0/3873553704 learned_addr learned my addr 172.21.15.82:0/3873553704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af8e67640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af4163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 -- 172.21.15.82:0/3873553704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 msgr2=0x7f7af4163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af4163960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 -- 172.21.15.82:0/3873553704 >> v1:172.21.15.82:6789/0 conn(0x7f7af4159fb0 legacy=0x7f7af40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 -- 172.21.15.82:0/3873553704 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af4071e50 con 0x7f7af4159bd0 2024-04-25T10:06:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af2ffd640 1 -- 172.21.15.82:0/3873553704 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963600852 0 0) 0x7f7af40c9ef0 con 0x7f7af4159fb0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af3fff640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159bd0 0x7f7af40c99b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400af60 tx=0x7f7ae401db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67bdcece9db325d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af2ffd640 1 -- 172.21.15.82:0/3873553704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400a2d0 con 0x7f7af4159bd0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af2ffd640 1 -- 172.21.15.82:0/3873553704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae40a6940 con 0x7f7af4159bd0 2024-04-25T10:06:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.282+0000 7f7af2ffd640 1 -- 172.21.15.82:0/3873553704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40a6d10 con 0x7f7af4159bd0 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3873553704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159bd0 msgr2=0x7f7af40c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159bd0 0x7f7af40c99b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400af60 tx=0x7f7ae401db00 comp rx=0 tx=0).stop 2024-04-25T10:06:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3873553704 shutdown_connections 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af4163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3873553704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159bd0 0x7f7af40c99b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3873553704 >> 172.21.15.82:0/3873553704 conn(0x7f7af406ae60 msgr2=0x7f7af406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3873553704 shutdown_connections 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3873553704 wait complete. 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 Processor -- start 2024-04-25T10:06:33.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- start start 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af40accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 0x7f7af41631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4166030 con 0x7f7af4159fb0 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4165eb0 con 0x7f7af4159bd0 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af41661b0 con 0x7f7af40b28c0 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58046/0 (socket says 172.21.15.82:58046) 2024-04-25T10:06:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 -- 172.21.15.82:0/3233770622 learned_addr learned my addr 172.21.15.82:0/3233770622 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af37fe640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 0x7f7af41631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af3fff640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af40accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 -- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 msgr2=0x7f7af40accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af40accf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 -- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 msgr2=0x7f7af41631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 0x7f7af41631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 -- 172.21.15.82:0/3233770622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40b2670 con 0x7f7af4159fb0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af37fe640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 0x7f7af41631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af8e67640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d560 tx=0x7f7aec00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0125f0 con 0x7f7af4159fb0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aec019070 con 0x7f7af4159fb0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af41625a0 con 0x7f7af4159fb0 2024-04-25T10:06:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aec0129f0 con 0x7f7af4159fb0 2024-04-25T10:06:33.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.286+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af4162860 con 0x7f7af4159fb0 2024-04-25T10:06:33.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af404efa0 con 0x7f7af4159fb0 2024-04-25T10:06:33.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7aec012b90 con 0x7f7af4159fb0 2024-04-25T10:06:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7af0ff9640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 0x7f7ae0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7af3fff640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 0x7f7ae0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f7aec085610 con 0x7f7af4159fb0 2024-04-25T10:06:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.290+0000 7f7af3fff640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 0x7f7ae0069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40af000 tx=0x7f7ae40a8070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.294+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7aec056400 con 0x7f7af4159fb0 2024-04-25T10:06:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.438+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7af40cea90 con 0x7f7af4159fb0 2024-04-25T10:06:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.438+0000 7f7af0ff9640 1 -- 172.21.15.82:0/3233770622 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7aec055d00 con 0x7f7af4159fb0 2024-04-25T10:06:33.443 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:06:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 msgr2=0x7f7ae0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 0x7f7ae0069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40af000 tx=0x7f7ae40a8070 comp rx=0 tx=0).stop 2024-04-25T10:06:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 msgr2=0x7f7af40ad5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00d560 tx=0x7f7aec00da20 comp rx=0 tx=0).stop 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 shutdown_connections 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ae0066f90 0x7f7ae0069450 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7af4159fb0 0x7f7af40ad5e0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7af4159bd0 0x7f7af41631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 --2- 172.21.15.82:0/3233770622 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7af40b28c0 0x7f7af40accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 >> 172.21.15.82:0/3233770622 conn(0x7f7af406ae60 msgr2=0x7f7af40af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 shutdown_connections 2024-04-25T10:06:33.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.442+0000 7f7afa8ca640 1 -- 172.21.15.82:0/3233770622 wait complete. 2024-04-25T10:06:33.461 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 Processor -- start 2024-04-25T10:06:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 -- start start 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fef1815da80 0x7fef180d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef1806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1806be60 con 0x7fef180d1cd0 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1806c440 con 0x7fef1815da80 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ef2b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1815de60 con 0x7fef180d07f0 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fef180d1cd0 0x7fef180d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54018/0 (socket says 172.21.15.82:54018) 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 -- 172.21.15.82:0/3910541715 learned_addr learned my addr 172.21.15.82:0/3910541715 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1ccc7640 1 --2- 172.21.15.82:0/3910541715 >> v2:172.21.15.196:3300/0 conn(0x7fef1815da80 0x7fef180d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366376096 0 0) 0x7fef1806be60 con 0x7fef180d1cd0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1d4c8640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef1806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef08003530 con 0x7fef180d1cd0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2509001233 0 0) 0x7fef08003530 con 0x7fef180d1cd0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 msgr2=0x7fef1806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef1806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 >> v2:172.21.15.196:3300/0 conn(0x7fef1815da80 msgr2=0x7fef180d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 --2- 172.21.15.82:0/3910541715 >> v2:172.21.15.196:3300/0 conn(0x7fef1815da80 0x7fef180d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180d2450 con 0x7fef180d1cd0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fef04001820 con 0x7fef180d1cd0 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1d4c8640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef1806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 >> v1:172.21.15.82:6789/0 conn(0x7fef180d1cd0 legacy=0x7fef180d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08003a30 0x7fef08003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef08004810 0x7fef08004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1806c440 con 0x7fef08004810 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1815de60 con 0x7fef08003a30 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0800ba80 con 0x7fef180d07f0 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef08004810 0x7fef08004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef1d4c8640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 -- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 msgr2=0x7fef080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef180d07f0 0x7fef080051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 -- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08003a30 msgr2=0x7fef08003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08003a30 0x7fef08003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.570+0000 7fef17fff640 1 -- 172.21.15.82:0/3910541715 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef08003530 con 0x7fef08004810 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef08004810 0x7fef08004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c210 tx=0x7fef0401f2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3e2e1fa794744b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04007d30 con 0x7fef08004810 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef04007270 con 0x7fef08004810 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef177fe640 1 -- 172.21.15.82:0/3910541715 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040057b0 con 0x7fef08004810 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- 172.21.15.82:0/3910541715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef08004810 msgr2=0x7fef08004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/3910541715 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef08004810 0x7fef08004c90 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fef0400c210 tx=0x7fef0401f2f0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1d4c8640 1 -- 172.21.15.82:0/3910541715 reap_dead start 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- 172.21.15.82:0/3910541715 shutdown_connections 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- 172.21.15.82:0/3910541715 >> 172.21.15.82:0/3910541715 conn(0x7fef1808b000 msgr2=0x7fef180d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- 172.21.15.82:0/3910541715 shutdown_connections 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- 172.21.15.82:0/3910541715 wait complete. 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 Processor -- start 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- start start 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08004810 0x7fef18157cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 0x7fef180b8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180d26f0 con 0x7fef180ca7c0 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef180d2120 con 0x7fef08004810 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ef2b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef180d2870 con 0x7fef181606e0 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58052/0 (socket says 172.21.15.82:58052) 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 -- 172.21.15.82:0/1634923513 learned_addr learned my addr 172.21.15.82:0/1634923513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1ccc7640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08004810 0x7fef18157cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1d4c8640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 0x7fef180b8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 -- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 msgr2=0x7fef180b8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 0x7fef180b8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 -- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08004810 msgr2=0x7fef18157cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08004810 0x7fef18157cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 -- 172.21.15.82:0/1634923513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef180cb7d0 con 0x7fef180ca7c0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef1d4c8640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 0x7fef180b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef17fff640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fef04000f00 tx=0x7fef08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.574+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040017e0 con 0x7fef180ca7c0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef040057b0 con 0x7fef180ca7c0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef180cb210 con 0x7fef180ca7c0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04005b80 con 0x7fef180ca7c0 2024-04-25T10:06:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1815f9a0 con 0x7fef180ca7c0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1804efa0 con 0x7fef180ca7c0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fef040b0430 con 0x7fef180ca7c0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef157fa640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fef04122930 con 0x7fef180ca7c0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef1ccc7640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.578+0000 7fef1ccc7640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 0x7feefc069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009af0 tx=0x7fef0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.582+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fef040c1060 con 0x7fef180ca7c0 2024-04-25T10:06:33.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fef1806d210 con 0x7fef180ca7c0 2024-04-25T10:06:33.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef157fa640 1 -- 172.21.15.82:0/1634923513 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7fef040b7070 con 0x7fef180ca7c0 2024-04-25T10:06:33.733 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 msgr2=0x7feefc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 0x7feefc069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fef0c009af0 tx=0x7fef0c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 msgr2=0x7fef181601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fef04000f00 tx=0x7fef08005840 comp rx=0 tx=0).stop 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 shutdown_connections 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feefc066f60 0x7feefc069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fef181606e0 0x7fef180b8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fef180ca7c0 0x7fef181601a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 --2- 172.21.15.82:0/1634923513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fef08004810 0x7fef18157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 >> 172.21.15.82:0/1634923513 conn(0x7fef1808b000 msgr2=0x7fef180d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 shutdown_connections 2024-04-25T10:06:33.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.730+0000 7fef1ef2b640 1 -- 172.21.15.82:0/1634923513 wait complete. 2024-04-25T10:06:33.751 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 Processor -- start 2024-04-25T10:06:33.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 -- start start 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8c9415a700 0x7f8c9415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8c940b24a0 0x7f8c940b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c940b2090 con 0x7f8c940b3f50 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9415f200 con 0x7f8c9415a700 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9b802640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940b43e0 con 0x7f8c940b24a0 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8c9415a700 0x7f8c9415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8c9415a700 0x7f8c9415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58062/0 (socket says 172.21.15.82:58062) 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 -- 172.21.15.82:0/3408278581 learned_addr learned my addr 172.21.15.82:0/3408278581 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 -- 172.21.15.82:0/3408278581 >> v2:172.21.15.196:3300/0 conn(0x7f8c940b24a0 msgr2=0x7f8c940b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 --2- 172.21.15.82:0/3408278581 >> v2:172.21.15.196:3300/0 conn(0x7f8c940b24a0 0x7f8c940b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 -- 172.21.15.82:0/3408278581 >> v1:172.21.15.196:6789/0 conn(0x7f8c940b3f50 legacy=0x7f8c940cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:33.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.866+0000 7f8c9959e640 1 -- 172.21.15.82:0/3408278581 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940b46c0 con 0x7f8c9415a700 2024-04-25T10:06:33.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9959e640 1 --2- 172.21.15.82:0/3408278581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c9415a700 0x7f8c9415ecc0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8c9406bd20 tx=0x7f8c9001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=458e231fd710938c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c8bfff640 1 -- 172.21.15.82:0/3408278581 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b0070 con 0x7f8c9415a700 2024-04-25T10:06:33.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c8bfff640 1 -- 172.21.15.82:0/3408278581 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c900ab600 con 0x7f8c9415a700 2024-04-25T10:06:33.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c8bfff640 1 -- 172.21.15.82:0/3408278581 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900ab9d0 con 0x7f8c9415a700 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- 172.21.15.82:0/3408278581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c9415a700 msgr2=0x7f8c9415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 --2- 172.21.15.82:0/3408278581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c9415a700 0x7f8c9415ecc0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8c9406bd20 tx=0x7f8c9001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- 172.21.15.82:0/3408278581 shutdown_connections 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 --2- 172.21.15.82:0/3408278581 >> v2:172.21.15.196:3300/0 conn(0x7f8c940b24a0 0x7f8c940b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 --2- 172.21.15.82:0/3408278581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c9415a700 0x7f8c9415ecc0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- 172.21.15.82:0/3408278581 >> 172.21.15.82:0/3408278581 conn(0x7f8c940b11a0 msgr2=0x7f8c940b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- 172.21.15.82:0/3408278581 shutdown_connections 2024-04-25T10:06:33.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- 172.21.15.82:0/3408278581 wait complete. 2024-04-25T10:06:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 Processor -- start 2024-04-25T10:06:33.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.870+0000 7f8c9b802640 1 -- start start 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 0x7f8c94163a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c9415a700 0x7f8c9413e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940cd310 con 0x7f8c94159b20 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940ce600 con 0x7f8c9415a700 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940cd490 con 0x7f8c940b24a0 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58076/0 (socket says 172.21.15.82:58076) 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 -- 172.21.15.82:0/509491565 learned_addr learned my addr 172.21.15.82:0/509491565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c98d9d640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c9415a700 0x7f8c9413e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9959e640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 0x7f8c94163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 -- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 msgr2=0x7f8c94163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 0x7f8c94163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 -- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c9415a700 msgr2=0x7f8c9413e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c9415a700 0x7f8c9413e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 -- 172.21.15.82:0/509491565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94164090 con 0x7f8c94159b20 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9959e640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 0x7f8c94163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c99d9f640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b9a0 tx=0x7f8c8c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c019070 con 0x7f8c94159b20 2024-04-25T10:06:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8c00cbc0 con 0x7f8c94159b20 2024-04-25T10:06:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c94164370 con 0x7f8c94159b20 2024-04-25T10:06:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c014440 con 0x7f8c94159b20 2024-04-25T10:06:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c94164930 con 0x7f8c94159b20 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c9404efa0 con 0x7f8c94159b20 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c8c01c070 con 0x7f8c94159b20 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 0x7f8c78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f8c8c084f30 con 0x7f8c94159b20 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9959e640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 0x7f8c78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.874+0000 7f8c9959e640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 0x7f8c78069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c900ad000 tx=0x7f8c90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:33.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:33.878+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c8c055d90 con 0x7f8c94159b20 2024-04-25T10:06:34.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.054+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c9406f260 con 0x7f8c94159b20 2024-04-25T10:06:34.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c89ffb640 1 -- 172.21.15.82:0/509491565 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f8c8c055690 con 0x7f8c94159b20 2024-04-25T10:06:34.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:34.060 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:06:06.293306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:34.061 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 msgr2=0x7f8c78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 0x7f8c78069420 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8c900ad000 tx=0x7f8c90008040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 msgr2=0x7f8c940cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00b9a0 tx=0x7f8c8c00be60 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 shutdown_connections 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8c78066f60 0x7f8c78069420 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c94159b20 0x7f8c940cf510 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8c9415a700 0x7f8c9413e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 --2- 172.21.15.82:0/509491565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c940b24a0 0x7f8c94163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 >> 172.21.15.82:0/509491565 conn(0x7f8c940b11a0 msgr2=0x7f8c94158910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 shutdown_connections 2024-04-25T10:06:34.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.058+0000 7f8c9b802640 1 -- 172.21.15.82:0/509491565 wait complete. 2024-04-25T10:06:34.077 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:34.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.194+0000 7efefb49d640 1 Processor -- start 2024-04-25T10:06:34.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.194+0000 7efefb49d640 1 -- start start 2024-04-25T10:06:34.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efefb49d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efef415da10 0x7efef4071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efefb49d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef4162310 con 0x7efef40b24a0 2024-04-25T10:06:34.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efefb49d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef40adbd0 con 0x7efef4071e90 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efefb49d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef40ae860 con 0x7efef415da10 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efef415da10 0x7efef4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef8a38640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efef40b24a0 0x7efef41614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54020/0 (socket says 172.21.15.82:54020) 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efef415da10 0x7efef4071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58084/0 (socket says 172.21.15.82:58084) 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef8a38640 1 -- 172.21.15.82:0/3066806552 learned_addr learned my addr 172.21.15.82:0/3066806552 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 -- 172.21.15.82:0/3066806552 >> v1:172.21.15.196:6789/0 conn(0x7efef4071e90 legacy=0x7efef4161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 -- 172.21.15.82:0/3066806552 >> v1:172.21.15.82:6789/0 conn(0x7efef40b24a0 legacy=0x7efef41614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efeebfff640 1 -- 172.21.15.82:0/3066806552 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2585481092 0 0) 0x7efef4162310 con 0x7efef40b24a0 2024-04-25T10:06:34.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 -- 172.21.15.82:0/3066806552 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef415e210 con 0x7efef415da10 2024-04-25T10:06:34.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efef9239640 1 --2- 172.21.15.82:0/3066806552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4071030 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efef00097c0 tx=0x7efef001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb1f3c21b2fcc78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efeebfff640 1 -- 172.21.15.82:0/3066806552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00b0070 con 0x7efef415da10 2024-04-25T10:06:34.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efeebfff640 1 -- 172.21.15.82:0/3066806552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef00ab600 con 0x7efef415da10 2024-04-25T10:06:34.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.198+0000 7efeebfff640 1 -- 172.21.15.82:0/3066806552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00ab9d0 con 0x7efef415da10 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/3066806552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 msgr2=0x7efef4071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 --2- 172.21.15.82:0/3066806552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4071030 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7efef00097c0 tx=0x7efef001c340 comp rx=0 tx=0).stop 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/3066806552 shutdown_connections 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 --2- 172.21.15.82:0/3066806552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4071030 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/3066806552 >> 172.21.15.82:0/3066806552 conn(0x7efef40b11a0 msgr2=0x7efef40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/3066806552 shutdown_connections 2024-04-25T10:06:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/3066806552 wait complete. 2024-04-25T10:06:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 Processor -- start 2024-04-25T10:06:34.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- start start 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efef4071e90 0x7efef414a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 0x7efef4147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef415ee70 con 0x7efef415da10 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef415ecf0 con 0x7efef40b24a0 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef416c4c0 con 0x7efef4071e90 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58086/0 (socket says 172.21.15.82:58086) 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 -- 172.21.15.82:0/1217586628 learned_addr learned my addr 172.21.15.82:0/1217586628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef8a38640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 0x7efef4147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9239640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efef4071e90 0x7efef414a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 -- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efef4071e90 msgr2=0x7efef414a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efef4071e90 0x7efef414a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 -- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 msgr2=0x7efef4147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 0x7efef4147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 -- 172.21.15.82:0/1217586628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4150e50 con 0x7efef415da10 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef8a38640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 0x7efef4147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efef9a3a640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b970 tx=0x7efeec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec019070 con 0x7efef415da10 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec00cb40 con 0x7efef415da10 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec014440 con 0x7efef415da10 2024-04-25T10:06:34.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef414dcd0 con 0x7efef415da10 2024-04-25T10:06:34.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.202+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef414e220 con 0x7efef415da10 2024-04-25T10:06:34.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef404efa0 con 0x7efef415da10 2024-04-25T10:06:34.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efeec01c070 con 0x7efef415da10 2024-04-25T10:06:34.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efee9ffb640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 0x7efed8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7efeec084e20 con 0x7efef415da10 2024-04-25T10:06:34.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efef9239640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 0x7efed8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.206+0000 7efef9239640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 0x7efed8069450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efef00ad000 tx=0x7efef00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.210+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efeec055c00 con 0x7efef415da10 2024-04-25T10:06:34.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.366+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efef414b460 con 0x7efef415da10 2024-04-25T10:06:34.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efee9ffb640 1 -- 172.21.15.82:0/1217586628 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17137 (secure 0 0 0) 0x7efeec055500 con 0x7efef415da10 2024-04-25T10:06:34.372 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:34.372 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":67,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:50.454515+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 msgr2=0x7efed8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 0x7efed8069450 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efef00ad000 tx=0x7efef00a6000 comp rx=0 tx=0).stop 2024-04-25T10:06:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 msgr2=0x7efef4154120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7efeec00b970 tx=0x7efeec00be30 comp rx=0 tx=0).stop 2024-04-25T10:06:34.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 shutdown_connections 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efed8066f90 0x7efed8069450 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efef415da10 0x7efef4154120 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efef40b24a0 0x7efef4147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 --2- 172.21.15.82:0/1217586628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efef4071e90 0x7efef414a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 >> 172.21.15.82:0/1217586628 conn(0x7efef40b11a0 msgr2=0x7efef40b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 shutdown_connections 2024-04-25T10:06:34.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.370+0000 7efefb49d640 1 -- 172.21.15.82:0/1217586628 wait complete. 2024-04-25T10:06:34.390 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:34.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:34.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.498+0000 7fb4b4d0f640 1 Processor -- start 2024-04-25T10:06:34.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.498+0000 7fb4b4d0f640 1 -- start start 2024-04-25T10:06:34.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4b4d0f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb4b006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4b4d0f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00ccc70 con 0x7fb4b006ec60 2024-04-25T10:06:34.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4b4d0f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b006dc40 con 0x7fb4b00ce510 2024-04-25T10:06:34.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4b4d0f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00cc2b0 con 0x7fb4b0144ca0 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4aed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb4b006ec60 0x7fb4b0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:54036/0 (socket says 172.21.15.82:54036) 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4aed9d640 1 -- 172.21.15.82:0/2682638014 learned_addr learned my addr 172.21.15.82:0/2682638014 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb4b006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505830040 0 0) 0x7fb4b00ccc70 con 0x7fb4b006ec60 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb49c003530 con 0x7fb4b006ec60 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3808082483 0 0) 0x7fb49c003530 con 0x7fb4b006ec60 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 msgr2=0x7fb4b006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb4b006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 >> v1:172.21.15.196:6789/0 conn(0x7fb4b00ce510 legacy=0x7fb4b00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b006df20 con 0x7fb4b006ec60 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb4a8001a00 con 0x7fb4b006ec60 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 >> v1:172.21.15.82:6789/0 conn(0x7fb4b006ec60 legacy=0x7fb4b0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb49c003a60 0x7fb49c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb49c004810 0x7fb49c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb49c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00cc2b0 con 0x7fb49c003a60 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00bb60 con 0x7fb49c004810 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49c00bd60 con 0x7fb4b0144ca0 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb49c003a60 0x7fb49c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb49c004810 0x7fb49c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4aed9d640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb49c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 msgr2=0x7fb49c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144ca0 0x7fb49c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb49c004810 msgr2=0x7fb49c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb49c004810 0x7fb49c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.502+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2682638014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb49c003530 con 0x7fb49c003a60 2024-04-25T10:06:34.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb49c003a60 0x7fb49c003e80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0001780 tx=0x7fb4a001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daa52631b66d679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00013a0 con 0x7fb49c003a60 2024-04-25T10:06:34.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a00050d0 con 0x7fb49c003a60 2024-04-25T10:06:34.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ad59a640 1 -- 172.21.15.82:0/2682638014 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00054a0 con 0x7fb49c003a60 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2682638014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb49c003a60 msgr2=0x7fb49c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2682638014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb49c003a60 0x7fb49c003e80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0001780 tx=0x7fb4a001c420 comp rx=0 tx=0).stop 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4aed9d640 1 -- 172.21.15.82:0/2682638014 reap_dead start 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2682638014 shutdown_connections 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2682638014 >> 172.21.15.82:0/2682638014 conn(0x7fb4b00b11a0 msgr2=0x7fb4b00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2682638014 shutdown_connections 2024-04-25T10:06:34.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2682638014 wait complete. 2024-04-25T10:06:34.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 Processor -- start 2024-04-25T10:06:34.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- start start 2024-04-25T10:06:34.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 0x7fb4b0143cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144210 0x7fb4b00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0142d90 con 0x7fb4b013cf90 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b006e290 con 0x7fb4b00cb920 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4b4d0f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b0142f10 con 0x7fb4b0144210 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58104/0 (socket says 172.21.15.82:58104) 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2888496151 learned_addr learned my addr 172.21.15.82:0/2888496151 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 0x7fb4b0143cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4aed9d640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144210 0x7fb4b00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144210 msgr2=0x7fb4b00c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144210 0x7fb4b00c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 msgr2=0x7fb4b0143cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 0x7fb4b0143cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 -- 172.21.15.82:0/2888496151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b006da70 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 0x7fb4b0143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.506+0000 7fb4ae59c640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001c250 tx=0x7fb49c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00b8070 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a0003a90 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0007020 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b013d370 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b01f1bc0 con 0x7fb4b013cf90 2024-04-25T10:06:34.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b0144620 con 0x7fb4b013cf90 2024-04-25T10:06:34.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4a00071c0 con 0x7fb4b013cf90 2024-04-25T10:06:34.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 0x7fb490069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fb4a011f670 con 0x7fb4b013cf90 2024-04-25T10:06:34.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 0x7fb490069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.510+0000 7fb4add9b640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 0x7fb490069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb498006d10 tx=0x7fb498008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.514+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4b0144620 con 0x7fb4b013cf90 2024-04-25T10:06:34.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.702+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b004efa0 con 0x7fb4b013cf90 2024-04-25T10:06:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4a6ffd640 1 -- 172.21.15.82:0/2888496151 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4937 (secure 0 0 0) 0x7fb4a0016130 con 0x7fb4b013cf90 2024-04-25T10:06:34.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:34.708 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:06:06.293306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:34.708 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 msgr2=0x7fb490069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 0x7fb490069420 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb498006d10 tx=0x7fb498008040 comp rx=0 tx=0).stop 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 msgr2=0x7fb4b00cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fb4a001c250 tx=0x7fb49c005840 comp rx=0 tx=0).stop 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 shutdown_connections 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb490066f60 0x7fb490069420 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4b0144210 0x7fb4b00c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4b00cb920 0x7fb4b0143cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 --2- 172.21.15.82:0/2888496151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4b013cf90 0x7fb4b00cb3e0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 >> 172.21.15.82:0/2888496151 conn(0x7fb4b00b11a0 msgr2=0x7fb4b00c4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 shutdown_connections 2024-04-25T10:06:34.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.706+0000 7fb4b4d0f640 1 -- 172.21.15.82:0/2888496151 wait complete. 2024-04-25T10:06:34.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.842+0000 7f0501dd6640 1 Processor -- start 2024-04-25T10:06:34.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 -- start start 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 0x7f04fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04fc06be60 con 0x7f04fc0d0620 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc0d0a70 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f0501dd6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04fc06c440 con 0x7f04fc15da80 2024-04-25T10:06:34.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39124/0 (socket says 172.21.15.82:39124) 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 -- 172.21.15.82:0/1084183091 learned_addr learned my addr 172.21.15.82:0/1084183091 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04fb7fe640 1 --2- 172.21.15.82:0/1084183091 >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 0x7f04fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 -- 172.21.15.82:0/1084183091 >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 msgr2=0x7f04fc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 --2- 172.21.15.82:0/1084183091 >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 0x7f04fc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 -- 172.21.15.82:0/1084183091 >> v1:172.21.15.196:6789/0 conn(0x7f04fc0d0620 legacy=0x7f04fc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 -- 172.21.15.82:0/1084183091 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc15de60 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04fb7fe640 1 --2- 172.21.15.82:0/1084183091 >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 0x7f04fc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04f2ffd640 1 --2- 172.21.15.82:0/1084183091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f04ec006fd0 tx=0x7f04ec01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edc53402a4c80a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04faffd640 1 -- 172.21.15.82:0/1084183091 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0b0070 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04faffd640 1 -- 172.21.15.82:0/1084183091 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ec004230 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.846+0000 7f04faffd640 1 -- 172.21.15.82:0/1084183091 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ec003620 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- 172.21.15.82:0/1084183091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d1cd0 msgr2=0x7f04fc0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- 172.21.15.82:0/1084183091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f04ec006fd0 tx=0x7f04ec01c4c0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- 172.21.15.82:0/1084183091 shutdown_connections 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- 172.21.15.82:0/1084183091 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc0d00e0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- 172.21.15.82:0/1084183091 >> v2:172.21.15.196:3300/0 conn(0x7f04fc15da80 0x7f04fc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- 172.21.15.82:0/1084183091 >> 172.21.15.82:0/1084183091 conn(0x7f04fc08b000 msgr2=0x7f04fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- 172.21.15.82:0/1084183091 shutdown_connections 2024-04-25T10:06:34.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- 172.21.15.82:0/1084183091 wait complete. 2024-04-25T10:06:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 Processor -- start 2024-04-25T10:06:34.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- start start 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc15ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 0x7f04fc0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16a730 con 0x7f04fc0d0620 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16a5b0 con 0x7f04fc0d1cd0 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f0501dd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04fc16a8b0 con 0x7f04fc15da80 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39128/0 (socket says 172.21.15.82:39128) 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 -- 172.21.15.82:0/3133999407 learned_addr learned my addr 172.21.15.82:0/3133999407 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04f2ffd640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc15ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fbfff640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 0x7f04fc0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 -- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 msgr2=0x7f04fc0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 0x7f04fc0b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 -- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04fc0d1cd0 msgr2=0x7f04fc15ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc15ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 -- 172.21.15.82:0/3133999407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04fc155620 con 0x7f04fc0d0620 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fbfff640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 0x7f04fc0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.850+0000 7f04fb7fe640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c820 tx=0x7f04e400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4015070 con 0x7f04fc0d0620 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4007a30 con 0x7f04fc0d0620 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04fc15f4c0 con 0x7f04fc0d0620 2024-04-25T10:06:34.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4007e40 con 0x7f04fc0d0620 2024-04-25T10:06:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04fc15fa80 con 0x7f04fc0d0620 2024-04-25T10:06:34.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04fc04efa0 con 0x7f04fc0d0620 2024-04-25T10:06:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04e4018070 con 0x7f04fc0d0620 2024-04-25T10:06:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 0x7f04dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f04e40850f0 con 0x7f04fc0d0620 2024-04-25T10:06:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f2ffd640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 0x7f04dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:34.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.854+0000 7f04f2ffd640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 0x7f04dc069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ad000 tx=0x7f04ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:34.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:34.858+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04e4055ef0 con 0x7f04fc0d0620 2024-04-25T10:06:35.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.038+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04fc0cb470 con 0x7f04fc0d0620 2024-04-25T10:06:35.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f04f8ff9640 1 -- 172.21.15.82:0/3133999407 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4937 (secure 0 0 0) 0x7f04e40557f0 con 0x7f04fc0d0620 2024-04-25T10:06:35.044 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:35.045 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:05:17.584870+0000","modified":"2024-04-25T10:06:06.293306+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6420":{"gid":6420,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":15,"addr":"172.21.15.82:6837/191077906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":191077906},{"type":"v1","addr":"172.21.15.82:6837","nonce":191077906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:06:35.045 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 34 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 msgr2=0x7f04dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 0x7f04dc069450 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ad000 tx=0x7f04ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 msgr2=0x7f04fc1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c820 tx=0x7f04e400cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 shutdown_connections 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f04dc066f90 0x7f04dc069450 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f04fc15da80 0x7f04fc0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f04fc0d1cd0 0x7f04fc15ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 --2- 172.21.15.82:0/3133999407 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f04fc0d0620 0x7f04fc1550e0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 >> 172.21.15.82:0/3133999407 conn(0x7f04fc08b000 msgr2=0x7f04fc160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 shutdown_connections 2024-04-25T10:06:35.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.042+0000 7f0501dd6640 1 -- 172.21.15.82:0/3133999407 wait complete. 2024-04-25T10:06:35.063 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:35.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 Processor -- start 2024-04-25T10:06:35.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 -- start start 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6abc0d07f0 0x7f6abc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc06be60 con 0x7f6abc15da80 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc06c440 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac295a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc15de60 con 0x7f6abc0d07f0 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39142/0 (socket says 172.21.15.82:39142) 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6abbfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6abc15da80 0x7f6abc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37394/0 (socket says 172.21.15.82:37394) 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 -- 172.21.15.82:0/2156979433 learned_addr learned my addr 172.21.15.82:0/2156979433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ab3fff640 1 --2- 172.21.15.82:0/2156979433 >> v2:172.21.15.196:3300/0 conn(0x7f6abc0d07f0 0x7f6abc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 -- 172.21.15.82:0/2156979433 >> v2:172.21.15.196:3300/0 conn(0x7f6abc0d07f0 msgr2=0x7f6abc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 --2- 172.21.15.82:0/2156979433 >> v2:172.21.15.196:3300/0 conn(0x7f6abc0d07f0 0x7f6abc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 -- 172.21.15.82:0/2156979433 >> v1:172.21.15.82:6789/0 conn(0x7f6abc15da80 legacy=0x7f6abc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6ac0ef7640 1 -- 172.21.15.82:0/2156979433 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0d2450 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.174+0000 7f6abb7fe640 1 -- 172.21.15.82:0/2156979433 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716394018 0 0) 0x7f6abc06be60 con 0x7f6abc15da80 2024-04-25T10:06:35.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac0ef7640 1 --2- 172.21.15.82:0/2156979433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6abc08d880 tx=0x7f6ab401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c032a9c1fb866c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6abb7fe640 1 -- 172.21.15.82:0/2156979433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40a58a0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6abb7fe640 1 -- 172.21.15.82:0/2156979433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab40a5a40 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6abb7fe640 1 -- 172.21.15.82:0/2156979433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40a5e10 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- 172.21.15.82:0/2156979433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 msgr2=0x7f6abc06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- 172.21.15.82:0/2156979433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f6abc08d880 tx=0x7f6ab401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- 172.21.15.82:0/2156979433 shutdown_connections 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- 172.21.15.82:0/2156979433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc06c8d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- 172.21.15.82:0/2156979433 >> v2:172.21.15.196:3300/0 conn(0x7f6abc0d07f0 0x7f6abc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- 172.21.15.82:0/2156979433 >> 172.21.15.82:0/2156979433 conn(0x7f6abc08b000 msgr2=0x7f6abc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- 172.21.15.82:0/2156979433 shutdown_connections 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- 172.21.15.82:0/2156979433 wait complete. 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 Processor -- start 2024-04-25T10:06:35.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- start start 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6abc0d07f0 0x7f6abc159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc15da80 0x7f6abc160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc06d7a0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc06d620 con 0x7f6abc15da80 2024-04-25T10:06:35.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac295a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc06d920 con 0x7f6abc0d07f0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ab3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39156/0 (socket says 172.21.15.82:39156) 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ab3fff640 1 -- 172.21.15.82:0/3895186469 learned_addr learned my addr 172.21.15.82:0/3895186469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6ac0ef7640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc15da80 0x7f6abc160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.178+0000 7f6abbfff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6abc0d07f0 0x7f6abc159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 -- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6abc0d07f0 msgr2=0x7f6abc159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6abc0d07f0 0x7f6abc159d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 -- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc15da80 msgr2=0x7f6abc160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc15da80 0x7f6abc160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 -- 172.21.15.82:0/3895186469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc06ca60 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab3fff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d7d0 tx=0x7f6aa400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400b6c0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa4019070 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc0b6d60 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa400bae0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc0b7320 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc0d2c00 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6aa400bc80 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 0x7f6a9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7f6aa4085650 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6abbfff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 0x7f6a9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.182+0000 7f6abbfff640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 0x7f6a9c069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6aac000f70 tx=0x7f6aac007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.186+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6aa40564d0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6abc04ef80 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ab97fa640 1 -- 172.21.15.82:0/3895186469 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17137 (secure 0 0 0) 0x7f6aa4055dd0 con 0x7f6abc0d1cd0 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:35.353 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":67,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:05:50.454515+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:05:15.838157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:05:16.556491+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.17","mappings":[{"from":7,"to":2}]},{"pgid":"10.19","mappings":[{"from":1,"to":3}]},{"pgid":"10.1b","mappings":[{"from":4,"to":5}]},{"pgid":"10.25","mappings":[{"from":4,"to":3}]},{"pgid":"10.38","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 msgr2=0x7f6a9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 0x7f6a9c069450 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6aac000f70 tx=0x7f6aac007c10 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 msgr2=0x7f6abc0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f6aa400d7d0 tx=0x7f6aa400dc90 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 shutdown_connections 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a9c066f90 0x7f6a9c069450 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6abc15da80 0x7f6abc160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6abc0d1cd0 0x7f6abc0b6310 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 --2- 172.21.15.82:0/3895186469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6abc0d07f0 0x7f6abc159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.350+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 >> 172.21.15.82:0/3895186469 conn(0x7f6abc08b000 msgr2=0x7f6abc0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.354+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 shutdown_connections 2024-04-25T10:06:35.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.354+0000 7f6ac295a640 1 -- 172.21.15.82:0/3895186469 wait complete. 2024-04-25T10:06:35.373 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:35.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 Processor -- start 2024-04-25T10:06:35.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 -- start start 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbda006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda00ccc70 con 0x7fbda00ce510 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda006dc40 con 0x7fbda006ec60 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbda5ab0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00cc2b0 con 0x7fbda0144ca0 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbda00ce510 0x7fbda00ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37406/0 (socket says 172.21.15.82:37406) 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9f7fe640 1 -- 172.21.15.82:0/4200247791 learned_addr learned my addr 172.21.15.82:0/4200247791 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9effd640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbda006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832012857 0 0) 0x7fbda00ccc70 con 0x7fbda00ce510 2024-04-25T10:06:35.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c003530 con 0x7fbda00ce510 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4283769875 0 0) 0x7fbd8c003530 con 0x7fbda00ce510 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 msgr2=0x7fbda006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbda006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 >> v1:172.21.15.196:6789/0 conn(0x7fbda006ec60 legacy=0x7fbda0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda006df20 con 0x7fbda00ce510 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fbd98001680 con 0x7fbda00ce510 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 >> v1:172.21.15.82:6789/0 conn(0x7fbda00ce510 legacy=0x7fbda00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd8c003a60 0x7fbd8c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbd8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbd8c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda00cc2b0 con 0x7fbd8c004d50 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c00bb70 con 0x7fbd8c003a60 2024-04-25T10:06:35.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c00bd70 con 0x7fbda0144ca0 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9ffff640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbd8c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.486+0000 7fbd9f7fe640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd8c003a60 0x7fbd8c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9effd640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbd8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 -- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 msgr2=0x7fbd8c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda0144ca0 0x7fbd8c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 -- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd8c003a60 msgr2=0x7fbd8c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd8c003a60 0x7fbd8c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 -- 172.21.15.82:0/4200247791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c003530 con 0x7fbd8c004d50 2024-04-25T10:06:35.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbd8c0051d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbda004f950 tx=0x7fbd9001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b566a719f747f3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90001a90 con 0x7fbd8c004d50 2024-04-25T10:06:35.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd90003b80 con 0x7fbd8c004d50 2024-04-25T10:06:35.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9e7fc640 1 -- 172.21.15.82:0/4200247791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd900050d0 con 0x7fbd8c004d50 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 -- 172.21.15.82:0/4200247791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 msgr2=0x7fbd8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/4200247791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbd8c0051d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbda004f950 tx=0x7fbd9001c700 comp rx=0 tx=0).stop 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbd9ffff640 1 -- 172.21.15.82:0/4200247791 reap_dead start 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 -- 172.21.15.82:0/4200247791 shutdown_connections 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 -- 172.21.15.82:0/4200247791 >> 172.21.15.82:0/4200247791 conn(0x7fbda00b11a0 msgr2=0x7fbda00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 -- 172.21.15.82:0/4200247791 shutdown_connections 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.490+0000 7fbda5ab0640 1 -- 172.21.15.82:0/4200247791 wait complete. 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 Processor -- start 2024-04-25T10:06:35.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- start start 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda01e3aa0 0x7fbda0134f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbda01e6230 0x7fbda01e66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0142d90 con 0x7fbd8c004d50 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda006e290 con 0x7fbda01e6230 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda0142f10 con 0x7fbda01e3aa0 2024-04-25T10:06:35.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39180/0 (socket says 172.21.15.82:39180) 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 -- 172.21.15.82:0/1989788293 learned_addr learned my addr 172.21.15.82:0/1989788293 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9ffff640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbda01e6230 0x7fbda01e66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9effd640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda01e3aa0 0x7fbda0134f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 -- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda01e3aa0 msgr2=0x7fbda0134f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda01e3aa0 0x7fbda0134f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 -- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbda01e6230 msgr2=0x7fbda01e66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbda01e6230 0x7fbda01e66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 -- 172.21.15.82:0/1989788293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda01e3f00 con 0x7fbd8c004d50 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd9f7fe640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbd980072f0 tx=0x7fbd980077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800f9c0 con 0x7fbd8c004d50 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98018070 con 0x7fbd8c004d50 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda01e1370 con 0x7fbd8c004d50 2024-04-25T10:06:35.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800fdc0 con 0x7fbd8c004d50 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda01e1930 con 0x7fbd8c004d50 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda004efa0 con 0x7fbd8c004d50 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.494+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd9801d460 con 0x7fbd8c004d50 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.498+0000 7fbd97fff640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 0x7fbd80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.498+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6246+0+0 (secure 0 0 0) 0x7fbd980853a0 con 0x7fbd8c004d50 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.498+0000 7fbd9effd640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 0x7fbd80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.498+0000 7fbd9effd640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 0x7fbd80069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbda01f4c40 tx=0x7fbd88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.502+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd98056190 con 0x7fbd8c004d50 2024-04-25T10:06:35.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.682+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbda00b1cc0 con 0x7fbd8c004d50 2024-04-25T10:06:35.685 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:06:35.682+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:35.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbd97fff640 1 -- 172.21.15.82:0/1989788293 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd98055a90 con 0x7fbd8c004d50 2024-04-25T10:06:35.688 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 msgr2=0x7fbd80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 0x7fbd80069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbda01f4c40 tx=0x7fbd88008040 comp rx=0 tx=0).stop 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 msgr2=0x7fbda01e0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbd980072f0 tx=0x7fbd980077b0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 shutdown_connections 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd80066f60 0x7fbd80069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbda01e3aa0 0x7fbda0134f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbda01e6230 0x7fbda01e66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 --2- 172.21.15.82:0/1989788293 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd8c004d50 0x7fbda01e0c90 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 >> 172.21.15.82:0/1989788293 conn(0x7fbda00b11a0 msgr2=0x7fbda0144070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 shutdown_connections 2024-04-25T10:06:35.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.686+0000 7fbda5ab0640 1 -- 172.21.15.82:0/1989788293 wait complete. 2024-04-25T10:06:35.697 INFO:tasks.ceph.mds.c.smithi082.stderr: -6431> 2024-04-25T10:05:23.506+0000 7feefa592640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:35.702 INFO:tasks.ceph.mds.c.smithi082.stderr: -4787> 2024-04-25T10:05:27.726+0000 7feef658a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6585=0-4194304@1} caps={6585=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6585 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x56489997fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:35.707 INFO:tasks.ceph.mds.c.smithi082.stderr: -2863> 2024-04-25T10:05:59.082+0000 7feef658a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T10:05:26.630910+0000 1=1+0) (iversion lock) caps={6585=pAsLsXsFs/-@2},l=6585(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x56489997fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:06:35.708 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:35.740 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:06:35.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 Processor -- start 2024-04-25T10:06:35.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 -- start start 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbae815ba10 0x7fbae809a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 0x7fbae80ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae809af10 con 0x7fbae80ccb50 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae809a540 con 0x7fbae815ba10 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbaed072640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8099c10 con 0x7fbae815faf0 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbae759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 0x7fbae80ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbae6d9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbae815ba10 0x7fbae809a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbae759e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 0x7fbae80ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:37628/0 (socket says 172.21.15.82:37628) 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbae759e640 1 -- 172.21.15.82:0/1256343964 learned_addr learned my addr 172.21.15.82:0/1256343964 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.822+0000 7fbae659c640 1 --1- 172.21.15.82:0/1256343964 >> v1:172.21.15.196:6789/0 conn(0x7fbae80ccb50 0x7fbae815f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:59438/0 (socket says 172.21.15.82:59438) 2024-04-25T10:06:35.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae6d9d640 1 -- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 msgr2=0x7fbae80ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:35.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 0x7fbae80ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae6d9d640 1 -- 172.21.15.82:0/1256343964 >> v1:172.21.15.196:6789/0 conn(0x7fbae80ccb50 legacy=0x7fbae815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae6d9d640 1 -- 172.21.15.82:0/1256343964 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae8099ef0 con 0x7fbae815ba10 2024-04-25T10:06:35.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815ba10 0x7fbae809a9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbad80097c0 tx=0x7fbad801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b93f64b4fd1568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae5d9b640 1 -- 172.21.15.82:0/1256343964 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80b1070 con 0x7fbae815ba10 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae5d9b640 1 -- 172.21.15.82:0/1256343964 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad80ac410 con 0x7fbae815ba10 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbae5d9b640 1 -- 172.21.15.82:0/1256343964 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad80ac7e0 con 0x7fbae815ba10 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 -- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815ba10 msgr2=0x7fbae809a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 --2- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815ba10 0x7fbae809a9d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbad80097c0 tx=0x7fbad801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 -- 172.21.15.82:0/1256343964 shutdown_connections 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 --2- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae815faf0 0x7fbae80ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 --2- 172.21.15.82:0/1256343964 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815ba10 0x7fbae809a9d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 -- 172.21.15.82:0/1256343964 >> 172.21.15.82:0/1256343964 conn(0x7fbae80b3060 msgr2=0x7fbae80b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 -- 172.21.15.82:0/1256343964 shutdown_connections 2024-04-25T10:06:35.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.826+0000 7fbaed072640 1 -- 172.21.15.82:0/1256343964 wait complete. 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 Processor -- start 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- start start 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 0x7fbae815a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 0x7fbae80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae80c5ca0 con 0x7fbae815ba10 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae80c5b20 con 0x7fbae815faf0 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8168720 con 0x7fbae80ccb50 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39214/0 (socket says 172.21.15.82:39214) 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 -- 172.21.15.82:0/2317766581 learned_addr learned my addr 172.21.15.82:0/2317766581 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae759e640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 0x7fbae80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 0x7fbae815a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 -- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 msgr2=0x7fbae815a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 0x7fbae815a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 -- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 msgr2=0x7fbae80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 0x7fbae80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 -- 172.21.15.82:0/2317766581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae815cc90 con 0x7fbae815ba10 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae759e640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 0x7fbae80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 0x7fbae815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbae659c640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c430 tx=0x7fbadc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0074e0 con 0x7fbae815ba10 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc007680 con 0x7fbae815ba10 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae815cf70 con 0x7fbae815ba10 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc004a10 con 0x7fbae815ba10 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae80c8640 con 0x7fbae815ba10 2024-04-25T10:06:35.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.830+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae804efa0 con 0x7fbae815ba10 2024-04-25T10:06:35.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.834+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbadc01a020 con 0x7fbae815ba10 2024-04-25T10:06:35.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.834+0000 7fbacf7fe640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 0x7fbad4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:35.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.834+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6332+0+0 (secure 0 0 0) 0x7fbadc084a80 con 0x7fbae815ba10 2024-04-25T10:06:35.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.834+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 0x7fbad4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:35.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.834+0000 7fbae6d9d640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 0x7fbad4069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbad80ae000 tx=0x7fbad8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:35.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:35.838+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbae809af10 con 0x7fbae815ba10 2024-04-25T10:06:36.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.030+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbae80b3c10 con 0x7fbae815ba10 2024-04-25T10:06:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbacf7fe640 1 -- 172.21.15.82:0/2317766581 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbadc0557a0 con 0x7fbae815ba10 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 msgr2=0x7fbad4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 0x7fbad4069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbad80ae000 tx=0x7fbad8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 msgr2=0x7fbae80c8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00c430 tx=0x7fbadc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 shutdown_connections 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbad4066f90 0x7fbad4069450 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbae815faf0 0x7fbae80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbae815ba10 0x7fbae80c8e40 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 --2- 172.21.15.82:0/2317766581 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbae80ccb50 0x7fbae815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 >> 172.21.15.82:0/2317766581 conn(0x7fbae80b3060 msgr2=0x7fbae815e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 shutdown_connections 2024-04-25T10:06:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.690+0000 7fbaed072640 1 -- 172.21.15.82:0/2317766581 wait complete. 2024-04-25T10:06:36.715 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 Processor -- start 2024-04-25T10:06:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 -- start start 2024-04-25T10:06:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4cb015ec90 0x7f4cb009a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb0099c10 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cb00cf930 con 0x7f4cb015ec90 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4cb54e7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0099ef0 con 0x7f4cb006f580 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4cb015ec90 0x7f4cb009a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4cb015ec90 0x7f4cb009a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39228/0 (socket says 172.21.15.82:39228) 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 -- 172.21.15.82:0/3861542538 learned_addr learned my addr 172.21.15.82:0/3861542538 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caf7fe640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 -- 172.21.15.82:0/3861542538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 msgr2=0x7f4cb00cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 -- 172.21.15.82:0/3861542538 >> v1:172.21.15.196:6789/0 conn(0x7f4cb00cf0e0 legacy=0x7f4cb006ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caeffd640 1 -- 172.21.15.82:0/3861542538 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb009a1d0 con 0x7f4cb015ec90 2024-04-25T10:06:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.874+0000 7f4caf7fe640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb015ec90 0x7f4cb009a9d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4cb006ead0 tx=0x7f4ca801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e2ae3714b03bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cadffb640 1 -- 172.21.15.82:0/3861542538 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80b2070 con 0x7f4cb015ec90 2024-04-25T10:06:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cadffb640 1 -- 172.21.15.82:0/3861542538 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca80ad640 con 0x7f4cb015ec90 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cadffb640 1 -- 172.21.15.82:0/3861542538 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca80ada10 con 0x7f4cb015ec90 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3861542538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb015ec90 msgr2=0x7f4cb009a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb015ec90 0x7f4cb009a9d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4cb006ead0 tx=0x7f4ca801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3861542538 shutdown_connections 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3861542538 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb015ec90 0x7f4cb009a9d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3861542538 >> 172.21.15.82:0/3861542538 conn(0x7f4cb00b3000 msgr2=0x7f4cb00b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3861542538 shutdown_connections 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3861542538 wait complete. 2024-04-25T10:06:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 Processor -- start 2024-04-25T10:06:36.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.878+0000 7f4cb54e7640 1 -- start start 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00c9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 0x7f4cb01f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb006fb10 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb006f990 con 0x7f4cb015ec90 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb0168730 con 0x7f4cb006f580 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39240/0 (socket says 172.21.15.82:39240) 2024-04-25T10:06:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 -- 172.21.15.82:0/3281467704 learned_addr learned my addr 172.21.15.82:0/3281467704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00c9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caf7fe640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 0x7f4cb01f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 -- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 msgr2=0x7f4cb00c9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00c9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 -- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 msgr2=0x7f4cb01f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 0x7f4cb01f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 -- 172.21.15.82:0/3281467704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00ca570 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caf7fe640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 0x7f4cb01f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cae7fc640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c7c0 tx=0x7f4ca400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4016020 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca400aa50 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb014c330 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4012420 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb014c8f0 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb014caf0 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ca4019020 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 0x7f4c9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6332+0+0 (secure 0 0 0) 0x7f4ca40852e0 con 0x7f4cb00cf0e0 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 0x7f4c9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:36.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.882+0000 7f4caeffd640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 0x7f4c9c069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80af000 tx=0x7f4ca80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:36.890+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4cb014caf0 con 0x7f4cb00cf0e0 2024-04-25T10:06:37.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.038+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4cb00cedc0 con 0x7f4cb00cf0e0 2024-04-25T10:06:37.176 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:37.179 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:37.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.690+0000 7f4c977fe640 1 -- 172.21.15.82:0/3281467704 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4ca4020030 con 0x7f4cb00cf0e0 2024-04-25T10:06:37.696 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 msgr2=0x7f4c9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 0x7f4c9c069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4ca80af000 tx=0x7f4ca80a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 msgr2=0x7f4cb006d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400c7c0 tx=0x7f4ca400cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 shutdown_connections 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4c9c066f60 0x7f4c9c069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4cb015ec90 0x7f4cb01f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4cb00cf0e0 0x7f4cb006d5a0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 --2- 172.21.15.82:0/3281467704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4cb006f580 0x7f4cb00c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 >> 172.21.15.82:0/3281467704 conn(0x7f4cb00b3000 msgr2=0x7f4cb00ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 shutdown_connections 2024-04-25T10:06:37.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.698+0000 7f4cb54e7640 1 -- 172.21.15.82:0/3281467704 wait complete. 2024-04-25T10:06:37.726 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 Processor -- start 2024-04-25T10:06:37.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 -- start start 2024-04-25T10:06:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb11815dd20 0x7fb11809a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180d0d00 con 0x7fb118099a70 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1180d0740 con 0x7fb1180cc4c0 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11cdbe640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb11809ad50 con 0x7fb11815dd20 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb118099a70 0x7fb118099e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37418/0 (socket says 172.21.15.82:37418) 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11659c640 1 -- 172.21.15.82:0/3514701817 learned_addr learned my addr 172.21.15.82:0/3514701817 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb115d9b640 1 --2- 172.21.15.82:0/3514701817 >> v2:172.21.15.196:3300/0 conn(0x7fb11815dd20 0x7fb11809a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818593555 0 0) 0x7fb1100014a0 con 0x7fb118099a70 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb104003530 con 0x7fb118099a70 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2996597716 0 0) 0x7fb104003530 con 0x7fb118099a70 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 >> v2:172.21.15.196:3300/0 conn(0x7fb11815dd20 msgr2=0x7fb11809a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 --2- 172.21.15.82:0/3514701817 >> v2:172.21.15.196:3300/0 conn(0x7fb11815dd20 0x7fb11809a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 >> v1:172.21.15.196:6789/0 conn(0x7fb1180cc4c0 legacy=0x7fb1180cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180d0a20 con 0x7fb118099a70 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb110001bd0 con 0x7fb118099a70 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 >> v1:172.21.15.82:6789/0 conn(0x7fb118099a70 legacy=0x7fb118099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1040039c0 0x7fb104003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11809ad50 con 0x7fb1040051d0 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10400c170 con 0x7fb104004810 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10400c370 con 0x7fb1040039c0 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb11659c640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1040039c0 0x7fb104003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb115d9b640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 -- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1040039c0 msgr2=0x7fb104003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1040039c0 0x7fb104003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 -- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb104004810 msgr2=0x7fb104004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb116d9d640 1 -- 172.21.15.82:0/3514701817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104003530 con 0x7fb1040051d0 2024-04-25T10:06:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.854+0000 7fb115d9b640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb116d9d640 1 -- 172.21.15.82:0/3514701817 reap_dead start 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb116d9d640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb108001780 tx=0x7fb10801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3460128c4d767ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080013a0 con 0x7fb1040051d0 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1080a9070 con 0x7fb1040051d0 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11559a640 1 -- 172.21.15.82:0/3514701817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb108003600 con 0x7fb1040051d0 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 -- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 msgr2=0x7fb10400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb108001780 tx=0x7fb10801c420 comp rx=0 tx=0).stop 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 -- 172.21.15.82:0/3514701817 shutdown_connections 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/3514701817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1040051d0 0x7fb10400b9d0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 -- 172.21.15.82:0/3514701817 >> 172.21.15.82:0/3514701817 conn(0x7fb1180b3030 msgr2=0x7fb1180b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 -- 172.21.15.82:0/3514701817 shutdown_connections 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.858+0000 7fb11cdbe640 1 -- 172.21.15.82:0/3514701817 wait complete. 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 Processor -- start 2024-04-25T10:06:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- start start 2024-04-25T10:06:37.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1040051d0 0x7fb1180c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1180b5e10 0x7fb118160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180b3d80 con 0x7fb118159ed0 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180cc030 con 0x7fb1040051d0 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1180d0320 con 0x7fb1180b5e10 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39270/0 (socket says 172.21.15.82:39270) 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 -- 172.21.15.82:0/2738641245 learned_addr learned my addr 172.21.15.82:0/2738641245 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11659c640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1040051d0 0x7fb1180c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 -- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1180b5e10 msgr2=0x7fb118160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1180b5e10 0x7fb118160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 -- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1040051d0 msgr2=0x7fb1180c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1040051d0 0x7fb1180c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 -- 172.21.15.82:0/2738641245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1180ceee0 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb116d9d640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb10801c250 tx=0x7fb104005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080bb070 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb108004ec0 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1080013a0 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11815a890 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1180c98e0 con 0x7fb118159ed0 2024-04-25T10:06:37.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb118068340 con 0x7fb118159ed0 2024-04-25T10:06:37.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.862+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1080be070 con 0x7fb118159ed0 2024-04-25T10:06:37.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.866+0000 7fb10effd640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 0x7fb0f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:37.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.866+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5865+0+0 (secure 0 0 0) 0x7fb10811e2d0 con 0x7fb118159ed0 2024-04-25T10:06:37.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.866+0000 7fb11659c640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 0x7fb0f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:37.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.866+0000 7fb11659c640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 0x7fb0f8069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb1100013e0 tx=0x7fb1100016b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:37.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:37.870+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb108016130 con 0x7fb118159ed0 2024-04-25T10:06:38.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.018+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb1180cecc0 con 0x7fb118159ed0 2024-04-25T10:06:38.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.706+0000 7fb10effd640 1 -- 172.21.15.82:0/2738641245 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb1080ef2d0 con 0x7fb118159ed0 2024-04-25T10:06:38.710 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 msgr2=0x7fb0f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 0x7fb0f8069420 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb1100013e0 tx=0x7fb1100016b0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 msgr2=0x7fb11815a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb10801c250 tx=0x7fb104005840 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 shutdown_connections 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb0f8066f60 0x7fb0f8069420 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1180b5e10 0x7fb118160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb118159ed0 0x7fb11815a350 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 --2- 172.21.15.82:0/2738641245 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1040051d0 0x7fb1180c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 >> 172.21.15.82:0/2738641245 conn(0x7fb1180b3030 msgr2=0x7fb118099e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 shutdown_connections 2024-04-25T10:06:38.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.714+0000 7fb11cdbe640 1 -- 172.21.15.82:0/2738641245 wait complete. 2024-04-25T10:06:38.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:38.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 Processor -- start 2024-04-25T10:06:38.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 -- start start 2024-04-25T10:06:38.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb5140cfca0 0x7fb51415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb5140caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5140cbf10 con 0x7fb5140b3cf0 2024-04-25T10:06:38.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5140cc1f0 con 0x7fb5140cfca0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb518d77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140cc4d0 con 0x7fb51415e4f0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb5140b3cf0 0x7fb5140cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37430/0 (socket says 172.21.15.82:37430) 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 -- 172.21.15.82:0/3360510969 learned_addr learned my addr 172.21.15.82:0/3360510969 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1165373504 0 0) 0x7fb5140cbf10 con 0x7fb5140b3cf0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4fc003530 con 0x7fb5140b3cf0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb511d9b640 1 --2- 172.21.15.82:0/3360510969 >> v2:172.21.15.196:3300/0 conn(0x7fb5140cfca0 0x7fb51415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb5140caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3925974852 0 0) 0x7fb4fc003530 con 0x7fb5140b3cf0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 msgr2=0x7fb5140caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb5140caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 >> v2:172.21.15.196:3300/0 conn(0x7fb5140cfca0 msgr2=0x7fb51415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 --2- 172.21.15.82:0/3360510969 >> v2:172.21.15.196:3300/0 conn(0x7fb5140cfca0 0x7fb51415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140cc7b0 con 0x7fb5140b3cf0 2024-04-25T10:06:38.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb504001680 con 0x7fb5140b3cf0 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 >> v1:172.21.15.82:6789/0 conn(0x7fb5140b3cf0 legacy=0x7fb5140cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4fc003a30 0x7fb4fc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb4fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4fc004d50 0x7fb4fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140cc1f0 con 0x7fb4fc003a30 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140cc4d0 con 0x7fb4fc004d50 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc00bab0 con 0x7fb51415e4f0 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4fc003a30 0x7fb4fc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4fc004d50 0x7fb4fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb511d9b640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb4fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 -- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 msgr2=0x7fb4fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb51415e4f0 0x7fb4fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 -- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4fc004d50 msgr2=0x7fb4fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:38.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4fc004d50 0x7fb4fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb51259c640 1 -- 172.21.15.82:0/3360510969 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140cc7b0 con 0x7fb4fc003a30 2024-04-25T10:06:38.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.866+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4fc004d50 0x7fb4fc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb51259c640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4fc003a30 0x7fb4fc003e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c340 tx=0x7fb50401cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851094f9730910a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504007e60 con 0x7fb4fc003a30 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5040a9020 con 0x7fb4fc003a30 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb51159a640 1 -- 172.21.15.82:0/3360510969 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb504005b00 con 0x7fb4fc003a30 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- 172.21.15.82:0/3360510969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4fc003a30 msgr2=0x7fb4fc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 --2- 172.21.15.82:0/3360510969 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4fc003a30 0x7fb4fc003e50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb50400c340 tx=0x7fb50401cda0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb512d9d640 1 -- 172.21.15.82:0/3360510969 reap_dead start 2024-04-25T10:06:38.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- 172.21.15.82:0/3360510969 shutdown_connections 2024-04-25T10:06:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- 172.21.15.82:0/3360510969 >> 172.21.15.82:0/3360510969 conn(0x7fb51406ae60 msgr2=0x7fb51406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- 172.21.15.82:0/3360510969 shutdown_connections 2024-04-25T10:06:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- 172.21.15.82:0/3360510969 wait complete. 2024-04-25T10:06:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 Processor -- start 2024-04-25T10:06:38.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.870+0000 7fb518d77640 1 -- start start 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 0x7fb514153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 0x7fb51415ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140cb300 con 0x7fb51415eed0 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51415eae0 con 0x7fb5141540b0 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5140cb480 con 0x7fb4fc003a30 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39288/0 (socket says 172.21.15.82:39288) 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 -- 172.21.15.82:0/3111203517 learned_addr learned my addr 172.21.15.82:0/3111203517 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb51259c640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 0x7fb514153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb511d9b640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 0x7fb51415ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 -- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 msgr2=0x7fb514153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 0x7fb514153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 -- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 msgr2=0x7fb51415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 0x7fb51415ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 -- 172.21.15.82:0/3111203517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140b0b70 con 0x7fb51415eed0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb51259c640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 0x7fb514153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb511d9b640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 0x7fb51415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb512d9d640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cf2c0 tx=0x7fb50c004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c007ce0 con 0x7fb51415eed0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50c019070 con 0x7fb51415eed0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51415c4c0 con 0x7fb51415eed0 2024-04-25T10:06:38.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50c014650 con 0x7fb51415eed0 2024-04-25T10:06:38.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5140c6430 con 0x7fb51415eed0 2024-04-25T10:06:38.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5140c66f0 con 0x7fb51415eed0 2024-04-25T10:06:38.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb50c0147f0 con 0x7fb51415eed0 2024-04-25T10:06:38.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 0x7fb4f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:38.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb50c0853c0 con 0x7fb51415eed0 2024-04-25T10:06:38.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb51259c640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 0x7fb4f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:38.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.874+0000 7fb51259c640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 0x7fb4f4069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb5040aa000 tx=0x7fb504001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:38.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:38.878+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb50c0565d0 con 0x7fb51415eed0 2024-04-25T10:06:39.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.042+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb51415c2b0 con 0x7fb51415eed0 2024-04-25T10:06:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb502ffd640 1 -- 172.21.15.82:0/3111203517 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb50c055ed0 con 0x7fb51415eed0 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 msgr2=0x7fb4f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 0x7fb4f4069420 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb5040aa000 tx=0x7fb504001a50 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 msgr2=0x7fb51415d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb5140cf2c0 tx=0x7fb50c004840 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 shutdown_connections 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb4f4066f60 0x7fb4f4069420 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb51415eed0 0x7fb51415d200 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb5141540b0 0x7fb51415ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 --2- 172.21.15.82:0/3111203517 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4fc003a30 0x7fb514153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 >> 172.21.15.82:0/3111203517 conn(0x7fb51406ae60 msgr2=0x7fb5140cd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 shutdown_connections 2024-04-25T10:06:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.046+0000 7fb518d77640 1 -- 172.21.15.82:0/3111203517 wait complete. 2024-04-25T10:06:39.069 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T10:06:39.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 Processor -- start 2024-04-25T10:06:39.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 -- start start 2024-04-25T10:06:39.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70180ce510 0x7f70180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f7018142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70180cee30 con 0x7f7018144ca0 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70180cc2b0 con 0x7f70180ce510 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f702008a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f701806e290 con 0x7f70181451f0 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70180ce510 0x7f70180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701d625640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f7018144ca0 0x7f701806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:59460/0 (socket says 172.21.15.82:59460) 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70180ce510 0x7f70180ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:58774/0 (socket says 172.21.15.82:58774) 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701d625640 1 -- 172.21.15.82:0/3665704637 learned_addr learned my addr 172.21.15.82:0/3665704637 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:39.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701e627640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f7018142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 -- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 msgr2=0x7f7018142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f7018142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 -- 172.21.15.82:0/3665704637 >> v1:172.21.15.196:6789/0 conn(0x7f7018144ca0 legacy=0x7f701806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.178+0000 7f701de26640 1 -- 172.21.15.82:0/3665704637 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180ccc70 con 0x7f70180ce510 2024-04-25T10:06:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701de26640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70180ce8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70140097c0 tx=0x7f701401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81a2637411d76c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701ce24640 1 -- 172.21.15.82:0/3665704637 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140b5070 con 0x7f70180ce510 2024-04-25T10:06:39.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701ce24640 1 -- 172.21.15.82:0/3665704637 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70140a8990 con 0x7f70180ce510 2024-04-25T10:06:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701ce24640 1 -- 172.21.15.82:0/3665704637 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70140a8d60 con 0x7f70180ce510 2024-04-25T10:06:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 msgr2=0x7f70180ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70180ce8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f70140097c0 tx=0x7f701401ec90 comp rx=0 tx=0).stop 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- 172.21.15.82:0/3665704637 shutdown_connections 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f7018142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- 172.21.15.82:0/3665704637 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70180ce8f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- 172.21.15.82:0/3665704637 >> 172.21.15.82:0/3665704637 conn(0x7f70180b11a0 msgr2=0x7f70180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- 172.21.15.82:0/3665704637 shutdown_connections 2024-04-25T10:06:39.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- 172.21.15.82:0/3665704637 wait complete. 2024-04-25T10:06:39.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 Processor -- start 2024-04-25T10:06:39.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- start start 2024-04-25T10:06:39.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70181e76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f70180c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701806ee30 con 0x7f7018144ca0 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701806df20 con 0x7f70180ce510 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f702008a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70180cc900 con 0x7f70181451f0 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701d625640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701d625640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39294/0 (socket says 172.21.15.82:39294) 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701d625640 1 -- 172.21.15.82:0/477873447 learned_addr learned my addr 172.21.15.82:0/477873447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.182+0000 7f701de26640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70181e76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701e627640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f70180c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 -- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 msgr2=0x7f70180c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f70180c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 -- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 msgr2=0x7f70181e76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70181e76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 -- 172.21.15.82:0/477873447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70181e0d60 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701d625640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f700800c430 tx=0x7f700800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7008017070 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7008005cf0 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70181e1000 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70080072f0 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70181e15c0 con 0x7f7018144ca0 2024-04-25T10:06:39.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f701804efa0 con 0x7f7018144ca0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f700801a070 con 0x7f7018144ca0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 0x7f6ffc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701de26640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 0x7f6ffc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7008084350 con 0x7f7018144ca0 2024-04-25T10:06:39.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.186+0000 7f701de26640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 0x7f6ffc069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70140b2000 tx=0x7f7014008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.190+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7008055430 con 0x7f7018144ca0 2024-04-25T10:06:39.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.406+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f701815bac0 con 0x7f7018144ca0 2024-04-25T10:06:39.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f700e7fc640 1 -- 172.21.15.82:0/477873447 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v239) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7008054d30 con 0x7f7018144ca0 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 msgr2=0x7f6ffc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 0x7f6ffc069450 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f70140b2000 tx=0x7f7014008040 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 msgr2=0x7f70180c4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f700800c430 tx=0x7f700800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 shutdown_connections 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ffc066f90 0x7f6ffc069450 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70181451f0 0x7f70180c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7018144ca0 0x7f70180c4ba0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 --2- 172.21.15.82:0/477873447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70180ce510 0x7f70181e76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 >> 172.21.15.82:0/477873447 conn(0x7f70180b11a0 msgr2=0x7f7018071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 shutdown_connections 2024-04-25T10:06:39.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.710+0000 7f702008a640 1 -- 172.21.15.82:0/477873447 wait complete. 2024-04-25T10:06:39.738 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:06:39.738 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:06:39.738 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:06:39.739 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:39.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 Processor -- start 2024-04-25T10:06:39.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 -- start start 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc0940cfb80 0x7fc0940cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940d04d0 con 0x7fc09409a480 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09415bd30 con 0x7fc09415b950 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc09b635640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0940cc0d0 con 0x7fc0940cfb80 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc098bd0640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc09409a480 0x7fc0940cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37436/0 (socket says 172.21.15.82:37436) 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc098bd0640 1 -- 172.21.15.82:0/2678404991 learned_addr learned my addr 172.21.15.82:0/2678404991 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc099bd2640 1 --2- 172.21.15.82:0/2678404991 >> v2:172.21.15.196:3300/0 conn(0x7fc0940cfb80 0x7fc0940cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3446471690 0 0) 0x7fc084001120 con 0x7fc09409a480 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc080003530 con 0x7fc09409a480 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508127884 0 0) 0x7fc080003530 con 0x7fc09409a480 2024-04-25T10:06:39.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 >> v2:172.21.15.196:3300/0 conn(0x7fc0940cfb80 msgr2=0x7fc0940cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 --2- 172.21.15.82:0/2678404991 >> v2:172.21.15.196:3300/0 conn(0x7fc0940cfb80 0x7fc0940cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 >> v1:172.21.15.196:6789/0 conn(0x7fc09415b950 legacy=0x7fc0940cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09409ab10 con 0x7fc09409a480 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc084001850 con 0x7fc09409a480 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 >> v1:172.21.15.82:6789/0 conn(0x7fc09409a480 legacy=0x7fc0940cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 0x7fc080003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc080004810 0x7fc080004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0800051d0 0x7fc08000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cc0d0 con 0x7fc080003a30 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08000c100 con 0x7fc0800051d0 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08000c300 con 0x7fc080004810 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 0x7fc080003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc099bd2640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0800051d0 0x7fc08000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc098bd0640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc080004810 0x7fc080004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 -- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc080004810 msgr2=0x7fc080004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc080004810 0x7fc080004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 -- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0800051d0 msgr2=0x7fc08000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0800051d0 0x7fc08000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc0993d1640 1 -- 172.21.15.82:0/2678404991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080003530 con 0x7fc080003a30 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc099bd2640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0800051d0 0x7fc08000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:39.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.862+0000 7fc099bd2640 1 -- 172.21.15.82:0/2678404991 reap_dead start 2024-04-25T10:06:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc0993d1640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 0x7fc080003e50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc090001e70 tx=0x7fc09001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7665b7114616a140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0900b0070 con 0x7fc080003a30 2024-04-25T10:06:39.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0900a7070 con 0x7fc080003a30 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc08bfff640 1 -- 172.21.15.82:0/2678404991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090001a90 con 0x7fc080003a30 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 msgr2=0x7fc080003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 0x7fc080003e50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fc090001e70 tx=0x7fc09001c760 comp rx=0 tx=0).stop 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- 172.21.15.82:0/2678404991 shutdown_connections 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 --2- 172.21.15.82:0/2678404991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc080003a30 0x7fc080003e50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- 172.21.15.82:0/2678404991 >> 172.21.15.82:0/2678404991 conn(0x7fc0940b30b0 msgr2=0x7fc0940b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- 172.21.15.82:0/2678404991 shutdown_connections 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- 172.21.15.82:0/2678404991 wait complete. 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 Processor -- start 2024-04-25T10:06:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- start start 2024-04-25T10:06:39.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc080003a30 0x7fc0940c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0941523e0 0x7fc09415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cf9b0 con 0x7fc094152860 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940cf830 con 0x7fc080003a30 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.866+0000 7fc09b635640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09406f290 con 0x7fc0941523e0 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39300/0 (socket says 172.21.15.82:39300) 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 -- 172.21.15.82:0/3467897386 learned_addr learned my addr 172.21.15.82:0/3467897386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc0993d1640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc080003a30 0x7fc0940c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc098bd0640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0941523e0 0x7fc09415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 -- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0941523e0 msgr2=0x7fc09415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0941523e0 0x7fc09415c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 -- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc080003a30 msgr2=0x7fc0940c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc080003a30 0x7fc0940c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 -- 172.21.15.82:0/3467897386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940c6e20 con 0x7fc094152860 2024-04-25T10:06:39.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc099bd2640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc094072430 tx=0x7fc08c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00f040 con 0x7fc094152860 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c00e840 con 0x7fc094152860 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0940c7100 con 0x7fc094152860 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00ebf0 con 0x7fc094152860 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0940c7cf0 con 0x7fc094152860 2024-04-25T10:06:39.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09404efa0 con 0x7fc094152860 2024-04-25T10:06:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc08c01c400 con 0x7fc094152860 2024-04-25T10:06:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc08c085120 con 0x7fc094152860 2024-04-25T10:06:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc0993d1640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:39.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.870+0000 7fc0993d1640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 0x7fc078069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0900ad000 tx=0x7fc090001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:39.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:39.874+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc08c018030 con 0x7fc094152860 2024-04-25T10:06:40.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.030+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0940b43e0 con 0x7fc094152860 2024-04-25T10:06:40.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc089ffb640 1 -- 172.21.15.82:0/3467897386 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+13304 (secure 0 0 0) 0x7fc08c081410 con 0x7fc094152860 2024-04-25T10:06:40.037 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:40.037 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":70,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:38.707117+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/191077906":"2024-04-26T10:06:35.685285+0000","172.21.15.82:6835/191077906":"2024-04-26T10:06:35.685285+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:40.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 msgr2=0x7fc078069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 0x7fc078069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0900ad000 tx=0x7fc090001520 comp rx=0 tx=0).stop 2024-04-25T10:06:40.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 msgr2=0x7fc094070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fc094072430 tx=0x7fc08c010c20 comp rx=0 tx=0).stop 2024-04-25T10:06:40.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 shutdown_connections 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc078066f60 0x7fc078069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0941523e0 0x7fc09415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc094152860 0x7fc094070610 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 --2- 172.21.15.82:0/3467897386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc080003a30 0x7fc0940c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 >> 172.21.15.82:0/3467897386 conn(0x7fc0940b30b0 msgr2=0x7fc0940b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 shutdown_connections 2024-04-25T10:06:40.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.034+0000 7fc09b635640 1 -- 172.21.15.82:0/3467897386 wait complete. 2024-04-25T10:06:40.055 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:40.055 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 Processor -- start 2024-04-25T10:06:40.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 -- start start 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b400cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0b400cfad0 0x7f0b4015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0b4015e4f0 0x7f0b40161370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400cff10 con 0x7f0b400cfad0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b400cbf10 con 0x7f0b4015e4f0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b45744640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400cc1f0 con 0x7f0b400b3cf0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0b400cfad0 0x7f0b4015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0b400cfad0 0x7f0b4015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39316/0 (socket says 172.21.15.82:39316) 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/2581768810 learned_addr learned my addr 172.21.15.82:0/2581768810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3f7fe640 1 --2- 172.21.15.82:0/2581768810 >> v2:172.21.15.196:3300/0 conn(0x7f0b4015e4f0 0x7f0b40161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3effd640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b400cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/2581768810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 msgr2=0x7f0b400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b400cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/2581768810 >> v2:172.21.15.196:3300/0 conn(0x7f0b4015e4f0 msgr2=0x7f0b40161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/2581768810 >> v2:172.21.15.196:3300/0 conn(0x7f0b4015e4f0 0x7f0b40161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.170+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/2581768810 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400cc4d0 con 0x7f0b400cfad0 2024-04-25T10:06:40.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b4015dfb0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0b400aedb0 tx=0x7f0b3401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43903bbcd64405e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3dffb640 1 -- 172.21.15.82:0/2581768810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b0070 con 0x7f0b400cfad0 2024-04-25T10:06:40.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3dffb640 1 -- 172.21.15.82:0/2581768810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340ab600 con 0x7f0b400cfad0 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3dffb640 1 -- 172.21.15.82:0/2581768810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340ab9d0 con 0x7f0b400cfad0 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/2581768810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 msgr2=0x7f0b4015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b4015dfb0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f0b400aedb0 tx=0x7f0b3401c260 comp rx=0 tx=0).stop 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/2581768810 shutdown_connections 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- 172.21.15.82:0/2581768810 >> v2:172.21.15.196:3300/0 conn(0x7f0b4015e4f0 0x7f0b40161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b4015dfb0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- 172.21.15.82:0/2581768810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b400cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/2581768810 >> 172.21.15.82:0/2581768810 conn(0x7f0b4006ae60 msgr2=0x7f0b4006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/2581768810 shutdown_connections 2024-04-25T10:06:40.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/2581768810 wait complete. 2024-04-25T10:06:40.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 Processor -- start 2024-04-25T10:06:40.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- start start 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b4006e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4015e4f0 0x7f0b400c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4015ea80 con 0x7f0b400cfad0 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4015e900 con 0x7f0b4015e4f0 2024-04-25T10:06:40.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400caaf0 con 0x7f0b400b3cf0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39326/0 (socket says 172.21.15.82:39326) 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/432672818 learned_addr learned my addr 172.21.15.82:0/432672818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3effd640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b4006e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 msgr2=0x7f0b4006e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b4006e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3f7fe640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4015e4f0 0x7f0b400c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4015e4f0 msgr2=0x7f0b400c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4015e4f0 0x7f0b400c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 -- 172.21.15.82:0/432672818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400c59a0 con 0x7f0b400cfad0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b3e7fc640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0b3401a910 tx=0x7f0b34019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b9070 con 0x7f0b400cfad0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b340b4400 con 0x7f0b400cfad0 2024-04-25T10:06:40.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b40158dd0 con 0x7f0b400cfad0 2024-04-25T10:06:40.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b340b0040 con 0x7f0b400cfad0 2024-04-25T10:06:40.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.174+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b40159390 con 0x7f0b400cfad0 2024-04-25T10:06:40.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b40159a50 con 0x7f0b400cfad0 2024-04-25T10:06:40.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b340bc070 con 0x7f0b400cfad0 2024-04-25T10:06:40.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b277fe640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 0x7f0b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0b3411dfe0 con 0x7f0b400cfad0 2024-04-25T10:06:40.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b3effd640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 0x7f0b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.178+0000 7f0b3effd640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 0x7f0b2c069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f30 tx=0x7f0b3000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.182+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b40159a50 con 0x7f0b400cfad0 2024-04-25T10:06:40.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.326+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b4004efa0 con 0x7f0b400cfad0 2024-04-25T10:06:40.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.326+0000 7f0b277fe640 1 -- 172.21.15.82:0/432672818 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b400c59a0 con 0x7f0b400cfad0 2024-04-25T10:06:40.330 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:06:40.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 msgr2=0x7f0b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 0x7f0b2c069420 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009f30 tx=0x7f0b3000b040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 msgr2=0x7f0b400c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f0b3401a910 tx=0x7f0b34019000 comp rx=0 tx=0).stop 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 shutdown_connections 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b2c066f60 0x7f0b2c069420 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0b4015e4f0 0x7f0b400c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0b400cfad0 0x7f0b400c4f20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 --2- 172.21.15.82:0/432672818 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0b400b3cf0 0x7f0b4006e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 >> 172.21.15.82:0/432672818 conn(0x7f0b4006ae60 msgr2=0x7f0b400ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 shutdown_connections 2024-04-25T10:06:40.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.330+0000 7f0b45744640 1 -- 172.21.15.82:0/432672818 wait complete. 2024-04-25T10:06:40.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:06:40.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 Processor -- start 2024-04-25T10:06:40.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 -- start start 2024-04-25T10:06:40.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 0x7fd9940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd994144ca0 0x7fd99406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9940cee30 con 0x7fd99406ec60 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9940cc2b0 con 0x7fd994144ca0 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd999eb9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99406e290 con 0x7fd9940ce510 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd994144ca0 0x7fd99406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd994144ca0 0x7fd99406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39332/0 (socket says 172.21.15.82:39332) 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 -- 172.21.15.82:0/1711925157 learned_addr learned my addr 172.21.15.82:0/1711925157 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd993fff640 1 --1- 172.21.15.82:0/1711925157 >> v1:172.21.15.82:6789/0 conn(0x7fd99406ec60 0x7fd994142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37440/0 (socket says 172.21.15.82:37440) 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd9937fe640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 0x7fd9940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 -- 172.21.15.82:0/1711925157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 msgr2=0x7fd9940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 0x7fd9940ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 -- 172.21.15.82:0/1711925157 >> v1:172.21.15.82:6789/0 conn(0x7fd99406ec60 legacy=0x7fd994142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 -- 172.21.15.82:0/1711925157 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940ccc70 con 0x7fd994144ca0 2024-04-25T10:06:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd9937fe640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 0x7fd9940ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:40.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.454+0000 7fd992ffd640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994144ca0 0x7fd99406e720 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd988009870 tx=0x7fd98801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88653dc73c0cd4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9927fc640 1 -- 172.21.15.82:0/1711925157 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880b5070 con 0x7fd994144ca0 2024-04-25T10:06:40.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9927fc640 1 -- 172.21.15.82:0/1711925157 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9880a8a50 con 0x7fd994144ca0 2024-04-25T10:06:40.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9927fc640 1 -- 172.21.15.82:0/1711925157 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9880a8e20 con 0x7fd994144ca0 2024-04-25T10:06:40.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/1711925157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994144ca0 msgr2=0x7fd99406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994144ca0 0x7fd99406e720 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fd988009870 tx=0x7fd98801ed50 comp rx=0 tx=0).stop 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/1711925157 shutdown_connections 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd994144ca0 0x7fd99406e720 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- 172.21.15.82:0/1711925157 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9940ce510 0x7fd9940ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/1711925157 >> 172.21.15.82:0/1711925157 conn(0x7fd9940b11a0 msgr2=0x7fd9940b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/1711925157 shutdown_connections 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/1711925157 wait complete. 2024-04-25T10:06:40.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 Processor -- start 2024-04-25T10:06:40.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- start start 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 0x7fd9940c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd994144ca0 0x7fd9940c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9940cc900 con 0x7fd99406ec60 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99406df20 con 0x7fd9940ce510 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd994142d90 con 0x7fd994144ca0 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39342/0 (socket says 172.21.15.82:39342) 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 -- 172.21.15.82:0/585650193 learned_addr learned my addr 172.21.15.82:0/585650193 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd992ffd640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 0x7fd9940c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 -- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd994144ca0 msgr2=0x7fd9940c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd994144ca0 0x7fd9940c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 -- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 msgr2=0x7fd9940c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 0x7fd9940c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 -- 172.21.15.82:0/585650193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9940c8d50 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd992ffd640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 0x7fd9940c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd9937fe640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c7f0 tx=0x7fd98400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd984007a30 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd984007bd0 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9941e1220 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9840113f0 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.458+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9940c95b0 con 0x7fd99406ec60 2024-04-25T10:06:40.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99404efa0 con 0x7fd99406ec60 2024-04-25T10:06:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd984011590 con 0x7fd99406ec60 2024-04-25T10:06:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd97bfff640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 0x7fd980069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd984085160 con 0x7fd99406ec60 2024-04-25T10:06:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd992ffd640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 0x7fd980069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.462+0000 7fd992ffd640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 0x7fd980069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9880b2000 tx=0x7fd9880aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.466+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd984015020 con 0x7fd99406ec60 2024-04-25T10:06:40.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.666+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fd9940b1cc0 con 0x7fd99406ec60 2024-04-25T10:06:40.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.710+0000 7fd97bfff640 1 -- 172.21.15.82:0/585650193 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v240) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd984056240 con 0x7fd99406ec60 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 msgr2=0x7fd980069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 0x7fd980069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9880b2000 tx=0x7fd9880aa070 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 msgr2=0x7fd9941e0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fd98400c7f0 tx=0x7fd98400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 shutdown_connections 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd980066f60 0x7fd980069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd994144ca0 0x7fd9940c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9940ce510 0x7fd9940c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 --2- 172.21.15.82:0/585650193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd99406ec60 0x7fd9941e0ce0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 >> 172.21.15.82:0/585650193 conn(0x7fd9940b11a0 msgr2=0x7fd9940b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 shutdown_connections 2024-04-25T10:06:40.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.714+0000 7fd999eb9640 1 -- 172.21.15.82:0/585650193 wait complete. 2024-04-25T10:06:40.733 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:40.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 Processor -- start 2024-04-25T10:06:40.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 -- start start 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 0x7ff8e40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e40cbd20 con 0x7ff8e415f130 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e415ba90 con 0x7ff8e40cf590 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e8cac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40cff30 con 0x7ff8e415b6b0 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e259c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e1d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 0x7ff8e40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e2d9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ff8e415f130 0x7ff8e415f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:59476/0 (socket says 172.21.15.82:59476) 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e2d9d640 1 -- 172.21.15.82:0/3186330858 learned_addr learned my addr 172.21.15.82:0/3186330858 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e1d9b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 0x7ff8e40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:58830/0 (socket says 172.21.15.82:58830) 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e259c640 1 -- 172.21.15.82:0/3186330858 >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 msgr2=0x7ff8e40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e259c640 1 --2- 172.21.15.82:0/3186330858 >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 0x7ff8e40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e259c640 1 -- 172.21.15.82:0/3186330858 >> v1:172.21.15.196:6789/0 conn(0x7ff8e415f130 legacy=0x7ff8e415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.850+0000 7ff8e259c640 1 -- 172.21.15.82:0/3186330858 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40cc000 con 0x7ff8e415b6b0 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e259c640 1 --2- 172.21.15.82:0/3186330858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40ca3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d90 tx=0x7ff8dc01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a94c8ec51d47d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e159a640 1 -- 172.21.15.82:0/3186330858 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0b2070 con 0x7ff8e415b6b0 2024-04-25T10:06:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e159a640 1 -- 172.21.15.82:0/3186330858 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8dc0a5a00 con 0x7ff8e415b6b0 2024-04-25T10:06:40.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e159a640 1 -- 172.21.15.82:0/3186330858 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0a5dd0 con 0x7ff8e415b6b0 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- 172.21.15.82:0/3186330858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 msgr2=0x7ff8e40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/3186330858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40ca3c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d90 tx=0x7ff8dc01c940 comp rx=0 tx=0).stop 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- 172.21.15.82:0/3186330858 shutdown_connections 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/3186330858 >> v2:172.21.15.196:3300/0 conn(0x7ff8e40cf590 0x7ff8e40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/3186330858 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40ca3c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- 172.21.15.82:0/3186330858 >> 172.21.15.82:0/3186330858 conn(0x7ff8e406ae60 msgr2=0x7ff8e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- 172.21.15.82:0/3186330858 shutdown_connections 2024-04-25T10:06:40.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- 172.21.15.82:0/3186330858 wait complete. 2024-04-25T10:06:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 Processor -- start 2024-04-25T10:06:40.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.854+0000 7ff8e8cac640 1 -- start start 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8e40cf590 0x7ff8e41648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40ca900 con 0x7ff8e415f130 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40cba30 con 0x7ff8e40cf590 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e40cadb0 con 0x7ff8e415b6b0 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39346/0 (socket says 172.21.15.82:39346) 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 -- 172.21.15.82:0/574254574 learned_addr learned my addr 172.21.15.82:0/574254574 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:40.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e1d9b640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e259c640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8e40cf590 0x7ff8e41648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 -- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 msgr2=0x7ff8e40c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 -- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8e40cf590 msgr2=0x7ff8e41648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8e40cf590 0x7ff8e41648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 -- 172.21.15.82:0/574254574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40adb40 con 0x7ff8e415f130 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e2d9d640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400c3c0 tx=0x7ff8d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4017070 con 0x7ff8e415f130 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d4005cf0 con 0x7ff8e415f130 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e40adde0 con 0x7ff8e415f130 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40072f0 con 0x7ff8e415f130 2024-04-25T10:06:40.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e40ae680 con 0x7ff8e415f130 2024-04-25T10:06:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e40ae1d0 con 0x7ff8e415f130 2024-04-25T10:06:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8d401a070 con 0x7ff8e415f130 2024-04-25T10:06:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 0x7ff8c4069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e259c640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 0x7ff8c4069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff8d40840f0 con 0x7ff8e415f130 2024-04-25T10:06:40.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.858+0000 7ff8e259c640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 0x7ff8c4069470 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0af000 tx=0x7ff8dc0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:40.862+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8d40553e0 con 0x7ff8e415f130 2024-04-25T10:06:41.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.010+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8e404efa0 con 0x7ff8e415f130 2024-04-25T10:06:41.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.018+0000 7ff8d2ffd640 1 -- 172.21.15.82:0/574254574 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8d4054ce0 con 0x7ff8e415f130 2024-04-25T10:06:41.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 msgr2=0x7ff8c4069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 0x7ff8c4069470 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0af000 tx=0x7ff8dc0a7070 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 msgr2=0x7ff8e40c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400c3c0 tx=0x7ff8d400c880 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 shutdown_connections 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff8c4066fb0 0x7ff8c4069470 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff8e415b6b0 0x7ff8e40c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff8e415f130 0x7ff8e40c1ee0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 --2- 172.21.15.82:0/574254574 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff8e40cf590 0x7ff8e41648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 >> 172.21.15.82:0/574254574 conn(0x7ff8e406ae60 msgr2=0x7ff8e40b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 shutdown_connections 2024-04-25T10:06:41.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.022+0000 7ff8e8cac640 1 -- 172.21.15.82:0/574254574 wait complete. 2024-04-25T10:06:41.040 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:41.040 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:41.101 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:41.150 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:41.150 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:06:41.150 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:41.156 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:41.156 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:06:41.156 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:41.162 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:41.162 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:06:41.162 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:41.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:41.204 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:06:41.204 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:41.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:41.210 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:41.210 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:41.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 Processor -- start 2024-04-25T10:06:41.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 -- start start 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f76e4144ca0 0x7f76e406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e40ccc70 con 0x7f76e406ec60 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76e406dc40 con 0x7f76e40ce510 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e8f12640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e40cc2b0 con 0x7f76e4144ca0 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e2d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f76e406ec60 0x7f76e4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37448/0 (socket says 172.21.15.82:37448) 2024-04-25T10:06:41.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e2d9d640 1 -- 172.21.15.82:0/1387973944 learned_addr learned my addr 172.21.15.82:0/1387973944 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563595147 0 0) 0x7f76e40ccc70 con 0x7f76e406ec60 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d0003530 con 0x7f76e406ec60 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e1d9b640 1 --2- 172.21.15.82:0/1387973944 >> v2:172.21.15.196:3300/0 conn(0x7f76e4144ca0 0x7f76e406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3440494249 0 0) 0x7f76d0003530 con 0x7f76e406ec60 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 >> v2:172.21.15.196:3300/0 conn(0x7f76e4144ca0 msgr2=0x7f76e406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 --2- 172.21.15.82:0/1387973944 >> v2:172.21.15.196:3300/0 conn(0x7f76e4144ca0 0x7f76e406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 >> v1:172.21.15.196:6789/0 conn(0x7f76e40ce510 legacy=0x7f76e40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e406df20 con 0x7f76e406ec60 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f76dc001680 con 0x7f76e406ec60 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 >> v1:172.21.15.82:6789/0 conn(0x7f76e406ec60 legacy=0x7f76e4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 0x7f76d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d0004810 0x7f76d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d00051d0 0x7f76d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e40cc2b0 con 0x7f76d0003a30 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d000c0e0 con 0x7f76d00051d0 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d000c2e0 con 0x7f76d0004810 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 0x7f76d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e2d9d640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d00051d0 0x7f76d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e1d9b640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d0004810 0x7f76d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 -- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d0004810 msgr2=0x7f76d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76d0004810 0x7f76d0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 -- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d00051d0 msgr2=0x7f76d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d00051d0 0x7f76d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e2d9d640 1 -- 172.21.15.82:0/1387973944 reap_dead start 2024-04-25T10:06:41.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.366+0000 7f76e259c640 1 -- 172.21.15.82:0/1387973944 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d0003530 con 0x7f76d0003a30 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e259c640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 0x7f76d0003e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f76d4001e70 tx=0x7f76d401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb754fed37dae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4001a90 con 0x7f76d0003a30 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d4003b80 con 0x7f76d0003a30 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e159a640 1 -- 172.21.15.82:0/1387973944 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d4005080 con 0x7f76d0003a30 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 msgr2=0x7f76d0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 0x7f76d0003e50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f76d4001e70 tx=0x7f76d401c760 comp rx=0 tx=0).stop 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- 172.21.15.82:0/1387973944 shutdown_connections 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 --2- 172.21.15.82:0/1387973944 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76d0003a30 0x7f76d0003e50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- 172.21.15.82:0/1387973944 >> 172.21.15.82:0/1387973944 conn(0x7f76e40b11a0 msgr2=0x7f76e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- 172.21.15.82:0/1387973944 shutdown_connections 2024-04-25T10:06:41.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- 172.21.15.82:0/1387973944 wait complete. 2024-04-25T10:06:41.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 Processor -- start 2024-04-25T10:06:41.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.370+0000 7f76e8f12640 1 -- start start 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d0003a30 0x7f76e40ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e4131470 0x7f76e4131920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e41451f0 con 0x7f76e4134160 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e406e290 con 0x7f76d0003a30 2024-04-25T10:06:41.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e4145370 con 0x7f76e4131470 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39368/0 (socket says 172.21.15.82:39368) 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 -- 172.21.15.82:0/3480639505 learned_addr learned my addr 172.21.15.82:0/3480639505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e2d9d640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e4131470 0x7f76e4131920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e259c640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d0003a30 0x7f76e40ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 -- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e4131470 msgr2=0x7f76e4131920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e4131470 0x7f76e4131920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 -- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d0003a30 msgr2=0x7f76e40ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d0003a30 0x7f76e40ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 -- 172.21.15.82:0/3480639505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e40ae970 con 0x7f76e4134160 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e1d9b640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f76cc010900 tx=0x7f76cc010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00f040 con 0x7f76e4134160 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00ec20 con 0x7f76e4134160 2024-04-25T10:06:41.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e4131e60 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc01c400 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e40aa4e0 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e404efa0 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76cc01c5a0 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 0x7f76c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f76cc085520 con 0x7f76e4134160 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e259c640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 0x7f76c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.374+0000 7f76e259c640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 0x7f76c4069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005000 tx=0x7f76d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.378+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76cc018030 con 0x7f76e4134160 2024-04-25T10:06:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.570+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76e40b1cc0 con 0x7f76e4134160 2024-04-25T10:06:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.570+0000 7f76daffd640 1 -- 172.21.15.82:0/3480639505 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f76cc055fd0 con 0x7f76e4134160 2024-04-25T10:06:41.575 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:41.575 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5054,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/500291942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":500291942},{"type":"v1","addr":"172.21.15.196:6833","nonce":500291942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2247935255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":2247935255},{"type":"v1","addr":"172.21.15.196:6835","nonce":2247935255}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/890244071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":890244071},{"type":"v1","addr":"172.21.15.82:6836","nonce":890244071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27},{"gid":6903,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3402639353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3402639353},{"type":"v1","addr":"172.21.15.82:6837","nonce":3402639353}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[]} 2024-04-25T10:06:41.575 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 36 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 msgr2=0x7f76c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 0x7f76c4069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f76d4005000 tx=0x7f76d4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 msgr2=0x7f76e41345c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f76cc010900 tx=0x7f76cc010dc0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 shutdown_connections 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f76c4066f60 0x7f76c4069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f76e4131470 0x7f76e4131920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f76e4134160 0x7f76e41345c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 --2- 172.21.15.82:0/3480639505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f76d0003a30 0x7f76e40ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 >> 172.21.15.82:0/3480639505 conn(0x7f76e40b11a0 msgr2=0x7f76e40a8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 shutdown_connections 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.574+0000 7f76e8f12640 1 -- 172.21.15.82:0/3480639505 wait complete. 2024-04-25T10:06:41.594 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:41.594 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:41.594 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:41.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:41.595 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:41.595 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:41.595 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:41.595 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:41.595 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:41.595 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:41.595 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:41.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:41.596 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:41.596 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:41.596 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:41.596 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:41.605 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:41.605 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:41.606 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:41.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:41.609 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:41.609 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:41.616 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:41.616 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:41.649 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:41.649 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:41.650 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:41.650 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:41.661 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:41.661 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:41.662 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:41.662 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:41.663 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:41.709 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:06:41.720 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:06:41.733 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:06:41.734 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:06:41.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.774+0000 7f66205ec640 1 Processor -- start 2024-04-25T10:06:41.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.774+0000 7f66205ec640 1 -- start start 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f66205ec640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66180d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f66205ec640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66180d1cd0 0x7f661806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f66205ec640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f661806be60 con 0x7f661815da80 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f66205ec640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f661806c440 con 0x7f66180d1cd0 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f66205ec640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661815de60 con 0x7f66180d07f0 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f661815da80 0x7f66180d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37470/0 (socket says 172.21.15.82:37470) 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 -- 172.21.15.82:0/817257548 learned_addr learned my addr 172.21.15.82:0/817257548 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661eb89640 1 --2- 172.21.15.82:0/817257548 >> v2:172.21.15.196:3300/0 conn(0x7f66180d1cd0 0x7f661806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661db87640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66180d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814725838 0 0) 0x7f661806be60 con 0x7f661815da80 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6604003530 con 0x7f661815da80 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873588738 0 0) 0x7f6604003530 con 0x7f661815da80 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 msgr2=0x7f66180d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66180d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 >> v2:172.21.15.196:3300/0 conn(0x7f66180d1cd0 msgr2=0x7f661806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 --2- 172.21.15.82:0/817257548 >> v2:172.21.15.196:3300/0 conn(0x7f66180d1cd0 0x7f661806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180d2450 con 0x7f661815da80 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6608001680 con 0x7f661815da80 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661eb89640 1 --2- 172.21.15.82:0/817257548 >> v2:172.21.15.196:3300/0 conn(0x7f66180d1cd0 0x7f661806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 >> v1:172.21.15.82:6789/0 conn(0x7f661815da80 legacy=0x7f66180d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6604003a30 0x7f6604003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6604004810 0x7f6604004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806c440 con 0x7f6604003a30 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661815de60 con 0x7f6604004810 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f660400b9d0 con 0x7f66180d07f0 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6604003a30 0x7f6604003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661db87640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6604004810 0x7f6604004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661eb89640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 -- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 msgr2=0x7f66040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 -- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6604004810 msgr2=0x7f6604004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6604004810 0x7f6604004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 -- 172.21.15.82:0/817257548 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604003530 con 0x7f6604003a30 2024-04-25T10:06:41.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661eb89640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66180d07f0 0x7f66040051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661e388640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6604003a30 0x7f6604003e50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f660800c2f0 tx=0x7f660801cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f031ed24facc4436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608007d60 con 0x7f6604003a30 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66080a9020 con 0x7f6604003a30 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.778+0000 7f661d386640 1 -- 172.21.15.82:0/817257548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608005b00 con 0x7f6604003a30 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/817257548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6604003a30 msgr2=0x7f6604003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 --2- 172.21.15.82:0/817257548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6604003a30 0x7f6604003e50 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f660800c2f0 tx=0x7f660801cd10 comp rx=0 tx=0).stop 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661eb89640 1 -- 172.21.15.82:0/817257548 reap_dead start 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/817257548 shutdown_connections 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/817257548 >> 172.21.15.82:0/817257548 conn(0x7f661808b000 msgr2=0x7f66180d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/817257548 shutdown_connections 2024-04-25T10:06:41.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/817257548 wait complete. 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 Processor -- start 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- start start 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 0x7f661809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6618159c90 0x7f661809e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806d7a0 con 0x7f661814eb30 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661806d620 con 0x7f6618159c90 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661806d920 con 0x7f661809ff90 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39418/0 (socket says 172.21.15.82:39418) 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 -- 172.21.15.82:0/632194114 learned_addr learned my addr 172.21.15.82:0/632194114 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661db87640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 0x7f661809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661eb89640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6618159c90 0x7f661809e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 -- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 msgr2=0x7f661809dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 0x7f661809dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 -- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6618159c90 msgr2=0x7f661809e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6618159c90 0x7f661809e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 -- 172.21.15.82:0/632194114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180a0430 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661db87640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 0x7f661809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f661e388640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f66080ab040 tx=0x7f6604005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608004ab0 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66080b2030 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66180a0710 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080070d0 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.782+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66180a0cd0 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661804efa0 con 0x7f661814eb30 2024-04-25T10:06:41.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6608007270 con 0x7f661814eb30 2024-04-25T10:06:41.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f660effd640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 0x7f65fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f660811e2c0 con 0x7f661814eb30 2024-04-25T10:06:41.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f661db87640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 0x7f65fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.786+0000 7f661db87640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 0x7f65fc069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6614009b40 tx=0x7f6614008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.790+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6608016a50 con 0x7f661814eb30 2024-04-25T10:06:41.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:06:41.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:41.954+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6618160b50 con 0x7f661814eb30 2024-04-25T10:06:43.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.046+0000 7f660effd640 1 -- 172.21.15.82:0/632194114 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f66080ef4d0 con 0x7f661814eb30 2024-04-25T10:06:43.051 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:06:43.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 msgr2=0x7f65fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 0x7f65fc069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f6614009b40 tx=0x7f6614008040 comp rx=0 tx=0).stop 2024-04-25T10:06:43.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 msgr2=0x7f661809fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f66080ab040 tx=0x7f6604005840 comp rx=0 tx=0).stop 2024-04-25T10:06:43.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 shutdown_connections 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65fc066f60 0x7f65fc069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f661809ff90 0x7f661809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6618159c90 0x7f661809e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 --2- 172.21.15.82:0/632194114 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f661814eb30 0x7f661809fa50 secure :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f66080ab040 tx=0x7f6604005840 comp rx=0 tx=0).stop 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 >> 172.21.15.82:0/632194114 conn(0x7f661808b000 msgr2=0x7f661815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 shutdown_connections 2024-04-25T10:06:43.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.050+0000 7f66205ec640 1 -- 172.21.15.82:0/632194114 wait complete. 2024-04-25T10:06:43.076 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:43.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 Processor -- start 2024-04-25T10:06:43.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 -- start start 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6400cc130 0x7ff640166650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64015f360 con 0x7ff64015ca90 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64015f640 con 0x7ff64006ac90 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64808d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64015ff70 con 0x7ff6400cc130 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64662a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff6400cc130 0x7ff640166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff645628640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff64015ca90 0x7ff6400d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37482/0 (socket says 172.21.15.82:37482) 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff645628640 1 -- 172.21.15.82:0/998717774 learned_addr learned my addr 172.21.15.82:0/998717774 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64662a640 1 -- 172.21.15.82:0/998717774 >> v1:172.21.15.196:6789/0 conn(0x7ff64006ac90 legacy=0x7ff64006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64662a640 1 -- 172.21.15.82:0/998717774 >> v1:172.21.15.82:6789/0 conn(0x7ff64015ca90 legacy=0x7ff6400d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff64662a640 1 -- 172.21.15.82:0/998717774 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64015f920 con 0x7ff6400cc130 2024-04-25T10:06:43.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.198+0000 7ff644e27640 1 -- 172.21.15.82:0/998717774 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539721044 0 0) 0x7ff64015f360 con 0x7ff64015ca90 2024-04-25T10:06:43.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64662a640 1 --2- 172.21.15.82:0/998717774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff640166650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0097c0 tx=0x7ff63c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5867bc3fa3528bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff644e27640 1 -- 172.21.15.82:0/998717774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0b0070 con 0x7ff6400cc130 2024-04-25T10:06:43.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff644e27640 1 -- 172.21.15.82:0/998717774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c0ab600 con 0x7ff6400cc130 2024-04-25T10:06:43.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff644e27640 1 -- 172.21.15.82:0/998717774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0ab9d0 con 0x7ff6400cc130 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- 172.21.15.82:0/998717774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 msgr2=0x7ff640166650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 --2- 172.21.15.82:0/998717774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff640166650 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7ff63c0097c0 tx=0x7ff63c01c460 comp rx=0 tx=0).stop 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- 172.21.15.82:0/998717774 shutdown_connections 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 --2- 172.21.15.82:0/998717774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff640166650 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- 172.21.15.82:0/998717774 >> 172.21.15.82:0/998717774 conn(0x7ff640099a70 msgr2=0x7ff6400d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- 172.21.15.82:0/998717774 shutdown_connections 2024-04-25T10:06:43.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- 172.21.15.82:0/998717774 wait complete. 2024-04-25T10:06:43.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 Processor -- start 2024-04-25T10:06:43.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- start start 2024-04-25T10:06:43.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 0x7ff640150900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 0x7ff6400b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff640169940 con 0x7ff6400cc130 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401697c0 con 0x7ff64015ca90 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64808d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff640169ac0 con 0x7ff64006ac90 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39438/0 (socket says 172.21.15.82:39438) 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 -- 172.21.15.82:0/152678675 learned_addr learned my addr 172.21.15.82:0/152678675 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64662a640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 0x7ff6400b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645e29640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 0x7ff640150900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 -- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 msgr2=0x7ff640150900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 0x7ff640150900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 -- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 msgr2=0x7ff6400b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 0x7ff6400b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 -- 172.21.15.82:0/152678675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64006f6b0 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff64662a640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 0x7ff6400b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645e29640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 0x7ff640150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.202+0000 7ff645628640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c4b0 tx=0x7ff63400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634018070 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff634007500 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64006f990 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff634004a30 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64006ff50 con 0x7ff6400cc130 2024-04-25T10:06:43.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64004efa0 con 0x7ff6400cc130 2024-04-25T10:06:43.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff63401b070 con 0x7ff6400cc130 2024-04-25T10:06:43.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 0x7ff624069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff6340842d0 con 0x7ff6400cc130 2024-04-25T10:06:43.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff645e29640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 0x7ff624069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.206+0000 7ff645e29640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 0x7ff624069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff638007fd0 tx=0x7ff638009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.210+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff634055610 con 0x7ff6400cc130 2024-04-25T10:06:43.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.430+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff6400b6480 con 0x7ff6400cc130 2024-04-25T10:06:43.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff6327fc640 1 -- 172.21.15.82:0/152678675 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff634054f10 con 0x7ff6400cc130 2024-04-25T10:06:43.436 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:43.437 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:06:43.437 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:06:43.437 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.437 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:06:43.437 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.438 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.439 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.440 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:06:43.441 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:06:43.442 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.443 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.444 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.445 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.446 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.447 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.448 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.449 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.450 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.451 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.452 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.453 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:06:43.454 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:06:43.455 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.456 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:06:43.457 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:06:43.458 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:43.459 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:43.460 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:06:43.461 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.461 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:43.461 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:06:43.461 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:06:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 msgr2=0x7ff624069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 0x7ff624069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7ff638007fd0 tx=0x7ff638009040 comp rx=0 tx=0).stop 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 msgr2=0x7ff6400b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7ff63400c4b0 tx=0x7ff63400c970 comp rx=0 tx=0).stop 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 shutdown_connections 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff624066f60 0x7ff624069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff64015ca90 0x7ff6400b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff6400cc130 0x7ff6400b1670 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 --2- 172.21.15.82:0/152678675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff64006ac90 0x7ff640150900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 >> 172.21.15.82:0/152678675 conn(0x7ff640099a70 msgr2=0x7ff64015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 shutdown_connections 2024-04-25T10:06:43.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.434+0000 7ff64808d640 1 -- 172.21.15.82:0/152678675 wait complete. 2024-04-25T10:06:43.467 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:43.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 Processor -- start 2024-04-25T10:06:43.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 -- start start 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0ae40d07f0 0x7f0ae40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae406be60 con 0x7f0ae415da80 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ae415de60 con 0x7f0ae40d1cd0 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0aeb6a7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ae406c440 con 0x7f0ae40d07f0 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9443640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0ae415da80 0x7f0ae40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37490/0 (socket says 172.21.15.82:37490) 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9443640 1 -- 172.21.15.82:0/2678942369 learned_addr learned my addr 172.21.15.82:0/2678942369 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae8c42640 1 --2- 172.21.15.82:0/2678942369 >> v2:172.21.15.196:3300/0 conn(0x7f0ae40d07f0 0x7f0ae40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955462748 0 0) 0x7f0ae406be60 con 0x7f0ae415da80 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad0003530 con 0x7f0ae415da80 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797967723 0 0) 0x7f0ad0003530 con 0x7f0ae415da80 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 >> v2:172.21.15.196:3300/0 conn(0x7f0ae40d07f0 msgr2=0x7f0ae40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 --2- 172.21.15.82:0/2678942369 >> v2:172.21.15.196:3300/0 conn(0x7f0ae40d07f0 0x7f0ae40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 >> v1:172.21.15.196:6789/0 conn(0x7f0ae40d1cd0 legacy=0x7f0ae406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40d2450 con 0x7f0ae415da80 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0ad4001680 con 0x7f0ae415da80 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 >> v1:172.21.15.82:6789/0 conn(0x7f0ae415da80 legacy=0x7f0ae40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ad0003a30 0x7f0ad0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ad000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae406c440 con 0x7f0ad00051d0 2024-04-25T10:06:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad000c100 con 0x7f0ad0004810 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad000c300 con 0x7f0ad0003a30 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ad000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae8c42640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 -- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ad0003a30 msgr2=0x7f0ad0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ad0003a30 0x7f0ad0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 -- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad0004810 msgr2=0x7f0ad0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ad0004810 0x7f0ad0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.586+0000 7f0ae9c44640 1 -- 172.21.15.82:0/2678942369 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0003530 con 0x7f0ad00051d0 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0ae9c44640 1 -- 172.21.15.82:0/2678942369 reap_dead start 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0ae9c44640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ad000b9d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001e70 tx=0x7f0adc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16f61b709364b37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc001a90 con 0x7f0ad00051d0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0adc003bc0 con 0x7f0ad00051d0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0adbfff640 1 -- 172.21.15.82:0/2678942369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0adc005020 con 0x7f0ad00051d0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 msgr2=0x7f0ad000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ad000b9d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0adc001e70 tx=0x7f0adc01c780 comp rx=0 tx=0).stop 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/2678942369 shutdown_connections 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/2678942369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ad000b9d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/2678942369 >> 172.21.15.82:0/2678942369 conn(0x7f0ae408b000 msgr2=0x7f0ae40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/2678942369 shutdown_connections 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/2678942369 wait complete. 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 Processor -- start 2024-04-25T10:06:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- start start 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ae415ad70 0x7f0ae40b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ae40b6780 0x7f0ae40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40d2870 con 0x7f0ad00051d0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40d26f0 con 0x7f0ae415ad70 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0aeb6a7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae40d20f0 con 0x7f0ae40b6780 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0ae9443640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0ae9443640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39464/0 (socket says 172.21.15.82:39464) 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.590+0000 7f0ae9443640 1 -- 172.21.15.82:0/1199395508 learned_addr learned my addr 172.21.15.82:0/1199395508 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae8c42640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ae415ad70 0x7f0ae40b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9c44640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ae40b6780 0x7f0ae40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 -- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ae40b6780 msgr2=0x7f0ae40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ae40b6780 0x7f0ae40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 -- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ae415ad70 msgr2=0x7f0ae40b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ae415ad70 0x7f0ae40b6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 -- 172.21.15.82:0/1199395508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40b6c90 con 0x7f0ad00051d0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae9443640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c790 tx=0x7f0ad400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4018070 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4007500 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae41f6940 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4004a30 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae41f6f00 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae40b6fd0 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ad401b070 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 0x7f0ac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0ad4084470 con 0x7f0ad00051d0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae8c42640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 0x7f0ac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.594+0000 7f0ae8c42640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 0x7f0ac8069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4073c20 tx=0x7f0ae0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.598+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ad4055510 con 0x7f0ad00051d0 2024-04-25T10:06:43.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.758+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ae40d1290 con 0x7f0ad00051d0 2024-04-25T10:06:43.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0ad9ffb640 1 -- 172.21.15.82:0/1199395508 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+13177 (secure 0 0 0) 0x7f0ad4054e10 con 0x7f0ad00051d0 2024-04-25T10:06:43.764 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:43.764 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:43.046659+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 msgr2=0x7f0ac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 0x7f0ac8069420 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4073c20 tx=0x7f0ae0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 msgr2=0x7f0ae41f7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400c790 tx=0x7f0ad400cc50 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 shutdown_connections 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ac8066f60 0x7f0ac8069420 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ae40b6780 0x7f0ae40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ae415ad70 0x7f0ae40b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 --2- 172.21.15.82:0/1199395508 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ad00051d0 0x7f0ae41f7420 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 >> 172.21.15.82:0/1199395508 conn(0x7f0ae408b000 msgr2=0x7f0ae40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 shutdown_connections 2024-04-25T10:06:43.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.762+0000 7f0aeb6a7640 1 -- 172.21.15.82:0/1199395508 wait complete. 2024-04-25T10:06:43.784 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:43.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:43.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 Processor -- start 2024-04-25T10:06:43.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 -- start start 2024-04-25T10:06:43.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c0cf590 0x7fbd2c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2c072000 0x7fbd2c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd2c0cbd20 con 0x7fbd2c15b6b0 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2c15ba90 con 0x7fbd2c072000 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd31caf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c0cff30 con 0x7fbd2c0cf590 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2c072000 0x7fbd2c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2c072000 0x7fbd2c072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39474/0 (socket says 172.21.15.82:39474) 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 -- 172.21.15.82:0/3294980849 learned_addr learned my addr 172.21.15.82:0/3294980849 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2b7fe640 1 --1- 172.21.15.82:0/3294980849 >> v1:172.21.15.82:6789/0 conn(0x7fbd2c15b6b0 0x7fbd2c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37502/0 (socket says 172.21.15.82:37502) 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2affd640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c0cf590 0x7fbd2c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 -- 172.21.15.82:0/3294980849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c0cf590 msgr2=0x7fbd2c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c0cf590 0x7fbd2c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 -- 172.21.15.82:0/3294980849 >> v1:172.21.15.82:6789/0 conn(0x7fbd2c15b6b0 legacy=0x7fbd2c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.902+0000 7fbd2bfff640 1 -- 172.21.15.82:0/3294980849 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c0cc000 con 0x7fbd2c072000 2024-04-25T10:06:43.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2bfff640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c072470 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c01c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fdd3cf1ac253f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2a7fc640 1 -- 172.21.15.82:0/3294980849 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0a5690 con 0x7fbd2c072000 2024-04-25T10:06:43.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2a7fc640 1 -- 172.21.15.82:0/3294980849 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c0a5830 con 0x7fbd2c072000 2024-04-25T10:06:43.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2a7fc640 1 -- 172.21.15.82:0/3294980849 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0a5c00 con 0x7fbd2c072000 2024-04-25T10:06:43.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- 172.21.15.82:0/3294980849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 msgr2=0x7fbd2c072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c072470 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c01c890 comp rx=0 tx=0).stop 2024-04-25T10:06:43.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- 172.21.15.82:0/3294980849 shutdown_connections 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c072470 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- 172.21.15.82:0/3294980849 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c0cf590 0x7fbd2c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- 172.21.15.82:0/3294980849 >> 172.21.15.82:0/3294980849 conn(0x7fbd2c06ae60 msgr2=0x7fbd2c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- 172.21.15.82:0/3294980849 shutdown_connections 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- 172.21.15.82:0/3294980849 wait complete. 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 Processor -- start 2024-04-25T10:06:43.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- start start 2024-04-25T10:06:43.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2c0cf590 0x7fbd2c098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c15b6b0 0x7fbd2c156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c0cadb0 con 0x7fbd2c072000 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c0cba30 con 0x7fbd2c0cf590 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd31caf640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2c0caf30 con 0x7fbd2c15b6b0 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39480/0 (socket says 172.21.15.82:39480) 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.906+0000 7fbd2b7fe640 1 -- 172.21.15.82:0/27246925 learned_addr learned my addr 172.21.15.82:0/27246925 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2affd640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2c0cf590 0x7fbd2c098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2bfff640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c15b6b0 0x7fbd2c156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 -- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c15b6b0 msgr2=0x7fbd2c156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c15b6b0 0x7fbd2c156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 -- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2c0cf590 msgr2=0x7fbd2c098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2c0cf590 0x7fbd2c098860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 -- 172.21.15.82:0/27246925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2c0cd140 con 0x7fbd2c072000 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2b7fe640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c780 tx=0x7fbd2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24018070 con 0x7fbd2c072000 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd240072d0 con 0x7fbd2c072000 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2c0cd420 con 0x7fbd2c072000 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd240047f0 con 0x7fbd2c072000 2024-04-25T10:06:43.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2c0cd9e0 con 0x7fbd2c072000 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2c165310 con 0x7fbd2c072000 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd2401b070 con 0x7fbd2c072000 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2affd640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbd240842c0 con 0x7fbd2c072000 2024-04-25T10:06:43.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.910+0000 7fbd2affd640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c15ec80 tx=0x7fbd14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:43.914+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd24055550 con 0x7fbd2c072000 2024-04-25T10:06:44.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.074+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd2c04efa0 con 0x7fbd2c072000 2024-04-25T10:06:44.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.074+0000 7fbd23fff640 1 -- 172.21.15.82:0/27246925 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd24054e50 con 0x7fbd2c072000 2024-04-25T10:06:44.079 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:06:44.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 msgr2=0x7fbd0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbd2c15ec80 tx=0x7fbd14008040 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 msgr2=0x7fbd2c164d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fbd2400c780 tx=0x7fbd2400cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 shutdown_connections 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2c15b6b0 0x7fbd2c156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2c0cf590 0x7fbd2c098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 --2- 172.21.15.82:0/27246925 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2c072000 0x7fbd2c164d70 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 >> 172.21.15.82:0/27246925 conn(0x7fbd2c06ae60 msgr2=0x7fbd2c157b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 shutdown_connections 2024-04-25T10:06:44.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.078+0000 7fbd31caf640 1 -- 172.21.15.82:0/27246925 wait complete. 2024-04-25T10:06:44.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:44.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.218+0000 7f2033b9c640 1 Processor -- start 2024-04-25T10:06:44.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.218+0000 7f2033b9c640 1 -- start start 2024-04-25T10:06:44.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f202c071e90 0x7f202c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f202c0adbd0 con 0x7f202c0b24a0 2024-04-25T10:06:44.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f202c0ae860 con 0x7f202c071e90 2024-04-25T10:06:44.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c15e210 con 0x7f202c15da10 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f202c071e90 0x7f202c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2031137640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f202c0b24a0 0x7f202c1614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37510/0 (socket says 172.21.15.82:37510) 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2031137640 1 -- 172.21.15.82:0/823809230 learned_addr learned my addr 172.21.15.82:0/823809230 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2031938640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 -- 172.21.15.82:0/823809230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 msgr2=0x7f202c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 -- 172.21.15.82:0/823809230 >> v1:172.21.15.82:6789/0 conn(0x7f202c0b24a0 legacy=0x7f202c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2030936640 1 -- 172.21.15.82:0/823809230 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804686885 0 0) 0x7f202c0adbd0 con 0x7f202c0b24a0 2024-04-25T10:06:44.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 -- 172.21.15.82:0/823809230 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c15e4f0 con 0x7f202c071e90 2024-04-25T10:06:44.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2032139640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c071e90 0x7f202c071570 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2024009d90 tx=0x7f202401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e67c2344277ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2030936640 1 -- 172.21.15.82:0/823809230 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20240b2070 con 0x7f202c071e90 2024-04-25T10:06:44.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2030936640 1 -- 172.21.15.82:0/823809230 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20240a5740 con 0x7f202c071e90 2024-04-25T10:06:44.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2030936640 1 -- 172.21.15.82:0/823809230 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20240a5b10 con 0x7f202c071e90 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 -- 172.21.15.82:0/823809230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c071e90 msgr2=0x7f202c071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.222+0000 7f2033b9c640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c071e90 0x7f202c071570 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f2024009d90 tx=0x7f202401c850 comp rx=0 tx=0).stop 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/823809230 shutdown_connections 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c071e90 0x7f202c071570 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 --2- 172.21.15.82:0/823809230 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/823809230 >> 172.21.15.82:0/823809230 conn(0x7f202c0b11a0 msgr2=0x7f202c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/823809230 shutdown_connections 2024-04-25T10:06:44.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/823809230 wait complete. 2024-04-25T10:06:44.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 Processor -- start 2024-04-25T10:06:44.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- start start 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f202c071e90 0x7f202c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c14aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c16aa10 con 0x7f202c0b24a0 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202c16a890 con 0x7f202c071e90 2024-04-25T10:06:44.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f202c16ab90 con 0x7f202c15da10 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39500/0 (socket says 172.21.15.82:39500) 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 -- 172.21.15.82:0/799196730 learned_addr learned my addr 172.21.15.82:0/799196730 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2032139640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c14aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031938640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f202c071e90 0x7f202c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 -- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 msgr2=0x7f202c14aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c14aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 -- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f202c071e90 msgr2=0x7f202c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f202c071e90 0x7f202c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 -- 172.21.15.82:0/799196730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202c1569d0 con 0x7f202c0b24a0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2031137640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c4e0 tx=0x7f201c00c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c018070 con 0x7f202c0b24a0 2024-04-25T10:06:44.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201c007500 con 0x7f202c0b24a0 2024-04-25T10:06:44.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201c004a30 con 0x7f202c0b24a0 2024-04-25T10:06:44.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202c15fb60 con 0x7f202c0b24a0 2024-04-25T10:06:44.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.226+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c160120 con 0x7f202c0b24a0 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f202c04efa0 con 0x7f202c0b24a0 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f201c004bd0 con 0x7f202c0b24a0 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f20227fc640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 0x7f2010069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f201c0841b0 con 0x7f202c0b24a0 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f2031938640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 0x7f2010069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.230+0000 7f2031938640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 0x7f2010069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f20280096f0 tx=0x7f2028008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.234+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f201c055440 con 0x7f202c0b24a0 2024-04-25T10:06:44.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f202c0b1dc0 con 0x7f202c0b24a0 2024-04-25T10:06:44.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f20227fc640 1 -- 172.21.15.82:0/799196730 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7f201c054d40 con 0x7f202c0b24a0 2024-04-25T10:06:44.394 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:44.394 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:06:44.394 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:06:44.394 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:06:44.395 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:06:44.395 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:44.395 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:06:44.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 msgr2=0x7f2010069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 0x7f2010069450 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f20280096f0 tx=0x7f2028008040 comp rx=0 tx=0).stop 2024-04-25T10:06:44.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 msgr2=0x7f202c160640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f201c00c4e0 tx=0x7f201c00c9a0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 shutdown_connections 2024-04-25T10:06:44.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2010066f90 0x7f2010069450 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f202c15da10 0x7f202c14aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f202c0b24a0 0x7f202c160640 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 --2- 172.21.15.82:0/799196730 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f202c071e90 0x7f202c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 >> 172.21.15.82:0/799196730 conn(0x7f202c0b11a0 msgr2=0x7f202c06d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 shutdown_connections 2024-04-25T10:06:44.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.390+0000 7f2033b9c640 1 -- 172.21.15.82:0/799196730 wait complete. 2024-04-25T10:06:44.411 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:44.411 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 Processor -- start 2024-04-25T10:06:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 -- start start 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8f00b36d0 0x7fc8f00b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00cd000 0x7fc8f00b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f00b3bd0 con 0x7fc8f0159f20 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f00b3eb0 con 0x7fc8f00b36d0 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f8206640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00b4190 con 0x7fc8f00cd000 2024-04-25T10:06:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f5fa2640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc8f0159f20 0x7fc8f006d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:37516/0 (socket says 172.21.15.82:37516) 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f5fa2640 1 -- 172.21.15.82:0/1562417883 learned_addr learned my addr 172.21.15.82:0/1562417883 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/1562417883 >> v2:172.21.15.82:3300/0 conn(0x7fc8f00b36d0 0x7fc8f00b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f57a1640 1 -- 172.21.15.82:0/1562417883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00cd000 msgr2=0x7fc8f00b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/1562417883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00cd000 0x7fc8f00b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f57a1640 1 -- 172.21.15.82:0/1562417883 >> v1:172.21.15.82:6789/0 conn(0x7fc8f0159f20 legacy=0x7fc8f006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f57a1640 1 -- 172.21.15.82:0/1562417883 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f006cf80 con 0x7fc8f00b36d0 2024-04-25T10:06:44.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.534+0000 7fc8f4fa0640 1 -- 172.21.15.82:0/1562417883 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460446229 0 0) 0x7fc8f00b3bd0 con 0x7fc8f0159f20 2024-04-25T10:06:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/1562417883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00b36d0 0x7fc8f00b2190 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097c0 tx=0x7fc8e001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799dc82ee511051 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f4fa0640 1 -- 172.21.15.82:0/1562417883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b2070 con 0x7fc8f00b36d0 2024-04-25T10:06:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f4fa0640 1 -- 172.21.15.82:0/1562417883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00ad640 con 0x7fc8f00b36d0 2024-04-25T10:06:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f4fa0640 1 -- 172.21.15.82:0/1562417883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ada10 con 0x7fc8f00b36d0 2024-04-25T10:06:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- 172.21.15.82:0/1562417883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00b36d0 msgr2=0x7fc8f00b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- 172.21.15.82:0/1562417883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00b36d0 0x7fc8f00b2190 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fc8e00097c0 tx=0x7fc8e001c470 comp rx=0 tx=0).stop 2024-04-25T10:06:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- 172.21.15.82:0/1562417883 shutdown_connections 2024-04-25T10:06:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- 172.21.15.82:0/1562417883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00cd000 0x7fc8f00b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- 172.21.15.82:0/1562417883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00b36d0 0x7fc8f00b2190 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- 172.21.15.82:0/1562417883 >> 172.21.15.82:0/1562417883 conn(0x7fc8f006ae60 msgr2=0x7fc8f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- 172.21.15.82:0/1562417883 shutdown_connections 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- 172.21.15.82:0/1562417883 wait complete. 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 Processor -- start 2024-04-25T10:06:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- start start 2024-04-25T10:06:44.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00b36d0 0x7fc8f01525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f0159f20 0x7fc8f0154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00cbd30 con 0x7fc8f00cd000 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00cbbb0 con 0x7fc8f0159f20 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f8206640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f00cbeb0 con 0x7fc8f00b36d0 2024-04-25T10:06:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f57a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f57a1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39522/0 (socket says 172.21.15.82:39522) 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.538+0000 7fc8f57a1640 1 -- 172.21.15.82:0/3480385447 learned_addr learned my addr 172.21.15.82:0/3480385447 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f5fa2640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00b36d0 0x7fc8f01525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 -- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00b36d0 msgr2=0x7fc8f01525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f67a3640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f0159f20 0x7fc8f0154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00b36d0 0x7fc8f01525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 -- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f0159f20 msgr2=0x7fc8f0154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f0159f20 0x7fc8f0154610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 -- 172.21.15.82:0/3480385447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f00ad480 con 0x7fc8f00cd000 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f57a1640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc8e001a0a0 tx=0x7fc8e0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00ba070 con 0x7fc8f00cd000 2024-04-25T10:06:44.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e00b6630 con 0x7fc8f00cd000 2024-04-25T10:06:44.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f00c6dc0 con 0x7fc8f00cd000 2024-04-25T10:06:44.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e00b2040 con 0x7fc8f00cd000 2024-04-25T10:06:44.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f00c7380 con 0x7fc8f00cd000 2024-04-25T10:06:44.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f0067c30 con 0x7fc8f00cd000 2024-04-25T10:06:44.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8e00bd070 con 0x7fc8f00cd000 2024-04-25T10:06:44.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 0x7fc8d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc8e011efd0 con 0x7fc8f00cd000 2024-04-25T10:06:44.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f5fa2640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 0x7fc8d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.542+0000 7fc8f5fa2640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 0x7fc8d4069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec000f70 tx=0x7fc8ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.546+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8e0016090 con 0x7fc8f00cd000 2024-04-25T10:06:44.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:44.702+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc8f006e150 con 0x7fc8f00cd000 2024-04-25T10:06:45.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.054+0000 7fc8e67fc640 1 -- 172.21.15.82:0/3480385447 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc8e00f01b0 con 0x7fc8f00cd000 2024-04-25T10:06:45.057 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:45.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 msgr2=0x7fc8d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 0x7fc8d4069450 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec000f70 tx=0x7fc8ec009040 comp rx=0 tx=0).stop 2024-04-25T10:06:45.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 msgr2=0x7fc8f01540d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fc8e001a0a0 tx=0x7fc8e0019210 comp rx=0 tx=0).stop 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 shutdown_connections 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8d4066f90 0x7fc8d4069450 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8f0159f20 0x7fc8f0154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8f00cd000 0x7fc8f01540d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 --2- 172.21.15.82:0/3480385447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8f00b36d0 0x7fc8f01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 >> 172.21.15.82:0/3480385447 conn(0x7fc8f006ae60 msgr2=0x7fc8f00c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 shutdown_connections 2024-04-25T10:06:45.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.058+0000 7fc8f8206640 1 -- 172.21.15.82:0/3480385447 wait complete. 2024-04-25T10:06:45.091 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:45.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 Processor -- start 2024-04-25T10:06:45.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 -- start start 2024-04-25T10:06:45.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f21281634c0 0x7f21281658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280ca4c0 con 0x7f212815ecb0 2024-04-25T10:06:45.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21280cd690 con 0x7f21280cf590 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212ce1c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21280ca780 con 0x7f21281634c0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f212815ecb0 0x7f21280c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35600/0 (socket says 172.21.15.82:35600) 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 -- 172.21.15.82:0/615944761 learned_addr learned my addr 172.21.15.82:0/615944761 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2126d9d640 1 --2- 172.21.15.82:0/615944761 >> v2:172.21.15.196:3300/0 conn(0x7f21281634c0 0x7f21281658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195408997 0 0) 0x7f21280ca4c0 con 0x7f212815ecb0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2110003530 con 0x7f212815ecb0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374116914 0 0) 0x7f2110003530 con 0x7f212815ecb0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 >> v2:172.21.15.196:3300/0 conn(0x7f21281634c0 msgr2=0x7f21281658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 --2- 172.21.15.82:0/615944761 >> v2:172.21.15.196:3300/0 conn(0x7f21281634c0 0x7f21281658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 >> v1:172.21.15.196:6789/0 conn(0x7f21280cf590 legacy=0x7f21280cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21280cd970 con 0x7f212815ecb0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2118001680 con 0x7f212815ecb0 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 >> v1:172.21.15.82:6789/0 conn(0x7f212815ecb0 legacy=0x7f21280c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2110003a30 0x7f2110003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 0x7f2110004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21100051d0 0x7f211000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211000c0d0 con 0x7f2110004810 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211000e490 con 0x7f21100051d0 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211000e690 con 0x7f2110003a30 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 0x7f2110004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f212659c640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2110003a30 0x7f2110003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2126d9d640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21100051d0 0x7f211000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 -- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2110003a30 msgr2=0x7f2110003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2110003a30 0x7f2110003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 -- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21100051d0 msgr2=0x7f211000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21100051d0 0x7f211000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2125d9b640 1 -- 172.21.15.82:0/615944761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110003530 con 0x7f2110004810 2024-04-25T10:06:45.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2126d9d640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21100051d0 0x7f211000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.218+0000 7f2126d9d640 1 -- 172.21.15.82:0/615944761 reap_dead start 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f2125d9b640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 0x7f2110004c90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f211800c240 tx=0x7f211801f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=514a41f17f36fe4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118007d20 con 0x7f2110004810 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2118005ae0 con 0x7f2110004810 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212559a640 1 -- 172.21.15.82:0/615944761 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118007070 con 0x7f2110004810 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 msgr2=0x7f2110004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 0x7f2110004c90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f211800c240 tx=0x7f211801f370 comp rx=0 tx=0).stop 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- 172.21.15.82:0/615944761 shutdown_connections 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 --2- 172.21.15.82:0/615944761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2110004810 0x7f2110004c90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- 172.21.15.82:0/615944761 >> 172.21.15.82:0/615944761 conn(0x7f212806ae60 msgr2=0x7f212806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- 172.21.15.82:0/615944761 shutdown_connections 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- 172.21.15.82:0/615944761 wait complete. 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 Processor -- start 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.222+0000 7f212ce1c640 1 -- start start 2024-04-25T10:06:45.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 0x7f212815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815c130 0x7f212806ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cd440 con 0x7f2128156e10 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212815f0f0 con 0x7f2110004810 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21280cdce0 con 0x7f212815c130 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57996/0 (socket says 172.21.15.82:57996) 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 -- 172.21.15.82:0/2219842076 learned_addr learned my addr 172.21.15.82:0/2219842076 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212659c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 0x7f212815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2126d9d640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815c130 0x7f212806ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 -- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815c130 msgr2=0x7f212806ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815c130 0x7f212806ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 -- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 msgr2=0x7f212815b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 0x7f212815b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 -- 172.21.15.82:0/2219842076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212815c760 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212659c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 0x7f212815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2125d9b640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f211801f1a0 tx=0x7f2110005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180047f0 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2118004990 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21280722d0 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118004d40 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2128072890 con 0x7f2128156e10 2024-04-25T10:06:45.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212804efa0 con 0x7f2128156e10 2024-04-25T10:06:45.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2118007070 con 0x7f2128156e10 2024-04-25T10:06:45.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 0x7f2108069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f21181222b0 con 0x7f2128156e10 2024-04-25T10:06:45.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212659c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 0x7f2108069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.226+0000 7f212659c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 0x7f2108069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f211c007fd0 tx=0x7f211c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.230+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f212815c760 con 0x7f2128156e10 2024-04-25T10:06:45.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:45.378+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f21280b4170 con 0x7f2128156e10 2024-04-25T10:06:46.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.066+0000 7f2116ffd640 1 -- 172.21.15.82:0/2219842076 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2118019080 con 0x7f2128156e10 2024-04-25T10:06:46.069 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:06:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 msgr2=0x7f2108069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 0x7f2108069420 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f211c007fd0 tx=0x7f211c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 msgr2=0x7f212815bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f211801f1a0 tx=0x7f2110005840 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 shutdown_connections 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2108066f60 0x7f2108069420 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f212815c130 0x7f212806ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2128156e10 0x7f212815bbf0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 --2- 172.21.15.82:0/2219842076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2110004810 0x7f212815b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 >> 172.21.15.82:0/2219842076 conn(0x7f212806ae60 msgr2=0x7f21280b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 shutdown_connections 2024-04-25T10:06:46.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.070+0000 7f212ce1c640 1 -- 172.21.15.82:0/2219842076 wait complete. 2024-04-25T10:06:46.104 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:46.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.226+0000 7fd03cf85640 1 Processor -- start 2024-04-25T10:06:46.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.226+0000 7fd03cf85640 1 -- start start 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.226+0000 7fd03cf85640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd03806b440 0x7fd0380b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.226+0000 7fd03cf85640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd03806b820 0x7fd0380b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.226+0000 7fd03cf85640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0380afb20 con 0x7fd0380b18f0 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0380b06c0 con 0x7fd03806b440 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0380afd90 con 0x7fd03806b820 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd03806b440 0x7fd0380b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd03806b440 0x7fd0380b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58000/0 (socket says 172.21.15.82:58000) 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 -- 172.21.15.82:0/3840600805 learned_addr learned my addr 172.21.15.82:0/3840600805 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd036d9d640 1 --1- 172.21.15.82:0/3840600805 >> v1:172.21.15.82:6789/0 conn(0x7fd0380b18f0 0x7fd0380cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35608/0 (socket says 172.21.15.82:35608) 2024-04-25T10:06:46.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 -- 172.21.15.82:0/3840600805 >> v2:172.21.15.196:3300/0 conn(0x7fd03806b820 msgr2=0x7fd0380b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd02ffff640 1 --2- 172.21.15.82:0/3840600805 >> v2:172.21.15.196:3300/0 conn(0x7fd03806b820 0x7fd0380b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 --2- 172.21.15.82:0/3840600805 >> v2:172.21.15.196:3300/0 conn(0x7fd03806b820 0x7fd0380b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 -- 172.21.15.82:0/3840600805 >> v1:172.21.15.82:6789/0 conn(0x7fd0380b18f0 legacy=0x7fd0380cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 -- 172.21.15.82:0/3840600805 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0380b1d40 con 0x7fd03806b440 2024-04-25T10:06:46.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03659c640 1 --2- 172.21.15.82:0/3840600805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd0380b0b50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd038161530 tx=0x7fd02801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafd06ab07914204 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd035d9b640 1 -- 172.21.15.82:0/3840600805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280b0070 con 0x7fd03806b440 2024-04-25T10:06:46.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd035d9b640 1 -- 172.21.15.82:0/3840600805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0280ab600 con 0x7fd03806b440 2024-04-25T10:06:46.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd035d9b640 1 -- 172.21.15.82:0/3840600805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280ab9d0 con 0x7fd03806b440 2024-04-25T10:06:46.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- 172.21.15.82:0/3840600805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 msgr2=0x7fd0380b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 --2- 172.21.15.82:0/3840600805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd0380b0b50 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fd038161530 tx=0x7fd02801c520 comp rx=0 tx=0).stop 2024-04-25T10:06:46.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- 172.21.15.82:0/3840600805 shutdown_connections 2024-04-25T10:06:46.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 --2- 172.21.15.82:0/3840600805 >> v2:172.21.15.196:3300/0 conn(0x7fd03806b820 0x7fd0380b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 --2- 172.21.15.82:0/3840600805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd0380b0b50 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- 172.21.15.82:0/3840600805 >> 172.21.15.82:0/3840600805 conn(0x7fd0380b1f10 msgr2=0x7fd0380b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- 172.21.15.82:0/3840600805 shutdown_connections 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.230+0000 7fd03cf85640 1 -- 172.21.15.82:0/3840600805 wait complete. 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 Processor -- start 2024-04-25T10:06:46.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- start start 2024-04-25T10:06:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd03806b820 0x7fd0381548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0380b18f0 0x7fd038158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380cd130 con 0x7fd03806b440 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0380ccfb0 con 0x7fd0380b18f0 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03815d080 con 0x7fd03806b820 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58004/0 (socket says 172.21.15.82:58004) 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 -- 172.21.15.82:0/800733172 learned_addr learned my addr 172.21.15.82:0/800733172 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd02ffff640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd03806b820 0x7fd0381548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd036d9d640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0380b18f0 0x7fd038158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 -- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd03806b820 msgr2=0x7fd0381548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd03806b820 0x7fd0381548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 -- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0380b18f0 msgr2=0x7fd038158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0380b18f0 0x7fd038158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 -- 172.21.15.82:0/800733172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0381588b0 con 0x7fd03806b440 2024-04-25T10:06:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03659c640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd02801abd0 tx=0x7fd0280192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280bb070 con 0x7fd03806b440 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0280b6630 con 0x7fd03806b440 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd038158b90 con 0x7fd03806b440 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0280b0040 con 0x7fd03806b440 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd038159150 con 0x7fd03806b440 2024-04-25T10:06:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.234+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03804efa0 con 0x7fd03806b440 2024-04-25T10:06:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.238+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0280be070 con 0x7fd03806b440 2024-04-25T10:06:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.238+0000 7fd02effd640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 0x7fd018069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.238+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd02811e8c0 con 0x7fd03806b440 2024-04-25T10:06:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.238+0000 7fd02ffff640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 0x7fd018069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.238+0000 7fd02ffff640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 0x7fd018069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd03806ed70 tx=0x7fd020008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.242+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd028016110 con 0x7fd03806b440 2024-04-25T10:06:46.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:46.422+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd0380cc160 con 0x7fd03806b440 2024-04-25T10:06:47.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.066+0000 7fd02effd640 1 -- 172.21.15.82:0/800733172 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd0280ef910 con 0x7fd03806b440 2024-04-25T10:06:47.071 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 msgr2=0x7fd018069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 0x7fd018069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd03806ed70 tx=0x7fd020008040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 msgr2=0x7fd03815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd02801abd0 tx=0x7fd0280192c0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 shutdown_connections 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd018066f90 0x7fd018069450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd03806b820 0x7fd0381548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0380b18f0 0x7fd038158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 --2- 172.21.15.82:0/800733172 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd03806b440 0x7fd03815aeb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 >> 172.21.15.82:0/800733172 conn(0x7fd0380b1f10 msgr2=0x7fd038155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 shutdown_connections 2024-04-25T10:06:47.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.070+0000 7fd03cf85640 1 -- 172.21.15.82:0/800733172 wait complete. 2024-04-25T10:06:47.100 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 Processor -- start 2024-04-25T10:06:47.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 -- start start 2024-04-25T10:06:47.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9e40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e406be60 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e415de60 con 0x7fd9e415da80 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9ea58b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406c440 con 0x7fd9e40d07f0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e8b28640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd9e40d1cd0 0x7fd9e406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35612/0 (socket says 172.21.15.82:35612) 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3100091024 learned_addr learned my addr 172.21.15.82:0/3100091024 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e37fe640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9e40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2574827730 0 0) 0x7fd9e406be60 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9cc003530 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2420174877 0 0) 0x7fd9cc003530 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 msgr2=0x7fd9e40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9e40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 >> v1:172.21.15.196:6789/0 conn(0x7fd9e415da80 legacy=0x7fd9e40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40d2450 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fd9dc001a00 con 0x7fd9e40d1cd0 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 >> v1:172.21.15.82:6789/0 conn(0x7fd9e40d1cd0 legacy=0x7fd9e406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9cc003a60 0x7fd9cc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9cc004810 0x7fd9cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406c440 con 0x7fd9cc003a60 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00bb00 con 0x7fd9cc004810 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc00bd00 con 0x7fd9e40d07f0 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9cc003a60 0x7fd9cc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 -- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 msgr2=0x7fd9cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e8b28640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e37fe640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9cc004810 0x7fd9cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 -- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9cc004810 msgr2=0x7fd9cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9cc004810 0x7fd9cc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e3fff640 1 -- 172.21.15.82:0/3100091024 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc003530 con 0x7fd9cc003a60 2024-04-25T10:06:47.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.218+0000 7fd9e8b28640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9e40d07f0 0x7fd9cc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9cc003a60 0x7fd9cc003e80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001780 tx=0x7fd9d401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab43ba2b2a137d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d40013a0 con 0x7fd9cc003a60 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d40052f0 con 0x7fd9cc003a60 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e2ffd640 1 -- 172.21.15.82:0/3100091024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4003840 con 0x7fd9cc003a60 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3100091024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9cc003a60 msgr2=0x7fd9cc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3100091024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9cc003a60 0x7fd9cc003e80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001780 tx=0x7fd9d401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3100091024 reap_dead start 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3100091024 shutdown_connections 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3100091024 >> 172.21.15.82:0/3100091024 conn(0x7fd9e408b000 msgr2=0x7fd9e40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3100091024 shutdown_connections 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3100091024 wait complete. 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 Processor -- start 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- start start 2024-04-25T10:06:47.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9cc003a60 0x7fd9e415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9e415c0e0 0x7fd9e4163060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406d970 con 0x7fd9e41635a0 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406d7f0 con 0x7fd9e415c0e0 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9ea58b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e406daf0 con 0x7fd9cc003a60 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58024/0 (socket says 172.21.15.82:58024) 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3676045215 learned_addr learned my addr 172.21.15.82:0/3676045215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9cc003a60 0x7fd9e415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e37fe640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9e415c0e0 0x7fd9e4163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9cc003a60 msgr2=0x7fd9e415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.222+0000 7fd9e8b28640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9cc003a60 0x7fd9e415c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9e415c0e0 msgr2=0x7fd9e4163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e8b28640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9e415c0e0 0x7fd9e4163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e8b28640 1 -- 172.21.15.82:0/3676045215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4073a90 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e8b28640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd9e406de30 tx=0x7fd9dc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc004d70 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc00bb80 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc007070 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e4162200 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e40cd1b0 con 0x7fd9e41635a0 2024-04-25T10:06:47.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e4162720 con 0x7fd9e41635a0 2024-04-25T10:06:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9dc007210 con 0x7fd9e41635a0 2024-04-25T10:06:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 0x7fd9d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9dc084fc0 con 0x7fd9e41635a0 2024-04-25T10:06:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 0x7fd9d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.226+0000 7fd9e3fff640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 0x7fd9d0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001750 tx=0x7fd9d40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.230+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9e4162720 con 0x7fd9e41635a0 2024-04-25T10:06:47.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.390+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd9e404efa0 con 0x7fd9e41635a0 2024-04-25T10:06:47.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.394+0000 7fd9e0ff9640 1 -- 172.21.15.82:0/3676045215 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16970 (secure 0 0 0) 0x7fd9dc0557f0 con 0x7fd9e41635a0 2024-04-25T10:06:47.398 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:47.398 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":78,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:46.427704+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 msgr2=0x7fd9d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 0x7fd9d0069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4001750 tx=0x7fd9d40011a0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 msgr2=0x7fd9e4163a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd9e406de30 tx=0x7fd9dc00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 shutdown_connections 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9d0066f60 0x7fd9d0069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9e41635a0 0x7fd9e4163a20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9e415c0e0 0x7fd9e4163060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 --2- 172.21.15.82:0/3676045215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9cc003a60 0x7fd9e415c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 >> 172.21.15.82:0/3676045215 conn(0x7fd9e408b000 msgr2=0x7fd9e4160250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 shutdown_connections 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.398+0000 7fd9ea58b640 1 -- 172.21.15.82:0/3676045215 wait complete. 2024-04-25T10:06:47.417 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:47.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.530+0000 7f543061e640 1 Processor -- start 2024-04-25T10:06:47.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.530+0000 7f543061e640 1 -- start start 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f543061e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f542815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f543061e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f542806b710 con 0x7f542806fc50 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f543061e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54281617b0 con 0x7f54280cf840 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f543061e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5428161560 con 0x7f542815d180 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542e3ba640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f542806fc50 0x7f542806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35628/0 (socket says 172.21.15.82:35628) 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542e3ba640 1 -- 172.21.15.82:0/4082696875 learned_addr learned my addr 172.21.15.82:0/4082696875 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f542815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768703259 0 0) 0x7f542806b710 con 0x7f542806fc50 2024-04-25T10:06:47.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5410003530 con 0x7f542806fc50 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1436851702 0 0) 0x7f5410003530 con 0x7f542806fc50 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 msgr2=0x7f542815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f542815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 >> v1:172.21.15.196:6789/0 conn(0x7f54280cf840 legacy=0x7f54280cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5428161a90 con 0x7f542806fc50 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5420001a00 con 0x7f542806fc50 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 >> v1:172.21.15.82:6789/0 conn(0x7f542806fc50 legacy=0x7f542806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5410003a60 0x7f5410003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f5410004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410004d50 0x7f54100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5428161560 con 0x7f5410004d50 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541000bb60 con 0x7f5410003a60 2024-04-25T10:06:47.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541000bd60 con 0x7f542815d180 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410004d50 0x7f54100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542e3ba640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5410003a60 0x7f5410003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542dbb9640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f5410004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 -- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 msgr2=0x7f5410004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f542815d180 0x7f5410004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 -- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5410003a60 msgr2=0x7f5410003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5410003a60 0x7f5410003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 -- 172.21.15.82:0/4082696875 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5410003530 con 0x7f5410004d50 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542ebbb640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410004d50 0x7f54100051d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5418009ae0 tx=0x7f541801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa59947594dfc664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180b2070 con 0x7f5410004d50 2024-04-25T10:06:47.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54180a5550 con 0x7f5410004d50 2024-04-25T10:06:47.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.534+0000 7f542d3b8640 1 -- 172.21.15.82:0/4082696875 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54180096b0 con 0x7f5410004d50 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/4082696875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410004d50 msgr2=0x7f54100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 --2- 172.21.15.82:0/4082696875 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5410004d50 0x7f54100051d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f5418009ae0 tx=0x7f541801ca70 comp rx=0 tx=0).stop 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542ebbb640 1 -- 172.21.15.82:0/4082696875 reap_dead start 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/4082696875 shutdown_connections 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/4082696875 >> 172.21.15.82:0/4082696875 conn(0x7f5428099a70 msgr2=0x7f54280d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/4082696875 shutdown_connections 2024-04-25T10:06:47.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/4082696875 wait complete. 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 Processor -- start 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- start start 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54280b1850 0x7f54280c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54280c5a70 0x7f54280c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54280d0e30 con 0x7f542815c3d0 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f542806b3a0 con 0x7f54280b1850 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5428070030 con 0x7f54280c5a70 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58036/0 (socket says 172.21.15.82:58036) 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 -- 172.21.15.82:0/842273870 learned_addr learned my addr 172.21.15.82:0/842273870 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:47.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542ebbb640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54280c5a70 0x7f54280c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542dbb9640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54280b1850 0x7f54280c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 -- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54280c5a70 msgr2=0x7f54280c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54280c5a70 0x7f54280c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 -- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54280b1850 msgr2=0x7f54280c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54280b1850 0x7f54280c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 -- 172.21.15.82:0/842273870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280c6260 con 0x7f542815c3d0 2024-04-25T10:06:47.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f542e3ba640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5420011790 tx=0x7f5420011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5420010040 con 0x7f542815c3d0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542001a070 con 0x7f542815c3d0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542000fda0 con 0x7f542815c3d0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54280c6540 con 0x7f542815c3d0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.538+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54280c6b00 con 0x7f542815c3d0 2024-04-25T10:06:47.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f542804efa0 con 0x7f542815c3d0 2024-04-25T10:06:47.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f542001f460 con 0x7f542815c3d0 2024-04-25T10:06:47.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f5426ffd640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 0x7f5414069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5420088e30 con 0x7f542815c3d0 2024-04-25T10:06:47.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f542dbb9640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 0x7f5414069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.542+0000 7f542dbb9640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 0x7f5414069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54280b2750 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.546+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5420059d90 con 0x7f542815c3d0 2024-04-25T10:06:47.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.706+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54280b1c30 con 0x7f542815c3d0 2024-04-25T10:06:47.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f5426ffd640 1 -- 172.21.15.82:0/842273870 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16970 (secure 0 0 0) 0x7f5420059690 con 0x7f542815c3d0 2024-04-25T10:06:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:47.712 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":78,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:46.427704+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:47.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 msgr2=0x7f5414069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 0x7f5414069420 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f54280b2750 tx=0x7f541c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:47.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 msgr2=0x7f54280b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f5420011790 tx=0x7f5420011c50 comp rx=0 tx=0).stop 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 shutdown_connections 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5414066f60 0x7f5414069420 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54280c5a70 0x7f54280c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54280b1850 0x7f54280c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 --2- 172.21.15.82:0/842273870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f542815c3d0 0x7f54280b1310 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 >> 172.21.15.82:0/842273870 conn(0x7f5428099a70 msgr2=0x7f54280c95e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 shutdown_connections 2024-04-25T10:06:47.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.710+0000 7f543061e640 1 -- 172.21.15.82:0/842273870 wait complete. 2024-04-25T10:06:47.731 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:47.736 INFO:tasks.ceph.mds.a.smithi082.stderr:2024-04-25T10:06:47.734+0000 7fc6dd70c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 Processor -- start 2024-04-25T10:06:47.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 -- start start 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469015b6b0 0x7f46900ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46900cf590 0x7f46900cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f469015f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f469015ba90 con 0x7f469015b6b0 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46900cbd20 con 0x7f46900cf590 2024-04-25T10:06:47.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f46956dc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900cc000 con 0x7f469015f130 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469015b6b0 0x7f46900ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f469015b6b0 0x7f46900ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58048/0 (socket says 172.21.15.82:58048) 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 -- 172.21.15.82:0/3518573120 learned_addr learned my addr 172.21.15.82:0/3518573120 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468e7fc640 1 --2- 172.21.15.82:0/3518573120 >> v2:172.21.15.196:3300/0 conn(0x7f46900cf590 0x7f46900cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468f7fe640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f469015f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 -- 172.21.15.82:0/3518573120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 msgr2=0x7f469015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f469015f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 -- 172.21.15.82:0/3518573120 >> v2:172.21.15.196:3300/0 conn(0x7f46900cf590 msgr2=0x7f46900cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 --2- 172.21.15.82:0/3518573120 >> v2:172.21.15.196:3300/0 conn(0x7f46900cf590 0x7f46900cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.858+0000 7f468effd640 1 -- 172.21.15.82:0/3518573120 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900cc2e0 con 0x7f469015b6b0 2024-04-25T10:06:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f468effd640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f46900ca3c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f46900b4430 tx=0x7f468401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fe7e1f381bef56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f468dffb640 1 -- 172.21.15.82:0/3518573120 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b1070 con 0x7f469015b6b0 2024-04-25T10:06:47.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f468dffb640 1 -- 172.21.15.82:0/3518573120 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840ac630 con 0x7f469015b6b0 2024-04-25T10:06:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f468dffb640 1 -- 172.21.15.82:0/3518573120 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840a8050 con 0x7f469015b6b0 2024-04-25T10:06:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- 172.21.15.82:0/3518573120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 msgr2=0x7f46900ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f46900ca3c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f46900b4430 tx=0x7f468401c520 comp rx=0 tx=0).stop 2024-04-25T10:06:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- 172.21.15.82:0/3518573120 shutdown_connections 2024-04-25T10:06:47.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f469015f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- 172.21.15.82:0/3518573120 >> v2:172.21.15.196:3300/0 conn(0x7f46900cf590 0x7f46900cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- 172.21.15.82:0/3518573120 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f46900ca3c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- 172.21.15.82:0/3518573120 >> 172.21.15.82:0/3518573120 conn(0x7f469006ae60 msgr2=0x7f469006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- 172.21.15.82:0/3518573120 shutdown_connections 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- 172.21.15.82:0/3518573120 wait complete. 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 Processor -- start 2024-04-25T10:06:47.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- start start 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 0x7f46901561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f4690165550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900ca900 con 0x7f469015b6b0 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469015fef0 con 0x7f46900cf590 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.862+0000 7f46956dc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900cadb0 con 0x7f469015f130 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58054/0 (socket says 172.21.15.82:58054) 2024-04-25T10:06:47.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 -- 172.21.15.82:0/2012448684 learned_addr learned my addr 172.21.15.82:0/2012448684 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468f7fe640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f4690165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468effd640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 0x7f46901561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 -- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 msgr2=0x7f4690165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f4690165550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 -- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 msgr2=0x7f46901561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 0x7f46901561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 -- 172.21.15.82:0/2012448684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900c2f60 con 0x7f469015b6b0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468effd640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 0x7f46901561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468e7fc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f468000d830 tx=0x7f468000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000b6c0 con 0x7f469015b6b0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4680019070 con 0x7f469015b6b0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46900c3240 con 0x7f469015b6b0 2024-04-25T10:06:47.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000ba60 con 0x7f469015b6b0 2024-04-25T10:06:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46900c3800 con 0x7f469015b6b0 2024-04-25T10:06:47.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469004efa0 con 0x7f469015b6b0 2024-04-25T10:06:47.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f468000bc00 con 0x7f469015b6b0 2024-04-25T10:06:47.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46800855d0 con 0x7f469015b6b0 2024-04-25T10:06:47.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468effd640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.866+0000 7f468effd640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae000 tx=0x7f4684008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:47.870+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46800564e0 con 0x7f469015b6b0 2024-04-25T10:06:48.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.066+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4690164a80 con 0x7f469015b6b0 2024-04-25T10:06:48.530 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:06:48.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.770+0000 7f46777fe640 1 -- 172.21.15.82:0/2012448684 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4680055de0 con 0x7f469015b6b0 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 msgr2=0x7f467c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f46840ae000 tx=0x7f4684008040 comp rx=0 tx=0).stop 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 msgr2=0x7f4690165010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f468000d830 tx=0x7f468000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 shutdown_connections 2024-04-25T10:06:48.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f469015f130 0x7f4690165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f469015b6b0 0x7f4690165010 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 --2- 172.21.15.82:0/2012448684 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46900cf590 0x7f46901561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 >> 172.21.15.82:0/2012448684 conn(0x7f469006ae60 msgr2=0x7f46900aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 shutdown_connections 2024-04-25T10:06:48.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.774+0000 7f46956dc640 1 -- 172.21.15.82:0/2012448684 wait complete. 2024-04-25T10:06:48.795 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:48.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:48.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 Processor -- start 2024-04-25T10:06:48.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 -- start start 2024-04-25T10:06:48.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78900cf590 0x7f78900cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 0x7f7890072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78900cbd20 con 0x7f789015b6b0 2024-04-25T10:06:48.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789015ba90 con 0x7f78900cf590 2024-04-25T10:06:48.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f7895fdd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cc000 con 0x7f7890072000 2024-04-25T10:06:48.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 0x7f7890072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788effd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78900cf590 0x7f78900cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 0x7f7890072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34716/0 (socket says 172.21.15.82:34716) 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788ffff640 1 -- 172.21.15.82:0/1574812629 learned_addr learned my addr 172.21.15.82:0/1574812629 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788f7fe640 1 --1- 172.21.15.82:0/1574812629 >> v1:172.21.15.196:6789/0 conn(0x7f789015b6b0 0x7f78900ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:52168/0 (socket says 172.21.15.82:52168) 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788effd640 1 -- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 msgr2=0x7f7890072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788effd640 1 --2- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 0x7f7890072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788effd640 1 -- 172.21.15.82:0/1574812629 >> v1:172.21.15.196:6789/0 conn(0x7f789015b6b0 legacy=0x7f78900ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.966+0000 7f788effd640 1 -- 172.21.15.82:0/1574812629 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78900cc2e0 con 0x7f78900cf590 2024-04-25T10:06:48.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f788effd640 1 --2- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900cf590 0x7f78900cfa20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78800097c0 tx=0x7f788001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d7f7826815ad55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f788e7fc640 1 -- 172.21.15.82:0/1574812629 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800b2070 con 0x7f78900cf590 2024-04-25T10:06:48.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f788e7fc640 1 -- 172.21.15.82:0/1574812629 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78800ad640 con 0x7f78900cf590 2024-04-25T10:06:48.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f788e7fc640 1 -- 172.21.15.82:0/1574812629 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800ada10 con 0x7f78900cf590 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900cf590 msgr2=0x7f78900cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900cf590 0x7f78900cfa20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f78800097c0 tx=0x7f788001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- 172.21.15.82:0/1574812629 shutdown_connections 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7890072000 0x7f7890072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- 172.21.15.82:0/1574812629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900cf590 0x7f78900cfa20 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- 172.21.15.82:0/1574812629 >> 172.21.15.82:0/1574812629 conn(0x7f789006ae60 msgr2=0x7f789006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- 172.21.15.82:0/1574812629 shutdown_connections 2024-04-25T10:06:48.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- 172.21.15.82:0/1574812629 wait complete. 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 Processor -- start 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- start start 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900cf590 0x7f78900c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 0x7f78901f7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900caf30 con 0x7f7890072000 2024-04-25T10:06:48.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78900cadb0 con 0x7f789015b6b0 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.970+0000 7f7895fdd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78901682a0 con 0x7f78900cf590 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58064/0 (socket says 172.21.15.82:58064) 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 -- 172.21.15.82:0/904394879 learned_addr learned my addr 172.21.15.82:0/904394879 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788ffff640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 0x7f78901f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788effd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900cf590 0x7f78900c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 -- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900cf590 msgr2=0x7f78900c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900cf590 0x7f78900c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 -- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 msgr2=0x7f78901f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 0x7f78901f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 -- 172.21.15.82:0/904394879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890153290 con 0x7f7890072000 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788ffff640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 0x7f78901f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:48.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788f7fe640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f787800c370 tx=0x7f787800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780074e0 con 0x7f7890072000 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7878007680 con 0x7f7890072000 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7878004a30 con 0x7f7890072000 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890097d90 con 0x7f7890072000 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7890098350 con 0x7f7890072000 2024-04-25T10:06:48.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789004ef80 con 0x7f7890072000 2024-04-25T10:06:48.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f787801a020 con 0x7f7890072000 2024-04-25T10:06:48.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 0x7f7870069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:48.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7878084980 con 0x7f7890072000 2024-04-25T10:06:48.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788effd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 0x7f7870069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:48.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.974+0000 7f788effd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 0x7f7870069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f78800af000 tx=0x7f7880008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:48.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:48.978+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7878055800 con 0x7f7890072000 2024-04-25T10:06:49.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.170+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7890165cf0 con 0x7f7890072000 2024-04-25T10:06:49.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.778+0000 7f7887fff640 1 -- 172.21.15.82:0/904394879 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7878055100 con 0x7f7890072000 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 msgr2=0x7f7870069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 0x7f7870069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f78800af000 tx=0x7f7880008040 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 msgr2=0x7f7890098870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f787800c370 tx=0x7f787800c830 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 shutdown_connections 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7870066f60 0x7f7870069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900cf590 0x7f78900c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f789015b6b0 0x7f78901f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 --2- 172.21.15.82:0/904394879 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7890072000 0x7f7890098870 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 >> 172.21.15.82:0/904394879 conn(0x7f789006ae60 msgr2=0x7f78900b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 shutdown_connections 2024-04-25T10:06:49.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.782+0000 7f7895fdd640 1 -- 172.21.15.82:0/904394879 wait complete. 2024-04-25T10:06:49.805 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:49.806 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:49.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 Processor -- start 2024-04-25T10:06:49.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 -- start start 2024-04-25T10:06:49.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa40c144ca0 0x7fa40c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 0x7fa40c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa40c0ccc70 con 0x7fa40c0ce510 2024-04-25T10:06:49.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c0cc2b0 con 0x7fa40c144ca0 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa412e92640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa40c06e290 con 0x7fa40c06ec60 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa40c144ca0 0x7fa40c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa40c144ca0 0x7fa40c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58070/0 (socket says 172.21.15.82:58070) 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 -- 172.21.15.82:0/1782977913 learned_addr learned my addr 172.21.15.82:0/1782977913 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa41142f640 1 --2- 172.21.15.82:0/1782977913 >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 0x7fa40c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 -- 172.21.15.82:0/1782977913 >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 msgr2=0x7fa40c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 --2- 172.21.15.82:0/1782977913 >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 0x7fa40c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 -- 172.21.15.82:0/1782977913 >> v1:172.21.15.196:6789/0 conn(0x7fa40c0ce510 legacy=0x7fa40c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa403fff640 1 -- 172.21.15.82:0/1782977913 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c06dc40 con 0x7fa40c144ca0 2024-04-25T10:06:49.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.918+0000 7fa41142f640 1 --2- 172.21.15.82:0/1782977913 >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 0x7fa40c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:49.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa403fff640 1 --2- 172.21.15.82:0/1782977913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c144ca0 0x7fa40c06e720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b48f0 tx=0x7fa3fc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c763dbcdbc5b0eea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4037fe640 1 -- 172.21.15.82:0/1782977913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0b5070 con 0x7fa40c144ca0 2024-04-25T10:06:49.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4037fe640 1 -- 172.21.15.82:0/1782977913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0a8a10 con 0x7fa40c144ca0 2024-04-25T10:06:49.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4037fe640 1 -- 172.21.15.82:0/1782977913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc0a8de0 con 0x7fa40c144ca0 2024-04-25T10:06:49.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/1782977913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c144ca0 msgr2=0x7fa40c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- 172.21.15.82:0/1782977913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c144ca0 0x7fa40c06e720 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa40c0b48f0 tx=0x7fa3fc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/1782977913 shutdown_connections 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- 172.21.15.82:0/1782977913 >> v2:172.21.15.196:3300/0 conn(0x7fa40c06ec60 0x7fa40c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- 172.21.15.82:0/1782977913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c144ca0 0x7fa40c06e720 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/1782977913 >> 172.21.15.82:0/1782977913 conn(0x7fa40c0b11a0 msgr2=0x7fa40c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/1782977913 shutdown_connections 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/1782977913 wait complete. 2024-04-25T10:06:49.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 Processor -- start 2024-04-25T10:06:49.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- start start 2024-04-25T10:06:49.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa40c0ce510 0x7fa40c1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40c144ca0 0x7fa40c1e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c1451f0 con 0x7fa40c06ec60 2024-04-25T10:06:49.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c0cc900 con 0x7fa40c144ca0 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40c145370 con 0x7fa40c0ce510 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58082/0 (socket says 172.21.15.82:58082) 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 -- 172.21.15.82:0/230498836 learned_addr learned my addr 172.21.15.82:0/230498836 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa403fff640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa40c0ce510 0x7fa40c1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa41142f640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40c144ca0 0x7fa40c1e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 -- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa40c0ce510 msgr2=0x7fa40c1f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa40c0ce510 0x7fa40c1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 -- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40c144ca0 msgr2=0x7fa40c1e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40c144ca0 0x7fa40c1e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 -- 172.21.15.82:0/230498836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa40c0ad7d0 con 0x7fa40c06ec60 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa410c2e640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c420 tx=0x7fa40800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4080052d0 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa408005470 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40c0adab0 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa408007260 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.922+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40c0ae070 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40c04efa0 con 0x7fa40c06ec60 2024-04-25T10:06:49.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa408007400 con 0x7fa40c06ec60 2024-04-25T10:06:49.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa4017fa640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 0x7fa3f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:49.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa408084740 con 0x7fa40c06ec60 2024-04-25T10:06:49.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa403fff640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 0x7fa3f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:49.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.926+0000 7fa403fff640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 0x7fa3f8069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0b2000 tx=0x7fa3fc0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:49.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:49.930+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa4080555c0 con 0x7fa40c06ec60 2024-04-25T10:06:50.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.114+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa40c0b1cc0 con 0x7fa40c06ec60 2024-04-25T10:06:50.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.114+0000 7fa4017fa640 1 -- 172.21.15.82:0/230498836 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fa408054ec0 con 0x7fa40c06ec60 2024-04-25T10:06:50.118 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:50.118 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:06:49.173277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/910515233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":910515233},{"type":"v1","addr":"172.21.15.82:6837","nonce":910515233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:06:50.118 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 41 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 msgr2=0x7fa3f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 0x7fa3f8069420 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc0b2000 tx=0x7fa3fc0aa070 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 msgr2=0x7fa40c0aaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa40800c420 tx=0x7fa40800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 shutdown_connections 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa3f8066f60 0x7fa3f8069420 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa40c0ce510 0x7fa40c1f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa40c144ca0 0x7fa40c1e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 --2- 172.21.15.82:0/230498836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa40c06ec60 0x7fa40c0aaeb0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 >> 172.21.15.82:0/230498836 conn(0x7fa40c0b11a0 msgr2=0x7fa40c0a8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 shutdown_connections 2024-04-25T10:06:50.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.118+0000 7fa412e92640 1 -- 172.21.15.82:0/230498836 wait complete. 2024-04-25T10:06:50.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.246+0000 7f745cfd2640 1 Processor -- start 2024-04-25T10:06:50.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.246+0000 7f745cfd2640 1 -- start start 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745cfd2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f745815a700 0x7f745815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745cfd2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74580b3f50 0x7f74580cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745cfd2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74580b2090 con 0x7f74580ce430 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745cfd2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f745815f200 con 0x7f745815a700 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745cfd2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580b43e0 con 0x7f74580b3f50 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f745815a700 0x7f745815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f745815a700 0x7f745815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58098/0 (socket says 172.21.15.82:58098) 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 -- 172.21.15.82:0/4203794732 learned_addr learned my addr 172.21.15.82:0/4203794732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f7455d9b640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74580b3f50 0x7f74580cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 -- 172.21.15.82:0/4203794732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74580b3f50 msgr2=0x7f74580cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74580b3f50 0x7f74580cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 -- 172.21.15.82:0/4203794732 >> v1:172.21.15.196:6789/0 conn(0x7f74580ce430 legacy=0x7f74580b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 -- 172.21.15.82:0/4203794732 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580b46c0 con 0x7f745815a700 2024-04-25T10:06:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745659c640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f745815ecc0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7448009870 tx=0x7f744801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766c15f38a7bb49a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745559a640 1 -- 172.21.15.82:0/4203794732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480ab610 con 0x7f745815a700 2024-04-25T10:06:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745559a640 1 -- 172.21.15.82:0/4203794732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74480b0070 con 0x7f745815a700 2024-04-25T10:06:50.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.250+0000 7f745559a640 1 -- 172.21.15.82:0/4203794732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74480ab9d0 con 0x7f745815a700 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/4203794732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 msgr2=0x7f745815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f745815ecc0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7448009870 tx=0x7f744801c410 comp rx=0 tx=0).stop 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/4203794732 shutdown_connections 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74580b3f50 0x7f74580cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- 172.21.15.82:0/4203794732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f745815ecc0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/4203794732 >> 172.21.15.82:0/4203794732 conn(0x7f74580b11a0 msgr2=0x7f74580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/4203794732 shutdown_connections 2024-04-25T10:06:50.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/4203794732 wait complete. 2024-04-25T10:06:50.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 Processor -- start 2024-04-25T10:06:50.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- start start 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 0x7f74581508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7458155cb0 0x7f7458156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580cd490 con 0x7f745815a700 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74580cd310 con 0x7f74580ce430 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74580b2bb0 con 0x7f7458155cb0 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58106/0 (socket says 172.21.15.82:58106) 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 -- 172.21.15.82:0/614394270 learned_addr learned my addr 172.21.15.82:0/614394270 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7456d9d640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7458155cb0 0x7f7458156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745659c640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 0x7f74581508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 -- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7458155cb0 msgr2=0x7f7458156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7458155cb0 0x7f7458156140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 -- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 msgr2=0x7f74581508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 0x7f74581508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 -- 172.21.15.82:0/614394270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74580b09a0 con 0x7f745815a700 2024-04-25T10:06:50.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745659c640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 0x7f74581508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f7455d9b640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f744000d620 tx=0x7f744000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000b620 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440019070 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74580a8880 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000b9e0 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.254+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74580a8e40 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745804efa0 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f744000bb80 con 0x7f745815a700 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f744effd640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 0x7f7438069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f745659c640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 0x7f7438069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7440085560 con 0x7f745815a700 2024-04-25T10:06:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.258+0000 7f745659c640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 0x7f7438069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f74480af000 tx=0x7f7448008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.262+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74400563e0 con 0x7f745815a700 2024-04-25T10:06:50.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.422+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f745806f260 con 0x7f745815a700 2024-04-25T10:06:50.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.422+0000 7f744effd640 1 -- 172.21.15.82:0/614394270 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16957 (secure 0 0 0) 0x7f7440055ce0 con 0x7f745815a700 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:50.426 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":79,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:47.754126+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 msgr2=0x7f7438069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 0x7f7438069450 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f74480af000 tx=0x7f7448008040 comp rx=0 tx=0).stop 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 msgr2=0x7f74580a9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f744000d620 tx=0x7f744000dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 shutdown_connections 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7438066f90 0x7f7438069450 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7458155cb0 0x7f7458156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f745815a700 0x7f74580a9920 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 --2- 172.21.15.82:0/614394270 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f74580ce430 0x7f74581508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 >> 172.21.15.82:0/614394270 conn(0x7f74580b11a0 msgr2=0x7f745815dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 shutdown_connections 2024-04-25T10:06:50.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.426+0000 7f745cfd2640 1 -- 172.21.15.82:0/614394270 wait complete. 2024-04-25T10:06:50.444 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:50.445 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:50.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.546+0000 7fe6c0801640 1 Processor -- start 2024-04-25T10:06:50.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.546+0000 7fe6c0801640 1 -- start start 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6c0801640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6bc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6c0801640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe6bc06ec60 0x7fe6bc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6c0801640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0ccc70 con 0x7fe6bc0ce510 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6c0801640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cc2b0 con 0x7fe6bc06ec60 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6c0801640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc06e290 con 0x7fe6bc144ca0 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe6bc0ce510 0x7fe6bc0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35642/0 (socket says 172.21.15.82:35642) 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 -- 172.21.15.82:0/3963271786 learned_addr learned my addr 172.21.15.82:0/3963271786 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6bad9d640 1 --2- 172.21.15.82:0/3963271786 >> v2:172.21.15.196:3300/0 conn(0x7fe6bc06ec60 0x7fe6bc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 543471532 0 0) 0x7fe6bc0ccc70 con 0x7fe6bc0ce510 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b9d9b640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6bc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a4003530 con 0x7fe6bc0ce510 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141377374 0 0) 0x7fe6a4003530 con 0x7fe6bc0ce510 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 msgr2=0x7fe6bc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6bc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 >> v2:172.21.15.196:3300/0 conn(0x7fe6bc06ec60 msgr2=0x7fe6bc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 --2- 172.21.15.82:0/3963271786 >> v2:172.21.15.196:3300/0 conn(0x7fe6bc06ec60 0x7fe6bc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc06dc40 con 0x7fe6bc0ce510 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6bad9d640 1 --2- 172.21.15.82:0/3963271786 >> v2:172.21.15.196:3300/0 conn(0x7fe6bc06ec60 0x7fe6bc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe6ac001680 con 0x7fe6bc0ce510 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 >> v1:172.21.15.82:6789/0 conn(0x7fe6bc0ce510 legacy=0x7fe6bc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4004d50 0x7fe6a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cc2b0 con 0x7fe6a4003a30 2024-04-25T10:06:50.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc06e290 con 0x7fe6a4004d50 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a400bae0 con 0x7fe6bc144ca0 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6b9d9b640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6bad9d640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4004d50 0x7fe6a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 -- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 msgr2=0x7fe6a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc144ca0 0x7fe6a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 -- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4004d50 msgr2=0x7fe6a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6a4004d50 0x7fe6a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.550+0000 7fe6ba59c640 1 -- 172.21.15.82:0/3963271786 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4003530 con 0x7fe6a4003a30 2024-04-25T10:06:50.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001060 tx=0x7fe6ac01f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88dbc904078ebb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac007d80 con 0x7fe6a4003a30 2024-04-25T10:06:50.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac005b00 con 0x7fe6a4003a30 2024-04-25T10:06:50.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6b959a640 1 -- 172.21.15.82:0/3963271786 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac007070 con 0x7fe6a4003a30 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 -- 172.21.15.82:0/3963271786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a4003a30 msgr2=0x7fe6a4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 --2- 172.21.15.82:0/3963271786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6a4003a30 0x7fe6a4003e50 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac001060 tx=0x7fe6ac01f410 comp rx=0 tx=0).stop 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6bad9d640 1 -- 172.21.15.82:0/3963271786 reap_dead start 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 -- 172.21.15.82:0/3963271786 shutdown_connections 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 -- 172.21.15.82:0/3963271786 >> 172.21.15.82:0/3963271786 conn(0x7fe6bc0b11a0 msgr2=0x7fe6bc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 -- 172.21.15.82:0/3963271786 shutdown_connections 2024-04-25T10:06:50.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.554+0000 7fe6c0801640 1 -- 172.21.15.82:0/3963271786 wait complete. 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 Processor -- start 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- start start 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc1e6b20 0x7fe6bc1e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc1e7440 0x7fe6bc1f1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1451f0 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0cc900 con 0x7fe6bc1e7440 2024-04-25T10:06:50.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc145370 con 0x7fe6bc1e6b20 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58120/0 (socket says 172.21.15.82:58120) 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 -- 172.21.15.82:0/1636752348 learned_addr learned my addr 172.21.15.82:0/1636752348 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 -- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc1e6b20 msgr2=0x7fe6bc1e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc1e6b20 0x7fe6bc1e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 -- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc1e7440 msgr2=0x7fe6bc1f1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc1e7440 0x7fe6bc1f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 -- 172.21.15.82:0/1636752348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc1e2530 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6b9d9b640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0011930 tx=0x7fe6b0011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b000fa70 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b000fc10 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc1e2810 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0014440 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc1e2dd0 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc04efa0 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6b0012040 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.558+0000 7fe6aaffd640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 0x7fe69c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.562+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6b0085790 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.562+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 0x7fe69c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.562+0000 7fe6ba59c640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 0x7fe69c069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0ae000 tx=0x7fe6ac000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.566+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6b0019030 con 0x7fe6bc1e1ce0 2024-04-25T10:06:50.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.706+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6bc1e4b50 con 0x7fe69c066f90 2024-04-25T10:06:50.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.722+0000 7fe6aaffd640 1 -- 172.21.15.82:0/1636752348 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237302 (secure 0 0 0) 0x7fe6ac0f8f10 con 0x7fe69c066f90 2024-04-25T10:06:50.727 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:50.729 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 msgr2=0x7fe69c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 0x7fe69c069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0ae000 tx=0x7fe6ac000ff0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 msgr2=0x7fe6bc1e6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0011930 tx=0x7fe6b0011df0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 shutdown_connections 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe69c066f90 0x7fe69c069450 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe6bc1e7440 0x7fe6bc1f1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe6bc1e1ce0 0x7fe6bc1e6f00 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 --2- 172.21.15.82:0/1636752348 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe6bc1e6b20 0x7fe6bc1e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 >> 172.21.15.82:0/1636752348 conn(0x7fe6bc0b11a0 msgr2=0x7fe6bc0c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 shutdown_connections 2024-04-25T10:06:50.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.726+0000 7fe6c0801640 1 -- 172.21.15.82:0/1636752348 wait complete. 2024-04-25T10:06:50.744 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-25T10:06:49.349069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":257804,"kb_used_data":8328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716916,"statfs":{"total":773094113280,"available":772830121984,"internally_reserved":0,"allocated":8527872,"data_stored":4505216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289688"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T10:06:43.052322+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:06:43.052322+0000","last_peered":"2024-04-25T10:06:43.052322+0000","last_clean":"2024-04-25T10:06:43.052322+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:06:43.052322+0000","last_undegraded":"2024-04-25T10:06:43.052322+0000","last_fullsized":"2024-04-25T10:06:43.052322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":1005928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1005928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:50.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:50.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 Processor -- start 2024-04-25T10:06:50.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 -- start start 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f365806ac90 0x7f365806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36580cc130 0x7f36580cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365806b5b0 con 0x7f365815ca90 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f365815ff70 con 0x7f365806ac90 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f365ddfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365815f360 con 0x7f36580cc130 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f3656ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f365815ca90 0x7f36580d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35654/0 (socket says 172.21.15.82:35654) 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f3656ffd640 1 -- 172.21.15.82:0/2039479842 learned_addr learned my addr 172.21.15.82:0/2039479842 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36577fe640 1 --2- 172.21.15.82:0/2039479842 >> v2:172.21.15.82:3300/0 conn(0x7f365806ac90 0x7f365806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f3657fff640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36580cc130 0x7f36580cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36567fc640 1 -- 172.21.15.82:0/2039479842 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540617505 0 0) 0x7f365806b5b0 con 0x7f365815ca90 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36567fc640 1 -- 172.21.15.82:0/2039479842 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3640003530 con 0x7f365815ca90 2024-04-25T10:06:50.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36577fe640 1 -- 172.21.15.82:0/2039479842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36580cc130 msgr2=0x7f36580cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:50.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36577fe640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36580cc130 0x7f36580cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36577fe640 1 -- 172.21.15.82:0/2039479842 >> v1:172.21.15.82:6789/0 conn(0x7f365815ca90 legacy=0x7f36580d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.902+0000 7f36577fe640 1 -- 172.21.15.82:0/2039479842 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365815f640 con 0x7f365806ac90 2024-04-25T10:06:50.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f36577fe640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365806ac90 0x7f365806b070 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f364c0097c0 tx=0x7f364c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc0d1aab10987bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f36567fc640 1 -- 172.21.15.82:0/2039479842 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0a7070 con 0x7f365806ac90 2024-04-25T10:06:50.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f36567fc640 1 -- 172.21.15.82:0/2039479842 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c0b0070 con 0x7f365806ac90 2024-04-25T10:06:50.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f36567fc640 1 -- 172.21.15.82:0/2039479842 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c0ab890 con 0x7f365806ac90 2024-04-25T10:06:50.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- 172.21.15.82:0/2039479842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365806ac90 msgr2=0x7f365806b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365806ac90 0x7f365806b070 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f364c0097c0 tx=0x7f364c01c470 comp rx=0 tx=0).stop 2024-04-25T10:06:50.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- 172.21.15.82:0/2039479842 shutdown_connections 2024-04-25T10:06:50.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36580cc130 0x7f36580cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- 172.21.15.82:0/2039479842 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f365806ac90 0x7f365806b070 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- 172.21.15.82:0/2039479842 >> 172.21.15.82:0/2039479842 conn(0x7f3658099a70 msgr2=0x7f36580d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:50.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- 172.21.15.82:0/2039479842 shutdown_connections 2024-04-25T10:06:50.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- 172.21.15.82:0/2039479842 wait complete. 2024-04-25T10:06:50.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 Processor -- start 2024-04-25T10:06:50.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- start start 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365806ac90 0x7f3658156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f365815ca90 0x7f36581512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3658159750 con 0x7f36580cc130 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36581595d0 con 0x7f365806ac90 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f365ddfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36580d0040 con 0x7f365815ca90 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f3656ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f3656ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58146/0 (socket says 172.21.15.82:58146) 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.906+0000 7f3656ffd640 1 -- 172.21.15.82:0/1794111366 learned_addr learned my addr 172.21.15.82:0/1794111366 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:50.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f36577fe640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365806ac90 0x7f3658156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 -- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f365815ca90 msgr2=0x7f36581512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3657fff640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f365815ca90 0x7f36581512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f365815ca90 0x7f36581512f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 -- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365806ac90 msgr2=0x7f3658156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365806ac90 0x7f3658156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 -- 172.21.15.82:0/1794111366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36580c9f30 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f3656ffd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f364800c450 tx=0x7f364800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648017070 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648005cf0 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36580c93b0 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648007270 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36580c9970 con 0x7f36580cc130 2024-04-25T10:06:50.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36580c4060 con 0x7f36580cc130 2024-04-25T10:06:50.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f364801a070 con 0x7f36580cc130 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 0x7f3644069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f36577fe640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 0x7f3644069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3648084430 con 0x7f36580cc130 2024-04-25T10:06:50.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.910+0000 7f36577fe640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 0x7f3644069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f364c0af000 tx=0x7f364c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:50.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:50.914+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36580c4060 con 0x7f36580cc130 2024-04-25T10:06:51.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.070+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36580b6230 con 0x7f3644066f60 2024-04-25T10:06:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.086+0000 7f363ffff640 1 -- 172.21.15.82:0/1794111366 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237302 (secure 0 0 0) 0x7f36580b6230 con 0x7f3644066f60 2024-04-25T10:06:51.092 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:51.093 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:06:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 msgr2=0x7f3644069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 0x7f3644069420 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f364c0af000 tx=0x7f364c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 msgr2=0x7f36580b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f364800c450 tx=0x7f364800c910 comp rx=0 tx=0).stop 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 shutdown_connections 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3644066f60 0x7f3644069420 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f365815ca90 0x7f36581512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f36580cc130 0x7f36580b52c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 --2- 172.21.15.82:0/1794111366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f365806ac90 0x7f3658156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 >> 172.21.15.82:0/1794111366 conn(0x7f3658099a70 msgr2=0x7f365815d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 shutdown_connections 2024-04-25T10:06:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.090+0000 7f365ddfd640 1 -- 172.21.15.82:0/1794111366 wait complete. 2024-04-25T10:06:51.110 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-04-25T10:06:49.349069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":257804,"kb_used_data":8328,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716916,"statfs":{"total":773094113280,"available":772830121984,"internally_reserved":0,"allocated":8527872,"data_stored":4505216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.289688"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:46.056936+0000","last_change":"2024-04-25T10:06:46.056936+0000","last_active":"2024-04-25T10:06:46.056936+0000","last_peered":"2024-04-25T10:06:46.056936+0000","last_clean":"2024-04-25T10:06:46.056936+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:46.056936+0000","last_undegraded":"2024-04-25T10:06:46.056936+0000","last_fullsized":"2024-04-25T10:06:46.056936+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:45.052320+0000","last_change":"2024-04-25T10:06:45.052320+0000","last_active":"2024-04-25T10:06:45.052320+0000","last_peered":"2024-04-25T10:06:45.052320+0000","last_clean":"2024-04-25T10:06:45.052320+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:45.052320+0000","last_undegraded":"2024-04-25T10:06:45.052320+0000","last_fullsized":"2024-04-25T10:06:45.052320+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":272,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-04-25T10:06:43.052322+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:06:43.052322+0000","last_peered":"2024-04-25T10:06:43.052322+0000","last_clean":"2024-04-25T10:06:43.052322+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:06:43.052322+0000","last_undegraded":"2024-04-25T10:06:43.052322+0000","last_fullsized":"2024-04-25T10:06:43.052322+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574909,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":1005928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32092,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339748,"statfs":{"total":96636764160,"available":96603901952,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574910,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":415560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":1005928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:51.114 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.262+0000 7f402a19b640 1 Processor -- start 2024-04-25T10:06:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.262+0000 7f402a19b640 1 -- start start 2024-04-25T10:06:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f402a19b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4024168920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f402a19b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40240ce6b0 con 0x7f402415d850 2024-04-25T10:06:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f402a19b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40240ce990 con 0x7f40240d0870 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f402a19b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40240cec70 con 0x7f40240d24f0 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f402415d850 0x7f40240d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35658/0 (socket says 172.21.15.82:35658) 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 -- 172.21.15.82:0/2590141350 learned_addr learned my addr 172.21.15.82:0/2590141350 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359721732 0 0) 0x7f40240ce6b0 con 0x7f402415d850 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4010003530 con 0x7f402415d850 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f4023fff640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4024168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1216828700 0 0) 0x7f4010003530 con 0x7f402415d850 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 msgr2=0x7f4024168920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4024168920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 >> v1:172.21.15.196:6789/0 conn(0x7f40240d0870 legacy=0x7f40240d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240cef50 con 0x7f402415d850 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f401c0016a0 con 0x7f402415d850 2024-04-25T10:06:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 >> v1:172.21.15.82:6789/0 conn(0x7f402415d850 legacy=0x7f40240d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4010003a60 0x7f4010003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4010004810 0x7f4010004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4010005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40240cec70 con 0x7f4010003a60 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401000bb90 con 0x7f4010004810 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401000bd90 con 0x7f40240d24f0 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4010003a60 0x7f4010003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f4022ffd640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4010004810 0x7f4010004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f4023fff640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4010005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 -- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 msgr2=0x7f4010005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40240d24f0 0x7f4010005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 -- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4010004810 msgr2=0x7f4010004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4010004810 0x7f4010004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.266+0000 7f40237fe640 1 -- 172.21.15.82:0/2590141350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4010003530 con 0x7f4010003a60 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f40237fe640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4010003a60 0x7f4010003e80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c290 tx=0x7f401c01cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6140e5bc528b716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c007d60 con 0x7f4010003a60 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c0a9070 con 0x7f4010003a60 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f40227fc640 1 -- 172.21.15.82:0/2590141350 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c005d40 con 0x7f4010003a60 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/2590141350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4010003a60 msgr2=0x7f4010003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 --2- 172.21.15.82:0/2590141350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4010003a60 0x7f4010003e80 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f401c00c290 tx=0x7f401c01cd00 comp rx=0 tx=0).stop 2024-04-25T10:06:51.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4023fff640 1 -- 172.21.15.82:0/2590141350 reap_dead start 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/2590141350 shutdown_connections 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/2590141350 >> 172.21.15.82:0/2590141350 conn(0x7f402408b000 msgr2=0x7f40240d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/2590141350 shutdown_connections 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/2590141350 wait complete. 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 Processor -- start 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- start start 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40241f9b60 0x7f40240b89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 0x7f402415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40241694c0 con 0x7f40240b8f30 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4024169340 con 0x7f40240b6490 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4024169640 con 0x7f40241f9b60 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58172/0 (socket says 172.21.15.82:58172) 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 -- 172.21.15.82:0/590573126 learned_addr learned my addr 172.21.15.82:0/590573126 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f40237fe640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40241f9b60 0x7f40240b89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4023fff640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 0x7f402415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 -- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40241f9b60 msgr2=0x7f40240b89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40241f9b60 0x7f40240b89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 -- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 msgr2=0x7f402415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 0x7f402415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 -- 172.21.15.82:0/590573126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240cc960 con 0x7f40240b8f30 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4023fff640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 0x7f402415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f4022ffd640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e4d0 tx=0x7f400c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0053a0 con 0x7f40240b8f30 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c018070 con 0x7f40240b8f30 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40240ccc00 con 0x7f40240b8f30 2024-04-25T10:06:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c003820 con 0x7f40240b8f30 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.270+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40240cd1c0 con 0x7f40240b8f30 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402404efa0 con 0x7f40240b8f30 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f400c0039c0 con 0x7f40240b8f30 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f401bfff640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 0x7f4004069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f400c084eb0 con 0x7f40240b8f30 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f40237fe640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 0x7f4004069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.274+0000 7f40237fe640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 0x7f4004069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f401c0ab000 tx=0x7f401c0b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.278+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f400c055d30 con 0x7f40240b8f30 2024-04-25T10:06:51.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.434+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40240d0e60 con 0x7f4004066f60 2024-04-25T10:06:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.450+0000 7f401bfff640 1 -- 172.21.15.82:0/590573126 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252072 (secure 0 0 0) 0x7f40240d0e60 con 0x7f4004066f60 2024-04-25T10:06:51.456 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:51.458 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 msgr2=0x7f4004069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 0x7f4004069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f401c0ab000 tx=0x7f401c0b0040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 msgr2=0x7f402415c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f400c00e4d0 tx=0x7f400c00e990 comp rx=0 tx=0).stop 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 shutdown_connections 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4004066f60 0x7f4004069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40240b6490 0x7f402415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40240b8f30 0x7f402415c850 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 --2- 172.21.15.82:0/590573126 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40241f9b60 0x7f40240b89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 >> 172.21.15.82:0/590573126 conn(0x7f402408b000 msgr2=0x7f402415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 shutdown_connections 2024-04-25T10:06:51.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.454+0000 7f402a19b640 1 -- 172.21.15.82:0/590573126 wait complete. 2024-04-25T10:06:51.475 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-25T10:06:51.349575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258236,"kb_used_data":8776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716484,"statfs":{"total":773094113280,"available":772829679616,"internally_reserved":0,"allocated":8986624,"data_stored":4723916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579140+0000","last_change":"2024-04-25T10:06:46.812042+0000","last_active":"2024-04-25T10:06:48.579140+0000","last_peered":"2024-04-25T10:06:48.579140+0000","last_clean":"2024-04-25T10:06:48.579140+0000","last_became_active":"2024-04-25T10:06:46.811330+0000","last_became_peered":"2024-04-25T10:06:46.811330+0000","last_unstale":"2024-04-25T10:06:48.579140+0000","last_undegraded":"2024-04-25T10:06:48.579140+0000","last_fullsized":"2024-04-25T10:06:48.579140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078792+0000","last_change":"2024-04-25T10:06:46.814055+0000","last_active":"2024-04-25T10:06:48.078792+0000","last_peered":"2024-04-25T10:06:48.078792+0000","last_clean":"2024-04-25T10:06:48.078792+0000","last_became_active":"2024-04-25T10:06:46.813374+0000","last_became_peered":"2024-04-25T10:06:46.813374+0000","last_unstale":"2024-04-25T10:06:48.078792+0000","last_undegraded":"2024-04-25T10:06:48.078792+0000","last_fullsized":"2024-04-25T10:06:48.078792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.433303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766684+0000","last_change":"2024-04-25T10:06:46.816949+0000","last_active":"2024-04-25T10:06:47.766684+0000","last_peered":"2024-04-25T10:06:47.766684+0000","last_clean":"2024-04-25T10:06:47.766684+0000","last_became_active":"2024-04-25T10:06:46.816384+0000","last_became_peered":"2024-04-25T10:06:46.816384+0000","last_unstale":"2024-04-25T10:06:47.766684+0000","last_undegraded":"2024-04-25T10:06:47.766684+0000","last_fullsized":"2024-04-25T10:06:47.766684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:41.271967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578047+0000","last_change":"2024-04-25T10:06:46.782326+0000","last_active":"2024-04-25T10:06:48.578047+0000","last_peered":"2024-04-25T10:06:48.578047+0000","last_clean":"2024-04-25T10:06:48.578047+0000","last_became_active":"2024-04-25T10:06:46.780178+0000","last_became_peered":"2024-04-25T10:06:46.780178+0000","last_unstale":"2024-04-25T10:06:48.578047+0000","last_undegraded":"2024-04-25T10:06:48.578047+0000","last_fullsized":"2024-04-25T10:06:48.578047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:11.581560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082074+0000","last_change":"2024-04-25T10:06:46.786463+0000","last_active":"2024-04-25T10:06:48.082074+0000","last_peered":"2024-04-25T10:06:48.082074+0000","last_clean":"2024-04-25T10:06:48.082074+0000","last_became_active":"2024-04-25T10:06:46.785559+0000","last_became_peered":"2024-04-25T10:06:46.785559+0000","last_unstale":"2024-04-25T10:06:48.082074+0000","last_undegraded":"2024-04-25T10:06:48.082074+0000","last_fullsized":"2024-04-25T10:06:48.082074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:01.174907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579158+0000","last_change":"2024-04-25T10:06:46.816049+0000","last_active":"2024-04-25T10:06:48.579158+0000","last_peered":"2024-04-25T10:06:48.579158+0000","last_clean":"2024-04-25T10:06:48.579158+0000","last_became_active":"2024-04-25T10:06:46.814728+0000","last_became_peered":"2024-04-25T10:06:46.814728+0000","last_unstale":"2024-04-25T10:06:48.579158+0000","last_undegraded":"2024-04-25T10:06:48.579158+0000","last_fullsized":"2024-04-25T10:06:48.579158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765258+0000","last_change":"2024-04-25T10:06:46.787007+0000","last_active":"2024-04-25T10:06:47.765258+0000","last_peered":"2024-04-25T10:06:47.765258+0000","last_clean":"2024-04-25T10:06:47.765258+0000","last_became_active":"2024-04-25T10:06:46.786255+0000","last_became_peered":"2024-04-25T10:06:46.786255+0000","last_unstale":"2024-04-25T10:06:47.765258+0000","last_undegraded":"2024-04-25T10:06:47.765258+0000","last_fullsized":"2024-04-25T10:06:47.765258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082256+0000","last_change":"2024-04-25T10:06:46.785378+0000","last_active":"2024-04-25T10:06:48.082256+0000","last_peered":"2024-04-25T10:06:48.082256+0000","last_clean":"2024-04-25T10:06:48.082256+0000","last_became_active":"2024-04-25T10:06:46.784540+0000","last_became_peered":"2024-04-25T10:06:46.784540+0000","last_unstale":"2024-04-25T10:06:48.082256+0000","last_undegraded":"2024-04-25T10:06:48.082256+0000","last_fullsized":"2024-04-25T10:06:48.082256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:56.712512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767669+0000","last_change":"2024-04-25T10:06:46.781725+0000","last_active":"2024-04-25T10:06:47.767669+0000","last_peered":"2024-04-25T10:06:47.767669+0000","last_clean":"2024-04-25T10:06:47.767669+0000","last_became_active":"2024-04-25T10:06:46.781039+0000","last_became_peered":"2024-04-25T10:06:46.781039+0000","last_unstale":"2024-04-25T10:06:47.767669+0000","last_undegraded":"2024-04-25T10:06:47.767669+0000","last_fullsized":"2024-04-25T10:06:47.767669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:11.102231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577706+0000","last_change":"2024-04-25T10:06:46.795742+0000","last_active":"2024-04-25T10:06:48.577706+0000","last_peered":"2024-04-25T10:06:48.577706+0000","last_clean":"2024-04-25T10:06:48.577706+0000","last_became_active":"2024-04-25T10:06:46.794090+0000","last_became_peered":"2024-04-25T10:06:46.794090+0000","last_unstale":"2024-04-25T10:06:48.577706+0000","last_undegraded":"2024-04-25T10:06:48.577706+0000","last_fullsized":"2024-04-25T10:06:48.577706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:12.588290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078775+0000","last_change":"2024-04-25T10:06:46.813315+0000","last_active":"2024-04-25T10:06:48.078775+0000","last_peered":"2024-04-25T10:06:48.078775+0000","last_clean":"2024-04-25T10:06:48.078775+0000","last_became_active":"2024-04-25T10:06:46.800516+0000","last_became_peered":"2024-04-25T10:06:46.800516+0000","last_unstale":"2024-04-25T10:06:48.078775+0000","last_undegraded":"2024-04-25T10:06:48.078775+0000","last_fullsized":"2024-04-25T10:06:48.078775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:05.005574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578300+0000","last_change":"2024-04-25T10:06:46.813321+0000","last_active":"2024-04-25T10:06:48.578300+0000","last_peered":"2024-04-25T10:06:48.578300+0000","last_clean":"2024-04-25T10:06:48.578300+0000","last_became_active":"2024-04-25T10:06:46.812941+0000","last_became_peered":"2024-04-25T10:06:46.812941+0000","last_unstale":"2024-04-25T10:06:48.578300+0000","last_undegraded":"2024-04-25T10:06:48.578300+0000","last_fullsized":"2024-04-25T10:06:48.578300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:34.026951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082448+0000","last_change":"2024-04-25T10:06:46.776909+0000","last_active":"2024-04-25T10:06:48.082448+0000","last_peered":"2024-04-25T10:06:48.082448+0000","last_clean":"2024-04-25T10:06:48.082448+0000","last_became_active":"2024-04-25T10:06:46.776153+0000","last_became_peered":"2024-04-25T10:06:46.776153+0000","last_unstale":"2024-04-25T10:06:48.082448+0000","last_undegraded":"2024-04-25T10:06:48.082448+0000","last_fullsized":"2024-04-25T10:06:48.082448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:40.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578644+0000","last_change":"2024-04-25T10:06:46.805290+0000","last_active":"2024-04-25T10:06:48.578644+0000","last_peered":"2024-04-25T10:06:48.578644+0000","last_clean":"2024-04-25T10:06:48.578644+0000","last_became_active":"2024-04-25T10:06:46.799720+0000","last_became_peered":"2024-04-25T10:06:46.799720+0000","last_unstale":"2024-04-25T10:06:48.578644+0000","last_undegraded":"2024-04-25T10:06:48.578644+0000","last_fullsized":"2024-04-25T10:06:48.578644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078045+0000","last_change":"2024-04-25T10:06:46.807971+0000","last_active":"2024-04-25T10:06:48.078045+0000","last_peered":"2024-04-25T10:06:48.078045+0000","last_clean":"2024-04-25T10:06:48.078045+0000","last_became_active":"2024-04-25T10:06:46.800983+0000","last_became_peered":"2024-04-25T10:06:46.800983+0000","last_unstale":"2024-04-25T10:06:48.078045+0000","last_undegraded":"2024-04-25T10:06:48.078045+0000","last_fullsized":"2024-04-25T10:06:48.078045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578470+0000","last_change":"2024-04-25T10:06:46.790688+0000","last_active":"2024-04-25T10:06:48.578470+0000","last_peered":"2024-04-25T10:06:48.578470+0000","last_clean":"2024-04-25T10:06:48.578470+0000","last_became_active":"2024-04-25T10:06:46.789841+0000","last_became_peered":"2024-04-25T10:06:46.789841+0000","last_unstale":"2024-04-25T10:06:48.578470+0000","last_undegraded":"2024-04-25T10:06:48.578470+0000","last_fullsized":"2024-04-25T10:06:48.578470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.083022+0000","last_change":"2024-04-25T10:06:46.803641+0000","last_active":"2024-04-25T10:06:48.083022+0000","last_peered":"2024-04-25T10:06:48.083022+0000","last_clean":"2024-04-25T10:06:48.083022+0000","last_became_active":"2024-04-25T10:06:46.802850+0000","last_became_peered":"2024-04-25T10:06:46.802850+0000","last_unstale":"2024-04-25T10:06:48.083022+0000","last_undegraded":"2024-04-25T10:06:48.083022+0000","last_fullsized":"2024-04-25T10:06:48.083022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:45.971536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766827+0000","last_change":"2024-04-25T10:06:46.782609+0000","last_active":"2024-04-25T10:06:47.766827+0000","last_peered":"2024-04-25T10:06:47.766827+0000","last_clean":"2024-04-25T10:06:47.766827+0000","last_became_active":"2024-04-25T10:06:46.781574+0000","last_became_peered":"2024-04-25T10:06:46.781574+0000","last_unstale":"2024-04-25T10:06:47.766827+0000","last_undegraded":"2024-04-25T10:06:47.766827+0000","last_fullsized":"2024-04-25T10:06:47.766827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579031+0000","last_change":"2024-04-25T10:06:46.788869+0000","last_active":"2024-04-25T10:06:48.579031+0000","last_peered":"2024-04-25T10:06:48.579031+0000","last_clean":"2024-04-25T10:06:48.579031+0000","last_became_active":"2024-04-25T10:06:46.788132+0000","last_became_peered":"2024-04-25T10:06:46.788132+0000","last_unstale":"2024-04-25T10:06:48.579031+0000","last_undegraded":"2024-04-25T10:06:48.579031+0000","last_fullsized":"2024-04-25T10:06:48.579031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082318+0000","last_change":"2024-04-25T10:06:46.785801+0000","last_active":"2024-04-25T10:06:48.082318+0000","last_peered":"2024-04-25T10:06:48.082318+0000","last_clean":"2024-04-25T10:06:48.082318+0000","last_became_active":"2024-04-25T10:06:46.784579+0000","last_became_peered":"2024-04-25T10:06:46.784579+0000","last_unstale":"2024-04-25T10:06:48.082318+0000","last_undegraded":"2024-04-25T10:06:48.082318+0000","last_fullsized":"2024-04-25T10:06:48.082318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:46.516701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767026+0000","last_change":"2024-04-25T10:06:46.803669+0000","last_active":"2024-04-25T10:06:47.767026+0000","last_peered":"2024-04-25T10:06:47.767026+0000","last_clean":"2024-04-25T10:06:47.767026+0000","last_became_active":"2024-04-25T10:06:46.802373+0000","last_became_peered":"2024-04-25T10:06:46.802373+0000","last_unstale":"2024-04-25T10:06:47.767026+0000","last_undegraded":"2024-04-25T10:06:47.767026+0000","last_fullsized":"2024-04-25T10:06:47.767026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:58.982166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.797809+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.797133+0000","last_became_peered":"2024-04-25T10:06:46.797133+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:27.886972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764762+0000","last_change":"2024-04-25T10:06:46.783667+0000","last_active":"2024-04-25T10:06:47.764762+0000","last_peered":"2024-04-25T10:06:47.764762+0000","last_clean":"2024-04-25T10:06:47.764762+0000","last_became_active":"2024-04-25T10:06:46.782958+0000","last_became_peered":"2024-04-25T10:06:46.782958+0000","last_unstale":"2024-04-25T10:06:47.764762+0000","last_undegraded":"2024-04-25T10:06:47.764762+0000","last_fullsized":"2024-04-25T10:06:47.764762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:45.364008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078138+0000","last_change":"2024-04-25T10:06:46.807236+0000","last_active":"2024-04-25T10:06:48.078138+0000","last_peered":"2024-04-25T10:06:48.078138+0000","last_clean":"2024-04-25T10:06:48.078138+0000","last_became_active":"2024-04-25T10:06:46.800093+0000","last_became_peered":"2024-04-25T10:06:46.800093+0000","last_unstale":"2024-04-25T10:06:48.078138+0000","last_undegraded":"2024-04-25T10:06:48.078138+0000","last_fullsized":"2024-04-25T10:06:48.078138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078877+0000","last_change":"2024-04-25T10:06:46.806479+0000","last_active":"2024-04-25T10:06:48.078877+0000","last_peered":"2024-04-25T10:06:48.078877+0000","last_clean":"2024-04-25T10:06:48.078877+0000","last_became_active":"2024-04-25T10:06:46.800600+0000","last_became_peered":"2024-04-25T10:06:46.800600+0000","last_unstale":"2024-04-25T10:06:48.078877+0000","last_undegraded":"2024-04-25T10:06:48.078877+0000","last_fullsized":"2024-04-25T10:06:48.078877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:06.913908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577901+0000","last_change":"2024-04-25T10:06:46.813601+0000","last_active":"2024-04-25T10:06:48.577901+0000","last_peered":"2024-04-25T10:06:48.577901+0000","last_clean":"2024-04-25T10:06:48.577901+0000","last_became_active":"2024-04-25T10:06:46.812901+0000","last_became_peered":"2024-04-25T10:06:46.812901+0000","last_unstale":"2024-04-25T10:06:48.577901+0000","last_undegraded":"2024-04-25T10:06:48.577901+0000","last_fullsized":"2024-04-25T10:06:48.577901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:15.598255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578852+0000","last_change":"2024-04-25T10:06:46.804827+0000","last_active":"2024-04-25T10:06:48.578852+0000","last_peered":"2024-04-25T10:06:48.578852+0000","last_clean":"2024-04-25T10:06:48.578852+0000","last_became_active":"2024-04-25T10:06:46.804083+0000","last_became_peered":"2024-04-25T10:06:46.804083+0000","last_unstale":"2024-04-25T10:06:48.578852+0000","last_undegraded":"2024-04-25T10:06:48.578852+0000","last_fullsized":"2024-04-25T10:06:48.578852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082404+0000","last_change":"2024-04-25T10:06:46.789728+0000","last_active":"2024-04-25T10:06:48.082404+0000","last_peered":"2024-04-25T10:06:48.082404+0000","last_clean":"2024-04-25T10:06:48.082404+0000","last_became_active":"2024-04-25T10:06:46.788957+0000","last_became_peered":"2024-04-25T10:06:46.788957+0000","last_unstale":"2024-04-25T10:06:48.082404+0000","last_undegraded":"2024-04-25T10:06:48.082404+0000","last_fullsized":"2024-04-25T10:06:48.082404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.998327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078446+0000","last_change":"2024-04-25T10:06:46.808484+0000","last_active":"2024-04-25T10:06:48.078446+0000","last_peered":"2024-04-25T10:06:48.078446+0000","last_clean":"2024-04-25T10:06:48.078446+0000","last_became_active":"2024-04-25T10:06:46.807681+0000","last_became_peered":"2024-04-25T10:06:46.807681+0000","last_unstale":"2024-04-25T10:06:48.078446+0000","last_undegraded":"2024-04-25T10:06:48.078446+0000","last_fullsized":"2024-04-25T10:06:48.078446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578440+0000","last_change":"2024-04-25T10:06:46.806504+0000","last_active":"2024-04-25T10:06:48.578440+0000","last_peered":"2024-04-25T10:06:48.578440+0000","last_clean":"2024-04-25T10:06:48.578440+0000","last_became_active":"2024-04-25T10:06:46.805910+0000","last_became_peered":"2024-04-25T10:06:46.805910+0000","last_unstale":"2024-04-25T10:06:48.578440+0000","last_undegraded":"2024-04-25T10:06:48.578440+0000","last_fullsized":"2024-04-25T10:06:48.578440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766290+0000","last_change":"2024-04-25T10:06:46.795871+0000","last_active":"2024-04-25T10:06:47.766290+0000","last_peered":"2024-04-25T10:06:47.766290+0000","last_clean":"2024-04-25T10:06:47.766290+0000","last_became_active":"2024-04-25T10:06:46.794978+0000","last_became_peered":"2024-04-25T10:06:46.794978+0000","last_unstale":"2024-04-25T10:06:47.766290+0000","last_undegraded":"2024-04-25T10:06:47.766290+0000","last_fullsized":"2024-04-25T10:06:47.766290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:20.023879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578885+0000","last_change":"2024-04-25T10:06:46.804579+0000","last_active":"2024-04-25T10:06:48.578885+0000","last_peered":"2024-04-25T10:06:48.578885+0000","last_clean":"2024-04-25T10:06:48.578885+0000","last_became_active":"2024-04-25T10:06:46.803668+0000","last_became_peered":"2024-04-25T10:06:46.803668+0000","last_unstale":"2024-04-25T10:06:48.578885+0000","last_undegraded":"2024-04-25T10:06:48.578885+0000","last_fullsized":"2024-04-25T10:06:48.578885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766189+0000","last_change":"2024-04-25T10:06:46.781678+0000","last_active":"2024-04-25T10:06:47.766189+0000","last_peered":"2024-04-25T10:06:47.766189+0000","last_clean":"2024-04-25T10:06:47.766189+0000","last_became_active":"2024-04-25T10:06:46.780704+0000","last_became_peered":"2024-04-25T10:06:46.780704+0000","last_unstale":"2024-04-25T10:06:47.766189+0000","last_undegraded":"2024-04-25T10:06:47.766189+0000","last_fullsized":"2024-04-25T10:06:47.766189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082982+0000","last_change":"2024-04-25T10:06:46.089223+0000","last_active":"2024-04-25T10:06:48.082982+0000","last_peered":"2024-04-25T10:06:48.082982+0000","last_clean":"2024-04-25T10:06:48.082982+0000","last_became_active":"2024-04-25T10:06:46.088197+0000","last_became_peered":"2024-04-25T10:06:46.088197+0000","last_unstale":"2024-04-25T10:06:48.082982+0000","last_undegraded":"2024-04-25T10:06:48.082982+0000","last_fullsized":"2024-04-25T10:06:48.082982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.519080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078305+0000","last_change":"2024-04-25T10:06:46.807437+0000","last_active":"2024-04-25T10:06:48.078305+0000","last_peered":"2024-04-25T10:06:48.078305+0000","last_clean":"2024-04-25T10:06:48.078305+0000","last_became_active":"2024-04-25T10:06:46.806601+0000","last_became_peered":"2024-04-25T10:06:46.806601+0000","last_unstale":"2024-04-25T10:06:48.078305+0000","last_undegraded":"2024-04-25T10:06:48.078305+0000","last_fullsized":"2024-04-25T10:06:48.078305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578947+0000","last_change":"2024-04-25T10:06:46.108323+0000","last_active":"2024-04-25T10:06:48.578947+0000","last_peered":"2024-04-25T10:06:48.578947+0000","last_clean":"2024-04-25T10:06:48.578947+0000","last_became_active":"2024-04-25T10:06:46.107822+0000","last_became_peered":"2024-04-25T10:06:46.107822+0000","last_unstale":"2024-04-25T10:06:48.578947+0000","last_undegraded":"2024-04-25T10:06:48.578947+0000","last_fullsized":"2024-04-25T10:06:48.578947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766972+0000","last_change":"2024-04-25T10:06:46.803343+0000","last_active":"2024-04-25T10:06:47.766972+0000","last_peered":"2024-04-25T10:06:47.766972+0000","last_clean":"2024-04-25T10:06:47.766972+0000","last_became_active":"2024-04-25T10:06:46.798722+0000","last_became_peered":"2024-04-25T10:06:46.798722+0000","last_unstale":"2024-04-25T10:06:47.766972+0000","last_undegraded":"2024-04-25T10:06:47.766972+0000","last_fullsized":"2024-04-25T10:06:47.766972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.202660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578371+0000","last_change":"2024-04-25T10:06:46.113290+0000","last_active":"2024-04-25T10:06:48.578371+0000","last_peered":"2024-04-25T10:06:48.578371+0000","last_clean":"2024-04-25T10:06:48.578371+0000","last_became_active":"2024-04-25T10:06:46.112709+0000","last_became_peered":"2024-04-25T10:06:46.112709+0000","last_unstale":"2024-04-25T10:06:48.578371+0000","last_undegraded":"2024-04-25T10:06:48.578371+0000","last_fullsized":"2024-04-25T10:06:48.578371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767427+0000","last_change":"2024-04-25T10:06:46.802403+0000","last_active":"2024-04-25T10:06:47.767427+0000","last_peered":"2024-04-25T10:06:47.767427+0000","last_clean":"2024-04-25T10:06:47.767427+0000","last_became_active":"2024-04-25T10:06:46.793094+0000","last_became_peered":"2024-04-25T10:06:46.793094+0000","last_unstale":"2024-04-25T10:06:47.767427+0000","last_undegraded":"2024-04-25T10:06:47.767427+0000","last_fullsized":"2024-04-25T10:06:47.767427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:09.368323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765745+0000","last_change":"2024-04-25T10:06:46.106011+0000","last_active":"2024-04-25T10:06:47.765745+0000","last_peered":"2024-04-25T10:06:47.765745+0000","last_clean":"2024-04-25T10:06:47.765745+0000","last_became_active":"2024-04-25T10:06:46.105343+0000","last_became_peered":"2024-04-25T10:06:46.105343+0000","last_unstale":"2024-04-25T10:06:47.765745+0000","last_undegraded":"2024-04-25T10:06:47.765745+0000","last_fullsized":"2024-04-25T10:06:47.765745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078833+0000","last_change":"2024-04-25T10:06:46.815307+0000","last_active":"2024-04-25T10:06:48.078833+0000","last_peered":"2024-04-25T10:06:48.078833+0000","last_clean":"2024-04-25T10:06:48.078833+0000","last_became_active":"2024-04-25T10:06:46.814081+0000","last_became_peered":"2024-04-25T10:06:46.814081+0000","last_unstale":"2024-04-25T10:06:48.078833+0000","last_undegraded":"2024-04-25T10:06:48.078833+0000","last_fullsized":"2024-04-25T10:06:48.078833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:17.800835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767873+0000","last_change":"2024-04-25T10:06:46.110424+0000","last_active":"2024-04-25T10:06:47.767873+0000","last_peered":"2024-04-25T10:06:47.767873+0000","last_clean":"2024-04-25T10:06:47.767873+0000","last_became_active":"2024-04-25T10:06:46.109677+0000","last_became_peered":"2024-04-25T10:06:46.109677+0000","last_unstale":"2024-04-25T10:06:47.767873+0000","last_undegraded":"2024-04-25T10:06:47.767873+0000","last_fullsized":"2024-04-25T10:06:47.767873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.659604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078976+0000","last_change":"2024-04-25T10:06:46.807062+0000","last_active":"2024-04-25T10:06:48.078976+0000","last_peered":"2024-04-25T10:06:48.078976+0000","last_clean":"2024-04-25T10:06:48.078976+0000","last_became_active":"2024-04-25T10:06:46.800704+0000","last_became_peered":"2024-04-25T10:06:46.800704+0000","last_unstale":"2024-04-25T10:06:48.078976+0000","last_undegraded":"2024-04-25T10:06:48.078976+0000","last_fullsized":"2024-04-25T10:06:48.078976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:12.930467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577283+0000","last_change":"2024-04-25T10:06:46.088468+0000","last_active":"2024-04-25T10:06:48.577283+0000","last_peered":"2024-04-25T10:06:48.577283+0000","last_clean":"2024-04-25T10:06:48.577283+0000","last_became_active":"2024-04-25T10:06:46.087685+0000","last_became_peered":"2024-04-25T10:06:46.087685+0000","last_unstale":"2024-04-25T10:06:48.577283+0000","last_undegraded":"2024-04-25T10:06:48.577283+0000","last_fullsized":"2024-04-25T10:06:48.577283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767923+0000","last_change":"2024-04-25T10:06:46.774632+0000","last_active":"2024-04-25T10:06:47.767923+0000","last_peered":"2024-04-25T10:06:47.767923+0000","last_clean":"2024-04-25T10:06:47.767923+0000","last_became_active":"2024-04-25T10:06:46.773829+0000","last_became_peered":"2024-04-25T10:06:46.773829+0000","last_unstale":"2024-04-25T10:06:47.767923+0000","last_undegraded":"2024-04-25T10:06:47.767923+0000","last_fullsized":"2024-04-25T10:06:47.767923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078146+0000","last_change":"2024-04-25T10:06:46.103414+0000","last_active":"2024-04-25T10:06:48.078146+0000","last_peered":"2024-04-25T10:06:48.078146+0000","last_clean":"2024-04-25T10:06:48.078146+0000","last_became_active":"2024-04-25T10:06:46.103164+0000","last_became_peered":"2024-04-25T10:06:46.103164+0000","last_unstale":"2024-04-25T10:06:48.078146+0000","last_undegraded":"2024-04-25T10:06:48.078146+0000","last_fullsized":"2024-04-25T10:06:48.078146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765294+0000","last_change":"2024-04-25T10:06:46.794249+0000","last_active":"2024-04-25T10:06:47.765294+0000","last_peered":"2024-04-25T10:06:47.765294+0000","last_clean":"2024-04-25T10:06:47.765294+0000","last_became_active":"2024-04-25T10:06:46.792753+0000","last_became_peered":"2024-04-25T10:06:46.792753+0000","last_unstale":"2024-04-25T10:06:47.765294+0000","last_undegraded":"2024-04-25T10:06:47.765294+0000","last_fullsized":"2024-04-25T10:06:47.765294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081812+0000","last_change":"2024-04-25T10:06:46.102417+0000","last_active":"2024-04-25T10:06:48.081812+0000","last_peered":"2024-04-25T10:06:48.081812+0000","last_clean":"2024-04-25T10:06:48.081812+0000","last_became_active":"2024-04-25T10:06:46.101880+0000","last_became_peered":"2024-04-25T10:06:46.101880+0000","last_unstale":"2024-04-25T10:06:48.081812+0000","last_undegraded":"2024-04-25T10:06:48.081812+0000","last_fullsized":"2024-04-25T10:06:48.081812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:27.992158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077906+0000","last_change":"2024-04-25T10:06:46.800819+0000","last_active":"2024-04-25T10:06:48.077906+0000","last_peered":"2024-04-25T10:06:48.077906+0000","last_clean":"2024-04-25T10:06:48.077906+0000","last_became_active":"2024-04-25T10:06:46.799180+0000","last_became_peered":"2024-04-25T10:06:46.799180+0000","last_unstale":"2024-04-25T10:06:48.077906+0000","last_undegraded":"2024-04-25T10:06:48.077906+0000","last_fullsized":"2024-04-25T10:06:48.077906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579230+0000","last_change":"2024-04-25T10:06:46.108693+0000","last_active":"2024-04-25T10:06:48.579230+0000","last_peered":"2024-04-25T10:06:48.579230+0000","last_clean":"2024-04-25T10:06:48.579230+0000","last_became_active":"2024-04-25T10:06:46.108114+0000","last_became_peered":"2024-04-25T10:06:46.108114+0000","last_unstale":"2024-04-25T10:06:48.579230+0000","last_undegraded":"2024-04-25T10:06:48.579230+0000","last_fullsized":"2024-04-25T10:06:48.579230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:05.951567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577767+0000","last_change":"2024-04-25T10:06:46.788689+0000","last_active":"2024-04-25T10:06:48.577767+0000","last_peered":"2024-04-25T10:06:48.577767+0000","last_clean":"2024-04-25T10:06:48.577767+0000","last_became_active":"2024-04-25T10:06:46.786763+0000","last_became_peered":"2024-04-25T10:06:46.786763+0000","last_unstale":"2024-04-25T10:06:48.577767+0000","last_undegraded":"2024-04-25T10:06:48.577767+0000","last_fullsized":"2024-04-25T10:06:48.577767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:38.545499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578022+0000","last_change":"2024-04-25T10:06:46.103056+0000","last_active":"2024-04-25T10:06:48.578022+0000","last_peered":"2024-04-25T10:06:48.578022+0000","last_clean":"2024-04-25T10:06:48.578022+0000","last_became_active":"2024-04-25T10:06:46.102214+0000","last_became_peered":"2024-04-25T10:06:46.102214+0000","last_unstale":"2024-04-25T10:06:48.578022+0000","last_undegraded":"2024-04-25T10:06:48.578022+0000","last_fullsized":"2024-04-25T10:06:48.578022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579421+0000","last_change":"2024-04-25T10:06:46.795073+0000","last_active":"2024-04-25T10:06:48.579421+0000","last_peered":"2024-04-25T10:06:48.579421+0000","last_clean":"2024-04-25T10:06:48.579421+0000","last_became_active":"2024-04-25T10:06:46.793560+0000","last_became_peered":"2024-04-25T10:06:46.793560+0000","last_unstale":"2024-04-25T10:06:48.579421+0000","last_undegraded":"2024-04-25T10:06:48.579421+0000","last_fullsized":"2024-04-25T10:06:48.579421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082711+0000","last_change":"2024-04-25T10:06:46.085445+0000","last_active":"2024-04-25T10:06:48.082711+0000","last_peered":"2024-04-25T10:06:48.082711+0000","last_clean":"2024-04-25T10:06:48.082711+0000","last_became_active":"2024-04-25T10:06:46.085042+0000","last_became_peered":"2024-04-25T10:06:46.085042+0000","last_unstale":"2024-04-25T10:06:48.082711+0000","last_undegraded":"2024-04-25T10:06:48.082711+0000","last_fullsized":"2024-04-25T10:06:48.082711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:22.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766428+0000","last_change":"2024-04-25T10:06:46.814843+0000","last_active":"2024-04-25T10:06:47.766428+0000","last_peered":"2024-04-25T10:06:47.766428+0000","last_clean":"2024-04-25T10:06:47.766428+0000","last_became_active":"2024-04-25T10:06:46.814095+0000","last_became_peered":"2024-04-25T10:06:46.814095+0000","last_unstale":"2024-04-25T10:06:47.766428+0000","last_undegraded":"2024-04-25T10:06:47.766428+0000","last_fullsized":"2024-04-25T10:06:47.766428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.490689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765998+0000","last_change":"2024-04-25T10:06:46.099679+0000","last_active":"2024-04-25T10:06:47.765998+0000","last_peered":"2024-04-25T10:06:47.765998+0000","last_clean":"2024-04-25T10:06:47.765998+0000","last_became_active":"2024-04-25T10:06:46.098811+0000","last_became_peered":"2024-04-25T10:06:46.098811+0000","last_unstale":"2024-04-25T10:06:47.765998+0000","last_undegraded":"2024-04-25T10:06:47.765998+0000","last_fullsized":"2024-04-25T10:06:47.765998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.533772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578436+0000","last_change":"2024-04-25T10:06:46.783730+0000","last_active":"2024-04-25T10:06:48.578436+0000","last_peered":"2024-04-25T10:06:48.578436+0000","last_clean":"2024-04-25T10:06:48.578436+0000","last_became_active":"2024-04-25T10:06:46.779298+0000","last_became_peered":"2024-04-25T10:06:46.779298+0000","last_unstale":"2024-04-25T10:06:48.578436+0000","last_undegraded":"2024-04-25T10:06:48.578436+0000","last_fullsized":"2024-04-25T10:06:48.578436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577400+0000","last_change":"2024-04-25T10:06:46.097662+0000","last_active":"2024-04-25T10:06:48.577400+0000","last_peered":"2024-04-25T10:06:48.577400+0000","last_clean":"2024-04-25T10:06:48.577400+0000","last_became_active":"2024-04-25T10:06:46.096906+0000","last_became_peered":"2024-04-25T10:06:46.096906+0000","last_unstale":"2024-04-25T10:06:48.577400+0000","last_undegraded":"2024-04-25T10:06:48.577400+0000","last_fullsized":"2024-04-25T10:06:48.577400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766293+0000","last_change":"2024-04-25T10:06:46.793225+0000","last_active":"2024-04-25T10:06:47.766293+0000","last_peered":"2024-04-25T10:06:47.766293+0000","last_clean":"2024-04-25T10:06:47.766293+0000","last_became_active":"2024-04-25T10:06:46.792341+0000","last_became_peered":"2024-04-25T10:06:46.792341+0000","last_unstale":"2024-04-25T10:06:47.766293+0000","last_undegraded":"2024-04-25T10:06:47.766293+0000","last_fullsized":"2024-04-25T10:06:47.766293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.586619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578713+0000","last_change":"2024-04-25T10:06:46.112336+0000","last_active":"2024-04-25T10:06:48.578713+0000","last_peered":"2024-04-25T10:06:48.578713+0000","last_clean":"2024-04-25T10:06:48.578713+0000","last_became_active":"2024-04-25T10:06:46.108377+0000","last_became_peered":"2024-04-25T10:06:46.108377+0000","last_unstale":"2024-04-25T10:06:48.578713+0000","last_undegraded":"2024-04-25T10:06:48.578713+0000","last_fullsized":"2024-04-25T10:06:48.578713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765621+0000","last_change":"2024-04-25T10:06:46.791957+0000","last_active":"2024-04-25T10:06:47.765621+0000","last_peered":"2024-04-25T10:06:47.765621+0000","last_clean":"2024-04-25T10:06:47.765621+0000","last_became_active":"2024-04-25T10:06:46.789267+0000","last_became_peered":"2024-04-25T10:06:46.789267+0000","last_unstale":"2024-04-25T10:06:47.765621+0000","last_undegraded":"2024-04-25T10:06:47.765621+0000","last_fullsized":"2024-04-25T10:06:47.765621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:11.919154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.095569+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.095284+0000","last_became_peered":"2024-04-25T10:06:46.095284+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:49.409583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765571+0000","last_change":"2024-04-25T10:06:46.797079+0000","last_active":"2024-04-25T10:06:47.765571+0000","last_peered":"2024-04-25T10:06:47.765571+0000","last_clean":"2024-04-25T10:06:47.765571+0000","last_became_active":"2024-04-25T10:06:46.796377+0000","last_became_peered":"2024-04-25T10:06:46.796377+0000","last_unstale":"2024-04-25T10:06:47.765571+0000","last_undegraded":"2024-04-25T10:06:47.765571+0000","last_fullsized":"2024-04-25T10:06:47.765571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:08.421167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077979+0000","last_change":"2024-04-25T10:06:46.096666+0000","last_active":"2024-04-25T10:06:48.077979+0000","last_peered":"2024-04-25T10:06:48.077979+0000","last_clean":"2024-04-25T10:06:48.077979+0000","last_became_active":"2024-04-25T10:06:46.096084+0000","last_became_peered":"2024-04-25T10:06:46.096084+0000","last_unstale":"2024-04-25T10:06:48.077979+0000","last_undegraded":"2024-04-25T10:06:48.077979+0000","last_fullsized":"2024-04-25T10:06:48.077979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578330+0000","last_change":"2024-04-25T10:06:46.789487+0000","last_active":"2024-04-25T10:06:48.578330+0000","last_peered":"2024-04-25T10:06:48.578330+0000","last_clean":"2024-04-25T10:06:48.578330+0000","last_became_active":"2024-04-25T10:06:46.788801+0000","last_became_peered":"2024-04-25T10:06:46.788801+0000","last_unstale":"2024-04-25T10:06:48.578330+0000","last_undegraded":"2024-04-25T10:06:48.578330+0000","last_fullsized":"2024-04-25T10:06:48.578330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579292+0000","last_change":"2024-04-25T10:06:46.109437+0000","last_active":"2024-04-25T10:06:48.579292+0000","last_peered":"2024-04-25T10:06:48.579292+0000","last_clean":"2024-04-25T10:06:48.579292+0000","last_became_active":"2024-04-25T10:06:46.108802+0000","last_became_peered":"2024-04-25T10:06:46.108802+0000","last_unstale":"2024-04-25T10:06:48.579292+0000","last_undegraded":"2024-04-25T10:06:48.579292+0000","last_fullsized":"2024-04-25T10:06:48.579292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:27.950431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081989+0000","last_change":"2024-04-25T10:06:46.794307+0000","last_active":"2024-04-25T10:06:48.081989+0000","last_peered":"2024-04-25T10:06:48.081989+0000","last_clean":"2024-04-25T10:06:48.081989+0000","last_became_active":"2024-04-25T10:06:46.793463+0000","last_became_peered":"2024-04-25T10:06:46.793463+0000","last_unstale":"2024-04-25T10:06:48.081989+0000","last_undegraded":"2024-04-25T10:06:48.081989+0000","last_fullsized":"2024-04-25T10:06:48.081989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:13.222579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767889+0000","last_change":"2024-04-25T10:06:46.098133+0000","last_active":"2024-04-25T10:06:47.767889+0000","last_peered":"2024-04-25T10:06:47.767889+0000","last_clean":"2024-04-25T10:06:47.767889+0000","last_became_active":"2024-04-25T10:06:46.097273+0000","last_became_peered":"2024-04-25T10:06:46.097273+0000","last_unstale":"2024-04-25T10:06:47.767889+0000","last_undegraded":"2024-04-25T10:06:47.767889+0000","last_fullsized":"2024-04-25T10:06:47.767889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:56.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082265+0000","last_change":"2024-04-25T10:06:46.801431+0000","last_active":"2024-04-25T10:06:48.082265+0000","last_peered":"2024-04-25T10:06:48.082265+0000","last_clean":"2024-04-25T10:06:48.082265+0000","last_became_active":"2024-04-25T10:06:46.800621+0000","last_became_peered":"2024-04-25T10:06:46.800621+0000","last_unstale":"2024-04-25T10:06:48.082265+0000","last_undegraded":"2024-04-25T10:06:48.082265+0000","last_fullsized":"2024-04-25T10:06:48.082265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:22.445395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082645+0000","last_change":"2024-04-25T10:06:46.098539+0000","last_active":"2024-04-25T10:06:48.082645+0000","last_peered":"2024-04-25T10:06:48.082645+0000","last_clean":"2024-04-25T10:06:48.082645+0000","last_became_active":"2024-04-25T10:06:46.098190+0000","last_became_peered":"2024-04-25T10:06:46.098190+0000","last_unstale":"2024-04-25T10:06:48.082645+0000","last_undegraded":"2024-04-25T10:06:48.082645+0000","last_fullsized":"2024-04-25T10:06:48.082645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:13.339487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.794288+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.788351+0000","last_became_peered":"2024-04-25T10:06:46.788351+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766444+0000","last_change":"2024-04-25T10:06:46.086234+0000","last_active":"2024-04-25T10:06:47.766444+0000","last_peered":"2024-04-25T10:06:47.766444+0000","last_clean":"2024-04-25T10:06:47.766444+0000","last_became_active":"2024-04-25T10:06:46.085918+0000","last_became_peered":"2024-04-25T10:06:46.085918+0000","last_unstale":"2024-04-25T10:06:47.766444+0000","last_undegraded":"2024-04-25T10:06:47.766444+0000","last_fullsized":"2024-04-25T10:06:47.766444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078587+0000","last_change":"2024-04-25T10:06:46.808180+0000","last_active":"2024-04-25T10:06:48.078587+0000","last_peered":"2024-04-25T10:06:48.078587+0000","last_clean":"2024-04-25T10:06:48.078587+0000","last_became_active":"2024-04-25T10:06:46.800573+0000","last_became_peered":"2024-04-25T10:06:46.800573+0000","last_unstale":"2024-04-25T10:06:48.078587+0000","last_undegraded":"2024-04-25T10:06:48.078587+0000","last_fullsized":"2024-04-25T10:06:48.078587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:22.585824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578979+0000","last_change":"2024-04-25T10:06:46.089883+0000","last_active":"2024-04-25T10:06:48.578979+0000","last_peered":"2024-04-25T10:06:48.578979+0000","last_clean":"2024-04-25T10:06:48.578979+0000","last_became_active":"2024-04-25T10:06:46.089591+0000","last_became_peered":"2024-04-25T10:06:46.089591+0000","last_unstale":"2024-04-25T10:06:48.578979+0000","last_undegraded":"2024-04-25T10:06:48.578979+0000","last_fullsized":"2024-04-25T10:06:48.578979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078910+0000","last_change":"2024-04-25T10:06:46.808415+0000","last_active":"2024-04-25T10:06:48.078910+0000","last_peered":"2024-04-25T10:06:48.078910+0000","last_clean":"2024-04-25T10:06:48.078910+0000","last_became_active":"2024-04-25T10:06:46.807612+0000","last_became_peered":"2024-04-25T10:06:46.807612+0000","last_unstale":"2024-04-25T10:06:48.078910+0000","last_undegraded":"2024-04-25T10:06:48.078910+0000","last_fullsized":"2024-04-25T10:06:48.078910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:06.294138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578529+0000","last_change":"2024-04-25T10:06:46.099936+0000","last_active":"2024-04-25T10:06:48.578529+0000","last_peered":"2024-04-25T10:06:48.578529+0000","last_clean":"2024-04-25T10:06:48.578529+0000","last_became_active":"2024-04-25T10:06:46.098806+0000","last_became_peered":"2024-04-25T10:06:46.098806+0000","last_unstale":"2024-04-25T10:06:48.578529+0000","last_undegraded":"2024-04-25T10:06:48.578529+0000","last_fullsized":"2024-04-25T10:06:48.578529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765840+0000","last_change":"2024-04-25T10:06:46.793643+0000","last_active":"2024-04-25T10:06:47.765840+0000","last_peered":"2024-04-25T10:06:47.765840+0000","last_clean":"2024-04-25T10:06:47.765840+0000","last_became_active":"2024-04-25T10:06:46.791775+0000","last_became_peered":"2024-04-25T10:06:46.791775+0000","last_unstale":"2024-04-25T10:06:47.765840+0000","last_undegraded":"2024-04-25T10:06:47.765840+0000","last_fullsized":"2024-04-25T10:06:47.765840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:15.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578533+0000","last_change":"2024-04-25T10:06:46.099543+0000","last_active":"2024-04-25T10:06:48.578533+0000","last_peered":"2024-04-25T10:06:48.578533+0000","last_clean":"2024-04-25T10:06:48.578533+0000","last_became_active":"2024-04-25T10:06:46.099269+0000","last_became_peered":"2024-04-25T10:06:46.099269+0000","last_unstale":"2024-04-25T10:06:48.578533+0000","last_undegraded":"2024-04-25T10:06:48.578533+0000","last_fullsized":"2024-04-25T10:06:48.578533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764669+0000","last_change":"2024-04-25T10:06:46.798507+0000","last_active":"2024-04-25T10:06:47.764669+0000","last_peered":"2024-04-25T10:06:47.764669+0000","last_clean":"2024-04-25T10:06:47.764669+0000","last_became_active":"2024-04-25T10:06:46.797703+0000","last_became_peered":"2024-04-25T10:06:46.797703+0000","last_unstale":"2024-04-25T10:06:47.764669+0000","last_undegraded":"2024-04-25T10:06:47.764669+0000","last_fullsized":"2024-04-25T10:06:47.764669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:41.538388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577874+0000","last_change":"2024-04-25T10:06:46.090113+0000","last_active":"2024-04-25T10:06:48.577874+0000","last_peered":"2024-04-25T10:06:48.577874+0000","last_clean":"2024-04-25T10:06:48.577874+0000","last_became_active":"2024-04-25T10:06:46.089353+0000","last_became_peered":"2024-04-25T10:06:46.089353+0000","last_unstale":"2024-04-25T10:06:48.577874+0000","last_undegraded":"2024-04-25T10:06:48.577874+0000","last_fullsized":"2024-04-25T10:06:48.577874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765642+0000","last_change":"2024-04-25T10:06:46.797032+0000","last_active":"2024-04-25T10:06:47.765642+0000","last_peered":"2024-04-25T10:06:47.765642+0000","last_clean":"2024-04-25T10:06:47.765642+0000","last_became_active":"2024-04-25T10:06:46.796310+0000","last_became_peered":"2024-04-25T10:06:46.796310+0000","last_unstale":"2024-04-25T10:06:47.765642+0000","last_undegraded":"2024-04-25T10:06:47.765642+0000","last_fullsized":"2024-04-25T10:06:47.765642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:39.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.080225+0000","last_change":"2024-04-25T10:06:46.087236+0000","last_active":"2024-04-25T10:06:48.080225+0000","last_peered":"2024-04-25T10:06:48.080225+0000","last_clean":"2024-04-25T10:06:48.080225+0000","last_became_active":"2024-04-25T10:06:46.086354+0000","last_became_peered":"2024-04-25T10:06:46.086354+0000","last_unstale":"2024-04-25T10:06:48.080225+0000","last_undegraded":"2024-04-25T10:06:48.080225+0000","last_fullsized":"2024-04-25T10:06:48.080225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766980+0000","last_change":"2024-04-25T10:06:46.801971+0000","last_active":"2024-04-25T10:06:47.766980+0000","last_peered":"2024-04-25T10:06:47.766980+0000","last_clean":"2024-04-25T10:06:47.766980+0000","last_became_active":"2024-04-25T10:06:46.798311+0000","last_became_peered":"2024-04-25T10:06:46.798311+0000","last_unstale":"2024-04-25T10:06:47.766980+0000","last_undegraded":"2024-04-25T10:06:47.766980+0000","last_fullsized":"2024-04-25T10:06:47.766980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:55.984280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579054+0000","last_change":"2024-04-25T10:06:46.113284+0000","last_active":"2024-04-25T10:06:48.579054+0000","last_peered":"2024-04-25T10:06:48.579054+0000","last_clean":"2024-04-25T10:06:48.579054+0000","last_became_active":"2024-04-25T10:06:46.112729+0000","last_became_peered":"2024-04-25T10:06:46.112729+0000","last_unstale":"2024-04-25T10:06:48.579054+0000","last_undegraded":"2024-04-25T10:06:48.579054+0000","last_fullsized":"2024-04-25T10:06:48.579054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078514+0000","last_change":"2024-04-25T10:06:46.806654+0000","last_active":"2024-04-25T10:06:48.078514+0000","last_peered":"2024-04-25T10:06:48.078514+0000","last_clean":"2024-04-25T10:06:48.078514+0000","last_became_active":"2024-04-25T10:06:46.799360+0000","last_became_peered":"2024-04-25T10:06:46.799360+0000","last_unstale":"2024-04-25T10:06:48.078514+0000","last_undegraded":"2024-04-25T10:06:48.078514+0000","last_fullsized":"2024-04-25T10:06:48.078514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578334+0000","last_change":"2024-04-25T10:06:46.109473+0000","last_active":"2024-04-25T10:06:48.578334+0000","last_peered":"2024-04-25T10:06:48.578334+0000","last_clean":"2024-04-25T10:06:48.578334+0000","last_became_active":"2024-04-25T10:06:46.108640+0000","last_became_peered":"2024-04-25T10:06:46.108640+0000","last_unstale":"2024-04-25T10:06:48.578334+0000","last_undegraded":"2024-04-25T10:06:48.578334+0000","last_fullsized":"2024-04-25T10:06:48.578334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765464+0000","last_change":"2024-04-25T10:06:46.786613+0000","last_active":"2024-04-25T10:06:47.765464+0000","last_peered":"2024-04-25T10:06:47.765464+0000","last_clean":"2024-04-25T10:06:47.765464+0000","last_became_active":"2024-04-25T10:06:46.785785+0000","last_became_peered":"2024-04-25T10:06:46.785785+0000","last_unstale":"2024-04-25T10:06:47.765464+0000","last_undegraded":"2024-04-25T10:06:47.765464+0000","last_fullsized":"2024-04-25T10:06:47.765464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:15.393383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764735+0000","last_change":"2024-04-25T10:06:46.100079+0000","last_active":"2024-04-25T10:06:47.764735+0000","last_peered":"2024-04-25T10:06:47.764735+0000","last_clean":"2024-04-25T10:06:47.764735+0000","last_became_active":"2024-04-25T10:06:46.099419+0000","last_became_peered":"2024-04-25T10:06:46.099419+0000","last_unstale":"2024-04-25T10:06:47.764735+0000","last_undegraded":"2024-04-25T10:06:47.764735+0000","last_fullsized":"2024-04-25T10:06:47.764735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765365+0000","last_change":"2024-04-25T10:06:46.793298+0000","last_active":"2024-04-25T10:06:47.765365+0000","last_peered":"2024-04-25T10:06:47.765365+0000","last_clean":"2024-04-25T10:06:47.765365+0000","last_became_active":"2024-04-25T10:06:46.791603+0000","last_became_peered":"2024-04-25T10:06:46.791603+0000","last_unstale":"2024-04-25T10:06:47.765365+0000","last_undegraded":"2024-04-25T10:06:47.765365+0000","last_fullsized":"2024-04-25T10:06:47.765365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082282+0000","last_change":"2024-04-25T10:06:46.088827+0000","last_active":"2024-04-25T10:06:48.082282+0000","last_peered":"2024-04-25T10:06:48.082282+0000","last_clean":"2024-04-25T10:06:48.082282+0000","last_became_active":"2024-04-25T10:06:46.088541+0000","last_became_peered":"2024-04-25T10:06:46.088541+0000","last_unstale":"2024-04-25T10:06:48.082282+0000","last_undegraded":"2024-04-25T10:06:48.082282+0000","last_fullsized":"2024-04-25T10:06:48.082282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:37.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766800+0000","last_change":"2024-04-25T10:06:46.815568+0000","last_active":"2024-04-25T10:06:47.766800+0000","last_peered":"2024-04-25T10:06:47.766800+0000","last_clean":"2024-04-25T10:06:47.766800+0000","last_became_active":"2024-04-25T10:06:46.814861+0000","last_became_peered":"2024-04-25T10:06:46.814861+0000","last_unstale":"2024-04-25T10:06:47.766800+0000","last_undegraded":"2024-04-25T10:06:47.766800+0000","last_fullsized":"2024-04-25T10:06:47.766800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765886+0000","last_change":"2024-04-25T10:06:46.092480+0000","last_active":"2024-04-25T10:06:47.765886+0000","last_peered":"2024-04-25T10:06:47.765886+0000","last_clean":"2024-04-25T10:06:47.765886+0000","last_became_active":"2024-04-25T10:06:46.091763+0000","last_became_peered":"2024-04-25T10:06:46.091763+0000","last_unstale":"2024-04-25T10:06:47.765886+0000","last_undegraded":"2024-04-25T10:06:47.765886+0000","last_fullsized":"2024-04-25T10:06:47.765886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:44.359393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766985+0000","last_change":"2024-04-25T10:06:46.783297+0000","last_active":"2024-04-25T10:06:47.766985+0000","last_peered":"2024-04-25T10:06:47.766985+0000","last_clean":"2024-04-25T10:06:47.766985+0000","last_became_active":"2024-04-25T10:06:46.782524+0000","last_became_peered":"2024-04-25T10:06:46.782524+0000","last_unstale":"2024-04-25T10:06:47.766985+0000","last_undegraded":"2024-04-25T10:06:47.766985+0000","last_fullsized":"2024-04-25T10:06:47.766985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766127+0000","last_change":"2024-04-25T10:06:46.079371+0000","last_active":"2024-04-25T10:06:47.766127+0000","last_peered":"2024-04-25T10:06:47.766127+0000","last_clean":"2024-04-25T10:06:47.766127+0000","last_became_active":"2024-04-25T10:06:46.079020+0000","last_became_peered":"2024-04-25T10:06:46.079020+0000","last_unstale":"2024-04-25T10:06:47.766127+0000","last_undegraded":"2024-04-25T10:06:47.766127+0000","last_fullsized":"2024-04-25T10:06:47.766127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.805681+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.805076+0000","last_became_peered":"2024-04-25T10:06:46.805076+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579678+0000","last_change":"2024-04-25T10:06:46.099980+0000","last_active":"2024-04-25T10:06:48.579678+0000","last_peered":"2024-04-25T10:06:48.579678+0000","last_clean":"2024-04-25T10:06:48.579678+0000","last_became_active":"2024-04-25T10:06:46.099518+0000","last_became_peered":"2024-04-25T10:06:46.099518+0000","last_unstale":"2024-04-25T10:06:48.579678+0000","last_undegraded":"2024-04-25T10:06:48.579678+0000","last_fullsized":"2024-04-25T10:06:48.579678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:15.012055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578707+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:06:48.578707+0000","last_peered":"2024-04-25T10:06:48.578707+0000","last_clean":"2024-04-25T10:06:48.578707+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:06:48.578707+0000","last_undegraded":"2024-04-25T10:06:48.578707+0000","last_fullsized":"2024-04-25T10:06:48.578707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1032685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":443907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":444011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":443037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":442407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":1032663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:51.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:06:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.626+0000 7f5625db1640 1 Processor -- start 2024-04-25T10:06:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.626+0000 7f5625db1640 1 -- start start 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.626+0000 7f5625db1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f562006e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f5625db1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f56200cf560 0x7f56200cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f5625db1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562006ee70 con 0x7f56200b2720 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f5625db1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56200cff10 con 0x7f56200cf560 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f5625db1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b37d0 con 0x7f562015a600 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f56200b2720 0x7f56200b3c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35662/0 (socket says 172.21.15.82:35662) 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 -- 172.21.15.82:0/287737017 learned_addr learned my addr 172.21.15.82:0/287737017 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561f7fe640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f562006e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 236372977 0 0) 0x7f562006ee70 con 0x7f56200b2720 2024-04-25T10:06:51.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561ffff640 1 --2- 172.21.15.82:0/287737017 >> v2:172.21.15.196:3300/0 conn(0x7f56200cf560 0x7f56200cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5608003530 con 0x7f56200b2720 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739237737 0 0) 0x7f5608003530 con 0x7f56200b2720 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 msgr2=0x7f562006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f562006e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 >> v2:172.21.15.196:3300/0 conn(0x7f56200cf560 msgr2=0x7f56200cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 --2- 172.21.15.82:0/287737017 >> v2:172.21.15.196:3300/0 conn(0x7f56200cf560 0x7f56200cf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200b2ca0 con 0x7f56200b2720 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5614001680 con 0x7f56200b2720 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 >> v1:172.21.15.82:6789/0 conn(0x7f56200b2720 legacy=0x7f56200b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608003a30 0x7f5608003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5608004810 0x7f5608004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f56080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200cff10 con 0x7f5608004810 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b37d0 con 0x7f5608003a30 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f560800b9d0 con 0x7f562015a600 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5608004810 0x7f5608004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561ffff640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f56080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561f7fe640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608003a30 0x7f5608003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 -- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 msgr2=0x7f56080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f562015a600 0x7f56080051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 -- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608003a30 msgr2=0x7f5608003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608003a30 0x7f5608003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 -- 172.21.15.82:0/287737017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608003530 con 0x7f5608004810 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561effd640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5608004810 0x7f5608004c90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f561400c270 tx=0x7f561401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df43a7be2882549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614007d60 con 0x7f5608004810 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614005b00 con 0x7f5608004810 2024-04-25T10:06:51.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.630+0000 7f561e7fc640 1 -- 172.21.15.82:0/287737017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614007070 con 0x7f5608004810 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/287737017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5608004810 msgr2=0x7f5608004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 --2- 172.21.15.82:0/287737017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5608004810 0x7f5608004c90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f561400c270 tx=0x7f561401f410 comp rx=0 tx=0).stop 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561ffff640 1 -- 172.21.15.82:0/287737017 reap_dead start 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/287737017 shutdown_connections 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/287737017 >> 172.21.15.82:0/287737017 conn(0x7f562006ae60 msgr2=0x7f562006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/287737017 shutdown_connections 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/287737017 wait complete. 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 Processor -- start 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- start start 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608004810 0x7f5620072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56200bdce0 0x7f56200ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b31e0 con 0x7f562015ac40 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200b2f40 con 0x7f5608004810 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562015a9e0 con 0x7f56200bdce0 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58188/0 (socket says 172.21.15.82:58188) 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 -- 172.21.15.82:0/1464836043 learned_addr learned my addr 172.21.15.82:0/1464836043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561ffff640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56200bdce0 0x7f56200ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561f7fe640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608004810 0x7f5620072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 -- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56200bdce0 msgr2=0x7f56200ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56200bdce0 0x7f56200ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 -- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608004810 msgr2=0x7f5620072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608004810 0x7f5620072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 -- 172.21.15.82:0/1464836043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200c3a80 con 0x7f562015ac40 2024-04-25T10:06:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f561effd640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f561400c270 tx=0x7f5608005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614001ca0 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614005cb0 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56200721f0 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614005e60 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562015b420 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.634+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562004efa0 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.638+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56140ba030 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.638+0000 7f5607fff640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.638+0000 7f561f7fe640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.638+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5614121c70 con 0x7f562015ac40 2024-04-25T10:06:51.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.638+0000 7f561f7fe640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 0x7f560c069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f56100098d0 tx=0x7f5610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.642+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5614019120 con 0x7f562015ac40 2024-04-25T10:06:51.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5620151d30 con 0x7f562015ac40 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5607fff640 1 -- 172.21.15.82:0/1464836043 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1175 (secure 0 0 0) 0x7f56140f2b00 con 0x7f562015ac40 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:51.848 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":309,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":79,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039313,"num_in_osds":8,"osd_in_since":1714039304,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":264433664,"bytes_avail":772829679616,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":41,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"a","status":"up:active","gid":6966}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T10:06:15.339291+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:06:51.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 msgr2=0x7f560c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 0x7f560c069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f56100098d0 tx=0x7f5610008040 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 msgr2=0x7f56200c3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f561400c270 tx=0x7f5608005840 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 shutdown_connections 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.846+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f560c066f60 0x7f560c069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56200bdce0 0x7f56200ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f562015ac40 0x7f56200c3540 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 --2- 172.21.15.82:0/1464836043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5608004810 0x7f5620072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 >> 172.21.15.82:0/1464836043 conn(0x7f562006ae60 msgr2=0x7f56200cd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 shutdown_connections 2024-04-25T10:06:51.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.850+0000 7f5625db1640 1 -- 172.21.15.82:0/1464836043 wait complete. 2024-04-25T10:06:51.867 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:06:51.867 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:51.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 Processor -- start 2024-04-25T10:06:51.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 -- start start 2024-04-25T10:06:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fac0c0b2c30 0x7fac0c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c066c30 con 0x7fac0c06c3b0 2024-04-25T10:06:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac0c0af660 con 0x7fac0c0af280 2024-04-25T10:06:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac139a9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac0c06daa0 con 0x7fac0c0b2c30 2024-04-25T10:06:51.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac10f44640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fac0c06c3b0 0x7fac0c06df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35678/0 (socket says 172.21.15.82:35678) 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac10f44640 1 -- 172.21.15.82:0/200206555 learned_addr learned my addr 172.21.15.82:0/200206555 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 848914660 0 0) 0x7fac0c066c30 con 0x7fac0c06c3b0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabf8003530 con 0x7fac0c06c3b0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 --2- 172.21.15.82:0/200206555 >> v2:172.21.15.196:3300/0 conn(0x7fac0c0b2c30 0x7fac0c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2093995979 0 0) 0x7fabf8003530 con 0x7fac0c06c3b0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 >> v2:172.21.15.196:3300/0 conn(0x7fac0c0b2c30 msgr2=0x7fac0c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 --2- 172.21.15.82:0/200206555 >> v2:172.21.15.196:3300/0 conn(0x7fac0c0b2c30 0x7fac0c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 >> v1:172.21.15.196:6789/0 conn(0x7fac0c0af280 legacy=0x7fac0c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c15e640 con 0x7fac0c06c3b0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fabfc001680 con 0x7fac0c06c3b0 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 >> v1:172.21.15.82:6789/0 conn(0x7fac0c06c3b0 legacy=0x7fac0c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8003a30 0x7fabf8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf8004810 0x7fabf8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 0x7fabf800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf800c170 con 0x7fabf8005240 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf800c320 con 0x7fabf8004810 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf800c520 con 0x7fabf8003a30 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 0x7fabf800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac10f44640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf8004810 0x7fabf8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 -- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8003a30 msgr2=0x7fabf8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11745640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8003a30 0x7fabf8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8003a30 0x7fabf8003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 -- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf8004810 msgr2=0x7fabf8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:51.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf8004810 0x7fabf8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac11f46640 1 -- 172.21.15.82:0/200206555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8003530 con 0x7fabf8005240 2024-04-25T10:06:51.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.982+0000 7fac10f44640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fabf8004810 0x7fabf8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:51.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac11f46640 1 -- 172.21.15.82:0/200206555 reap_dead start 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac11f46640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 0x7fabf800b9d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ba10 tx=0x7fac0801c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b57af4246091504 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800b610 con 0x7fabf8005240 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac080aa630 con 0x7fabf8005240 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac03fff640 1 -- 172.21.15.82:0/200206555 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac080aaa20 con 0x7fabf8005240 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 msgr2=0x7fabf800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 0x7fabf800b9d0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fac0800ba10 tx=0x7fac0801c6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- 172.21.15.82:0/200206555 shutdown_connections 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 --2- 172.21.15.82:0/200206555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fabf8005240 0x7fabf800b9d0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- 172.21.15.82:0/200206555 >> 172.21.15.82:0/200206555 conn(0x7fac0c071810 msgr2=0x7fac0c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- 172.21.15.82:0/200206555 shutdown_connections 2024-04-25T10:06:51.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- 172.21.15.82:0/200206555 wait complete. 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 Processor -- start 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.986+0000 7fac139a9640 1 -- start start 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8005240 0x7fac0c0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac0c150c80 0x7fac0c151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06bca0 con 0x7fac0c14c800 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c15ec00 con 0x7fac0c150c80 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0c06be20 con 0x7fabf8005240 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58202/0 (socket says 172.21.15.82:58202) 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 -- 172.21.15.82:0/4263017032 learned_addr learned my addr 172.21.15.82:0/4263017032 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:51.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac10f44640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac0c150c80 0x7fac0c151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11745640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8005240 0x7fac0c0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 -- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8005240 msgr2=0x7fac0c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8005240 0x7fac0c0cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 -- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac0c150c80 msgr2=0x7fac0c151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac0c150c80 0x7fac0c151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 -- 172.21.15.82:0/4263017032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac0c152080 con 0x7fac0c14c800 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11f46640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fac080980d0 tx=0x7fabf8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac0800b610 con 0x7fac0c14c800 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac080ba070 con 0x7fac0c14c800 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac080a8850 con 0x7fac0c14c800 2024-04-25T10:06:51.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac0c152310 con 0x7fac0c14c800 2024-04-25T10:06:51.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac0c1565b0 con 0x7fac0c14c800 2024-04-25T10:06:51.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac0c04efa0 con 0x7fac0c14c800 2024-04-25T10:06:51.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac080b2070 con 0x7fac0c14c800 2024-04-25T10:06:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac0811f6a0 con 0x7fac0c14c800 2024-04-25T10:06:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11745640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:51.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.990+0000 7fac11745640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 0x7fabf0069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac04000f90 tx=0x7fac04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:51.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:51.994+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac080163d0 con 0x7fac0c14c800 2024-04-25T10:06:52.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.138+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac0c0727c0 con 0x7fabf0066f60 2024-04-25T10:06:52.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac01ffb640 1 -- 172.21.15.82:0/4263017032 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252072 (secure 0 0 0) 0x7fac0c0727c0 con 0x7fabf0066f60 2024-04-25T10:06:52.150 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:52.151 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:06:52.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 msgr2=0x7fabf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 0x7fabf0069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fac04000f90 tx=0x7fac04009040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 msgr2=0x7fac0c153310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fac080980d0 tx=0x7fabf8005840 comp rx=0 tx=0).stop 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 shutdown_connections 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fabf0066f60 0x7fabf0069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fac0c14c800 0x7fac0c153310 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.146+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fac0c150c80 0x7fac0c151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.150+0000 7fac139a9640 1 --2- 172.21.15.82:0/4263017032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fabf8005240 0x7fac0c0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.150+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 >> 172.21.15.82:0/4263017032 conn(0x7fac0c071810 msgr2=0x7fac0c15df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.150+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 shutdown_connections 2024-04-25T10:06:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:52.150+0000 7fac139a9640 1 -- 172.21.15.82:0/4263017032 wait complete. 2024-04-25T10:06:52.167 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-04-25T10:06:51.349575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258236,"kb_used_data":8776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716484,"statfs":{"total":773094113280,"available":772829679616,"internally_reserved":0,"allocated":8986624,"data_stored":4723916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001350"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579140+0000","last_change":"2024-04-25T10:06:46.812042+0000","last_active":"2024-04-25T10:06:48.579140+0000","last_peered":"2024-04-25T10:06:48.579140+0000","last_clean":"2024-04-25T10:06:48.579140+0000","last_became_active":"2024-04-25T10:06:46.811330+0000","last_became_peered":"2024-04-25T10:06:46.811330+0000","last_unstale":"2024-04-25T10:06:48.579140+0000","last_undegraded":"2024-04-25T10:06:48.579140+0000","last_fullsized":"2024-04-25T10:06:48.579140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078792+0000","last_change":"2024-04-25T10:06:46.814055+0000","last_active":"2024-04-25T10:06:48.078792+0000","last_peered":"2024-04-25T10:06:48.078792+0000","last_clean":"2024-04-25T10:06:48.078792+0000","last_became_active":"2024-04-25T10:06:46.813374+0000","last_became_peered":"2024-04-25T10:06:46.813374+0000","last_unstale":"2024-04-25T10:06:48.078792+0000","last_undegraded":"2024-04-25T10:06:48.078792+0000","last_fullsized":"2024-04-25T10:06:48.078792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.433303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766684+0000","last_change":"2024-04-25T10:06:46.816949+0000","last_active":"2024-04-25T10:06:47.766684+0000","last_peered":"2024-04-25T10:06:47.766684+0000","last_clean":"2024-04-25T10:06:47.766684+0000","last_became_active":"2024-04-25T10:06:46.816384+0000","last_became_peered":"2024-04-25T10:06:46.816384+0000","last_unstale":"2024-04-25T10:06:47.766684+0000","last_undegraded":"2024-04-25T10:06:47.766684+0000","last_fullsized":"2024-04-25T10:06:47.766684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:41.271967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578047+0000","last_change":"2024-04-25T10:06:46.782326+0000","last_active":"2024-04-25T10:06:48.578047+0000","last_peered":"2024-04-25T10:06:48.578047+0000","last_clean":"2024-04-25T10:06:48.578047+0000","last_became_active":"2024-04-25T10:06:46.780178+0000","last_became_peered":"2024-04-25T10:06:46.780178+0000","last_unstale":"2024-04-25T10:06:48.578047+0000","last_undegraded":"2024-04-25T10:06:48.578047+0000","last_fullsized":"2024-04-25T10:06:48.578047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:11.581560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082074+0000","last_change":"2024-04-25T10:06:46.786463+0000","last_active":"2024-04-25T10:06:48.082074+0000","last_peered":"2024-04-25T10:06:48.082074+0000","last_clean":"2024-04-25T10:06:48.082074+0000","last_became_active":"2024-04-25T10:06:46.785559+0000","last_became_peered":"2024-04-25T10:06:46.785559+0000","last_unstale":"2024-04-25T10:06:48.082074+0000","last_undegraded":"2024-04-25T10:06:48.082074+0000","last_fullsized":"2024-04-25T10:06:48.082074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:01.174907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579158+0000","last_change":"2024-04-25T10:06:46.816049+0000","last_active":"2024-04-25T10:06:48.579158+0000","last_peered":"2024-04-25T10:06:48.579158+0000","last_clean":"2024-04-25T10:06:48.579158+0000","last_became_active":"2024-04-25T10:06:46.814728+0000","last_became_peered":"2024-04-25T10:06:46.814728+0000","last_unstale":"2024-04-25T10:06:48.579158+0000","last_undegraded":"2024-04-25T10:06:48.579158+0000","last_fullsized":"2024-04-25T10:06:48.579158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765258+0000","last_change":"2024-04-25T10:06:46.787007+0000","last_active":"2024-04-25T10:06:47.765258+0000","last_peered":"2024-04-25T10:06:47.765258+0000","last_clean":"2024-04-25T10:06:47.765258+0000","last_became_active":"2024-04-25T10:06:46.786255+0000","last_became_peered":"2024-04-25T10:06:46.786255+0000","last_unstale":"2024-04-25T10:06:47.765258+0000","last_undegraded":"2024-04-25T10:06:47.765258+0000","last_fullsized":"2024-04-25T10:06:47.765258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082256+0000","last_change":"2024-04-25T10:06:46.785378+0000","last_active":"2024-04-25T10:06:48.082256+0000","last_peered":"2024-04-25T10:06:48.082256+0000","last_clean":"2024-04-25T10:06:48.082256+0000","last_became_active":"2024-04-25T10:06:46.784540+0000","last_became_peered":"2024-04-25T10:06:46.784540+0000","last_unstale":"2024-04-25T10:06:48.082256+0000","last_undegraded":"2024-04-25T10:06:48.082256+0000","last_fullsized":"2024-04-25T10:06:48.082256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:56.712512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767669+0000","last_change":"2024-04-25T10:06:46.781725+0000","last_active":"2024-04-25T10:06:47.767669+0000","last_peered":"2024-04-25T10:06:47.767669+0000","last_clean":"2024-04-25T10:06:47.767669+0000","last_became_active":"2024-04-25T10:06:46.781039+0000","last_became_peered":"2024-04-25T10:06:46.781039+0000","last_unstale":"2024-04-25T10:06:47.767669+0000","last_undegraded":"2024-04-25T10:06:47.767669+0000","last_fullsized":"2024-04-25T10:06:47.767669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:11.102231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577706+0000","last_change":"2024-04-25T10:06:46.795742+0000","last_active":"2024-04-25T10:06:48.577706+0000","last_peered":"2024-04-25T10:06:48.577706+0000","last_clean":"2024-04-25T10:06:48.577706+0000","last_became_active":"2024-04-25T10:06:46.794090+0000","last_became_peered":"2024-04-25T10:06:46.794090+0000","last_unstale":"2024-04-25T10:06:48.577706+0000","last_undegraded":"2024-04-25T10:06:48.577706+0000","last_fullsized":"2024-04-25T10:06:48.577706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:12.588290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078775+0000","last_change":"2024-04-25T10:06:46.813315+0000","last_active":"2024-04-25T10:06:48.078775+0000","last_peered":"2024-04-25T10:06:48.078775+0000","last_clean":"2024-04-25T10:06:48.078775+0000","last_became_active":"2024-04-25T10:06:46.800516+0000","last_became_peered":"2024-04-25T10:06:46.800516+0000","last_unstale":"2024-04-25T10:06:48.078775+0000","last_undegraded":"2024-04-25T10:06:48.078775+0000","last_fullsized":"2024-04-25T10:06:48.078775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:05.005574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578300+0000","last_change":"2024-04-25T10:06:46.813321+0000","last_active":"2024-04-25T10:06:48.578300+0000","last_peered":"2024-04-25T10:06:48.578300+0000","last_clean":"2024-04-25T10:06:48.578300+0000","last_became_active":"2024-04-25T10:06:46.812941+0000","last_became_peered":"2024-04-25T10:06:46.812941+0000","last_unstale":"2024-04-25T10:06:48.578300+0000","last_undegraded":"2024-04-25T10:06:48.578300+0000","last_fullsized":"2024-04-25T10:06:48.578300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:34.026951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082448+0000","last_change":"2024-04-25T10:06:46.776909+0000","last_active":"2024-04-25T10:06:48.082448+0000","last_peered":"2024-04-25T10:06:48.082448+0000","last_clean":"2024-04-25T10:06:48.082448+0000","last_became_active":"2024-04-25T10:06:46.776153+0000","last_became_peered":"2024-04-25T10:06:46.776153+0000","last_unstale":"2024-04-25T10:06:48.082448+0000","last_undegraded":"2024-04-25T10:06:48.082448+0000","last_fullsized":"2024-04-25T10:06:48.082448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:40.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578644+0000","last_change":"2024-04-25T10:06:46.805290+0000","last_active":"2024-04-25T10:06:48.578644+0000","last_peered":"2024-04-25T10:06:48.578644+0000","last_clean":"2024-04-25T10:06:48.578644+0000","last_became_active":"2024-04-25T10:06:46.799720+0000","last_became_peered":"2024-04-25T10:06:46.799720+0000","last_unstale":"2024-04-25T10:06:48.578644+0000","last_undegraded":"2024-04-25T10:06:48.578644+0000","last_fullsized":"2024-04-25T10:06:48.578644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078045+0000","last_change":"2024-04-25T10:06:46.807971+0000","last_active":"2024-04-25T10:06:48.078045+0000","last_peered":"2024-04-25T10:06:48.078045+0000","last_clean":"2024-04-25T10:06:48.078045+0000","last_became_active":"2024-04-25T10:06:46.800983+0000","last_became_peered":"2024-04-25T10:06:46.800983+0000","last_unstale":"2024-04-25T10:06:48.078045+0000","last_undegraded":"2024-04-25T10:06:48.078045+0000","last_fullsized":"2024-04-25T10:06:48.078045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578470+0000","last_change":"2024-04-25T10:06:46.790688+0000","last_active":"2024-04-25T10:06:48.578470+0000","last_peered":"2024-04-25T10:06:48.578470+0000","last_clean":"2024-04-25T10:06:48.578470+0000","last_became_active":"2024-04-25T10:06:46.789841+0000","last_became_peered":"2024-04-25T10:06:46.789841+0000","last_unstale":"2024-04-25T10:06:48.578470+0000","last_undegraded":"2024-04-25T10:06:48.578470+0000","last_fullsized":"2024-04-25T10:06:48.578470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.083022+0000","last_change":"2024-04-25T10:06:46.803641+0000","last_active":"2024-04-25T10:06:48.083022+0000","last_peered":"2024-04-25T10:06:48.083022+0000","last_clean":"2024-04-25T10:06:48.083022+0000","last_became_active":"2024-04-25T10:06:46.802850+0000","last_became_peered":"2024-04-25T10:06:46.802850+0000","last_unstale":"2024-04-25T10:06:48.083022+0000","last_undegraded":"2024-04-25T10:06:48.083022+0000","last_fullsized":"2024-04-25T10:06:48.083022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:45.971536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766827+0000","last_change":"2024-04-25T10:06:46.782609+0000","last_active":"2024-04-25T10:06:47.766827+0000","last_peered":"2024-04-25T10:06:47.766827+0000","last_clean":"2024-04-25T10:06:47.766827+0000","last_became_active":"2024-04-25T10:06:46.781574+0000","last_became_peered":"2024-04-25T10:06:46.781574+0000","last_unstale":"2024-04-25T10:06:47.766827+0000","last_undegraded":"2024-04-25T10:06:47.766827+0000","last_fullsized":"2024-04-25T10:06:47.766827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579031+0000","last_change":"2024-04-25T10:06:46.788869+0000","last_active":"2024-04-25T10:06:48.579031+0000","last_peered":"2024-04-25T10:06:48.579031+0000","last_clean":"2024-04-25T10:06:48.579031+0000","last_became_active":"2024-04-25T10:06:46.788132+0000","last_became_peered":"2024-04-25T10:06:46.788132+0000","last_unstale":"2024-04-25T10:06:48.579031+0000","last_undegraded":"2024-04-25T10:06:48.579031+0000","last_fullsized":"2024-04-25T10:06:48.579031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082318+0000","last_change":"2024-04-25T10:06:46.785801+0000","last_active":"2024-04-25T10:06:48.082318+0000","last_peered":"2024-04-25T10:06:48.082318+0000","last_clean":"2024-04-25T10:06:48.082318+0000","last_became_active":"2024-04-25T10:06:46.784579+0000","last_became_peered":"2024-04-25T10:06:46.784579+0000","last_unstale":"2024-04-25T10:06:48.082318+0000","last_undegraded":"2024-04-25T10:06:48.082318+0000","last_fullsized":"2024-04-25T10:06:48.082318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:46.516701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767026+0000","last_change":"2024-04-25T10:06:46.803669+0000","last_active":"2024-04-25T10:06:47.767026+0000","last_peered":"2024-04-25T10:06:47.767026+0000","last_clean":"2024-04-25T10:06:47.767026+0000","last_became_active":"2024-04-25T10:06:46.802373+0000","last_became_peered":"2024-04-25T10:06:46.802373+0000","last_unstale":"2024-04-25T10:06:47.767026+0000","last_undegraded":"2024-04-25T10:06:47.767026+0000","last_fullsized":"2024-04-25T10:06:47.767026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:58.982166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.797809+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.797133+0000","last_became_peered":"2024-04-25T10:06:46.797133+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:27.886972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764762+0000","last_change":"2024-04-25T10:06:46.783667+0000","last_active":"2024-04-25T10:06:47.764762+0000","last_peered":"2024-04-25T10:06:47.764762+0000","last_clean":"2024-04-25T10:06:47.764762+0000","last_became_active":"2024-04-25T10:06:46.782958+0000","last_became_peered":"2024-04-25T10:06:46.782958+0000","last_unstale":"2024-04-25T10:06:47.764762+0000","last_undegraded":"2024-04-25T10:06:47.764762+0000","last_fullsized":"2024-04-25T10:06:47.764762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:45.364008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078138+0000","last_change":"2024-04-25T10:06:46.807236+0000","last_active":"2024-04-25T10:06:48.078138+0000","last_peered":"2024-04-25T10:06:48.078138+0000","last_clean":"2024-04-25T10:06:48.078138+0000","last_became_active":"2024-04-25T10:06:46.800093+0000","last_became_peered":"2024-04-25T10:06:46.800093+0000","last_unstale":"2024-04-25T10:06:48.078138+0000","last_undegraded":"2024-04-25T10:06:48.078138+0000","last_fullsized":"2024-04-25T10:06:48.078138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078877+0000","last_change":"2024-04-25T10:06:46.806479+0000","last_active":"2024-04-25T10:06:48.078877+0000","last_peered":"2024-04-25T10:06:48.078877+0000","last_clean":"2024-04-25T10:06:48.078877+0000","last_became_active":"2024-04-25T10:06:46.800600+0000","last_became_peered":"2024-04-25T10:06:46.800600+0000","last_unstale":"2024-04-25T10:06:48.078877+0000","last_undegraded":"2024-04-25T10:06:48.078877+0000","last_fullsized":"2024-04-25T10:06:48.078877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:06.913908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577901+0000","last_change":"2024-04-25T10:06:46.813601+0000","last_active":"2024-04-25T10:06:48.577901+0000","last_peered":"2024-04-25T10:06:48.577901+0000","last_clean":"2024-04-25T10:06:48.577901+0000","last_became_active":"2024-04-25T10:06:46.812901+0000","last_became_peered":"2024-04-25T10:06:46.812901+0000","last_unstale":"2024-04-25T10:06:48.577901+0000","last_undegraded":"2024-04-25T10:06:48.577901+0000","last_fullsized":"2024-04-25T10:06:48.577901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:15.598255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578852+0000","last_change":"2024-04-25T10:06:46.804827+0000","last_active":"2024-04-25T10:06:48.578852+0000","last_peered":"2024-04-25T10:06:48.578852+0000","last_clean":"2024-04-25T10:06:48.578852+0000","last_became_active":"2024-04-25T10:06:46.804083+0000","last_became_peered":"2024-04-25T10:06:46.804083+0000","last_unstale":"2024-04-25T10:06:48.578852+0000","last_undegraded":"2024-04-25T10:06:48.578852+0000","last_fullsized":"2024-04-25T10:06:48.578852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082404+0000","last_change":"2024-04-25T10:06:46.789728+0000","last_active":"2024-04-25T10:06:48.082404+0000","last_peered":"2024-04-25T10:06:48.082404+0000","last_clean":"2024-04-25T10:06:48.082404+0000","last_became_active":"2024-04-25T10:06:46.788957+0000","last_became_peered":"2024-04-25T10:06:46.788957+0000","last_unstale":"2024-04-25T10:06:48.082404+0000","last_undegraded":"2024-04-25T10:06:48.082404+0000","last_fullsized":"2024-04-25T10:06:48.082404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.998327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078446+0000","last_change":"2024-04-25T10:06:46.808484+0000","last_active":"2024-04-25T10:06:48.078446+0000","last_peered":"2024-04-25T10:06:48.078446+0000","last_clean":"2024-04-25T10:06:48.078446+0000","last_became_active":"2024-04-25T10:06:46.807681+0000","last_became_peered":"2024-04-25T10:06:46.807681+0000","last_unstale":"2024-04-25T10:06:48.078446+0000","last_undegraded":"2024-04-25T10:06:48.078446+0000","last_fullsized":"2024-04-25T10:06:48.078446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578440+0000","last_change":"2024-04-25T10:06:46.806504+0000","last_active":"2024-04-25T10:06:48.578440+0000","last_peered":"2024-04-25T10:06:48.578440+0000","last_clean":"2024-04-25T10:06:48.578440+0000","last_became_active":"2024-04-25T10:06:46.805910+0000","last_became_peered":"2024-04-25T10:06:46.805910+0000","last_unstale":"2024-04-25T10:06:48.578440+0000","last_undegraded":"2024-04-25T10:06:48.578440+0000","last_fullsized":"2024-04-25T10:06:48.578440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766290+0000","last_change":"2024-04-25T10:06:46.795871+0000","last_active":"2024-04-25T10:06:47.766290+0000","last_peered":"2024-04-25T10:06:47.766290+0000","last_clean":"2024-04-25T10:06:47.766290+0000","last_became_active":"2024-04-25T10:06:46.794978+0000","last_became_peered":"2024-04-25T10:06:46.794978+0000","last_unstale":"2024-04-25T10:06:47.766290+0000","last_undegraded":"2024-04-25T10:06:47.766290+0000","last_fullsized":"2024-04-25T10:06:47.766290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:20.023879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578885+0000","last_change":"2024-04-25T10:06:46.804579+0000","last_active":"2024-04-25T10:06:48.578885+0000","last_peered":"2024-04-25T10:06:48.578885+0000","last_clean":"2024-04-25T10:06:48.578885+0000","last_became_active":"2024-04-25T10:06:46.803668+0000","last_became_peered":"2024-04-25T10:06:46.803668+0000","last_unstale":"2024-04-25T10:06:48.578885+0000","last_undegraded":"2024-04-25T10:06:48.578885+0000","last_fullsized":"2024-04-25T10:06:48.578885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766189+0000","last_change":"2024-04-25T10:06:46.781678+0000","last_active":"2024-04-25T10:06:47.766189+0000","last_peered":"2024-04-25T10:06:47.766189+0000","last_clean":"2024-04-25T10:06:47.766189+0000","last_became_active":"2024-04-25T10:06:46.780704+0000","last_became_peered":"2024-04-25T10:06:46.780704+0000","last_unstale":"2024-04-25T10:06:47.766189+0000","last_undegraded":"2024-04-25T10:06:47.766189+0000","last_fullsized":"2024-04-25T10:06:47.766189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082982+0000","last_change":"2024-04-25T10:06:46.089223+0000","last_active":"2024-04-25T10:06:48.082982+0000","last_peered":"2024-04-25T10:06:48.082982+0000","last_clean":"2024-04-25T10:06:48.082982+0000","last_became_active":"2024-04-25T10:06:46.088197+0000","last_became_peered":"2024-04-25T10:06:46.088197+0000","last_unstale":"2024-04-25T10:06:48.082982+0000","last_undegraded":"2024-04-25T10:06:48.082982+0000","last_fullsized":"2024-04-25T10:06:48.082982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.519080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078305+0000","last_change":"2024-04-25T10:06:46.807437+0000","last_active":"2024-04-25T10:06:48.078305+0000","last_peered":"2024-04-25T10:06:48.078305+0000","last_clean":"2024-04-25T10:06:48.078305+0000","last_became_active":"2024-04-25T10:06:46.806601+0000","last_became_peered":"2024-04-25T10:06:46.806601+0000","last_unstale":"2024-04-25T10:06:48.078305+0000","last_undegraded":"2024-04-25T10:06:48.078305+0000","last_fullsized":"2024-04-25T10:06:48.078305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578947+0000","last_change":"2024-04-25T10:06:46.108323+0000","last_active":"2024-04-25T10:06:48.578947+0000","last_peered":"2024-04-25T10:06:48.578947+0000","last_clean":"2024-04-25T10:06:48.578947+0000","last_became_active":"2024-04-25T10:06:46.107822+0000","last_became_peered":"2024-04-25T10:06:46.107822+0000","last_unstale":"2024-04-25T10:06:48.578947+0000","last_undegraded":"2024-04-25T10:06:48.578947+0000","last_fullsized":"2024-04-25T10:06:48.578947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766972+0000","last_change":"2024-04-25T10:06:46.803343+0000","last_active":"2024-04-25T10:06:47.766972+0000","last_peered":"2024-04-25T10:06:47.766972+0000","last_clean":"2024-04-25T10:06:47.766972+0000","last_became_active":"2024-04-25T10:06:46.798722+0000","last_became_peered":"2024-04-25T10:06:46.798722+0000","last_unstale":"2024-04-25T10:06:47.766972+0000","last_undegraded":"2024-04-25T10:06:47.766972+0000","last_fullsized":"2024-04-25T10:06:47.766972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.202660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578371+0000","last_change":"2024-04-25T10:06:46.113290+0000","last_active":"2024-04-25T10:06:48.578371+0000","last_peered":"2024-04-25T10:06:48.578371+0000","last_clean":"2024-04-25T10:06:48.578371+0000","last_became_active":"2024-04-25T10:06:46.112709+0000","last_became_peered":"2024-04-25T10:06:46.112709+0000","last_unstale":"2024-04-25T10:06:48.578371+0000","last_undegraded":"2024-04-25T10:06:48.578371+0000","last_fullsized":"2024-04-25T10:06:48.578371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767427+0000","last_change":"2024-04-25T10:06:46.802403+0000","last_active":"2024-04-25T10:06:47.767427+0000","last_peered":"2024-04-25T10:06:47.767427+0000","last_clean":"2024-04-25T10:06:47.767427+0000","last_became_active":"2024-04-25T10:06:46.793094+0000","last_became_peered":"2024-04-25T10:06:46.793094+0000","last_unstale":"2024-04-25T10:06:47.767427+0000","last_undegraded":"2024-04-25T10:06:47.767427+0000","last_fullsized":"2024-04-25T10:06:47.767427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:09.368323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765745+0000","last_change":"2024-04-25T10:06:46.106011+0000","last_active":"2024-04-25T10:06:47.765745+0000","last_peered":"2024-04-25T10:06:47.765745+0000","last_clean":"2024-04-25T10:06:47.765745+0000","last_became_active":"2024-04-25T10:06:46.105343+0000","last_became_peered":"2024-04-25T10:06:46.105343+0000","last_unstale":"2024-04-25T10:06:47.765745+0000","last_undegraded":"2024-04-25T10:06:47.765745+0000","last_fullsized":"2024-04-25T10:06:47.765745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078833+0000","last_change":"2024-04-25T10:06:46.815307+0000","last_active":"2024-04-25T10:06:48.078833+0000","last_peered":"2024-04-25T10:06:48.078833+0000","last_clean":"2024-04-25T10:06:48.078833+0000","last_became_active":"2024-04-25T10:06:46.814081+0000","last_became_peered":"2024-04-25T10:06:46.814081+0000","last_unstale":"2024-04-25T10:06:48.078833+0000","last_undegraded":"2024-04-25T10:06:48.078833+0000","last_fullsized":"2024-04-25T10:06:48.078833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:17.800835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767873+0000","last_change":"2024-04-25T10:06:46.110424+0000","last_active":"2024-04-25T10:06:47.767873+0000","last_peered":"2024-04-25T10:06:47.767873+0000","last_clean":"2024-04-25T10:06:47.767873+0000","last_became_active":"2024-04-25T10:06:46.109677+0000","last_became_peered":"2024-04-25T10:06:46.109677+0000","last_unstale":"2024-04-25T10:06:47.767873+0000","last_undegraded":"2024-04-25T10:06:47.767873+0000","last_fullsized":"2024-04-25T10:06:47.767873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.659604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078976+0000","last_change":"2024-04-25T10:06:46.807062+0000","last_active":"2024-04-25T10:06:48.078976+0000","last_peered":"2024-04-25T10:06:48.078976+0000","last_clean":"2024-04-25T10:06:48.078976+0000","last_became_active":"2024-04-25T10:06:46.800704+0000","last_became_peered":"2024-04-25T10:06:46.800704+0000","last_unstale":"2024-04-25T10:06:48.078976+0000","last_undegraded":"2024-04-25T10:06:48.078976+0000","last_fullsized":"2024-04-25T10:06:48.078976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:12.930467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577283+0000","last_change":"2024-04-25T10:06:46.088468+0000","last_active":"2024-04-25T10:06:48.577283+0000","last_peered":"2024-04-25T10:06:48.577283+0000","last_clean":"2024-04-25T10:06:48.577283+0000","last_became_active":"2024-04-25T10:06:46.087685+0000","last_became_peered":"2024-04-25T10:06:46.087685+0000","last_unstale":"2024-04-25T10:06:48.577283+0000","last_undegraded":"2024-04-25T10:06:48.577283+0000","last_fullsized":"2024-04-25T10:06:48.577283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767923+0000","last_change":"2024-04-25T10:06:46.774632+0000","last_active":"2024-04-25T10:06:47.767923+0000","last_peered":"2024-04-25T10:06:47.767923+0000","last_clean":"2024-04-25T10:06:47.767923+0000","last_became_active":"2024-04-25T10:06:46.773829+0000","last_became_peered":"2024-04-25T10:06:46.773829+0000","last_unstale":"2024-04-25T10:06:47.767923+0000","last_undegraded":"2024-04-25T10:06:47.767923+0000","last_fullsized":"2024-04-25T10:06:47.767923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078146+0000","last_change":"2024-04-25T10:06:46.103414+0000","last_active":"2024-04-25T10:06:48.078146+0000","last_peered":"2024-04-25T10:06:48.078146+0000","last_clean":"2024-04-25T10:06:48.078146+0000","last_became_active":"2024-04-25T10:06:46.103164+0000","last_became_peered":"2024-04-25T10:06:46.103164+0000","last_unstale":"2024-04-25T10:06:48.078146+0000","last_undegraded":"2024-04-25T10:06:48.078146+0000","last_fullsized":"2024-04-25T10:06:48.078146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765294+0000","last_change":"2024-04-25T10:06:46.794249+0000","last_active":"2024-04-25T10:06:47.765294+0000","last_peered":"2024-04-25T10:06:47.765294+0000","last_clean":"2024-04-25T10:06:47.765294+0000","last_became_active":"2024-04-25T10:06:46.792753+0000","last_became_peered":"2024-04-25T10:06:46.792753+0000","last_unstale":"2024-04-25T10:06:47.765294+0000","last_undegraded":"2024-04-25T10:06:47.765294+0000","last_fullsized":"2024-04-25T10:06:47.765294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081812+0000","last_change":"2024-04-25T10:06:46.102417+0000","last_active":"2024-04-25T10:06:48.081812+0000","last_peered":"2024-04-25T10:06:48.081812+0000","last_clean":"2024-04-25T10:06:48.081812+0000","last_became_active":"2024-04-25T10:06:46.101880+0000","last_became_peered":"2024-04-25T10:06:46.101880+0000","last_unstale":"2024-04-25T10:06:48.081812+0000","last_undegraded":"2024-04-25T10:06:48.081812+0000","last_fullsized":"2024-04-25T10:06:48.081812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:27.992158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077906+0000","last_change":"2024-04-25T10:06:46.800819+0000","last_active":"2024-04-25T10:06:48.077906+0000","last_peered":"2024-04-25T10:06:48.077906+0000","last_clean":"2024-04-25T10:06:48.077906+0000","last_became_active":"2024-04-25T10:06:46.799180+0000","last_became_peered":"2024-04-25T10:06:46.799180+0000","last_unstale":"2024-04-25T10:06:48.077906+0000","last_undegraded":"2024-04-25T10:06:48.077906+0000","last_fullsized":"2024-04-25T10:06:48.077906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579230+0000","last_change":"2024-04-25T10:06:46.108693+0000","last_active":"2024-04-25T10:06:48.579230+0000","last_peered":"2024-04-25T10:06:48.579230+0000","last_clean":"2024-04-25T10:06:48.579230+0000","last_became_active":"2024-04-25T10:06:46.108114+0000","last_became_peered":"2024-04-25T10:06:46.108114+0000","last_unstale":"2024-04-25T10:06:48.579230+0000","last_undegraded":"2024-04-25T10:06:48.579230+0000","last_fullsized":"2024-04-25T10:06:48.579230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:05.951567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577767+0000","last_change":"2024-04-25T10:06:46.788689+0000","last_active":"2024-04-25T10:06:48.577767+0000","last_peered":"2024-04-25T10:06:48.577767+0000","last_clean":"2024-04-25T10:06:48.577767+0000","last_became_active":"2024-04-25T10:06:46.786763+0000","last_became_peered":"2024-04-25T10:06:46.786763+0000","last_unstale":"2024-04-25T10:06:48.577767+0000","last_undegraded":"2024-04-25T10:06:48.577767+0000","last_fullsized":"2024-04-25T10:06:48.577767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:38.545499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578022+0000","last_change":"2024-04-25T10:06:46.103056+0000","last_active":"2024-04-25T10:06:48.578022+0000","last_peered":"2024-04-25T10:06:48.578022+0000","last_clean":"2024-04-25T10:06:48.578022+0000","last_became_active":"2024-04-25T10:06:46.102214+0000","last_became_peered":"2024-04-25T10:06:46.102214+0000","last_unstale":"2024-04-25T10:06:48.578022+0000","last_undegraded":"2024-04-25T10:06:48.578022+0000","last_fullsized":"2024-04-25T10:06:48.578022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579421+0000","last_change":"2024-04-25T10:06:46.795073+0000","last_active":"2024-04-25T10:06:48.579421+0000","last_peered":"2024-04-25T10:06:48.579421+0000","last_clean":"2024-04-25T10:06:48.579421+0000","last_became_active":"2024-04-25T10:06:46.793560+0000","last_became_peered":"2024-04-25T10:06:46.793560+0000","last_unstale":"2024-04-25T10:06:48.579421+0000","last_undegraded":"2024-04-25T10:06:48.579421+0000","last_fullsized":"2024-04-25T10:06:48.579421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082711+0000","last_change":"2024-04-25T10:06:46.085445+0000","last_active":"2024-04-25T10:06:48.082711+0000","last_peered":"2024-04-25T10:06:48.082711+0000","last_clean":"2024-04-25T10:06:48.082711+0000","last_became_active":"2024-04-25T10:06:46.085042+0000","last_became_peered":"2024-04-25T10:06:46.085042+0000","last_unstale":"2024-04-25T10:06:48.082711+0000","last_undegraded":"2024-04-25T10:06:48.082711+0000","last_fullsized":"2024-04-25T10:06:48.082711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:22.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766428+0000","last_change":"2024-04-25T10:06:46.814843+0000","last_active":"2024-04-25T10:06:47.766428+0000","last_peered":"2024-04-25T10:06:47.766428+0000","last_clean":"2024-04-25T10:06:47.766428+0000","last_became_active":"2024-04-25T10:06:46.814095+0000","last_became_peered":"2024-04-25T10:06:46.814095+0000","last_unstale":"2024-04-25T10:06:47.766428+0000","last_undegraded":"2024-04-25T10:06:47.766428+0000","last_fullsized":"2024-04-25T10:06:47.766428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.490689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765998+0000","last_change":"2024-04-25T10:06:46.099679+0000","last_active":"2024-04-25T10:06:47.765998+0000","last_peered":"2024-04-25T10:06:47.765998+0000","last_clean":"2024-04-25T10:06:47.765998+0000","last_became_active":"2024-04-25T10:06:46.098811+0000","last_became_peered":"2024-04-25T10:06:46.098811+0000","last_unstale":"2024-04-25T10:06:47.765998+0000","last_undegraded":"2024-04-25T10:06:47.765998+0000","last_fullsized":"2024-04-25T10:06:47.765998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.533772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578436+0000","last_change":"2024-04-25T10:06:46.783730+0000","last_active":"2024-04-25T10:06:48.578436+0000","last_peered":"2024-04-25T10:06:48.578436+0000","last_clean":"2024-04-25T10:06:48.578436+0000","last_became_active":"2024-04-25T10:06:46.779298+0000","last_became_peered":"2024-04-25T10:06:46.779298+0000","last_unstale":"2024-04-25T10:06:48.578436+0000","last_undegraded":"2024-04-25T10:06:48.578436+0000","last_fullsized":"2024-04-25T10:06:48.578436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577400+0000","last_change":"2024-04-25T10:06:46.097662+0000","last_active":"2024-04-25T10:06:48.577400+0000","last_peered":"2024-04-25T10:06:48.577400+0000","last_clean":"2024-04-25T10:06:48.577400+0000","last_became_active":"2024-04-25T10:06:46.096906+0000","last_became_peered":"2024-04-25T10:06:46.096906+0000","last_unstale":"2024-04-25T10:06:48.577400+0000","last_undegraded":"2024-04-25T10:06:48.577400+0000","last_fullsized":"2024-04-25T10:06:48.577400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766293+0000","last_change":"2024-04-25T10:06:46.793225+0000","last_active":"2024-04-25T10:06:47.766293+0000","last_peered":"2024-04-25T10:06:47.766293+0000","last_clean":"2024-04-25T10:06:47.766293+0000","last_became_active":"2024-04-25T10:06:46.792341+0000","last_became_peered":"2024-04-25T10:06:46.792341+0000","last_unstale":"2024-04-25T10:06:47.766293+0000","last_undegraded":"2024-04-25T10:06:47.766293+0000","last_fullsized":"2024-04-25T10:06:47.766293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.586619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578713+0000","last_change":"2024-04-25T10:06:46.112336+0000","last_active":"2024-04-25T10:06:48.578713+0000","last_peered":"2024-04-25T10:06:48.578713+0000","last_clean":"2024-04-25T10:06:48.578713+0000","last_became_active":"2024-04-25T10:06:46.108377+0000","last_became_peered":"2024-04-25T10:06:46.108377+0000","last_unstale":"2024-04-25T10:06:48.578713+0000","last_undegraded":"2024-04-25T10:06:48.578713+0000","last_fullsized":"2024-04-25T10:06:48.578713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765621+0000","last_change":"2024-04-25T10:06:46.791957+0000","last_active":"2024-04-25T10:06:47.765621+0000","last_peered":"2024-04-25T10:06:47.765621+0000","last_clean":"2024-04-25T10:06:47.765621+0000","last_became_active":"2024-04-25T10:06:46.789267+0000","last_became_peered":"2024-04-25T10:06:46.789267+0000","last_unstale":"2024-04-25T10:06:47.765621+0000","last_undegraded":"2024-04-25T10:06:47.765621+0000","last_fullsized":"2024-04-25T10:06:47.765621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:11.919154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.095569+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.095284+0000","last_became_peered":"2024-04-25T10:06:46.095284+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:49.409583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765571+0000","last_change":"2024-04-25T10:06:46.797079+0000","last_active":"2024-04-25T10:06:47.765571+0000","last_peered":"2024-04-25T10:06:47.765571+0000","last_clean":"2024-04-25T10:06:47.765571+0000","last_became_active":"2024-04-25T10:06:46.796377+0000","last_became_peered":"2024-04-25T10:06:46.796377+0000","last_unstale":"2024-04-25T10:06:47.765571+0000","last_undegraded":"2024-04-25T10:06:47.765571+0000","last_fullsized":"2024-04-25T10:06:47.765571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:08.421167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077979+0000","last_change":"2024-04-25T10:06:46.096666+0000","last_active":"2024-04-25T10:06:48.077979+0000","last_peered":"2024-04-25T10:06:48.077979+0000","last_clean":"2024-04-25T10:06:48.077979+0000","last_became_active":"2024-04-25T10:06:46.096084+0000","last_became_peered":"2024-04-25T10:06:46.096084+0000","last_unstale":"2024-04-25T10:06:48.077979+0000","last_undegraded":"2024-04-25T10:06:48.077979+0000","last_fullsized":"2024-04-25T10:06:48.077979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578330+0000","last_change":"2024-04-25T10:06:46.789487+0000","last_active":"2024-04-25T10:06:48.578330+0000","last_peered":"2024-04-25T10:06:48.578330+0000","last_clean":"2024-04-25T10:06:48.578330+0000","last_became_active":"2024-04-25T10:06:46.788801+0000","last_became_peered":"2024-04-25T10:06:46.788801+0000","last_unstale":"2024-04-25T10:06:48.578330+0000","last_undegraded":"2024-04-25T10:06:48.578330+0000","last_fullsized":"2024-04-25T10:06:48.578330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579292+0000","last_change":"2024-04-25T10:06:46.109437+0000","last_active":"2024-04-25T10:06:48.579292+0000","last_peered":"2024-04-25T10:06:48.579292+0000","last_clean":"2024-04-25T10:06:48.579292+0000","last_became_active":"2024-04-25T10:06:46.108802+0000","last_became_peered":"2024-04-25T10:06:46.108802+0000","last_unstale":"2024-04-25T10:06:48.579292+0000","last_undegraded":"2024-04-25T10:06:48.579292+0000","last_fullsized":"2024-04-25T10:06:48.579292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:27.950431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081989+0000","last_change":"2024-04-25T10:06:46.794307+0000","last_active":"2024-04-25T10:06:48.081989+0000","last_peered":"2024-04-25T10:06:48.081989+0000","last_clean":"2024-04-25T10:06:48.081989+0000","last_became_active":"2024-04-25T10:06:46.793463+0000","last_became_peered":"2024-04-25T10:06:46.793463+0000","last_unstale":"2024-04-25T10:06:48.081989+0000","last_undegraded":"2024-04-25T10:06:48.081989+0000","last_fullsized":"2024-04-25T10:06:48.081989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:13.222579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767889+0000","last_change":"2024-04-25T10:06:46.098133+0000","last_active":"2024-04-25T10:06:47.767889+0000","last_peered":"2024-04-25T10:06:47.767889+0000","last_clean":"2024-04-25T10:06:47.767889+0000","last_became_active":"2024-04-25T10:06:46.097273+0000","last_became_peered":"2024-04-25T10:06:46.097273+0000","last_unstale":"2024-04-25T10:06:47.767889+0000","last_undegraded":"2024-04-25T10:06:47.767889+0000","last_fullsized":"2024-04-25T10:06:47.767889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:56.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082265+0000","last_change":"2024-04-25T10:06:46.801431+0000","last_active":"2024-04-25T10:06:48.082265+0000","last_peered":"2024-04-25T10:06:48.082265+0000","last_clean":"2024-04-25T10:06:48.082265+0000","last_became_active":"2024-04-25T10:06:46.800621+0000","last_became_peered":"2024-04-25T10:06:46.800621+0000","last_unstale":"2024-04-25T10:06:48.082265+0000","last_undegraded":"2024-04-25T10:06:48.082265+0000","last_fullsized":"2024-04-25T10:06:48.082265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:22.445395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082645+0000","last_change":"2024-04-25T10:06:46.098539+0000","last_active":"2024-04-25T10:06:48.082645+0000","last_peered":"2024-04-25T10:06:48.082645+0000","last_clean":"2024-04-25T10:06:48.082645+0000","last_became_active":"2024-04-25T10:06:46.098190+0000","last_became_peered":"2024-04-25T10:06:46.098190+0000","last_unstale":"2024-04-25T10:06:48.082645+0000","last_undegraded":"2024-04-25T10:06:48.082645+0000","last_fullsized":"2024-04-25T10:06:48.082645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:13.339487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.794288+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.788351+0000","last_became_peered":"2024-04-25T10:06:46.788351+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766444+0000","last_change":"2024-04-25T10:06:46.086234+0000","last_active":"2024-04-25T10:06:47.766444+0000","last_peered":"2024-04-25T10:06:47.766444+0000","last_clean":"2024-04-25T10:06:47.766444+0000","last_became_active":"2024-04-25T10:06:46.085918+0000","last_became_peered":"2024-04-25T10:06:46.085918+0000","last_unstale":"2024-04-25T10:06:47.766444+0000","last_undegraded":"2024-04-25T10:06:47.766444+0000","last_fullsized":"2024-04-25T10:06:47.766444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078587+0000","last_change":"2024-04-25T10:06:46.808180+0000","last_active":"2024-04-25T10:06:48.078587+0000","last_peered":"2024-04-25T10:06:48.078587+0000","last_clean":"2024-04-25T10:06:48.078587+0000","last_became_active":"2024-04-25T10:06:46.800573+0000","last_became_peered":"2024-04-25T10:06:46.800573+0000","last_unstale":"2024-04-25T10:06:48.078587+0000","last_undegraded":"2024-04-25T10:06:48.078587+0000","last_fullsized":"2024-04-25T10:06:48.078587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:22.585824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578979+0000","last_change":"2024-04-25T10:06:46.089883+0000","last_active":"2024-04-25T10:06:48.578979+0000","last_peered":"2024-04-25T10:06:48.578979+0000","last_clean":"2024-04-25T10:06:48.578979+0000","last_became_active":"2024-04-25T10:06:46.089591+0000","last_became_peered":"2024-04-25T10:06:46.089591+0000","last_unstale":"2024-04-25T10:06:48.578979+0000","last_undegraded":"2024-04-25T10:06:48.578979+0000","last_fullsized":"2024-04-25T10:06:48.578979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078910+0000","last_change":"2024-04-25T10:06:46.808415+0000","last_active":"2024-04-25T10:06:48.078910+0000","last_peered":"2024-04-25T10:06:48.078910+0000","last_clean":"2024-04-25T10:06:48.078910+0000","last_became_active":"2024-04-25T10:06:46.807612+0000","last_became_peered":"2024-04-25T10:06:46.807612+0000","last_unstale":"2024-04-25T10:06:48.078910+0000","last_undegraded":"2024-04-25T10:06:48.078910+0000","last_fullsized":"2024-04-25T10:06:48.078910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:06.294138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578529+0000","last_change":"2024-04-25T10:06:46.099936+0000","last_active":"2024-04-25T10:06:48.578529+0000","last_peered":"2024-04-25T10:06:48.578529+0000","last_clean":"2024-04-25T10:06:48.578529+0000","last_became_active":"2024-04-25T10:06:46.098806+0000","last_became_peered":"2024-04-25T10:06:46.098806+0000","last_unstale":"2024-04-25T10:06:48.578529+0000","last_undegraded":"2024-04-25T10:06:48.578529+0000","last_fullsized":"2024-04-25T10:06:48.578529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765840+0000","last_change":"2024-04-25T10:06:46.793643+0000","last_active":"2024-04-25T10:06:47.765840+0000","last_peered":"2024-04-25T10:06:47.765840+0000","last_clean":"2024-04-25T10:06:47.765840+0000","last_became_active":"2024-04-25T10:06:46.791775+0000","last_became_peered":"2024-04-25T10:06:46.791775+0000","last_unstale":"2024-04-25T10:06:47.765840+0000","last_undegraded":"2024-04-25T10:06:47.765840+0000","last_fullsized":"2024-04-25T10:06:47.765840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:15.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578533+0000","last_change":"2024-04-25T10:06:46.099543+0000","last_active":"2024-04-25T10:06:48.578533+0000","last_peered":"2024-04-25T10:06:48.578533+0000","last_clean":"2024-04-25T10:06:48.578533+0000","last_became_active":"2024-04-25T10:06:46.099269+0000","last_became_peered":"2024-04-25T10:06:46.099269+0000","last_unstale":"2024-04-25T10:06:48.578533+0000","last_undegraded":"2024-04-25T10:06:48.578533+0000","last_fullsized":"2024-04-25T10:06:48.578533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764669+0000","last_change":"2024-04-25T10:06:46.798507+0000","last_active":"2024-04-25T10:06:47.764669+0000","last_peered":"2024-04-25T10:06:47.764669+0000","last_clean":"2024-04-25T10:06:47.764669+0000","last_became_active":"2024-04-25T10:06:46.797703+0000","last_became_peered":"2024-04-25T10:06:46.797703+0000","last_unstale":"2024-04-25T10:06:47.764669+0000","last_undegraded":"2024-04-25T10:06:47.764669+0000","last_fullsized":"2024-04-25T10:06:47.764669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:41.538388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577874+0000","last_change":"2024-04-25T10:06:46.090113+0000","last_active":"2024-04-25T10:06:48.577874+0000","last_peered":"2024-04-25T10:06:48.577874+0000","last_clean":"2024-04-25T10:06:48.577874+0000","last_became_active":"2024-04-25T10:06:46.089353+0000","last_became_peered":"2024-04-25T10:06:46.089353+0000","last_unstale":"2024-04-25T10:06:48.577874+0000","last_undegraded":"2024-04-25T10:06:48.577874+0000","last_fullsized":"2024-04-25T10:06:48.577874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765642+0000","last_change":"2024-04-25T10:06:46.797032+0000","last_active":"2024-04-25T10:06:47.765642+0000","last_peered":"2024-04-25T10:06:47.765642+0000","last_clean":"2024-04-25T10:06:47.765642+0000","last_became_active":"2024-04-25T10:06:46.796310+0000","last_became_peered":"2024-04-25T10:06:46.796310+0000","last_unstale":"2024-04-25T10:06:47.765642+0000","last_undegraded":"2024-04-25T10:06:47.765642+0000","last_fullsized":"2024-04-25T10:06:47.765642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:39.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.080225+0000","last_change":"2024-04-25T10:06:46.087236+0000","last_active":"2024-04-25T10:06:48.080225+0000","last_peered":"2024-04-25T10:06:48.080225+0000","last_clean":"2024-04-25T10:06:48.080225+0000","last_became_active":"2024-04-25T10:06:46.086354+0000","last_became_peered":"2024-04-25T10:06:46.086354+0000","last_unstale":"2024-04-25T10:06:48.080225+0000","last_undegraded":"2024-04-25T10:06:48.080225+0000","last_fullsized":"2024-04-25T10:06:48.080225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766980+0000","last_change":"2024-04-25T10:06:46.801971+0000","last_active":"2024-04-25T10:06:47.766980+0000","last_peered":"2024-04-25T10:06:47.766980+0000","last_clean":"2024-04-25T10:06:47.766980+0000","last_became_active":"2024-04-25T10:06:46.798311+0000","last_became_peered":"2024-04-25T10:06:46.798311+0000","last_unstale":"2024-04-25T10:06:47.766980+0000","last_undegraded":"2024-04-25T10:06:47.766980+0000","last_fullsized":"2024-04-25T10:06:47.766980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:55.984280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579054+0000","last_change":"2024-04-25T10:06:46.113284+0000","last_active":"2024-04-25T10:06:48.579054+0000","last_peered":"2024-04-25T10:06:48.579054+0000","last_clean":"2024-04-25T10:06:48.579054+0000","last_became_active":"2024-04-25T10:06:46.112729+0000","last_became_peered":"2024-04-25T10:06:46.112729+0000","last_unstale":"2024-04-25T10:06:48.579054+0000","last_undegraded":"2024-04-25T10:06:48.579054+0000","last_fullsized":"2024-04-25T10:06:48.579054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078514+0000","last_change":"2024-04-25T10:06:46.806654+0000","last_active":"2024-04-25T10:06:48.078514+0000","last_peered":"2024-04-25T10:06:48.078514+0000","last_clean":"2024-04-25T10:06:48.078514+0000","last_became_active":"2024-04-25T10:06:46.799360+0000","last_became_peered":"2024-04-25T10:06:46.799360+0000","last_unstale":"2024-04-25T10:06:48.078514+0000","last_undegraded":"2024-04-25T10:06:48.078514+0000","last_fullsized":"2024-04-25T10:06:48.078514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578334+0000","last_change":"2024-04-25T10:06:46.109473+0000","last_active":"2024-04-25T10:06:48.578334+0000","last_peered":"2024-04-25T10:06:48.578334+0000","last_clean":"2024-04-25T10:06:48.578334+0000","last_became_active":"2024-04-25T10:06:46.108640+0000","last_became_peered":"2024-04-25T10:06:46.108640+0000","last_unstale":"2024-04-25T10:06:48.578334+0000","last_undegraded":"2024-04-25T10:06:48.578334+0000","last_fullsized":"2024-04-25T10:06:48.578334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765464+0000","last_change":"2024-04-25T10:06:46.786613+0000","last_active":"2024-04-25T10:06:47.765464+0000","last_peered":"2024-04-25T10:06:47.765464+0000","last_clean":"2024-04-25T10:06:47.765464+0000","last_became_active":"2024-04-25T10:06:46.785785+0000","last_became_peered":"2024-04-25T10:06:46.785785+0000","last_unstale":"2024-04-25T10:06:47.765464+0000","last_undegraded":"2024-04-25T10:06:47.765464+0000","last_fullsized":"2024-04-25T10:06:47.765464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:15.393383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764735+0000","last_change":"2024-04-25T10:06:46.100079+0000","last_active":"2024-04-25T10:06:47.764735+0000","last_peered":"2024-04-25T10:06:47.764735+0000","last_clean":"2024-04-25T10:06:47.764735+0000","last_became_active":"2024-04-25T10:06:46.099419+0000","last_became_peered":"2024-04-25T10:06:46.099419+0000","last_unstale":"2024-04-25T10:06:47.764735+0000","last_undegraded":"2024-04-25T10:06:47.764735+0000","last_fullsized":"2024-04-25T10:06:47.764735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765365+0000","last_change":"2024-04-25T10:06:46.793298+0000","last_active":"2024-04-25T10:06:47.765365+0000","last_peered":"2024-04-25T10:06:47.765365+0000","last_clean":"2024-04-25T10:06:47.765365+0000","last_became_active":"2024-04-25T10:06:46.791603+0000","last_became_peered":"2024-04-25T10:06:46.791603+0000","last_unstale":"2024-04-25T10:06:47.765365+0000","last_undegraded":"2024-04-25T10:06:47.765365+0000","last_fullsized":"2024-04-25T10:06:47.765365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082282+0000","last_change":"2024-04-25T10:06:46.088827+0000","last_active":"2024-04-25T10:06:48.082282+0000","last_peered":"2024-04-25T10:06:48.082282+0000","last_clean":"2024-04-25T10:06:48.082282+0000","last_became_active":"2024-04-25T10:06:46.088541+0000","last_became_peered":"2024-04-25T10:06:46.088541+0000","last_unstale":"2024-04-25T10:06:48.082282+0000","last_undegraded":"2024-04-25T10:06:48.082282+0000","last_fullsized":"2024-04-25T10:06:48.082282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:37.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766800+0000","last_change":"2024-04-25T10:06:46.815568+0000","last_active":"2024-04-25T10:06:47.766800+0000","last_peered":"2024-04-25T10:06:47.766800+0000","last_clean":"2024-04-25T10:06:47.766800+0000","last_became_active":"2024-04-25T10:06:46.814861+0000","last_became_peered":"2024-04-25T10:06:46.814861+0000","last_unstale":"2024-04-25T10:06:47.766800+0000","last_undegraded":"2024-04-25T10:06:47.766800+0000","last_fullsized":"2024-04-25T10:06:47.766800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765886+0000","last_change":"2024-04-25T10:06:46.092480+0000","last_active":"2024-04-25T10:06:47.765886+0000","last_peered":"2024-04-25T10:06:47.765886+0000","last_clean":"2024-04-25T10:06:47.765886+0000","last_became_active":"2024-04-25T10:06:46.091763+0000","last_became_peered":"2024-04-25T10:06:46.091763+0000","last_unstale":"2024-04-25T10:06:47.765886+0000","last_undegraded":"2024-04-25T10:06:47.765886+0000","last_fullsized":"2024-04-25T10:06:47.765886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:44.359393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766985+0000","last_change":"2024-04-25T10:06:46.783297+0000","last_active":"2024-04-25T10:06:47.766985+0000","last_peered":"2024-04-25T10:06:47.766985+0000","last_clean":"2024-04-25T10:06:47.766985+0000","last_became_active":"2024-04-25T10:06:46.782524+0000","last_became_peered":"2024-04-25T10:06:46.782524+0000","last_unstale":"2024-04-25T10:06:47.766985+0000","last_undegraded":"2024-04-25T10:06:47.766985+0000","last_fullsized":"2024-04-25T10:06:47.766985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766127+0000","last_change":"2024-04-25T10:06:46.079371+0000","last_active":"2024-04-25T10:06:47.766127+0000","last_peered":"2024-04-25T10:06:47.766127+0000","last_clean":"2024-04-25T10:06:47.766127+0000","last_became_active":"2024-04-25T10:06:46.079020+0000","last_became_peered":"2024-04-25T10:06:46.079020+0000","last_unstale":"2024-04-25T10:06:47.766127+0000","last_undegraded":"2024-04-25T10:06:47.766127+0000","last_fullsized":"2024-04-25T10:06:47.766127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.805681+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.805076+0000","last_became_peered":"2024-04-25T10:06:46.805076+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579678+0000","last_change":"2024-04-25T10:06:46.099980+0000","last_active":"2024-04-25T10:06:48.579678+0000","last_peered":"2024-04-25T10:06:48.579678+0000","last_clean":"2024-04-25T10:06:48.579678+0000","last_became_active":"2024-04-25T10:06:46.099518+0000","last_became_peered":"2024-04-25T10:06:46.099518+0000","last_unstale":"2024-04-25T10:06:48.579678+0000","last_undegraded":"2024-04-25T10:06:48.579678+0000","last_fullsized":"2024-04-25T10:06:48.579678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:15.012055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578707+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:06:48.578707+0000","last_peered":"2024-04-25T10:06:48.578707+0000","last_clean":"2024-04-25T10:06:48.578707+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:06:48.578707+0000","last_undegraded":"2024-04-25T10:06:48.578707+0000","last_fullsized":"2024-04-25T10:06:48.578707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1032685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574911,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":443907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574912,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":444011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574911,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":443037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":442407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":1032663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:55.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.298+0000 7f22a6005640 1 Processor -- start 2024-04-25T10:06:55.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.298+0000 7f22a6005640 1 -- start start 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f22a6005640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22a015a700 0x7f22a015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f22a6005640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22a00b24a0 0x7f22a00b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f22a6005640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a00b2090 con 0x7f22a00b3f50 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f22a6005640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a015f200 con 0x7f22a00b24a0 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f22a6005640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a00b43e0 con 0x7f22a015a700 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22a00b24a0 0x7f22a00b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22a00b24a0 0x7f22a00b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35982/0 (socket says 172.21.15.82:35982) 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 -- 172.21.15.82:0/1366286552 learned_addr learned my addr 172.21.15.82:0/1366286552 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229f7fe640 1 --2- 172.21.15.82:0/1366286552 >> v2:172.21.15.196:3300/0 conn(0x7f22a015a700 0x7f22a015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 -- 172.21.15.82:0/1366286552 >> v2:172.21.15.196:3300/0 conn(0x7f22a015a700 msgr2=0x7f22a015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 --2- 172.21.15.82:0/1366286552 >> v2:172.21.15.196:3300/0 conn(0x7f22a015a700 0x7f22a015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 -- 172.21.15.82:0/1366286552 >> v1:172.21.15.196:6789/0 conn(0x7f22a00b3f50 legacy=0x7f22a00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 -- 172.21.15.82:0/1366286552 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00b46c0 con 0x7f22a00b24a0 2024-04-25T10:06:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229ffff640 1 --2- 172.21.15.82:0/1366286552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a00b24a0 0x7f22a00b28f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f22a006c980 tx=0x7f229801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4562754570ab429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.302+0000 7f229effd640 1 -- 172.21.15.82:0/1366286552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980a5620 con 0x7f22a00b24a0 2024-04-25T10:06:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f229effd640 1 -- 172.21.15.82:0/1366286552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22980a57c0 con 0x7f22a00b24a0 2024-04-25T10:06:55.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f229effd640 1 -- 172.21.15.82:0/1366286552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22980a5b90 con 0x7f22a00b24a0 2024-04-25T10:06:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- 172.21.15.82:0/1366286552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a00b24a0 msgr2=0x7f22a00b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- 172.21.15.82:0/1366286552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a00b24a0 0x7f22a00b28f0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f22a006c980 tx=0x7f229801c870 comp rx=0 tx=0).stop 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- 172.21.15.82:0/1366286552 shutdown_connections 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- 172.21.15.82:0/1366286552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a00b24a0 0x7f22a00b28f0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- 172.21.15.82:0/1366286552 >> v2:172.21.15.196:3300/0 conn(0x7f22a015a700 0x7f22a015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- 172.21.15.82:0/1366286552 >> 172.21.15.82:0/1366286552 conn(0x7f22a00b11a0 msgr2=0x7f22a00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- 172.21.15.82:0/1366286552 shutdown_connections 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- 172.21.15.82:0/1366286552 wait complete. 2024-04-25T10:06:55.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 Processor -- start 2024-04-25T10:06:55.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- start start 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a00b24a0 0x7f22a013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 0x7f22a01519b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00cd310 con 0x7f22a015a700 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00ce600 con 0x7f22a0151520 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f22a6005640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a00cd490 con 0x7f22a00b24a0 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35988/0 (socket says 172.21.15.82:35988) 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 -- 172.21.15.82:0/2245116347 learned_addr learned my addr 172.21.15.82:0/2245116347 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f229ffff640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 0x7f22a01519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f229f7fe640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a00b24a0 0x7f22a013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 -- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a00b24a0 msgr2=0x7f22a013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a00b24a0 0x7f22a013fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 -- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 msgr2=0x7f22a01519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 0x7f22a01519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f2297fff640 1 -- 172.21.15.82:0/2245116347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a00ae510 con 0x7f22a015a700 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.306+0000 7f229ffff640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 0x7f22a01519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f2297fff640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f228800e760 tx=0x7f228800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288004260 con 0x7f22a015a700 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288004420 con 0x7f22a015a700 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22880116b0 con 0x7f22a015a700 2024-04-25T10:06:55.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a0152300 con 0x7f22a015a700 2024-04-25T10:06:55.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a013cd80 con 0x7f22a015a700 2024-04-25T10:06:55.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a013c720 con 0x7f22a015a700 2024-04-25T10:06:55.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2288011850 con 0x7f22a015a700 2024-04-25T10:06:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2288087a20 con 0x7f22a015a700 2024-04-25T10:06:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229f7fe640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.310+0000 7f229f7fe640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 0x7f2284069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f229000b800 tx=0x7f2290008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.314+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f22a013c720 con 0x7f22a015a700 2024-04-25T10:06:55.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.470+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22a006f260 con 0x7f2284066f60 2024-04-25T10:06:55.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.486+0000 7f229cff9640 1 -- 172.21.15.82:0/2245116347 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251961 (secure 0 0 0) 0x7f22a006f260 con 0x7f2284066f60 2024-04-25T10:06:55.492 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:55.494 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 msgr2=0x7f2284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 0x7f2284069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f229000b800 tx=0x7f2290008040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 msgr2=0x7f22a00adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f228800e760 tx=0x7f228800ec20 comp rx=0 tx=0).stop 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 shutdown_connections 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a0151520 0x7f22a01519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a015a700 0x7f22a00adfd0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 --2- 172.21.15.82:0/2245116347 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a00b24a0 0x7f22a013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 >> 172.21.15.82:0/2245116347 conn(0x7f22a00b11a0 msgr2=0x7f22a015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 shutdown_connections 2024-04-25T10:06:55.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.490+0000 7f22a6005640 1 -- 172.21.15.82:0/2245116347 wait complete. 2024-04-25T10:06:55.510 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-04-25T10:06:55.350610+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258236,"kb_used_data":8776,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716484,"statfs":{"total":773094113280,"available":772829679616,"internally_reserved":0,"allocated":8986624,"data_stored":4723916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.595791"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579140+0000","last_change":"2024-04-25T10:06:46.812042+0000","last_active":"2024-04-25T10:06:48.579140+0000","last_peered":"2024-04-25T10:06:48.579140+0000","last_clean":"2024-04-25T10:06:48.579140+0000","last_became_active":"2024-04-25T10:06:46.811330+0000","last_became_peered":"2024-04-25T10:06:46.811330+0000","last_unstale":"2024-04-25T10:06:48.579140+0000","last_undegraded":"2024-04-25T10:06:48.579140+0000","last_fullsized":"2024-04-25T10:06:48.579140+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078792+0000","last_change":"2024-04-25T10:06:46.814055+0000","last_active":"2024-04-25T10:06:48.078792+0000","last_peered":"2024-04-25T10:06:48.078792+0000","last_clean":"2024-04-25T10:06:48.078792+0000","last_became_active":"2024-04-25T10:06:46.813374+0000","last_became_peered":"2024-04-25T10:06:46.813374+0000","last_unstale":"2024-04-25T10:06:48.078792+0000","last_undegraded":"2024-04-25T10:06:48.078792+0000","last_fullsized":"2024-04-25T10:06:48.078792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:10.433303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766684+0000","last_change":"2024-04-25T10:06:46.816949+0000","last_active":"2024-04-25T10:06:47.766684+0000","last_peered":"2024-04-25T10:06:47.766684+0000","last_clean":"2024-04-25T10:06:47.766684+0000","last_became_active":"2024-04-25T10:06:46.816384+0000","last_became_peered":"2024-04-25T10:06:46.816384+0000","last_unstale":"2024-04-25T10:06:47.766684+0000","last_undegraded":"2024-04-25T10:06:47.766684+0000","last_fullsized":"2024-04-25T10:06:47.766684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:41.271967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578047+0000","last_change":"2024-04-25T10:06:46.782326+0000","last_active":"2024-04-25T10:06:48.578047+0000","last_peered":"2024-04-25T10:06:48.578047+0000","last_clean":"2024-04-25T10:06:48.578047+0000","last_became_active":"2024-04-25T10:06:46.780178+0000","last_became_peered":"2024-04-25T10:06:46.780178+0000","last_unstale":"2024-04-25T10:06:48.578047+0000","last_undegraded":"2024-04-25T10:06:48.578047+0000","last_fullsized":"2024-04-25T10:06:48.578047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:11.581560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082074+0000","last_change":"2024-04-25T10:06:46.786463+0000","last_active":"2024-04-25T10:06:48.082074+0000","last_peered":"2024-04-25T10:06:48.082074+0000","last_clean":"2024-04-25T10:06:48.082074+0000","last_became_active":"2024-04-25T10:06:46.785559+0000","last_became_peered":"2024-04-25T10:06:46.785559+0000","last_unstale":"2024-04-25T10:06:48.082074+0000","last_undegraded":"2024-04-25T10:06:48.082074+0000","last_fullsized":"2024-04-25T10:06:48.082074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:01.174907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579158+0000","last_change":"2024-04-25T10:06:46.816049+0000","last_active":"2024-04-25T10:06:48.579158+0000","last_peered":"2024-04-25T10:06:48.579158+0000","last_clean":"2024-04-25T10:06:48.579158+0000","last_became_active":"2024-04-25T10:06:46.814728+0000","last_became_peered":"2024-04-25T10:06:46.814728+0000","last_unstale":"2024-04-25T10:06:48.579158+0000","last_undegraded":"2024-04-25T10:06:48.579158+0000","last_fullsized":"2024-04-25T10:06:48.579158+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765258+0000","last_change":"2024-04-25T10:06:46.787007+0000","last_active":"2024-04-25T10:06:47.765258+0000","last_peered":"2024-04-25T10:06:47.765258+0000","last_clean":"2024-04-25T10:06:47.765258+0000","last_became_active":"2024-04-25T10:06:46.786255+0000","last_became_peered":"2024-04-25T10:06:46.786255+0000","last_unstale":"2024-04-25T10:06:47.765258+0000","last_undegraded":"2024-04-25T10:06:47.765258+0000","last_fullsized":"2024-04-25T10:06:47.765258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082256+0000","last_change":"2024-04-25T10:06:46.785378+0000","last_active":"2024-04-25T10:06:48.082256+0000","last_peered":"2024-04-25T10:06:48.082256+0000","last_clean":"2024-04-25T10:06:48.082256+0000","last_became_active":"2024-04-25T10:06:46.784540+0000","last_became_peered":"2024-04-25T10:06:46.784540+0000","last_unstale":"2024-04-25T10:06:48.082256+0000","last_undegraded":"2024-04-25T10:06:48.082256+0000","last_fullsized":"2024-04-25T10:06:48.082256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:56.712512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767669+0000","last_change":"2024-04-25T10:06:46.781725+0000","last_active":"2024-04-25T10:06:47.767669+0000","last_peered":"2024-04-25T10:06:47.767669+0000","last_clean":"2024-04-25T10:06:47.767669+0000","last_became_active":"2024-04-25T10:06:46.781039+0000","last_became_peered":"2024-04-25T10:06:46.781039+0000","last_unstale":"2024-04-25T10:06:47.767669+0000","last_undegraded":"2024-04-25T10:06:47.767669+0000","last_fullsized":"2024-04-25T10:06:47.767669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:11.102231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577706+0000","last_change":"2024-04-25T10:06:46.795742+0000","last_active":"2024-04-25T10:06:48.577706+0000","last_peered":"2024-04-25T10:06:48.577706+0000","last_clean":"2024-04-25T10:06:48.577706+0000","last_became_active":"2024-04-25T10:06:46.794090+0000","last_became_peered":"2024-04-25T10:06:46.794090+0000","last_unstale":"2024-04-25T10:06:48.577706+0000","last_undegraded":"2024-04-25T10:06:48.577706+0000","last_fullsized":"2024-04-25T10:06:48.577706+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:12.588290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078775+0000","last_change":"2024-04-25T10:06:46.813315+0000","last_active":"2024-04-25T10:06:48.078775+0000","last_peered":"2024-04-25T10:06:48.078775+0000","last_clean":"2024-04-25T10:06:48.078775+0000","last_became_active":"2024-04-25T10:06:46.800516+0000","last_became_peered":"2024-04-25T10:06:46.800516+0000","last_unstale":"2024-04-25T10:06:48.078775+0000","last_undegraded":"2024-04-25T10:06:48.078775+0000","last_fullsized":"2024-04-25T10:06:48.078775+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:05.005574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578300+0000","last_change":"2024-04-25T10:06:46.813321+0000","last_active":"2024-04-25T10:06:48.578300+0000","last_peered":"2024-04-25T10:06:48.578300+0000","last_clean":"2024-04-25T10:06:48.578300+0000","last_became_active":"2024-04-25T10:06:46.812941+0000","last_became_peered":"2024-04-25T10:06:46.812941+0000","last_unstale":"2024-04-25T10:06:48.578300+0000","last_undegraded":"2024-04-25T10:06:48.578300+0000","last_fullsized":"2024-04-25T10:06:48.578300+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:34.026951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082448+0000","last_change":"2024-04-25T10:06:46.776909+0000","last_active":"2024-04-25T10:06:48.082448+0000","last_peered":"2024-04-25T10:06:48.082448+0000","last_clean":"2024-04-25T10:06:48.082448+0000","last_became_active":"2024-04-25T10:06:46.776153+0000","last_became_peered":"2024-04-25T10:06:46.776153+0000","last_unstale":"2024-04-25T10:06:48.082448+0000","last_undegraded":"2024-04-25T10:06:48.082448+0000","last_fullsized":"2024-04-25T10:06:48.082448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:40.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578644+0000","last_change":"2024-04-25T10:06:46.805290+0000","last_active":"2024-04-25T10:06:48.578644+0000","last_peered":"2024-04-25T10:06:48.578644+0000","last_clean":"2024-04-25T10:06:48.578644+0000","last_became_active":"2024-04-25T10:06:46.799720+0000","last_became_peered":"2024-04-25T10:06:46.799720+0000","last_unstale":"2024-04-25T10:06:48.578644+0000","last_undegraded":"2024-04-25T10:06:48.578644+0000","last_fullsized":"2024-04-25T10:06:48.578644+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078045+0000","last_change":"2024-04-25T10:06:46.807971+0000","last_active":"2024-04-25T10:06:48.078045+0000","last_peered":"2024-04-25T10:06:48.078045+0000","last_clean":"2024-04-25T10:06:48.078045+0000","last_became_active":"2024-04-25T10:06:46.800983+0000","last_became_peered":"2024-04-25T10:06:46.800983+0000","last_unstale":"2024-04-25T10:06:48.078045+0000","last_undegraded":"2024-04-25T10:06:48.078045+0000","last_fullsized":"2024-04-25T10:06:48.078045+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578470+0000","last_change":"2024-04-25T10:06:46.790688+0000","last_active":"2024-04-25T10:06:48.578470+0000","last_peered":"2024-04-25T10:06:48.578470+0000","last_clean":"2024-04-25T10:06:48.578470+0000","last_became_active":"2024-04-25T10:06:46.789841+0000","last_became_peered":"2024-04-25T10:06:46.789841+0000","last_unstale":"2024-04-25T10:06:48.578470+0000","last_undegraded":"2024-04-25T10:06:48.578470+0000","last_fullsized":"2024-04-25T10:06:48.578470+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:07.727264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.083022+0000","last_change":"2024-04-25T10:06:46.803641+0000","last_active":"2024-04-25T10:06:48.083022+0000","last_peered":"2024-04-25T10:06:48.083022+0000","last_clean":"2024-04-25T10:06:48.083022+0000","last_became_active":"2024-04-25T10:06:46.802850+0000","last_became_peered":"2024-04-25T10:06:46.802850+0000","last_unstale":"2024-04-25T10:06:48.083022+0000","last_undegraded":"2024-04-25T10:06:48.083022+0000","last_fullsized":"2024-04-25T10:06:48.083022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:45.971536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766827+0000","last_change":"2024-04-25T10:06:46.782609+0000","last_active":"2024-04-25T10:06:47.766827+0000","last_peered":"2024-04-25T10:06:47.766827+0000","last_clean":"2024-04-25T10:06:47.766827+0000","last_became_active":"2024-04-25T10:06:46.781574+0000","last_became_peered":"2024-04-25T10:06:46.781574+0000","last_unstale":"2024-04-25T10:06:47.766827+0000","last_undegraded":"2024-04-25T10:06:47.766827+0000","last_fullsized":"2024-04-25T10:06:47.766827+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579031+0000","last_change":"2024-04-25T10:06:46.788869+0000","last_active":"2024-04-25T10:06:48.579031+0000","last_peered":"2024-04-25T10:06:48.579031+0000","last_clean":"2024-04-25T10:06:48.579031+0000","last_became_active":"2024-04-25T10:06:46.788132+0000","last_became_peered":"2024-04-25T10:06:46.788132+0000","last_unstale":"2024-04-25T10:06:48.579031+0000","last_undegraded":"2024-04-25T10:06:48.579031+0000","last_fullsized":"2024-04-25T10:06:48.579031+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082318+0000","last_change":"2024-04-25T10:06:46.785801+0000","last_active":"2024-04-25T10:06:48.082318+0000","last_peered":"2024-04-25T10:06:48.082318+0000","last_clean":"2024-04-25T10:06:48.082318+0000","last_became_active":"2024-04-25T10:06:46.784579+0000","last_became_peered":"2024-04-25T10:06:46.784579+0000","last_unstale":"2024-04-25T10:06:48.082318+0000","last_undegraded":"2024-04-25T10:06:48.082318+0000","last_fullsized":"2024-04-25T10:06:48.082318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:46.516701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767026+0000","last_change":"2024-04-25T10:06:46.803669+0000","last_active":"2024-04-25T10:06:47.767026+0000","last_peered":"2024-04-25T10:06:47.767026+0000","last_clean":"2024-04-25T10:06:47.767026+0000","last_became_active":"2024-04-25T10:06:46.802373+0000","last_became_peered":"2024-04-25T10:06:46.802373+0000","last_unstale":"2024-04-25T10:06:47.767026+0000","last_undegraded":"2024-04-25T10:06:47.767026+0000","last_fullsized":"2024-04-25T10:06:47.767026+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:58.982166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.797809+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.797133+0000","last_became_peered":"2024-04-25T10:06:46.797133+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:27.886972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764762+0000","last_change":"2024-04-25T10:06:46.783667+0000","last_active":"2024-04-25T10:06:47.764762+0000","last_peered":"2024-04-25T10:06:47.764762+0000","last_clean":"2024-04-25T10:06:47.764762+0000","last_became_active":"2024-04-25T10:06:46.782958+0000","last_became_peered":"2024-04-25T10:06:46.782958+0000","last_unstale":"2024-04-25T10:06:47.764762+0000","last_undegraded":"2024-04-25T10:06:47.764762+0000","last_fullsized":"2024-04-25T10:06:47.764762+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:45.364008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078138+0000","last_change":"2024-04-25T10:06:46.807236+0000","last_active":"2024-04-25T10:06:48.078138+0000","last_peered":"2024-04-25T10:06:48.078138+0000","last_clean":"2024-04-25T10:06:48.078138+0000","last_became_active":"2024-04-25T10:06:46.800093+0000","last_became_peered":"2024-04-25T10:06:46.800093+0000","last_unstale":"2024-04-25T10:06:48.078138+0000","last_undegraded":"2024-04-25T10:06:48.078138+0000","last_fullsized":"2024-04-25T10:06:48.078138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078877+0000","last_change":"2024-04-25T10:06:46.806479+0000","last_active":"2024-04-25T10:06:48.078877+0000","last_peered":"2024-04-25T10:06:48.078877+0000","last_clean":"2024-04-25T10:06:48.078877+0000","last_became_active":"2024-04-25T10:06:46.800600+0000","last_became_peered":"2024-04-25T10:06:46.800600+0000","last_unstale":"2024-04-25T10:06:48.078877+0000","last_undegraded":"2024-04-25T10:06:48.078877+0000","last_fullsized":"2024-04-25T10:06:48.078877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:06.913908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577901+0000","last_change":"2024-04-25T10:06:46.813601+0000","last_active":"2024-04-25T10:06:48.577901+0000","last_peered":"2024-04-25T10:06:48.577901+0000","last_clean":"2024-04-25T10:06:48.577901+0000","last_became_active":"2024-04-25T10:06:46.812901+0000","last_became_peered":"2024-04-25T10:06:46.812901+0000","last_unstale":"2024-04-25T10:06:48.577901+0000","last_undegraded":"2024-04-25T10:06:48.577901+0000","last_fullsized":"2024-04-25T10:06:48.577901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:15.598255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578852+0000","last_change":"2024-04-25T10:06:46.804827+0000","last_active":"2024-04-25T10:06:48.578852+0000","last_peered":"2024-04-25T10:06:48.578852+0000","last_clean":"2024-04-25T10:06:48.578852+0000","last_became_active":"2024-04-25T10:06:46.804083+0000","last_became_peered":"2024-04-25T10:06:46.804083+0000","last_unstale":"2024-04-25T10:06:48.578852+0000","last_undegraded":"2024-04-25T10:06:48.578852+0000","last_fullsized":"2024-04-25T10:06:48.578852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082404+0000","last_change":"2024-04-25T10:06:46.789728+0000","last_active":"2024-04-25T10:06:48.082404+0000","last_peered":"2024-04-25T10:06:48.082404+0000","last_clean":"2024-04-25T10:06:48.082404+0000","last_became_active":"2024-04-25T10:06:46.788957+0000","last_became_peered":"2024-04-25T10:06:46.788957+0000","last_unstale":"2024-04-25T10:06:48.082404+0000","last_undegraded":"2024-04-25T10:06:48.082404+0000","last_fullsized":"2024-04-25T10:06:48.082404+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.998327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078446+0000","last_change":"2024-04-25T10:06:46.808484+0000","last_active":"2024-04-25T10:06:48.078446+0000","last_peered":"2024-04-25T10:06:48.078446+0000","last_clean":"2024-04-25T10:06:48.078446+0000","last_became_active":"2024-04-25T10:06:46.807681+0000","last_became_peered":"2024-04-25T10:06:46.807681+0000","last_unstale":"2024-04-25T10:06:48.078446+0000","last_undegraded":"2024-04-25T10:06:48.078446+0000","last_fullsized":"2024-04-25T10:06:48.078446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578440+0000","last_change":"2024-04-25T10:06:46.806504+0000","last_active":"2024-04-25T10:06:48.578440+0000","last_peered":"2024-04-25T10:06:48.578440+0000","last_clean":"2024-04-25T10:06:48.578440+0000","last_became_active":"2024-04-25T10:06:46.805910+0000","last_became_peered":"2024-04-25T10:06:46.805910+0000","last_unstale":"2024-04-25T10:06:48.578440+0000","last_undegraded":"2024-04-25T10:06:48.578440+0000","last_fullsized":"2024-04-25T10:06:48.578440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766290+0000","last_change":"2024-04-25T10:06:46.795871+0000","last_active":"2024-04-25T10:06:47.766290+0000","last_peered":"2024-04-25T10:06:47.766290+0000","last_clean":"2024-04-25T10:06:47.766290+0000","last_became_active":"2024-04-25T10:06:46.794978+0000","last_became_peered":"2024-04-25T10:06:46.794978+0000","last_unstale":"2024-04-25T10:06:47.766290+0000","last_undegraded":"2024-04-25T10:06:47.766290+0000","last_fullsized":"2024-04-25T10:06:47.766290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:20.023879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578885+0000","last_change":"2024-04-25T10:06:46.804579+0000","last_active":"2024-04-25T10:06:48.578885+0000","last_peered":"2024-04-25T10:06:48.578885+0000","last_clean":"2024-04-25T10:06:48.578885+0000","last_became_active":"2024-04-25T10:06:46.803668+0000","last_became_peered":"2024-04-25T10:06:46.803668+0000","last_unstale":"2024-04-25T10:06:48.578885+0000","last_undegraded":"2024-04-25T10:06:48.578885+0000","last_fullsized":"2024-04-25T10:06:48.578885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:31.038651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766189+0000","last_change":"2024-04-25T10:06:46.781678+0000","last_active":"2024-04-25T10:06:47.766189+0000","last_peered":"2024-04-25T10:06:47.766189+0000","last_clean":"2024-04-25T10:06:47.766189+0000","last_became_active":"2024-04-25T10:06:46.780704+0000","last_became_peered":"2024-04-25T10:06:46.780704+0000","last_unstale":"2024-04-25T10:06:47.766189+0000","last_undegraded":"2024-04-25T10:06:47.766189+0000","last_fullsized":"2024-04-25T10:06:47.766189+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"78'4","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082982+0000","last_change":"2024-04-25T10:06:46.089223+0000","last_active":"2024-04-25T10:06:48.082982+0000","last_peered":"2024-04-25T10:06:48.082982+0000","last_clean":"2024-04-25T10:06:48.082982+0000","last_became_active":"2024-04-25T10:06:46.088197+0000","last_became_peered":"2024-04-25T10:06:46.088197+0000","last_unstale":"2024-04-25T10:06:48.082982+0000","last_undegraded":"2024-04-25T10:06:48.082982+0000","last_fullsized":"2024-04-25T10:06:48.082982+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:53.519080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078305+0000","last_change":"2024-04-25T10:06:46.807437+0000","last_active":"2024-04-25T10:06:48.078305+0000","last_peered":"2024-04-25T10:06:48.078305+0000","last_clean":"2024-04-25T10:06:48.078305+0000","last_became_active":"2024-04-25T10:06:46.806601+0000","last_became_peered":"2024-04-25T10:06:46.806601+0000","last_unstale":"2024-04-25T10:06:48.078305+0000","last_undegraded":"2024-04-25T10:06:48.078305+0000","last_fullsized":"2024-04-25T10:06:48.078305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:27.830558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578947+0000","last_change":"2024-04-25T10:06:46.108323+0000","last_active":"2024-04-25T10:06:48.578947+0000","last_peered":"2024-04-25T10:06:48.578947+0000","last_clean":"2024-04-25T10:06:48.578947+0000","last_became_active":"2024-04-25T10:06:46.107822+0000","last_became_peered":"2024-04-25T10:06:46.107822+0000","last_unstale":"2024-04-25T10:06:48.578947+0000","last_undegraded":"2024-04-25T10:06:48.578947+0000","last_fullsized":"2024-04-25T10:06:48.578947+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766972+0000","last_change":"2024-04-25T10:06:46.803343+0000","last_active":"2024-04-25T10:06:47.766972+0000","last_peered":"2024-04-25T10:06:47.766972+0000","last_clean":"2024-04-25T10:06:47.766972+0000","last_became_active":"2024-04-25T10:06:46.798722+0000","last_became_peered":"2024-04-25T10:06:46.798722+0000","last_unstale":"2024-04-25T10:06:47.766972+0000","last_undegraded":"2024-04-25T10:06:47.766972+0000","last_fullsized":"2024-04-25T10:06:47.766972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:42.202660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578371+0000","last_change":"2024-04-25T10:06:46.113290+0000","last_active":"2024-04-25T10:06:48.578371+0000","last_peered":"2024-04-25T10:06:48.578371+0000","last_clean":"2024-04-25T10:06:48.578371+0000","last_became_active":"2024-04-25T10:06:46.112709+0000","last_became_peered":"2024-04-25T10:06:46.112709+0000","last_unstale":"2024-04-25T10:06:48.578371+0000","last_undegraded":"2024-04-25T10:06:48.578371+0000","last_fullsized":"2024-04-25T10:06:48.578371+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767427+0000","last_change":"2024-04-25T10:06:46.802403+0000","last_active":"2024-04-25T10:06:47.767427+0000","last_peered":"2024-04-25T10:06:47.767427+0000","last_clean":"2024-04-25T10:06:47.767427+0000","last_became_active":"2024-04-25T10:06:46.793094+0000","last_became_peered":"2024-04-25T10:06:46.793094+0000","last_unstale":"2024-04-25T10:06:47.767427+0000","last_undegraded":"2024-04-25T10:06:47.767427+0000","last_fullsized":"2024-04-25T10:06:47.767427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:09.368323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765745+0000","last_change":"2024-04-25T10:06:46.106011+0000","last_active":"2024-04-25T10:06:47.765745+0000","last_peered":"2024-04-25T10:06:47.765745+0000","last_clean":"2024-04-25T10:06:47.765745+0000","last_became_active":"2024-04-25T10:06:46.105343+0000","last_became_peered":"2024-04-25T10:06:46.105343+0000","last_unstale":"2024-04-25T10:06:47.765745+0000","last_undegraded":"2024-04-25T10:06:47.765745+0000","last_fullsized":"2024-04-25T10:06:47.765745+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078833+0000","last_change":"2024-04-25T10:06:46.815307+0000","last_active":"2024-04-25T10:06:48.078833+0000","last_peered":"2024-04-25T10:06:48.078833+0000","last_clean":"2024-04-25T10:06:48.078833+0000","last_became_active":"2024-04-25T10:06:46.814081+0000","last_became_peered":"2024-04-25T10:06:46.814081+0000","last_unstale":"2024-04-25T10:06:48.078833+0000","last_undegraded":"2024-04-25T10:06:48.078833+0000","last_fullsized":"2024-04-25T10:06:48.078833+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:17.800835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767873+0000","last_change":"2024-04-25T10:06:46.110424+0000","last_active":"2024-04-25T10:06:47.767873+0000","last_peered":"2024-04-25T10:06:47.767873+0000","last_clean":"2024-04-25T10:06:47.767873+0000","last_became_active":"2024-04-25T10:06:46.109677+0000","last_became_peered":"2024-04-25T10:06:46.109677+0000","last_unstale":"2024-04-25T10:06:47.767873+0000","last_undegraded":"2024-04-25T10:06:47.767873+0000","last_fullsized":"2024-04-25T10:06:47.767873+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:07.659604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078976+0000","last_change":"2024-04-25T10:06:46.807062+0000","last_active":"2024-04-25T10:06:48.078976+0000","last_peered":"2024-04-25T10:06:48.078976+0000","last_clean":"2024-04-25T10:06:48.078976+0000","last_became_active":"2024-04-25T10:06:46.800704+0000","last_became_peered":"2024-04-25T10:06:46.800704+0000","last_unstale":"2024-04-25T10:06:48.078976+0000","last_undegraded":"2024-04-25T10:06:48.078976+0000","last_fullsized":"2024-04-25T10:06:48.078976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:12.930467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577283+0000","last_change":"2024-04-25T10:06:46.088468+0000","last_active":"2024-04-25T10:06:48.577283+0000","last_peered":"2024-04-25T10:06:48.577283+0000","last_clean":"2024-04-25T10:06:48.577283+0000","last_became_active":"2024-04-25T10:06:46.087685+0000","last_became_peered":"2024-04-25T10:06:46.087685+0000","last_unstale":"2024-04-25T10:06:48.577283+0000","last_undegraded":"2024-04-25T10:06:48.577283+0000","last_fullsized":"2024-04-25T10:06:48.577283+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767923+0000","last_change":"2024-04-25T10:06:46.774632+0000","last_active":"2024-04-25T10:06:47.767923+0000","last_peered":"2024-04-25T10:06:47.767923+0000","last_clean":"2024-04-25T10:06:47.767923+0000","last_became_active":"2024-04-25T10:06:46.773829+0000","last_became_peered":"2024-04-25T10:06:46.773829+0000","last_unstale":"2024-04-25T10:06:47.767923+0000","last_undegraded":"2024-04-25T10:06:47.767923+0000","last_fullsized":"2024-04-25T10:06:47.767923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:40.812202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078146+0000","last_change":"2024-04-25T10:06:46.103414+0000","last_active":"2024-04-25T10:06:48.078146+0000","last_peered":"2024-04-25T10:06:48.078146+0000","last_clean":"2024-04-25T10:06:48.078146+0000","last_became_active":"2024-04-25T10:06:46.103164+0000","last_became_peered":"2024-04-25T10:06:46.103164+0000","last_unstale":"2024-04-25T10:06:48.078146+0000","last_undegraded":"2024-04-25T10:06:48.078146+0000","last_fullsized":"2024-04-25T10:06:48.078146+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:10.914538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765294+0000","last_change":"2024-04-25T10:06:46.794249+0000","last_active":"2024-04-25T10:06:47.765294+0000","last_peered":"2024-04-25T10:06:47.765294+0000","last_clean":"2024-04-25T10:06:47.765294+0000","last_became_active":"2024-04-25T10:06:46.792753+0000","last_became_peered":"2024-04-25T10:06:46.792753+0000","last_unstale":"2024-04-25T10:06:47.765294+0000","last_undegraded":"2024-04-25T10:06:47.765294+0000","last_fullsized":"2024-04-25T10:06:47.765294+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:41.917771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081812+0000","last_change":"2024-04-25T10:06:46.102417+0000","last_active":"2024-04-25T10:06:48.081812+0000","last_peered":"2024-04-25T10:06:48.081812+0000","last_clean":"2024-04-25T10:06:48.081812+0000","last_became_active":"2024-04-25T10:06:46.101880+0000","last_became_peered":"2024-04-25T10:06:46.101880+0000","last_unstale":"2024-04-25T10:06:48.081812+0000","last_undegraded":"2024-04-25T10:06:48.081812+0000","last_fullsized":"2024-04-25T10:06:48.081812+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:27.992158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077906+0000","last_change":"2024-04-25T10:06:46.800819+0000","last_active":"2024-04-25T10:06:48.077906+0000","last_peered":"2024-04-25T10:06:48.077906+0000","last_clean":"2024-04-25T10:06:48.077906+0000","last_became_active":"2024-04-25T10:06:46.799180+0000","last_became_peered":"2024-04-25T10:06:46.799180+0000","last_unstale":"2024-04-25T10:06:48.077906+0000","last_undegraded":"2024-04-25T10:06:48.077906+0000","last_fullsized":"2024-04-25T10:06:48.077906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579230+0000","last_change":"2024-04-25T10:06:46.108693+0000","last_active":"2024-04-25T10:06:48.579230+0000","last_peered":"2024-04-25T10:06:48.579230+0000","last_clean":"2024-04-25T10:06:48.579230+0000","last_became_active":"2024-04-25T10:06:46.108114+0000","last_became_peered":"2024-04-25T10:06:46.108114+0000","last_unstale":"2024-04-25T10:06:48.579230+0000","last_undegraded":"2024-04-25T10:06:48.579230+0000","last_fullsized":"2024-04-25T10:06:48.579230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:05.951567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577767+0000","last_change":"2024-04-25T10:06:46.788689+0000","last_active":"2024-04-25T10:06:48.577767+0000","last_peered":"2024-04-25T10:06:48.577767+0000","last_clean":"2024-04-25T10:06:48.577767+0000","last_became_active":"2024-04-25T10:06:46.786763+0000","last_became_peered":"2024-04-25T10:06:46.786763+0000","last_unstale":"2024-04-25T10:06:48.577767+0000","last_undegraded":"2024-04-25T10:06:48.577767+0000","last_fullsized":"2024-04-25T10:06:48.577767+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:38.545499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578022+0000","last_change":"2024-04-25T10:06:46.103056+0000","last_active":"2024-04-25T10:06:48.578022+0000","last_peered":"2024-04-25T10:06:48.578022+0000","last_clean":"2024-04-25T10:06:48.578022+0000","last_became_active":"2024-04-25T10:06:46.102214+0000","last_became_peered":"2024-04-25T10:06:46.102214+0000","last_unstale":"2024-04-25T10:06:48.578022+0000","last_undegraded":"2024-04-25T10:06:48.578022+0000","last_fullsized":"2024-04-25T10:06:48.578022+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579421+0000","last_change":"2024-04-25T10:06:46.795073+0000","last_active":"2024-04-25T10:06:48.579421+0000","last_peered":"2024-04-25T10:06:48.579421+0000","last_clean":"2024-04-25T10:06:48.579421+0000","last_became_active":"2024-04-25T10:06:46.793560+0000","last_became_peered":"2024-04-25T10:06:46.793560+0000","last_unstale":"2024-04-25T10:06:48.579421+0000","last_undegraded":"2024-04-25T10:06:48.579421+0000","last_fullsized":"2024-04-25T10:06:48.579421+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"78'3","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082711+0000","last_change":"2024-04-25T10:06:46.085445+0000","last_active":"2024-04-25T10:06:48.082711+0000","last_peered":"2024-04-25T10:06:48.082711+0000","last_clean":"2024-04-25T10:06:48.082711+0000","last_became_active":"2024-04-25T10:06:46.085042+0000","last_became_peered":"2024-04-25T10:06:46.085042+0000","last_unstale":"2024-04-25T10:06:48.082711+0000","last_undegraded":"2024-04-25T10:06:48.082711+0000","last_fullsized":"2024-04-25T10:06:48.082711+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:22.190109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766428+0000","last_change":"2024-04-25T10:06:46.814843+0000","last_active":"2024-04-25T10:06:47.766428+0000","last_peered":"2024-04-25T10:06:47.766428+0000","last_clean":"2024-04-25T10:06:47.766428+0000","last_became_active":"2024-04-25T10:06:46.814095+0000","last_became_peered":"2024-04-25T10:06:46.814095+0000","last_unstale":"2024-04-25T10:06:47.766428+0000","last_undegraded":"2024-04-25T10:06:47.766428+0000","last_fullsized":"2024-04-25T10:06:47.766428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:47.490689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"78'6","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765998+0000","last_change":"2024-04-25T10:06:46.099679+0000","last_active":"2024-04-25T10:06:47.765998+0000","last_peered":"2024-04-25T10:06:47.765998+0000","last_clean":"2024-04-25T10:06:47.765998+0000","last_became_active":"2024-04-25T10:06:46.098811+0000","last_became_peered":"2024-04-25T10:06:46.098811+0000","last_unstale":"2024-04-25T10:06:47.765998+0000","last_undegraded":"2024-04-25T10:06:47.765998+0000","last_fullsized":"2024-04-25T10:06:47.765998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:40.533772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578436+0000","last_change":"2024-04-25T10:06:46.783730+0000","last_active":"2024-04-25T10:06:48.578436+0000","last_peered":"2024-04-25T10:06:48.578436+0000","last_clean":"2024-04-25T10:06:48.578436+0000","last_became_active":"2024-04-25T10:06:46.779298+0000","last_became_peered":"2024-04-25T10:06:46.779298+0000","last_unstale":"2024-04-25T10:06:48.578436+0000","last_undegraded":"2024-04-25T10:06:48.578436+0000","last_fullsized":"2024-04-25T10:06:48.578436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577400+0000","last_change":"2024-04-25T10:06:46.097662+0000","last_active":"2024-04-25T10:06:48.577400+0000","last_peered":"2024-04-25T10:06:48.577400+0000","last_clean":"2024-04-25T10:06:48.577400+0000","last_became_active":"2024-04-25T10:06:46.096906+0000","last_became_peered":"2024-04-25T10:06:46.096906+0000","last_unstale":"2024-04-25T10:06:48.577400+0000","last_undegraded":"2024-04-25T10:06:48.577400+0000","last_fullsized":"2024-04-25T10:06:48.577400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766293+0000","last_change":"2024-04-25T10:06:46.793225+0000","last_active":"2024-04-25T10:06:47.766293+0000","last_peered":"2024-04-25T10:06:47.766293+0000","last_clean":"2024-04-25T10:06:47.766293+0000","last_became_active":"2024-04-25T10:06:46.792341+0000","last_became_peered":"2024-04-25T10:06:46.792341+0000","last_unstale":"2024-04-25T10:06:47.766293+0000","last_undegraded":"2024-04-25T10:06:47.766293+0000","last_fullsized":"2024-04-25T10:06:47.766293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:51.586619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578713+0000","last_change":"2024-04-25T10:06:46.112336+0000","last_active":"2024-04-25T10:06:48.578713+0000","last_peered":"2024-04-25T10:06:48.578713+0000","last_clean":"2024-04-25T10:06:48.578713+0000","last_became_active":"2024-04-25T10:06:46.108377+0000","last_became_peered":"2024-04-25T10:06:46.108377+0000","last_unstale":"2024-04-25T10:06:48.578713+0000","last_undegraded":"2024-04-25T10:06:48.578713+0000","last_fullsized":"2024-04-25T10:06:48.578713+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:50.582004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765621+0000","last_change":"2024-04-25T10:06:46.791957+0000","last_active":"2024-04-25T10:06:47.765621+0000","last_peered":"2024-04-25T10:06:47.765621+0000","last_clean":"2024-04-25T10:06:47.765621+0000","last_became_active":"2024-04-25T10:06:46.789267+0000","last_became_peered":"2024-04-25T10:06:46.789267+0000","last_unstale":"2024-04-25T10:06:47.765621+0000","last_undegraded":"2024-04-25T10:06:47.765621+0000","last_fullsized":"2024-04-25T10:06:47.765621+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:11.919154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082124+0000","last_change":"2024-04-25T10:06:46.095569+0000","last_active":"2024-04-25T10:06:48.082124+0000","last_peered":"2024-04-25T10:06:48.082124+0000","last_clean":"2024-04-25T10:06:48.082124+0000","last_became_active":"2024-04-25T10:06:46.095284+0000","last_became_peered":"2024-04-25T10:06:46.095284+0000","last_unstale":"2024-04-25T10:06:48.082124+0000","last_undegraded":"2024-04-25T10:06:48.082124+0000","last_fullsized":"2024-04-25T10:06:48.082124+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:49.409583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765571+0000","last_change":"2024-04-25T10:06:46.797079+0000","last_active":"2024-04-25T10:06:47.765571+0000","last_peered":"2024-04-25T10:06:47.765571+0000","last_clean":"2024-04-25T10:06:47.765571+0000","last_became_active":"2024-04-25T10:06:46.796377+0000","last_became_peered":"2024-04-25T10:06:46.796377+0000","last_unstale":"2024-04-25T10:06:47.765571+0000","last_undegraded":"2024-04-25T10:06:47.765571+0000","last_fullsized":"2024-04-25T10:06:47.765571+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:08.421167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"78'8","reported_seq":22,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.077979+0000","last_change":"2024-04-25T10:06:46.096666+0000","last_active":"2024-04-25T10:06:48.077979+0000","last_peered":"2024-04-25T10:06:48.077979+0000","last_clean":"2024-04-25T10:06:48.077979+0000","last_became_active":"2024-04-25T10:06:46.096084+0000","last_became_peered":"2024-04-25T10:06:46.096084+0000","last_unstale":"2024-04-25T10:06:48.077979+0000","last_undegraded":"2024-04-25T10:06:48.077979+0000","last_fullsized":"2024-04-25T10:06:48.077979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:14.594339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578330+0000","last_change":"2024-04-25T10:06:46.789487+0000","last_active":"2024-04-25T10:06:48.578330+0000","last_peered":"2024-04-25T10:06:48.578330+0000","last_clean":"2024-04-25T10:06:48.578330+0000","last_became_active":"2024-04-25T10:06:46.788801+0000","last_became_peered":"2024-04-25T10:06:46.788801+0000","last_unstale":"2024-04-25T10:06:48.578330+0000","last_undegraded":"2024-04-25T10:06:48.578330+0000","last_fullsized":"2024-04-25T10:06:48.578330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:14.905327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579292+0000","last_change":"2024-04-25T10:06:46.109437+0000","last_active":"2024-04-25T10:06:48.579292+0000","last_peered":"2024-04-25T10:06:48.579292+0000","last_clean":"2024-04-25T10:06:48.579292+0000","last_became_active":"2024-04-25T10:06:46.108802+0000","last_became_peered":"2024-04-25T10:06:46.108802+0000","last_unstale":"2024-04-25T10:06:48.579292+0000","last_undegraded":"2024-04-25T10:06:48.579292+0000","last_fullsized":"2024-04-25T10:06:48.579292+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:27.950431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.081989+0000","last_change":"2024-04-25T10:06:46.794307+0000","last_active":"2024-04-25T10:06:48.081989+0000","last_peered":"2024-04-25T10:06:48.081989+0000","last_clean":"2024-04-25T10:06:48.081989+0000","last_became_active":"2024-04-25T10:06:46.793463+0000","last_became_peered":"2024-04-25T10:06:46.793463+0000","last_unstale":"2024-04-25T10:06:48.081989+0000","last_undegraded":"2024-04-25T10:06:48.081989+0000","last_fullsized":"2024-04-25T10:06:48.081989+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:13.222579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.767889+0000","last_change":"2024-04-25T10:06:46.098133+0000","last_active":"2024-04-25T10:06:47.767889+0000","last_peered":"2024-04-25T10:06:47.767889+0000","last_clean":"2024-04-25T10:06:47.767889+0000","last_became_active":"2024-04-25T10:06:46.097273+0000","last_became_peered":"2024-04-25T10:06:46.097273+0000","last_unstale":"2024-04-25T10:06:47.767889+0000","last_undegraded":"2024-04-25T10:06:47.767889+0000","last_fullsized":"2024-04-25T10:06:47.767889+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:56.587351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082265+0000","last_change":"2024-04-25T10:06:46.801431+0000","last_active":"2024-04-25T10:06:48.082265+0000","last_peered":"2024-04-25T10:06:48.082265+0000","last_clean":"2024-04-25T10:06:48.082265+0000","last_became_active":"2024-04-25T10:06:46.800621+0000","last_became_peered":"2024-04-25T10:06:46.800621+0000","last_unstale":"2024-04-25T10:06:48.082265+0000","last_undegraded":"2024-04-25T10:06:48.082265+0000","last_fullsized":"2024-04-25T10:06:48.082265+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:22.445395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"78'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082645+0000","last_change":"2024-04-25T10:06:46.098539+0000","last_active":"2024-04-25T10:06:48.082645+0000","last_peered":"2024-04-25T10:06:48.082645+0000","last_clean":"2024-04-25T10:06:48.082645+0000","last_became_active":"2024-04-25T10:06:46.098190+0000","last_became_peered":"2024-04-25T10:06:46.098190+0000","last_unstale":"2024-04-25T10:06:48.082645+0000","last_undegraded":"2024-04-25T10:06:48.082645+0000","last_fullsized":"2024-04-25T10:06:48.082645+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:13.339487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.794288+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.788351+0000","last_became_peered":"2024-04-25T10:06:46.788351+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766444+0000","last_change":"2024-04-25T10:06:46.086234+0000","last_active":"2024-04-25T10:06:47.766444+0000","last_peered":"2024-04-25T10:06:47.766444+0000","last_clean":"2024-04-25T10:06:47.766444+0000","last_became_active":"2024-04-25T10:06:46.085918+0000","last_became_peered":"2024-04-25T10:06:46.085918+0000","last_unstale":"2024-04-25T10:06:47.766444+0000","last_undegraded":"2024-04-25T10:06:47.766444+0000","last_fullsized":"2024-04-25T10:06:47.766444+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:44.045570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078587+0000","last_change":"2024-04-25T10:06:46.808180+0000","last_active":"2024-04-25T10:06:48.078587+0000","last_peered":"2024-04-25T10:06:48.078587+0000","last_clean":"2024-04-25T10:06:48.078587+0000","last_became_active":"2024-04-25T10:06:46.800573+0000","last_became_peered":"2024-04-25T10:06:46.800573+0000","last_unstale":"2024-04-25T10:06:48.078587+0000","last_undegraded":"2024-04-25T10:06:48.078587+0000","last_fullsized":"2024-04-25T10:06:48.078587+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:22.585824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578979+0000","last_change":"2024-04-25T10:06:46.089883+0000","last_active":"2024-04-25T10:06:48.578979+0000","last_peered":"2024-04-25T10:06:48.578979+0000","last_clean":"2024-04-25T10:06:48.578979+0000","last_became_active":"2024-04-25T10:06:46.089591+0000","last_became_peered":"2024-04-25T10:06:46.089591+0000","last_unstale":"2024-04-25T10:06:48.578979+0000","last_undegraded":"2024-04-25T10:06:48.578979+0000","last_fullsized":"2024-04-25T10:06:48.578979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:24.851984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078910+0000","last_change":"2024-04-25T10:06:46.808415+0000","last_active":"2024-04-25T10:06:48.078910+0000","last_peered":"2024-04-25T10:06:48.078910+0000","last_clean":"2024-04-25T10:06:48.078910+0000","last_became_active":"2024-04-25T10:06:46.807612+0000","last_became_peered":"2024-04-25T10:06:46.807612+0000","last_unstale":"2024-04-25T10:06:48.078910+0000","last_undegraded":"2024-04-25T10:06:48.078910+0000","last_fullsized":"2024-04-25T10:06:48.078910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:06.294138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578529+0000","last_change":"2024-04-25T10:06:46.099936+0000","last_active":"2024-04-25T10:06:48.578529+0000","last_peered":"2024-04-25T10:06:48.578529+0000","last_clean":"2024-04-25T10:06:48.578529+0000","last_became_active":"2024-04-25T10:06:46.098806+0000","last_became_peered":"2024-04-25T10:06:46.098806+0000","last_unstale":"2024-04-25T10:06:48.578529+0000","last_undegraded":"2024-04-25T10:06:48.578529+0000","last_fullsized":"2024-04-25T10:06:48.578529+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:26.129369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765840+0000","last_change":"2024-04-25T10:06:46.793643+0000","last_active":"2024-04-25T10:06:47.765840+0000","last_peered":"2024-04-25T10:06:47.765840+0000","last_clean":"2024-04-25T10:06:47.765840+0000","last_became_active":"2024-04-25T10:06:46.791775+0000","last_became_peered":"2024-04-25T10:06:46.791775+0000","last_unstale":"2024-04-25T10:06:47.765840+0000","last_undegraded":"2024-04-25T10:06:47.765840+0000","last_fullsized":"2024-04-25T10:06:47.765840+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:15.598955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578533+0000","last_change":"2024-04-25T10:06:46.099543+0000","last_active":"2024-04-25T10:06:48.578533+0000","last_peered":"2024-04-25T10:06:48.578533+0000","last_clean":"2024-04-25T10:06:48.578533+0000","last_became_active":"2024-04-25T10:06:46.099269+0000","last_became_peered":"2024-04-25T10:06:46.099269+0000","last_unstale":"2024-04-25T10:06:48.578533+0000","last_undegraded":"2024-04-25T10:06:48.578533+0000","last_fullsized":"2024-04-25T10:06:48.578533+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764669+0000","last_change":"2024-04-25T10:06:46.798507+0000","last_active":"2024-04-25T10:06:47.764669+0000","last_peered":"2024-04-25T10:06:47.764669+0000","last_clean":"2024-04-25T10:06:47.764669+0000","last_became_active":"2024-04-25T10:06:46.797703+0000","last_became_peered":"2024-04-25T10:06:46.797703+0000","last_unstale":"2024-04-25T10:06:47.764669+0000","last_undegraded":"2024-04-25T10:06:47.764669+0000","last_fullsized":"2024-04-25T10:06:47.764669+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:41.538388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"78'2","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.577874+0000","last_change":"2024-04-25T10:06:46.090113+0000","last_active":"2024-04-25T10:06:48.577874+0000","last_peered":"2024-04-25T10:06:48.577874+0000","last_clean":"2024-04-25T10:06:48.577874+0000","last_became_active":"2024-04-25T10:06:46.089353+0000","last_became_peered":"2024-04-25T10:06:46.089353+0000","last_unstale":"2024-04-25T10:06:48.577874+0000","last_undegraded":"2024-04-25T10:06:48.577874+0000","last_fullsized":"2024-04-25T10:06:48.577874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765642+0000","last_change":"2024-04-25T10:06:46.797032+0000","last_active":"2024-04-25T10:06:47.765642+0000","last_peered":"2024-04-25T10:06:47.765642+0000","last_clean":"2024-04-25T10:06:47.765642+0000","last_became_active":"2024-04-25T10:06:46.796310+0000","last_became_peered":"2024-04-25T10:06:46.796310+0000","last_unstale":"2024-04-25T10:06:47.765642+0000","last_undegraded":"2024-04-25T10:06:47.765642+0000","last_fullsized":"2024-04-25T10:06:47.765642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:39.607765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"79'1","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.080225+0000","last_change":"2024-04-25T10:06:46.087236+0000","last_active":"2024-04-25T10:06:48.080225+0000","last_peered":"2024-04-25T10:06:48.080225+0000","last_clean":"2024-04-25T10:06:48.080225+0000","last_became_active":"2024-04-25T10:06:46.086354+0000","last_became_peered":"2024-04-25T10:06:46.086354+0000","last_unstale":"2024-04-25T10:06:48.080225+0000","last_undegraded":"2024-04-25T10:06:48.080225+0000","last_fullsized":"2024-04-25T10:06:48.080225+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:38.603149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766980+0000","last_change":"2024-04-25T10:06:46.801971+0000","last_active":"2024-04-25T10:06:47.766980+0000","last_peered":"2024-04-25T10:06:47.766980+0000","last_clean":"2024-04-25T10:06:47.766980+0000","last_became_active":"2024-04-25T10:06:46.798311+0000","last_became_peered":"2024-04-25T10:06:46.798311+0000","last_unstale":"2024-04-25T10:06:47.766980+0000","last_undegraded":"2024-04-25T10:06:47.766980+0000","last_fullsized":"2024-04-25T10:06:47.766980+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:55.984280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579054+0000","last_change":"2024-04-25T10:06:46.113284+0000","last_active":"2024-04-25T10:06:48.579054+0000","last_peered":"2024-04-25T10:06:48.579054+0000","last_clean":"2024-04-25T10:06:48.579054+0000","last_became_active":"2024-04-25T10:06:46.112729+0000","last_became_peered":"2024-04-25T10:06:46.112729+0000","last_unstale":"2024-04-25T10:06:48.579054+0000","last_undegraded":"2024-04-25T10:06:48.579054+0000","last_fullsized":"2024-04-25T10:06:48.579054+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:13.189267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.078514+0000","last_change":"2024-04-25T10:06:46.806654+0000","last_active":"2024-04-25T10:06:48.078514+0000","last_peered":"2024-04-25T10:06:48.078514+0000","last_clean":"2024-04-25T10:06:48.078514+0000","last_became_active":"2024-04-25T10:06:46.799360+0000","last_became_peered":"2024-04-25T10:06:46.799360+0000","last_unstale":"2024-04-25T10:06:48.078514+0000","last_undegraded":"2024-04-25T10:06:48.078514+0000","last_fullsized":"2024-04-25T10:06:48.078514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:08.999570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578334+0000","last_change":"2024-04-25T10:06:46.109473+0000","last_active":"2024-04-25T10:06:48.578334+0000","last_peered":"2024-04-25T10:06:48.578334+0000","last_clean":"2024-04-25T10:06:48.578334+0000","last_became_active":"2024-04-25T10:06:46.108640+0000","last_became_peered":"2024-04-25T10:06:46.108640+0000","last_unstale":"2024-04-25T10:06:48.578334+0000","last_undegraded":"2024-04-25T10:06:48.578334+0000","last_fullsized":"2024-04-25T10:06:48.578334+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:43.158673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765464+0000","last_change":"2024-04-25T10:06:46.786613+0000","last_active":"2024-04-25T10:06:47.765464+0000","last_peered":"2024-04-25T10:06:47.765464+0000","last_clean":"2024-04-25T10:06:47.765464+0000","last_became_active":"2024-04-25T10:06:46.785785+0000","last_became_peered":"2024-04-25T10:06:46.785785+0000","last_unstale":"2024-04-25T10:06:47.765464+0000","last_undegraded":"2024-04-25T10:06:47.765464+0000","last_fullsized":"2024-04-25T10:06:47.765464+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:15.393383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.764735+0000","last_change":"2024-04-25T10:06:46.100079+0000","last_active":"2024-04-25T10:06:47.764735+0000","last_peered":"2024-04-25T10:06:47.764735+0000","last_clean":"2024-04-25T10:06:47.764735+0000","last_became_active":"2024-04-25T10:06:46.099419+0000","last_became_peered":"2024-04-25T10:06:46.099419+0000","last_unstale":"2024-04-25T10:06:47.764735+0000","last_undegraded":"2024-04-25T10:06:47.764735+0000","last_fullsized":"2024-04-25T10:06:47.764735+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765365+0000","last_change":"2024-04-25T10:06:46.793298+0000","last_active":"2024-04-25T10:06:47.765365+0000","last_peered":"2024-04-25T10:06:47.765365+0000","last_clean":"2024-04-25T10:06:47.765365+0000","last_became_active":"2024-04-25T10:06:46.791603+0000","last_became_peered":"2024-04-25T10:06:46.791603+0000","last_unstale":"2024-04-25T10:06:47.765365+0000","last_undegraded":"2024-04-25T10:06:47.765365+0000","last_fullsized":"2024-04-25T10:06:47.765365+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.082282+0000","last_change":"2024-04-25T10:06:46.088827+0000","last_active":"2024-04-25T10:06:48.082282+0000","last_peered":"2024-04-25T10:06:48.082282+0000","last_clean":"2024-04-25T10:06:48.082282+0000","last_became_active":"2024-04-25T10:06:46.088541+0000","last_became_peered":"2024-04-25T10:06:46.088541+0000","last_unstale":"2024-04-25T10:06:48.082282+0000","last_undegraded":"2024-04-25T10:06:48.082282+0000","last_fullsized":"2024-04-25T10:06:48.082282+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:37.206201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766800+0000","last_change":"2024-04-25T10:06:46.815568+0000","last_active":"2024-04-25T10:06:47.766800+0000","last_peered":"2024-04-25T10:06:47.766800+0000","last_clean":"2024-04-25T10:06:47.766800+0000","last_became_active":"2024-04-25T10:06:46.814861+0000","last_became_peered":"2024-04-25T10:06:46.814861+0000","last_unstale":"2024-04-25T10:06:47.766800+0000","last_undegraded":"2024-04-25T10:06:47.766800+0000","last_fullsized":"2024-04-25T10:06:47.766800+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:47.965250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.765886+0000","last_change":"2024-04-25T10:06:46.092480+0000","last_active":"2024-04-25T10:06:47.765886+0000","last_peered":"2024-04-25T10:06:47.765886+0000","last_clean":"2024-04-25T10:06:47.765886+0000","last_became_active":"2024-04-25T10:06:46.091763+0000","last_became_peered":"2024-04-25T10:06:46.091763+0000","last_unstale":"2024-04-25T10:06:47.765886+0000","last_undegraded":"2024-04-25T10:06:47.765886+0000","last_fullsized":"2024-04-25T10:06:47.765886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:44.359393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766985+0000","last_change":"2024-04-25T10:06:46.783297+0000","last_active":"2024-04-25T10:06:47.766985+0000","last_peered":"2024-04-25T10:06:47.766985+0000","last_clean":"2024-04-25T10:06:47.766985+0000","last_became_active":"2024-04-25T10:06:46.782524+0000","last_became_peered":"2024-04-25T10:06:46.782524+0000","last_unstale":"2024-04-25T10:06:47.766985+0000","last_undegraded":"2024-04-25T10:06:47.766985+0000","last_fullsized":"2024-04-25T10:06:47.766985+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:41.186568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:47.766127+0000","last_change":"2024-04-25T10:06:46.079371+0000","last_active":"2024-04-25T10:06:47.766127+0000","last_peered":"2024-04-25T10:06:47.766127+0000","last_clean":"2024-04-25T10:06:47.766127+0000","last_became_active":"2024-04-25T10:06:46.079020+0000","last_became_peered":"2024-04-25T10:06:46.079020+0000","last_unstale":"2024-04-25T10:06:47.766127+0000","last_undegraded":"2024-04-25T10:06:47.766127+0000","last_fullsized":"2024-04-25T10:06:47.766127+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:44.633904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578863+0000","last_change":"2024-04-25T10:06:46.805681+0000","last_active":"2024-04-25T10:06:48.578863+0000","last_peered":"2024-04-25T10:06:48.578863+0000","last_clean":"2024-04-25T10:06:48.578863+0000","last_became_active":"2024-04-25T10:06:46.805076+0000","last_became_peered":"2024-04-25T10:06:46.805076+0000","last_unstale":"2024-04-25T10:06:48.578863+0000","last_undegraded":"2024-04-25T10:06:48.578863+0000","last_fullsized":"2024-04-25T10:06:48.578863+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:46.056936+0000","last_clean_scrub_stamp":"2024-04-25T10:06:46.056936+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:43.954275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.579678+0000","last_change":"2024-04-25T10:06:46.099980+0000","last_active":"2024-04-25T10:06:48.579678+0000","last_peered":"2024-04-25T10:06:48.579678+0000","last_clean":"2024-04-25T10:06:48.579678+0000","last_became_active":"2024-04-25T10:06:46.099518+0000","last_became_peered":"2024-04-25T10:06:46.099518+0000","last_unstale":"2024-04-25T10:06:48.579678+0000","last_undegraded":"2024-04-25T10:06:48.579678+0000","last_fullsized":"2024-04-25T10:06:48.579678+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:45.052320+0000","last_clean_scrub_stamp":"2024-04-25T10:06:45.052320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:15.012055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":280,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-04-25T10:06:48.578707+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:06:48.578707+0000","last_peered":"2024-04-25T10:06:48.578707+0000","last_clean":"2024-04-25T10:06:48.578707+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:06:48.578707+0000","last_undegraded":"2024-04-25T10:06:48.578707+0000","last_fullsized":"2024-04-25T10:06:48.578707+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":1032685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574912,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":978944,"data_stored":443907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":444011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":970752,"data_stored":442877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574912,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":978944,"data_stored":443037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":442407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":1032663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:55.514 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:55.514 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 Processor -- start 2024-04-25T10:06:55.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 -- start start 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0fb40cfa20 0x7f0fb4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40cbd20 con 0x7f0fb40cf590 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fb40cc000 con 0x7f0fb415b6b0 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fbb4e2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fb415ba90 con 0x7f0fb40cfa20 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fb8a7d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0fb40cf590 0x7f0fb4072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59384/0 (socket says 172.21.15.82:59384) 2024-04-25T10:06:55.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fb8a7d640 1 -- 172.21.15.82:0/2284857197 learned_addr learned my addr 172.21.15.82:0/2284857197 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fb9a7f640 1 --2- 172.21.15.82:0/2284857197 >> v2:172.21.15.196:3300/0 conn(0x7f0fb40cfa20 0x7f0fb4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790145098 0 0) 0x7f0fb40cbd20 con 0x7f0fb40cf590 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.674+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fa0003530 con 0x7f0fb40cf590 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1803436042 0 0) 0x7f0fa0003530 con 0x7f0fb40cf590 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 >> v2:172.21.15.196:3300/0 conn(0x7f0fb40cfa20 msgr2=0x7f0fb4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 --2- 172.21.15.82:0/2284857197 >> v2:172.21.15.196:3300/0 conn(0x7f0fb40cfa20 0x7f0fb4072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 >> v1:172.21.15.196:6789/0 conn(0x7f0fb415b6b0 legacy=0x7f0fb40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb40cc2e0 con 0x7f0fb40cf590 2024-04-25T10:06:55.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0fa4001680 con 0x7f0fb40cf590 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb9a7f640 1 --2- 172.21.15.82:0/2284857197 >> v2:172.21.15.196:3300/0 conn(0x7f0fb40cfa20 0x7f0fb4072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 >> v1:172.21.15.82:6789/0 conn(0x7f0fb40cf590 legacy=0x7f0fb4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fa0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa0004810 0x7f0fa0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa0005240 0x7f0fa000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb415ba90 con 0x7f0fa0003a30 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa000c1a0 con 0x7f0fa0005240 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa000c3a0 con 0x7f0fa0004810 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fa0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb9a7f640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa0005240 0x7f0fa000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 -- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa0004810 msgr2=0x7f0fa0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fa0004810 0x7f0fa0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 -- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa0005240 msgr2=0x7f0fa000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa0005240 0x7f0fa000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 -- 172.21.15.82:0/2284857197 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa0003530 con 0x7f0fa0003a30 2024-04-25T10:06:55.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb9a7f640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fa0005240 0x7f0fa000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb9a7f640 1 -- 172.21.15.82:0/2284857197 reap_dead start 2024-04-25T10:06:55.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fb927e640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fa0003e50 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001780 tx=0x7f0fb001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67da8f8cdcdaf7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00013a0 con 0x7f0fa0003a30 2024-04-25T10:06:55.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00052f0 con 0x7f0fa0003a30 2024-04-25T10:06:55.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fabfff640 1 -- 172.21.15.82:0/2284857197 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0003840 con 0x7f0fa0003a30 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 msgr2=0x7f0fa0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fa0003e50 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0001780 tx=0x7f0fb001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/2284857197 shutdown_connections 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/2284857197 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fa0003e50 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/2284857197 >> 172.21.15.82:0/2284857197 conn(0x7f0fb406ae60 msgr2=0x7f0fb406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/2284857197 shutdown_connections 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.678+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/2284857197 wait complete. 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 Processor -- start 2024-04-25T10:06:55.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- start start 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 0x7f0fb40c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fb40c55c0 0x7f0fb415ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40cfe00 con 0x7f0fa0003a30 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40cba30 con 0x7f0fb41f56a0 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb40cadb0 con 0x7f0fb40c55c0 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35996/0 (socket says 172.21.15.82:35996) 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 -- 172.21.15.82:0/4193546560 learned_addr learned my addr 172.21.15.82:0/4193546560 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:55.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb8a7d640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 0x7f0fb40c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 -- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fb40c55c0 msgr2=0x7f0fb415ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb9a7f640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fb40c55c0 0x7f0fb415ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fb40c55c0 0x7f0fb415ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 -- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 msgr2=0x7f0fb40c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 0x7f0fb40c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 -- 172.21.15.82:0/4193546560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb4155a90 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb8a7d640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 0x7f0fb40c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fb927e640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005040 tx=0x7f0fa0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb0007a40 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb00ba070 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb40c3210 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00013a0 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb40c37d0 con 0x7f0fa0003a30 2024-04-25T10:06:55.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fb404efa0 con 0x7f0fa0003a30 2024-04-25T10:06:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.682+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0fb00052f0 con 0x7f0fa0003a30 2024-04-25T10:06:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.686+0000 7f0fa9ffb640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 0x7f0f98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.686+0000 7f0fb8a7d640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 0x7f0f98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.686+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0fb011f550 con 0x7f0fa0003a30 2024-04-25T10:06:55.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.686+0000 7f0fb8a7d640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 0x7f0f98069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40c5eb0 tx=0x7f0fa4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.690+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0fb0016110 con 0x7f0fa0003a30 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.890+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0fb415aee0 con 0x7f0fa0003a30 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fa9ffb640 1 -- 172.21.15.82:0/4193546560 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0fb00f03d0 con 0x7f0fa0003a30 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:06:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 msgr2=0x7f0f98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 0x7f0f98069450 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40c5eb0 tx=0x7f0fa4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 msgr2=0x7f0fb4155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005040 tx=0x7f0fa0005840 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 shutdown_connections 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0f98066f90 0x7f0f98069450 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0fb40c55c0 0x7f0fb415ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0fb41f56a0 0x7f0fb40c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 --2- 172.21.15.82:0/4193546560 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0fa0003a30 0x7f0fb4155550 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 >> 172.21.15.82:0/4193546560 conn(0x7f0fb406ae60 msgr2=0x7f0fb40aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 shutdown_connections 2024-04-25T10:06:55.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:55.894+0000 7f0fbb4e2640 1 -- 172.21.15.82:0/4193546560 wait complete. 2024-04-25T10:06:55.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:56.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 Processor -- start 2024-04-25T10:06:56.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 -- start start 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99a015da80 0x7f99a00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a006be60 con 0x7f99a00d07f0 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a015de60 con 0x7f99a00d1cd0 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a6b70640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a006c440 con 0x7f99a015da80 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99a015da80 0x7f99a00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99a015da80 0x7f99a00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36002/0 (socket says 172.21.15.82:36002) 2024-04-25T10:06:56.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a510d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f99a00d07f0 0x7f99a006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59398/0 (socket says 172.21.15.82:59398) 2024-04-25T10:06:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 -- 172.21.15.82:0/3032644849 learned_addr learned my addr 172.21.15.82:0/3032644849 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 -- 172.21.15.82:0/3032644849 >> v1:172.21.15.196:6789/0 conn(0x7f99a00d1cd0 legacy=0x7f99a00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 -- 172.21.15.82:0/3032644849 >> v1:172.21.15.82:6789/0 conn(0x7f99a00d07f0 legacy=0x7f99a006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f999f7fe640 1 -- 172.21.15.82:0/3032644849 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112292130 0 0) 0x7f99a006be60 con 0x7f99a00d07f0 2024-04-25T10:06:56.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.030+0000 7f99a490c640 1 -- 172.21.15.82:0/3032644849 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00d2450 con 0x7f99a015da80 2024-04-25T10:06:56.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a490c640 1 --2- 172.21.15.82:0/3032644849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a015da80 0x7f99a00d4230 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f99a006c5c0 tx=0x7f999401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b6a9329d657087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f999f7fe640 1 -- 172.21.15.82:0/3032644849 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940a7070 con 0x7f99a015da80 2024-04-25T10:06:56.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f999f7fe640 1 -- 172.21.15.82:0/3032644849 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99940b0070 con 0x7f99a015da80 2024-04-25T10:06:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f999f7fe640 1 -- 172.21.15.82:0/3032644849 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940ab890 con 0x7f99a015da80 2024-04-25T10:06:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- 172.21.15.82:0/3032644849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a015da80 msgr2=0x7f99a00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 --2- 172.21.15.82:0/3032644849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a015da80 0x7f99a00d4230 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f99a006c5c0 tx=0x7f999401c510 comp rx=0 tx=0).stop 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- 172.21.15.82:0/3032644849 shutdown_connections 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 --2- 172.21.15.82:0/3032644849 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a015da80 0x7f99a00d4230 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- 172.21.15.82:0/3032644849 >> 172.21.15.82:0/3032644849 conn(0x7f99a008b000 msgr2=0x7f99a00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- 172.21.15.82:0/3032644849 shutdown_connections 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- 172.21.15.82:0/3032644849 wait complete. 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 Processor -- start 2024-04-25T10:06:56.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.034+0000 7f99a6b70640 1 -- start start 2024-04-25T10:06:56.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 0x7f99a014ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 0x7f99a00b8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a006d970 con 0x7f99a00b6310 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a006d7f0 con 0x7f99a00d07f0 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a006daf0 con 0x7f99a00d1cd0 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36004/0 (socket says 172.21.15.82:36004) 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 -- 172.21.15.82:0/123204699 learned_addr learned my addr 172.21.15.82:0/123204699 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a490c640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 0x7f99a014ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999ffff640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 0x7f99a00b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 -- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 msgr2=0x7f99a00b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 0x7f99a00b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 -- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 msgr2=0x7f99a014ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 0x7f99a014ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 -- 172.21.15.82:0/123204699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00b6810 con 0x7f99a00b6310 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a490c640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 0x7f99a014ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999ffff640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 0x7f99a00b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a510d640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f999800c7d0 tx=0x7f999800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998018070 con 0x7f99a00b6310 2024-04-25T10:06:56.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f999800a640 con 0x7f99a00b6310 2024-04-25T10:06:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999800aa70 con 0x7f99a00b6310 2024-04-25T10:06:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a0159a80 con 0x7f99a00b6310 2024-04-25T10:06:56.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a015a040 con 0x7f99a00b6310 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a004efa0 con 0x7f99a00b6310 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f999800ac10 con 0x7f99a00b6310 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 0x7f998c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9998084760 con 0x7f99a00b6310 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.038+0000 7f99a490c640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 0x7f998c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.042+0000 7f99a490c640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 0x7f998c069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f99940af000 tx=0x7f9994008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.046+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9998055670 con 0x7f99a00b6310 2024-04-25T10:06:56.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.238+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99a00b1e70 con 0x7f99a00b6310 2024-04-25T10:06:56.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.238+0000 7f999d7fa640 1 -- 172.21.15.82:0/123204699 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9998054f70 con 0x7f99a00b6310 2024-04-25T10:06:56.243 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:06:56.243 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:06:49.173277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/910515233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":910515233},{"type":"v1","addr":"172.21.15.82:6837","nonce":910515233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:06:56.243 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 41 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 msgr2=0x7f998c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 0x7f998c069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f99940af000 tx=0x7f9994008040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 msgr2=0x7f99a014e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f999800c7d0 tx=0x7f999800cc90 comp rx=0 tx=0).stop 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 shutdown_connections 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f998c066f60 0x7f998c069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99a00d1cd0 0x7f99a00b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99a00b6310 0x7f99a014e250 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 --2- 172.21.15.82:0/123204699 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99a00d07f0 0x7f99a014ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 >> 172.21.15.82:0/123204699 conn(0x7f99a008b000 msgr2=0x7f99a00d1470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 shutdown_connections 2024-04-25T10:06:56.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.242+0000 7f99a6b70640 1 -- 172.21.15.82:0/123204699 wait complete. 2024-04-25T10:06:56.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:46.428439+0000', 'modified': '2024-04-25T10:06:49.173277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6966}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6966': {'gid': 6966, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/910515233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 910515233}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 910515233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:56.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:56.664 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:06:56.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 Processor -- start 2024-04-25T10:06:56.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 -- start start 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 0x7ffb180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb18144ca0 0x7ffb1806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb180cee30 con 0x7ffb1806ec60 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb180cc2b0 con 0x7ffb18144ca0 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb1e61b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1806e290 con 0x7ffb180ce510 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb18144ca0 0x7ffb1806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb18144ca0 0x7ffb1806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36012/0 (socket says 172.21.15.82:36012) 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 -- 172.21.15.82:0/3035955628 learned_addr learned my addr 172.21.15.82:0/3035955628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb17fff640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 0x7ffb180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 -- 172.21.15.82:0/3035955628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 msgr2=0x7ffb180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 0x7ffb180ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 -- 172.21.15.82:0/3035955628 >> v1:172.21.15.196:6789/0 conn(0x7ffb1806ec60 legacy=0x7ffb18142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb177fe640 1 -- 172.21.15.82:0/3035955628 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb180ccc70 con 0x7ffb18144ca0 2024-04-25T10:06:56.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.790+0000 7ffb17fff640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 0x7ffb180ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb177fe640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb1806e720 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7903eab27c3cf379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb16ffd640 1 -- 172.21.15.82:0/3035955628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000b5070 con 0x7ffb18144ca0 2024-04-25T10:06:56.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb16ffd640 1 -- 172.21.15.82:0/3035955628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000a89d0 con 0x7ffb18144ca0 2024-04-25T10:06:56.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb16ffd640 1 -- 172.21.15.82:0/3035955628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000a8da0 con 0x7ffb18144ca0 2024-04-25T10:06:56.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3035955628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 msgr2=0x7ffb1806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb1806e720 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009870 tx=0x7ffb0001ed10 comp rx=0 tx=0).stop 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3035955628 shutdown_connections 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb1806e720 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3035955628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb180ce510 0x7ffb180ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3035955628 >> 172.21.15.82:0/3035955628 conn(0x7ffb180b11a0 msgr2=0x7ffb180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3035955628 shutdown_connections 2024-04-25T10:06:56.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3035955628 wait complete. 2024-04-25T10:06:56.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 Processor -- start 2024-04-25T10:06:56.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.794+0000 7ffb1e61b640 1 -- start start 2024-04-25T10:06:56.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb1806ec60 0x7ffb1813ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb180ce510 0x7ffb18143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb180cc900 con 0x7ffb18144ca0 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1806df20 con 0x7ffb180ce510 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb18142d90 con 0x7ffb1806ec60 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36014/0 (socket says 172.21.15.82:36014) 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/3049258989 learned_addr learned my addr 172.21.15.82:0/3049258989 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb177fe640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb180ce510 0x7ffb18143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb17fff640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb1806ec60 0x7ffb1813ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb1806ec60 msgr2=0x7ffb1813ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb1806ec60 0x7ffb1813ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb180ce510 msgr2=0x7ffb18143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb180ce510 0x7ffb18143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/3049258989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1813d3b0 con 0x7ffb18144ca0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c390 tx=0x7ffb1000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 -- 172.21.15.82:0/3049258989 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10011790 con 0x7ffb18144ca0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 -- 172.21.15.82:0/3049258989 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb10017070 con 0x7ffb18144ca0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb180b35b0 con 0x7ffb18144ca0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 -- 172.21.15.82:0/3049258989 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10011b70 con 0x7ffb18144ca0 2024-04-25T10:06:56.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb181e0ce0 con 0x7ffb18144ca0 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 -- 172.21.15.82:0/3049258989 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb10011d10 con 0x7ffb18144ca0 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 0x7ffaf8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb14ff9640 1 -- 172.21.15.82:0/3049258989 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb10084ff0 con 0x7ffb18144ca0 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb17fff640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 0x7ffaf8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.798+0000 7ffb17fff640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 0x7ffaf8069450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb080075b0 tx=0x7ffb08007540 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 Processor -- start 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- start start 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110be4a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bd325f0 con 0x56110be4af30 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bc99da0 con 0x56110be95240 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110bc22150 con 0x56110be94920 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36020/0 (socket says 172.21.15.82:36020) 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/258912017 learned_addr learned my addr 172.21.15.82:0/258912017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb17fff640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb177fe640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110be4a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 msgr2=0x56110be94d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be94d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 msgr2=0x56110be4a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110be4a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/258912017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110bc3bca0 con 0x56110be4af30 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb17fff640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffb100556d0 tx=0x7ffb10055f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0817f122978a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb0dffb640 1 -- 172.21.15.82:0/258912017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1006d020 con 0x56110be4af30 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb0dffb640 1 -- 172.21.15.82:0/258912017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1005de50 con 0x56110be4af30 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb0dffb640 1 -- 172.21.15.82:0/258912017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1006d1c0 con 0x56110be4af30 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- 172.21.15.82:0/258912017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 msgr2=0x56110be4b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7ffb100556d0 tx=0x7ffb10055f80 comp rx=0 tx=0).stop 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- 172.21.15.82:0/258912017 shutdown_connections 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be4af30 0x56110be4b310 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110be4a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 --2- 172.21.15.82:0/258912017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- 172.21.15.82:0/258912017 >> 172.21.15.82:0/258912017 conn(0x56110be512a0 msgr2=0x56110be492d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- 172.21.15.82:0/258912017 shutdown_connections 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- 172.21.15.82:0/258912017 wait complete. 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 -- start start 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.802+0000 7ffb21788000 1 Processor -- start 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110be9bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be9c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bc54990 con 0x56110be95240 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bc95e40 con 0x56110be4af30 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110bc25900 con 0x56110be94920 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 10 client.0 _ll_register_callbacks cb 0x56110bd4d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36026/0 (socket says 172.21.15.82:36026) 2024-04-25T10:06:56.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/959874544 learned_addr learned my addr 172.21.15.82:0/959874544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb17fff640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110be9bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 msgr2=0x56110be9c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be9c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 msgr2=0x56110be9bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110be9bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 -- 172.21.15.82:0/959874544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110bc52630 con 0x56110be95240 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffb100556d0 tx=0x7ffb1001e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91aa6f1852490f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10081060 con 0x56110be95240 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb10051e30 con 0x56110be95240 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb1005de50 con 0x56110be95240 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb1006d030 con 0x56110be95240 2024-04-25T10:06:56.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 10 client.7140 fetch_fsmap learned FSMap version 41 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x56110bc223a0 con 0x56110be95240 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1005de50 con 0x56110be95240 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb1005d9c0 con 0x56110be95240 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 10 client.7140 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 10 client.7140 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 20 client.7140 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110be4f4b0 0x56110be4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 4 client.7140 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:56.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 --> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56110bc35a50 con 0x56110be4f4b0 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb17fff640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110be4f4b0 0x56110be4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb17fff640 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110be4f4b0 0x56110be4f890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d2c4583166152a server_cookie=91f42f5224d04d4 in_seq=0 out_seq=0 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 10 client.7140 ms_handle_connect on v2:172.21.15.82:6836/910515233 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/959874544 <== mds.0 v2:172.21.15.82:6836/910515233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56110bc35a50 con 0x56110be4f4b0 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 10 client.7140 handle_command_reply: tid=0 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 client.7140 shutdown 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 msgr2=0x56110be4eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ffb100556d0 tx=0x7ffb1001e070 comp rx=0 tx=0).stop 2024-04-25T10:06:56.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 shutdown_connections 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110be4f4b0 0x56110be4f890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be94920 0x56110be9c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be95240 0x56110be4eaa0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 --2- 172.21.15.82:0/959874544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110be9bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffaf6ffd640 0 client.7140 ms_handle_reset on v2:172.21.15.82:6836/910515233 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 >> 172.21.15.82:0/959874544 conn(0x56110be512a0 msgr2=0x56110be499e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 shutdown_connections 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 1 -- 172.21.15.82:0/959874544 wait complete. 2024-04-25T10:06:56.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.806+0000 7ffb21788000 20 client.7140 trim_cache size 0 max 16384 2024-04-25T10:06:56.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 Processor -- start 2024-04-25T10:06:56.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 -- start start 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110bef0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be95240 0x56110bef0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb181ff3e0 con 0x56110be94920 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb181e2970 con 0x56110be4af30 2024-04-25T10:06:56.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb181fede0 con 0x56110be95240 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36040/0 (socket says 172.21.15.82:36040) 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.822+0000 7ffb177fe640 1 -- 172.21.15.82:0/1089753651 learned_addr learned my addr 172.21.15.82:0/1089753651 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb17fff640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110bef0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be95240 msgr2=0x56110bef0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb1cbb8640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be95240 0x56110bef0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be95240 0x56110bef0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 msgr2=0x56110bef0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110bef0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/1089753651 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56110be95070 con 0x56110be94920 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb17fff640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110bef0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ffb0001f5b0 tx=0x7ffb00004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aad5c622c666c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb0d7fa640 1 -- 172.21.15.82:0/1089753651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000b5040 con 0x56110be94920 2024-04-25T10:06:56.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb0d7fa640 1 -- 172.21.15.82:0/1089753651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000c0070 con 0x56110be94920 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb0d7fa640 1 -- 172.21.15.82:0/1089753651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000a8c30 con 0x56110be94920 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/1089753651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 msgr2=0x56110bef0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ffb0001f5b0 tx=0x7ffb00004740 comp rx=0 tx=0).stop 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/1089753651 shutdown_connections 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be95240 0x56110bef0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef0660 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- 172.21.15.82:0/1089753651 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be4af30 0x56110bef0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/1089753651 >> 172.21.15.82:0/1089753651 conn(0x56110be512a0 msgr2=0x56110be4e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/1089753651 shutdown_connections 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/1089753651 wait complete. 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- start start 2024-04-25T10:06:56.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 Processor -- start 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be4af30 0x56110bea16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110bef18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bc9e3d0 con 0x56110be94920 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56110bd55d50 con 0x56110be95240 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56110bc34980 con 0x56110be4af30 2024-04-25T10:06:56.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 10 client.0 _ll_register_callbacks cb 0x7ffb10051a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36044/0 (socket says 172.21.15.82:36044) 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/141429308 learned_addr learned my addr 172.21.15.82:0/141429308 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be4af30 msgr2=0x56110bea16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be4af30 0x56110bea16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 msgr2=0x56110bef18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110bef18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 -- 172.21.15.82:0/141429308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56110bd1b600 con 0x56110be94920 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb177fe640 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ffb000bd040 tx=0x7ffb0001d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d6d3ecd68f53b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb000b5040 con 0x56110be94920 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb000b0500 con 0x56110be94920 2024-04-25T10:06:56.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb000c0070 con 0x56110be94920 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb000cf050 con 0x56110be94920 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 10 client.7146 fetch_fsmap learned FSMap version 41 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) v3 -- 0x56110bc2e9b0 con 0x56110be94920 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb000c0070 con 0x56110be94920 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 41) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb000d2020 con 0x56110be94920 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 10 client.7146 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 10 client.7146 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 20 client.7146 populate_metadata read hostname 'smithi082' 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110bef1de0 0x56110bef21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 4 client.7146 mds_command: new command op to 6966 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.826+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 --> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56110bca72e0 con 0x56110bef1de0 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb17fff640 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110bef1de0 0x56110bef21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffaf6ffd640 10 client.7146 ms_handle_connect on v2:172.21.15.82:6836/910515233 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb17fff640 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110bef1de0 0x56110bef21c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c24390d93aaa417b server_cookie=c69725c2c006cdea in_seq=0 out_seq=0 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffaf6ffd640 1 -- 172.21.15.82:0/141429308 <== mds.0 v2:172.21.15.82:6836/910515233 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x56110bca72e0 con 0x56110bef1de0 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffaf6ffd640 10 client.7146 handle_command_reply: tid=0 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 client.7146 shutdown 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 msgr2=0x56110bef1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7ffb000bd040 tx=0x7ffb0001d550 comp rx=0 tx=0).stop 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 shutdown_connections 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:6836/910515233,v1:172.21.15.82:6837/910515233] conn(0x56110bef1de0 0x56110bef21c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56110be95240 0x56110bef18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56110be94920 0x56110bef1360 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 --2- 172.21.15.82:0/141429308 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56110be4af30 0x56110bea16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffaf6ffd640 0 client.7146 ms_handle_reset on v2:172.21.15.82:6836/910515233 2024-04-25T10:06:56.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 >> 172.21.15.82:0/141429308 conn(0x56110be512a0 msgr2=0x56110be9c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 shutdown_connections 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 1 -- 172.21.15.82:0/141429308 wait complete. 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb21788000 20 client.7146 trim_cache size 0 max 16384 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 6966, 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:06:56.839 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 9.7607044349999992, 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 41, 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 79, 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 79, 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 15.110891375 2024-04-25T10:06:56.840 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 msgr2=0x7ffaf8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 0x7ffaf8069450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ffb080075b0 tx=0x7ffb08007540 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 msgr2=0x7ffb181435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7ffb1000c390 tx=0x7ffb1000c850 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 shutdown_connections 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffaf8066f90 0x7ffaf8069450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb18144ca0 0x7ffb181435d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb180ce510 0x7ffb18143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 --2- 172.21.15.82:0/3049258989 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb1806ec60 0x7ffb1813ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 >> 172.21.15.82:0/3049258989 conn(0x7ffb180b11a0 msgr2=0x7ffb180b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 shutdown_connections 2024-04-25T10:06:56.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.830+0000 7ffb1e61b640 1 -- 172.21.15.82:0/3049258989 wait complete. 2024-04-25T10:06:56.849 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:56.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 Processor -- start 2024-04-25T10:06:56.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 -- start start 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05e40ce510 0x7f05e40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e40cee30 con 0x7f05e406ec60 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e40cc2b0 con 0x7f05e40ce510 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e939d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e406e290 con 0x7f05e4144ca0 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05e40ce510 0x7f05e40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f05e40ce510 0x7f05e40ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36052/0 (socket says 172.21.15.82:36052) 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 -- 172.21.15.82:0/3057521732 learned_addr learned my addr 172.21.15.82:0/3057521732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e37fe640 1 --1- 172.21.15.82:0/3057521732 >> v1:172.21.15.82:6789/0 conn(0x7f05e406ec60 0x7f05e4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59410/0 (socket says 172.21.15.82:59410) 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e27fc640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 -- 172.21.15.82:0/3057521732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 msgr2=0x7f05e406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 -- 172.21.15.82:0/3057521732 >> v1:172.21.15.82:6789/0 conn(0x7f05e406ec60 legacy=0x7f05e4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e1ffb640 1 -- 172.21.15.82:0/3057521732 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141982042 0 0) 0x7f05e40cee30 con 0x7f05e406ec60 2024-04-25T10:06:56.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.954+0000 7f05e2ffd640 1 -- 172.21.15.82:0/3057521732 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e40ccc70 con 0x7f05e40ce510 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e2ffd640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e40ce8f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009e50 tx=0x7f05d401f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5513363fec51cc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e1ffb640 1 -- 172.21.15.82:0/3057521732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40b4070 con 0x7f05e40ce510 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e1ffb640 1 -- 172.21.15.82:0/3057521732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d40af400 con 0x7f05e40ce510 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e1ffb640 1 -- 172.21.15.82:0/3057521732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d40af7d0 con 0x7f05e40ce510 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- 172.21.15.82:0/3057521732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 msgr2=0x7f05e40ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e40ce8f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009e50 tx=0x7f05d401f210 comp rx=0 tx=0).stop 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- 172.21.15.82:0/3057521732 shutdown_connections 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 --2- 172.21.15.82:0/3057521732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e40ce8f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- 172.21.15.82:0/3057521732 >> 172.21.15.82:0/3057521732 conn(0x7f05e40b11a0 msgr2=0x7f05e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- 172.21.15.82:0/3057521732 shutdown_connections 2024-04-25T10:06:56.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- 172.21.15.82:0/3057521732 wait complete. 2024-04-25T10:06:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 Processor -- start 2024-04-25T10:06:56.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 -- start start 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e406ec60 0x7f05e41e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.958+0000 7f05e939d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e41e60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e40cc900 con 0x7f05e40ce510 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e406df20 con 0x7f05e406ec60 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e4142d90 con 0x7f05e4144ca0 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36066/0 (socket says 172.21.15.82:36066) 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 -- 172.21.15.82:0/294073795 learned_addr learned my addr 172.21.15.82:0/294073795 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e37fe640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e41e60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 -- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 msgr2=0x7f05e41e60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e2ffd640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e406ec60 0x7f05e41e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e41e60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 -- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e406ec60 msgr2=0x7f05e41e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e406ec60 0x7f05e41e3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 -- 172.21.15.82:0/294073795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e41e4540 con 0x7f05e40ce510 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e27fc640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c5e0 tx=0x7f05d800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8004230 con 0x7f05e40ce510 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d80043f0 con 0x7f05e40ce510 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8003860 con 0x7f05e40ce510 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e41e4820 con 0x7f05e40ce510 2024-04-25T10:06:56.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e41e4de0 con 0x7f05e40ce510 2024-04-25T10:06:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e404efa0 con 0x7f05e40ce510 2024-04-25T10:06:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05d8003a00 con 0x7f05e40ce510 2024-04-25T10:06:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 0x7f05d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05d80851c0 con 0x7f05e40ce510 2024-04-25T10:06:56.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e2ffd640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 0x7f05d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.962+0000 7f05e2ffd640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 0x7f05d0069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05d40b1000 tx=0x7f05d40aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:56.966+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05d8015020 con 0x7f05e40ce510 2024-04-25T10:06:57.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05e40b1cc0 con 0x7f05e40ce510 2024-04-25T10:06:57.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05cb7fe640 1 -- 172.21.15.82:0/294073795 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05d8056040 con 0x7f05e40ce510 2024-04-25T10:06:57.149 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 msgr2=0x7f05d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 0x7f05d0069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05d40b1000 tx=0x7f05d40aa000 comp rx=0 tx=0).stop 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 msgr2=0x7f05e41e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f05d800c5e0 tx=0x7f05d800caa0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 shutdown_connections 2024-04-25T10:06:57.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f05d0066f60 0x7f05d0069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05e4144ca0 0x7f05e41e60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05e40ce510 0x7f05e41e5ba0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 --2- 172.21.15.82:0/294073795 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05e406ec60 0x7f05e41e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 >> 172.21.15.82:0/294073795 conn(0x7f05e40b11a0 msgr2=0x7f05e40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 shutdown_connections 2024-04-25T10:06:57.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:57.146+0000 7f05e939d640 1 -- 172.21.15.82:0/294073795 wait complete. 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:57.166 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:06:57.167 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:06:57.167 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:57.167 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:57.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.167 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:57.173 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42253sec preferred_lft 42253sec 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:06:57.174 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:57.175 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:57.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.176 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:57.225 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:06:57.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.227 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:57.276 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:06:57.277 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:57.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.277 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:06:57.277 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:06:57.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:06:57.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:06:57.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:06:57.278 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:06:57.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.349 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:06:57.396 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:57.396 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:57.397 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:57.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:06:57.452 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:06:57.452 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:57.453 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:57.500 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:57.500 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:57.500 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:57.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:06:57.605 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:57.653 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:06:57.653 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:06:57.653 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:06:57.653 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:06:57.653 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:57.654 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:57.655 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:57.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.656 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:06:57.704 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:06:57.704 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:06:57.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:06:57.765 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:06:57.812 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:06:57.813 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:57.814 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:57.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.815 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:06:57.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:06:57.826+0000 7fd9d1434580 -1 init, newargv = 0x55ff20fedfe0 newargc=15 2024-04-25T10:06:57.828 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45252]: starting ceph client 2024-04-25T10:06:57.834 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45252]: starting fuse 2024-04-25T10:06:58.933 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:06:58.950 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:06:58.997 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:06:58.997 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:06:58.997 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:06:58.997 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:06:58.998 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:06:58.999 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:59.000 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:59.000 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:59.000 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:59.000 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.001 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:06:59.048 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:06:59.048 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:06:59.048 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:06:59.049 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:06:59.050 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:06:59.173 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.45252.asok 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:59.185 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45252.asok 2024-04-25T10:06:59.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45252.asok status 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "45252", 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:59.339 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7161, 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 7161 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3243013765 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:59.340 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3243013765 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.7161 192.168.0.1:0/3243013765", 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/3243013765", 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 41, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 79, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:06:59.341 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:06:59.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.352 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:06:59.352 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.358 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:06:59.358 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.359 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:59.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.412 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T10:06:59.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.470 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T10:06:59.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:59.523 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T10:06:59.577 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:06:59.687 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:06:59.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:59.697 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:06:59.697 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:59.744 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:59.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.918+0000 7f0751d18640 1 Processor -- start 2024-04-25T10:06:59.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.918+0000 7f0751d18640 1 -- start start 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f0751d18640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f074c09a480 0x7f074c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f0751d18640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f074c0cfb80 0x7f074c0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f0751d18640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f074c0d04d0 con 0x7f074c15b950 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f0751d18640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c15bd30 con 0x7f074c09a480 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f0751d18640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f074c0cc0d0 con 0x7f074c0cfb80 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f074c15b950 0x7f074c0cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59426/0 (socket says 172.21.15.82:59426) 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074b7fe640 1 -- 172.21.15.82:0/2337262905 learned_addr learned my addr 172.21.15.82:0/2337262905 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 --2- 172.21.15.82:0/2337262905 >> v2:172.21.15.82:3300/0 conn(0x7f074c09a480 0x7f074c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074bfff640 1 --2- 172.21.15.82:0/2337262905 >> v2:172.21.15.196:3300/0 conn(0x7f074c0cfb80 0x7f074c0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074a7fc640 1 -- 172.21.15.82:0/2337262905 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091300527 0 0) 0x7f073c001120 con 0x7f074c15b950 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074a7fc640 1 -- 172.21.15.82:0/2337262905 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0734003530 con 0x7f074c15b950 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 -- 172.21.15.82:0/2337262905 >> v2:172.21.15.196:3300/0 conn(0x7f074c0cfb80 msgr2=0x7f074c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 --2- 172.21.15.82:0/2337262905 >> v2:172.21.15.196:3300/0 conn(0x7f074c0cfb80 0x7f074c0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 -- 172.21.15.82:0/2337262905 >> v1:172.21.15.82:6789/0 conn(0x7f074c15b950 legacy=0x7f074c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 -- 172.21.15.82:0/2337262905 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c09ab10 con 0x7f074c09a480 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074affd640 1 --2- 172.21.15.82:0/2337262905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c0cce50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f074c0b3bd0 tx=0x7f074001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3fe1fc271da04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074a7fc640 1 -- 172.21.15.82:0/2337262905 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400b0070 con 0x7f074c09a480 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074a7fc640 1 -- 172.21.15.82:0/2337262905 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07400ab600 con 0x7f074c09a480 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.922+0000 7f074a7fc640 1 -- 172.21.15.82:0/2337262905 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400ab9b0 con 0x7f074c09a480 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/2337262905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 msgr2=0x7f074c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- 172.21.15.82:0/2337262905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c0cce50 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f074c0b3bd0 tx=0x7f074001c480 comp rx=0 tx=0).stop 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/2337262905 shutdown_connections 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- 172.21.15.82:0/2337262905 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c0cce50 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- 172.21.15.82:0/2337262905 >> v2:172.21.15.196:3300/0 conn(0x7f074c0cfb80 0x7f074c0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/2337262905 >> 172.21.15.82:0/2337262905 conn(0x7f074c0b30b0 msgr2=0x7f074c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/2337262905 shutdown_connections 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/2337262905 wait complete. 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 Processor -- start 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- start start 2024-04-25T10:06:59.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 0x7f074c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f074c15b950 0x7f074c15d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0cd390 con 0x7f074c09a480 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074c06faa0 con 0x7f074c0cfb80 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074c0af8b0 con 0x7f074c15b950 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36088/0 (socket says 172.21.15.82:36088) 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 -- 172.21.15.82:0/209996168 learned_addr learned my addr 172.21.15.82:0/209996168 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074affd640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 0x7f074c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 -- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f074c15b950 msgr2=0x7f074c15d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074bfff640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f074c15b950 0x7f074c15d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f074c15b950 0x7f074c15d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 -- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 msgr2=0x7f074c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 0x7f074c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 -- 172.21.15.82:0/209996168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074c0af5d0 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074affd640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 0x7f074c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f074b7fe640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c3f0 tx=0x7f073c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c017070 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c005cf0 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074c15ca60 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c007310 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.926+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f074c15d020 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f074c04efa0 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f073c01a070 con 0x7f074c09a480 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f0733fff640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 0x7f0738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f073c084570 con 0x7f074c09a480 2024-04-25T10:06:59.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f074affd640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 0x7f0738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.930+0000 7f074affd640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 0x7f0738069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f07400ad000 tx=0x7f07400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:06:59.934+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f074c0d04d0 con 0x7f074c09a480 2024-04-25T10:07:00.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.094+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f074c157590 con 0x7f074c09a480 2024-04-25T10:07:00.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.094+0000 7f0733fff640 1 -- 172.21.15.82:0/209996168 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16957 (secure 0 0 0) 0x7f073c055480 con 0x7f074c09a480 2024-04-25T10:07:00.099 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:00.099 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":79,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:47.754126+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:00.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 msgr2=0x7f0738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 0x7f0738069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f07400ad000 tx=0x7f07400a6000 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 msgr2=0x7f074c14ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f073c00c3f0 tx=0x7f073c00c8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 shutdown_connections 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0738066f60 0x7f0738069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f074c15b950 0x7f074c15d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f074c0cfb80 0x7f074c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 --2- 172.21.15.82:0/209996168 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f074c09a480 0x7f074c14ed20 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 >> 172.21.15.82:0/209996168 conn(0x7f074c0b30b0 msgr2=0x7f074c0b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 shutdown_connections 2024-04-25T10:07:00.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.098+0000 7f0751d18640 1 -- 172.21.15.82:0/209996168 wait complete. 2024-04-25T10:07:00.120 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:00.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:00.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.242+0000 7f3b4f30a640 1 Processor -- start 2024-04-25T10:07:00.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.242+0000 7f3b4f30a640 1 -- start start 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4f30a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b480cd350 0x7f3b4815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4f30a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b4815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4f30a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b480ce320 con 0x7f3b4815aac0 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4f30a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b480ce0d0 con 0x7f3b480cd350 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4f30a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480ce600 con 0x7f3b480cf0e0 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b480cd350 0x7f3b4815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b480cd350 0x7f3b4815aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36100/0 (socket says 172.21.15.82:36100) 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 -- 172.21.15.82:0/3077169468 learned_addr learned my addr 172.21.15.82:0/3077169468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4d8a7640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b4815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 -- 172.21.15.82:0/3077169468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 msgr2=0x7f3b4815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b4815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 -- 172.21.15.82:0/3077169468 >> v1:172.21.15.196:6789/0 conn(0x7f3b4815aac0 legacy=0x7f3b480ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 -- 172.21.15.82:0/3077169468 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480ce8e0 con 0x7f3b480cd350 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b4c8a5640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b4815aea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009870 tx=0x7f3b3801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ca01424d0ac03e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b3ffff640 1 -- 172.21.15.82:0/3077169468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380a7070 con 0x7f3b480cd350 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.246+0000 7f3b3ffff640 1 -- 172.21.15.82:0/3077169468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b380b0070 con 0x7f3b480cd350 2024-04-25T10:07:00.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b3ffff640 1 -- 172.21.15.82:0/3077169468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b380ab890 con 0x7f3b480cd350 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/3077169468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 msgr2=0x7f3b4815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b4815aea0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f3b38009870 tx=0x7f3b3801c560 comp rx=0 tx=0).stop 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/3077169468 shutdown_connections 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b4815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/3077169468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b4815aea0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/3077169468 >> 172.21.15.82:0/3077169468 conn(0x7f3b48099a70 msgr2=0x7f3b480d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/3077169468 shutdown_connections 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/3077169468 wait complete. 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 Processor -- start 2024-04-25T10:07:00.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- start start 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b480c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 0x7f3b48143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480cbbb0 con 0x7f3b480cd350 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480cba30 con 0x7f3b4815aac0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b480cbd30 con 0x7f3b480cf0e0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36116/0 (socket says 172.21.15.82:36116) 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 -- 172.21.15.82:0/4257985822 learned_addr learned my addr 172.21.15.82:0/4257985822 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d8a7640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 0x7f3b48143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4c8a5640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b480c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 -- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 msgr2=0x7f3b480c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b480c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 -- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 msgr2=0x7f3b48143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 0x7f3b48143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 -- 172.21.15.82:0/4257985822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48144420 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d8a7640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 0x7f3b48143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4d0a6640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400c760 tx=0x7f3b4400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44016020 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b44006d60 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b48144700 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b44003bc0 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.250+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b480c8c40 con 0x7f3b480cd350 2024-04-25T10:07:00.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4804efa0 con 0x7f3b480cd350 2024-04-25T10:07:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b4400a850 con 0x7f3b480cd350 2024-04-25T10:07:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b3dffb640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b44084e50 con 0x7f3b480cd350 2024-04-25T10:07:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b4c8a5640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.254+0000 7f3b4c8a5640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 0x7f3b2c069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3b380af000 tx=0x7f3b380a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.258+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b44020020 con 0x7f3b480cd350 2024-04-25T10:07:00.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.418+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b480c47c0 con 0x7f3b480cd350 2024-04-25T10:07:00.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.418+0000 7f3b3dffb640 1 -- 172.21.15.82:0/4257985822 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b44055d60 con 0x7f3b480cd350 2024-04-25T10:07:00.423 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 msgr2=0x7f3b2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 0x7f3b2c069420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3b380af000 tx=0x7f3b380a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 msgr2=0x7f3b481552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400c760 tx=0x7f3b4400cc20 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 shutdown_connections 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b2c066f60 0x7f3b2c069420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b480cf0e0 0x7f3b480c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b4815aac0 0x7f3b48143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 --2- 172.21.15.82:0/4257985822 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b480cd350 0x7f3b481552f0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 >> 172.21.15.82:0/4257985822 conn(0x7f3b48099a70 msgr2=0x7f3b480ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 shutdown_connections 2024-04-25T10:07:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.422+0000 7f3b4f30a640 1 -- 172.21.15.82:0/4257985822 wait complete. 2024-04-25T10:07:00.441 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.558+0000 7f6c149de640 1 Processor -- start 2024-04-25T10:07:00.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.558+0000 7f6c149de640 1 -- start start 2024-04-25T10:07:00.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c1015ecb0 0x7f6c100c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100ca4c0 con 0x7f6c100cf590 2024-04-25T10:07:00.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c100cd690 con 0x7f6c100c9ef0 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100ca780 con 0x7f6c1015ecb0 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c100cf590 0x7f6c100cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59430/0 (socket says 172.21.15.82:59430) 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0e59c640 1 -- 172.21.15.82:0/4005563602 learned_addr learned my addr 172.21.15.82:0/4005563602 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0dd9b640 1 --2- 172.21.15.82:0/4005563602 >> v2:172.21.15.82:3300/0 conn(0x7f6c1015ecb0 0x7f6c100c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0d59a640 1 -- 172.21.15.82:0/4005563602 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916851274 0 0) 0x7f6c100ca4c0 con 0x7f6c100cf590 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0d59a640 1 -- 172.21.15.82:0/4005563602 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bf8003530 con 0x7f6c100cf590 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0dd9b640 1 -- 172.21.15.82:0/4005563602 >> v1:172.21.15.196:6789/0 conn(0x7f6c100c9ef0 legacy=0x7f6c100ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0dd9b640 1 -- 172.21.15.82:0/4005563602 >> v1:172.21.15.82:6789/0 conn(0x7f6c100cf590 legacy=0x7f6c100cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0dd9b640 1 -- 172.21.15.82:0/4005563602 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100cd970 con 0x7f6c1015ecb0 2024-04-25T10:07:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0dd9b640 1 --2- 172.21.15.82:0/4005563602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1015ecb0 0x7f6c100c99b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097c0 tx=0x7f6c0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49ef4670d0727d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0d59a640 1 -- 172.21.15.82:0/4005563602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040b1070 con 0x7f6c1015ecb0 2024-04-25T10:07:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0d59a640 1 -- 172.21.15.82:0/4005563602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c040ac410 con 0x7f6c1015ecb0 2024-04-25T10:07:00.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c0d59a640 1 -- 172.21.15.82:0/4005563602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c040ac7c0 con 0x7f6c1015ecb0 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- 172.21.15.82:0/4005563602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1015ecb0 msgr2=0x7f6c100c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 --2- 172.21.15.82:0/4005563602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1015ecb0 0x7f6c100c99b0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097c0 tx=0x7f6c0401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- 172.21.15.82:0/4005563602 shutdown_connections 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 --2- 172.21.15.82:0/4005563602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c1015ecb0 0x7f6c100c99b0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.562+0000 7f6c149de640 1 -- 172.21.15.82:0/4005563602 >> 172.21.15.82:0/4005563602 conn(0x7f6c1006ae60 msgr2=0x7f6c1006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- 172.21.15.82:0/4005563602 shutdown_connections 2024-04-25T10:07:00.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- 172.21.15.82:0/4005563602 wait complete. 2024-04-25T10:07:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 Processor -- start 2024-04-25T10:07:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- start start 2024-04-25T10:07:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c100cf590 0x7f6c10164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1015ecb0 0x7f6c101582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100cdce0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100cd440 con 0x7f6c1015ecb0 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c10168f10 con 0x7f6c100cf590 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36130/0 (socket says 172.21.15.82:36130) 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 -- 172.21.15.82:0/3758631756 learned_addr learned my addr 172.21.15.82:0/3758631756 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 -- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c100cf590 msgr2=0x7f6c10164ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c100cf590 0x7f6c10164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 -- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1015ecb0 msgr2=0x7f6c101582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1015ecb0 0x7f6c101582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 -- 172.21.15.82:0/3758631756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c10163a10 con 0x7f6c100c9ef0 2024-04-25T10:07:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c0e59c640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c500 tx=0x7f6c0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00017070 con 0x7f6c100c9ef0 2024-04-25T10:07:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c00005cf0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10163cf0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000072f0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c101642b0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.566+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1004efa0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.570+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c0001a070 con 0x7f6c100c9ef0 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.570+0000 7f6bfeffd640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.570+0000 7f6c0dd9b640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.570+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c00084500 con 0x7f6c100c9ef0 2024-04-25T10:07:00.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.570+0000 7f6c0dd9b640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 0x7f6bf0069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c040ae000 tx=0x7f6c04004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.574+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c00054d30 con 0x7f6c100c9ef0 2024-04-25T10:07:00.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c100ae3a0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6bfeffd640 1 -- 172.21.15.82:0/3758631756 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c00058dd0 con 0x7f6c100c9ef0 2024-04-25T10:07:00.737 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 msgr2=0x7f6bf0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 0x7f6bf0069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6c040ae000 tx=0x7f6c04004740 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 msgr2=0x7f6c1006cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c500 tx=0x7f6c0000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 shutdown_connections 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6bf0066f60 0x7f6bf0069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c100cf590 0x7f6c10164ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c1015ecb0 0x7f6c101582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 --2- 172.21.15.82:0/3758631756 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c100c9ef0 0x7f6c1006cf50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 >> 172.21.15.82:0/3758631756 conn(0x7f6c1006ae60 msgr2=0x7f6c1015a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 shutdown_connections 2024-04-25T10:07:00.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.734+0000 7f6c149de640 1 -- 172.21.15.82:0/3758631756 wait complete. 2024-04-25T10:07:00.756 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:00.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 Processor -- start 2024-04-25T10:07:00.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 -- start start 2024-04-25T10:07:00.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46600cba90 0x7f46600cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4660071b70 con 0x7f4660159bd0 2024-04-25T10:07:00.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4660071e50 con 0x7f4660159fb0 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4667d23640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4660072780 con 0x7f46600cba90 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4665abf640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4660159bd0 0x7f46600c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59436/0 (socket says 172.21.15.82:59436) 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4665abf640 1 -- 172.21.15.82:0/823612651 learned_addr learned my addr 172.21.15.82:0/823612651 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f46662c0640 1 --2- 172.21.15.82:0/823612651 >> v2:172.21.15.82:3300/0 conn(0x7f46600cba90 0x7f46600cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f46662c0640 1 -- 172.21.15.82:0/823612651 >> v1:172.21.15.196:6789/0 conn(0x7f4660159fb0 legacy=0x7f46600cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f46662c0640 1 -- 172.21.15.82:0/823612651 >> v1:172.21.15.82:6789/0 conn(0x7f4660159bd0 legacy=0x7f46600c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f4664abd640 1 -- 172.21.15.82:0/823612651 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370220038 0 0) 0x7f4660071b70 con 0x7f4660159bd0 2024-04-25T10:07:00.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.878+0000 7f46662c0640 1 -- 172.21.15.82:0/823612651 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660072130 con 0x7f46600cba90 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 --2- 172.21.15.82:0/823612651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46600cba90 0x7f46600cbf40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aaa11329c2a2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4664abd640 1 -- 172.21.15.82:0/823612651 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580b2070 con 0x7f46600cba90 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4664abd640 1 -- 172.21.15.82:0/823612651 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46580a59a0 con 0x7f46600cba90 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4664abd640 1 -- 172.21.15.82:0/823612651 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580a5d70 con 0x7f46600cba90 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- 172.21.15.82:0/823612651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46600cba90 msgr2=0x7f46600cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 --2- 172.21.15.82:0/823612651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46600cba90 0x7f46600cbf40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f4658009d90 tx=0x7f465801c940 comp rx=0 tx=0).stop 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- 172.21.15.82:0/823612651 shutdown_connections 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 --2- 172.21.15.82:0/823612651 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46600cba90 0x7f46600cbf40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- 172.21.15.82:0/823612651 >> 172.21.15.82:0/823612651 conn(0x7f466006ae60 msgr2=0x7f466006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- 172.21.15.82:0/823612651 shutdown_connections 2024-04-25T10:07:00.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- 172.21.15.82:0/823612651 wait complete. 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 Processor -- start 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- start start 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46600cba90 0x7f466015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 0x7f46600b2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660167e30 con 0x7f4660159fb0 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660167cb0 con 0x7f4660159bd0 2024-04-25T10:07:00.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4667d23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4660167fb0 con 0x7f46600cba90 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36150/0 (socket says 172.21.15.82:36150) 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 -- 172.21.15.82:0/1299752337 learned_addr learned my addr 172.21.15.82:0/1299752337 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46652be640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 0x7f46600b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f4665abf640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46600cba90 0x7f466015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 -- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46600cba90 msgr2=0x7f466015aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46600cba90 0x7f466015aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 -- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 msgr2=0x7f46600b2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 0x7f46600b2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46662c0640 1 -- 172.21.15.82:0/1299752337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466015c3d0 con 0x7f4660159fb0 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.882+0000 7f46652be640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 0x7f46600b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46662c0640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f46580b0040 tx=0x7f46580196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580b2040 con 0x7f4660159fb0 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46580b8530 con 0x7f4660159fb0 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46601f8240 con 0x7f4660159fb0 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580b88e0 con 0x7f4660159fb0 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466015c930 con 0x7f4660159fb0 2024-04-25T10:07:00.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466004efa0 con 0x7f4660159fb0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46580c1020 con 0x7f4660159fb0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 0x7f4644069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f465811fcf0 con 0x7f4660159fb0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f4665abf640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 0x7f4644069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:00.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.886+0000 7f4665abf640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 0x7f4644069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f465c000f70 tx=0x7f465c007bc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:00.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:00.890+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4658124070 con 0x7f4660159fb0 2024-04-25T10:07:01.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.050+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4660156ed0 con 0x7f4660159fb0 2024-04-25T10:07:01.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.050+0000 7f46567fc640 1 -- 172.21.15.82:0/1299752337 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f466015c3d0 con 0x7f4660159fb0 2024-04-25T10:07:01.053 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 msgr2=0x7f4644069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 0x7f4644069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f465c000f70 tx=0x7f465c007bc0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 msgr2=0x7f46600b2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f46580b0040 tx=0x7f46580196e0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 shutdown_connections 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4644066f60 0x7f4644069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4660159fb0 0x7f46600b2ad0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660159bd0 0x7f46600b2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 --2- 172.21.15.82:0/1299752337 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46600cba90 0x7f466015aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 >> 172.21.15.82:0/1299752337 conn(0x7f466006ae60 msgr2=0x7f4660153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 shutdown_connections 2024-04-25T10:07:01.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.054+0000 7f4667d23640 1 -- 172.21.15.82:0/1299752337 wait complete. 2024-04-25T10:07:01.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:01.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 Processor -- start 2024-04-25T10:07:01.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 -- start start 2024-04-25T10:07:01.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60c00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f60c00d1cd0 0x7f60c00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c006be60 con 0x7f60c00d0620 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c006c440 con 0x7f60c00d1cd0 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60c5eb9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c015de60 con 0x7f60c015da80 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60bffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f60c00d0620 0x7f60c00ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59450/0 (socket says 172.21.15.82:59450) 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60bffff640 1 -- 172.21.15.82:0/4068564606 learned_addr learned my addr 172.21.15.82:0/4068564606 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 --2- 172.21.15.82:0/4068564606 >> v2:172.21.15.196:3300/0 conn(0x7f60c00d1cd0 0x7f60c00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60bf7fe640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60c00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667861561 0 0) 0x7f60c006be60 con 0x7f60c00d0620 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a8003530 con 0x7f60c00d0620 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 532816876 0 0) 0x7f60a8003530 con 0x7f60c00d0620 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 msgr2=0x7f60c00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60c00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 >> v2:172.21.15.196:3300/0 conn(0x7f60c00d1cd0 msgr2=0x7f60c00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 --2- 172.21.15.82:0/4068564606 >> v2:172.21.15.196:3300/0 conn(0x7f60c00d1cd0 0x7f60c00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00d2450 con 0x7f60c00d0620 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f60b8001a00 con 0x7f60c00d0620 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 >> v1:172.21.15.82:6789/0 conn(0x7f60c00d0620 legacy=0x7f60c00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a8003a30 0x7f60a8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c006c440 con 0x7f60a8004810 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c015de60 con 0x7f60a8003a30 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a800b9d0 con 0x7f60c015da80 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60bffff640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 -- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 msgr2=0x7f60a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015da80 0x7f60a80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 -- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a8003a30 msgr2=0x7f60a8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60a8003a30 0x7f60a8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.186+0000 7f60beffd640 1 -- 172.21.15.82:0/4068564606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8003530 con 0x7f60a8004810 2024-04-25T10:07:01.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60beffd640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60a8004c90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009af0 tx=0x7f60b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106c13290d0bd759 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0009690 con 0x7f60a8004810 2024-04-25T10:07:01.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b00059a0 con 0x7f60a8004810 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60be7fc640 1 -- 172.21.15.82:0/4068564606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0005d90 con 0x7f60a8004810 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4068564606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 msgr2=0x7f60a8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4068564606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60a8004c90 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009af0 tx=0x7f60b001ca70 comp rx=0 tx=0).stop 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60bffff640 1 -- 172.21.15.82:0/4068564606 reap_dead start 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4068564606 shutdown_connections 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4068564606 >> 172.21.15.82:0/4068564606 conn(0x7f60c008b000 msgr2=0x7f60c00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4068564606 shutdown_connections 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4068564606 wait complete. 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 Processor -- start 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.190+0000 7f60c5eb9640 1 -- start start 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c00cc510 0x7f60c00b7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 0x7f60c00b1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00d26f0 con 0x7f60a8004810 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00d20f0 con 0x7f60c00cc510 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00d2870 con 0x7f60c015a7a0 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36160/0 (socket says 172.21.15.82:36160) 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 -- 172.21.15.82:0/4220915895 learned_addr learned my addr 172.21.15.82:0/4220915895 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60beffd640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c00cc510 0x7f60c00b7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bffff640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 0x7f60c00b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 -- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 msgr2=0x7f60c00b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 0x7f60c00b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 -- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c00cc510 msgr2=0x7f60c00b7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c00cc510 0x7f60c00b7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 -- 172.21.15.82:0/4220915895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c00cc970 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bffff640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 0x7f60c00b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60bf7fe640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e7c0 tx=0x7f60b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b4017070 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b400c940 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c00b6a10 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b400cd30 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c00b6fd0 con 0x7f60a8004810 2024-04-25T10:07:01.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c004efa0 con 0x7f60a8004810 2024-04-25T10:07:01.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60b401a070 con 0x7f60a8004810 2024-04-25T10:07:01.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 0x7f60ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f60b4084ff0 con 0x7f60a8004810 2024-04-25T10:07:01.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60beffd640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 0x7f60ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.194+0000 7f60beffd640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 0x7f60ac069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60b00ae000 tx=0x7f60b00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.198+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60b4055e70 con 0x7f60a8004810 2024-04-25T10:07:01.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.358+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f60c00b0640 con 0x7f60a8004810 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60a7fff640 1 -- 172.21.15.82:0/4220915895 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) v1 ==== 80+0+0 (secure 0 0 0) 0x7f60b4055770 con 0x7f60a8004810 2024-04-25T10:07:01.364 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:07:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 msgr2=0x7f60ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 0x7f60ac069420 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60b00ae000 tx=0x7f60b00046d0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 msgr2=0x7f60c00b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f60b400e7c0 tx=0x7f60b400ec80 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 shutdown_connections 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60ac066f60 0x7f60ac069420 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60c015a7a0 0x7f60c00b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60c00cc510 0x7f60c00b7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 --2- 172.21.15.82:0/4220915895 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60a8004810 0x7f60c00b6310 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 >> 172.21.15.82:0/4220915895 conn(0x7f60c008b000 msgr2=0x7f60c00b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 shutdown_connections 2024-04-25T10:07:01.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.362+0000 7f60c5eb9640 1 -- 172.21.15.82:0/4220915895 wait complete. 2024-04-25T10:07:01.381 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:07:01.381 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:01.387 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:01.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.546+0000 7f845e296640 1 Processor -- start 2024-04-25T10:07:01.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.546+0000 7f845e296640 1 -- start start 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845e296640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f845815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845e296640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84580b3f50 0x7f84580cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845e296640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84580b2090 con 0x7f84580ce430 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845e296640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845815f200 con 0x7f84580b3f50 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845e296640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b43e0 con 0x7f845815a700 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84580b3f50 0x7f84580cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845c833640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f84580ce430 0x7f84580b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59454/0 (socket says 172.21.15.82:59454) 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84580b3f50 0x7f84580cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36168/0 (socket says 172.21.15.82:36168) 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f845c833640 1 -- 172.21.15.82:0/3004066249 learned_addr learned my addr 172.21.15.82:0/3004066249 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f8457fff640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f845815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 -- 172.21.15.82:0/3004066249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 msgr2=0x7f845815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f845815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 -- 172.21.15.82:0/3004066249 >> v1:172.21.15.82:6789/0 conn(0x7f84580ce430 legacy=0x7f84580b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 -- 172.21.15.82:0/3004066249 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84580b46c0 con 0x7f84580b3f50 2024-04-25T10:07:01.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f8456ffd640 1 -- 172.21.15.82:0/3004066249 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656025761 0 0) 0x7f84580b2090 con 0x7f84580ce430 2024-04-25T10:07:01.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f84577fe640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84580b3f50 0x7f84580cdef0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f844c0097c0 tx=0x7f844c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5da89eb65ceba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f8456ffd640 1 -- 172.21.15.82:0/3004066249 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c0b0070 con 0x7f84580b3f50 2024-04-25T10:07:01.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f8456ffd640 1 -- 172.21.15.82:0/3004066249 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c004230 con 0x7f84580b3f50 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.550+0000 7f8456ffd640 1 -- 172.21.15.82:0/3004066249 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c003620 con 0x7f84580b3f50 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/3004066249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84580b3f50 msgr2=0x7f84580cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84580b3f50 0x7f84580cdef0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f844c0097c0 tx=0x7f844c01c350 comp rx=0 tx=0).stop 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/3004066249 shutdown_connections 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84580b3f50 0x7f84580cdef0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- 172.21.15.82:0/3004066249 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f845815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/3004066249 >> 172.21.15.82:0/3004066249 conn(0x7f84580b11a0 msgr2=0x7f84580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/3004066249 shutdown_connections 2024-04-25T10:07:01.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/3004066249 wait complete. 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 Processor -- start 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- start start 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 0x7f84580a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f8458145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580cd490 con 0x7f84581467d0 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84580cd310 con 0x7f84580ce430 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84580b2bb0 con 0x7f845815a700 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36174/0 (socket says 172.21.15.82:36174) 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 -- 172.21.15.82:0/4087962872 learned_addr learned my addr 172.21.15.82:0/4087962872 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f84577fe640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f8458145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f8457fff640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 0x7f84580a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 -- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 msgr2=0x7f8458145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f8458145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 -- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 msgr2=0x7f84580a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 0x7f84580a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 -- 172.21.15.82:0/4087962872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8458158040 con 0x7f84581467d0 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f8457fff640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 0x7f84580a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f84577fe640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f8458145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845c833640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f845000c7a0 tx=0x7f845000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8450018070 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f845000a5e0 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8458158320 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f845000aa50 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.554+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84581588e0 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845804efa0 con 0x7f84581467d0 2024-04-25T10:07:01.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f845001b070 con 0x7f84581467d0 2024-04-25T10:07:01.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f8454ff9640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f8457fff640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8450084790 con 0x7f84581467d0 2024-04-25T10:07:01.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.558+0000 7f8457fff640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 0x7f8444069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8448009840 tx=0x7f8448008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.562+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8450055610 con 0x7f84581467d0 2024-04-25T10:07:01.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.718+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f845815d450 con 0x7f84581467d0 2024-04-25T10:07:01.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.722+0000 7f8454ff9640 1 -- 172.21.15.82:0/4087962872 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16957 (secure 0 0 0) 0x7f8450054f10 con 0x7f84581467d0 2024-04-25T10:07:01.726 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:01.726 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":79,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:06:47.754126+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 msgr2=0x7f8444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 0x7f8444069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f8448009840 tx=0x7f8448008040 comp rx=0 tx=0).stop 2024-04-25T10:07:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 msgr2=0x7f84580a8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f845000c7a0 tx=0x7f845000cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:01.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 shutdown_connections 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8444066f60 0x7f8444069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f845815a700 0x7f8458145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84581467d0 0x7f84580a8e40 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 --2- 172.21.15.82:0/4087962872 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84580ce430 0x7f84580a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 >> 172.21.15.82:0/4087962872 conn(0x7f84580b11a0 msgr2=0x7f84580af680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 shutdown_connections 2024-04-25T10:07:01.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.726+0000 7f845e296640 1 -- 172.21.15.82:0/4087962872 wait complete. 2024-04-25T10:07:01.745 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:01.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:01.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.866+0000 7f7de2d2b640 1 Processor -- start 2024-04-25T10:07:01.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.866+0000 7f7de2d2b640 1 -- start start 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de2d2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc15b420 0x7f7ddc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de2d2b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ddc15b800 0x7f7ddc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de2d2b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc0cbd20 con 0x7f7ddc0cfad0 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de2d2b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cff20 con 0x7f7ddc15b800 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de2d2b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cc930 con 0x7f7ddc15b420 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ddc15b800 0x7f7ddc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7ddc15b800 0x7f7ddc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36188/0 (socket says 172.21.15.82:36188) 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3428352610 learned_addr learned my addr 172.21.15.82:0/3428352610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7de0ac7640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc15b420 0x7f7ddc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3428352610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc15b420 msgr2=0x7f7ddc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc15b420 0x7f7ddc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3428352610 >> v1:172.21.15.196:6789/0 conn(0x7f7ddc0cfad0 legacy=0x7f7ddc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:01.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3428352610 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0cc000 con 0x7f7ddc15b800 2024-04-25T10:07:01.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddbfff640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b800 0x7f7ddc0cf590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009e50 tx=0x7f7dc401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d045ac8ddb8e195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddb7fe640 1 -- 172.21.15.82:0/3428352610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40a5840 con 0x7f7ddc15b800 2024-04-25T10:07:01.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddb7fe640 1 -- 172.21.15.82:0/3428352610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc40a59e0 con 0x7f7ddc15b800 2024-04-25T10:07:01.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.870+0000 7f7ddb7fe640 1 -- 172.21.15.82:0/3428352610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40a5db0 con 0x7f7ddc15b800 2024-04-25T10:07:01.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3428352610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b800 msgr2=0x7f7ddc0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b800 0x7f7ddc0cf590 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4009e50 tx=0x7f7dc401c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3428352610 shutdown_connections 2024-04-25T10:07:01.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b800 0x7f7ddc0cf590 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3428352610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc15b420 0x7f7ddc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3428352610 >> 172.21.15.82:0/3428352610 conn(0x7f7ddc06ae60 msgr2=0x7f7ddc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:01.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3428352610 shutdown_connections 2024-04-25T10:07:01.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3428352610 wait complete. 2024-04-25T10:07:01.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 Processor -- start 2024-04-25T10:07:01.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- start start 2024-04-25T10:07:01.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc0cfad0 0x7f7ddc0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ddc15b800 0x7f7ddc15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0caa80 con 0x7f7ddc15b420 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0ca900 con 0x7f7ddc15b800 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cadb0 con 0x7f7ddc0cfad0 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36202/0 (socket says 172.21.15.82:36202) 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3076035220 learned_addr learned my addr 172.21.15.82:0/3076035220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de0ac7640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc0cfad0 0x7f7ddc0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de12c8640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ddc15b800 0x7f7ddc15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc0cfad0 msgr2=0x7f7ddc0b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc0cfad0 0x7f7ddc0b0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ddc15b800 msgr2=0x7f7ddc15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:01.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ddc15b800 0x7f7ddc15c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 -- 172.21.15.82:0/3076035220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0b1ad0 con 0x7f7ddc15b420 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7ddbfff640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40af040 tx=0x7f7dc4019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40bc020 con 0x7f7ddc15b420 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc40b86b0 con 0x7f7ddc15b420 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc0b3550 con 0x7f7ddc15b420 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc40b8aa0 con 0x7f7ddc15b420 2024-04-25T10:07:01.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.874+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc15be90 con 0x7f7ddc15b420 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc04efa0 con 0x7f7ddc15b420 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dc40bf020 con 0x7f7ddc15b420 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7dd97fa640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7dc4120320 con 0x7f7ddc15b420 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7de0ac7640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:01.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.878+0000 7f7de0ac7640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 0x7f7dbc069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b9d0 tx=0x7f7dcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:01.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:01.882+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dc40c6030 con 0x7f7ddc15b420 2024-04-25T10:07:02.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.026+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ddc073d90 con 0x7f7ddc15b420 2024-04-25T10:07:02.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7dd97fa640 1 -- 172.21.15.82:0/3076035220 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7dc40165e0 con 0x7f7ddc15b420 2024-04-25T10:07:02.032 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 msgr2=0x7f7dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 0x7f7dbc069420 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00b9d0 tx=0x7f7dcc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 msgr2=0x7f7ddc1553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40af040 tx=0x7f7dc4019760 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 shutdown_connections 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dbc066f60 0x7f7dbc069420 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ddc15b800 0x7f7ddc15c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ddc15b420 0x7f7ddc1553c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 --2- 172.21.15.82:0/3076035220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ddc0cfad0 0x7f7ddc0b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 >> 172.21.15.82:0/3076035220 conn(0x7f7ddc06ae60 msgr2=0x7f7ddc15a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 shutdown_connections 2024-04-25T10:07:02.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.030+0000 7f7de2d2b640 1 -- 172.21.15.82:0/3076035220 wait complete. 2024-04-25T10:07:02.049 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:02.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 Processor -- start 2024-04-25T10:07:02.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 -- start start 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe9cc15da80 0x7fe9cc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d0620 0x7fe9cc0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc06c440 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc06be60 con 0x7fe9cc15da80 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d2c4d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc15de60 con 0x7fe9cc0d0620 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36214/0 (socket says 172.21.15.82:36214) 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 -- 172.21.15.82:0/3752518828 learned_addr learned my addr 172.21.15.82:0/3752518828 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d09e9640 1 --2- 172.21.15.82:0/3752518828 >> v2:172.21.15.196:3300/0 conn(0x7fe9cc15da80 0x7fe9cc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9d11ea640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d0620 0x7fe9cc0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 -- 172.21.15.82:0/3752518828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d0620 msgr2=0x7fe9cc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d0620 0x7fe9cc0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 -- 172.21.15.82:0/3752518828 >> v2:172.21.15.196:3300/0 conn(0x7fe9cc15da80 msgr2=0x7fe9cc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 --2- 172.21.15.82:0/3752518828 >> v2:172.21.15.196:3300/0 conn(0x7fe9cc15da80 0x7fe9cc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.170+0000 7fe9cbfff640 1 -- 172.21.15.82:0/3752518828 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc0d2450 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9cbfff640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006fd0 tx=0x7fe9b801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776334f646c93576 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9cb7fe640 1 -- 172.21.15.82:0/3752518828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80b0070 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9cb7fe640 1 -- 172.21.15.82:0/3752518828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8004230 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9cb7fe640 1 -- 172.21.15.82:0/3752518828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003620 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3752518828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc0d1cd0 msgr2=0x7fe9cc0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8006fd0 tx=0x7fe9b801c500 comp rx=0 tx=0).stop 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3752518828 shutdown_connections 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d0620 0x7fe9cc0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3752518828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc0d1cd0 0x7fe9cc0d00e0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3752518828 >> v2:172.21.15.196:3300/0 conn(0x7fe9cc15da80 0x7fe9cc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3752518828 >> 172.21.15.82:0/3752518828 conn(0x7fe9cc08b000 msgr2=0x7fe9cc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3752518828 shutdown_connections 2024-04-25T10:07:02.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3752518828 wait complete. 2024-04-25T10:07:02.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 Processor -- start 2024-04-25T10:07:02.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- start start 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc0d0620 0x7fe9cc15fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d1cd0 0x7fe9cc153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc06ca30 con 0x7fe9cc15da80 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc06c8b0 con 0x7fe9cc0d0620 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc06cbb0 con 0x7fe9cc0d1cd0 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36224/0 (socket says 172.21.15.82:36224) 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 -- 172.21.15.82:0/3157383767 learned_addr learned my addr 172.21.15.82:0/3157383767 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d09e9640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc0d0620 0x7fe9cc15fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9cbfff640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d1cd0 0x7fe9cc153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 -- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d1cd0 msgr2=0x7fe9cc153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d1cd0 0x7fe9cc153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 -- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc0d0620 msgr2=0x7fe9cc15fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc0d0620 0x7fe9cc15fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 -- 172.21.15.82:0/3157383767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc154010 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d11ea640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400cd20 tx=0x7fe9c400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4003ba0 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c4003d40 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4004240 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc15eab0 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.174+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc15f070 con 0x7fe9cc15da80 2024-04-25T10:07:02.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc04efa0 con 0x7fe9cc15da80 2024-04-25T10:07:02.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9c40043e0 con 0x7fe9cc15da80 2024-04-25T10:07:02.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9c97fa640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9d09e9640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9c4085c40 con 0x7fe9cc15da80 2024-04-25T10:07:02.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.178+0000 7fe9d09e9640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b930 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.182+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9c401d340 con 0x7fe9cc15da80 2024-04-25T10:07:02.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.326+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9cc0d3660 con 0x7fe9cc15da80 2024-04-25T10:07:02.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.326+0000 7fe9c97fa640 1 -- 172.21.15.82:0/3157383767 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9c4056ac0 con 0x7fe9cc15da80 2024-04-25T10:07:02.331 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 msgr2=0x7fe9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000b930 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 msgr2=0x7fe9cc15e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400cd20 tx=0x7fe9c400a660 comp rx=0 tx=0).stop 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 shutdown_connections 2024-04-25T10:07:02.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9cc0d1cd0 0x7fe9cc153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9cc15da80 0x7fe9cc15e060 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 --2- 172.21.15.82:0/3157383767 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9cc0d0620 0x7fe9cc15fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 >> 172.21.15.82:0/3157383767 conn(0x7fe9cc08b000 msgr2=0x7fe9cc0d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 shutdown_connections 2024-04-25T10:07:02.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.330+0000 7fe9d2c4d640 1 -- 172.21.15.82:0/3157383767 wait complete. 2024-04-25T10:07:02.348 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:02.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.470+0000 7f65d83ae640 1 Processor -- start 2024-04-25T10:07:02.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.470+0000 7f65d83ae640 1 -- start start 2024-04-25T10:07:02.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d83ae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f65d00d07f0 0x7f65d00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d83ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65d006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d83ae640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65d006be60 con 0x7f65d015da80 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d83ae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d006c440 con 0x7f65d00d07f0 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d83ae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d015de60 con 0x7f65d00d1cd0 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f65d015da80 0x7f65d00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59466/0 (socket says 172.21.15.82:59466) 2024-04-25T10:07:02.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 -- 172.21.15.82:0/737080760 learned_addr learned my addr 172.21.15.82:0/737080760 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d694b640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65d006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5949640 1 --2- 172.21.15.82:0/737080760 >> v2:172.21.15.196:3300/0 conn(0x7f65d00d07f0 0x7f65d00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201344007 0 0) 0x7f65d006be60 con 0x7f65d015da80 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65bc003530 con 0x7f65d015da80 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 465190929 0 0) 0x7f65bc003530 con 0x7f65d015da80 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 msgr2=0x7f65d006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65d006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 >> v2:172.21.15.196:3300/0 conn(0x7f65d00d07f0 msgr2=0x7f65d00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 --2- 172.21.15.82:0/737080760 >> v2:172.21.15.196:3300/0 conn(0x7f65d00d07f0 0x7f65d00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00d2450 con 0x7f65d015da80 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f65cc0016a0 con 0x7f65d015da80 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 >> v1:172.21.15.82:6789/0 conn(0x7f65d015da80 legacy=0x7f65d00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65bc003a30 0x7f65bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc004d50 0x7f65bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006c440 con 0x7f65bc003a30 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d015de60 con 0x7f65bc004d50 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65bc00b9d0 con 0x7f65d00d1cd0 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65bc003a30 0x7f65bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d5949640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d694b640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc004d50 0x7f65bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 -- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 msgr2=0x7f65bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00d1cd0 0x7f65bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 -- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc004d50 msgr2=0x7f65bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc004d50 0x7f65bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d614a640 1 -- 172.21.15.82:0/737080760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65bc003530 con 0x7f65bc003a30 2024-04-25T10:07:02.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.474+0000 7f65d694b640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc004d50 0x7f65bc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d614a640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65bc003a30 0x7f65bc003e50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c210 tx=0x7f65cc01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f66e294bc700d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc007d70 con 0x7f65bc003a30 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65cc0a9020 con 0x7f65bc003a30 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5148640 1 -- 172.21.15.82:0/737080760 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65cc005b20 con 0x7f65bc003a30 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/737080760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65bc003a30 msgr2=0x7f65bc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 --2- 172.21.15.82:0/737080760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65bc003a30 0x7f65bc003e50 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f65cc00c210 tx=0x7f65cc01cc70 comp rx=0 tx=0).stop 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d694b640 1 -- 172.21.15.82:0/737080760 reap_dead start 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/737080760 shutdown_connections 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/737080760 >> 172.21.15.82:0/737080760 conn(0x7f65d008b000 msgr2=0x7f65d00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/737080760 shutdown_connections 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/737080760 wait complete. 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 Processor -- start 2024-04-25T10:07:02.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- start start 2024-04-25T10:07:02.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc003a30 0x7f65d0154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00c8bf0 0x7f65d00c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006d240 con 0x7f65d0156890 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006d0c0 con 0x7f65bc003a30 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d006d3c0 con 0x7f65d00c8bf0 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36244/0 (socket says 172.21.15.82:36244) 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 -- 172.21.15.82:0/688678195 learned_addr learned my addr 172.21.15.82:0/688678195 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d694b640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00c8bf0 0x7f65d00c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d614a640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc003a30 0x7f65d0154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 -- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00c8bf0 msgr2=0x7f65d00c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00c8bf0 0x7f65d00c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 -- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc003a30 msgr2=0x7f65d0154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc003a30 0x7f65d0154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 -- 172.21.15.82:0/688678195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00b06d0 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d5949640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f65c0011580 tx=0x7f65c0011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0019070 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00b09b0 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c0005db0 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.478+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0006250 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00b0f70 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d004efa0 con 0x7f65d0156890 2024-04-25T10:07:02.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65c001c070 con 0x7f65d0156890 2024-04-25T10:07:02.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65c6ffd640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65c00855b0 con 0x7f65d0156890 2024-04-25T10:07:02.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65d614a640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.482+0000 7f65d614a640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0aa000 tx=0x7f65cc001a70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.486+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65c0056440 con 0x7f65d0156890 2024-04-25T10:07:02.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65d0160b60 con 0x7f65d0156890 2024-04-25T10:07:02.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65c6ffd640 1 -- 172.21.15.82:0/688678195 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65c0055d40 con 0x7f65d0156890 2024-04-25T10:07:02.649 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 msgr2=0x7f65b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0aa000 tx=0x7f65cc001a70 comp rx=0 tx=0).stop 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 msgr2=0x7f65d0156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.646+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f65c0011580 tx=0x7f65c0011a40 comp rx=0 tx=0).stop 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 shutdown_connections 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65d00c8bf0 0x7f65d00c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65d0156890 0x7f65d0156d10 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 --2- 172.21.15.82:0/688678195 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65bc003a30 0x7f65d0154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 >> 172.21.15.82:0/688678195 conn(0x7f65d008b000 msgr2=0x7f65d0073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 shutdown_connections 2024-04-25T10:07:02.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.650+0000 7f65d83ae640 1 -- 172.21.15.82:0/688678195 wait complete. 2024-04-25T10:07:02.668 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:02.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.774+0000 7fd43df6f640 1 Processor -- start 2024-04-25T10:07:02.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.774+0000 7fd43df6f640 1 -- start start 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd43df6f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd4380ce510 0x7fd4380ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd43df6f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd438144ca0 0x7fd43806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd43df6f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4380cee30 con 0x7fd43806ec60 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd43df6f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4380cc2b0 con 0x7fd438144ca0 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd43df6f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43806e290 con 0x7fd4380ce510 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd438144ca0 0x7fd43806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd437fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd43806ec60 0x7fd438142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59472/0 (socket says 172.21.15.82:59472) 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd437fff640 1 -- 172.21.15.82:0/1067885335 learned_addr learned my addr 172.21.15.82:0/1067885335 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd4377fe640 1 --2- 172.21.15.82:0/1067885335 >> v2:172.21.15.196:3300/0 conn(0x7fd4380ce510 0x7fd4380ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 -- 172.21.15.82:0/1067885335 >> v2:172.21.15.196:3300/0 conn(0x7fd4380ce510 msgr2=0x7fd4380ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:02.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 --2- 172.21.15.82:0/1067885335 >> v2:172.21.15.196:3300/0 conn(0x7fd4380ce510 0x7fd4380ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 -- 172.21.15.82:0/1067885335 >> v1:172.21.15.82:6789/0 conn(0x7fd43806ec60 legacy=0x7fd438142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 -- 172.21.15.82:0/1067885335 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380ccc70 con 0x7fd438144ca0 2024-04-25T10:07:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd42effd640 1 --2- 172.21.15.82:0/1067885335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd438144ca0 0x7fd43806e720 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd438144c50 tx=0x7fd42001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef02df270f093d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.778+0000 7fd436ffd640 1 -- 172.21.15.82:0/1067885335 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4200b0070 con 0x7fd438144ca0 2024-04-25T10:07:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd436ffd640 1 -- 172.21.15.82:0/1067885335 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4200ab600 con 0x7fd438144ca0 2024-04-25T10:07:02.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd436ffd640 1 -- 172.21.15.82:0/1067885335 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4200ab9d0 con 0x7fd438144ca0 2024-04-25T10:07:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/1067885335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd438144ca0 msgr2=0x7fd43806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- 172.21.15.82:0/1067885335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd438144ca0 0x7fd43806e720 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd438144c50 tx=0x7fd42001c3a0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/1067885335 shutdown_connections 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- 172.21.15.82:0/1067885335 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd438144ca0 0x7fd43806e720 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- 172.21.15.82:0/1067885335 >> v2:172.21.15.196:3300/0 conn(0x7fd4380ce510 0x7fd4380ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/1067885335 >> 172.21.15.82:0/1067885335 conn(0x7fd4380b11a0 msgr2=0x7fd4380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/1067885335 shutdown_connections 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/1067885335 wait complete. 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 Processor -- start 2024-04-25T10:07:02.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- start start 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 0x7fd4381f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd438144ca0 0x7fd43813ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380cc900 con 0x7fd43806ec60 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806df20 con 0x7fd4380ce510 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd438142d90 con 0x7fd438144ca0 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36266/0 (socket says 172.21.15.82:36266) 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 -- 172.21.15.82:0/2381162835 learned_addr learned my addr 172.21.15.82:0/2381162835 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd42effd640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 0x7fd4381f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd437fff640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd438144ca0 0x7fd43813ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 -- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd438144ca0 msgr2=0x7fd43813ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd438144ca0 0x7fd43813ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 -- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 msgr2=0x7fd4381f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 0x7fd4381f3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 -- 172.21.15.82:0/2381162835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4381f43e0 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd42effd640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 0x7fd4381f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd4377fe640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d830 tx=0x7fd42800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800b6c0 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd428019070 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800ba60 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4380b2b70 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.782+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4380b3130 con 0x7fd43806ec60 2024-04-25T10:07:02.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43804efa0 con 0x7fd43806ec60 2024-04-25T10:07:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd42801e460 con 0x7fd43806ec60 2024-04-25T10:07:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd434ff9640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 0x7fd418069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd428085810 con 0x7fd43806ec60 2024-04-25T10:07:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd42effd640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 0x7fd418069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:02.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.786+0000 7fd42effd640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 0x7fd418069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd4380b2550 tx=0x7fd420008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:02.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.790+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4381f43e0 con 0x7fd43806ec60 2024-04-25T10:07:02.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.950+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4380b1890 con 0x7fd43806ec60 2024-04-25T10:07:02.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.950+0000 7fd434ff9640 1 -- 172.21.15.82:0/2381162835 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v79) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd428056720 con 0x7fd43806ec60 2024-04-25T10:07:02.955 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:07:02.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 msgr2=0x7fd418069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 0x7fd418069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd4380b2550 tx=0x7fd420008040 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 msgr2=0x7fd4381ed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd42800d830 tx=0x7fd42800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 shutdown_connections 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd418066f60 0x7fd418069420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd438144ca0 0x7fd43813ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd4380ce510 0x7fd4381f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 --2- 172.21.15.82:0/2381162835 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd43806ec60 0x7fd4381ed490 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 >> 172.21.15.82:0/2381162835 conn(0x7fd4380b11a0 msgr2=0x7fd438071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 shutdown_connections 2024-04-25T10:07:02.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:02.954+0000 7fd43df6f640 1 -- 172.21.15.82:0/2381162835 wait complete. 2024-04-25T10:07:02.972 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:07:02.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:02.973 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:02.983 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45252]: fuse finished with error 0 and tester_r 0 2024-04-25T10:07:02.985 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:07.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:07.258 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:09.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:07:09.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.090 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:09.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 Processor -- start 2024-04-25T10:07:09.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 -- start start 2024-04-25T10:07:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4f0409a180 0x7f4f0415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f040b0e10 con 0x7f4f0406f4a0 2024-04-25T10:07:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f040d0180 con 0x7f4f040b0a30 2024-04-25T10:07:09.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0c361640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f040d0d90 con 0x7f4f0409a180 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4f0406f4a0 0x7f4f04099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60980/0 (socket says 172.21.15.82:60980) 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 -- 172.21.15.82:0/4041894554 learned_addr learned my addr 172.21.15.82:0/4041894554 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0a8fe640 1 --2- 172.21.15.82:0/4041894554 >> v2:172.21.15.196:3300/0 conn(0x7f4f0409a180 0x7f4f0415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050626 0 0) 0x7f4f040b0e10 con 0x7f4f0406f4a0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ef0003530 con 0x7f4f0406f4a0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228245134 0 0) 0x7f4ef0003530 con 0x7f4f0406f4a0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 >> v2:172.21.15.196:3300/0 conn(0x7f4f0409a180 msgr2=0x7f4f0415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 --2- 172.21.15.82:0/4041894554 >> v2:172.21.15.196:3300/0 conn(0x7f4f0409a180 0x7f4f0415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 >> v1:172.21.15.196:6789/0 conn(0x7f4f040b0a30 legacy=0x7f4f040cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f040d0460 con 0x7f4f0406f4a0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4f00001680 con 0x7f4f0406f4a0 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 >> v1:172.21.15.82:6789/0 conn(0x7f4f0406f4a0 legacy=0x7f4f04099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0003a30 0x7f4ef0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 0x7f4ef0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00051d0 0x7f4ef000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040d0d90 con 0x7f4ef0004810 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef000c100 con 0x7f4ef00051d0 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef000c300 con 0x7f4ef0003a30 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 0x7f4ef0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0a8fe640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00051d0 0x7f4ef000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0a0fd640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0003a30 0x7f4ef0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 -- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0003a30 msgr2=0x7f4ef0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0003a30 0x7f4ef0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 -- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00051d0 msgr2=0x7f4ef000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4ef00051d0 0x7f4ef000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0a8fe640 1 -- 172.21.15.82:0/4041894554 reap_dead start 2024-04-25T10:07:09.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f098fc640 1 -- 172.21.15.82:0/4041894554 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0003530 con 0x7f4ef0004810 2024-04-25T10:07:09.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.266+0000 7f4f0a0fd640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0003a30 0x7f4ef0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f098fc640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 0x7f4ef0004c90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c210 tx=0x7f4f0001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de262caf3fea2d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00007d40 con 0x7f4ef0004810 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f00005b00 con 0x7f4ef0004810 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f090fb640 1 -- 172.21.15.82:0/4041894554 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00007070 con 0x7f4ef0004810 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 msgr2=0x7f4ef0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 0x7f4ef0004c90 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000c210 tx=0x7f4f0001f3b0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- 172.21.15.82:0/4041894554 shutdown_connections 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 --2- 172.21.15.82:0/4041894554 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4ef0004810 0x7f4ef0004c90 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- 172.21.15.82:0/4041894554 >> 172.21.15.82:0/4041894554 conn(0x7f4f040b30b0 msgr2=0x7f4f040b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- 172.21.15.82:0/4041894554 shutdown_connections 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- 172.21.15.82:0/4041894554 wait complete. 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 Processor -- start 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.270+0000 7f4f0c361640 1 -- start start 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0004810 0x7f4f040c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f041e73a0 0x7f4f041e7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04099a70 con 0x7f4f041e5730 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f040d0a20 con 0x7f4f041e73a0 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04099bf0 con 0x7f4ef0004810 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57230/0 (socket says 172.21.15.82:57230) 2024-04-25T10:07:09.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 -- 172.21.15.82:0/3048224830 learned_addr learned my addr 172.21.15.82:0/3048224830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0a8fe640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f041e73a0 0x7f4f041e7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0a0fd640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0004810 0x7f4f040c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 -- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0004810 msgr2=0x7f4f040c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0004810 0x7f4f040c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 -- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f041e73a0 msgr2=0x7f4f041e7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f041e73a0 0x7f4f041e7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 -- 172.21.15.82:0/3048224830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f041e5bc0 con 0x7f4f041e5730 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f098fc640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4f0001f1e0 tx=0x7f4ef0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00001680 con 0x7f4f041e5730 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f000b6070 con 0x7f4f041e5730 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f041e5ea0 con 0x7f4f041e5730 2024-04-25T10:07:09.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004a10 con 0x7f4f041e5730 2024-04-25T10:07:09.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f041e6460 con 0x7f4f041e5730 2024-04-25T10:07:09.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0404efa0 con 0x7f4f041e5730 2024-04-25T10:07:09.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f000b8020 con 0x7f4f041e5730 2024-04-25T10:07:09.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 0x7f4ee8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4f00122630 con 0x7f4f041e5730 2024-04-25T10:07:09.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0a0fd640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 0x7f4ee8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.274+0000 7f4f0a0fd640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 0x7f4ee8069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c10 tx=0x7f4ef4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.278+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f000190c0 con 0x7f4f041e5730 2024-04-25T10:07:09.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f0406fa80 con 0x7f4f041e5730 2024-04-25T10:07:09.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4efaffd640 1 -- 172.21.15.82:0/3048224830 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4f000f3380 con 0x7f4f041e5730 2024-04-25T10:07:09.476 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:09.477 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:06:49.173277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6966},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6966":{"gid":6966,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/910515233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":910515233},{"type":"v1","addr":"172.21.15.82:6837","nonce":910515233}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:09.477 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 41 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 msgr2=0x7f4ee8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 0x7f4ee8069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4000c10 tx=0x7f4ef4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 msgr2=0x7f4f040b56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4f0001f1e0 tx=0x7f4ef0005840 comp rx=0 tx=0).stop 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 shutdown_connections 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4ee8066f60 0x7f4ee8069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f041e73a0 0x7f4f041e7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f041e5730 0x7f4f040b56a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 --2- 172.21.15.82:0/3048224830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4ef0004810 0x7f4f040c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 >> 172.21.15.82:0/3048224830 conn(0x7f4f040b30b0 msgr2=0x7f4f040af940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 shutdown_connections 2024-04-25T10:07:09.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.474+0000 7f4f0c361640 1 -- 172.21.15.82:0/3048224830 wait complete. 2024-04-25T10:07:09.494 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:07:09.641 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:07:09.642 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:09.642 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:09.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 Processor -- start 2024-04-25T10:07:09.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 -- start start 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdebc15a280 0x7fdebc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdebc0718c0 0x7fdebc071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc0b2990 con 0x7fdebc0718c0 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc15ae30 con 0x7fdebc15a280 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec24da640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15b110 con 0x7fdebc159ea0 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdebc0718c0 0x7fdebc071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdebc0718c0 0x7fdebc071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57232/0 (socket says 172.21.15.82:57232) 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 -- 172.21.15.82:0/3873189592 learned_addr learned my addr 172.21.15.82:0/3873189592 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdeb3fff640 1 --2- 172.21.15.82:0/3873189592 >> v2:172.21.15.196:3300/0 conn(0x7fdebc15a280 0x7fdebc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdebbfff640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 -- 172.21.15.82:0/3873189592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 msgr2=0x7fdebc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 -- 172.21.15.82:0/3873189592 >> v2:172.21.15.196:3300/0 conn(0x7fdebc15a280 msgr2=0x7fdebc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 --2- 172.21.15.82:0/3873189592 >> v2:172.21.15.196:3300/0 conn(0x7fdebc15a280 0x7fdebc15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 -- 172.21.15.82:0/3873189592 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0b2620 con 0x7fdebc0718c0 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdec0a77640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc0718c0 0x7fdebc071d50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06cea0 tx=0x7fdeb401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f71616ebbd9089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.750+0000 7fdebb7fe640 1 -- 172.21.15.82:0/3873189592 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40a5700 con 0x7fdebc0718c0 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdebb7fe640 1 -- 172.21.15.82:0/3873189592 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb40a58a0 con 0x7fdebc0718c0 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdebb7fe640 1 -- 172.21.15.82:0/3873189592 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40a5c70 con 0x7fdebc0718c0 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- 172.21.15.82:0/3873189592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc0718c0 msgr2=0x7fdebc071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc0718c0 0x7fdebc071d50 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fdebc06cea0 tx=0x7fdeb401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- 172.21.15.82:0/3873189592 shutdown_connections 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc0718c0 0x7fdebc071d50 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- 172.21.15.82:0/3873189592 >> v2:172.21.15.196:3300/0 conn(0x7fdebc15a280 0x7fdebc15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- 172.21.15.82:0/3873189592 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- 172.21.15.82:0/3873189592 >> 172.21.15.82:0/3873189592 conn(0x7fdebc0b11a0 msgr2=0x7fdebc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- 172.21.15.82:0/3873189592 shutdown_connections 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- 172.21.15.82:0/3873189592 wait complete. 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 Processor -- start 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- start start 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc0718c0 0x7fdebc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06d420 con 0x7fdebc15a280 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06d2a0 con 0x7fdebc0718c0 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec24da640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06d5a0 con 0x7fdebc159ea0 2024-04-25T10:07:09.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57238/0 (socket says 172.21.15.82:57238) 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 -- 172.21.15.82:0/2687639056 learned_addr learned my addr 172.21.15.82:0/2687639056 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdeb3fff640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdebbfff640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc0718c0 0x7fdebc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 -- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 msgr2=0x7fdebc14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 -- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc0718c0 msgr2=0x7fdebc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc0718c0 0x7fdebc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 -- 172.21.15.82:0/2687639056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc149090 con 0x7fdebc15a280 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdec0a77640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fdeb401b0a0 tx=0x7fdeb4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.754+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40b8670 con 0x7fdebc15a280 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb40be070 con 0x7fdebc15a280 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40b8a50 con 0x7fdebc15a280 2024-04-25T10:07:09.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc147e30 con 0x7fdebc15a280 2024-04-25T10:07:09.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc1485b0 con 0x7fdebc15a280 2024-04-25T10:07:09.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc04efa0 con 0x7fdebc15a280 2024-04-25T10:07:09.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdeb40b8bf0 con 0x7fdebc15a280 2024-04-25T10:07:09.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdeb97fa640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 0x7fde9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdeb4120280 con 0x7fdebc15a280 2024-04-25T10:07:09.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdebbfff640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 0x7fde9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.758+0000 7fdebbfff640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 0x7fde9c069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ba30 tx=0x7fdeac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.762+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdeb40165e0 con 0x7fdebc15a280 2024-04-25T10:07:09.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.942+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdebc06e6b0 con 0x7fdebc15a280 2024-04-25T10:07:09.947 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:07:09.946+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:09.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.946+0000 7fdeb97fa640 1 -- 172.21.15.82:0/2687639056 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdeb40f1100 con 0x7fdebc15a280 2024-04-25T10:07:09.950 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:09.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 msgr2=0x7fde9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 0x7fde9c069420 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdeac00ba30 tx=0x7fdeac008040 comp rx=0 tx=0).stop 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 msgr2=0x7fdebc146ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fdeb401b0a0 tx=0x7fdeb4019790 comp rx=0 tx=0).stop 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 shutdown_connections 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fde9c066f60 0x7fde9c069420 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdebc159ea0 0x7fdebc14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdebc15a280 0x7fdebc146ed0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 --2- 172.21.15.82:0/2687639056 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdebc0718c0 0x7fdebc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 >> 172.21.15.82:0/2687639056 conn(0x7fdebc0b11a0 msgr2=0x7fdebc15ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 shutdown_connections 2024-04-25T10:07:09.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:09.950+0000 7fdec24da640 1 -- 172.21.15.82:0/2687639056 wait complete. 2024-04-25T10:07:09.958 INFO:tasks.ceph.mds.a.smithi082.stderr: -3741> 2024-04-25T10:06:47.734+0000 7fc6dd70c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:09.972 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:07:10.004 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:07:10.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 Processor -- start 2024-04-25T10:07:10.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- start start 2024-04-25T10:07:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 0x5564d457fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d457de50 0x5564d4578b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d44d91b0 con 0x5564d44d7e40 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d44509f0 con 0x5564d44f20f0 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564d44d9330 con 0x5564d457de50 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57260/0 (socket says 172.21.15.82:57260) 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 -- 172.21.15.82:0/169712910 learned_addr learned my addr 172.21.15.82:0/169712910 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 -- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d457de50 msgr2=0x5564d4578b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d457de50 0x5564d4578b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 -- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 msgr2=0x5564d457fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5ce4b3640 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 0x5564d457fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 0x5564d457fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 -- 172.21.15.82:0/169712910 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564d457ef60 con 0x5564d44d7e40 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5ce4b3640 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 0x5564d457fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000a400 tx=0x7fd5c000a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e215da9089c8c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cd4b1640 1 -- 172.21.15.82:0/169712910 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0010070 con 0x5564d44d7e40 2024-04-25T10:07:10.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cd4b1640 1 -- 172.21.15.82:0/169712910 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c000c900 con 0x5564d44d7e40 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5cd4b1640 1 -- 172.21.15.82:0/169712910 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000ccf0 con 0x5564d44d7e40 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- 172.21.15.82:0/169712910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 msgr2=0x5564d457e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000a400 tx=0x7fd5c000a8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- 172.21.15.82:0/169712910 shutdown_connections 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d457de50 0x5564d4578b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d44f20f0 0x5564d457fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 --2- 172.21.15.82:0/169712910 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d457e3f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- 172.21.15.82:0/169712910 >> 172.21.15.82:0/169712910 conn(0x5564d44d4ab0 msgr2=0x5564d44ad320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- 172.21.15.82:0/169712910 shutdown_connections 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.006+0000 7fd5d1196080 1 -- 172.21.15.82:0/169712910 wait complete. 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 Processor -- start 2024-04-25T10:07:10.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- start start 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d4619f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d44f20f0 0x5564d4577530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d457de50 0x5564d45875c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d457ea20 con 0x5564d44d7e40 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564d457f770 con 0x5564d457de50 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564d457ecc0 con 0x5564d44f20f0 2024-04-25T10:07:10.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d4619f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d4619f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57272/0 (socket says 172.21.15.82:57272) 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 -- 172.21.15.82:0/2616359847 learned_addr learned my addr 172.21.15.82:0/2616359847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 -- 172.21.15.82:0/2616359847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d44f20f0 msgr2=0x5564d4577530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5564d44f20f0 0x5564d4577530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 -- 172.21.15.82:0/2616359847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d457de50 msgr2=0x5564d45875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5564d457de50 0x5564d45875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 -- 172.21.15.82:0/2616359847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564d4577eb0 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cecb4640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d4619f00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000adc0 tx=0x7fd5c0002f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 -- 172.21.15.82:0/2616359847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0025070 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 -- 172.21.15.82:0/2616359847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c0015d40 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5564d4578150 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 -- 172.21.15.82:0/2616359847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0010050 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564d461a870 con 0x5564d44d7e40 2024-04-25T10:07:10.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 -- 172.21.15.82:0/2616359847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5c0028070 con 0x5564d44d7e40 2024-04-25T10:07:10.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd5bc065970 0x7fd5bc067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5ce4b3640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd5bc065970 0x7fd5bc067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5b7fff640 1 -- 172.21.15.82:0/2616359847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd5c0085780 con 0x5564d44d7e40 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5564d461b220 0x5564d461b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) v9 -- 0x5564d4577a70 con 0x5564d461b220 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.010+0000 7fd5cf4b5640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5564d461b220 0x5564d461b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.014+0000 7fd5ce4b3640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd5bc065970 0x7fd5bc067e30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009540 tx=0x7fd5c4005600 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.014+0000 7fd5cf4b5640 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5564d461b220 0x5564d461b620 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5cf4b5640 1 -- 172.21.15.82:0/2616359847 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd5c80ab410 con 0x5564d461b220 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5564d461b220 msgr2=0x5564d461b620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5564d461b220 0x5564d461b620 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd5bc065970 msgr2=0x7fd5bc067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd5bc065970 0x7fd5bc067e30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd5c4009540 tx=0x7fd5c4005600 comp rx=0 tx=0).stop 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 msgr2=0x5564d4619f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 --2- 172.21.15.82:0/2616359847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5564d44d7e40 0x5564d4619f00 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000adc0 tx=0x7fd5c0002f40 comp rx=0 tx=0).stop 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5cf4b5640 1 -- 172.21.15.82:0/2616359847 reap_dead start 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 shutdown_connections 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 >> 172.21.15.82:0/2616359847 conn(0x5564d44d4ab0 msgr2=0x5564d4580f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 shutdown_connections 2024-04-25T10:07:10.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.018+0000 7fd5d1196080 1 -- 172.21.15.82:0/2616359847 wait complete. 2024-04-25T10:07:10.027 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:07:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.178+0000 7f5f4fc53640 1 Processor -- start 2024-04-25T10:07:10.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.178+0000 7f5f4fc53640 1 -- start start 2024-04-25T10:07:10.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4fc53640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f48159bd0 0x7f5f480c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4fc53640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f48159fb0 0x7f5f480b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4fc53640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f480c9ef0 con 0x7f5f480b26f0 2024-04-25T10:07:10.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4fc53640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f48072780 con 0x7f5f48159fb0 2024-04-25T10:07:10.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4fc53640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f48071b70 con 0x7f5f48159bd0 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f48159fb0 0x7f5f480b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f48159fb0 0x7f5f480b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57280/0 (socket says 172.21.15.82:57280) 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 -- 172.21.15.82:0/156790686 learned_addr learned my addr 172.21.15.82:0/156790686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d9ef640 1 --2- 172.21.15.82:0/156790686 >> v2:172.21.15.196:3300/0 conn(0x7f5f48159bd0 0x7f5f480c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 -- 172.21.15.82:0/156790686 >> v2:172.21.15.196:3300/0 conn(0x7f5f48159bd0 msgr2=0x7f5f480c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 --2- 172.21.15.82:0/156790686 >> v2:172.21.15.196:3300/0 conn(0x7f5f48159bd0 0x7f5f480c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 -- 172.21.15.82:0/156790686 >> v1:172.21.15.196:6789/0 conn(0x7f5f480b26f0 legacy=0x7f5f480cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 -- 172.21.15.82:0/156790686 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48071e50 con 0x7f5f48159fb0 2024-04-25T10:07:10.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4d1ee640 1 --2- 172.21.15.82:0/156790686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f480b21b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5f480cf2c0 tx=0x7f5f3801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e8d96694910ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4c9ed640 1 -- 172.21.15.82:0/156790686 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380ab650 con 0x7f5f48159fb0 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4c9ed640 1 -- 172.21.15.82:0/156790686 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380b0070 con 0x7f5f48159fb0 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.182+0000 7f5f4c9ed640 1 -- 172.21.15.82:0/156790686 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380aba10 con 0x7f5f48159fb0 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/156790686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 msgr2=0x7f5f480b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/156790686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f480b21b0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f5f480cf2c0 tx=0x7f5f3801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/156790686 shutdown_connections 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/156790686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f480b21b0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/156790686 >> v2:172.21.15.196:3300/0 conn(0x7f5f48159bd0 0x7f5f480c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/156790686 >> 172.21.15.82:0/156790686 conn(0x7f5f4806ae60 msgr2=0x7f5f4806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/156790686 shutdown_connections 2024-04-25T10:07:10.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/156790686 wait complete. 2024-04-25T10:07:10.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 Processor -- start 2024-04-25T10:07:10.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- start start 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f480b26f0 0x7f5f48098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f48159bd0 0x7f5f481f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cba90 con 0x7f5f48159fb0 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cefd0 con 0x7f5f480b26f0 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f480cbc10 con 0x7f5f48159bd0 2024-04-25T10:07:10.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57292/0 (socket says 172.21.15.82:57292) 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 -- 172.21.15.82:0/550093116 learned_addr learned my addr 172.21.15.82:0/550093116 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4d9ef640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f480b26f0 0x7f5f48098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 -- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f48159bd0 msgr2=0x7f5f481f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f48159bd0 0x7f5f481f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 -- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f480b26f0 msgr2=0x7f5f48098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f480b26f0 0x7f5f48098880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 -- 172.21.15.82:0/550093116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48157f50 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4e1f0640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b970 tx=0x7f5f4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40019070 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f4000cb40 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f40014440 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f48158230 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.186+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f481587f0 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4804efa0 con 0x7f5f48159fb0 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f4001c070 con 0x7f5f48159fb0 2024-04-25T10:07:10.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f3e7fc640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f4d9ef640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5f40084e80 con 0x7f5f48159fb0 2024-04-25T10:07:10.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.190+0000 7f5f4d9ef640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 0x7f5f2c069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5f4806edf0 tx=0x7f5f44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.194+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5f40055ca0 con 0x7f5f48159fb0 2024-04-25T10:07:10.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.374+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5f48151290 con 0x7f5f48159fb0 2024-04-25T10:07:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f3e7fc640 1 -- 172.21.15.82:0/550093116 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v43) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5f400555a0 con 0x7f5f48159fb0 2024-04-25T10:07:10.953 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 msgr2=0x7f5f2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 0x7f5f2c069420 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5f4806edf0 tx=0x7f5f44008040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 msgr2=0x7f5f4806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b970 tx=0x7f5f4000be30 comp rx=0 tx=0).stop 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 shutdown_connections 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f2c066f60 0x7f5f2c069420 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f48159bd0 0x7f5f481f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f48159fb0 0x7f5f4806e110 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.950+0000 7f5f4fc53640 1 --2- 172.21.15.82:0/550093116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f480b26f0 0x7f5f48098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.954+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 >> 172.21.15.82:0/550093116 conn(0x7f5f4806ae60 msgr2=0x7f5f48156780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.954+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 shutdown_connections 2024-04-25T10:07:10.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:10.954+0000 7f5f4fc53640 1 -- 172.21.15.82:0/550093116 wait complete. 2024-04-25T10:07:10.980 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:11.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 Processor -- start 2024-04-25T10:07:11.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 -- start start 2024-04-25T10:07:11.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5304066e30 0x7f530406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 0x7f53040ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530406ec30 con 0x7f530406bca0 2024-04-25T10:07:11.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040669a0 con 0x7f5304066e30 2024-04-25T10:07:11.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530c6ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530406c5c0 con 0x7f53040af370 2024-04-25T10:07:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530ac87640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 0x7f53040ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f5309c85640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5304066e30 0x7f530406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530ac87640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 0x7f53040ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:58192/0 (socket says 172.21.15.82:58192) 2024-04-25T10:07:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f530ac87640 1 -- 172.21.15.82:0/563150671 learned_addr learned my addr 172.21.15.82:0/563150671 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:11.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f5309c85640 1 -- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 msgr2=0x7f53040ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f5309c85640 1 --2- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 0x7f53040ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f5309c85640 1 -- 172.21.15.82:0/563150671 >> v1:172.21.15.196:6789/0 conn(0x7f530406bca0 legacy=0x7f53040cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.102+0000 7f5309c85640 1 -- 172.21.15.82:0/563150671 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530406f230 con 0x7f5304066e30 2024-04-25T10:07:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f5309c85640 1 --2- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304066e30 0x7f530406c080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097c0 tx=0x7f52f401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8812ef6585fc7af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f5309484640 1 -- 172.21.15.82:0/563150671 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b0070 con 0x7f5304066e30 2024-04-25T10:07:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f5309484640 1 -- 172.21.15.82:0/563150671 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40ab600 con 0x7f5304066e30 2024-04-25T10:07:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f5309484640 1 -- 172.21.15.82:0/563150671 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab9d0 con 0x7f5304066e30 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 -- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304066e30 msgr2=0x7f530406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 --2- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304066e30 0x7f530406c080 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f52f40097c0 tx=0x7f52f401c350 comp rx=0 tx=0).stop 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 -- 172.21.15.82:0/563150671 shutdown_connections 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 --2- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f53040af370 0x7f53040ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 --2- 172.21.15.82:0/563150671 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5304066e30 0x7f530406c080 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 -- 172.21.15.82:0/563150671 >> 172.21.15.82:0/563150671 conn(0x7f5304071810 msgr2=0x7f5304071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 -- 172.21.15.82:0/563150671 shutdown_connections 2024-04-25T10:07:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 -- 172.21.15.82:0/563150671 wait complete. 2024-04-25T10:07:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.106+0000 7f530c6ea640 1 Processor -- start 2024-04-25T10:07:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- start start 2024-04-25T10:07:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5304066e30 0x7f5304146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53040af370 0x7f530414c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53041687b0 con 0x7f530406bca0 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304168630 con 0x7f53040af370 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304168930 con 0x7f5304066e30 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57294/0 (socket says 172.21.15.82:57294) 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 -- 172.21.15.82:0/869422189 learned_addr learned my addr 172.21.15.82:0/869422189 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530a486640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5304066e30 0x7f5304146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530ac87640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53040af370 0x7f530414c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 -- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5304066e30 msgr2=0x7f5304146e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5304066e30 0x7f5304146e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 -- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53040af370 msgr2=0x7f530414c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53040af370 0x7f530414c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 -- 172.21.15.82:0/869422189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53041488c0 con 0x7f530406bca0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f5309c85640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f52f4019f80 tx=0x7f52f401ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b0050 con 0x7f530406bca0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40bc070 con 0x7f530406bca0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5304147540 con 0x7f530406bca0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b5830 con 0x7f530406bca0 2024-04-25T10:07:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5304147b00 con 0x7f530406bca0 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530404efa0 con 0x7f530406bca0 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.110+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52f40b59d0 con 0x7f530406bca0 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.114+0000 7f52faffd640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.114+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f52f411f7f0 con 0x7f530406bca0 2024-04-25T10:07:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.114+0000 7f530a486640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.114+0000 7f530a486640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 0x7f52e8069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5304156b20 tx=0x7f5300009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.118+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53041488c0 con 0x7f530406bca0 2024-04-25T10:07:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5304166400 con 0x7f530406bca0 2024-04-25T10:07:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f52faffd640 1 -- 172.21.15.82:0/869422189 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f52f4016090 con 0x7f530406bca0 2024-04-25T10:07:11.313 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:11.313 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:07:10.957353+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:11.314 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 44 2024-04-25T10:07:11.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 msgr2=0x7f52e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 0x7f52e8069420 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5304156b20 tx=0x7f5300009040 comp rx=0 tx=0).stop 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 msgr2=0x7f530414bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f52f4019f80 tx=0x7f52f401ab90 comp rx=0 tx=0).stop 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 shutdown_connections 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f52e8066f60 0x7f52e8069420 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f53040af370 0x7f530414c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f530406bca0 0x7f530414bae0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 --2- 172.21.15.82:0/869422189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5304066e30 0x7f5304146e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 >> 172.21.15.82:0/869422189 conn(0x7f5304071810 msgr2=0x7f5304151fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 shutdown_connections 2024-04-25T10:07:11.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:11.310+0000 7f530c6ea640 1 -- 172.21.15.82:0/869422189 wait complete. 2024-04-25T10:07:11.735 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:46.428439+0000', 'modified': '2024-04-25T10:07:10.957353+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7260': {'gid': 7260, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/3956208581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 3956208581}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3956208581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:11.735 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7260:up:replay 2024-04-25T10:07:12.737 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.865+0000 7f29518e6640 1 Processor -- start 2024-04-25T10:07:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.865+0000 7f29518e6640 1 -- start start 2024-04-25T10:07:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f29518e6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f294c15da10 0x7f294c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f29518e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f294c071e90 0x7f294c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f29518e6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f294c0adbd0 con 0x7f294c0b24a0 2024-04-25T10:07:12.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f29518e6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c0ae860 con 0x7f294c15da10 2024-04-25T10:07:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f29518e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f294c15e210 con 0x7f294c071e90 2024-04-25T10:07:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f294c15da10 0x7f294c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f294c15da10 0x7f294c071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57302/0 (socket says 172.21.15.82:57302) 2024-04-25T10:07:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 -- 172.21.15.82:0/3793427264 learned_addr learned my addr 172.21.15.82:0/3793427264 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294b7fe640 1 --2- 172.21.15.82:0/3793427264 >> v2:172.21.15.196:3300/0 conn(0x7f294c071e90 0x7f294c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 -- 172.21.15.82:0/3793427264 >> v2:172.21.15.196:3300/0 conn(0x7f294c071e90 msgr2=0x7f294c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 --2- 172.21.15.82:0/3793427264 >> v2:172.21.15.196:3300/0 conn(0x7f294c071e90 0x7f294c071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 -- 172.21.15.82:0/3793427264 >> v1:172.21.15.196:6789/0 conn(0x7f294c0b24a0 legacy=0x7f294c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.869+0000 7f294affd640 1 -- 172.21.15.82:0/3793427264 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c15e4f0 con 0x7f294c15da10 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- 172.21.15.82:0/3793427264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c15da10 0x7f294c071030 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b43e0 tx=0x7f293c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f6a6f830c16d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2949ffb640 1 -- 172.21.15.82:0/3793427264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b0070 con 0x7f294c15da10 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2949ffb640 1 -- 172.21.15.82:0/3793427264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c0ab600 con 0x7f294c15da10 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2949ffb640 1 -- 172.21.15.82:0/3793427264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0ab9d0 con 0x7f294c15da10 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3793427264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c15da10 msgr2=0x7f294c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- 172.21.15.82:0/3793427264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c15da10 0x7f294c071030 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f294c0b43e0 tx=0x7f293c01c400 comp rx=0 tx=0).stop 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3793427264 shutdown_connections 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- 172.21.15.82:0/3793427264 >> v2:172.21.15.196:3300/0 conn(0x7f294c071e90 0x7f294c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- 172.21.15.82:0/3793427264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c15da10 0x7f294c071030 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3793427264 >> 172.21.15.82:0/3793427264 conn(0x7f294c0b11a0 msgr2=0x7f294c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3793427264 shutdown_connections 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3793427264 wait complete. 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 Processor -- start 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- start start 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f294c0b24a0 0x7f294c1538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f294c14e110 0x7f294c1f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c16aa10 con 0x7f294c071e90 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294c16a890 con 0x7f294c14e110 2024-04-25T10:07:12.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294c16ab90 con 0x7f294c0b24a0 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57308/0 (socket says 172.21.15.82:57308) 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 -- 172.21.15.82:0/3803896638 learned_addr learned my addr 172.21.15.82:0/3803896638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294b7fe640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f294c14e110 0x7f294c1f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294a7fc640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f294c0b24a0 0x7f294c1538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 -- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f294c0b24a0 msgr2=0x7f294c1538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f294c0b24a0 0x7f294c1538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 -- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f294c14e110 msgr2=0x7f294c1f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f294c14e110 0x7f294c1f9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 -- 172.21.15.82:0/3803896638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c15a970 con 0x7f294c071e90 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f294affd640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f293c0ae040 tx=0x7f293c0191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b0040 con 0x7f294c071e90 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c0b4430 con 0x7f294c071e90 2024-04-25T10:07:12.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c14e590 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.873+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c0b4840 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c1540c0 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f294c04efa0 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f293c0b49e0 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f2933fff640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 0x7f2938069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f293c11ecd0 con 0x7f294c071e90 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f294a7fc640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 0x7f2938069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.877+0000 7f294a7fc640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 0x7f2938069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2940007fd0 tx=0x7f2940009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:12.881+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f293c016110 con 0x7f294c071e90 2024-04-25T10:07:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.069+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f294c0b1dc0 con 0x7f294c071e90 2024-04-25T10:07:13.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.069+0000 7f2933fff640 1 -- 172.21.15.82:0/3803896638 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f293c0efb00 con 0x7f294c071e90 2024-04-25T10:07:13.075 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:13.075 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:07:11.969610+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:13.075 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 46 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 msgr2=0x7f2938069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 0x7f2938069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2940007fd0 tx=0x7f2940009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 msgr2=0x7f294c14fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f293c0ae040 tx=0x7f293c0191a0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 shutdown_connections 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2938066f60 0x7f2938069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f294c0b24a0 0x7f294c1538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f294c14e110 0x7f294c1f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 --2- 172.21.15.82:0/3803896638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f294c071e90 0x7f294c14fd10 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 >> 172.21.15.82:0/3803896638 conn(0x7f294c0b11a0 msgr2=0x7f294c156910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 shutdown_connections 2024-04-25T10:07:13.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:13.073+0000 7f29518e6640 1 -- 172.21.15.82:0/3803896638 wait complete. 2024-04-25T10:07:13.495 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:46.428439+0000', 'modified': '2024-04-25T10:07:11.969610+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7260': {'gid': 7260, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/3956208581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 3956208581}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3956208581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:13.496 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7260:up:rejoin 2024-04-25T10:07:14.496 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:14.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.621+0000 7f968af5e640 1 Processor -- start 2024-04-25T10:07:14.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f968af5e640 1 -- start start 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f968af5e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f968415a280 0x7f968415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f968af5e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968415ae30 con 0x7f96840718c0 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f968af5e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968415b110 con 0x7f9684159ea0 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f968af5e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96840b2990 con 0x7f968415a280 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f968415a280 0x7f968415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f968415a280 0x7f968415a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57318/0 (socket says 172.21.15.82:57318) 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 -- 172.21.15.82:0/1209231818 learned_addr learned my addr 172.21.15.82:0/1209231818 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 -- 172.21.15.82:0/1209231818 >> v1:172.21.15.196:6789/0 conn(0x7f9684159ea0 legacy=0x7f96840b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 -- 172.21.15.82:0/1209231818 >> v1:172.21.15.82:6789/0 conn(0x7f96840718c0 legacy=0x7f968406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 -- 172.21.15.82:0/1209231818 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840b2620 con 0x7f968415a280 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.625+0000 7f9683fff640 1 --2- 172.21.15.82:0/1209231818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f968415a700 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f967000aa10 tx=0x7f967001d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f819141e8c8f3cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96837fe640 1 -- 172.21.15.82:0/1209231818 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96700b3070 con 0x7f968415a280 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96837fe640 1 -- 172.21.15.82:0/1209231818 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96700a68e0 con 0x7f968415a280 2024-04-25T10:07:14.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96837fe640 1 -- 172.21.15.82:0/1209231818 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000b040 con 0x7f968415a280 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1209231818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 msgr2=0x7f968415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 --2- 172.21.15.82:0/1209231818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f968415a700 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f967000aa10 tx=0x7f967001d8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1209231818 shutdown_connections 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 --2- 172.21.15.82:0/1209231818 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f968415a700 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1209231818 >> 172.21.15.82:0/1209231818 conn(0x7f96840b11a0 msgr2=0x7f96840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1209231818 shutdown_connections 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1209231818 wait complete. 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 Processor -- start 2024-04-25T10:07:14.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- start start 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 0x7f968413a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9684159ea0 0x7f96841590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684167bd0 con 0x7f968415a280 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840b3530 con 0x7f96840718c0 2024-04-25T10:07:14.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9684167d50 con 0x7f9684159ea0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57322/0 (socket says 172.21.15.82:57322) 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 -- 172.21.15.82:0/1391414532 learned_addr learned my addr 172.21.15.82:0/1391414532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f9683fff640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9684159ea0 0x7f96841590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f9688cfa640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 0x7f968413a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 -- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9684159ea0 msgr2=0x7f96841590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9684159ea0 0x7f96841590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 -- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 msgr2=0x7f968413a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 0x7f968413a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 -- 172.21.15.82:0/1391414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968413ab30 con 0x7f968415a280 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f9688cfa640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 0x7f968413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96894fb640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c710 tx=0x7f967c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c016020 con 0x7f968415a280 2024-04-25T10:07:14.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c006d60 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c003bc0 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968413ae10 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.629+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f968413b3d0 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968404efa0 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f967c00a850 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f96817fa640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 0x7f9668069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f967c084ea0 con 0x7f968415a280 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f9688cfa640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 0x7f9668069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.633+0000 7f9688cfa640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 0x7f9668069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9678003e40 tx=0x7f9678009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.637+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f967c020020 con 0x7f968415a280 2024-04-25T10:07:14.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96840cb810 con 0x7f968415a280 2024-04-25T10:07:14.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f96817fa640 1 -- 172.21.15.82:0/1391414532 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f967c055d60 con 0x7f968415a280 2024-04-25T10:07:14.837 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:14.837 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:07:12.979512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:14.837 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 47 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 msgr2=0x7f9668069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 0x7f9668069420 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f9678003e40 tx=0x7f9678009040 comp rx=0 tx=0).stop 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 msgr2=0x7f9684137af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f967c00c710 tx=0x7f967c00cbd0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 shutdown_connections 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9668066f60 0x7f9668069420 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9684159ea0 0x7f96841590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f968415a280 0x7f9684137af0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 --2- 172.21.15.82:0/1391414532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96840718c0 0x7f968413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 >> 172.21.15.82:0/1391414532 conn(0x7f96840b11a0 msgr2=0x7f9684073600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 shutdown_connections 2024-04-25T10:07:14.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:14.833+0000 7f968af5e640 1 -- 172.21.15.82:0/1391414532 wait complete. 2024-04-25T10:07:15.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:46.428439+0000', 'modified': '2024-04-25T10:07:12.979512+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 80, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7260': {'gid': 7260, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/3956208581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 3956208581}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3956208581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:15.258 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:15.258 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:07:15.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 Processor -- start 2024-04-25T10:07:15.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 -- start start 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdbe80ce510 0x7fdbe80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbe8144ca0 0x7fdbe806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 0x7fdbe8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe80cc2b0 con 0x7fdbe8144ca0 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe806e290 con 0x7fdbe80ce510 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbed629640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80cee30 con 0x7fdbe806ec60 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbe8144ca0 0x7fdbe806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdbe8144ca0 0x7fdbe806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52360/0 (socket says 172.21.15.82:52360) 2024-04-25T10:07:15.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 -- 172.21.15.82:0/2379325709 learned_addr learned my addr 172.21.15.82:0/2379325709 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/2379325709 >> v2:172.21.15.196:3300/0 conn(0x7fdbe80ce510 0x7fdbe80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 0x7fdbe8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 -- 172.21.15.82:0/2379325709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 msgr2=0x7fdbe8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 0x7fdbe8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 -- 172.21.15.82:0/2379325709 >> v2:172.21.15.196:3300/0 conn(0x7fdbe80ce510 msgr2=0x7fdbe80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2379325709 >> v2:172.21.15.196:3300/0 conn(0x7fdbe80ce510 0x7fdbe80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe67fc640 1 -- 172.21.15.82:0/2379325709 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80ccc70 con 0x7fdbe8144ca0 2024-04-25T10:07:15.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.389+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 0x7fdbe8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:15.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe8144ca0 0x7fdbe806e720 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8144c50 tx=0x7fdbdc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526c84a65b40e6c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbe5ffb640 1 -- 172.21.15.82:0/2379325709 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0b5070 con 0x7fdbe8144ca0 2024-04-25T10:07:15.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbe5ffb640 1 -- 172.21.15.82:0/2379325709 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc0a89d0 con 0x7fdbe8144ca0 2024-04-25T10:07:15.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbe5ffb640 1 -- 172.21.15.82:0/2379325709 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0a8da0 con 0x7fdbe8144ca0 2024-04-25T10:07:15.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- 172.21.15.82:0/2379325709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe8144ca0 msgr2=0x7fdbe806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe8144ca0 0x7fdbe806e720 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8144c50 tx=0x7fdbdc01ed10 comp rx=0 tx=0).stop 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- 172.21.15.82:0/2379325709 shutdown_connections 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe806ec60 0x7fdbe8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 --2- 172.21.15.82:0/2379325709 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe8144ca0 0x7fdbe806e720 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 --2- 172.21.15.82:0/2379325709 >> v2:172.21.15.196:3300/0 conn(0x7fdbe80ce510 0x7fdbe80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- 172.21.15.82:0/2379325709 >> 172.21.15.82:0/2379325709 conn(0x7fdbe80b11a0 msgr2=0x7fdbe80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- 172.21.15.82:0/2379325709 shutdown_connections 2024-04-25T10:07:15.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- 172.21.15.82:0/2379325709 wait complete. 2024-04-25T10:07:15.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 Processor -- start 2024-04-25T10:07:15.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.393+0000 7fdbed629640 1 -- start start 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe80ce510 0x7fdbe81e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8144ca0 0x7fdbe8132580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe80cc900 con 0x7fdbe806ec60 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe806df20 con 0x7fdbe8144ca0 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81451f0 con 0x7fdbe80ce510 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52370/0 (socket says 172.21.15.82:52370) 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/2957936158 learned_addr learned my addr 172.21.15.82:0/2957936158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8144ca0 0x7fdbe8132580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe80ce510 0x7fdbe81e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe80ce510 msgr2=0x7fdbe81e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe80ce510 0x7fdbe81e57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8144ca0 msgr2=0x7fdbe8132580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8144ca0 0x7fdbe8132580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/2957936158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe81e4720 con 0x7fdbe806ec60 2024-04-25T10:07:15.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e930 tx=0x7fdbd800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 -- 172.21.15.82:0/2957936158 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8017070 con 0x7fdbe806ec60 2024-04-25T10:07:15.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 -- 172.21.15.82:0/2957936158 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd800ca00 con 0x7fdbe806ec60 2024-04-25T10:07:15.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe81e4a00 con 0x7fdbe806ec60 2024-04-25T10:07:15.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 -- 172.21.15.82:0/2957936158 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800cdb0 con 0x7fdbe806ec60 2024-04-25T10:07:15.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe81e4fc0 con 0x7fdbe806ec60 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 -- 172.21.15.82:0/2957936158 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdbd801a070 con 0x7fdbe806ec60 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbcf7fe640 1 -- 172.21.15.82:0/2957936158 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdbd80855d0 con 0x7fdbe806ec60 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.397+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 0x7fdbd4069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b2000 tx=0x7fdbdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 Processor -- start 2024-04-25T10:07:15.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- start start 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a656caf30 0x562a656cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a655b25f0 con 0x562a65714920 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a65519da0 con 0x562a65715240 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a654a2150 con 0x562a656caf30 2024-04-25T10:07:15.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52376/0 (socket says 172.21.15.82:52376) 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1604793773 learned_addr learned my addr 172.21.15.82:0/1604793773 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a656caf30 msgr2=0x562a656cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a656caf30 0x562a656cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 msgr2=0x562a656ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1604793773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a654bbca0 con 0x562a65714920 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8055d20 tx=0x7fdbd8052100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c10b9a10bdd59e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbccff9640 1 -- 172.21.15.82:0/1604793773 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80540e0 con 0x562a65714920 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbccff9640 1 -- 172.21.15.82:0/1604793773 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8052450 con 0x562a65714920 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbccff9640 1 -- 172.21.15.82:0/1604793773 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8055450 con 0x562a65714920 2024-04-25T10:07:15.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- 172.21.15.82:0/1604793773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 msgr2=0x562a65714d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8055d20 tx=0x7fdbd8052100 comp rx=0 tx=0).stop 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- 172.21.15.82:0/1604793773 shutdown_connections 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a656caf30 0x562a656cb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1604793773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65714d00 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- 172.21.15.82:0/1604793773 >> 172.21.15.82:0/1604793773 conn(0x562a656d12a0 msgr2=0x562a656c92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- 172.21.15.82:0/1604793773 shutdown_connections 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- 172.21.15.82:0/1604793773 wait complete. 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 -- start start 2024-04-25T10:07:15.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.401+0000 7fdbf0796000 1 Processor -- start 2024-04-25T10:07:15.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a6571c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a654d4990 con 0x562a656caf30 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a65515e40 con 0x562a65715240 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a654a5900 con 0x562a65714920 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 10 client.0 _ll_register_callbacks cb 0x562a655cd1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52390/0 (socket says 172.21.15.82:52390) 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1224426853 learned_addr learned my addr 172.21.15.82:0/1224426853 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a6571c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 msgr2=0x562a6571c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a6571c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 msgr2=0x562a656ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 -- 172.21.15.82:0/1224426853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a654d2630 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e900 tx=0x7fdbd8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd41784f35814b65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80852b0 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdbd807cd00 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbd8052450 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd8063050 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 10 client.7302 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x562a654a23a0 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbd8052450 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdbd8066020 con 0x562a656caf30 2024-04-25T10:07:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 10 client.7302 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 10 client.7302 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 20 client.7302 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a656d0160 0x562a65725c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 4 client.7302 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a654b5a50 con 0x562a656d0160 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a656d0160 0x562a65725c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 10 client.7302 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a656d0160 0x562a65725c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29be479f6fa800b9 server_cookie=e68ff71ad113bbcb in_seq=0 out_seq=0 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/1224426853 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562a654b5a50 con 0x562a656d0160 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 10 client.7302 handle_command_reply: tid=0 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 client.7302 shutdown 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 msgr2=0x562a6571bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e900 tx=0x7fdbd8000bd0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 shutdown_connections 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a6571c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a656d0160 0x562a65725c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a65715240 0x562a656ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1224426853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a656caf30 0x562a6571bc80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbc5ffb640 0 client.7302 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 >> 172.21.15.82:0/1224426853 conn(0x562a656d12a0 msgr2=0x562a656c99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 shutdown_connections 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 1 -- 172.21.15.82:0/1224426853 wait complete. 2024-04-25T10:07:15.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.405+0000 7fdbf0796000 20 client.7302 trim_cache size 0 max 16384 2024-04-25T10:07:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 Processor -- start 2024-04-25T10:07:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 -- start start 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a65770120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a65770660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81fd450 con 0x562a65715240 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81e17c0 con 0x562a656caf30 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81fce50 con 0x562a65714920 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52392/0 (socket says 172.21.15.82:52392) 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 -- 172.21.15.82:0/1022370432 learned_addr learned my addr 172.21.15.82:0/1022370432 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 -- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 msgr2=0x562a65770660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a65770660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 -- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 msgr2=0x562a65770120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a65770120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 -- 172.21.15.82:0/1022370432 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a65715070 con 0x562a65715240 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbe77fe640 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009f40 tx=0x7fdbe0003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c4cfadbf2b21e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbc7fff640 1 -- 172.21.15.82:0/1022370432 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0015480 con 0x562a65715240 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbc7fff640 1 -- 172.21.15.82:0/1022370432 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe001a070 con 0x562a65715240 2024-04-25T10:07:15.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.421+0000 7fdbc7fff640 1 -- 172.21.15.82:0/1022370432 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0015880 con 0x562a65715240 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/1022370432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 msgr2=0x562a65770ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009f40 tx=0x7fdbe0003e30 comp rx=0 tx=0).stop 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/1022370432 shutdown_connections 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65714920 0x562a65770660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65715240 0x562a65770ba0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/1022370432 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a65770120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/1022370432 >> 172.21.15.82:0/1022370432 conn(0x562a656d12a0 msgr2=0x562a656ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/1022370432 shutdown_connections 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/1022370432 wait complete. 2024-04-25T10:07:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- start start 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 Processor -- start 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a657588d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65715240 0x562a657211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a6551e3d0 con 0x562a65714920 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a655d5d50 con 0x562a656caf30 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a654b4980 con 0x562a65715240 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 10 client.0 _ll_register_callbacks cb 0x7fdbdc0b3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52396/0 (socket says 172.21.15.82:52396) 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 -- 172.21.15.82:0/4119797032 learned_addr learned my addr 172.21.15.82:0/4119797032 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a657588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 -- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65715240 msgr2=0x562a657211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65715240 0x562a657211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 -- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 msgr2=0x562a657588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a657588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 -- 172.21.15.82:0/4119797032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a6559b600 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe67fc640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0090b0 tx=0x7fdbdc01d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8d7e4eb48b6a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0b5040 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdbdc0d2c80 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbe001a070 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbdc0a8e60 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 10 client.7308 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x562a654ae9b0 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe001a070 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdbdc0d0030 con 0x562a65714920 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 10 client.7308 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 10 client.7308 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 20 client.7308 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a657591c0 0x562a657595a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 4 client.7308 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562a655272e0 con 0x562a657591c0 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a657591c0 0x562a657595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 10 client.7308 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbe6ffd640 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a657591c0 0x562a657595a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d779fe51b66b6a9 server_cookie=8028cca0ec8396e5 in_seq=0 out_seq=0 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 1 -- 172.21.15.82:0/4119797032 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562a655272e0 con 0x562a657591c0 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 10 client.7308 handle_command_reply: tid=0 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 client.7308 shutdown 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 msgr2=0x562a65720c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0090b0 tx=0x7fdbdc01d550 comp rx=0 tx=0).stop 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 shutdown_connections 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x562a65715240 0x562a657211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x562a657591c0 0x562a657595a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x562a65714920 0x562a65720c90 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbc5ffb640 0 client.7308 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 --2- 172.21.15.82:0/4119797032 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x562a656caf30 0x562a657588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.425+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 >> 172.21.15.82:0/4119797032 conn(0x562a656d12a0 msgr2=0x562a6571c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 shutdown_connections 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbf0796000 1 -- 172.21.15.82:0/4119797032 wait complete. 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbf0796000 20 client.7308 trim_cache size 0 max 16384 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7260, 2024-04-25T10:07:15.437 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4686612769999998, 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 47, 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 80, 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 80, 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 5.4253052420000003 2024-04-25T10:07:15.438 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 msgr2=0x7fdbd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 0x7fdbd4069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0b2000 tx=0x7fdbdc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 msgr2=0x7fdbe81e6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800e930 tx=0x7fdbd800edf0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 shutdown_connections 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdbd4066f60 0x7fdbd4069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdbe80ce510 0x7fdbe81e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdbe8144ca0 0x7fdbe8132580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 --2- 172.21.15.82:0/2957936158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdbe806ec60 0x7fdbe81e6470 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 >> 172.21.15.82:0/2957936158 conn(0x7fdbe80b11a0 msgr2=0x7fdbe8071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 shutdown_connections 2024-04-25T10:07:15.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.429+0000 7fdbed629640 1 -- 172.21.15.82:0/2957936158 wait complete. 2024-04-25T10:07:15.448 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:15.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 Processor -- start 2024-04-25T10:07:15.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 -- start start 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec15b950 0x7f4bec09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4bec0cf420 0x7f4bec0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec15bd30 con 0x7f4bec0cfb80 2024-04-25T10:07:15.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec0cc0d0 con 0x7f4bec0cf420 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf3827640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec09ab10 con 0x7f4bec15b950 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4bec0cf420 0x7f4bec0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4bec0cf420 0x7f4bec0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52402/0 (socket says 172.21.15.82:52402) 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 -- 172.21.15.82:0/1267834826 learned_addr learned my addr 172.21.15.82:0/1267834826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf15c3640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec15b950 0x7f4bec09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 -- 172.21.15.82:0/1267834826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec15b950 msgr2=0x7f4bec09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec15b950 0x7f4bec09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 -- 172.21.15.82:0/1267834826 >> v1:172.21.15.196:6789/0 conn(0x7f4bec0cfb80 legacy=0x7f4bec06f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.557+0000 7f4bf0dc2640 1 -- 172.21.15.82:0/1267834826 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0cc370 con 0x7f4bec0cf420 2024-04-25T10:07:15.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf0dc2640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec0cf420 0x7f4bec0cce50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009870 tx=0x7f4bdc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f25d6591da9c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4be3fff640 1 -- 172.21.15.82:0/1267834826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0ab640 con 0x7f4bec0cf420 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4be3fff640 1 -- 172.21.15.82:0/1267834826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc0b0070 con 0x7f4bec0cf420 2024-04-25T10:07:15.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4be3fff640 1 -- 172.21.15.82:0/1267834826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0aba00 con 0x7f4bec0cf420 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- 172.21.15.82:0/1267834826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec0cf420 msgr2=0x7f4bec0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec0cf420 0x7f4bec0cce50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009870 tx=0x7f4bdc01c530 comp rx=0 tx=0).stop 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- 172.21.15.82:0/1267834826 shutdown_connections 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec0cf420 0x7f4bec0cce50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1267834826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec15b950 0x7f4bec09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- 172.21.15.82:0/1267834826 >> 172.21.15.82:0/1267834826 conn(0x7f4bec0b30b0 msgr2=0x7f4bec0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- 172.21.15.82:0/1267834826 shutdown_connections 2024-04-25T10:07:15.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- 172.21.15.82:0/1267834826 wait complete. 2024-04-25T10:07:15.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 Processor -- start 2024-04-25T10:07:15.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- start start 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 0x7f4bec0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec0cfb80 0x7f4bec153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0cd390 con 0x7f4bec15b950 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0d03a0 con 0x7f4bec0cf420 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf3827640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec0cad00 con 0x7f4bec0cfb80 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52406/0 (socket says 172.21.15.82:52406) 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 -- 172.21.15.82:0/1360990680 learned_addr learned my addr 172.21.15.82:0/1360990680 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf15c3640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 0x7f4bec0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf0dc2640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec0cfb80 0x7f4bec153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 -- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec0cfb80 msgr2=0x7f4bec153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec0cfb80 0x7f4bec153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 -- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 msgr2=0x7f4bec0c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 0x7f4bec0c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 -- 172.21.15.82:0/1360990680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0cafe0 con 0x7f4bec15b950 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf15c3640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 0x7f4bec0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4bf1dc4640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c730 tx=0x7f4be400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.561+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be4003b80 con 0x7f4bec15b950 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be4003d20 con 0x7f4bec15b950 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0cb2c0 con 0x7f4bec15b950 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be400aa30 con 0x7f4bec15b950 2024-04-25T10:07:15.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0cb880 con 0x7f4bec15b950 2024-04-25T10:07:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec04efa0 con 0x7f4bec15b950 2024-04-25T10:07:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4be400abd0 con 0x7f4bec15b950 2024-04-25T10:07:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4be1ffb640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 0x7f4bd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4be4085020 con 0x7f4bec15b950 2024-04-25T10:07:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4bf15c3640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 0x7f4bd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.565+0000 7f4bf15c3640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 0x7f4bd0069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4bec1f84e0 tx=0x7f4be8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.569+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4be4055ee0 con 0x7f4bec15b950 2024-04-25T10:07:15.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4bec0d13a0 con 0x7f4bec15b950 2024-04-25T10:07:15.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4be1ffb640 1 -- 172.21.15.82:0/1360990680 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4be40557e0 con 0x7f4bec15b950 2024-04-25T10:07:15.753 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 msgr2=0x7f4bd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 0x7f4bd0069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4bec1f84e0 tx=0x7f4be8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 msgr2=0x7f4bec0ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4be400c730 tx=0x7f4be400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 shutdown_connections 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bd0066f60 0x7f4bd0069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.749+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4bec0cfb80 0x7f4bec153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.753+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bec15b950 0x7f4bec0ca420 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.753+0000 7f4bf3827640 1 --2- 172.21.15.82:0/1360990680 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bec0cf420 0x7f4bec0c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.753+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 >> 172.21.15.82:0/1360990680 conn(0x7f4bec0b30b0 msgr2=0x7f4bec0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.753+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 shutdown_connections 2024-04-25T10:07:15.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:15.753+0000 7f4bf3827640 1 -- 172.21.15.82:0/1360990680 wait complete. 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:15.771 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:15.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.772 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:07:15.777 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:15.777 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:15.778 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42234sec preferred_lft 42234sec 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.779 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:15.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.780 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:15.829 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:07:15.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.831 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:07:15.885 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:15.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:15.885 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:07:15.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.958 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:07:15.964 INFO:tasks.ceph.mds.a.smithi082.stderr:2024-04-25T10:07:15.961+0000 7f19b316f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:15.969 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:15.969 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:15.969 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:15.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:15.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:07:16.024 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:07:16.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:16.024 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:16.074 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:16.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:16.075 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:16.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:07:16.177 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:16.226 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:07:16.226 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:07:16.227 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:16.228 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:07:16.229 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:16.229 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:16.229 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:16.229 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:16.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.230 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:07:16.278 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:07:16.279 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:07:16.321 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:07:16.336 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:16.389 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:07:16.390 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:16.391 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:16.392 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:16.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:16.393 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:07:16.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:07:16.397+0000 7fc2c4322580 -1 init, newargv = 0x561ef4633b20 newargc=15 2024-04-25T10:07:16.401 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45904]: starting ceph client 2024-04-25T10:07:16.407 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45904]: starting fuse 2024-04-25T10:07:17.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:07:17.526 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:07:17.574 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:07:17.575 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:07:17.576 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:17.577 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.578 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:07:17.624 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:07:17.624 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:07:17.625 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:07:17.626 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:07:17.626 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:17.626 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:07:17.627 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:07:17.749 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.45904.asok 2024-04-25T10:07:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:17.761 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45904.asok 2024-04-25T10:07:17.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.762 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45904.asok status 2024-04-25T10:07:17.916 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "45904", 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:07:17.917 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7323, 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 7323 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 694932443 2024-04-25T10:07:17.918 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 694932443 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.7323 192.168.0.1:0/694932443", 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/694932443", 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 47, 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 80, 2024-04-25T10:07:17.919 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:07:17.920 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:07:17.920 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:07:17.920 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:17.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.931 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:07:17.931 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.936 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:07:17.937 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.937 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:17.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:17.991 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T10:07:18.042 INFO:teuthology.orchestra.run.smithi082.stdout:file_undamaged 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> import json 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> import sys 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> try: 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> except OSError as e: 2024-04-25T10:07:18.043 DEBUG:teuthology.orchestra.run.smithi082:> sys.exit(e.errno) 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> print(json.dumps( 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> indent=2)) 2024-04-25T10:07:18.044 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:07:18.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:07:18.165 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:07:18.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:18.165 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:07:18.215 INFO:teuthology.orchestra.run.smithi082.stdout:2 2024-04-25T10:07:18.215 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:07:18.215 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:18.264 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:18.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 Processor -- start 2024-04-25T10:07:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 -- start start 2024-04-25T10:07:18.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdabc0ce510 0x7fdabc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdabc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdabc0cee30 con 0x7fdabc06ec60 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdabc0cc2b0 con 0x7fdabc0ce510 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdac1bc1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc06e290 con 0x7fdabc144ca0 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabbfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdabc06ec60 0x7fdabc142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51034/0 (socket says 172.21.15.82:51034) 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabbfff640 1 -- 172.21.15.82:0/403268128 learned_addr learned my addr 172.21.15.82:0/403268128 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265120791 0 0) 0x7fdabc0cee30 con 0x7fdabc06ec60 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa8003530 con 0x7fdabc06ec60 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/403268128 >> v2:172.21.15.196:3300/0 conn(0x7fdabc0ce510 0x7fdabc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdab3fff640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdabc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 19386500 0 0) 0x7fdaa8003530 con 0x7fdabc06ec60 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 msgr2=0x7fdabc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdabc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 >> v2:172.21.15.196:3300/0 conn(0x7fdabc0ce510 msgr2=0x7fdabc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 --2- 172.21.15.82:0/403268128 >> v2:172.21.15.196:3300/0 conn(0x7fdabc0ce510 0x7fdabc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0ccc70 con 0x7fdabc06ec60 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fdab4001a00 con 0x7fdabc06ec60 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 >> v1:172.21.15.82:6789/0 conn(0x7fdabc06ec60 legacy=0x7fdabc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa8003a30 0x7fdaa8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdaa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa8004d50 0x7fdaa80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0cc2b0 con 0x7fdaa8003a30 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc06e290 con 0x7fdaa8004d50 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa800bae0 con 0x7fdabc144ca0 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa8003a30 0x7fdaa8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdab3fff640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdaa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 -- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 msgr2=0x7fdaa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdabc144ca0 0x7fdaa8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 -- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa8004d50 msgr2=0x7fdaa80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabbfff640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa8004d50 0x7fdaa80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdaa8004d50 0x7fdaa80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.433+0000 7fdabb7fe640 1 -- 172.21.15.82:0/403268128 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8003530 con 0x7fdaa8003a30 2024-04-25T10:07:18.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa8003a30 0x7fdaa8003e50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009ae0 tx=0x7fdaac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b456466f46fe30e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2070 con 0x7fdaa8003a30 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab40020c0 con 0x7fdaa8003a30 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdabaffd640 1 -- 172.21.15.82:0/403268128 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0096b0 con 0x7fdaa8003a30 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- 172.21.15.82:0/403268128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa8003a30 msgr2=0x7fdaa8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/403268128 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdaa8003a30 0x7fdaa8003e50 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009ae0 tx=0x7fdaac01cab0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdabbfff640 1 -- 172.21.15.82:0/403268128 reap_dead start 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- 172.21.15.82:0/403268128 shutdown_connections 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- 172.21.15.82:0/403268128 >> 172.21.15.82:0/403268128 conn(0x7fdabc0b11a0 msgr2=0x7fdabc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- 172.21.15.82:0/403268128 shutdown_connections 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- 172.21.15.82:0/403268128 wait complete. 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 Processor -- start 2024-04-25T10:07:18.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.437+0000 7fdac1bc1640 1 -- start start 2024-04-25T10:07:18.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaa8003a30 0x7fdabc13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdabc143160 0x7fdabc1435e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc0cc900 con 0x7fdabc1f24f0 2024-04-25T10:07:18.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc06df20 con 0x7fdabc143160 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdabc142d90 con 0x7fdaa8003a30 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52424/0 (socket says 172.21.15.82:52424) 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 -- 172.21.15.82:0/1120153107 learned_addr learned my addr 172.21.15.82:0/1120153107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab3fff640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdabc143160 0x7fdabc1435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaa8003a30 0x7fdabc13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 -- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaa8003a30 msgr2=0x7fdabc13bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaa8003a30 0x7fdabc13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 -- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdabc143160 msgr2=0x7fdabc1435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdabc143160 0x7fdabc1435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 -- 172.21.15.82:0/1120153107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc1f4300 con 0x7fdabc1f24f0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabbfff640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f90 tx=0x7fdab4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab401a070 con 0x7fdabc1f24f0 2024-04-25T10:07:18.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab4011040 con 0x7fdabc1f24f0 2024-04-25T10:07:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdabc1f3260 con 0x7fdabc1f24f0 2024-04-25T10:07:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab400ed40 con 0x7fdabc1f24f0 2024-04-25T10:07:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc1f3820 con 0x7fdabc1f24f0 2024-04-25T10:07:18.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdabc143b20 con 0x7fdabc1f24f0 2024-04-25T10:07:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdab401d070 con 0x7fdabc1f24f0 2024-04-25T10:07:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 0x7fda9c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdab4087fe0 con 0x7fdabc1f24f0 2024-04-25T10:07:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 0x7fda9c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.441+0000 7fdabb7fe640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 0x7fda9c069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0af000 tx=0x7fdaac0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.445+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdab40587c0 con 0x7fdabc1f24f0 2024-04-25T10:07:18.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.605+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdabc0008e0 con 0x7fdabc1f24f0 2024-04-25T10:07:18.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdab8ff9640 1 -- 172.21.15.82:0/1120153107 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fdab405c860 con 0x7fdabc1f24f0 2024-04-25T10:07:18.613 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:18.613 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 msgr2=0x7fda9c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 0x7fda9c069450 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdaac0af000 tx=0x7fdaac0046d0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 msgr2=0x7fdabc1f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fdab4000f90 tx=0x7fdab4010d70 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 shutdown_connections 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda9c066f90 0x7fda9c069450 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdabc1f24f0 0x7fdabc1f2970 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdabc143160 0x7fdabc1435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 --2- 172.21.15.82:0/1120153107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdaa8003a30 0x7fdabc13bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 >> 172.21.15.82:0/1120153107 conn(0x7fdabc0b11a0 msgr2=0x7fdabc0ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 shutdown_connections 2024-04-25T10:07:18.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.609+0000 7fdac1bc1640 1 -- 172.21.15.82:0/1120153107 wait complete. 2024-04-25T10:07:18.631 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:18.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:18.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.749+0000 7f3e766d4640 1 Processor -- start 2024-04-25T10:07:18.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 -- start start 2024-04-25T10:07:18.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700cf590 0x7f3e7015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e700cf970 0x7f3e700c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 0x7f3e700b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e700ca780 con 0x7f3e700cf970 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e700cd440 con 0x7f3e700c9ef0 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e766d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cdce0 con 0x7f3e700cf590 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e700cf970 0x7f3e700c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3e700cf970 0x7f3e700c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52440/0 (socket says 172.21.15.82:52440) 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 -- 172.21.15.82:0/3833695570 learned_addr learned my addr 172.21.15.82:0/3833695570 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e74c71640 1 --2- 172.21.15.82:0/3833695570 >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 0x7f3e700b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 -- 172.21.15.82:0/3833695570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700cf590 msgr2=0x7f3e7015ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e6ffff640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700cf590 0x7f3e7015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700cf590 0x7f3e7015ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 -- 172.21.15.82:0/3833695570 >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 msgr2=0x7f3e700b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 --2- 172.21.15.82:0/3833695570 >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 0x7f3e700b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 -- 172.21.15.82:0/3833695570 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700ca4c0 con 0x7f3e700cf970 2024-04-25T10:07:18.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e74c71640 1 --2- 172.21.15.82:0/3833695570 >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 0x7f3e700b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.753+0000 7f3e677fe640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf970 0x7f3e700c99b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f3e700cf310 tx=0x7f3e5801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f33af047787f7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6f7fe640 1 -- 172.21.15.82:0/3833695570 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b0070 con 0x7f3e700cf970 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6f7fe640 1 -- 172.21.15.82:0/3833695570 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e580ab600 con 0x7f3e700cf970 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6f7fe640 1 -- 172.21.15.82:0/3833695570 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580ab9d0 con 0x7f3e700cf970 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/3833695570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf970 msgr2=0x7f3e700c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf970 0x7f3e700c99b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f3e700cf310 tx=0x7f3e5801c230 comp rx=0 tx=0).stop 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/3833695570 shutdown_connections 2024-04-25T10:07:18.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- 172.21.15.82:0/3833695570 >> v2:172.21.15.196:3300/0 conn(0x7f3e700c9ef0 0x7f3e700b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf970 0x7f3e700c99b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- 172.21.15.82:0/3833695570 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700cf590 0x7f3e7015ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/3833695570 >> 172.21.15.82:0/3833695570 conn(0x7f3e7006ae60 msgr2=0x7f3e7006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/3833695570 shutdown_connections 2024-04-25T10:07:18.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/3833695570 wait complete. 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 Processor -- start 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- start start 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700c9ef0 0x7f3e700c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e700cf970 0x7f3e701563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700ac2f0 con 0x7f3e700cf590 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700ac170 con 0x7f3e700cf970 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cbde0 con 0x7f3e700c9ef0 2024-04-25T10:07:18.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52456/0 (socket says 172.21.15.82:52456) 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 -- 172.21.15.82:0/168114272 learned_addr learned my addr 172.21.15.82:0/168114272 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e74c71640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e700cf970 0x7f3e701563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6ffff640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700c9ef0 0x7f3e700c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 -- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700c9ef0 msgr2=0x7f3e700c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700c9ef0 0x7f3e700c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 -- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e700cf970 msgr2=0x7f3e701563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e700cf970 0x7f3e701563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 -- 172.21.15.82:0/168114272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e700c49a0 con 0x7f3e700cf590 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e677fe640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5801a8e0 tx=0x7f3e58018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b9070 con 0x7f3e700cf590 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e580b4600 con 0x7f3e700cf590 2024-04-25T10:07:18.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e580b0040 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e700c4c80 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.757+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e700c5240 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e7004efa0 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e580bc070 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e6d7fa640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 0x7f3e50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e6ffff640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 0x7f3e50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3e5811e140 con 0x7f3e700cf590 2024-04-25T10:07:18.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.761+0000 7f3e6ffff640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 0x7f3e50069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000a970 tx=0x7f3e60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.765+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e58016110 con 0x7f3e700cf590 2024-04-25T10:07:18.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.925+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e700b4170 con 0x7f3e700cf590 2024-04-25T10:07:18.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e6d7fa640 1 -- 172.21.15.82:0/168114272 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e580ef090 con 0x7f3e700cf590 2024-04-25T10:07:18.932 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 msgr2=0x7f3e50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 0x7f3e50069420 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000a970 tx=0x7f3e60008040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 msgr2=0x7f3e70155e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5801a8e0 tx=0x7f3e58018fd0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 shutdown_connections 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3e50066f60 0x7f3e50069420 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3e700cf970 0x7f3e701563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3e700cf590 0x7f3e70155e90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 --2- 172.21.15.82:0/168114272 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3e700c9ef0 0x7f3e700c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 >> 172.21.15.82:0/168114272 conn(0x7f3e7006ae60 msgr2=0x7f3e7006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 shutdown_connections 2024-04-25T10:07:18.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:18.929+0000 7f3e766d4640 1 -- 172.21.15.82:0/168114272 wait complete. 2024-04-25T10:07:18.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:19.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.069+0000 7ff794663640 1 Processor -- start 2024-04-25T10:07:19.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.069+0000 7ff794663640 1 -- start start 2024-04-25T10:07:19.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff794663640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff794663640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff78c0cf590 0x7ff78c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff794663640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78c0cbd20 con 0x7ff78c15f130 2024-04-25T10:07:19.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff794663640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c15ba90 con 0x7ff78c0cf590 2024-04-25T10:07:19.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff794663640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cff30 con 0x7ff78c15b6b0 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff7923ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff791bfe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff78c0cf590 0x7ff78c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff7923ff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:33252/0 (socket says 172.21.15.82:33252) 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff7923ff640 1 -- 172.21.15.82:0/3934430366 learned_addr learned my addr 172.21.15.82:0/3934430366 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff791bfe640 1 -- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 msgr2=0x7ff78c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff791bfe640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff791bfe640 1 -- 172.21.15.82:0/3934430366 >> v1:172.21.15.196:6789/0 conn(0x7ff78c15f130 legacy=0x7ff78c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff791bfe640 1 -- 172.21.15.82:0/3934430366 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0cc000 con 0x7ff78c0cf590 2024-04-25T10:07:19.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.073+0000 7ff7923ff640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:19.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff791bfe640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c0cf590 0x7ff78c0cf9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0097c0 tx=0x7ff77c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9378720d4367240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7913fd640 1 -- 172.21.15.82:0/3934430366 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0a7070 con 0x7ff78c0cf590 2024-04-25T10:07:19.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7913fd640 1 -- 172.21.15.82:0/3934430366 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c0b0070 con 0x7ff78c0cf590 2024-04-25T10:07:19.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7913fd640 1 -- 172.21.15.82:0/3934430366 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0ab890 con 0x7ff78c0cf590 2024-04-25T10:07:19.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c0cf590 msgr2=0x7ff78c0cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c0cf590 0x7ff78c0cf9f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0097c0 tx=0x7ff77c01c470 comp rx=0 tx=0).stop 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- 172.21.15.82:0/3934430366 shutdown_connections 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c0cf590 0x7ff78c0cf9f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- 172.21.15.82:0/3934430366 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15b6b0 0x7ff78c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- 172.21.15.82:0/3934430366 >> 172.21.15.82:0/3934430366 conn(0x7ff78c06ae60 msgr2=0x7ff78c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- 172.21.15.82:0/3934430366 shutdown_connections 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- 172.21.15.82:0/3934430366 wait complete. 2024-04-25T10:07:19.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 Processor -- start 2024-04-25T10:07:19.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- start start 2024-04-25T10:07:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 0x7ff78c06de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f130 0x7ff78c1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0ca900 con 0x7ff78c0cf590 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cba30 con 0x7ff78c15b6b0 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff794663640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cadb0 con 0x7ff78c15f130 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52470/0 (socket says 172.21.15.82:52470) 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 -- 172.21.15.82:0/1812780841 learned_addr learned my addr 172.21.15.82:0/1812780841 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff791bfe640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 0x7ff78c06de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff792c00640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f130 0x7ff78c1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 -- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f130 msgr2=0x7ff78c1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f130 0x7ff78c1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 -- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 msgr2=0x7ff78c06de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 0x7ff78c06de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 -- 172.21.15.82:0/1812780841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c152820 con 0x7ff78c0cf590 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff791bfe640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 0x7ff78c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff7923ff640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9d0 tx=0x7ff78800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.077+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788017070 con 0x7ff78c0cf590 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c155e10 con 0x7ff78c0cf590 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800ca60 con 0x7ff78c0cf590 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ce10 con 0x7ff78c0cf590 2024-04-25T10:07:19.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c1560f0 con 0x7ff78c0cf590 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c04efa0 con 0x7ff78c0cf590 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff78801a070 con 0x7ff78c0cf590 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff782ffd640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 0x7ff770069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff791bfe640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 0x7ff770069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff788085210 con 0x7ff78c0cf590 2024-04-25T10:07:19.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.081+0000 7ff791bfe640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 0x7ff770069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0af000 tx=0x7ff77c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.085+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff788055fc0 con 0x7ff78c0cf590 2024-04-25T10:07:19.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.233+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff78c165dd0 con 0x7ff78c0cf590 2024-04-25T10:07:19.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.233+0000 7ff782ffd640 1 -- 172.21.15.82:0/1812780841 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7880558c0 con 0x7ff78c0cf590 2024-04-25T10:07:19.238 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 msgr2=0x7ff770069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 0x7ff770069420 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0af000 tx=0x7ff77c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 msgr2=0x7ff78c0c1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9d0 tx=0x7ff78800ee90 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 shutdown_connections 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff770066f60 0x7ff770069420 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff78c15f130 0x7ff78c1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff78c15b6b0 0x7ff78c06de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 --2- 172.21.15.82:0/1812780841 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff78c0cf590 0x7ff78c0c1f50 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 >> 172.21.15.82:0/1812780841 conn(0x7ff78c06ae60 msgr2=0x7ff78c0af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 shutdown_connections 2024-04-25T10:07:19.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.237+0000 7ff794663640 1 -- 172.21.15.82:0/1812780841 wait complete. 2024-04-25T10:07:19.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:19.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 Processor -- start 2024-04-25T10:07:19.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 -- start start 2024-04-25T10:07:19.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc01406bca0 0x7fc0140cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc014066e30 0x7fc01406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc01406d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0140669a0 con 0x7fc01406bca0 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01406ec30 con 0x7fc014066e30 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc019043640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140cd5f0 con 0x7fc0140af370 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc01406bca0 0x7fc0140cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc01406bca0 0x7fc0140cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52486/0 (socket says 172.21.15.82:52486) 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 -- 172.21.15.82:0/683361170 learned_addr learned my addr 172.21.15.82:0/683361170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc01259c640 1 --2- 172.21.15.82:0/683361170 >> v2:172.21.15.196:3300/0 conn(0x7fc014066e30 0x7fc01406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc01359e640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc01406d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 -- 172.21.15.82:0/683361170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 msgr2=0x7fc01406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc01406d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 -- 172.21.15.82:0/683361170 >> v2:172.21.15.196:3300/0 conn(0x7fc014066e30 msgr2=0x7fc01406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 --2- 172.21.15.82:0/683361170 >> v2:172.21.15.196:3300/0 conn(0x7fc014066e30 0x7fc01406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.373+0000 7fc012d9d640 1 -- 172.21.15.82:0/683361170 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01406c5c0 con 0x7fc01406bca0 2024-04-25T10:07:19.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc012d9d640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc01406bca0 0x7fc0140cd0b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc0140731a0 tx=0x7fc00401c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd8b4c1f125f17b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc011d9b640 1 -- 172.21.15.82:0/683361170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040b0070 con 0x7fc01406bca0 2024-04-25T10:07:19.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc011d9b640 1 -- 172.21.15.82:0/683361170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0040ab600 con 0x7fc01406bca0 2024-04-25T10:07:19.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc011d9b640 1 -- 172.21.15.82:0/683361170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040ab9d0 con 0x7fc01406bca0 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- 172.21.15.82:0/683361170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc01406bca0 msgr2=0x7fc0140cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc01406bca0 0x7fc0140cd0b0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc0140731a0 tx=0x7fc00401c110 comp rx=0 tx=0).stop 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- 172.21.15.82:0/683361170 shutdown_connections 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc01406d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 --2- 172.21.15.82:0/683361170 >> v2:172.21.15.196:3300/0 conn(0x7fc014066e30 0x7fc01406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 --2- 172.21.15.82:0/683361170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc01406bca0 0x7fc0140cd0b0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- 172.21.15.82:0/683361170 >> 172.21.15.82:0/683361170 conn(0x7fc014071810 msgr2=0x7fc014071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- 172.21.15.82:0/683361170 shutdown_connections 2024-04-25T10:07:19.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- 172.21.15.82:0/683361170 wait complete. 2024-04-25T10:07:19.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 Processor -- start 2024-04-25T10:07:19.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.377+0000 7fc019043640 1 -- start start 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 0x7fc0140b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc014158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415cc80 con 0x7fc014066e30 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01415cb00 con 0x7fc01406bca0 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0140ceaf0 con 0x7fc0140af370 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52500/0 (socket says 172.21.15.82:52500) 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 -- 172.21.15.82:0/976998480 learned_addr learned my addr 172.21.15.82:0/976998480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:19.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc01259c640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 0x7fc0140b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 -- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 msgr2=0x7fc014158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc014158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 -- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 msgr2=0x7fc0140b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 0x7fc0140b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 -- 172.21.15.82:0/976998480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0141547c0 con 0x7fc014066e30 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc01259c640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 0x7fc0140b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:19.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc012d9d640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc0040ae040 tx=0x7fc004018eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040b0040 con 0x7fc014066e30 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0040b9070 con 0x7fc014066e30 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc014154aa0 con 0x7fc014066e30 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0040b4680 con 0x7fc014066e30 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc014156c80 con 0x7fc014066e30 2024-04-25T10:07:19.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01404efa0 con 0x7fc014066e30 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0040b4820 con 0x7fc014066e30 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc01259c640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc00411eda0 con 0x7fc014066e30 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.381+0000 7fc01259c640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 0x7fc000069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc0141fa340 tx=0x7fc008008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.385+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc004016110 con 0x7fc014066e30 2024-04-25T10:07:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.533+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc014072400 con 0x7fc014066e30 2024-04-25T10:07:19.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.533+0000 7fbffb7fe640 1 -- 172.21.15.82:0/976998480 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0040efd80 con 0x7fc014066e30 2024-04-25T10:07:19.538 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 msgr2=0x7fc000069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 0x7fc000069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc0141fa340 tx=0x7fc008008040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 msgr2=0x7fc014146020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fc0040ae040 tx=0x7fc004018eb0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 shutdown_connections 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc000066f60 0x7fc000069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0140af370 0x7fc014158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc01406bca0 0x7fc0140b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 --2- 172.21.15.82:0/976998480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc014066e30 0x7fc014146020 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 >> 172.21.15.82:0/976998480 conn(0x7fc014071810 msgr2=0x7fc014152a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 shutdown_connections 2024-04-25T10:07:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.537+0000 7fc019043640 1 -- 172.21.15.82:0/976998480 wait complete. 2024-04-25T10:07:19.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:19.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 Processor -- start 2024-04-25T10:07:19.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 -- start start 2024-04-25T10:07:19.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f175809a180 0x7f17580b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f175809a7b0 0x7f17580ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cbec0 con 0x7f17580b2c30 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580cc1a0 con 0x7f175809a180 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175ca63640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580cc480 con 0x7f175809a7b0 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1755d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f175809a180 0x7f17580b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f17580b2c30 0x7f1758099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51046/0 (socket says 172.21.15.82:51046) 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175659c640 1 -- 172.21.15.82:0/3232168829 learned_addr learned my addr 172.21.15.82:0/3232168829 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1756d9d640 1 --2- 172.21.15.82:0/3232168829 >> v2:172.21.15.196:3300/0 conn(0x7f175809a7b0 0x7f17580ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1755d9b640 1 -- 172.21.15.82:0/3232168829 >> v2:172.21.15.196:3300/0 conn(0x7f175809a7b0 msgr2=0x7f17580ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1755d9b640 1 --2- 172.21.15.82:0/3232168829 >> v2:172.21.15.196:3300/0 conn(0x7f175809a7b0 0x7f17580ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1755d9b640 1 -- 172.21.15.82:0/3232168829 >> v1:172.21.15.82:6789/0 conn(0x7f17580b2c30 legacy=0x7f1758099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f1755d9b640 1 -- 172.21.15.82:0/3232168829 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580cc760 con 0x7f175809a180 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.673+0000 7f175559a640 1 -- 172.21.15.82:0/3232168829 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163539532 0 0) 0x7f17580cbec0 con 0x7f17580b2c30 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f1755d9b640 1 --2- 172.21.15.82:0/3232168829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f17580b0ba0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1758072bf0 tx=0x7f174c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de9032161f8c20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175559a640 1 -- 172.21.15.82:0/3232168829 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0a7070 con 0x7f175809a180 2024-04-25T10:07:19.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175559a640 1 -- 172.21.15.82:0/3232168829 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174c0b0070 con 0x7f175809a180 2024-04-25T10:07:19.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175559a640 1 -- 172.21.15.82:0/3232168829 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174c0ab690 con 0x7f175809a180 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- 172.21.15.82:0/3232168829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 msgr2=0x7f17580b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- 172.21.15.82:0/3232168829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f17580b0ba0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f1758072bf0 tx=0x7f174c01c450 comp rx=0 tx=0).stop 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- 172.21.15.82:0/3232168829 shutdown_connections 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- 172.21.15.82:0/3232168829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f17580b0ba0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- 172.21.15.82:0/3232168829 >> v2:172.21.15.196:3300/0 conn(0x7f175809a7b0 0x7f17580ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- 172.21.15.82:0/3232168829 >> 172.21.15.82:0/3232168829 conn(0x7f17580b3010 msgr2=0x7f17580b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- 172.21.15.82:0/3232168829 shutdown_connections 2024-04-25T10:07:19.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- 172.21.15.82:0/3232168829 wait complete. 2024-04-25T10:07:19.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 Processor -- start 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- start start 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f175809a7b0 0x7f175814d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17580b2c30 0x7f175814dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806c380 con 0x7f175809a180 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175806c200 con 0x7f175809a7b0 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.677+0000 7f175ca63640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175815d190 con 0x7f17580b2c30 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52520/0 (socket says 172.21.15.82:52520) 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 -- 172.21.15.82:0/1962966729 learned_addr learned my addr 172.21.15.82:0/1962966729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 -- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17580b2c30 msgr2=0x7f175814dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17580b2c30 0x7f175814dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1755d9b640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f175809a7b0 0x7f175814d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 -- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f175809a7b0 msgr2=0x7f175814d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f175809a7b0 0x7f175814d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 -- 172.21.15.82:0/1962966729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f175814e550 con 0x7f175809a180 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175659c640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f174800c480 tx=0x7f174800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748018070 con 0x7f175809a180 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1748007500 con 0x7f175809a180 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175814e830 con 0x7f175809a180 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1748004a30 con 0x7f175809a180 2024-04-25T10:07:19.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1758151c30 con 0x7f175809a180 2024-04-25T10:07:19.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175804efa0 con 0x7f175809a180 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f174801b070 con 0x7f175809a180 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1755d9b640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f17480845e0 con 0x7f175809a180 2024-04-25T10:07:19.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.681+0000 7f1755d9b640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 0x7f1738069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f174c0af000 tx=0x7f174c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.685+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1748055400 con 0x7f175809a180 2024-04-25T10:07:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.857+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f175806ba80 con 0x7f175809a180 2024-04-25T10:07:19.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.857+0000 7f1746ffd640 1 -- 172.21.15.82:0/1962966729 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+120 (secure 0 0 0) 0x7f1748054d00 con 0x7f175809a180 2024-04-25T10:07:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:07:19.863 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:07:19.863 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:07:19.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 msgr2=0x7f1738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 0x7f1738069420 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f174c0af000 tx=0x7f174c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 msgr2=0x7f1758152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f174800c480 tx=0x7f174800c940 comp rx=0 tx=0).stop 2024-04-25T10:07:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 shutdown_connections 2024-04-25T10:07:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1738066f60 0x7f1738069420 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f17580b2c30 0x7f175814dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f175809a7b0 0x7f175814d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 --2- 172.21.15.82:0/1962966729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f175809a180 0x7f1758152500 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 >> 172.21.15.82:0/1962966729 conn(0x7f17580b3010 msgr2=0x7f175806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 shutdown_connections 2024-04-25T10:07:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:19.861+0000 7f175ca63640 1 -- 172.21.15.82:0/1962966729 wait complete. 2024-04-25T10:07:19.881 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:07:19.881 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:19.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:20.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.037+0000 7efc317ce640 1 Processor -- start 2024-04-25T10:07:20.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.037+0000 7efc317ce640 1 -- start start 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc317ce640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc2c0ce510 0x7efc2c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc317ce640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c0cee30 con 0x7efc2c1451f0 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc317ce640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc2c0ccc70 con 0x7efc2c144ca0 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc317ce640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2c0cc2b0 con 0x7efc2c0ce510 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc2c0ce510 0x7efc2c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efc2c0ce510 0x7efc2c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52530/0 (socket says 172.21.15.82:52530) 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 -- 172.21.15.82:0/4014958811 learned_addr learned my addr 172.21.15.82:0/4014958811 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:20.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 -- 172.21.15.82:0/4014958811 >> v1:172.21.15.196:6789/0 conn(0x7efc2c144ca0 legacy=0x7efc2c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 -- 172.21.15.82:0/4014958811 >> v1:172.21.15.82:6789/0 conn(0x7efc2c1451f0 legacy=0x7efc2c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc29ffb640 1 -- 172.21.15.82:0/4014958811 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281587415 0 0) 0x7efc2c0cee30 con 0x7efc2c1451f0 2024-04-25T10:07:20.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.041+0000 7efc2affd640 1 -- 172.21.15.82:0/4014958811 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c06dc40 con 0x7efc2c0ce510 2024-04-25T10:07:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2affd640 1 --2- 172.21.15.82:0/4014958811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c0ce510 0x7efc2c0ce8f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24772e0b8703da87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc29ffb640 1 -- 172.21.15.82:0/4014958811 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0b5070 con 0x7efc2c0ce510 2024-04-25T10:07:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc29ffb640 1 -- 172.21.15.82:0/4014958811 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c0a8a50 con 0x7efc2c0ce510 2024-04-25T10:07:20.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc29ffb640 1 -- 172.21.15.82:0/4014958811 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0a8e20 con 0x7efc2c0ce510 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- 172.21.15.82:0/4014958811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c0ce510 msgr2=0x7efc2c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 --2- 172.21.15.82:0/4014958811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c0ce510 0x7efc2c0ce8f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efc1c009870 tx=0x7efc1c01ed50 comp rx=0 tx=0).stop 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- 172.21.15.82:0/4014958811 shutdown_connections 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 --2- 172.21.15.82:0/4014958811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c0ce510 0x7efc2c0ce8f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- 172.21.15.82:0/4014958811 >> 172.21.15.82:0/4014958811 conn(0x7efc2c0b11a0 msgr2=0x7efc2c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- 172.21.15.82:0/4014958811 shutdown_connections 2024-04-25T10:07:20.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- 172.21.15.82:0/4014958811 wait complete. 2024-04-25T10:07:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 Processor -- start 2024-04-25T10:07:20.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- start start 2024-04-25T10:07:20.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2c0ce510 0x7efc2c1e7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2c1451f0 0x7efc2c136dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c06ee30 con 0x7efc2c144ca0 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c06e290 con 0x7efc2c0ce510 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc317ce640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2c0cc900 con 0x7efc2c1451f0 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52544/0 (socket says 172.21.15.82:52544) 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 -- 172.21.15.82:0/4071107935 learned_addr learned my addr 172.21.15.82:0/4071107935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2b7fe640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2c1451f0 0x7efc2c136dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 -- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2c1451f0 msgr2=0x7efc2c136dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2affd640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2c0ce510 0x7efc2c1e7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2c1451f0 0x7efc2c136dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 -- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2c0ce510 msgr2=0x7efc2c1e7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2c0ce510 0x7efc2c1e7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 -- 172.21.15.82:0/4071107935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c1e35c0 con 0x7efc2c144ca0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.045+0000 7efc2a7fc640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c500 tx=0x7efc2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200052d0 con 0x7efc2c144ca0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc20005470 con 0x7efc2c144ca0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc20012830 con 0x7efc2c144ca0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc2c1e4a20 con 0x7efc2c144ca0 2024-04-25T10:07:20.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc2c1e7790 con 0x7efc2c144ca0 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc200129d0 con 0x7efc2c144ca0 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2c04efa0 con 0x7efc2c144ca0 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 0x7efc18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc2affd640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 0x7efc18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efc20084e90 con 0x7efc2c144ca0 2024-04-25T10:07:20.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.049+0000 7efc2affd640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 0x7efc18069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0b2000 tx=0x7efc1c0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.053+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc20055c30 con 0x7efc2c144ca0 2024-04-25T10:07:20.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.209+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc2c0b1900 con 0x7efc2c144ca0 2024-04-25T10:07:20.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.209+0000 7efc137fe640 1 -- 172.21.15.82:0/4071107935 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7efc20055530 con 0x7efc2c144ca0 2024-04-25T10:07:20.214 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:20.215 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:20.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 msgr2=0x7efc18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 0x7efc18069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0b2000 tx=0x7efc1c0aa070 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 msgr2=0x7efc2c136880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7efc2000c500 tx=0x7efc2000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 shutdown_connections 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efc18066f60 0x7efc18069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efc2c1451f0 0x7efc2c136dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efc2c144ca0 0x7efc2c136880 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 --2- 172.21.15.82:0/4071107935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efc2c0ce510 0x7efc2c1e7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 >> 172.21.15.82:0/4071107935 conn(0x7efc2c0b11a0 msgr2=0x7efc2c0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 shutdown_connections 2024-04-25T10:07:20.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.213+0000 7efc317ce640 1 -- 172.21.15.82:0/4071107935 wait complete. 2024-04-25T10:07:20.231 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:20.232 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:20.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.341+0000 7f9ca5f05640 1 Processor -- start 2024-04-25T10:07:20.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.341+0000 7f9ca5f05640 1 -- start start 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9ca5f05640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9ca015da80 0x7f9ca00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9ca5f05640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca006be60 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9ca5f05640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca015de60 con 0x7f9ca00d07f0 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9ca5f05640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca006c440 con 0x7f9ca015da80 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9ca00d1cd0 0x7f9ca00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51072/0 (socket says 172.21.15.82:51072) 2024-04-25T10:07:20.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9effd640 1 -- 172.21.15.82:0/233808811 learned_addr learned my addr 172.21.15.82:0/233808811 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:20.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195703819 0 0) 0x7f9ca006be60 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/233808811 >> v2:172.21.15.196:3300/0 conn(0x7f9ca015da80 0x7f9ca00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c88003530 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589560940 0 0) 0x7f9c88003530 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 >> v2:172.21.15.196:3300/0 conn(0x7f9ca015da80 msgr2=0x7f9ca00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 --2- 172.21.15.82:0/233808811 >> v2:172.21.15.196:3300/0 conn(0x7f9ca015da80 0x7f9ca00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 >> v1:172.21.15.196:6789/0 conn(0x7f9ca00d07f0 legacy=0x7f9ca006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca00d2450 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9c900016d0 con 0x7f9ca00d1cd0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 >> v1:172.21.15.82:6789/0 conn(0x7f9ca00d1cd0 legacy=0x7f9ca00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9c88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88004810 0x7f9c88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c880051d0 0x7f9c8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca006c440 con 0x7f9c88003a30 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8800c100 con 0x7f9c88004810 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8800c300 con 0x7f9c880051d0 2024-04-25T10:07:20.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9c88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c880051d0 msgr2=0x7f9c8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9c880051d0 0x7f9c8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9effd640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88004810 0x7f9c88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88004810 msgr2=0x7f9c88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9c88004810 0x7f9c88004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/233808811 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c88003530 con 0x7f9c88003a30 2024-04-25T10:07:20.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.345+0000 7f9c9ffff640 1 -- 172.21.15.82:0/233808811 reap_dead start 2024-04-25T10:07:20.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9c88003e70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ab0 tx=0x7f9c9401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3da5a4e395431871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940b2070 con 0x7f9c88003a30 2024-04-25T10:07:20.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940a56c0 con 0x7f9c88003a30 2024-04-25T10:07:20.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9e7fc640 1 -- 172.21.15.82:0/233808811 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940096b0 con 0x7f9c88003a30 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 msgr2=0x7f9c88003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9c88003e70 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ab0 tx=0x7f9c9401ca40 comp rx=0 tx=0).stop 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/233808811 shutdown_connections 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/233808811 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9c88003e70 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/233808811 >> 172.21.15.82:0/233808811 conn(0x7f9ca008b000 msgr2=0x7f9ca00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/233808811 shutdown_connections 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/233808811 wait complete. 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 Processor -- start 2024-04-25T10:07:20.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- start start 2024-04-25T10:07:20.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca00c8230 0x7f9ca00b8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca01fadb0 0x7f9ca01fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00d26f0 con 0x7f9c88003a30 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00d4640 con 0x7f9ca01fadb0 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00d2870 con 0x7f9ca00c8230 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52554/0 (socket says 172.21.15.82:52554) 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/2691998593 learned_addr learned my addr 172.21.15.82:0/2691998593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9effd640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca00c8230 0x7f9ca00b8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca00c8230 msgr2=0x7f9ca00b8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca00c8230 0x7f9ca00b8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca01fadb0 msgr2=0x7f9ca01fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca01fadb0 0x7f9ca01fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 -- 172.21.15.82:0/2691998593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca00b6c50 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c9f7fe640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ab0 tx=0x7f9c88005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940b2040 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9400f000 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca00b63b0 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c940096b0 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.349+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca00b6970 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca004efa0 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9c940c0030 con 0x7f9c88003a30 2024-04-25T10:07:20.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9c87fff640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 0x7f9c8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9c9effd640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 0x7f9c8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9c9411eec0 con 0x7f9c88003a30 2024-04-25T10:07:20.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.353+0000 7f9c9effd640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 0x7f9c8c069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00c8650 tx=0x7f9c90009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.357+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9c940c7060 con 0x7f9c88003a30 2024-04-25T10:07:20.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ca0160e70 con 0x7f9c88003a30 2024-04-25T10:07:20.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9c87fff640 1 -- 172.21.15.82:0/2691998593 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9c94016780 con 0x7f9c88003a30 2024-04-25T10:07:20.740 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 msgr2=0x7f9c8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 0x7f9c8c069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9ca00c8650 tx=0x7f9c90009040 comp rx=0 tx=0).stop 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 msgr2=0x7f9ca0156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9c94009ab0 tx=0x7f9c88005840 comp rx=0 tx=0).stop 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 shutdown_connections 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9c8c066f60 0x7f9c8c069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9ca00c8230 0x7f9ca00b8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9ca01fadb0 0x7f9ca01fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 --2- 172.21.15.82:0/2691998593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9c88003a30 0x7f9ca0156760 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 >> 172.21.15.82:0/2691998593 conn(0x7f9ca008b000 msgr2=0x7f9ca00b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 shutdown_connections 2024-04-25T10:07:20.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.505+0000 7f9ca5f05640 1 -- 172.21.15.82:0/2691998593 wait complete. 2024-04-25T10:07:20.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:20.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 Processor -- start 2024-04-25T10:07:20.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 -- start start 2024-04-25T10:07:20.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1d6c0cf590 0x7f1d6c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0cbd20 con 0x7f1d6c072000 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d6c0cc000 con 0x7f1d6c15b6b0 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d72c04640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d6c15ba90 con 0x7f1d6c0cf590 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d711a1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1d6c072000 0x7f1d6c072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51084/0 (socket says 172.21.15.82:51084) 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d711a1640 1 -- 172.21.15.82:0/3845864284 learned_addr learned my addr 172.21.15.82:0/3845864284 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d6bfff640 1 --2- 172.21.15.82:0/3845864284 >> v2:172.21.15.82:3300/0 conn(0x7f1d6c0cf590 0x7f1d6c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d6bfff640 1 -- 172.21.15.82:0/3845864284 >> v1:172.21.15.196:6789/0 conn(0x7f1d6c15b6b0 legacy=0x7f1d6c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d6bfff640 1 -- 172.21.15.82:0/3845864284 >> v1:172.21.15.82:6789/0 conn(0x7f1d6c072000 legacy=0x7f1d6c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.873+0000 7f1d6bfff640 1 -- 172.21.15.82:0/3845864284 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0cc2e0 con 0x7f1d6c0cf590 2024-04-25T10:07:20.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d6bfff640 1 --2- 172.21.15.82:0/3845864284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0cfa20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b1a60c6f8f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d6b7fe640 1 -- 172.21.15.82:0/3845864284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0b0070 con 0x7f1d6c0cf590 2024-04-25T10:07:20.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d6b7fe640 1 -- 172.21.15.82:0/3845864284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c004230 con 0x7f1d6c0cf590 2024-04-25T10:07:20.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d6b7fe640 1 -- 172.21.15.82:0/3845864284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003620 con 0x7f1d6c0cf590 2024-04-25T10:07:20.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- 172.21.15.82:0/3845864284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 msgr2=0x7f1d6c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 --2- 172.21.15.82:0/3845864284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0cfa20 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c01c460 comp rx=0 tx=0).stop 2024-04-25T10:07:20.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- 172.21.15.82:0/3845864284 shutdown_connections 2024-04-25T10:07:20.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 --2- 172.21.15.82:0/3845864284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0cfa20 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- 172.21.15.82:0/3845864284 >> 172.21.15.82:0/3845864284 conn(0x7f1d6c06ae60 msgr2=0x7f1d6c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- 172.21.15.82:0/3845864284 shutdown_connections 2024-04-25T10:07:20.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- 172.21.15.82:0/3845864284 wait complete. 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 Processor -- start 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.877+0000 7f1d72c04640 1 -- start start 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 0x7f1d6c0aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d6c15b6b0 0x7f1d6c098860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0caf30 con 0x7f1d6c0cf590 2024-04-25T10:07:20.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c0cadb0 con 0x7f1d6c15b6b0 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6c1692b0 con 0x7f1d6c072000 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52578/0 (socket says 172.21.15.82:52578) 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 -- 172.21.15.82:0/2353880329 learned_addr learned my addr 172.21.15.82:0/2353880329 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d709a0640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 0x7f1d6c0aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 -- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 msgr2=0x7f1d6c0aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 0x7f1d6c0aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 -- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d6c15b6b0 msgr2=0x7f1d6c098860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d6c15b6b0 0x7f1d6c098860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 -- 172.21.15.82:0/2353880329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d6c0afc50 con 0x7f1d6c0cf590 2024-04-25T10:07:20.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d709a0640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 0x7f1d6c0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d6bfff640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0b9020 con 0x7f1d6c0cf590 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5c0043e0 con 0x7f1d6c0cf590 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6c0ada00 con 0x7f1d6c0cf590 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0b0040 con 0x7f1d6c0cf590 2024-04-25T10:07:20.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6c0adfc0 con 0x7f1d6c0cf590 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6c04efa0 con 0x7f1d6c0cf590 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d5c0bc020 con 0x7f1d6c0cf590 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d709a0640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1d5c11f2d0 con 0x7f1d6c0cf590 2024-04-25T10:07:20.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.881+0000 7f1d709a0640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400a850 tx=0x7f1d54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:20.885+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d5c0c3030 con 0x7f1d6c0cf590 2024-04-25T10:07:21.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.033+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d6c06bf80 con 0x7f1d6c0cf590 2024-04-25T10:07:21.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.033+0000 7f1d697fa640 1 -- 172.21.15.82:0/2353880329 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d6c0afc50 con 0x7f1d6c0cf590 2024-04-25T10:07:21.039 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 msgr2=0x7f1d4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 0x7f1d4c069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1d5400a850 tx=0x7f1d54008040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 msgr2=0x7f1d6c0ad390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c0097c0 tx=0x7f1d5c019200 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 shutdown_connections 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d4c066f60 0x7f1d4c069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d6c15b6b0 0x7f1d6c098860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d6c0cf590 0x7f1d6c0ad390 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 --2- 172.21.15.82:0/2353880329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d6c072000 0x7f1d6c0aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 >> 172.21.15.82:0/2353880329 conn(0x7f1d6c06ae60 msgr2=0x7f1d6c15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 shutdown_connections 2024-04-25T10:07:21.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.037+0000 7f1d72c04640 1 -- 172.21.15.82:0/2353880329 wait complete. 2024-04-25T10:07:21.056 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:21.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.169+0000 7f2234a27640 1 Processor -- start 2024-04-25T10:07:21.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.169+0000 7f2234a27640 1 -- start start 2024-04-25T10:07:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f2234a27640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f22301634c0 0x7f22301658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f2234a27640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300ca4c0 con 0x7f22300cf590 2024-04-25T10:07:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f2234a27640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300cd690 con 0x7f223015ecb0 2024-04-25T10:07:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f2234a27640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22300ca780 con 0x7f22301634c0 2024-04-25T10:07:21.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f22300cf590 0x7f22300cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51100/0 (socket says 172.21.15.82:51100) 2024-04-25T10:07:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222e59c640 1 -- 172.21.15.82:0/1401655995 learned_addr learned my addr 172.21.15.82:0/1401655995 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222ed9d640 1 --2- 172.21.15.82:0/1401655995 >> v2:172.21.15.82:3300/0 conn(0x7f22301634c0 0x7f22301658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222d59a640 1 -- 172.21.15.82:0/1401655995 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1251600470 0 0) 0x7f22300ca4c0 con 0x7f22300cf590 2024-04-25T10:07:21.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222d59a640 1 -- 172.21.15.82:0/1401655995 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2218003530 con 0x7f22300cf590 2024-04-25T10:07:21.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222ed9d640 1 -- 172.21.15.82:0/1401655995 >> v1:172.21.15.196:6789/0 conn(0x7f223015ecb0 legacy=0x7f22300c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222ed9d640 1 -- 172.21.15.82:0/1401655995 >> v1:172.21.15.82:6789/0 conn(0x7f22300cf590 legacy=0x7f22300cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222ed9d640 1 -- 172.21.15.82:0/1401655995 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22300cd970 con 0x7f22301634c0 2024-04-25T10:07:21.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222ed9d640 1 --2- 172.21.15.82:0/1401655995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22301634c0 0x7f22301658b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2228009d90 tx=0x7f222801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3def979601d45630 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222d59a640 1 -- 172.21.15.82:0/1401655995 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22280b2070 con 0x7f22301634c0 2024-04-25T10:07:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222d59a640 1 -- 172.21.15.82:0/1401655995 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22280a5730 con 0x7f22301634c0 2024-04-25T10:07:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.173+0000 7f222d59a640 1 -- 172.21.15.82:0/1401655995 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22280a5ae0 con 0x7f22301634c0 2024-04-25T10:07:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/1401655995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22301634c0 msgr2=0x7f22301658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 --2- 172.21.15.82:0/1401655995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22301634c0 0x7f22301658b0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f2228009d90 tx=0x7f222801c970 comp rx=0 tx=0).stop 2024-04-25T10:07:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/1401655995 shutdown_connections 2024-04-25T10:07:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 --2- 172.21.15.82:0/1401655995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22301634c0 0x7f22301658b0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/1401655995 >> 172.21.15.82:0/1401655995 conn(0x7f223006ae60 msgr2=0x7f223006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/1401655995 shutdown_connections 2024-04-25T10:07:21.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/1401655995 wait complete. 2024-04-25T10:07:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 Processor -- start 2024-04-25T10:07:21.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- start start 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 0x7f2230072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 0x7f22301648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300cc980 con 0x7f223015ecb0 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22300cc800 con 0x7f22301634c0 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300ccb00 con 0x7f22300cf590 2024-04-25T10:07:21.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52596/0 (socket says 172.21.15.82:52596) 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 -- 172.21.15.82:0/2999854994 learned_addr learned my addr 172.21.15.82:0/2999854994 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222e59c640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 0x7f2230072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222ed9d640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 0x7f22301648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 -- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 msgr2=0x7f2230072980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 0x7f2230072980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 -- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 msgr2=0x7f22301648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 0x7f22301648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 -- 172.21.15.82:0/2999854994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2230151880 con 0x7f223015ecb0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222e59c640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 0x7f2230072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222ed9d640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 0x7f22301648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f222dd9b640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f222400c500 tx=0x7f222400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224016020 con 0x7f223015ecb0 2024-04-25T10:07:21.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2224005220 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2230151b60 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224007030 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.177+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2230152120 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223004efa0 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22240071d0 con 0x7f223015ecb0 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f221effd640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 0x7f2210069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2224084bf0 con 0x7f223015ecb0 2024-04-25T10:07:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f222e59c640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 0x7f2210069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.181+0000 7f222e59c640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 0x7f2210069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f22200011f0 tx=0x7f2220009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.185+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2224020020 con 0x7f223015ecb0 2024-04-25T10:07:21.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f22300b3430 con 0x7f223015ecb0 2024-04-25T10:07:21.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f221effd640 1 -- 172.21.15.82:0/2999854994 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f22240074c0 con 0x7f223015ecb0 2024-04-25T10:07:21.336 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:07:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 msgr2=0x7f2210069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 0x7f2210069450 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f22200011f0 tx=0x7f2220009040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 msgr2=0x7f22300b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f222400c500 tx=0x7f222400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 shutdown_connections 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2210066f90 0x7f2210069450 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22301634c0 0x7f22301648f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f223015ecb0 0x7f22300b1c00 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 --2- 172.21.15.82:0/2999854994 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22300cf590 0x7f2230072980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 >> 172.21.15.82:0/2999854994 conn(0x7f223006ae60 msgr2=0x7f22300b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 shutdown_connections 2024-04-25T10:07:21.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.333+0000 7f2234a27640 1 -- 172.21.15.82:0/2999854994 wait complete. 2024-04-25T10:07:21.354 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:21.423 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:07:21.417+0000 7ff48c596640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:07:21.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 Processor -- start 2024-04-25T10:07:21.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 -- start start 2024-04-25T10:07:21.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effe015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe00b2090 con 0x7effe00b24a0 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe00b43e0 con 0x7effe00b3f50 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe7e11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe015f200 con 0x7effe015a700 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe63ae640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7effe00b24a0 0x7effe006e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51104/0 (socket says 172.21.15.82:51104) 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe63ae640 1 -- 172.21.15.82:0/2060606221 learned_addr learned my addr 172.21.15.82:0/2060606221 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe5bad640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effe015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877464935 0 0) 0x7effe00b2090 con 0x7effe00b24a0 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effc8003530 con 0x7effe00b24a0 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686863408 0 0) 0x7effc8003530 con 0x7effe00b24a0 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 msgr2=0x7effe015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effe015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 >> v1:172.21.15.196:6789/0 conn(0x7effe00b3f50 legacy=0x7effe00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00b46c0 con 0x7effe00b24a0 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7effd8001680 con 0x7effe00b24a0 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 >> v1:172.21.15.82:6789/0 conn(0x7effe00b24a0 legacy=0x7effe006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8003a60 0x7effc8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe015f200 con 0x7effc8004d50 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc800bc00 con 0x7effc8003a60 2024-04-25T10:07:21.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc800e010 con 0x7effe015a700 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe63ae640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe53ac640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.473+0000 7effe5bad640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8003a60 0x7effc8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 -- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 msgr2=0x7effc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe015a700 0x7effc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 -- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8003a60 msgr2=0x7effc8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8003a60 0x7effc8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 -- 172.21.15.82:0/2060606221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc8003530 con 0x7effc8004d50 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe5bad640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effc8003a60 0x7effc8003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effc80051d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7effd800c240 tx=0x7effd801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1fa13244a2852f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8007d20 con 0x7effc8004d50 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80a9070 con 0x7effc8004d50 2024-04-25T10:07:21.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe4bab640 1 -- 172.21.15.82:0/2060606221 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8005d60 con 0x7effc8004d50 2024-04-25T10:07:21.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 -- 172.21.15.82:0/2060606221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 msgr2=0x7effc80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 --2- 172.21.15.82:0/2060606221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effc80051d0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7effd800c240 tx=0x7effd801cc50 comp rx=0 tx=0).stop 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe63ae640 1 -- 172.21.15.82:0/2060606221 reap_dead start 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 -- 172.21.15.82:0/2060606221 shutdown_connections 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 -- 172.21.15.82:0/2060606221 >> 172.21.15.82:0/2060606221 conn(0x7effe00b11a0 msgr2=0x7effe00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 -- 172.21.15.82:0/2060606221 shutdown_connections 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.477+0000 7effe7e11640 1 -- 172.21.15.82:0/2060606221 wait complete. 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 Processor -- start 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- start start 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe00cecc0 0x7effe00cf140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe01f65c0 0x7effe01f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cd310 con 0x7effc8004d50 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe00ce600 con 0x7effe01f65c0 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe00cd490 con 0x7effe00cecc0 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:52624/0 (socket says 172.21.15.82:52624) 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 -- 172.21.15.82:0/2676962038 learned_addr learned my addr 172.21.15.82:0/2676962038 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 -- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe00cecc0 msgr2=0x7effe00cf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe00cecc0 0x7effe00cf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 -- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe01f65c0 msgr2=0x7effe01f6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe01f65c0 0x7effe01f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 -- 172.21.15.82:0/2676962038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe00af2c0 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe5bad640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7effd0010890 tx=0x7effd0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000f040 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd0004220 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe00b0300 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd000e8e0 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe00af820 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe00cf680 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7effd000ea80 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 0x7effcc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7effd0085440 con 0x7effc8004d50 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.481+0000 7effe53ac640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 0x7effcc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.485+0000 7effe53ac640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 0x7effcc069450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7effd4007e30 tx=0x7effd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.489+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7effe00af2c0 con 0x7effc8004d50 2024-04-25T10:07:21.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7effe004efa0 con 0x7effc8004d50 2024-04-25T10:07:21.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effde7fc640 1 -- 172.21.15.82:0/2676962038 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+120 (secure 0 0 0) 0x7effd0018030 con 0x7effc8004d50 2024-04-25T10:07:21.660 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:07:21.660 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:07:21.660 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 msgr2=0x7effcc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 0x7effcc069450 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7effd4007e30 tx=0x7effd4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 msgr2=0x7effe014c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7effd0010890 tx=0x7effd0010d50 comp rx=0 tx=0).stop 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 shutdown_connections 2024-04-25T10:07:21.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7effcc066f90 0x7effcc069450 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7effe00cecc0 0x7effe00cf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7effe01f65c0 0x7effe01f6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 --2- 172.21.15.82:0/2676962038 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7effc8004d50 0x7effe014c560 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 >> 172.21.15.82:0/2676962038 conn(0x7effe00b11a0 msgr2=0x7effe006e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 shutdown_connections 2024-04-25T10:07:21.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:21.657+0000 7effe7e11640 1 -- 172.21.15.82:0/2676962038 wait complete. 2024-04-25T10:07:21.677 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:07:21.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:21.677 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:21.688 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[45904]: fuse finished with error 0 and tester_r 0 2024-04-25T10:07:21.689 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:27.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:07:27.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:27.791 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:27.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:27.799 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T10:07:27.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.973+0000 7f47f6e24640 1 Processor -- start 2024-04-25T10:07:27.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.973+0000 7f47f6e24640 1 -- start start 2024-04-25T10:07:27.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f6e24640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f47f0159f20 0x7f47f006d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f6e24640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00b3bd0 con 0x7f47f00b28a0 2024-04-25T10:07:27.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f6e24640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00b3eb0 con 0x7f47f00b36d0 2024-04-25T10:07:27.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f6e24640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f00b4190 con 0x7f47f0159f20 2024-04-25T10:07:27.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f53c1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f47f00b28a0 0x7f47f00cbbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40560/0 (socket says 172.21.15.82:40560) 2024-04-25T10:07:27.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f53c1640 1 -- 172.21.15.82:0/3939773055 learned_addr learned my addr 172.21.15.82:0/3939773055 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540247819 0 0) 0x7f47f00b3bd0 con 0x7f47f00b28a0 2024-04-25T10:07:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0003450 con 0x7f47f00b28a0 2024-04-25T10:07:27.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3939773055 >> v2:172.21.15.196:3300/0 conn(0x7f47f0159f20 0x7f47f006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 373538604 0 0) 0x7f47e0003450 con 0x7f47f00b28a0 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 >> v2:172.21.15.196:3300/0 conn(0x7f47f0159f20 msgr2=0x7f47f006d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 --2- 172.21.15.82:0/3939773055 >> v2:172.21.15.196:3300/0 conn(0x7f47f0159f20 0x7f47f006d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 >> v1:172.21.15.196:6789/0 conn(0x7f47f00b36d0 legacy=0x7f47f00b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f006cf80 con 0x7f47f00b28a0 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f47e8001a00 con 0x7f47f00b28a0 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 >> v1:172.21.15.82:6789/0 conn(0x7f47f00b28a0 legacy=0x7f47f00cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 0x7f47e0003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0008860 0x7f47e000acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47e000b4d0 0x7f47e000d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00b4190 con 0x7f47e0003ab0 2024-04-25T10:07:27.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000e270 con 0x7f47e000b4d0 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000e470 con 0x7f47e0008860 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 0x7f47e0003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0008860 msgr2=0x7f47e000acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0008860 0x7f47e000acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47e000b4d0 msgr2=0x7f47e000d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47e000b4d0 0x7f47e000d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3939773055 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0003450 con 0x7f47e0003ab0 2024-04-25T10:07:27.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f53c1640 1 -- 172.21.15.82:0/3939773055 reap_dead start 2024-04-25T10:07:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 0x7f47e0003ed0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009ae0 tx=0x7f47e401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57432aa0983ea63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40b2070 con 0x7f47e0003ab0 2024-04-25T10:07:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40a55f0 con 0x7f47e0003ab0 2024-04-25T10:07:27.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.977+0000 7f47ef7fe640 1 -- 172.21.15.82:0/3939773055 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40096b0 con 0x7f47e0003ab0 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 msgr2=0x7f47e0003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 0x7f47e0003ed0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f47e4009ae0 tx=0x7f47e401ca80 comp rx=0 tx=0).stop 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- 172.21.15.82:0/3939773055 shutdown_connections 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 --2- 172.21.15.82:0/3939773055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47e0003ab0 0x7f47e0003ed0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- 172.21.15.82:0/3939773055 >> 172.21.15.82:0/3939773055 conn(0x7f47f006ae60 msgr2=0x7f47f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- 172.21.15.82:0/3939773055 shutdown_connections 2024-04-25T10:07:27.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- 172.21.15.82:0/3939773055 wait complete. 2024-04-25T10:07:27.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 Processor -- start 2024-04-25T10:07:27.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- start start 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0003ab0 0x7f47f01f2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015aa10 0x7f47f015c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00b2520 con 0x7f47f015a590 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00b23a0 con 0x7f47f015aa10 2024-04-25T10:07:27.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f6e24640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00b26a0 con 0x7f47e0003ab0 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39148/0 (socket says 172.21.15.82:39148) 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 -- 172.21.15.82:0/1189153580 learned_addr learned my addr 172.21.15.82:0/1189153580 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0003ab0 0x7f47f01f2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47f53c1640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015aa10 0x7f47f015c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 -- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0003ab0 msgr2=0x7f47f01f2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0003ab0 0x7f47f01f2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 -- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015aa10 msgr2=0x7f47f015c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015aa10 0x7f47f015c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 -- 172.21.15.82:0/1189153580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f01f2590 con 0x7f47f015a590 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.981+0000 7f47effff640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c450 tx=0x7f47dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 -- 172.21.15.82:0/1189153580 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc017070 con 0x7f47f015a590 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 -- 172.21.15.82:0/1189153580 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc0052f0 con 0x7f47f015a590 2024-04-25T10:07:27.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f015c080 con 0x7f47f015a590 2024-04-25T10:07:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 -- 172.21.15.82:0/1189153580 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc003840 con 0x7f47f015a590 2024-04-25T10:07:27.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f015b2c0 con 0x7f47f015a590 2024-04-25T10:07:27.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 -- 172.21.15.82:0/1189153580 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47dc01a070 con 0x7f47f015a590 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 0x7f47d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47ed7fa640 1 -- 172.21.15.82:0/1189153580 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f47dc084580 con 0x7f47f015a590 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 0x7f47d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 Processor -- start 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 -- start start 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 0x7f47d4069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f47e40af000 tx=0x7f47e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6c1e0 0x558a56f21110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f21a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.985+0000 7f47f9f91000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a56d6fdd0 con 0x558a56f6b8c0 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a56cf8010 con 0x558a56f21650 2024-04-25T10:07:27.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a56d6be70 con 0x558a56f6c1e0 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39162/0 (socket says 172.21.15.82:39162) 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3638644726 learned_addr learned my addr 172.21.15.82:0/3638644726 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f21a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6c1e0 msgr2=0x558a56f21110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6c1e0 0x558a56f21110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 msgr2=0x558a56f21a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f21a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 -- 172.21.15.82:0/3638644726 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a56d289c0 con 0x558a56f6b8c0 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f21a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f47e40b0040 tx=0x7f47e4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4a9804595a9d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47da7fc640 1 -- 172.21.15.82:0/3638644726 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40c8070 con 0x558a56f6b8c0 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47da7fc640 1 -- 172.21.15.82:0/3638644726 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40b3040 con 0x558a56f6b8c0 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47da7fc640 1 -- 172.21.15.82:0/3638644726 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40038e0 con 0x558a56f6b8c0 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/3638644726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 msgr2=0x558a56f6bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f47e40b0040 tx=0x7f47e4004740 comp rx=0 tx=0).stop 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/3638644726 shutdown_connections 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6c1e0 0x558a56f21110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f21a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- 172.21.15.82:0/3638644726 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6b8c0 0x558a56f6bca0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/3638644726 >> 172.21.15.82:0/3638644726 conn(0x558a56e09870 msgr2=0x558a56f1f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/3638644726 shutdown_connections 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/3638644726 wait complete. 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- start start 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 Processor -- start 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f25d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56f262a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a56d2a420 con 0x558a56f6c1e0 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a56d6f0e0 con 0x558a56f21650 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a56f2ac20 con 0x558a56f6b8c0 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 10 client.0 _ll_register_callbacks cb 0x558a56f0cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39172/0 (socket says 172.21.15.82:39172) 2024-04-25T10:07:28.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 -- 172.21.15.82:0/2595037629 learned_addr learned my addr 172.21.15.82:0/2595037629 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 -- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 msgr2=0x558a56f262a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56f262a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f25d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 -- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 msgr2=0x558a56f25d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f25d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 -- 172.21.15.82:0/2595037629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a56cf8260 con 0x558a56f6c1e0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f53c1640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cf210 tx=0x7f47e800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1bda1a0ea89e5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8012590 con 0x558a56f6c1e0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f47e8012730 con 0x558a56f6c1e0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.989+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47e40b3040 con 0x558a56f6c1e0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e8020050 con 0x558a56f6c1e0 2024-04-25T10:07:28.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 10 client.7404 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x558a56d2ab30 con 0x558a56f6c1e0 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40b3040 con 0x558a56f6c1e0 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47e8023020 con 0x558a56f6c1e0 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 10 client.7404 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 10 client.7404 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 20 client.7404 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f7a950 0x558a56f7cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 4 client.7404 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a56d0bc80 con 0x558a56f7a950 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f7a950 0x558a56f7cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f7a950 0x558a56f7cd40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa51c312b278ef65 server_cookie=95618b31986ef594 in_seq=0 out_seq=0 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 10 client.7404 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 1 -- 172.21.15.82:0/2595037629 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558a56d0bc80 con 0x558a56f7a950 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 10 client.7404 handle_command_reply: tid=0 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 client.7404 shutdown 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 msgr2=0x558a56f267e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f47f00cf210 tx=0x7f47e800e940 comp rx=0 tx=0).stop 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 shutdown_connections 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56f262a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f7a950 0x558a56f7cd40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f6c1e0 0x558a56f267e0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 --2- 172.21.15.82:0/2595037629 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f21650 0x558a56f25d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47cf7fe640 0 client.7404 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 >> 172.21.15.82:0/2595037629 conn(0x558a56e09870 msgr2=0x558a56f1f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 shutdown_connections 2024-04-25T10:07:28.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 1 -- 172.21.15.82:0/2595037629 wait complete. 2024-04-25T10:07:28.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:27.993+0000 7f47f9f91000 20 client.7404 trim_cache size 0 max 16384 2024-04-25T10:07:28.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 Processor -- start 2024-04-25T10:07:28.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- start start 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fcfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56fd0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01ffc90 con 0x558a56f21650 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01fedf0 con 0x558a56f6c1e0 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01fe1f0 con 0x558a56f6b8c0 2024-04-25T10:07:28.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39184/0 (socket says 172.21.15.82:39184) 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 -- 172.21.15.82:0/553126202 learned_addr learned my addr 172.21.15.82:0/553126202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 -- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 msgr2=0x558a56fcfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fcfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 -- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 msgr2=0x558a56fd0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56fd0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 -- 172.21.15.82:0/553126202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a56df56d0 con 0x558a56f21650 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f47e401cf60 tx=0x7f47e4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e765927a5d5484c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47d9ffb640 1 -- 172.21.15.82:0/553126202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40c8040 con 0x558a56f21650 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47d9ffb640 1 -- 172.21.15.82:0/553126202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40b3030 con 0x558a56f21650 2024-04-25T10:07:28.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47d9ffb640 1 -- 172.21.15.82:0/553126202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e4003ae0 con 0x558a56f21650 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- 172.21.15.82:0/553126202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 msgr2=0x558a56fcf7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f47e401cf60 tx=0x7f47e4009190 comp rx=0 tx=0).stop 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- 172.21.15.82:0/553126202 shutdown_connections 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fcfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56fd0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 --2- 172.21.15.82:0/553126202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fcf7f0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- 172.21.15.82:0/553126202 >> 172.21.15.82:0/553126202 conn(0x558a56e09870 msgr2=0x558a56f21a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- 172.21.15.82:0/553126202 shutdown_connections 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.009+0000 7f47f9f91000 1 -- 172.21.15.82:0/553126202 wait complete. 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- start start 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 Processor -- start 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fd4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56f8efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a56df2470 con 0x558a56f21650 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f01fe7f0 con 0x558a56f6c1e0 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558a56d74400 con 0x558a56f6b8c0 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 10 client.0 _ll_register_callbacks cb 0x558a56d0c200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39188/0 (socket says 172.21.15.82:39188) 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 -- 172.21.15.82:0/641449988 learned_addr learned my addr 172.21.15.82:0/641449988 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 -- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 msgr2=0x558a56fd4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fd4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 -- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 msgr2=0x558a56f8efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56f8efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 -- 172.21.15.82:0/641449988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a56df2900 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f4bc0640 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f47e401cac0 tx=0x7f47e4019820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c2a2ed0ba2bb95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e40c8070 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f47e40038e0 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47e40b3030 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f47e4003ae0 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 10 client.7410 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x558a56d16f20 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e40b3030 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f47e4003c80 con 0x558a56f21650 2024-04-25T10:07:28.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 10 client.7410 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 10 client.7410 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 20 client.7410 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f8f510 0x558a56f8f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 4 client.7410 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x558a56d7d310 con 0x558a56f8f510 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47effff640 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f8f510 0x558a56f8f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 10 client.7410 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47effff640 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f8f510 0x558a56f8f920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9d43b20b19f3fe server_cookie=1ec66b5e6e4cc609 in_seq=0 out_seq=0 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 1 -- 172.21.15.82:0/641449988 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558a56d7d310 con 0x558a56f8f510 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 10 client.7410 handle_command_reply: tid=0 2024-04-25T10:07:28.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 client.7410 shutdown 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 msgr2=0x558a56fd4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f47e401cac0 tx=0x7f47e4019820 comp rx=0 tx=0).stop 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 shutdown_connections 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558a56f6b8c0 0x558a56fd4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x558a56f8f510 0x558a56f8f920 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558a56f6c1e0 0x558a56f8efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 --2- 172.21.15.82:0/641449988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558a56f21650 0x558a56fd4690 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 >> 172.21.15.82:0/641449988 conn(0x558a56e09870 msgr2=0x558a56f222c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47cf7fe640 0 client.7410 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 shutdown_connections 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 1 -- 172.21.15.82:0/641449988 wait complete. 2024-04-25T10:07:28.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.013+0000 7f47f9f91000 20 client.7410 trim_cache size 0 max 16384 2024-04-25T10:07:28.026 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:28.026 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:07:28.026 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "f320bf46-8fba-4e96-ada1-9e36418fe947", 2024-04-25T10:07:28.026 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:07:28.026 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 msgr2=0x7f47d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 0x7f47d4069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f47e40af000 tx=0x7f47e4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 msgr2=0x7f47f015c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f47dc00c450 tx=0x7f47dc00c910 comp rx=0 tx=0).stop 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 shutdown_connections 2024-04-25T10:07:28.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f47d4066f60 0x7f47d4069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f47f015aa10 0x7f47f015c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47f015a590 0x7f47f015c450 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 --2- 172.21.15.82:0/1189153580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f47e0003ab0 0x7f47f01f2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 >> 172.21.15.82:0/1189153580 conn(0x7f47f006ae60 msgr2=0x7f47f01591d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 shutdown_connections 2024-04-25T10:07:28.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.017+0000 7f47f6e24640 1 -- 172.21.15.82:0/1189153580 wait complete. 2024-04-25T10:07:28.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:28.037 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:07:28.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 Processor -- start 2024-04-25T10:07:28.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 -- start start 2024-04-25T10:07:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e4815b6b0 0x7f8e480ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 0x7f8e480cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8e4815f130 0x7f8e4815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4815ba90 con 0x7f8e4815b6b0 2024-04-25T10:07:28.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e480cff30 con 0x7f8e4815f130 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e50586640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480cbd20 con 0x7f8e480cf590 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e4815b6b0 0x7f8e480ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8e4815b6b0 0x7f8e480ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39204/0 (socket says 172.21.15.82:39204) 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 -- 172.21.15.82:0/663656058 learned_addr learned my addr 172.21.15.82:0/663656058 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4db21640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 0x7f8e480cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/663656058 >> v2:172.21.15.196:3300/0 conn(0x7f8e4815f130 0x7f8e4815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 -- 172.21.15.82:0/663656058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 msgr2=0x7f8e480cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 0x7f8e480cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 -- 172.21.15.82:0/663656058 >> v2:172.21.15.196:3300/0 conn(0x7f8e4815f130 msgr2=0x7f8e4815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 --2- 172.21.15.82:0/663656058 >> v2:172.21.15.196:3300/0 conn(0x7f8e4815f130 0x7f8e4815f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4e322640 1 -- 172.21.15.82:0/663656058 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480cc000 con 0x7f8e4815b6b0 2024-04-25T10:07:28.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.149+0000 7f8e4db21640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 0x7f8e480cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e4e322640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480ca3c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8e480c2de0 tx=0x7f8e4001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43917162751179e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e4d320640 1 -- 172.21.15.82:0/663656058 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400aa610 con 0x7f8e4815b6b0 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e4d320640 1 -- 172.21.15.82:0/663656058 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400af070 con 0x7f8e4815b6b0 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e4d320640 1 -- 172.21.15.82:0/663656058 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400aa9d0 con 0x7f8e4815b6b0 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- 172.21.15.82:0/663656058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 msgr2=0x7f8e480ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480ca3c0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f8e480c2de0 tx=0x7f8e4001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- 172.21.15.82:0/663656058 shutdown_connections 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 --2- 172.21.15.82:0/663656058 >> v2:172.21.15.196:3300/0 conn(0x7f8e4815f130 0x7f8e4815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e480cf590 0x7f8e480cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 --2- 172.21.15.82:0/663656058 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480ca3c0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- 172.21.15.82:0/663656058 >> 172.21.15.82:0/663656058 conn(0x7f8e4806ae60 msgr2=0x7f8e4806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- 172.21.15.82:0/663656058 shutdown_connections 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- 172.21.15.82:0/663656058 wait complete. 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 Processor -- start 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.153+0000 7f8e50586640 1 -- start start 2024-04-25T10:07:28.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e480cf590 0x7f8e480c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815f130 0x7f8e4806d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4815fef0 con 0x7f8e4815b6b0 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480cba30 con 0x7f8e480cf590 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480ca900 con 0x7f8e4815f130 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39216/0 (socket says 172.21.15.82:39216) 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 -- 172.21.15.82:0/3903277468 learned_addr learned my addr 172.21.15.82:0/3903277468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 -- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815f130 msgr2=0x7f8e4806d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815f130 0x7f8e4806d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 -- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e480cf590 msgr2=0x7f8e480c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4e322640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e480cf590 0x7f8e480c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e480cf590 0x7f8e480c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 -- 172.21.15.82:0/3903277468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e480c7e50 con 0x7f8e4815b6b0 2024-04-25T10:07:28.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4db21640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00cf60 tx=0x7f8e3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 -- 172.21.15.82:0/3903277468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c01a070 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 -- 172.21.15.82:0/3903277468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c00d840 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e480c8130 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 -- 172.21.15.82:0/3903277468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c00dc10 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e480c86f0 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 -- 172.21.15.82:0/3903277468 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8e3c01d070 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 0x7f8e2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e3affd640 1 -- 172.21.15.82:0/3903277468 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8e3c0858a0 con 0x7f8e4815b6b0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4e322640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 0x7f8e2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.157+0000 7f8e4e322640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 0x7f8e2c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8e400ae000 tx=0x7f8e40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 Processor -- start 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- start start 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c773af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c773b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c7583ff0 con 0x5628c7784e30 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c750c150 con 0x5628c7785750 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628c7580090 con 0x5628c773b4b0 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39218/0 (socket says 172.21.15.82:39218) 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 -- 172.21.15.82:0/2813666725 learned_addr learned my addr 172.21.15.82:0/2813666725 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4db21640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c773af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c773b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 -- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 msgr2=0x5628c773b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c773b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 -- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 msgr2=0x5628c773af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c773af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 -- 172.21.15.82:0/2813666725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628c771a0d0 con 0x5628c7784e30 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4db21640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c773af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8e4001cd90 tx=0x7f8e40004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59569a9da428ede9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e2bfff640 1 -- 172.21.15.82:0/2813666725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b37b0 con 0x5628c7784e30 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e2bfff640 1 -- 172.21.15.82:0/2813666725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400c3070 con 0x5628c7784e30 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e2bfff640 1 -- 172.21.15.82:0/2813666725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b3b50 con 0x5628c7784e30 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- 172.21.15.82:0/2813666725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 msgr2=0x5628c7785210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f8e4001cd90 tx=0x7f8e40004740 comp rx=0 tx=0).stop 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- 172.21.15.82:0/2813666725 shutdown_connections 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c773b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c773af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2813666725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7785210 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- 172.21.15.82:0/2813666725 >> 172.21.15.82:0/2813666725 conn(0x5628c7733020 msgr2=0x5628c7735440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- 172.21.15.82:0/2813666725 shutdown_connections 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- 172.21.15.82:0/2813666725 wait complete. 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- start start 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 Processor -- start 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c77847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c77418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c7588620 con 0x5628c7784e30 2024-04-25T10:07:28.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c753e690 con 0x5628c7785750 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628c750ca40 con 0x5628c773b4b0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e536f3000 10 client.0 _ll_register_callbacks cb 0x5628c7721e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4db21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.161+0000 7f8e4db21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39230/0 (socket says 172.21.15.82:39230) 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 -- 172.21.15.82:0/2203861506 learned_addr learned my addr 172.21.15.82:0/2203861506 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c77418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c77847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 -- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 msgr2=0x5628c77847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c77847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 -- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 msgr2=0x5628c77418a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c77418a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 -- 172.21.15.82:0/2203861506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628c76311c0 con 0x5628c7784e30 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4db21640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00cf30 tx=0x7f8e3c05e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2484cf0d84586b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c082060 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8e3c055560 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e400c3070 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3c072050 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 10 client.7422 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5628c762d170 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e400c3070 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e3c05c140 con 0x5628c7784e30 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 10 client.7422 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 10 client.7422 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 20 client.7422 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77425e0 0x5628c77970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 4 client.7422 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:28.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5628c751fc60 con 0x5628c77425e0 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77425e0 0x5628c77970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e4e322640 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77425e0 0x5628c77970f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=362db80c0e661257 server_cookie=9e7ed145b5c1be5b in_seq=0 out_seq=0 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 10 client.7422 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 1 -- 172.21.15.82:0/2203861506 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5628c751fc60 con 0x5628c77425e0 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 10 client.7422 handle_command_reply: tid=0 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 client.7422 shutdown 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 msgr2=0x5628c7741360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00cf30 tx=0x7f8e3c05e770 comp rx=0 tx=0).stop 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 shutdown_connections 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77425e0 0x5628c77970f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7785750 0x5628c77418a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7784e30 0x5628c7741360 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 --2- 172.21.15.82:0/2203861506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c773b4b0 0x5628c77847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e297fa640 0 client.7422 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 >> 172.21.15.82:0/2203861506 conn(0x5628c7733020 msgr2=0x5628c773c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 shutdown_connections 2024-04-25T10:07:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 1 -- 172.21.15.82:0/2203861506 wait complete. 2024-04-25T10:07:28.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.165+0000 7f8e536f3000 20 client.7422 trim_cache size 0 max 16384 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 Processor -- start 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- start start 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c773b4b0 0x5628c7763ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7784e30 0x5628c77922a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e481fee30 con 0x5628c7785750 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480ced90 con 0x5628c773b4b0 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480ce190 con 0x5628c7784e30 2024-04-25T10:07:28.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39242/0 (socket says 172.21.15.82:39242) 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 -- 172.21.15.82:0/102102521 learned_addr learned my addr 172.21.15.82:0/102102521 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 -- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7784e30 msgr2=0x5628c77922a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7784e30 0x5628c77922a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 -- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c773b4b0 msgr2=0x5628c7763ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c773b4b0 0x5628c7763ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 -- 172.21.15.82:0/102102521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628c75cade0 con 0x5628c7785750 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d7e0 tx=0x7f8e4400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac1693214c05eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e2b7fe640 1 -- 172.21.15.82:0/102102521 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44019030 con 0x5628c7785750 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e2b7fe640 1 -- 172.21.15.82:0/102102521 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400f9b0 con 0x5628c7785750 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e2b7fe640 1 -- 172.21.15.82:0/102102521 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4400fd80 con 0x5628c7785750 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- 172.21.15.82:0/102102521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 msgr2=0x5628c77927e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400d7e0 tx=0x7f8e4400dca0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- 172.21.15.82:0/102102521 shutdown_connections 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7784e30 0x5628c77922a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c7785750 0x5628c77927e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 --2- 172.21.15.82:0/102102521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c773b4b0 0x5628c7763ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- 172.21.15.82:0/102102521 >> 172.21.15.82:0/102102521 conn(0x5628c7733020 msgr2=0x5628c7791b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- 172.21.15.82:0/102102521 shutdown_connections 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- 172.21.15.82:0/102102521 wait complete. 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 -- start start 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.181+0000 7f8e536f3000 1 Processor -- start 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7784e30 0x5628c7797a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7785750 0x5628c7797f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628c751ee00 con 0x5628c773b4b0 2024-04-25T10:07:28.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e480ce790 con 0x5628c7784e30 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628c7583300 con 0x5628c7785750 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 10 client.0 _ll_register_callbacks cb 0x5628c7702040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:39258/0 (socket says 172.21.15.82:39258) 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 -- 172.21.15.82:0/539196331 learned_addr learned my addr 172.21.15.82:0/539196331 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4eb23640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7785750 0x5628c7797f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 -- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7785750 msgr2=0x5628c7797f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7785750 0x5628c7797f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 -- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7784e30 msgr2=0x5628c7797a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7784e30 0x5628c7797a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 -- 172.21.15.82:0/539196331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5628c75258e0 con 0x5628c773b4b0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4e322640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004780 tx=0x7f8e40019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28987d6a44422b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:28.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e400b3a50 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8e400b3bf0 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e4400f9b0 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e400d6050 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 10 client.7428 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5628c75dd8f0 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4400f9b0 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e400d9020 con 0x5628c773b4b0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 10 client.7428 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 10 client.7428 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 20 client.7428 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77eafd0 0x5628c77ed3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 4 client.7428 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5628c7591530 con 0x5628c77eafd0 2024-04-25T10:07:28.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4db21640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77eafd0 0x5628c77ed3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 10 client.7428 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e4db21640 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77eafd0 0x5628c77ed3c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee90edeb1fbd16a server_cookie=7169d59870e68429 in_seq=0 out_seq=0 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 1 -- 172.21.15.82:0/539196331 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5628c7591530 con 0x5628c77eafd0 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 10 client.7428 handle_command_reply: tid=0 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 client.7428 shutdown 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 msgr2=0x5628c77974e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f8e40004780 tx=0x7f8e40019290 comp rx=0 tx=0).stop 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 shutdown_connections 2024-04-25T10:07:28.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628c7785750 0x5628c7797f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5628c77eafd0 0x5628c77ed3c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5628c7784e30 0x5628c7797a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 --2- 172.21.15.82:0/539196331 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628c773b4b0 0x5628c77974e0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e297fa640 0 client.7428 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 >> 172.21.15.82:0/539196331 conn(0x5628c7733020 msgr2=0x5628c7792420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 shutdown_connections 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 1 -- 172.21.15.82:0/539196331 wait complete. 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e536f3000 20 client.7428 trim_cache size 0 max 16384 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:28.202 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: "f320bf46-8fba-4e96-ada1-9e36418fe947": { 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/subdir", 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "f320bf46-8fba-4e96-ada1-9e36418fe947", 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive" 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:28.203 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 msgr2=0x7f8e2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 0x7f8e2c069420 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f8e400ae000 tx=0x7f8e40008040 comp rx=0 tx=0).stop 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 msgr2=0x7f8e480c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c00cf60 tx=0x7f8e3c00cf90 comp rx=0 tx=0).stop 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 shutdown_connections 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8e2c066f60 0x7f8e2c069420 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8e4815f130 0x7f8e4806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8e4815b6b0 0x7f8e480c3dd0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 --2- 172.21.15.82:0/3903277468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8e480cf590 0x7f8e480c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 >> 172.21.15.82:0/3903277468 conn(0x7f8e4806ae60 msgr2=0x7f8e480af690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 shutdown_connections 2024-04-25T10:07:28.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:28.185+0000 7f8e50586640 1 -- 172.21.15.82:0/3903277468 wait complete. 2024-04-25T10:07:28.206 INFO:tasks.cephfs.filesystem:scrub status for tag:f320bf46-8fba-4e96-ada1-9e36418fe947 - {'path': '/subdir', 'tag': 'f320bf46-8fba-4e96-ada1-9e36418fe947', 'options': 'recursive'} 2024-04-25T10:07:37.419 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:37.422 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:58.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:58.207 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 Processor -- start 2024-04-25T10:07:58.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 -- start start 2024-04-25T10:07:58.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44540cf590 0x7f445415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f44540ce170 0x7f44540ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44540ca4c0 con 0x7f44540cf970 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540ca780 con 0x7f44540cf590 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f445b510640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540cd440 con 0x7f44540ce170 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f4458aab640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f44540cf970 0x7f44540c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60794/0 (socket says 172.21.15.82:60794) 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f4458aab640 1 -- 172.21.15.82:0/1107691320 learned_addr learned my addr 172.21.15.82:0/1107691320 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f44592ac640 1 --2- 172.21.15.82:0/1107691320 >> v2:172.21.15.82:3300/0 conn(0x7f44540cf590 0x7f445415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.337+0000 7f4459aad640 1 --2- 172.21.15.82:0/1107691320 >> v2:172.21.15.196:3300/0 conn(0x7f44540ce170 0x7f44540ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f44592ac640 1 -- 172.21.15.82:0/1107691320 >> v2:172.21.15.196:3300/0 conn(0x7f44540ce170 msgr2=0x7f44540ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f44592ac640 1 --2- 172.21.15.82:0/1107691320 >> v2:172.21.15.196:3300/0 conn(0x7f44540ce170 0x7f44540ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f44592ac640 1 -- 172.21.15.82:0/1107691320 >> v1:172.21.15.82:6789/0 conn(0x7f44540cf970 legacy=0x7f44540c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f44592ac640 1 -- 172.21.15.82:0/1107691320 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540cd690 con 0x7f44540cf590 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f4447fff640 1 -- 172.21.15.82:0/1107691320 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466638903 0 0) 0x7f44540ca4c0 con 0x7f44540cf970 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f44592ac640 1 --2- 172.21.15.82:0/1107691320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540cf590 0x7f445415ecb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f44540c9940 tx=0x7f444801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91eea8fccf78b15e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f4447fff640 1 -- 172.21.15.82:0/1107691320 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480b1070 con 0x7f44540cf590 2024-04-25T10:07:58.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f4447fff640 1 -- 172.21.15.82:0/1107691320 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44480ac5f0 con 0x7f44540cf590 2024-04-25T10:07:58.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f4447fff640 1 -- 172.21.15.82:0/1107691320 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480ac9c0 con 0x7f44540cf590 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 -- 172.21.15.82:0/1107691320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540cf590 msgr2=0x7f445415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 --2- 172.21.15.82:0/1107691320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540cf590 0x7f445415ecb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f44540c9940 tx=0x7f444801c480 comp rx=0 tx=0).stop 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 -- 172.21.15.82:0/1107691320 shutdown_connections 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 --2- 172.21.15.82:0/1107691320 >> v2:172.21.15.196:3300/0 conn(0x7f44540ce170 0x7f44540ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 --2- 172.21.15.82:0/1107691320 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540cf590 0x7f445415ecb0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 -- 172.21.15.82:0/1107691320 >> 172.21.15.82:0/1107691320 conn(0x7f445406ae60 msgr2=0x7f445406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 -- 172.21.15.82:0/1107691320 shutdown_connections 2024-04-25T10:07:58.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.341+0000 7f445b510640 1 -- 172.21.15.82:0/1107691320 wait complete. 2024-04-25T10:07:58.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 Processor -- start 2024-04-25T10:07:58.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- start start 2024-04-25T10:07:58.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44540cf590 0x7f44540c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44540cf970 0x7f4454098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ca240 con 0x7f44540ce170 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ca0c0 con 0x7f44540cf590 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540ac170 con 0x7f44540cf970 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43422/0 (socket says 172.21.15.82:43422) 2024-04-25T10:07:58.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 -- 172.21.15.82:0/349539778 learned_addr learned my addr 172.21.15.82:0/349539778 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4458aab640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44540cf590 0x7f44540c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 -- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44540cf970 msgr2=0x7f4454098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44540cf970 0x7f4454098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 -- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44540cf590 msgr2=0x7f44540c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44540cf590 0x7f44540c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 -- 172.21.15.82:0/349539778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540c41c0 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f44592ac640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f444801ab30 tx=0x7f4448019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 -- 172.21.15.82:0/349539778 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480b1040 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 -- 172.21.15.82:0/349539778 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44480bc070 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540c44a0 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 -- 172.21.15.82:0/349539778 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448006e20 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4454071340 con 0x7f44540ce170 2024-04-25T10:07:58.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 -- 172.21.15.82:0/349539778 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44480b5630 con 0x7f44540ce170 2024-04-25T10:07:58.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 0x7f4438069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.345+0000 7f4445ffb640 1 -- 172.21.15.82:0/349539778 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f444811ee10 con 0x7f44540ce170 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 0x7f4438069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 0x7f4438069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44540adde0 tx=0x7f4450009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 Processor -- start 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- start start 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 0x55c71312d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c7130e3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c712f2bff0 con 0x55c71312d750 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c712eb4150 con 0x55c71312ce30 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c712f28090 con 0x55c7130e34b0 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43428/0 (socket says 172.21.15.82:43428) 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 -- 172.21.15.82:0/1931846133 learned_addr learned my addr 172.21.15.82:0/1931846133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f44592ac640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 0x55c71312d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4459aad640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c7130e3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 -- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 msgr2=0x55c7130e3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c7130e3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 -- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 msgr2=0x55c71312d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 0x55c71312d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 -- 172.21.15.82:0/1931846133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7130c20d0 con 0x55c71312d750 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f44592ac640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 0x55c71312d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f4458aab640 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4450007360 tx=0x7f4450010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7f6b2bd0add3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f443effd640 1 -- 172.21.15.82:0/1931846133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445000d9a0 con 0x55c71312d750 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f443effd640 1 -- 172.21.15.82:0/1931846133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445000db40 con 0x55c71312d750 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f443effd640 1 -- 172.21.15.82:0/1931846133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450020450 con 0x55c71312d750 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- 172.21.15.82:0/1931846133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 msgr2=0x55c7130e2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4450007360 tx=0x7f4450010000 comp rx=0 tx=0).stop 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- 172.21.15.82:0/1931846133 shutdown_connections 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c7130e3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312d750 0x55c7130e2f70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 --2- 172.21.15.82:0/1931846133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312ce30 0x55c71312d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- 172.21.15.82:0/1931846133 >> 172.21.15.82:0/1931846133 conn(0x55c7130db020 msgr2=0x55c7130dd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- 172.21.15.82:0/1931846133 shutdown_connections 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- 172.21.15.82:0/1931846133 wait complete. 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 -- start start 2024-04-25T10:07:58.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.349+0000 7f445e67d000 1 Processor -- start 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c71312c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312d750 0x55c7130e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c712f30620 con 0x55c71312ce30 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c712ee6690 con 0x55c71312d750 2024-04-25T10:07:58.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c712eb4a40 con 0x55c7130e34b0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 10 client.0 _ll_register_callbacks cb 0x55c7130c9e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43444/0 (socket says 172.21.15.82:43444) 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 -- 172.21.15.82:0/1123890787 learned_addr learned my addr 172.21.15.82:0/1123890787 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f44592ac640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c71312c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4459aad640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312d750 0x55c7130e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 -- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 msgr2=0x55c71312c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c71312c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 -- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312d750 msgr2=0x55c7130e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312d750 0x55c7130e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 -- 172.21.15.82:0/1123890787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c712fd91c0 con 0x55c71312ce30 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4458aab640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4450001460 tx=0x7f4450012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a4e018ce56fbf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4450022820 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44500229c0 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f445000db40 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4450032050 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 10 client.7443 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55c712fd5170 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f445000db40 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4450035020 con 0x55c71312ce30 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 10 client.7443 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 10 client.7443 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 20 client.7443 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:58.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c7130ea640 0x55c71313f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 4 client.7443 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c712ec7c60 con 0x55c7130ea640 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f44592ac640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c7130ea640 0x55c71313f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 10 client.7443 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f44592ac640 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c7130ea640 0x55c71313f0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa40ace5cf2e26aa server_cookie=65b9530abd12111f in_seq=0 out_seq=0 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 1 -- 172.21.15.82:0/1123890787 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c712ec7c60 con 0x55c7130ea640 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 10 client.7443 handle_command_reply: tid=0 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 client.7443 shutdown 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 msgr2=0x55c7130e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f4450001460 tx=0x7f4450012000 comp rx=0 tx=0).stop 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 shutdown_connections 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c7130ea640 0x55c71313f0f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c71312d750 0x55c7130e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c7130e9360 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f4433fff640 0 client.7443 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 --2- 172.21.15.82:0/1123890787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130e34b0 0x55c71312c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 >> 172.21.15.82:0/1123890787 conn(0x55c7130db020 msgr2=0x55c7130e4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 shutdown_connections 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 1 -- 172.21.15.82:0/1123890787 wait complete. 2024-04-25T10:07:58.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.353+0000 7f445e67d000 20 client.7443 trim_cache size 0 max 16384 2024-04-25T10:07:58.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 Processor -- start 2024-04-25T10:07:58.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 -- start start 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130db020 0x55c71310bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c713141110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541fee60 con 0x55c71312ce30 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454152740 con 0x55c7130e34b0 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540724a0 con 0x55c7130db020 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43446/0 (socket says 172.21.15.82:43446) 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 -- 172.21.15.82:0/3531944776 learned_addr learned my addr 172.21.15.82:0/3531944776 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4458aab640 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c713141110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 -- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130db020 msgr2=0x55c71310bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130db020 0x55c71310bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 -- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 msgr2=0x55c713141110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c713141110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 -- 172.21.15.82:0/3531944776 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c712f72de0 con 0x55c71312ce30 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f4459aad640 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f444c009d00 tx=0x7f444c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10b16a9b9cffbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f443e7fc640 1 -- 172.21.15.82:0/3531944776 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c015680 con 0x55c71312ce30 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f443e7fc640 1 -- 172.21.15.82:0/3531944776 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c01a070 con 0x55c71312ce30 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.369+0000 7f443e7fc640 1 -- 172.21.15.82:0/3531944776 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c015a60 con 0x55c71312ce30 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/3531944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 msgr2=0x55c713141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f444c009d00 tx=0x7f444c007700 comp rx=0 tx=0).stop 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/3531944776 shutdown_connections 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c71312ce30 0x55c713141650 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c713141110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/3531944776 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c7130db020 0x55c71310bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/3531944776 >> 172.21.15.82:0/3531944776 conn(0x55c7130dab00 msgr2=0x55c713139b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/3531944776 shutdown_connections 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/3531944776 wait complete. 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- start start 2024-04-25T10:07:58.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 Processor -- start 2024-04-25T10:07:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c71313f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c71312ce30 0x55c71313fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c712ec6e00 con 0x55c7130db020 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4454072aa0 con 0x55c7130e34b0 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c712f2b300 con 0x55c71312ce30 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 10 client.0 _ll_register_callbacks cb 0x55c7130aa040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43454/0 (socket says 172.21.15.82:43454) 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 -- 172.21.15.82:0/2529152688 learned_addr learned my addr 172.21.15.82:0/2529152688 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 -- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c71312ce30 msgr2=0x55c71313fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c71312ce30 0x55c71313fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 -- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 msgr2=0x55c71313f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c71313f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 -- 172.21.15.82:0/2529152688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c712ecd8e0 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f44592ac640 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4448097e60 tx=0x7f44480ef600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28631410601e839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480ebbb0 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44480f77c0 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f444c01a070 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44480ebd50 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 10 client.7449 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55c712f858f0 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c01a070 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f44480b4050 con 0x55c7130db020 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 10 client.7449 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 10 client.7449 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 20 client.7449 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:58.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c71312d750 0x55c7131404d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 4 client.7449 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c712f39530 con 0x55c71312d750 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4458aab640 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c71312d750 0x55c7131404d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 10 client.7449 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4458aab640 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c71312d750 0x55c7131404d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea7712aeb531be5 server_cookie=90c18a14e19cc78 in_seq=0 out_seq=0 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 1 -- 172.21.15.82:0/2529152688 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c712f39530 con 0x55c71312d750 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 10 client.7449 handle_command_reply: tid=0 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 client.7449 shutdown 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 msgr2=0x55c71313f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4448097e60 tx=0x7f44480ef600 comp rx=0 tx=0).stop 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 shutdown_connections 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c71312ce30 0x55c71313fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55c71312d750 0x55c7131404d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c7130e34b0 0x55c71313f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 --2- 172.21.15.82:0/2529152688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c7130db020 0x55c71313f0f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f4433fff640 0 client.7449 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 >> 172.21.15.82:0/2529152688 conn(0x55c7130dab00 msgr2=0x55c71313a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 shutdown_connections 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 1 -- 172.21.15.82:0/2529152688 wait complete. 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.373+0000 7f445e67d000 20 client.7449 trim_cache size 0 max 16384 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:07:58.385 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 msgr2=0x7f4438069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 0x7f4438069450 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f44540adde0 tx=0x7f4450009040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 msgr2=0x7f445415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f444801ab30 tx=0x7f4448019220 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 shutdown_connections 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4438066f90 0x7f4438069450 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44540cf970 0x7f4454098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44540cf590 0x7f44540c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 --2- 172.21.15.82:0/349539778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44540ce170 0x7f445415ab40 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 >> 172.21.15.82:0/349539778 conn(0x7f445406ae60 msgr2=0x7f445415a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 shutdown_connections 2024-04-25T10:07:58.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.377+0000 7f445b510640 1 -- 172.21.15.82:0/349539778 wait complete. 2024-04-25T10:07:58.396 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:07:58.396 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:58.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.501+0000 7fb7e9f0c640 1 Processor -- start 2024-04-25T10:07:58.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.501+0000 7fb7e9f0c640 1 -- start start 2024-04-25T10:07:58.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e9f0c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb7e40cd220 0x7fb7e40cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e9f0c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7e4099cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e9f0c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e40b1240 con 0x7fb7e40b2740 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e9f0c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e415f360 con 0x7fb7e40cd220 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e9f0c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e415f640 con 0x7fb7e40d0010 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb7e40b2740 0x7fb7e40cf900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60796/0 (socket says 172.21.15.82:60796) 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 -- 172.21.15.82:0/2635322582 learned_addr learned my addr 172.21.15.82:0/2635322582 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e37fe640 1 --2- 172.21.15.82:0/2635322582 >> v2:172.21.15.196:3300/0 conn(0x7fb7e40cd220 0x7fb7e40cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e3fff640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7e4099cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3859339039 0 0) 0x7fb7cc001120 con 0x7fb7e40b2740 2024-04-25T10:07:58.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d0003530 con 0x7fb7e40b2740 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2213555586 0 0) 0x7fb7d0003530 con 0x7fb7e40b2740 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 msgr2=0x7fb7e4099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7e4099cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 >> v2:172.21.15.196:3300/0 conn(0x7fb7e40cd220 msgr2=0x7fb7e40cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 --2- 172.21.15.82:0/2635322582 >> v2:172.21.15.196:3300/0 conn(0x7fb7e40cd220 0x7fb7e40cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e415f920 con 0x7fb7e40b2740 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb7cc001850 con 0x7fb7e40b2740 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 >> v1:172.21.15.82:6789/0 conn(0x7fb7e40b2740 legacy=0x7fb7e40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7d0003a30 0x7fb7d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7d0004810 0x7fb7d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7d00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e415f360 con 0x7fb7d0004810 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e415f640 con 0x7fb7d0003a30 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d000ba80 con 0x7fb7e40d0010 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7d0004810 0x7fb7d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e37fe640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7d0003a30 0x7fb7d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e3fff640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7d00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 -- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 msgr2=0x7fb7d00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e40d0010 0x7fb7d00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 -- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7d0003a30 msgr2=0x7fb7d0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7d0003a30 0x7fb7d0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.505+0000 7fb7e2ffd640 1 -- 172.21.15.82:0/2635322582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d0003530 con 0x7fb7d0004810 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e2ffd640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7d0004810 0x7fb7d0004c90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00c4c0 tx=0x7fb7cc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb04a0148d442429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00c0e0 con 0x7fb7d0004810 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc0a9070 con 0x7fb7d0004810 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e27fc640 1 -- 172.21.15.82:0/2635322582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc005d40 con 0x7fb7d0004810 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/2635322582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7d0004810 msgr2=0x7fb7d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/2635322582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7d0004810 0x7fb7d0004c90 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00c4c0 tx=0x7fb7cc01cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e3fff640 1 -- 172.21.15.82:0/2635322582 reap_dead start 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/2635322582 shutdown_connections 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/2635322582 >> 172.21.15.82:0/2635322582 conn(0x7fb7e40d15b0 msgr2=0x7fb7e40d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/2635322582 shutdown_connections 2024-04-25T10:07:58.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/2635322582 wait complete. 2024-04-25T10:07:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 Processor -- start 2024-04-25T10:07:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.509+0000 7fb7e9f0c640 1 -- start start 2024-04-25T10:07:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e4159820 0x7fb7e40afb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e4159c00 0x7fb7e40b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e409a850 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e409a6d0 con 0x7fb7e4159820 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e409a9d0 con 0x7fb7e4159c00 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43462/0 (socket says 172.21.15.82:43462) 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 -- 172.21.15.82:0/796114327 learned_addr learned my addr 172.21.15.82:0/796114327 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e37fe640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e4159820 0x7fb7e40afb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e2ffd640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e4159c00 0x7fb7e40b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 -- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e4159c00 msgr2=0x7fb7e40b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e4159c00 0x7fb7e40b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 -- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e4159820 msgr2=0x7fb7e40afb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e4159820 0x7fb7e40afb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 -- 172.21.15.82:0/796114327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40b5c00 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e3fff640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40d25f0 tx=0x7fb7dc011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc015870 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7dc01c070 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e40aea70 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc015c50 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e40af030 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e404efa0 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.513+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb7dc020410 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.517+0000 7fb7dbfff640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 0x7fb7c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.517+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb7dc084d90 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.517+0000 7fb7e37fe640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 0x7fb7c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.517+0000 7fb7e37fe640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 0x7fb7c4069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b07e0 tx=0x7fb7d400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.521+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb7dc025020 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.697+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7e40cede0 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.697+0000 7fb7dbfff640 1 -- 172.21.15.82:0/796114327 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb7dc055bc0 con 0x7fb7e40ae4d0 2024-04-25T10:07:58.702 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:58.702 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:07:20.979076+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:07:58.702 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 48 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 msgr2=0x7fb7c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 0x7fb7c4069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40b07e0 tx=0x7fb7d400b040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 msgr2=0x7fb7e40b6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb7e40d25f0 tx=0x7fb7dc011860 comp rx=0 tx=0).stop 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 shutdown_connections 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb7c4066f90 0x7fb7c4069450 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb7e4159c00 0x7fb7e40b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb7e40ae4d0 0x7fb7e40b6020 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 --2- 172.21.15.82:0/796114327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb7e4159820 0x7fb7e40afb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 >> 172.21.15.82:0/796114327 conn(0x7fb7e40d15b0 msgr2=0x7fb7e415c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 shutdown_connections 2024-04-25T10:07:58.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.701+0000 7fb7e9f0c640 1 -- 172.21.15.82:0/796114327 wait complete. 2024-04-25T10:07:58.720 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:07:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 Processor -- start 2024-04-25T10:07:58.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 -- start start 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a500cf590 0x7f7a5015ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500cf970 0x7f7a500c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500ca4c0 con 0x7f7a500c9ef0 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a500ca780 con 0x7f7a500cf590 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a5858b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500cd440 con 0x7f7a500cf970 2024-04-25T10:07:58.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56327640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7a500cf590 0x7f7a5015ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56b28640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7a500c9ef0 0x7f7a500b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60810/0 (socket says 172.21.15.82:60810) 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56b28640 1 -- 172.21.15.82:0/4002083956 learned_addr learned my addr 172.21.15.82:0/4002083956 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56327640 1 -- 172.21.15.82:0/4002083956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500cf970 msgr2=0x7f7a500c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56327640 1 --2- 172.21.15.82:0/4002083956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500cf970 0x7f7a500c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56327640 1 -- 172.21.15.82:0/4002083956 >> v1:172.21.15.82:6789/0 conn(0x7f7a500c9ef0 legacy=0x7f7a500b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a56327640 1 -- 172.21.15.82:0/4002083956 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500cd690 con 0x7f7a500cf590 2024-04-25T10:07:58.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.841+0000 7f7a55325640 1 -- 172.21.15.82:0/4002083956 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758812010 0 0) 0x7f7a500ca4c0 con 0x7f7a500c9ef0 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a56327640 1 --2- 172.21.15.82:0/4002083956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a5015ecb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097c0 tx=0x7f7a4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f3e1f11d2055fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55325640 1 -- 172.21.15.82:0/4002083956 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b0070 con 0x7f7a500cf590 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55325640 1 -- 172.21.15.82:0/4002083956 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0ab600 con 0x7f7a500cf590 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55325640 1 -- 172.21.15.82:0/4002083956 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0ab9d0 con 0x7f7a500cf590 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- 172.21.15.82:0/4002083956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 msgr2=0x7f7a5015ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- 172.21.15.82:0/4002083956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a5015ecb0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0097c0 tx=0x7f7a4c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- 172.21.15.82:0/4002083956 shutdown_connections 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- 172.21.15.82:0/4002083956 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500cf970 0x7f7a500c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- 172.21.15.82:0/4002083956 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a5015ecb0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- 172.21.15.82:0/4002083956 >> 172.21.15.82:0/4002083956 conn(0x7f7a5006ae60 msgr2=0x7f7a5006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- 172.21.15.82:0/4002083956 shutdown_connections 2024-04-25T10:07:58.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- 172.21.15.82:0/4002083956 wait complete. 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 Processor -- start 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- start start 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500c9ef0 0x7f7a50098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a500cf970 0x7f7a500ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50167660 con 0x7f7a500cf590 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501674e0 con 0x7f7a500cf970 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a5858b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501677e0 con 0x7f7a500c9ef0 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43484/0 (socket says 172.21.15.82:43484) 2024-04-25T10:07:58.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 -- 172.21.15.82:0/2444216158 learned_addr learned my addr 172.21.15.82:0/2444216158 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 -- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500c9ef0 msgr2=0x7f7a50098990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500c9ef0 0x7f7a50098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 -- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a500cf970 msgr2=0x7f7a500ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a56b28640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a500cf970 0x7f7a500ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a500cf970 0x7f7a500ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.845+0000 7f7a55b26640 1 -- 172.21.15.82:0/2444216158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50098ed0 con 0x7f7a500cf590 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b410 tx=0x7f7a4000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 -- 172.21.15.82:0/2444216158 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40015020 con 0x7f7a500cf590 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 -- 172.21.15.82:0/2444216158 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40011830 con 0x7f7a500cf590 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a50157020 con 0x7f7a500cf590 2024-04-25T10:07:58.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 -- 172.21.15.82:0/2444216158 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a40011c20 con 0x7f7a500cf590 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a501575e0 con 0x7f7a500cf590 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 -- 172.21.15.82:0/2444216158 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a40018020 con 0x7f7a500cf590 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 0x7f7a34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a56327640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 0x7f7a34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a46ffd640 1 -- 172.21.15.82:0/2444216158 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7a40085120 con 0x7f7a500cf590 2024-04-25T10:07:58.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a56327640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 0x7f7a34069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ad000 tx=0x7f7a4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 Processor -- start 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 -- start start 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b9371c00 0x5653b93273a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9327cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b920eba0 con 0x5653b93712e0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b9174090 con 0x5653b9371c00 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b90fc160 con 0x5653b93278e0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a56327640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a56327640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43488/0 (socket says 172.21.15.82:43488) 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.849+0000 7f7a56327640 1 -- 172.21.15.82:0/2327051039 learned_addr learned my addr 172.21.15.82:0/2327051039 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 -- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 msgr2=0x5653b9327cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9327cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 -- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b9371c00 msgr2=0x5653b93273a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b9371c00 0x5653b93273a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 -- 172.21.15.82:0/2327051039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b921dc20 con 0x5653b93712e0 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c008010 tx=0x7f7a4c0a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0674d945ab8209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a33fff640 1 -- 172.21.15.82:0/2327051039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0c7070 con 0x5653b93712e0 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a33fff640 1 -- 172.21.15.82:0/2327051039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0b1040 con 0x5653b93712e0 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a33fff640 1 -- 172.21.15.82:0/2327051039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c004230 con 0x5653b93712e0 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2327051039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 msgr2=0x5653b93716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c008010 tx=0x7f7a4c0a8000 comp rx=0 tx=0).stop 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2327051039 shutdown_connections 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9327cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b9371c00 0x5653b93273a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2327051039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b93716c0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2327051039 >> 172.21.15.82:0/2327051039 conn(0x5653b9323890 msgr2=0x5653b9325c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2327051039 shutdown_connections 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2327051039 wait complete. 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- start start 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 Processor -- start 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93278e0 0x5653b9378640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9371c00 0x5653b935f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b9170130 con 0x5653b93712e0 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b9138130 con 0x5653b93278e0 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b9232300 con 0x5653b9371c00 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 10 client.0 _ll_register_callbacks cb 0x5653b9312bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43496/0 (socket says 172.21.15.82:43496) 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 -- 172.21.15.82:0/802384719 learned_addr learned my addr 172.21.15.82:0/802384719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 -- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9371c00 msgr2=0x5653b935f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9371c00 0x5653b935f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 -- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93278e0 msgr2=0x5653b9378640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93278e0 0x5653b9378640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 -- 172.21.15.82:0/802384719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653b9151060 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a55b26640 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7a4005da20 tx=0x7f7a40056080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7f4df661b51f6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4006d030 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7a40051eb0 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4c0b1040 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a4006d1d0 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 10 client.7470 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5653b91157c0 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0b1040 con 0x5653b93712e0 2024-04-25T10:07:58.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7a40056b70 con 0x5653b93712e0 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 10 client.7470 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 10 client.7470 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 20 client.7470 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b935fe30 0x5653b9360210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 4 client.7470 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5653b910fcd0 con 0x5653b935fe30 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.853+0000 7f7a56327640 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b935fe30 0x5653b9360210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a317fa640 10 client.7470 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a56327640 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b935fe30 0x5653b9360210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52194f4aaee7317 server_cookie=fe15f0d9ac9b03c8 in_seq=0 out_seq=0 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a317fa640 1 -- 172.21.15.82:0/802384719 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5653b910fcd0 con 0x5653b935fe30 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a317fa640 10 client.7470 handle_command_reply: tid=0 2024-04-25T10:07:58.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 client.7470 shutdown 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 msgr2=0x5653b9378b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7a4005da20 tx=0x7f7a40056080 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 shutdown_connections 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9371c00 0x5653b935f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b935fe30 0x5653b9360210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93712e0 0x5653b9378b80 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/802384719 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93278e0 0x5653b9378640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a317fa640 0 client.7470 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 >> 172.21.15.82:0/802384719 conn(0x5653b9323890 msgr2=0x5653b9326510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 shutdown_connections 2024-04-25T10:07:58.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/802384719 wait complete. 2024-04-25T10:07:58.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.857+0000 7f7a5b6f8000 20 client.7470 trim_cache size 0 max 16384 2024-04-25T10:07:58.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 Processor -- start 2024-04-25T10:07:58.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- start start 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9325c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b93261c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a50200170 con 0x5653b9323890 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500b18f0 con 0x5653b93712e0 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a501ffba0 con 0x5653b93278e0 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43508/0 (socket says 172.21.15.82:43508) 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 -- 172.21.15.82:0/3319777129 learned_addr learned my addr 172.21.15.82:0/3319777129 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 -- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 msgr2=0x5653b9325c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9325c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 -- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 msgr2=0x5653b93261c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b93261c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 -- 172.21.15.82:0/3319777129 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653b912ffc0 con 0x5653b9323890 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a56327640 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c01c980 tx=0x7f7a4c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963326a1178ff326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a337fe640 1 -- 172.21.15.82:0/3319777129 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0c7040 con 0x5653b9323890 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a337fe640 1 -- 172.21.15.82:0/3319777129 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0b1070 con 0x5653b9323890 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a337fe640 1 -- 172.21.15.82:0/3319777129 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c003620 con 0x5653b9323890 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/3319777129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 msgr2=0x5653b9377c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c01c980 tx=0x7f7a4c004740 comp rx=0 tx=0).stop 2024-04-25T10:07:58.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/3319777129 shutdown_connections 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b93278e0 0x5653b9325c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b93261c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/3319777129 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b9323890 0x5653b9377c40 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/3319777129 >> 172.21.15.82:0/3319777129 conn(0x5653b932cb90 msgr2=0x5653b937e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/3319777129 shutdown_connections 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/3319777129 wait complete. 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- start start 2024-04-25T10:07:58.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 Processor -- start 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9323890 0x5653b93d58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b9390d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b910ee70 con 0x5653b93278e0 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653b90fca50 con 0x5653b93712e0 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653b91733a0 con 0x5653b9323890 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a5b6f8000 10 client.0 _ll_register_callbacks cb 0x5653b9120240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a55b26640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43518/0 (socket says 172.21.15.82:43518) 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.869+0000 7f7a55b26640 1 -- 172.21.15.82:0/2424304627 learned_addr learned my addr 172.21.15.82:0/2424304627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 -- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9323890 msgr2=0x5653b93d58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a56327640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9323890 0x5653b93d58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9323890 0x5653b93d58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a56b28640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b9390d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 -- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 msgr2=0x5653b9390d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b9390d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 -- 172.21.15.82:0/2424304627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653b91cd9e0 con 0x5653b93278e0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a55b26640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7a40011fd0 tx=0x7f7a40009200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f209bf5cf3c1b24a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4006d030 con 0x5653b93278e0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7a4006ee60 con 0x5653b93278e0 2024-04-25T10:07:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7a4c0b1070 con 0x5653b93278e0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7a40054c90 con 0x5653b93278e0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 10 client.7476 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5653b9113630 con 0x5653b93278e0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0b1070 con 0x5653b93278e0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7a40051590 con 0x5653b93278e0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 10 client.7476 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 10 client.7476 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 20 client.7476 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b9371c00 0x5653b93917f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 4 client.7476 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5653b91815d0 con 0x5653b9371c00 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a56327640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b9371c00 0x5653b93917f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a56327640 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b9371c00 0x5653b93917f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b067ac5c0b2da8bb server_cookie=9e30e69240220ba8 in_seq=0 out_seq=0 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 10 client.7476 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 1 -- 172.21.15.82:0/2424304627 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+237 (crc 0 0 0) 0x5653b91815d0 con 0x5653b9371c00 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 10 client.7476 handle_command_reply: tid=0 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 client.7476 shutdown 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 msgr2=0x5653b93d5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7a40011fd0 tx=0x7f7a40009200 comp rx=0 tx=0).stop 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 shutdown_connections 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5653b9371c00 0x5653b93917f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5653b93712e0 0x5653b9390d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5653b93278e0 0x5653b93d5de0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a317fa640 0 client.7476 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 --2- 172.21.15.82:0/2424304627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5653b9323890 0x5653b93d58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 >> 172.21.15.82:0/2424304627 conn(0x5653b932cb90 msgr2=0x5653b937ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 shutdown_connections 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 1 -- 172.21.15.82:0/2424304627 wait complete. 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5b6f8000 20 client.7476 trim_cache size 0 max 16384 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "dentry", 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 2774689891, 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627776, 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "frag": "*", 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "dname": "file_to_be_damaged", 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "snap_id": "head", 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:58.886 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 msgr2=0x7f7a34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 0x7f7a34069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c0ad000 tx=0x7f7a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 msgr2=0x7f7a50156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b410 tx=0x7f7a4000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 shutdown_connections 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a34066f60 0x7f7a34069420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a500cf970 0x7f7a500ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a500cf590 0x7f7a50156640 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 --2- 172.21.15.82:0/2444216158 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a500c9ef0 0x7f7a50098990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:58.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 >> 172.21.15.82:0/2444216158 conn(0x7f7a5006ae60 msgr2=0x7f7a5015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:58.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 shutdown_connections 2024-04-25T10:07:58.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.873+0000 7f7a5858b640 1 -- 172.21.15.82:0/2444216158 wait complete. 2024-04-25T10:07:58.892 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:07:59.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 Processor -- start 2024-04-25T10:07:59.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 -- start start 2024-04-25T10:07:59.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c0cf560 0x7f576c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f576c06e680 0x7f576c0ad300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f576c15bbb0 con 0x7f576c070fa0 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576c0ace70 con 0x7f576c06e680 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:58.997+0000 7f577245c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c0ac280 con 0x7f576c0cf560 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576b7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f576c070fa0 0x7f576c0cf940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60826/0 (socket says 172.21.15.82:60826) 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576b7fe640 1 -- 172.21.15.82:0/160574572 learned_addr learned my addr 172.21.15.82:0/160574572 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 --2- 172.21.15.82:0/160574572 >> v2:172.21.15.82:3300/0 conn(0x7f576c06e680 0x7f576c0ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576bfff640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c0cf560 0x7f576c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576affd640 1 -- 172.21.15.82:0/160574572 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807407215 0 0) 0x7f576c15bbb0 con 0x7f576c070fa0 2024-04-25T10:07:59.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576affd640 1 -- 172.21.15.82:0/160574572 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f574c003530 con 0x7f576c070fa0 2024-04-25T10:07:59.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 -- 172.21.15.82:0/160574572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c0cf560 msgr2=0x7f576c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c0cf560 0x7f576c0b22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 -- 172.21.15.82:0/160574572 >> v1:172.21.15.82:6789/0 conn(0x7f576c070fa0 legacy=0x7f576c0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 -- 172.21.15.82:0/160574572 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c0ac540 con 0x7f576c06e680 2024-04-25T10:07:59.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f57709f9640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c06e680 0x7f576c0ad300 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f575401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2389da51d0944a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576affd640 1 -- 172.21.15.82:0/160574572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540b2070 con 0x7f576c06e680 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576affd640 1 -- 172.21.15.82:0/160574572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c0ae050 con 0x7f576c06e680 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f576affd640 1 -- 172.21.15.82:0/160574572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540ad860 con 0x7f576c06e680 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 -- 172.21.15.82:0/160574572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c06e680 msgr2=0x7f576c0ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c06e680 0x7f576c0ad300 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f57540097c0 tx=0x7f575401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 -- 172.21.15.82:0/160574572 shutdown_connections 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c06e680 0x7f576c0ad300 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 --2- 172.21.15.82:0/160574572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c0cf560 0x7f576c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 -- 172.21.15.82:0/160574572 >> 172.21.15.82:0/160574572 conn(0x7f576c06ae60 msgr2=0x7f576c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 -- 172.21.15.82:0/160574572 shutdown_connections 2024-04-25T10:07:59.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.001+0000 7f577245c640 1 -- 172.21.15.82:0/160574572 wait complete. 2024-04-25T10:07:59.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 Processor -- start 2024-04-25T10:07:59.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- start start 2024-04-25T10:07:59.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 0x7f576c15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 0x7f576c1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c073310 con 0x7f576c0cf560 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c073190 con 0x7f576c06e680 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f576c1682f0 con 0x7f576c070fa0 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43524/0 (socket says 172.21.15.82:43524) 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 -- 172.21.15.82:0/980936697 learned_addr learned my addr 172.21.15.82:0/980936697 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f576b7fe640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 0x7f576c1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f576bfff640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 0x7f576c15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 -- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 msgr2=0x7f576c1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 0x7f576c1531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 -- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 msgr2=0x7f576c15d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 0x7f576c15d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 -- 172.21.15.82:0/980936697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f576c15c960 con 0x7f576c0cf560 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f576b7fe640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 0x7f576c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f576bfff640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 0x7f576c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f57709f9640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f575401a0d0 tx=0x7f5754019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540ba020 con 0x7f576c0cf560 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57540adcc0 con 0x7f576c0cf560 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c15cc00 con 0x7f576c0cf560 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57540b2040 con 0x7f576c0cf560 2024-04-25T10:07:59.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c15d180 con 0x7f576c0cf560 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.005+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c04efa0 con 0x7f576c0cf560 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.009+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57540bd020 con 0x7f576c0cf560 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.009+0000 7f5768ff9640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.009+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f575411eee0 con 0x7f576c0cf560 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.009+0000 7f576bfff640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.009+0000 7f576bfff640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 0x7f5750069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f575c007520 tx=0x7f575c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.013+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57540c4030 con 0x7f576c0cf560 2024-04-25T10:07:59.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f576c071930 con 0x7f576c0cf560 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f5768ff9640 1 -- 172.21.15.82:0/980936697 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5754016090 con 0x7f576c0cf560 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:07:59.204 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 msgr2=0x7f5750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 0x7f5750069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f575c007520 tx=0x7f575c00b040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 msgr2=0x7f576c0b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f575401a0d0 tx=0x7f5754019240 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 shutdown_connections 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5750066f60 0x7f5750069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f576c070fa0 0x7f576c1531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f576c0cf560 0x7f576c0b1fc0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 --2- 172.21.15.82:0/980936697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f576c06e680 0x7f576c15d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 >> 172.21.15.82:0/980936697 conn(0x7f576c06ae60 msgr2=0x7f576c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 shutdown_connections 2024-04-25T10:07:59.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.201+0000 7f577245c640 1 -- 172.21.15.82:0/980936697 wait complete. 2024-04-25T10:07:59.222 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:07:59.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:59.222 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 Processor -- start 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 -- start start 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce340ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce340b0a50 con 0x7fce3415e7e0 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3415f420 con 0x7fce340b1040 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce39384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce340b1a60 con 0x7fce3415e0c0 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce32ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fce3415e7e0 0x7fce3415ebc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60836/0 (socket says 172.21.15.82:60836) 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.333+0000 7fce32ffd640 1 -- 172.21.15.82:0/149984356 learned_addr learned my addr 172.21.15.82:0/149984356 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:07:59.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce340ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900231928 0 0) 0x7fce340b0a50 con 0x7fce3415e7e0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce20003530 con 0x7fce3415e7e0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2720392891 0 0) 0x7fce20003530 con 0x7fce3415e7e0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 msgr2=0x7fce340ceed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce340ceed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 >> v1:172.21.15.196:6789/0 conn(0x7fce340b1040 legacy=0x7fce3415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340b0cf0 con 0x7fce3415e7e0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fce24001680 con 0x7fce3415e7e0 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 >> v1:172.21.15.82:6789/0 conn(0x7fce3415e7e0 legacy=0x7fce3415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce200039f0 0x7fce20003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce20004d50 0x7fce200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce340b1a60 con 0x7fce20004d50 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2000bb90 con 0x7fce200039f0 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2000bd90 con 0x7fce3415e0c0 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce20004d50 0x7fce200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 -- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 msgr2=0x7fce20004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce20004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 -- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce200039f0 msgr2=0x7fce20003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce200039f0 0x7fce20003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 -- 172.21.15.82:0/149984356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce20003530 con 0x7fce20004d50 2024-04-25T10:07:59.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce3415e0c0 0x7fce20004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce20004d50 0x7fce200051d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fc0 tx=0x7fce2c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b9668549105523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0b2070 con 0x7fce20004d50 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340cf2e0 con 0x7fce20004d50 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce327fc640 1 -- 172.21.15.82:0/149984356 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c009c10 con 0x7fce20004d50 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 -- 172.21.15.82:0/149984356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce20004d50 msgr2=0x7fce200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 --2- 172.21.15.82:0/149984356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce20004d50 0x7fce200051d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fc0 tx=0x7fce2c01cc30 comp rx=0 tx=0).stop 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce337fe640 1 -- 172.21.15.82:0/149984356 reap_dead start 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 -- 172.21.15.82:0/149984356 shutdown_connections 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 -- 172.21.15.82:0/149984356 >> 172.21.15.82:0/149984356 conn(0x7fce340b1fa0 msgr2=0x7fce340b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 -- 172.21.15.82:0/149984356 shutdown_connections 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.337+0000 7fce39384640 1 -- 172.21.15.82:0/149984356 wait complete. 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 Processor -- start 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- start start 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce20004d50 0x7fce340b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce340b4180 0x7fce340b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34166450 con 0x7fce341561b0 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce341662d0 con 0x7fce20004d50 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce341665d0 con 0x7fce340b4180 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43536/0 (socket says 172.21.15.82:43536) 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3949537392 learned_addr learned my addr 172.21.15.82:0/3949537392 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce20004d50 0x7fce340b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce337fe640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce340b4180 0x7fce340b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce340b4180 msgr2=0x7fce340b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce340b4180 0x7fce340b4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce20004d50 msgr2=0x7fce340b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce20004d50 0x7fce340b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3949537392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34151ee0 con 0x7fce341561b0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00d7d0 tx=0x7fce1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2bfff640 1 -- 172.21.15.82:0/3949537392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c005270 con 0x7fce341561b0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2bfff640 1 -- 172.21.15.82:0/3949537392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c017070 con 0x7fce341561b0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce34156920 con 0x7fce341561b0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce2bfff640 1 -- 172.21.15.82:0/3949537392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c00b650 con 0x7fce341561b0 2024-04-25T10:07:59.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.341+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce34156ee0 con 0x7fce341561b0 2024-04-25T10:07:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce2bfff640 1 -- 172.21.15.82:0/3949537392 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce1c00b7f0 con 0x7fce341561b0 2024-04-25T10:07:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce2bfff640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 0x7fce14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce2bfff640 1 -- 172.21.15.82:0/3949537392 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce1c085080 con 0x7fce341561b0 2024-04-25T10:07:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 0x7fce14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 0x7fce14069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce24001870 tx=0x7fce24001640 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 Processor -- start 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- start start 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d690b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66bb7c40 con 0x555e66db2d40 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66b401d0 con 0x555e66d695f0 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e66bb3ce0 con 0x555e66db3660 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43540/0 (socket says 172.21.15.82:43540) 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 -- 172.21.15.82:0/1129698417 learned_addr learned my addr 172.21.15.82:0/1129698417 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce337fe640 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 -- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 msgr2=0x555e66d690b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d690b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 -- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 msgr2=0x555e66d699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 -- 172.21.15.82:0/1129698417 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e66bfd2a0 con 0x555e66db2d40 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce32ffd640 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fce2400eaf0 tx=0x7fce24011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a2098b95b9cfc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce28ff9640 1 -- 172.21.15.82:0/1129698417 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24004e30 con 0x555e66db2d40 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce28ff9640 1 -- 172.21.15.82:0/1129698417 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24005260 con 0x555e66db2d40 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce28ff9640 1 -- 172.21.15.82:0/1129698417 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce24005610 con 0x555e66db2d40 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1129698417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 msgr2=0x555e66db3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fce2400eaf0 tx=0x7fce24011000 comp rx=0 tx=0).stop 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1129698417 shutdown_connections 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d690b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1129698417 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66db3120 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1129698417 >> 172.21.15.82:0/1129698417 conn(0x555e66d655a0 msgr2=0x555e66d67990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1129698417 shutdown_connections 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.345+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1129698417 wait complete. 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- start start 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 Processor -- start 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d97d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66b72a90 con 0x555e66db2d40 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66bb6c10 con 0x555e66d695f0 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e66b7b910 con 0x555e66db3660 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 10 client.0 _ll_register_callbacks cb 0x555e66d54410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:59.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43544/0 (socket says 172.21.15.82:43544) 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3971327259 learned_addr learned my addr 172.21.15.82:0/3971327259 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce337fe640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d97d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 msgr2=0x555e66d97d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d97d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 msgr2=0x555e66d6e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d6e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 -- 172.21.15.82:0/3971327259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e66b70730 con 0x555e66db2d40 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce337fe640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fce1c001ec0 tx=0x7fce1c055ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a382a18a1c6f04cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c072060 con 0x555e66db2d40 2024-04-25T10:07:59.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce1c06d210 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce24005260 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce1c081050 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 10 client.7497 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x555e66b40420 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce24005260 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce1c054c00 con 0x555e66db2d40 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 10 client.7497 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 10 client.7497 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 20 client.7497 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66d98780 0x555e66dc5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 4 client.7497 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e66b53b50 con 0x555e66d98780 2024-04-25T10:07:59.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66d98780 0x555e66dc5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 10 client.7497 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce32ffd640 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66d98780 0x555e66dc5000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be64f51132999c0c server_cookie=18a7be2e76cf1fe2 in_seq=0 out_seq=0 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 1 -- 172.21.15.82:0/3971327259 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e66b53b50 con 0x555e66d98780 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 10 client.7497 handle_command_reply: tid=0 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 client.7497 shutdown 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 msgr2=0x555e66d6ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fce1c001ec0 tx=0x7fce1c055ff0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 shutdown_connections 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66d97d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66d98780 0x555e66dc5000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66d6ef20 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce11ffb640 0 client.7497 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/3971327259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66d695f0 0x555e66d6e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 >> 172.21.15.82:0/3971327259 conn(0x555e66d655a0 msgr2=0x555e66d68220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 shutdown_connections 2024-04-25T10:07:59.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 1 -- 172.21.15.82:0/3971327259 wait complete. 2024-04-25T10:07:59.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.349+0000 7fce3c4f1000 20 client.7497 trim_cache size 0 max 16384 2024-04-25T10:07:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 Processor -- start 2024-04-25T10:07:59.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- start start 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66d695f0 0x555e66d68fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db3660 0x555e66dc06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34201b80 con 0x555e66db2d40 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce34200db0 con 0x555e66db3660 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce342001b0 con 0x555e66d695f0 2024-04-25T10:07:59.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43550/0 (socket says 172.21.15.82:43550) 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 -- 172.21.15.82:0/1598873171 learned_addr learned my addr 172.21.15.82:0/1598873171 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 -- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66d695f0 msgr2=0x555e66d68fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66d695f0 0x555e66d68fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 -- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db3660 msgr2=0x555e66dc06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db3660 0x555e66dc06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 -- 172.21.15.82:0/1598873171 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e66c3c550 con 0x555e66db2d40 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0554b0 tx=0x7fce1c051b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c277db9b8eba73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce13fff640 1 -- 172.21.15.82:0/1598873171 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c06d070 con 0x555e66db2d40 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce13fff640 1 -- 172.21.15.82:0/1598873171 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c06d210 con 0x555e66db2d40 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce13fff640 1 -- 172.21.15.82:0/1598873171 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c055110 con 0x555e66db2d40 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1598873171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 msgr2=0x555e66dc01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0554b0 tx=0x7fce1c051b80 comp rx=0 tx=0).stop 2024-04-25T10:07:59.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1598873171 shutdown_connections 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db3660 0x555e66dc06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66db2d40 0x555e66dc01b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/1598873171 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66d695f0 0x555e66d68fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1598873171 >> 172.21.15.82:0/1598873171 conn(0x555e66d655a0 msgr2=0x555e66dbfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1598873171 shutdown_connections 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.365+0000 7fce3c4f1000 1 -- 172.21.15.82:0/1598873171 wait complete. 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- start start 2024-04-25T10:07:59.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 Processor -- start 2024-04-25T10:07:59.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db2d40 0x555e66e112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66e11800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66b50850 con 0x555e66d695f0 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce342007b0 con 0x555e66db2d40 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e66b72a90 con 0x555e66db3660 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 10 client.0 _ll_register_callbacks cb 0x7fce24005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43562/0 (socket says 172.21.15.82:43562) 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 -- 172.21.15.82:0/2123469475 learned_addr learned my addr 172.21.15.82:0/2123469475 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 -- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 msgr2=0x555e66e11800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66e11800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 -- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db2d40 msgr2=0x555e66e112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db2d40 0x555e66e112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 -- 172.21.15.82:0/2123469475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e66b6a490 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce32ffd640 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fce24017000 tx=0x7fce340b2ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c50f8c89ee2fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2402a470 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce24021450 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce1c06d210 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce24021650 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 10 client.7503 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x555e66b5f1b0 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c06d210 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce240217f0 con 0x555e66d695f0 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 10 client.7503 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 10 client.7503 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 20 client.7503 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:59.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66e11d40 0x555e66e1c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 4 client.7503 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x555e66bc5180 con 0x555e66e11d40 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66e11d40 0x555e66e1c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 10 client.7503 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce2a7fc640 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66e11d40 0x555e66e1c6c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74651b8662518804 server_cookie=8031f993212aac88 in_seq=0 out_seq=0 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 1 -- 172.21.15.82:0/2123469475 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555e66bc5180 con 0x555e66e11d40 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 10 client.7503 handle_command_reply: tid=0 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 client.7503 shutdown 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 msgr2=0x555e66e10d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fce24017000 tx=0x7fce340b2ff0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 shutdown_connections 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x555e66db3660 0x555e66e11800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x555e66e11d40 0x555e66e1c6c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x555e66db2d40 0x555e66e112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 --2- 172.21.15.82:0/2123469475 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x555e66d695f0 0x555e66e10d80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce11ffb640 0 client.7503 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 >> 172.21.15.82:0/2123469475 conn(0x555e66d655a0 msgr2=0x555e66dc0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 shutdown_connections 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 1 -- 172.21.15.82:0/2123469475 wait complete. 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.369+0000 7fce3c4f1000 20 client.7503 trim_cache size 0 max 16384 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:07:59.380 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "fcd14b4f-3dbc-4c0f-8d87-360cbdd21fe7", 2024-04-25T10:07:59.381 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:07:59.381 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:59.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 msgr2=0x7fce14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 0x7fce14069420 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fce24001870 tx=0x7fce24001640 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 msgr2=0x7fce341519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fce1c00d7d0 tx=0x7fce1c00dc90 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 shutdown_connections 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fce14066f60 0x7fce14069420 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fce340b4180 0x7fce340b4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fce341561b0 0x7fce341519a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 --2- 172.21.15.82:0/3949537392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fce20004d50 0x7fce340b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 >> 172.21.15.82:0/3949537392 conn(0x7fce340b1fa0 msgr2=0x7fce3415ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 shutdown_connections 2024-04-25T10:07:59.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.373+0000 7fce39384640 1 -- 172.21.15.82:0/3949537392 wait complete. 2024-04-25T10:07:59.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:59.391 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.489+0000 7f16df4e3640 1 Processor -- start 2024-04-25T10:07:59.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.489+0000 7f16df4e3640 1 -- start start 2024-04-25T10:07:59.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16df4e3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f16d80cc820 0x7f16d80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16df4e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16df4e3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d8160310 con 0x7f16d80b0b00 2024-04-25T10:07:59.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16df4e3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d815cfa0 con 0x7f16d80cc820 2024-04-25T10:07:59.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16df4e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815cdf0 con 0x7f16d80b1910 2024-04-25T10:07:59.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dca7e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f16d80cc820 0x7f16d80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34976/0 (socket says 172.21.15.82:34976) 2024-04-25T10:07:59.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dd27f640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f16d80b0b00 0x7f16d80cc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:59696/0 (socket says 172.21.15.82:59696) 2024-04-25T10:07:59.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 -- 172.21.15.82:0/2848817312 learned_addr learned my addr 172.21.15.82:0/2848817312 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16d7fff640 1 -- 172.21.15.82:0/2848817312 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1008416242 0 0) 0x7f16d8160310 con 0x7f16d80b0b00 2024-04-25T10:07:59.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 -- 172.21.15.82:0/2848817312 >> v2:172.21.15.196:3300/0 conn(0x7f16d80cc820 msgr2=0x7f16d80b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 --2- 172.21.15.82:0/2848817312 >> v2:172.21.15.196:3300/0 conn(0x7f16d80cc820 0x7f16d80b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 -- 172.21.15.82:0/2848817312 >> v1:172.21.15.196:6789/0 conn(0x7f16d80b0b00 legacy=0x7f16d80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.493+0000 7f16dda80640 1 -- 172.21.15.82:0/2848817312 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d815f550 con 0x7f16d80b1910 2024-04-25T10:07:59.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dda80640 1 --2- 172.21.15.82:0/2848817312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009e50 tx=0x7f16c801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0362ef609cdcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16d7fff640 1 -- 172.21.15.82:0/2848817312 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80a5510 con 0x7f16d80b1910 2024-04-25T10:07:59.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16d7fff640 1 -- 172.21.15.82:0/2848817312 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c80a56b0 con 0x7f16d80b1910 2024-04-25T10:07:59.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16d7fff640 1 -- 172.21.15.82:0/2848817312 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80a5a80 con 0x7f16d80b1910 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- 172.21.15.82:0/2848817312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 msgr2=0x7f16d8151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- 172.21.15.82:0/2848817312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009e50 tx=0x7f16c801c850 comp rx=0 tx=0).stop 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- 172.21.15.82:0/2848817312 shutdown_connections 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- 172.21.15.82:0/2848817312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80b1910 0x7f16d8151fd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- 172.21.15.82:0/2848817312 >> v2:172.21.15.196:3300/0 conn(0x7f16d80cc820 0x7f16d80b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- 172.21.15.82:0/2848817312 >> 172.21.15.82:0/2848817312 conn(0x7f16d80b1f10 msgr2=0x7f16d80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- 172.21.15.82:0/2848817312 shutdown_connections 2024-04-25T10:07:59.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- 172.21.15.82:0/2848817312 wait complete. 2024-04-25T10:07:59.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 Processor -- start 2024-04-25T10:07:59.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- start start 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80b1910 0x7f16d80aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 0x7f16d80af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b1060 con 0x7f16d80b0b00 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b0ee0 con 0x7f16d80b1910 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16df4e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815bde0 con 0x7f16d80ae900 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43566/0 (socket says 172.21.15.82:43566) 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 -- 172.21.15.82:0/3525593778 learned_addr learned my addr 172.21.15.82:0/3525593778 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dda80640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 0x7f16d80af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80b1910 0x7f16d80aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 -- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 msgr2=0x7f16d80af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 0x7f16d80af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 -- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80b1910 msgr2=0x7f16d80aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80b1910 0x7f16d80aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 -- 172.21.15.82:0/3525593778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80c82c0 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dda80640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 0x7f16d80af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.497+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f16d00011f0 tx=0x7f16d000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 -- 172.21.15.82:0/3525593778 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0018070 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 -- 172.21.15.82:0/3525593778 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00074e0 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d80c85a0 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 -- 172.21.15.82:0/3525593778 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0004a10 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d80c8b60 con 0x7f16d80b0b00 2024-04-25T10:07:59.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 -- 172.21.15.82:0/3525593778 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16d001b070 con 0x7f16d80b0b00 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 0x7f16c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16d5ffb640 1 -- 172.21.15.82:0/3525593778 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f16d00844c0 con 0x7f16d80b0b00 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 0x7f16c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 0x7f16c4069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16d80c9300 tx=0x7f16cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 Processor -- start 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 -- start start 2024-04-25T10:07:59.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f54db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f0ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629f7d57ff0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629f7ce0150 con 0x5629f7f552f0 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629f7d54090 con 0x5629f7f549d0 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43570/0 (socket says 172.21.15.82:43570) 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 -- 172.21.15.82:0/3820062175 learned_addr learned my addr 172.21.15.82:0/3820062175 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f54db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 -- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 msgr2=0x5629f7f54db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f54db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 -- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 msgr2=0x5629f7f0ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f0ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.501+0000 7f16dda80640 1 -- 172.21.15.82:0/3820062175 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629f7eee0d0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dda80640 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f16c80af040 tx=0x7f16c8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4af88a90daec261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16beffd640 1 -- 172.21.15.82:0/3820062175 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80be070 con 0x5629f7f0b0a0 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16beffd640 1 -- 172.21.15.82:0/3820062175 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c80a5510 con 0x5629f7f0b0a0 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16beffd640 1 -- 172.21.15.82:0/3820062175 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80a58e0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/3820062175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 msgr2=0x5629f7f0b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f16c80af040 tx=0x7f16c8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/3820062175 shutdown_connections 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f0b480 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f0ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- 172.21.15.82:0/3820062175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f54db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/3820062175 >> 172.21.15.82:0/3820062175 conn(0x5629f7f07020 msgr2=0x5629f7f09440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/3820062175 shutdown_connections 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/3820062175 wait complete. 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- start start 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 Processor -- start 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f39d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629f7d1c250 con 0x5629f7f0b0a0 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629f7d12690 con 0x5629f7f552f0 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629f7ce0a40 con 0x5629f7f549d0 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 10 client.0 _ll_register_callbacks cb 0x5629f7ef5e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43580/0 (socket says 172.21.15.82:43580) 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 -- 172.21.15.82:0/1118557877 learned_addr learned my addr 172.21.15.82:0/1118557877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dda80640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dca7e640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f39d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 -- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 msgr2=0x5629f7f39d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f39d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 -- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 msgr2=0x5629f7f3a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f3a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 -- 172.21.15.82:0/1118557877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629f7e051c0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dda80640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dd27f640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f16d0000fc0 tx=0x7f16d0054cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4849e161b3b15b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0055050 con 0x5629f7f0b0a0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f16d0013c90 con 0x5629f7f0b0a0 2024-04-25T10:07:59.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16c80a5510 con 0x5629f7f0b0a0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d007d050 con 0x5629f7f0b0a0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 10 client.7512 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5629f7e01170 con 0x5629f7f0b0a0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c80a5510 con 0x5629f7f0b0a0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f16d005ce70 con 0x5629f7f0b0a0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 10 client.7512 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 10 client.7512 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 20 client.7512 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f64c80 0x5629f7f67070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 4 client.7512 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629f7cf3c60 con 0x5629f7f64c80 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dca7e640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f64c80 0x5629f7f67070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16dca7e640 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f64c80 0x5629f7f67070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7f473148395921 server_cookie=b9809e4df0327804 in_seq=0 out_seq=0 2024-04-25T10:07:59.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.505+0000 7f16b7fff640 10 client.7512 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16b7fff640 1 -- 172.21.15.82:0/1118557877 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5629f7cf3c60 con 0x5629f7f64c80 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16b7fff640 10 client.7512 handle_command_reply: tid=0 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 client.7512 shutdown 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 msgr2=0x5629f7f397f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f16d0000fc0 tx=0x7f16d0054cb0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 shutdown_connections 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f549d0 0x5629f7f39d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f64c80 0x5629f7f67070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f552f0 0x5629f7f3a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 --2- 172.21.15.82:0/1118557877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7f397f0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16b7fff640 0 client.7512 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 >> 172.21.15.82:0/1118557877 conn(0x5629f7f07020 msgr2=0x5629f7f09ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 shutdown_connections 2024-04-25T10:07:59.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 1 -- 172.21.15.82:0/1118557877 wait complete. 2024-04-25T10:07:59.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.509+0000 7f16e2650000 20 client.7512 trim_cache size 0 max 16384 2024-04-25T10:07:59.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.521+0000 7f16e2650000 1 Processor -- start 2024-04-25T10:07:59.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.521+0000 7f16e2650000 1 -- start start 2024-04-25T10:07:59.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7fb61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d8200ff0 con 0x5629f7f549d0 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d82001f0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80c5cd0 con 0x5629f7f552f0 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43584/0 (socket says 172.21.15.82:43584) 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 -- 172.21.15.82:0/3042145246 learned_addr learned my addr 172.21.15.82:0/3042145246 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 -- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 msgr2=0x5629f7fb61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7fb61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 -- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f0b0a0 msgr2=0x5629f7fb5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 -- 172.21.15.82:0/3042145246 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629f7d9ede0 con 0x5629f7f549d0 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f16d80c3870 tx=0x7f16cc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b327a5b31359518 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16be7fc640 1 -- 172.21.15.82:0/3042145246 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc029030 con 0x5629f7f549d0 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16be7fc640 1 -- 172.21.15.82:0/3042145246 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc016870 con 0x5629f7f549d0 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16be7fc640 1 -- 172.21.15.82:0/3042145246 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00f040 con 0x5629f7f549d0 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3042145246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 msgr2=0x5629f7fb5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f16d80c3870 tx=0x7f16cc013000 comp rx=0 tx=0).stop 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3042145246 shutdown_connections 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7fb61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f549d0 0x5629f7fb5ca0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- 172.21.15.82:0/3042145246 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3042145246 >> 172.21.15.82:0/3042145246 conn(0x5629f7f07020 msgr2=0x5629f7f61fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3042145246 shutdown_connections 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3042145246 wait complete. 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- start start 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 Processor -- start 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f549d0 0x5629f7fb9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7f74e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629f7cf2e00 con 0x5629f7f0b0a0 2024-04-25T10:07:59.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d81ffc50 con 0x5629f7f549d0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629f7d57300 con 0x5629f7f552f0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 10 client.0 _ll_register_callbacks cb 0x5629f7ed6040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43596/0 (socket says 172.21.15.82:43596) 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 -- 172.21.15.82:0/3079145356 learned_addr learned my addr 172.21.15.82:0/3079145356 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f549d0 0x5629f7fb9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 -- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 msgr2=0x5629f7f74e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7f74e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 -- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f549d0 msgr2=0x5629f7fb9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f549d0 0x5629f7fb9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 -- 172.21.15.82:0/3079145356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629f7cf98e0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16dd27f640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f16d0052ea0 tx=0x7f16d0057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8da98e68b73e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0050e90 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f16d0004bb0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16cc016870 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16d0004db0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 10 client.7518 fetch_fsmap learned FSMap version 48 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x5629f7db18f0 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc016870 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f16d0054040 con 0x5629f7f0b0a0 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 10 client.7518 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 10 client.7518 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 20 client.7518 populate_metadata read hostname 'smithi082' 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f75870 0x5629f7f75c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 4 client.7518 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.525+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5629f7d65530 con 0x5629f7f75870 2024-04-25T10:07:59.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f75870 0x5629f7f75c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16b7fff640 10 client.7518 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16dca7e640 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f75870 0x5629f7f75c80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d796b707f13d6df1 server_cookie=612df29f1a712ae3 in_seq=0 out_seq=0 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16b7fff640 1 -- 172.21.15.82:0/3079145356 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x5629f7d65530 con 0x5629f7f75870 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16b7fff640 10 client.7518 handle_command_reply: tid=0 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 client.7518 shutdown 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 msgr2=0x5629f7fb9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f16d0052ea0 tx=0x7f16d0057040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 shutdown_connections 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5629f7f552f0 0x5629f7f74e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x5629f7f75870 0x5629f7f75c80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5629f7f549d0 0x5629f7fb9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 --2- 172.21.15.82:0/3079145356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5629f7f0b0a0 0x5629f7fb9590 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 >> 172.21.15.82:0/3079145356 conn(0x5629f7f07020 msgr2=0x5629f7f530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16b7fff640 0 client.7518 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 shutdown_connections 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 1 -- 172.21.15.82:0/3079145356 wait complete. 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16e2650000 20 client.7518 trim_cache size 0 max 16384 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:07:59.541 INFO:teuthology.orchestra.run.smithi082.stdout: "fcd14b4f-3dbc-4c0f-8d87-360cbdd21fe7": { 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/subdir", 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "fcd14b4f-3dbc-4c0f-8d87-360cbdd21fe7", 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:07:59.542 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:07:59.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 msgr2=0x7f16c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 0x7f16c4069450 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f16d80c9300 tx=0x7f16cc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 msgr2=0x7f16d80ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f16d00011f0 tx=0x7f16d000c9b0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 shutdown_connections 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16c4066f90 0x7f16c4069450 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d80ae900 0x7f16d80af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d80b1910 0x7f16d80aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 --2- 172.21.15.82:0/3525593778 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16d80b0b00 0x7f16d80ae3c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 >> 172.21.15.82:0/3525593778 conn(0x7f16d80b1f10 msgr2=0x7f16d815c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 shutdown_connections 2024-04-25T10:07:59.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:07:59.529+0000 7f16df4e3640 1 -- 172.21.15.82:0/3525593778 wait complete. 2024-04-25T10:07:59.547 INFO:tasks.cephfs.filesystem:scrub status for tag:fcd14b4f-3dbc-4c0f-8d87-360cbdd21fe7 - {'path': '/subdir', 'tag': 'fcd14b4f-3dbc-4c0f-8d87-360cbdd21fe7', 'options': 'recursive,repair'} 2024-04-25T10:08:07.472 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:07.476 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:29.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:29.549 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:08:29.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.676+0000 7fe221cb4640 1 Processor -- start 2024-04-25T10:08:29.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.676+0000 7fe221cb4640 1 -- start start 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe221cb4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe21c0cf590 0x7fe21c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe221cb4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe221cb4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c0ca4c0 con 0x7fe21c0c9ef0 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe221cb4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21c0ca780 con 0x7fe21c0cf590 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe221cb4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0cd440 con 0x7fe21c0cf970 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe21c0cf590 0x7fe21c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:48206/0 (socket says 172.21.15.82:48206) 2024-04-25T10:08:29.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21bfff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe21c0c9ef0 0x7fe21c0b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:40842/0 (socket says 172.21.15.82:40842) 2024-04-25T10:08:29.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21affd640 1 -- 172.21.15.82:0/2271274383 learned_addr learned my addr 172.21.15.82:0/2271274383 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 -- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 msgr2=0x7fe21c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 -- 172.21.15.82:0/2271274383 >> v1:172.21.15.196:6789/0 conn(0x7fe21c0c9ef0 legacy=0x7fe21c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 -- 172.21.15.82:0/2271274383 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c0cd690 con 0x7fe21c0cf590 2024-04-25T10:08:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21affd640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:29.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.680+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15ecb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d9e37646621d44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21a7fc640 1 -- 172.21.15.82:0/2271274383 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0b0070 con 0x7fe21c0cf590 2024-04-25T10:08:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21a7fc640 1 -- 172.21.15.82:0/2271274383 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c0ab600 con 0x7fe21c0cf590 2024-04-25T10:08:29.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21a7fc640 1 -- 172.21.15.82:0/2271274383 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c0ab9d0 con 0x7fe21c0cf590 2024-04-25T10:08:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 msgr2=0x7fe21c15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15ecb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0097c0 tx=0x7fe20c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- 172.21.15.82:0/2271274383 shutdown_connections 2024-04-25T10:08:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0cf970 0x7fe21c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- 172.21.15.82:0/2271274383 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15ecb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- 172.21.15.82:0/2271274383 >> 172.21.15.82:0/2271274383 conn(0x7fe21c06ae60 msgr2=0x7fe21c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- 172.21.15.82:0/2271274383 shutdown_connections 2024-04-25T10:08:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- 172.21.15.82:0/2271274383 wait complete. 2024-04-25T10:08:29.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 Processor -- start 2024-04-25T10:08:29.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- start start 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0c9ef0 0x7fe21c098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0cbf60 con 0x7fe21c0cf970 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0cbde0 con 0x7fe21c0cf590 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe221cb4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0cc0e0 con 0x7fe21c0c9ef0 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34404/0 (socket says 172.21.15.82:34404) 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21bfff640 1 -- 172.21.15.82:0/819406303 learned_addr learned my addr 172.21.15.82:0/819406303 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0c9ef0 0x7fe21c098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21affd640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21bfff640 1 -- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0c9ef0 msgr2=0x7fe21c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.684+0000 7fe21bfff640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0c9ef0 0x7fe21c098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21bfff640 1 -- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 msgr2=0x7fe21c15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21bfff640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21bfff640 1 -- 172.21.15.82:0/819406303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c15c070 con 0x7fe21c0cf970 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21affd640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21bfff640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c7a0 tx=0x7fe21400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 -- 172.21.15.82:0/819406303 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214017070 con 0x7fe21c0cf970 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 -- 172.21.15.82:0/819406303 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21400a640 con 0x7fe21c0cf970 2024-04-25T10:08:29.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c15c350 con 0x7fe21c0cf970 2024-04-25T10:08:29.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 -- 172.21.15.82:0/819406303 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21400a9f0 con 0x7fe21c0cf970 2024-04-25T10:08:29.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c15c910 con 0x7fe21c0cf970 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 -- 172.21.15.82:0/819406303 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe21401a070 con 0x7fe21c0cf970 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe203fff640 1 -- 172.21.15.82:0/819406303 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe2140847e0 con 0x7fe21c0cf970 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 0x7fe208069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0ad000 tx=0x7fe20c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe224e21000 1 Processor -- start 2024-04-25T10:08:29.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.688+0000 7fe224e21000 1 -- start start 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598ce69d0 0x559598ce6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598c9cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598ae9ff0 con 0x559598c9d0a0 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598a72150 con 0x559598ce69d0 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559598ae6090 con 0x559598ce72f0 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34408/0 (socket says 172.21.15.82:34408) 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 -- 172.21.15.82:0/2552629942 learned_addr learned my addr 172.21.15.82:0/2552629942 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598ce69d0 0x559598ce6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598c9cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 -- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 msgr2=0x559598c9cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598c9cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 -- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598ce69d0 msgr2=0x559598ce6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598ce69d0 0x559598ce6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 -- 172.21.15.82:0/2552629942 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559598c800d0 con 0x559598c9d0a0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe214054f40 tx=0x7fe214059110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd71eec6105ca53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe2017fa640 1 -- 172.21.15.82:0/2552629942 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe214051620 con 0x559598c9d0a0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe2017fa640 1 -- 172.21.15.82:0/2552629942 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21405b0c0 con 0x559598c9d0a0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe2017fa640 1 -- 172.21.15.82:0/2552629942 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2140517c0 con 0x559598c9d0a0 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/2552629942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 msgr2=0x559598c9d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fe214054f40 tx=0x7fe214059110 comp rx=0 tx=0).stop 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/2552629942 shutdown_connections 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598c9cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598c9d0a0 0x559598c9d480 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- 172.21.15.82:0/2552629942 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598ce69d0 0x559598ce6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/2552629942 >> 172.21.15.82:0/2552629942 conn(0x559598c99020 msgr2=0x559598c9b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/2552629942 shutdown_connections 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/2552629942 wait complete. 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- start start 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 Processor -- start 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598ccb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598ccc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598aae250 con 0x559598ce69d0 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598aa4690 con 0x559598c9d0a0 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559598a72a40 con 0x559598ce72f0 2024-04-25T10:08:29.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 10 client.0 _ll_register_callbacks cb 0x559598c87e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34414/0 (socket says 172.21.15.82:34414) 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 -- 172.21.15.82:0/506735386 learned_addr learned my addr 172.21.15.82:0/506735386 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598ccc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598ccb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 -- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 msgr2=0x559598ccc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598ccc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 -- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 msgr2=0x559598ccb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598ccb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 -- 172.21.15.82:0/506735386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559598b971c0 con 0x559598ce69d0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21bfff640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598ccc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598ccb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe21affd640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe210009ef0 tx=0x7fe210011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b6d9cd984f50e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe1fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210019070 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe21405b0c0 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.692+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe210015870 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe210024050 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 10 client.7527 fetch_fsmap learned FSMap version 49 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x559598b93170 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21405b0c0 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe210027020 con 0x559598ce69d0 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 10 client.7527 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 10 client.7527 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 20 client.7527 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598cf6c80 0x559598cf9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 4 client.7527 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559598a85c60 con 0x559598cf6c80 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598cf6c80 0x559598cf9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 10 client.7527 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598cf6c80 0x559598cf9070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e1dbc6b2c06083 server_cookie=89d795c629f7f62b in_seq=0 out_seq=0 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/506735386 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559598a85c60 con 0x559598cf6c80 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 10 client.7527 handle_command_reply: tid=0 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 client.7527 shutdown 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 msgr2=0x559598ccbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe210009ef0 tx=0x7fe210011630 comp rx=0 tx=0).stop 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 shutdown_connections 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598ccc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598cf6c80 0x559598cf9070 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598ccbd30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 --2- 172.21.15.82:0/506735386 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598ccb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe1fa7fc640 0 client.7527 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:29.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 >> 172.21.15.82:0/506735386 conn(0x559598c99020 msgr2=0x559598c9bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 shutdown_connections 2024-04-25T10:08:29.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 1 -- 172.21.15.82:0/506735386 wait complete. 2024-04-25T10:08:29.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.696+0000 7fe224e21000 20 client.7527 trim_cache size 0 max 16384 2024-04-25T10:08:29.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 Processor -- start 2024-04-25T10:08:29.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- start start 2024-04-25T10:08:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce69d0 0x559598d47880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1fee10 con 0x559598ce72f0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1fe010 con 0x559598c9d0a0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c158470 con 0x559598ce69d0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34422/0 (socket says 172.21.15.82:34422) 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 -- 172.21.15.82:0/3829576111 learned_addr learned my addr 172.21.15.82:0/3829576111 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 -- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce69d0 msgr2=0x559598d47880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce69d0 0x559598d47880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 -- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 msgr2=0x559598d47340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d47340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 -- 172.21.15.82:0/3829576111 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559598b30de0 con 0x559598ce72f0 2024-04-25T10:08:29.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe21bfff640 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe214059150 tx=0x7fe21405b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993acc59e19388bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe200ff9640 1 -- 172.21.15.82:0/3829576111 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21406c040 con 0x559598ce72f0 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe200ff9640 1 -- 172.21.15.82:0/3829576111 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214051620 con 0x559598ce72f0 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe200ff9640 1 -- 172.21.15.82:0/3829576111 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21405b300 con 0x559598ce72f0 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- 172.21.15.82:0/3829576111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 msgr2=0x559598d47dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe214059150 tx=0x7fe21405b050 comp rx=0 tx=0).stop 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- 172.21.15.82:0/3829576111 shutdown_connections 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce69d0 0x559598d47880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce72f0 0x559598d47dc0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 --2- 172.21.15.82:0/3829576111 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- 172.21.15.82:0/3829576111 >> 172.21.15.82:0/3829576111 conn(0x559598c99020 msgr2=0x559598cf3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- 172.21.15.82:0/3829576111 shutdown_connections 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- 172.21.15.82:0/3829576111 wait complete. 2024-04-25T10:08:29.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 -- start start 2024-04-25T10:08:29.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.712+0000 7fe224e21000 1 Processor -- start 2024-04-25T10:08:29.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598d064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559598a84e00 con 0x559598ce69d0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1fda10 con 0x559598c9d0a0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559598ae9300 con 0x559598ce72f0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 10 client.0 _ll_register_callbacks cb 0x559598c68040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34426/0 (socket says 172.21.15.82:34426) 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 -- 172.21.15.82:0/2052438959 learned_addr learned my addr 172.21.15.82:0/2052438959 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 -- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 msgr2=0x559598d064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598d064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 -- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 msgr2=0x559598d4ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d4ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 -- 172.21.15.82:0/2052438959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559598a8b8e0 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21affd640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe2100094b0 tx=0x7fe210004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406d0bc36c8b780c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210019040 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe210006b50 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe214051620 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe210034050 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 10 client.7533 fetch_fsmap learned FSMap version 49 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x559598b438f0 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe214051620 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe210037020 con 0x559598ce69d0 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 10 client.7533 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 10 client.7533 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.a 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 20 client.7533 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:29.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598d07250 0x559598d07660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 4 client.7533 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559598af7530 con 0x559598d07250 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598d07250 0x559598d07660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 10 client.7533 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe21b7fe640 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598d07250 0x559598d07660 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b935c72c13387 server_cookie=168eac8b87899b9d in_seq=0 out_seq=0 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 1 -- 172.21.15.82:0/2052438959 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559598af7530 con 0x559598d07250 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 10 client.7533 handle_command_reply: tid=0 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 client.7533 shutdown 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 msgr2=0x559598d4b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe2100094b0 tx=0x7fe210004740 comp rx=0 tx=0).stop 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 shutdown_connections 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559598ce72f0 0x559598d064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x559598d07250 0x559598d07660 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559598ce69d0 0x559598d4b170 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 --2- 172.21.15.82:0/2052438959 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559598c9d0a0 0x559598d4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe1fa7fc640 0 client.7533 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 >> 172.21.15.82:0/2052438959 conn(0x559598c99020 msgr2=0x559598ce50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 shutdown_connections 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 1 -- 172.21.15.82:0/2052438959 wait complete. 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe224e21000 20 client.7533 trim_cache size 0 max 16384 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:08:29.728 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:08:29.729 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:08:29.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 msgr2=0x7fe208069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 0x7fe208069420 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0ad000 tx=0x7fe20c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 msgr2=0x7fe21c152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fe21400c7a0 tx=0x7fe21400cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 shutdown_connections 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe208066f60 0x7fe208069420 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe21c0cf970 0x7fe21c152bc0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe21c0cf590 0x7fe21c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 --2- 172.21.15.82:0/819406303 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe21c0c9ef0 0x7fe21c098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 >> 172.21.15.82:0/819406303 conn(0x7fe21c06ae60 msgr2=0x7fe21c06cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 shutdown_connections 2024-04-25T10:08:29.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.716+0000 7fe221cb4640 1 -- 172.21.15.82:0/819406303 wait complete. 2024-04-25T10:08:29.738 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:08:29.738 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.844+0000 7f6ad94ba640 1 Processor -- start 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad94ba640 1 -- start start 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad94ba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ad40b3f50 0x7f6ad415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad94ba640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad40b2090 con 0x7f6ad415a700 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad94ba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad40b43e0 con 0x7f6ad40b24a0 2024-04-25T10:08:29.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad94ba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad415f200 con 0x7f6ad40b3f50 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad2ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6ad415a700 0x7f6ad40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40522/0 (socket says 172.21.15.82:40522) 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad2ffd640 1 -- 172.21.15.82:0/3875127972 learned_addr learned my addr 172.21.15.82:0/3875127972 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/3875127972 >> v2:172.21.15.196:3300/0 conn(0x7f6ad40b3f50 0x7f6ad415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107264559 0 0) 0x7f6ad40b2090 con 0x7f6ad415a700 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ac0003530 con 0x7f6ad415a700 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2488784612 0 0) 0x7f6ac0003530 con 0x7f6ad415a700 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 >> v2:172.21.15.196:3300/0 conn(0x7f6ad40b3f50 msgr2=0x7f6ad415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 --2- 172.21.15.82:0/3875127972 >> v2:172.21.15.196:3300/0 conn(0x7f6ad40b3f50 0x7f6ad415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 >> v1:172.21.15.196:6789/0 conn(0x7f6ad40b24a0 legacy=0x7f6ad406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad40b46c0 con 0x7f6ad415a700 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6ac40016a0 con 0x7f6ad415a700 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 >> v1:172.21.15.82:6789/0 conn(0x7f6ad415a700 legacy=0x7f6ad40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ac0003a30 0x7f6ac0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac00052d0 0x7f6ac000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad415f200 con 0x7f6ac0004810 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac000c210 con 0x7f6ac00052d0 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ac000c410 con 0x7f6ac0003a30 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 -- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ac0003a30 msgr2=0x7f6ac0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ac0003a30 0x7f6ac0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad37fe640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac00052d0 0x7f6ac000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 -- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac00052d0 msgr2=0x7f6ac000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ac00052d0 0x7f6ac000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad37fe640 1 -- 172.21.15.82:0/3875127972 reap_dead start 2024-04-25T10:08:29.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.848+0000 7f6ad27fc640 1 -- 172.21.15.82:0/3875127972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac0003530 con 0x7f6ac0004810 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009ac0 tx=0x7f6abc01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e21da1fe893d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc009690 con 0x7f6ac0004810 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc0a6510 con 0x7f6ac0004810 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad1ffb640 1 -- 172.21.15.82:0/3875127972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc0a68e0 con 0x7f6ac0004810 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 msgr2=0x7f6ac0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c90 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009ac0 tx=0x7f6abc01ca20 comp rx=0 tx=0).stop 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- 172.21.15.82:0/3875127972 shutdown_connections 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/3875127972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ac0004c90 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- 172.21.15.82:0/3875127972 >> 172.21.15.82:0/3875127972 conn(0x7f6ad40b11a0 msgr2=0x7f6ad40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- 172.21.15.82:0/3875127972 shutdown_connections 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- 172.21.15.82:0/3875127972 wait complete. 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 Processor -- start 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- start start 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad41f54b0 0x7f6ad4153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ad4153980 0x7f6ad415bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40b2c90 con 0x7f6ac0004810 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40ce700 con 0x7f6ad41f54b0 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad94ba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad40cd310 con 0x7f6ad4153980 2024-04-25T10:08:29.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34438/0 (socket says 172.21.15.82:34438) 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 -- 172.21.15.82:0/1734360215 learned_addr learned my addr 172.21.15.82:0/1734360215 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad41f54b0 0x7f6ad4153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad37fe640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ad4153980 0x7f6ad415bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 -- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ad4153980 msgr2=0x7f6ad415bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ad4153980 0x7f6ad415bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 -- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad41f54b0 msgr2=0x7f6ad4153440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad41f54b0 0x7f6ad4153440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.852+0000 7f6ad2ffd640 1 -- 172.21.15.82:0/1734360215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4067e40 con 0x7f6ac0004810 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad2ffd640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c680 tx=0x7f6ac400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4018070 con 0x7f6ac0004810 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac40072f0 con 0x7f6ac0004810 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad41f5b50 con 0x7f6ac0004810 2024-04-25T10:08:29.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac40047f0 con 0x7f6ac0004810 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad41f6110 con 0x7f6ac0004810 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad404efa0 con 0x7f6ac0004810 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ac401b070 con 0x7f6ac0004810 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6ac4084570 con 0x7f6ac0004810 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:29.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.856+0000 7f6ad27fc640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 0x7f6ab4069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0af000 tx=0x7f6abc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:29.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:29.860+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ac4055430 con 0x7f6ac0004810 2024-04-25T10:08:30.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.048+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ad406f260 con 0x7f6ac0004810 2024-04-25T10:08:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.048+0000 7f6acb7fe640 1 -- 172.21.15.82:0/1734360215 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f6ac4054d30 con 0x7f6ac0004810 2024-04-25T10:08:30.055 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:30.055 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:08:00.987254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:30.055 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-25T10:08:30.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 msgr2=0x7f6ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 0x7f6ab4069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6abc0af000 tx=0x7f6abc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 msgr2=0x7f6ad41f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400c680 tx=0x7f6ac400cb40 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 shutdown_connections 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6ab4066f60 0x7f6ab4069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ad4153980 0x7f6ad415bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ad41f54b0 0x7f6ad4153440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 --2- 172.21.15.82:0/1734360215 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ac0004810 0x7f6ad41f3720 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 >> 172.21.15.82:0/1734360215 conn(0x7f6ad40b11a0 msgr2=0x7f6ad40b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 shutdown_connections 2024-04-25T10:08:30.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.052+0000 7f6ad94ba640 1 -- 172.21.15.82:0/1734360215 wait complete. 2024-04-25T10:08:30.072 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T10:08:30.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.188+0000 7f88d2f0e640 1 Processor -- start 2024-04-25T10:08:30.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.188+0000 7f88d2f0e640 1 -- start start 2024-04-25T10:08:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d2f0e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88cc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d2f0e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f88cc0d07f0 0x7f88cc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d2f0e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88cc06be60 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d2f0e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc06c440 con 0x7f88cc0d07f0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d2f0e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc15de60 con 0x7f88cc15da80 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f88cc0d1cd0 0x7f88cc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40534/0 (socket says 172.21.15.82:40534) 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 -- 172.21.15.82:0/571511660 learned_addr learned my addr 172.21.15.82:0/571511660 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d14ab640 1 --2- 172.21.15.82:0/571511660 >> v2:172.21.15.196:3300/0 conn(0x7f88cc0d07f0 0x7f88cc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d0caa640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88cc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369055242 0 0) 0x7f88cc06be60 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b4003530 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328734896 0 0) 0x7f88b4003530 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 msgr2=0x7f88cc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88cc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 >> v2:172.21.15.196:3300/0 conn(0x7f88cc0d07f0 msgr2=0x7f88cc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 --2- 172.21.15.82:0/571511660 >> v2:172.21.15.196:3300/0 conn(0x7f88cc0d07f0 0x7f88cc06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0d2450 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f88bc001800 con 0x7f88cc0d1cd0 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 >> v1:172.21.15.82:6789/0 conn(0x7f88cc0d1cd0 legacy=0x7f88cc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4003a30 0x7f88b4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4004810 0x7f88b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc06c440 con 0x7f88b4004810 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc15de60 con 0x7f88b4003a30 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b400b9d0 con 0x7f88cc15da80 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4004810 0x7f88b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d14ab640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d0caa640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4003a30 0x7f88b4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 -- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 msgr2=0x7f88b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88b40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 -- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4003a30 msgr2=0x7f88b4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88b4003a30 0x7f88b4003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 -- 172.21.15.82:0/571511660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b4003530 con 0x7f88b4004810 2024-04-25T10:08:30.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88d14ab640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15da80 0x7f88b40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:30.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c3fff640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4004810 0x7f88b4004c90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c210 tx=0x7f88bc01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6504c8cca650ff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc007d50 con 0x7f88b4004810 2024-04-25T10:08:30.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0ac020 con 0x7f88b4004810 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.192+0000 7f88c37fe640 1 -- 172.21.15.82:0/571511660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0070d0 con 0x7f88b4004810 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- 172.21.15.82:0/571511660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4004810 msgr2=0x7f88b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/571511660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88b4004810 0x7f88b4004c90 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c210 tx=0x7f88bc01f3d0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d14ab640 1 -- 172.21.15.82:0/571511660 reap_dead start 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- 172.21.15.82:0/571511660 shutdown_connections 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- 172.21.15.82:0/571511660 >> 172.21.15.82:0/571511660 conn(0x7f88cc08b000 msgr2=0x7f88cc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- 172.21.15.82:0/571511660 shutdown_connections 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- 172.21.15.82:0/571511660 wait complete. 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 Processor -- start 2024-04-25T10:08:30.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- start start 2024-04-25T10:08:30.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88cc15b140 0x7f88cc15bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 0x7f88cc0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0d26f0 con 0x7f88cc151c60 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0d20f0 con 0x7f88cc15b140 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d2f0e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc0d2870 con 0x7f88cc15c1e0 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34458/0 (socket says 172.21.15.82:34458) 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 -- 172.21.15.82:0/3084785240 learned_addr learned my addr 172.21.15.82:0/3084785240 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88c3fff640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88cc15b140 0x7f88cc15bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d14ab640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 0x7f88cc0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 -- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 msgr2=0x7f88cc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 0x7f88cc0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 -- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88cc15b140 msgr2=0x7f88cc15bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88cc15b140 0x7f88cc15bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 -- 172.21.15.82:0/3084785240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc0cc0b0 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d14ab640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 0x7f88cc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.196+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f88c8010890 tx=0x7f88c8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 -- 172.21.15.82:0/3084785240 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800f040 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 -- 172.21.15.82:0/3084785240 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c800eb00 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc0cb010 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 -- 172.21.15.82:0/3084785240 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c801c400 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc0cb5d0 con 0x7f88cc151c60 2024-04-25T10:08:30.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 -- 172.21.15.82:0/3084785240 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88c801c5a0 con 0x7f88cc151c60 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 0x7f88b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c17fa640 1 -- 172.21.15.82:0/3084785240 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f88c80856d0 con 0x7f88cc151c60 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c3fff640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 0x7f88b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c3fff640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 0x7f88b8069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ad000 tx=0x7f88bc001170 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 Processor -- start 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 -- start start 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c3f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9bf5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff9adcba0 con 0x55fff9c3fc00 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff9a42090 con 0x55fff9c3f2e0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fff99ca160 con 0x55fff9bf58e0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34462/0 (socket says 172.21.15.82:34462) 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.200+0000 7f88c3fff640 1 -- 172.21.15.82:0/1288748631 learned_addr learned my addr 172.21.15.82:0/1288748631 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d0caa640 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c3f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d14ab640 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9bf5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 msgr2=0x55fff9bf5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9bf5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 msgr2=0x55fff9c3f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c3f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1288748631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fff9aebc20 con 0x55fff9c3fc00 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000f00 tx=0x7f88cc06c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75a58fcbba72b5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88b27fc640 1 -- 172.21.15.82:0/1288748631 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0c1070 con 0x55fff9c3fc00 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88b27fc640 1 -- 172.21.15.82:0/1288748631 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0ac040 con 0x55fff9c3fc00 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88b27fc640 1 -- 172.21.15.82:0/1288748631 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0074d0 con 0x55fff9c3fc00 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1288748631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 msgr2=0x55fff9bf53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000f00 tx=0x7f88cc06c3b0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1288748631 shutdown_connections 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9bf5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3fc00 0x55fff9bf53a0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- 172.21.15.82:0/1288748631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c3f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1288748631 >> 172.21.15.82:0/1288748631 conn(0x55fff9bf1890 msgr2=0x55fff9bf3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1288748631 shutdown_connections 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1288748631 wait complete. 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- start start 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 Processor -- start 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9bf58e0 0x55fff9c46640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c2d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff9a3e130 con 0x55fff9c3f2e0 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff9a06130 con 0x55fff9bf58e0 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fff9b00300 con 0x55fff9c3fc00 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 10 client.0 _ll_register_callbacks cb 0x55fff9be0bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34468/0 (socket says 172.21.15.82:34468) 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1698768209 learned_addr learned my addr 172.21.15.82:0/1698768209 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 msgr2=0x55fff9c2d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c2d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9bf58e0 msgr2=0x55fff9c46640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9bf58e0 0x55fff9c46640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 -- 172.21.15.82:0/1698768209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fff9a1f060 con 0x55fff9c3f2e0 2024-04-25T10:08:30.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88c3fff640 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000f00 tx=0x7f88bc01c170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ee93163c3aed444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0c1040 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f88bc0c6460 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88bc0ac040 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88bc0d4050 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 10 client.7551 fetch_fsmap learned FSMap version 49 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55fff99e37c0 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0ac040 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88bc0d7020 con 0x55fff9c3f2e0 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 10 client.7551 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 10 client.7551 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 20 client.7551 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:30.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9c2de30 0x55fff9c2e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 4 client.7551 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.204+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fff99ddcd0 con 0x55fff9c2de30 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d0caa640 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9c2de30 0x55fff9c2e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d0caa640 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9c2de30 0x55fff9c2e210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28a0a440727d291e server_cookie=63fdb282b9b01008 in_seq=0 out_seq=0 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88ab7fe640 10 client.7551 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88ab7fe640 1 -- 172.21.15.82:0/1698768209 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fff99ddcd0 con 0x55fff9c2de30 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88ab7fe640 10 client.7551 handle_command_reply: tid=0 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 client.7551 shutdown 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 msgr2=0x55fff9c46b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f88bc000f00 tx=0x7f88bc01c170 comp rx=0 tx=0).stop 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 shutdown_connections 2024-04-25T10:08:30.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c2d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9c2de30 0x55fff9c2e210 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c46b80 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 --2- 172.21.15.82:0/1698768209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9bf58e0 0x55fff9c46640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88ab7fe640 0 client.7551 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 >> 172.21.15.82:0/1698768209 conn(0x55fff9bf1890 msgr2=0x55fff9bf4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 shutdown_connections 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 1 -- 172.21.15.82:0/1698768209 wait complete. 2024-04-25T10:08:30.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.208+0000 7f88d607b000 20 client.7551 trim_cache size 0 max 16384 2024-04-25T10:08:30.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 Processor -- start 2024-04-25T10:08:30.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- start start 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9c2c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3fc00 0x55fff9bf3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc200280 con 0x55fff9c3f2e0 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1ff6e0 con 0x55fff9c3fc00 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc1ffcb0 con 0x55fff9bf58e0 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34476/0 (socket says 172.21.15.82:34476) 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 -- 172.21.15.82:0/2533560986 learned_addr learned my addr 172.21.15.82:0/2533560986 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 -- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 msgr2=0x55fff9c2c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9c2c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 -- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3fc00 msgr2=0x55fff9bf3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3fc00 0x55fff9bf3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 -- 172.21.15.82:0/2533560986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fff99fdfc0 con 0x55fff9c3f2e0 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88c3fff640 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f88bc001e00 tx=0x7f88bc0c8070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75dd61065ad0b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88b1ffb640 1 -- 172.21.15.82:0/2533560986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0c1070 con 0x55fff9c3f2e0 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88b1ffb640 1 -- 172.21.15.82:0/2533560986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0ac070 con 0x55fff9c3f2e0 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88b1ffb640 1 -- 172.21.15.82:0/2533560986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0c6900 con 0x55fff9c3f2e0 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- 172.21.15.82:0/2533560986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 msgr2=0x55fff9c2c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f88bc001e00 tx=0x7f88bc0c8070 comp rx=0 tx=0).stop 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- 172.21.15.82:0/2533560986 shutdown_connections 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3fc00 0x55fff9bf3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9c3f2e0 0x55fff9c2c730 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 --2- 172.21.15.82:0/2533560986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9bf58e0 0x55fff9c2c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- 172.21.15.82:0/2533560986 >> 172.21.15.82:0/2533560986 conn(0x55fff9bf1890 msgr2=0x55fff9c4cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- 172.21.15.82:0/2533560986 shutdown_connections 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- 172.21.15.82:0/2533560986 wait complete. 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 -- start start 2024-04-25T10:08:30.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.220+0000 7f88d607b000 1 Processor -- start 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c9f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c43040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff99dce70 con 0x55fff9bf58e0 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fff99caa50 con 0x55fff9c3f2e0 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fff9a413a0 con 0x55fff9c3fc00 2024-04-25T10:08:30.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 10 client.0 _ll_register_callbacks cb 0x55fff99ee240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34488/0 (socket says 172.21.15.82:34488) 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 -- 172.21.15.82:0/3326068973 learned_addr learned my addr 172.21.15.82:0/3326068973 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 -- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 msgr2=0x55fff9c43040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c43040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 -- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 msgr2=0x55fff9c9f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c9f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 -- 172.21.15.82:0/3326068973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fff9a9b9e0 con 0x55fff9bf58e0 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d0caa640 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f88c80522f0 tx=0x7f88c8056650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ce6b0e6e75c2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:30.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8081070 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f88c8052500 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88bc0ac070 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88c805e5a0 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 10 client.7557 fetch_fsmap learned FSMap version 49 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) v3 -- 0x55fff99e1630 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc0ac070 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 49) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f88c8024030 con 0x55fff9bf58e0 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 10 client.7557 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 10 client.7557 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 20 client.7557 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9ca5ba0 0x55fff9ca7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 4 client.7557 mds_command: new command op to 7260 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 --> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55fff9a4f5d0 con 0x55fff9ca5ba0 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88c3fff640 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9ca5ba0 0x55fff9ca7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88c3fff640 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9ca5ba0 0x55fff9ca7f90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a17e4815cd66898e server_cookie=562c57d719c5c823 in_seq=0 out_seq=0 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 10 client.7557 ms_handle_connect on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 1 -- 172.21.15.82:0/3326068973 <== mds.0 v2:172.21.15.82:6836/3956208581 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fff9a4f5d0 con 0x55fff9ca5ba0 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 10 client.7557 handle_command_reply: tid=0 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 client.7557 shutdown 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 msgr2=0x55fff9c9ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f88c80522f0 tx=0x7f88c8056650 comp rx=0 tx=0).stop 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 shutdown_connections 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fff9c3fc00 0x55fff9c43040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:6836/3956208581,v1:172.21.15.82:6837/3956208581] conn(0x55fff9ca5ba0 0x55fff9ca7f90 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fff9c3f2e0 0x55fff9c9f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 --2- 172.21.15.82:0/3326068973 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fff9bf58e0 0x55fff9c9ed40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 >> 172.21.15.82:0/3326068973 conn(0x55fff9bf1890 msgr2=0x55fff9c4cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88ab7fe640 0 client.7557 ms_handle_reset on v2:172.21.15.82:6836/3956208581 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 shutdown_connections 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 1 -- 172.21.15.82:0/3326068973 wait complete. 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d607b000 20 client.7557 trim_cache size 0 max 16384 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:30.238 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-25T10:08:30.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 msgr2=0x7f88b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 0x7f88b8069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f88bc0ad000 tx=0x7f88bc001170 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 msgr2=0x7f88cc0ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f88c8010890 tx=0x7f88c8010d50 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 shutdown_connections 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88b8066f60 0x7f88b8069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88cc15c1e0 0x7f88cc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88cc15b140 0x7f88cc15bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 --2- 172.21.15.82:0/3084785240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88cc151c60 0x7f88cc0ca7c0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 >> 172.21.15.82:0/3084785240 conn(0x7f88cc08b000 msgr2=0x7f88cc0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 shutdown_connections 2024-04-25T10:08:30.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.224+0000 7f88d2f0e640 1 -- 172.21.15.82:0/3084785240 wait complete. 2024-04-25T10:08:30.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:08:30.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 Processor -- start 2024-04-25T10:08:30.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 -- start start 2024-04-25T10:08:30.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f7404070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740415bbb0 con 0x7f740406e680 2024-04-25T10:08:30.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040ac280 con 0x7f74040cf560 2024-04-25T10:08:30.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740bbcb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040ace70 con 0x7f74040b22e0 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740a168640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f740406e680 0x7f74040ad300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40548/0 (socket says 172.21.15.82:40548) 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740a168640 1 -- 172.21.15.82:0/2869624473 learned_addr learned my addr 172.21.15.82:0/2869624473 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653535360 0 0) 0x7f740415bbb0 con 0x7f740406e680 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73f0003530 con 0x7f740406e680 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f7404070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319755802 0 0) 0x7f73f0003530 con 0x7f740406e680 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 msgr2=0x7f7404070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f7404070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 >> v1:172.21.15.196:6789/0 conn(0x7f74040cf560 legacy=0x7f74040cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040ac540 con 0x7f740406e680 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f73f4001680 con 0x7f740406e680 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 >> v1:172.21.15.82:6789/0 conn(0x7f740406e680 legacy=0x7f74040ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0003a60 0x7f73f0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73f0004810 0x7f73f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f73f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040ace70 con 0x7f73f0004810 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f0005530 con 0x7f73f0003a60 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f000bf10 con 0x7f74040b22e0 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73f0004810 0x7f73f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409967640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0003a60 0x7f73f0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740a168640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f73f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 -- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 msgr2=0x7f73f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f73f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 -- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0003a60 msgr2=0x7f73f0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0003a60 0x7f73f0003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f7409166640 1 -- 172.21.15.82:0/2869624473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f0003530 con 0x7f73f0004810 2024-04-25T10:08:30.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.344+0000 7f740a168640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040b22e0 0x7f73f000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f7409166640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73f0004810 0x7f73f0004c90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7400007360 tx=0x7f740001ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9627f2dcd8d5b073 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74000b2070 con 0x7f73f0004810 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74000a5c00 con 0x7f73f0004810 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f7408965640 1 -- 172.21.15.82:0/2869624473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7400009be0 con 0x7f73f0004810 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 -- 172.21.15.82:0/2869624473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73f0004810 msgr2=0x7f73f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 --2- 172.21.15.82:0/2869624473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f73f0004810 0x7f73f0004c90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f7400007360 tx=0x7f740001ce10 comp rx=0 tx=0).stop 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740a168640 1 -- 172.21.15.82:0/2869624473 reap_dead start 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 -- 172.21.15.82:0/2869624473 shutdown_connections 2024-04-25T10:08:30.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 -- 172.21.15.82:0/2869624473 >> 172.21.15.82:0/2869624473 conn(0x7f740406ae60 msgr2=0x7f740406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 -- 172.21.15.82:0/2869624473 shutdown_connections 2024-04-25T10:08:30.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.348+0000 7f740bbcb640 1 -- 172.21.15.82:0/2869624473 wait complete. 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 Processor -- start 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- start start 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 0x7f7404154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040aedf0 0x7f7404154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7404073310 con 0x7f74040c3290 2024-04-25T10:08:30.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7404073190 con 0x7f73f0004810 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040ade20 con 0x7f74040aedf0 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34500/0 (socket says 172.21.15.82:34500) 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 -- 172.21.15.82:0/977265805 learned_addr learned my addr 172.21.15.82:0/977265805 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f7409967640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 0x7f7404154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f7409166640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040aedf0 0x7f7404154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 -- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040aedf0 msgr2=0x7f7404154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040aedf0 0x7f7404154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 -- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 msgr2=0x7f7404154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 0x7f7404154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 -- 172.21.15.82:0/977265805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74040c55e0 con 0x7f74040c3290 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f7409967640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 0x7f7404154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740a168640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f740415f200 tx=0x7f73f400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f40075a0 con 0x7f74040c3290 2024-04-25T10:08:30.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f40047f0 con 0x7f74040c3290 2024-04-25T10:08:30.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7404160030 con 0x7f74040c3290 2024-04-25T10:08:30.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f4004be0 con 0x7f74040c3290 2024-04-25T10:08:30.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74040c5880 con 0x7f74040c3290 2024-04-25T10:08:30.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f740404efa0 con 0x7f74040c3290 2024-04-25T10:08:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.352+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73f4014430 con 0x7f74040c3290 2024-04-25T10:08:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.356+0000 7f73fa7fc640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 0x7f73e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.356+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f73f40849b0 con 0x7f74040c3290 2024-04-25T10:08:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.356+0000 7f7409967640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 0x7f73e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.356+0000 7f7409967640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 0x7f73e8069450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74040cb3d0 tx=0x7f73fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.360+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73f4055870 con 0x7f74040c3290 2024-04-25T10:08:30.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f74040cfd50 con 0x7f74040c3290 2024-04-25T10:08:30.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f73fa7fc640 1 -- 172.21.15.82:0/977265805 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f73f4055170 con 0x7f74040c3290 2024-04-25T10:08:30.541 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:30.541 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:08:30.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 msgr2=0x7f73e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 0x7f73e8069450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f74040cb3d0 tx=0x7f73fc009040 comp rx=0 tx=0).stop 2024-04-25T10:08:30.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 msgr2=0x7f74040c3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f740415f200 tx=0x7f73f400d9d0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 shutdown_connections 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f73e8066f90 0x7f73e8069450 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f74040aedf0 0x7f7404154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f74040c3290 0x7f74040c3740 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 --2- 172.21.15.82:0/977265805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f73f0004810 0x7f7404154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 >> 172.21.15.82:0/977265805 conn(0x7f740406ae60 msgr2=0x7f7404166a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 shutdown_connections 2024-04-25T10:08:30.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.536+0000 7f740bbcb640 1 -- 172.21.15.82:0/977265805 wait complete. 2024-04-25T10:08:30.558 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:08:30.558 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.660+0000 7f6975ddd640 1 Processor -- start 2024-04-25T10:08:30.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.660+0000 7f6975ddd640 1 -- start start 2024-04-25T10:08:30.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69700ce510 0x7f69700ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6970144ca0 0x7f697006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f697006ec60 0x7f6970142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700cc2b0 con 0x7f6970144ca0 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697006e290 con 0x7f697006ec60 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f6975ddd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700cee30 con 0x7f69700ce510 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6970144ca0 0x7f697006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6970144ca0 0x7f697006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34508/0 (socket says 172.21.15.82:34508) 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 -- 172.21.15.82:0/1322611000 learned_addr learned my addr 172.21.15.82:0/1322611000 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696f7fe640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69700ce510 0x7f69700ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696ffff640 1 --2- 172.21.15.82:0/1322611000 >> v2:172.21.15.196:3300/0 conn(0x7f697006ec60 0x7f6970142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 -- 172.21.15.82:0/1322611000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69700ce510 msgr2=0x7f69700ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69700ce510 0x7f69700ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 -- 172.21.15.82:0/1322611000 >> v2:172.21.15.196:3300/0 conn(0x7f697006ec60 msgr2=0x7f6970142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:30.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 --2- 172.21.15.82:0/1322611000 >> v2:172.21.15.196:3300/0 conn(0x7f697006ec60 0x7f6970142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 -- 172.21.15.82:0/1322611000 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700ccc70 con 0x7f6970144ca0 2024-04-25T10:08:30.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696effd640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6970144ca0 0x7f697006e720 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f69700cdbc0 tx=0x7f696401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623b3348b05c9976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696e7fc640 1 -- 172.21.15.82:0/1322611000 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640b5070 con 0x7f6970144ca0 2024-04-25T10:08:30.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696e7fc640 1 -- 172.21.15.82:0/1322611000 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69640a89d0 con 0x7f6970144ca0 2024-04-25T10:08:30.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.664+0000 7f696e7fc640 1 -- 172.21.15.82:0/1322611000 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640a8da0 con 0x7f6970144ca0 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1322611000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6970144ca0 msgr2=0x7f697006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6970144ca0 0x7f697006e720 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f69700cdbc0 tx=0x7f696401ed10 comp rx=0 tx=0).stop 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1322611000 shutdown_connections 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1322611000 >> v2:172.21.15.196:3300/0 conn(0x7f697006ec60 0x7f6970142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6970144ca0 0x7f697006e720 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1322611000 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69700ce510 0x7f69700ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1322611000 >> 172.21.15.82:0/1322611000 conn(0x7f69700b11a0 msgr2=0x7f69700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1322611000 shutdown_connections 2024-04-25T10:08:30.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1322611000 wait complete. 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 Processor -- start 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- start start 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f697006ec60 0x7f69701f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6970144ca0 0x7f697013e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700cc900 con 0x7f69700ce510 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697006df20 con 0x7f697006ec60 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69701451f0 con 0x7f6970144ca0 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34522/0 (socket says 172.21.15.82:34522) 2024-04-25T10:08:30.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 -- 172.21.15.82:0/1131464314 learned_addr learned my addr 172.21.15.82:0/1131464314 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 -- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6970144ca0 msgr2=0x7f697013e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6970144ca0 0x7f697013e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 -- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f697006ec60 msgr2=0x7f69701f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696f7fe640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f697006ec60 0x7f69701f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f697006ec60 0x7f69701f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 -- 172.21.15.82:0/1131464314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69701f4810 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f696effd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f696401d3c0 tx=0x7f696401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640b5070 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69640b0a50 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697013d510 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69640b0e20 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697013dad0 con 0x7f69700ce510 2024-04-25T10:08:30.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.668+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697004efa0 con 0x7f69700ce510 2024-04-25T10:08:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.672+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69640c2060 con 0x7f69700ce510 2024-04-25T10:08:30.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.672+0000 7f6957fff640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 0x7f695c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:30.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.672+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6964122440 con 0x7f69700ce510 2024-04-25T10:08:30.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.672+0000 7f696f7fe640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 0x7f695c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:30.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.672+0000 7f696f7fe640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 0x7f695c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69701f6da0 tx=0x7f6960008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:30.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.676+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69701f4810 con 0x7f69700ce510 2024-04-25T10:08:30.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.868+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f69700ccfe0 con 0x7f69700ce510 2024-04-25T10:08:30.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.868+0000 7f6957fff640 1 -- 172.21.15.82:0/1131464314 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6964018870 con 0x7f69700ce510 2024-04-25T10:08:30.874 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:30.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 msgr2=0x7f695c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 0x7f695c069420 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f69701f6da0 tx=0x7f6960008040 comp rx=0 tx=0).stop 2024-04-25T10:08:30.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 msgr2=0x7f697013f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f696401d3c0 tx=0x7f696401bab0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 shutdown_connections 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f695c066f60 0x7f695c069420 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6970144ca0 0x7f697013e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f69700ce510 0x7f697013f7c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 --2- 172.21.15.82:0/1131464314 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f697006ec60 0x7f69701f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 >> 172.21.15.82:0/1131464314 conn(0x7f69700b11a0 msgr2=0x7f6970144230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 shutdown_connections 2024-04-25T10:08:30.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:30.872+0000 7f6975ddd640 1 -- 172.21.15.82:0/1131464314 wait complete. 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:30.891 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:30.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:30.892 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:30.898 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.899 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42159sec preferred_lft 42159sec 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:30.900 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:30.901 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:30.902 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:08:30.902 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:30.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:30.902 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:30.951 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:08:30.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:30.952 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:08:31.002 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:08:31.002 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:31.002 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:08:31.002 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:31.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:31.003 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:08:31.003 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:08:31.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:08:31.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:08:31.004 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:08:31.004 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:08:31.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:31.073 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:08:31.119 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:31.119 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:31.120 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:31.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:31.120 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:08:31.174 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:08:31.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:31.174 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:31.219 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:31.220 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:31.220 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:31.267 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:08:31.325 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:08:31.373 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:08:31.374 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:31.375 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:31.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:31.376 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:08:31.424 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:08:31.424 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:08:31.469 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:08:31.483 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:08:31.533 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:08:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:31.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:31.535 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:08:31.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:08:31.540+0000 7f6add1ce580 -1 init, newargv = 0x561fa805dd80 newargc=15 2024-04-25T10:08:31.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[46800]: starting ceph client 2024-04-25T10:08:31.553 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[46800]: starting fuse 2024-04-25T10:08:32.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:08:32.668 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:08:32.717 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:08:32.718 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:32.719 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:32.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:32.720 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:08:32.768 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:08:32.769 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:08:32.769 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:08:32.770 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:08:32.771 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:08:32.772 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:08:32.772 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:08:32.893 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.46800.asok 2024-04-25T10:08:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:08:32.904 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46800.asok 2024-04-25T10:08:32.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:32.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46800.asok status 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:08:33.061 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "46800", 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7578, 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 7578 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:08:33.062 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3273465034 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3273465034 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.7578 192.168.0.1:0/3273465034", 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/3273465034", 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 49, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 80, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:08:33.063 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:08:33.064 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:08:33.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:33.074 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:08:33.075 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:33.081 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:08:33.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:33.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:33.081 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:33.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:33.134 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T10:08:33.188 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:08:33.188 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T10:08:33.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 Processor -- start 2024-04-25T10:08:33.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- start start 2024-04-25T10:08:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891593540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628915d9050 0x562891666560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628915d9f80 con 0x56289158ebb0 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5628915369f0 con 0x56289158e7d0 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5628915bec00 con 0x5628915d9050 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34546/0 (socket says 172.21.15.82:34546) 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 -- 172.21.15.82:0/2344797439 learned_addr learned my addr 172.21.15.82:0/2344797439 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec93368640 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891593540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec93b69640 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628915d9050 0x562891666560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 -- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628915d9050 msgr2=0x562891666560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628915d9050 0x562891666560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 -- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 msgr2=0x562891593540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891593540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 -- 172.21.15.82:0/2344797439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5628915925a0 con 0x56289158ebb0 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec92b67640 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fec800097d0 tx=0x7fec80009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa92d0566f289be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec91b65640 1 -- 172.21.15.82:0/2344797439 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80017070 con 0x56289158ebb0 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec91b65640 1 -- 172.21.15.82:0/2344797439 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec80006970 con 0x56289158ebb0 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec91b65640 1 -- 172.21.15.82:0/2344797439 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec80006d20 con 0x56289158ebb0 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- 172.21.15.82:0/2344797439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 msgr2=0x562891668c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fec800097d0 tx=0x7fec80009c90 comp rx=0 tx=0).stop 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- 172.21.15.82:0/2344797439 shutdown_connections 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5628915d9050 0x562891666560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56289158ebb0 0x562891668c50 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 --2- 172.21.15.82:0/2344797439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891593540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- 172.21.15.82:0/2344797439 >> 172.21.15.82:0/2344797439 conn(0x5628915baab0 msgr2=0x56289166aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- 172.21.15.82:0/2344797439 shutdown_connections 2024-04-25T10:08:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- 172.21.15.82:0/2344797439 wait complete. 2024-04-25T10:08:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 Processor -- start 2024-04-25T10:08:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.268+0000 7fec9584a080 1 -- start start 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891704880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56289158ebb0 0x562891707600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 0x56289164df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56289166b850 con 0x5628915d9050 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x562891592840 con 0x56289158e7d0 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x562891592ae0 con 0x56289158ebb0 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 0x56289164df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 0x56289164df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34560/0 (socket says 172.21.15.82:34560) 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 -- 172.21.15.82:0/2443382660 learned_addr learned my addr 172.21.15.82:0/2443382660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec92b67640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56289158ebb0 0x562891707600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93368640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891704880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 -- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56289158ebb0 msgr2=0x562891707600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56289158ebb0 0x562891707600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 -- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 msgr2=0x562891704880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891704880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 -- 172.21.15.82:0/2443382660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56289164e890 con 0x5628915d9050 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93368640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56289158e7d0 0x562891704880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93b69640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 0x56289164df30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fec8c006d80 tx=0x7fec8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 -- 172.21.15.82:0/2443382660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c017070 con 0x5628915d9050 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 -- 172.21.15.82:0/2443382660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec8c00ce30 con 0x5628915d9050 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56289164eb30 con 0x5628915d9050 2024-04-25T10:08:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 -- 172.21.15.82:0/2443382660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec8c012690 con 0x5628915d9050 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56289164f070 con 0x5628915d9050 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 -- 172.21.15.82:0/2443382660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec8c0013f0 con 0x5628915d9050 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec7c065970 0x7fec7c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec90b63640 1 -- 172.21.15.82:0/2443382660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fec8c0833f0 con 0x5628915d9050 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93368640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec7c065970 0x7fec7c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5628916ffb50 0x56289164f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e80) v9 -- 0x56289164fcf0 con 0x5628916ffb50 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec92b67640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5628916ffb50 0x56289164f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec93368640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec7c065970 0x7fec7c067e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec840ac000 tx=0x7fec840a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.272+0000 7fec92b67640 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5628916ffb50 0x56289164f7b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec92b67640 1 -- 172.21.15.82:0/2443382660 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v80'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fec80006c30 con 0x5628916ffb50 2024-04-25T10:08:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5628916ffb50 msgr2=0x56289164f7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5628916ffb50 0x56289164f7b0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec7c065970 msgr2=0x7fec7c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fec7c065970 0x7fec7c067e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fec840ac000 tx=0x7fec840a5000 comp rx=0 tx=0).stop 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 msgr2=0x56289164df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 --2- 172.21.15.82:0/2443382660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5628915d9050 0x56289164df30 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fec8c006d80 tx=0x7fec8c00c520 comp rx=0 tx=0).stop 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec93b69640 1 -- 172.21.15.82:0/2443382660 reap_dead start 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 shutdown_connections 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 >> 172.21.15.82:0/2443382660 conn(0x5628915baab0 msgr2=0x5628915bdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 shutdown_connections 2024-04-25T10:08:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.276+0000 7fec9584a080 1 -- 172.21.15.82:0/2443382660 wait complete. 2024-04-25T10:08:33.285 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:33.285 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:33.385 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:33.425 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:33.425 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:08:33.425 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:33.431 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 Processor -- start 2024-04-25T10:08:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 -- start start 2024-04-25T10:08:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33dc15a700 0x7f33dc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 0x7f33dc0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33dc0b2090 con 0x7f33dc0ce430 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc15f200 con 0x7f33dc15a700 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e2775640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33dc0b43e0 con 0x7f33dc0b3f50 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33dbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f33dc15a700 0x7f33dc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e0d12640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f33dc0ce430 0x7f33dc0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40572/0 (socket says 172.21.15.82:40572) 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33e0d12640 1 -- 172.21.15.82:0/2072430203 learned_addr learned my addr 172.21.15.82:0/2072430203 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33d3fff640 1 --2- 172.21.15.82:0/2072430203 >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 0x7f33dc0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33dbfff640 1 -- 172.21.15.82:0/2072430203 >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 msgr2=0x7f33dc0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33dbfff640 1 --2- 172.21.15.82:0/2072430203 >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 0x7f33dc0cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33dbfff640 1 -- 172.21.15.82:0/2072430203 >> v1:172.21.15.82:6789/0 conn(0x7f33dc0ce430 legacy=0x7f33dc0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33db7fe640 1 -- 172.21.15.82:0/2072430203 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3391623910 0 0) 0x7f33dc0b2090 con 0x7f33dc0ce430 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33dbfff640 1 -- 172.21.15.82:0/2072430203 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0b46c0 con 0x7f33dc15a700 2024-04-25T10:08:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.592+0000 7f33d3fff640 1 --2- 172.21.15.82:0/2072430203 >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 0x7f33dc0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33dbfff640 1 --2- 172.21.15.82:0/2072430203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc15ecc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0b3e20 tx=0x7f33cc01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b053c30dd52d34f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33db7fe640 1 -- 172.21.15.82:0/2072430203 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0b0070 con 0x7f33dc15a700 2024-04-25T10:08:33.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33db7fe640 1 -- 172.21.15.82:0/2072430203 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0ab600 con 0x7f33dc15a700 2024-04-25T10:08:33.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33db7fe640 1 -- 172.21.15.82:0/2072430203 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0ab9d0 con 0x7f33dc15a700 2024-04-25T10:08:33.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 -- 172.21.15.82:0/2072430203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 msgr2=0x7f33dc15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 --2- 172.21.15.82:0/2072430203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc15ecc0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0b3e20 tx=0x7f33cc01c330 comp rx=0 tx=0).stop 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 -- 172.21.15.82:0/2072430203 shutdown_connections 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 --2- 172.21.15.82:0/2072430203 >> v2:172.21.15.196:3300/0 conn(0x7f33dc0b3f50 0x7f33dc0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 --2- 172.21.15.82:0/2072430203 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc15ecc0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 -- 172.21.15.82:0/2072430203 >> 172.21.15.82:0/2072430203 conn(0x7f33dc0b11a0 msgr2=0x7f33dc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 -- 172.21.15.82:0/2072430203 shutdown_connections 2024-04-25T10:08:33.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.596+0000 7f33e2775640 1 -- 172.21.15.82:0/2072430203 wait complete. 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 Processor -- start 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- start start 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33dc0ce430 0x7f33dc13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 0x7f33dc159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0cd490 con 0x7f33dc15a700 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0cd310 con 0x7f33dc159b00 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33dc0b2bb0 con 0x7f33dc0ce430 2024-04-25T10:08:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34588/0 (socket says 172.21.15.82:34588) 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 -- 172.21.15.82:0/4193766221 learned_addr learned my addr 172.21.15.82:0/4193766221 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33dbfff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33dc0ce430 0x7f33dc13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e0d12640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 0x7f33dc159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 -- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33dc0ce430 msgr2=0x7f33dc13ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33dc0ce430 0x7f33dc13ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 -- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 msgr2=0x7f33dc159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 0x7f33dc159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 -- 172.21.15.82:0/4193766221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc13f9c0 con 0x7f33dc15a700 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e0d12640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 0x7f33dc159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d3fff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f33c400c7f0 tx=0x7f33c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4015070 con 0x7f33dc15a700 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4007820 con 0x7f33dc15a700 2024-04-25T10:08:33.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc0ac980 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4007c10 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33dc0acf40 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33dc04efa0 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.600+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33c4018070 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.604+0000 7f33d97fa640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 0x7f33bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.604+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f33c40852f0 con 0x7f33dc15a700 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.604+0000 7f33dbfff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 0x7f33bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.604+0000 7f33dbfff640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 0x7f33bc069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0ad000 tx=0x7f33cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.608+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33c40561b0 con 0x7f33dc15a700 2024-04-25T10:08:33.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.764+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33dc071f80 con 0x7f33dc15a700 2024-04-25T10:08:33.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33d97fa640 1 -- 172.21.15.82:0/4193766221 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f33c4055ab0 con 0x7f33dc15a700 2024-04-25T10:08:33.783 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:33.783 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 msgr2=0x7f33bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 0x7f33bc069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0ad000 tx=0x7f33cc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 msgr2=0x7f33dc13c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f33c400c7f0 tx=0x7f33c400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 shutdown_connections 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33bc066f60 0x7f33bc069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33dc159b00 0x7f33dc159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33dc15a700 0x7f33dc13c110 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 --2- 172.21.15.82:0/4193766221 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33dc0ce430 0x7f33dc13ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 >> 172.21.15.82:0/4193766221 conn(0x7f33dc0b11a0 msgr2=0x7f33dc1571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 shutdown_connections 2024-04-25T10:08:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.768+0000 7f33e2775640 1 -- 172.21.15.82:0/4193766221 wait complete. 2024-04-25T10:08:33.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:33.792 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:33.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.904+0000 7f5c703d0640 1 Processor -- start 2024-04-25T10:08:33.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.904+0000 7f5c703d0640 1 -- start start 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c703d0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5c680cba90 0x7f5c680cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c703d0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c68071b70 con 0x7f5c68159bd0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c703d0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c68071e50 con 0x7f5c68159fb0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c703d0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c68072780 con 0x7f5c680cba90 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e16c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5c68159bd0 0x7f5c680c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40574/0 (socket says 172.21.15.82:40574) 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e16c640 1 -- 172.21.15.82:0/2805572976 learned_addr learned my addr 172.21.15.82:0/2805572976 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e96d640 1 --2- 172.21.15.82:0/2805572976 >> v2:172.21.15.196:3300/0 conn(0x7f5c680cba90 0x7f5c680cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333615301 0 0) 0x7f5c68071b70 con 0x7f5c68159bd0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c54003530 con 0x7f5c68159bd0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363173705 0 0) 0x7f5c54003530 con 0x7f5c68159bd0 2024-04-25T10:08:33.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 >> v2:172.21.15.196:3300/0 conn(0x7f5c680cba90 msgr2=0x7f5c680cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 --2- 172.21.15.82:0/2805572976 >> v2:172.21.15.196:3300/0 conn(0x7f5c680cba90 0x7f5c680cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 >> v1:172.21.15.196:6789/0 conn(0x7f5c68159fb0 legacy=0x7f5c680cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68072130 con 0x7f5c68159bd0 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5c60001680 con 0x7f5c68159bd0 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 >> v1:172.21.15.82:6789/0 conn(0x7f5c68159bd0 legacy=0x7f5c680c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c54003a30 0x7f5c54003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 0x7f5c54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c540051d0 0x7f5c5400cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68072780 con 0x7f5c54004810 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5400d210 con 0x7f5c540051d0 2024-04-25T10:08:33.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5400d410 con 0x7f5c54003a30 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 0x7f5c54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e96d640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c540051d0 0x7f5c5400cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e16c640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c54003a30 0x7f5c54003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 -- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c54003a30 msgr2=0x7f5c54003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c54003a30 0x7f5c54003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 -- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c540051d0 msgr2=0x7f5c5400cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c540051d0 0x7f5c5400cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e96d640 1 -- 172.21.15.82:0/2805572976 reap_dead start 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6d96b640 1 -- 172.21.15.82:0/2805572976 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c54003530 con 0x7f5c54004810 2024-04-25T10:08:33.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.908+0000 7f5c6e16c640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c54003a30 0x7f5c54003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 0x7f5c54004c90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c001e40 tx=0x7f5c5c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a22634086d62cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c001a90 con 0x7f5c54004810 2024-04-25T10:08:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c003b80 con 0x7f5c54004810 2024-04-25T10:08:33.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d16a640 1 -- 172.21.15.82:0/2805572976 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c005080 con 0x7f5c54004810 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 msgr2=0x7f5c54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 0x7f5c54004c90 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c001e40 tx=0x7f5c5c01c740 comp rx=0 tx=0).stop 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- 172.21.15.82:0/2805572976 shutdown_connections 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 --2- 172.21.15.82:0/2805572976 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c54004810 0x7f5c54004c90 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- 172.21.15.82:0/2805572976 >> 172.21.15.82:0/2805572976 conn(0x7f5c6806ae60 msgr2=0x7f5c6806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- 172.21.15.82:0/2805572976 shutdown_connections 2024-04-25T10:08:33.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- 172.21.15.82:0/2805572976 wait complete. 2024-04-25T10:08:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 Processor -- start 2024-04-25T10:08:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- start start 2024-04-25T10:08:33.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c54004810 0x7f5c680c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c680b1950 0x7f5c680b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680cc0c0 con 0x7f5c68070f50 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680cbf40 con 0x7f5c54004810 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c703d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680cc240 con 0x7f5c680b1950 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34596/0 (socket says 172.21.15.82:34596) 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 -- 172.21.15.82:0/3404093834 learned_addr learned my addr 172.21.15.82:0/3404093834 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6e16c640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c54004810 0x7f5c680c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6e96d640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c680b1950 0x7f5c680b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 -- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c680b1950 msgr2=0x7f5c680b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c680b1950 0x7f5c680b1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 -- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c54004810 msgr2=0x7f5c680c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:33.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c54004810 0x7f5c680c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.912+0000 7f5c6d96b640 1 -- 172.21.15.82:0/3404093834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c680b36a0 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c6d96b640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c01c570 tx=0x7f5c54005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c006d60 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c005030 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c680c64d0 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0053e0 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c680c6770 con 0x7f5c68070f50 2024-04-25T10:08:33.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c680b2310 con 0x7f5c68070f50 2024-04-25T10:08:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c5c005580 con 0x7f5c68070f50 2024-04-25T10:08:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 0x7f5c4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5c5c11f3d0 con 0x7f5c68070f50 2024-04-25T10:08:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c6e16c640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 0x7f5c4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:33.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.916+0000 7f5c6e16c640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 0x7f5c4c069450 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5c60000c10 tx=0x7f5c60001440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:33.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:33.920+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c5c0b5030 con 0x7f5c68070f50 2024-04-25T10:08:34.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c680732f0 con 0x7f5c68070f50 2024-04-25T10:08:34.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c5affd640 1 -- 172.21.15.82:0/3404093834 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c5c0ac020 con 0x7f5c68070f50 2024-04-25T10:08:34.081 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 msgr2=0x7f5c4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 0x7f5c4c069450 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5c60000c10 tx=0x7f5c60001440 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 msgr2=0x7f5c68098920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.076+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c01c570 tx=0x7f5c54005840 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 shutdown_connections 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c4c066f90 0x7f5c4c069450 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c680b1950 0x7f5c680b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68070f50 0x7f5c68098920 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 --2- 172.21.15.82:0/3404093834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c54004810 0x7f5c680c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 >> 172.21.15.82:0/3404093834 conn(0x7f5c6806ae60 msgr2=0x7f5c680acdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 shutdown_connections 2024-04-25T10:08:34.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.080+0000 7f5c703d0640 1 -- 172.21.15.82:0/3404093834 wait complete. 2024-04-25T10:08:34.098 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:34.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 Processor -- start 2024-04-25T10:08:34.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 -- start start 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9240d1cd0 0x7ff9240d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9240d0620 0x7ff9240ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92406c440 con 0x7ff9240d1cd0 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92406be60 con 0x7ff9240d0620 2024-04-25T10:08:34.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9296b8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92415de60 con 0x7ff92415da80 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9240d1cd0 0x7ff9240d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff9240d1cd0 0x7ff9240d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34602/0 (socket says 172.21.15.82:34602) 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 -- 172.21.15.82:0/3252226866 learned_addr learned my addr 172.21.15.82:0/3252226866 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff9237fe640 1 --2- 172.21.15.82:0/3252226866 >> v2:172.21.15.196:3300/0 conn(0x7ff9240d0620 0x7ff9240ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff922ffd640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 -- 172.21.15.82:0/3252226866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 msgr2=0x7ff9240d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 -- 172.21.15.82:0/3252226866 >> v2:172.21.15.196:3300/0 conn(0x7ff9240d0620 msgr2=0x7ff9240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3252226866 >> v2:172.21.15.196:3300/0 conn(0x7ff9240d0620 0x7ff9240ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 -- 172.21.15.82:0/3252226866 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240d2450 con 0x7ff9240d1cd0 2024-04-25T10:08:34.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.216+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff9240d00e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff92406de60 tx=0x7ff90c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06f5a60db7339be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9227fc640 1 -- 172.21.15.82:0/3252226866 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0ab5f0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9227fc640 1 -- 172.21.15.82:0/3252226866 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c0b0070 con 0x7ff9240d1cd0 2024-04-25T10:08:34.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9227fc640 1 -- 172.21.15.82:0/3252226866 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0ab9b0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3252226866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 msgr2=0x7ff9240d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff9240d00e0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff92406de60 tx=0x7ff90c01c520 comp rx=0 tx=0).stop 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3252226866 shutdown_connections 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3252226866 >> v2:172.21.15.196:3300/0 conn(0x7ff9240d0620 0x7ff9240ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff9240d00e0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3252226866 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3252226866 >> 172.21.15.82:0/3252226866 conn(0x7ff92408b000 msgr2=0x7ff9240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3252226866 shutdown_connections 2024-04-25T10:08:34.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3252226866 wait complete. 2024-04-25T10:08:34.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 Processor -- start 2024-04-25T10:08:34.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- start start 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9240d0620 0x7ff924157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92406ca30 con 0x7ff9240d1cd0 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92406c8b0 con 0x7ff9240d0620 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92406cbb0 con 0x7ff92415da80 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34616/0 (socket says 172.21.15.82:34616) 2024-04-25T10:08:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 -- 172.21.15.82:0/3486685837 learned_addr learned my addr 172.21.15.82:0/3486685837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 -- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 msgr2=0x7ff9240c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 -- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9240d0620 msgr2=0x7ff924157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9240d0620 0x7ff924157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 -- 172.21.15.82:0/3486685837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924157a60 con 0x7ff9240d1cd0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91bfff640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff90c01cdc0 tx=0x7ff90c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c003ab0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90c003c50 con 0x7ff9240d1cd0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9240b6440 con 0x7ff9240d1cd0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90c0bd400 con 0x7ff9240d1cd0 2024-04-25T10:08:34.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.220+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9240b6a00 con 0x7ff9240d1cd0 2024-04-25T10:08:34.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92404efa0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff90c0bd5a0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff91b7fe640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff90c11f7b0 con 0x7ff9240d1cd0 2024-04-25T10:08:34.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff922ffd640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.224+0000 7ff922ffd640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 0x7ff904069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff92415ae50 tx=0x7ff914008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.228+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff924157a60 con 0x7ff9240d1cd0 2024-04-25T10:08:34.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.384+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff924158940 con 0x7ff9240d1cd0 2024-04-25T10:08:34.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.384+0000 7ff91b7fe640 1 -- 172.21.15.82:0/3486685837 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff90c0b9030 con 0x7ff9240d1cd0 2024-04-25T10:08:34.390 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 msgr2=0x7ff904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 0x7ff904069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff92415ae50 tx=0x7ff914008040 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 msgr2=0x7ff924158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff90c01cdc0 tx=0x7ff90c0192c0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 shutdown_connections 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff904066f60 0x7ff904069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff92415da80 0x7ff9240c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9240d1cd0 0x7ff924158400 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 --2- 172.21.15.82:0/3486685837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9240d0620 0x7ff924157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 >> 172.21.15.82:0/3486685837 conn(0x7ff92408b000 msgr2=0x7ff924160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 shutdown_connections 2024-04-25T10:08:34.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.388+0000 7ff9296b8640 1 -- 172.21.15.82:0/3486685837 wait complete. 2024-04-25T10:08:34.408 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:34.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.520+0000 7f8ba4f5e640 1 Processor -- start 2024-04-25T10:08:34.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.520+0000 7f8ba4f5e640 1 -- start start 2024-04-25T10:08:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8ba4f5e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ba015da80 0x7f8ba00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8ba4f5e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ba00d1cd0 0x7f8ba00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8ba4f5e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ba006be60 con 0x7f8ba00d0620 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8ba4f5e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba00d0a70 con 0x7f8ba015da80 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8ba4f5e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ba006c440 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ba015da80 0x7f8ba00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9ed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8ba00d0620 0x7f8ba00ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:40590/0 (socket says 172.21.15.82:40590) 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9ed9d640 1 -- 172.21.15.82:0/3575482201 learned_addr learned my addr 172.21.15.82:0/3575482201 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9dd9b640 1 --2- 172.21.15.82:0/3575482201 >> v2:172.21.15.196:3300/0 conn(0x7f8ba00d1cd0 0x7f8ba00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9d59a640 1 -- 172.21.15.82:0/3575482201 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2835092946 0 0) 0x7f8ba006be60 con 0x7f8ba00d0620 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9d59a640 1 -- 172.21.15.82:0/3575482201 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b88003530 con 0x7f8ba00d0620 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 -- 172.21.15.82:0/3575482201 >> v2:172.21.15.196:3300/0 conn(0x7f8ba00d1cd0 msgr2=0x7f8ba00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 --2- 172.21.15.82:0/3575482201 >> v2:172.21.15.196:3300/0 conn(0x7f8ba00d1cd0 0x7f8ba00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 -- 172.21.15.82:0/3575482201 >> v1:172.21.15.82:6789/0 conn(0x7f8ba00d0620 legacy=0x7f8ba00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 -- 172.21.15.82:0/3575482201 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba015de60 con 0x7f8ba015da80 2024-04-25T10:08:34.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9e59c640 1 --2- 172.21.15.82:0/3575482201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba015da80 0x7f8ba00d4230 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8b900097c0 tx=0x7f8b9001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7bbcfb4d7b1b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9d59a640 1 -- 172.21.15.82:0/3575482201 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b900b0070 con 0x7f8ba015da80 2024-04-25T10:08:34.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9d59a640 1 -- 172.21.15.82:0/3575482201 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b900ab600 con 0x7f8ba015da80 2024-04-25T10:08:34.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.524+0000 7f8b9d59a640 1 -- 172.21.15.82:0/3575482201 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b900ab9d0 con 0x7f8ba015da80 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/3575482201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba015da80 msgr2=0x7f8ba00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/3575482201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba015da80 0x7f8ba00d4230 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8b900097c0 tx=0x7f8b9001c460 comp rx=0 tx=0).stop 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/3575482201 shutdown_connections 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/3575482201 >> v2:172.21.15.196:3300/0 conn(0x7f8ba00d1cd0 0x7f8ba00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/3575482201 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba015da80 0x7f8ba00d4230 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/3575482201 >> 172.21.15.82:0/3575482201 conn(0x7f8ba008b000 msgr2=0x7f8ba00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/3575482201 shutdown_connections 2024-04-25T10:08:34.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/3575482201 wait complete. 2024-04-25T10:08:34.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 Processor -- start 2024-04-25T10:08:34.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- start start 2024-04-25T10:08:34.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00d0620 0x7f8ba00c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba015da80 0x7f8ba00b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba016a730 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba016a5b0 con 0x7f8ba015da80 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba016a8b0 con 0x7f8ba00d0620 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34630/0 (socket says 172.21.15.82:34630) 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 -- 172.21.15.82:0/2576925191 learned_addr learned my addr 172.21.15.82:0/2576925191 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 -- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00d0620 msgr2=0x7f8ba00c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00d0620 0x7f8ba00c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9ed9d640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba015da80 0x7f8ba00b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 -- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba015da80 msgr2=0x7f8ba00b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba015da80 0x7f8ba00b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 -- 172.21.15.82:0/2576925191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba00ca960 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b9dd9b640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400d800 tx=0x7f8b9400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b940059a0 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b94005b40 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ba00b8ee0 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94005d30 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ba00b8b00 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.528+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ba004efa0 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.532+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b9401b020 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.532+0000 7f8b8effd640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.532+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8b94085030 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.532+0000 7f8b9e59c640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.532+0000 7f8b9e59c640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 0x7f8b80069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8b900ad000 tx=0x7f8b90004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.536+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b94055e60 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.680+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ba006ad40 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8b8effd640 1 -- 172.21.15.82:0/2576925191 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b94055760 con 0x7f8ba00d1cd0 2024-04-25T10:08:34.688 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 msgr2=0x7f8b80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 0x7f8b80069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8b900ad000 tx=0x7f8b90004740 comp rx=0 tx=0).stop 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 msgr2=0x7f8ba00b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400d800 tx=0x7f8b9400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 shutdown_connections 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b80066f60 0x7f8b80069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ba015da80 0x7f8ba00b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ba00d1cd0 0x7f8ba00b51c0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 --2- 172.21.15.82:0/2576925191 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ba00d0620 0x7f8ba00c9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 >> 172.21.15.82:0/2576925191 conn(0x7f8ba008b000 msgr2=0x7f8ba00b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 shutdown_connections 2024-04-25T10:08:34.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.684+0000 7f8ba4f5e640 1 -- 172.21.15.82:0/2576925191 wait complete. 2024-04-25T10:08:34.705 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:08:34.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 Processor -- start 2024-04-25T10:08:34.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 -- start start 2024-04-25T10:08:34.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f972815b950 0x7f972809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f97280cfb80 0x7f97280cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97280d04d0 con 0x7f97280cf420 2024-04-25T10:08:34.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972815bd30 con 0x7f972815b950 2024-04-25T10:08:34.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f972e9c3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97280cc0d0 con 0x7f97280cfb80 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f9727fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f972815b950 0x7f972809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f9727fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f972815b950 0x7f972809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41044/0 (socket says 172.21.15.82:41044) 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.820+0000 7f9727fff640 1 -- 172.21.15.82:0/2755734273 learned_addr learned my addr 172.21.15.82:0/2755734273 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972cf60640 1 --2- 172.21.15.82:0/2755734273 >> v2:172.21.15.196:3300/0 conn(0x7f97280cfb80 0x7f97280cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9727fff640 1 -- 172.21.15.82:0/2755734273 >> v2:172.21.15.196:3300/0 conn(0x7f97280cfb80 msgr2=0x7f97280cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9727fff640 1 --2- 172.21.15.82:0/2755734273 >> v2:172.21.15.196:3300/0 conn(0x7f97280cfb80 0x7f97280cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9727fff640 1 -- 172.21.15.82:0/2755734273 >> v1:172.21.15.196:6789/0 conn(0x7f97280cf420 legacy=0x7f97280cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:34.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9727fff640 1 -- 172.21.15.82:0/2755734273 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972809ab10 con 0x7f972815b950 2024-04-25T10:08:34.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9727fff640 1 --2- 172.21.15.82:0/2755734273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972815b950 0x7f972809a480 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f97280b3bd0 tx=0x7f971801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171006cade1c30fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9726ffd640 1 -- 172.21.15.82:0/2755734273 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180b0070 con 0x7f972815b950 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9726ffd640 1 -- 172.21.15.82:0/2755734273 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180ab600 con 0x7f972815b950 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f9726ffd640 1 -- 172.21.15.82:0/2755734273 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180ab9d0 con 0x7f972815b950 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 -- 172.21.15.82:0/2755734273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972815b950 msgr2=0x7f972809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 --2- 172.21.15.82:0/2755734273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972815b950 0x7f972809a480 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f97280b3bd0 tx=0x7f971801c300 comp rx=0 tx=0).stop 2024-04-25T10:08:34.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 -- 172.21.15.82:0/2755734273 shutdown_connections 2024-04-25T10:08:34.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 --2- 172.21.15.82:0/2755734273 >> v2:172.21.15.196:3300/0 conn(0x7f97280cfb80 0x7f97280cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 --2- 172.21.15.82:0/2755734273 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f972815b950 0x7f972809a480 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 -- 172.21.15.82:0/2755734273 >> 172.21.15.82:0/2755734273 conn(0x7f97280b30b0 msgr2=0x7f97280b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:34.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 -- 172.21.15.82:0/2755734273 shutdown_connections 2024-04-25T10:08:34.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.824+0000 7f972e9c3640 1 -- 172.21.15.82:0/2755734273 wait complete. 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 Processor -- start 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- start start 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 0x7f972814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f972815b950 0x7f9728155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972806faa0 con 0x7f97280cf420 2024-04-25T10:08:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97280cf8b0 con 0x7f97280cfb80 2024-04-25T10:08:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972806fc20 con 0x7f972815b950 2024-04-25T10:08:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41060/0 (socket says 172.21.15.82:41060) 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 -- 172.21.15.82:0/150196804 learned_addr learned my addr 172.21.15.82:0/150196804 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f97277fe640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 0x7f972814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972cf60640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f972815b950 0x7f9728155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 -- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f972815b950 msgr2=0x7f9728155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f972815b950 0x7f9728155b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 -- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 msgr2=0x7f972814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 0x7f972814fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 -- 172.21.15.82:0/150196804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9728150100 con 0x7f97280cf420 2024-04-25T10:08:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f97277fe640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 0x7f972814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9727fff640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f9718019f30 tx=0x7f97180190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180b9070 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180b4400 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9728152b60 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97180b0040 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9728151200 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972804efa0 con 0x7f97280cf420 2024-04-25T10:08:34.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.828+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97180bc070 con 0x7f97280cf420 2024-04-25T10:08:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.832+0000 7f9724ff9640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 0x7f9714069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.832+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f971811e1e0 con 0x7f97280cf420 2024-04-25T10:08:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.832+0000 7f97277fe640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 0x7f9714069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:34.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.832+0000 7f97277fe640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 0x7f9714069450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9728099e50 tx=0x7f971c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:34.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:34.836+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97280d04d0 con 0x7f97280cf420 2024-04-25T10:08:35.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.004+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f97280cd840 con 0x7f97280cf420 2024-04-25T10:08:35.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.004+0000 7f9724ff9640 1 -- 172.21.15.82:0/150196804 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v80) v1 ==== 80+0+120 (secure 0 0 0) 0x7f97180ef010 con 0x7f97280cf420 2024-04-25T10:08:35.011 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:08:35.011 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/910515233 2024-04-26T10:07:09.947207+0000 2024-04-25T10:08:35.011 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:08:35.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 msgr2=0x7f9714069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 0x7f9714069450 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9728099e50 tx=0x7f971c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 msgr2=0x7f97281522a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f9718019f30 tx=0x7f97180190a0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 shutdown_connections 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9714066f90 0x7f9714069450 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f972815b950 0x7f9728155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97280cfb80 0x7f972814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 --2- 172.21.15.82:0/150196804 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97280cf420 0x7f97281522a0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 >> 172.21.15.82:0/150196804 conn(0x7f97280b30b0 msgr2=0x7f97280ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 shutdown_connections 2024-04-25T10:08:35.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.008+0000 7f972e9c3640 1 -- 172.21.15.82:0/150196804 wait complete. 2024-04-25T10:08:35.029 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:08:35.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:35.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[46800]: fuse finished with error 0 and tester_r 0 2024-04-25T10:08:35.041 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:08:35.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.041 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:08:35.141 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:08:35.142 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:35.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:35.149 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:08:35.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.149 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:08:35.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:35.198 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:08:35.198 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:08:35.198 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:35.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:35.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:08:35.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.205 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:08:35.252 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:08:35.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:35.253 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:08:35.253 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:35.259 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:35.259 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:08:35.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:35.260 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:08:35.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:08:35.309 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:08:35.309 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 Processor -- start 2024-04-25T10:08:35.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 -- start start 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 0x7f71f809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f71f80cf420 0x7f71f80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f815bd30 con 0x7f71f80cfb80 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f80cc0d0 con 0x7f71f80cf420 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fec05640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f809ab10 con 0x7f71f815b950 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fd1a2640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71f80cfb80 0x7f71f806f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43142/0 (socket says 172.21.15.82:43142) 2024-04-25T10:08:35.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f71f80cf420 0x7f71f80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fd1a2640 1 -- 172.21.15.82:0/1544707742 learned_addr learned my addr 172.21.15.82:0/1544707742 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f7fff640 1 -- 172.21.15.82:0/1544707742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 msgr2=0x7f71f809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fc9a1640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 0x7f71f809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f7fff640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 0x7f71f809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f7fff640 1 -- 172.21.15.82:0/1544707742 >> v1:172.21.15.82:6789/0 conn(0x7f71f80cfb80 legacy=0x7f71f806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f7fff640 1 -- 172.21.15.82:0/1544707742 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80cc370 con 0x7f71f80cf420 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71f77fe640 1 -- 172.21.15.82:0/1544707742 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704976204 0 0) 0x7f71f815bd30 con 0x7f71f80cfb80 2024-04-25T10:08:35.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.432+0000 7f71fc9a1640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 0x7f71f809a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cf420 0x7f71f80cce50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097c0 tx=0x7f71e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0fdc8c90735c2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f77fe640 1 -- 172.21.15.82:0/1544707742 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b2070 con 0x7f71f80cf420 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f77fe640 1 -- 172.21.15.82:0/1544707742 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e80ad680 con 0x7f71f80cf420 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f77fe640 1 -- 172.21.15.82:0/1544707742 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80ada50 con 0x7f71f80cf420 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- 172.21.15.82:0/1544707742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cf420 msgr2=0x7f71f80cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cf420 0x7f71f80cce50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097c0 tx=0x7f71e801c470 comp rx=0 tx=0).stop 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- 172.21.15.82:0/1544707742 shutdown_connections 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cf420 0x7f71f80cce50 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- 172.21.15.82:0/1544707742 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f815b950 0x7f71f809a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- 172.21.15.82:0/1544707742 >> 172.21.15.82:0/1544707742 conn(0x7f71f80b30b0 msgr2=0x7f71f80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- 172.21.15.82:0/1544707742 shutdown_connections 2024-04-25T10:08:35.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- 172.21.15.82:0/1544707742 wait complete. 2024-04-25T10:08:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 Processor -- start 2024-04-25T10:08:35.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- start start 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f80cf420 0x7f71f80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71f815b950 0x7f71f80c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80cd390 con 0x7f71f80cfb80 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80d03e0 con 0x7f71f815b950 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71fec05640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80b5780 con 0x7f71f80cf420 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41090/0 (socket says 172.21.15.82:41090) 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 -- 172.21.15.82:0/3296428290 learned_addr learned my addr 172.21.15.82:0/3296428290 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:35.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 -- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f80cf420 msgr2=0x7f71f80c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f80cf420 0x7f71f80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 -- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71f815b950 msgr2=0x7f71f80c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71f815b950 0x7f71f80c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.436+0000 7f71f7fff640 1 -- 172.21.15.82:0/3296428290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f80b5a60 con 0x7f71f80cfb80 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f7fff640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f71e80b0040 tx=0x7f71e8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b2040 con 0x7f71f80cfb80 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e80adbf0 con 0x7f71f80cfb80 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f80b5d40 con 0x7f71f80cfb80 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004020 con 0x7f71f80cfb80 2024-04-25T10:08:35.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f80b6300 con 0x7f71f80cfb80 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f804efa0 con 0x7f71f80cfb80 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71e80bf060 con 0x7f71f80cfb80 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 0x7f71e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f71e811e790 con 0x7f71f80cfb80 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71fc9a1640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 0x7f71e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.440+0000 7f71fc9a1640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 0x7f71e4069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00a890 tx=0x7f71ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.444+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71e8016090 con 0x7f71f80cfb80 2024-04-25T10:08:35.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f80d13a0 con 0x7f71f80cfb80 2024-04-25T10:08:35.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71f57fa640 1 -- 172.21.15.82:0/3296428290 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f71e80ef5a0 con 0x7f71f80cfb80 2024-04-25T10:08:35.637 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:35.637 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:08:00.987254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:35.638 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 msgr2=0x7f71e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 0x7f71e4069420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00a890 tx=0x7f71ec008040 comp rx=0 tx=0).stop 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 msgr2=0x7f71f80cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f71e80b0040 tx=0x7f71e8019210 comp rx=0 tx=0).stop 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 shutdown_connections 2024-04-25T10:08:35.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71e4066f60 0x7f71e4069420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71f815b950 0x7f71f80c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71f80cfb80 0x7f71f80cb160 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 --2- 172.21.15.82:0/3296428290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71f80cf420 0x7f71f80c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 >> 172.21.15.82:0/3296428290 conn(0x7f71f80b30b0 msgr2=0x7f71f80b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 shutdown_connections 2024-04-25T10:08:35.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.632+0000 7f71fec05640 1 -- 172.21.15.82:0/3296428290 wait complete. 2024-04-25T10:08:35.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:35.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.768+0000 7fa0cd431640 1 Processor -- start 2024-04-25T10:08:35.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.768+0000 7fa0cd431640 1 -- start start 2024-04-25T10:08:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0cd431640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c8144ca0 0x7fa0c806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0cd431640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c80ccc70 con 0x7fa0c806ec60 2024-04-25T10:08:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0cd431640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c806dc40 con 0x7fa0c80ce510 2024-04-25T10:08:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0cd431640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c80cc2b0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c8144ca0 0x7fa0c806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0c8144ca0 0x7fa0c806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41096/0 (socket says 172.21.15.82:41096) 2024-04-25T10:08:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 -- 172.21.15.82:0/2212642034 learned_addr learned my addr 172.21.15.82:0/2212642034 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 -- 172.21.15.82:0/2212642034 >> v1:172.21.15.196:6789/0 conn(0x7fa0c80ce510 legacy=0x7fa0c80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 -- 172.21.15.82:0/2212642034 >> v1:172.21.15.82:6789/0 conn(0x7fa0c806ec60 legacy=0x7fa0c8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 -- 172.21.15.82:0/2212642034 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c806df20 con 0x7fa0c8144ca0 2024-04-25T10:08:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c67fc640 1 --2- 172.21.15.82:0/2212642034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c806e720 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009870 tx=0x7fa0bc01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56411566bcc543d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2212642034 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0b5070 con 0x7fa0c8144ca0 2024-04-25T10:08:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2212642034 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc0a8a50 con 0x7fa0c8144ca0 2024-04-25T10:08:35.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.772+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2212642034 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0a8e20 con 0x7fa0c8144ca0 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- 172.21.15.82:0/2212642034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 msgr2=0x7fa0c806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 --2- 172.21.15.82:0/2212642034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c806e720 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009870 tx=0x7fa0bc01ed50 comp rx=0 tx=0).stop 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- 172.21.15.82:0/2212642034 shutdown_connections 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 --2- 172.21.15.82:0/2212642034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c806e720 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- 172.21.15.82:0/2212642034 >> 172.21.15.82:0/2212642034 conn(0x7fa0c80b11a0 msgr2=0x7fa0c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- 172.21.15.82:0/2212642034 shutdown_connections 2024-04-25T10:08:35.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- 172.21.15.82:0/2212642034 wait complete. 2024-04-25T10:08:35.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 Processor -- start 2024-04-25T10:08:35.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- start start 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ec60 0x7fa0c81f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c80ce510 0x7fa0c80c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8142d90 con 0x7fa0c8144ca0 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c806e290 con 0x7fa0c80ce510 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0cd431640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c8142f10 con 0x7fa0c806ec60 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41108/0 (socket says 172.21.15.82:41108) 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c67fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c80ce510 0x7fa0c80c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c77fe640 1 -- 172.21.15.82:0/3802524665 learned_addr learned my addr 172.21.15.82:0/3802524665 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c6ffd640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ec60 0x7fa0c81f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c77fe640 1 -- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ec60 msgr2=0x7fa0c81f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.776+0000 7fa0c77fe640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ec60 0x7fa0c81f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c77fe640 1 -- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c80ce510 msgr2=0x7fa0c80c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c77fe640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c80ce510 0x7fa0c80c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c77fe640 1 -- 172.21.15.82:0/3802524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c813c6e0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c77fe640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c7a0 tx=0x7fa0c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0003b80 con 0x7fa0c8144ca0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c0003d20 con 0x7fa0c8144ca0 2024-04-25T10:08:35.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c000aa90 con 0x7fa0c8144ca0 2024-04-25T10:08:35.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c80c4b40 con 0x7fa0c8144ca0 2024-04-25T10:08:35.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c80ad130 con 0x7fa0c8144ca0 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c804efa0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0c000ac30 con 0x7fa0c8144ca0 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 0x7fa0b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa0c0085010 con 0x7fa0c8144ca0 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c6ffd640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 0x7fa0b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:35.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.780+0000 7fa0c6ffd640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 0x7fa0b4069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8001650 tx=0x7fa0b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:35.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.784+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0c813c6e0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.944+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0c80b1cc0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.944+0000 7fa0af7fe640 1 -- 172.21.15.82:0/3802524665 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7fa0c0055ed0 con 0x7fa0c8144ca0 2024-04-25T10:08:35.951 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:35.951 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:35.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 msgr2=0x7fa0b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 0x7fa0b4069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8001650 tx=0x7fa0b8009040 comp rx=0 tx=0).stop 2024-04-25T10:08:35.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 msgr2=0x7fa0c813c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fa0c000c7a0 tx=0x7fa0c000cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 shutdown_connections 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0b4066f60 0x7fa0b4069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0c8144ca0 0x7fa0c813c1a0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0c80ce510 0x7fa0c80c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 --2- 172.21.15.82:0/3802524665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0c806ec60 0x7fa0c81f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 >> 172.21.15.82:0/3802524665 conn(0x7fa0c80b11a0 msgr2=0x7fa0c8141af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 shutdown_connections 2024-04-25T10:08:35.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:35.948+0000 7fa0cd431640 1 -- 172.21.15.82:0/3802524665 wait complete. 2024-04-25T10:08:35.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:44.746226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:45.417744+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:35.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.092+0000 7f54ae0e4640 1 Processor -- start 2024-04-25T10:08:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.092+0000 7f54ae0e4640 1 -- start start 2024-04-25T10:08:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54ae0e4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54a815b6b0 0x7f54a80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54ae0e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 0x7f54a80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54ae0e4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54a80cbd20 con 0x7f54a80cf590 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54ae0e4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54a815ba90 con 0x7f54a815b6b0 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54ae0e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80cc000 con 0x7f54a8073010 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54a815b6b0 0x7f54a80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f54a815b6b0 0x7f54a80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41120/0 (socket says 172.21.15.82:41120) 2024-04-25T10:08:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 -- 172.21.15.82:0/2002780522 learned_addr learned my addr 172.21.15.82:0/2002780522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a7fff640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 0x7f54a80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 -- 172.21.15.82:0/2002780522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 msgr2=0x7f54a80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 0x7f54a80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 -- 172.21.15.82:0/2002780522 >> v1:172.21.15.196:6789/0 conn(0x7f54a80cf590 legacy=0x7f54a815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 -- 172.21.15.82:0/2002780522 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80cc2e0 con 0x7f54a815b6b0 2024-04-25T10:08:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a7fff640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 0x7f54a80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:36.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a77fe640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a815b6b0 0x7f54a80ca3c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5498009870 tx=0x7f549801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d081937293f2063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a67fc640 1 -- 172.21.15.82:0/2002780522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980b0070 con 0x7f54a815b6b0 2024-04-25T10:08:36.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a67fc640 1 -- 172.21.15.82:0/2002780522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54980ab600 con 0x7f54a815b6b0 2024-04-25T10:08:36.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.096+0000 7f54a67fc640 1 -- 172.21.15.82:0/2002780522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54980ab9d0 con 0x7f54a815b6b0 2024-04-25T10:08:36.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/2002780522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a815b6b0 msgr2=0x7f54a80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a815b6b0 0x7f54a80ca3c0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f5498009870 tx=0x7f549801c560 comp rx=0 tx=0).stop 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/2002780522 shutdown_connections 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a8073010 0x7f54a80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/2002780522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a815b6b0 0x7f54a80ca3c0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/2002780522 >> 172.21.15.82:0/2002780522 conn(0x7f54a806ae60 msgr2=0x7f54a806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/2002780522 shutdown_connections 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/2002780522 wait complete. 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 Processor -- start 2024-04-25T10:08:36.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- start start 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a8073010 0x7f54a8154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 0x7f54a806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80cadb0 con 0x7f54a80cf590 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80ca900 con 0x7f54a8073010 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a80caf30 con 0x7f54a815b6b0 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41136/0 (socket says 172.21.15.82:41136) 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 -- 172.21.15.82:0/596285342 learned_addr learned my addr 172.21.15.82:0/596285342 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a77fe640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a8073010 0x7f54a8154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a7fff640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 0x7f54a806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 -- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 msgr2=0x7f54a806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 0x7f54a806dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 -- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a8073010 msgr2=0x7f54a8154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a8073010 0x7f54a8154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 -- 172.21.15.82:0/596285342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a8155510 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a7fff640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 0x7f54a806dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54a6ffd640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c500 tx=0x7f549c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c016020 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549c005220 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a80c5900 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549c007030 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.100+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a81557d0 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a804ef80 con 0x7f54a80cf590 2024-04-25T10:08:36.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f549c0071f0 con 0x7f54a80cf590 2024-04-25T10:08:36.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f548ffff640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 0x7f5494069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f549c084cc0 con 0x7f54a80cf590 2024-04-25T10:08:36.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f54a77fe640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 0x7f5494069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.104+0000 7f54a77fe640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 0x7f5494069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f54980ad000 tx=0x7f54980a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.108+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f549c020020 con 0x7f54a80cf590 2024-04-25T10:08:36.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.260+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54a8073b80 con 0x7f54a80cf590 2024-04-25T10:08:36.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f548ffff640 1 -- 172.21.15.82:0/596285342 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f549c055a60 con 0x7f54a80cf590 2024-04-25T10:08:36.268 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 msgr2=0x7f5494069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 0x7f5494069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f54980ad000 tx=0x7f54980a6000 comp rx=0 tx=0).stop 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 msgr2=0x7f54a80c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f549c00c500 tx=0x7f549c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 shutdown_connections 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5494066f60 0x7f5494069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f54a815b6b0 0x7f54a806dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f54a80cf590 0x7f54a80c3a30 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 --2- 172.21.15.82:0/596285342 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f54a8073010 0x7f54a8154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 >> 172.21.15.82:0/596285342 conn(0x7f54a806ae60 msgr2=0x7f54a80af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 shutdown_connections 2024-04-25T10:08:36.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.264+0000 7f54ae0e4640 1 -- 172.21.15.82:0/596285342 wait complete. 2024-04-25T10:08:36.286 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:36.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.392+0000 7f35341ad640 1 Processor -- start 2024-04-25T10:08:36.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.392+0000 7f35341ad640 1 -- start start 2024-04-25T10:08:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f35341ad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f352c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f35341ad640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c15ff50 con 0x7f352c073010 2024-04-25T10:08:36.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f35341ad640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0cbd20 con 0x7f352c0cf590 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f35341ad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c15ba90 con 0x7f352c15b6b0 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f352c073010 0x7f352c15f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43164/0 (socket says 172.21.15.82:43164) 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 -- 172.21.15.82:0/1728363029 learned_addr learned my addr 172.21.15.82:0/1728363029 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 265871305 0 0) 0x7f352c15ff50 con 0x7f352c073010 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3514003530 con 0x7f352c073010 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3531f49640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f352c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258155930 0 0) 0x7f3514003530 con 0x7f352c073010 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 msgr2=0x7f352c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f352c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 >> v1:172.21.15.196:6789/0 conn(0x7f352c0cf590 legacy=0x7f352c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0cc000 con 0x7f352c073010 2024-04-25T10:08:36.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3524001a00 con 0x7f352c073010 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 >> v1:172.21.15.82:6789/0 conn(0x7f352c073010 legacy=0x7f352c15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f3514003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514004090 0x7f3514004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3514004da0 0x7f3514005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c15ba90 con 0x7f3514004da0 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351400bbe0 con 0x7f3514004090 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351400bde0 con 0x7f352c15b6b0 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3514004da0 0x7f3514005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3531f49640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f3514003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f3531748640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514004090 0x7f3514004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 -- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 msgr2=0x7f3514003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c15b6b0 0x7f3514003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 -- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514004090 msgr2=0x7f3514004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3514004090 0x7f3514004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 -- 172.21.15.82:0/1728363029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514003530 con 0x7f3514004da0 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.396+0000 7f353274a640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3514004da0 0x7f3514005220 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f352c073480 tx=0x7f352401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c489e5916075a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352400a4a0 con 0x7f3514004da0 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35240ad650 con 0x7f3514004da0 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3530f47640 1 -- 172.21.15.82:0/1728363029 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240ada40 con 0x7f3514004da0 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/1728363029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3514004da0 msgr2=0x7f3514005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 --2- 172.21.15.82:0/1728363029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3514004da0 0x7f3514005220 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f352c073480 tx=0x7f352401d3c0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f353274a640 1 -- 172.21.15.82:0/1728363029 reap_dead start 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/1728363029 shutdown_connections 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/1728363029 >> 172.21.15.82:0/1728363029 conn(0x7f352c06ae60 msgr2=0x7f352c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/1728363029 shutdown_connections 2024-04-25T10:08:36.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/1728363029 wait complete. 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 Processor -- start 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- start start 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c1550b0 0x7f352c155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c1f3af0 0x7f352c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0cc8a0 con 0x7f352c1f3670 2024-04-25T10:08:36.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0cc5c0 con 0x7f352c1f3af0 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0cba30 con 0x7f352c1550b0 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41160/0 (socket says 172.21.15.82:41160) 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 -- 172.21.15.82:0/3954538181 learned_addr learned my addr 172.21.15.82:0/3954538181 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f353274a640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c1f3af0 0x7f352c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531f49640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c1550b0 0x7f352c155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 -- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c1550b0 msgr2=0x7f352c155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c1550b0 0x7f352c155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 -- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c1f3af0 msgr2=0x7f352c0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c1f3af0 0x7f352c0cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 -- 172.21.15.82:0/3954538181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0b1b10 con 0x7f352c1f3670 2024-04-25T10:08:36.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f3531748640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f352000e550 tx=0x7f352000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35200052e0 con 0x7f352c1f3670 2024-04-25T10:08:36.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3520018070 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c0cd140 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520003820 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.400+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c0cdc20 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c0ccf30 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35200039c0 con 0x7f352c1f3670 2024-04-25T10:08:36.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f352a7fc640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 0x7f3518069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3520084aa0 con 0x7f352c1f3670 2024-04-25T10:08:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f3531f49640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 0x7f3518069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.404+0000 7f3531f49640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 0x7f3518069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b9b0 tx=0x7f351c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.408+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3520055960 con 0x7f352c1f3670 2024-04-25T10:08:36.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.552+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f352c06cff0 con 0x7f352c1f3670 2024-04-25T10:08:36.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.552+0000 7f352a7fc640 1 -- 172.21.15.82:0/3954538181 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3520005480 con 0x7f352c1f3670 2024-04-25T10:08:36.558 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 msgr2=0x7f3518069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 0x7f3518069450 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f351c00b9b0 tx=0x7f351c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 msgr2=0x7f352c0ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f352000e550 tx=0x7f352000ea10 comp rx=0 tx=0).stop 2024-04-25T10:08:36.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 shutdown_connections 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3518066f90 0x7f3518069450 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f352c1f3af0 0x7f352c0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f352c1f3670 0x7f352c0ca840 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 --2- 172.21.15.82:0/3954538181 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f352c1550b0 0x7f352c155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 >> 172.21.15.82:0/3954538181 conn(0x7f352c06ae60 msgr2=0x7f352c06e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 shutdown_connections 2024-04-25T10:08:36.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.556+0000 7f35341ad640 1 -- 172.21.15.82:0/3954538181 wait complete. 2024-04-25T10:08:36.575 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:36.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.680+0000 7fc0127c4640 1 Processor -- start 2024-04-25T10:08:36.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.680+0000 7fc0127c4640 1 -- start start 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc0127c4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc00c0d07f0 0x7fc00c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc0127c4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc00c0d1cd0 0x7fc00c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc0127c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00c06be60 con 0x7fc00c15da80 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc0127c4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c06c440 con 0x7fc00c0d07f0 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc0127c4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00c15de60 con 0x7fc00c0d1cd0 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc00c0d07f0 0x7fc00c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc00c0d07f0 0x7fc00c0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41174/0 (socket says 172.21.15.82:41174) 2024-04-25T10:08:36.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 -- 172.21.15.82:0/3313495121 learned_addr learned my addr 172.21.15.82:0/3313495121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 -- 172.21.15.82:0/3313495121 >> v2:172.21.15.196:3300/0 conn(0x7fc00c0d1cd0 msgr2=0x7fc00c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc010d61640 1 --2- 172.21.15.82:0/3313495121 >> v2:172.21.15.196:3300/0 conn(0x7fc00c0d1cd0 0x7fc00c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 --2- 172.21.15.82:0/3313495121 >> v2:172.21.15.196:3300/0 conn(0x7fc00c0d1cd0 0x7fc00c06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 -- 172.21.15.82:0/3313495121 >> v1:172.21.15.196:6789/0 conn(0x7fc00c15da80 legacy=0x7fc00c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 -- 172.21.15.82:0/3313495121 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0d2450 con 0x7fc00c0d07f0 2024-04-25T10:08:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00b7fe640 1 --2- 172.21.15.82:0/3313495121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c0d4230 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06de30 tx=0x7fc00001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400226739a4ec87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00affd640 1 -- 172.21.15.82:0/3313495121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000b1070 con 0x7fc00c0d07f0 2024-04-25T10:08:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00affd640 1 -- 172.21.15.82:0/3313495121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0000a8050 con 0x7fc00c0d07f0 2024-04-25T10:08:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.684+0000 7fc00affd640 1 -- 172.21.15.82:0/3313495121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0000ac860 con 0x7fc00c0d07f0 2024-04-25T10:08:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/3313495121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 msgr2=0x7fc00c0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- 172.21.15.82:0/3313495121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c0d4230 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fc00c06de30 tx=0x7fc00001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/3313495121 shutdown_connections 2024-04-25T10:08:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- 172.21.15.82:0/3313495121 >> v2:172.21.15.196:3300/0 conn(0x7fc00c0d1cd0 0x7fc00c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- 172.21.15.82:0/3313495121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c0d4230 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/3313495121 >> 172.21.15.82:0/3313495121 conn(0x7fc00c08b000 msgr2=0x7fc00c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/3313495121 shutdown_connections 2024-04-25T10:08:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/3313495121 wait complete. 2024-04-25T10:08:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 Processor -- start 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- start start 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc00c0d1cd0 0x7fc00c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc00c15da80 0x7fc00c155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06d7a0 con 0x7fc00c0d07f0 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06d620 con 0x7fc00c15da80 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c06d920 con 0x7fc00c0d1cd0 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41188/0 (socket says 172.21.15.82:41188) 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 -- 172.21.15.82:0/2637238010 learned_addr learned my addr 172.21.15.82:0/2637238010 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 -- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc00c0d1cd0 msgr2=0x7fc00c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc00c0d1cd0 0x7fc00c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 -- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc00c15da80 msgr2=0x7fc00c155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc00c15da80 0x7fc00c155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 -- 172.21.15.82:0/2637238010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c153570 con 0x7fc00c0d07f0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc00bfff640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b410 tx=0x7fbffc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc015020 con 0x7fc00c0d07f0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc011830 con 0x7fc00c0d07f0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c153850 con 0x7fc00c0d07f0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc011c20 con 0x7fc00c0d07f0 2024-04-25T10:08:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c15e3c0 con 0x7fc00c0d07f0 2024-04-25T10:08:36.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.688+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00c04efa0 con 0x7fc00c0d07f0 2024-04-25T10:08:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.692+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbffc018020 con 0x7fc00c0d07f0 2024-04-25T10:08:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.692+0000 7fc008ff9640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 0x7fbff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.692+0000 7fc00b7fe640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 0x7fbff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.692+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbffc0850d0 con 0x7fc00c0d07f0 2024-04-25T10:08:36.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.692+0000 7fc00b7fe640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 0x7fbff8069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ae000 tx=0x7fc000004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.696+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbffc01f020 con 0x7fc00c0d07f0 2024-04-25T10:08:36.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.840+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc00c160e70 con 0x7fc00c0d07f0 2024-04-25T10:08:36.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc008ff9640 1 -- 172.21.15.82:0/2637238010 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbffc0172c0 con 0x7fc00c0d07f0 2024-04-25T10:08:36.848 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:08:36.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 msgr2=0x7fbff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 0x7fbff8069420 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc0000ae000 tx=0x7fc000004740 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 msgr2=0x7fc00c152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fbffc00b410 tx=0x7fbffc00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 shutdown_connections 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbff8066f60 0x7fbff8069420 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc00c0d1cd0 0x7fc00c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc00c15da80 0x7fc00c155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 --2- 172.21.15.82:0/2637238010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc00c0d07f0 0x7fc00c152140 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 >> 172.21.15.82:0/2637238010 conn(0x7fc00c08b000 msgr2=0x7fc00c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 shutdown_connections 2024-04-25T10:08:36.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.844+0000 7fc0127c4640 1 -- 172.21.15.82:0/2637238010 wait complete. 2024-04-25T10:08:36.865 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.976+0000 7f382bfc6640 1 Processor -- start 2024-04-25T10:08:36.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.976+0000 7f382bfc6640 1 -- start start 2024-04-25T10:08:36.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f382415da80 0x7f38240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f38240d1cd0 0x7f38240d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 0x7f38240ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382406c440 con 0x7f382415da80 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382406be60 con 0x7f38240d1cd0 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382415de60 con 0x7f38240d0620 2024-04-25T10:08:36.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f382415da80 0x7f38240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f382415da80 0x7f38240d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41198/0 (socket says 172.21.15.82:41198) 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 -- 172.21.15.82:0/519403718 learned_addr learned my addr 172.21.15.82:0/519403718 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829561640 1 --2- 172.21.15.82:0/519403718 >> v2:172.21.15.196:3300/0 conn(0x7f38240d1cd0 0x7f38240d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382a563640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 0x7f38240ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 -- 172.21.15.82:0/519403718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 msgr2=0x7f38240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 0x7f38240ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 -- 172.21.15.82:0/519403718 >> v2:172.21.15.196:3300/0 conn(0x7f38240d1cd0 msgr2=0x7f38240d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 --2- 172.21.15.82:0/519403718 >> v2:172.21.15.196:3300/0 conn(0x7f38240d1cd0 0x7f38240d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 -- 172.21.15.82:0/519403718 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38240d2450 con 0x7f382415da80 2024-04-25T10:08:36.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382a563640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 0x7f38240ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:36.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3829d62640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f382415da80 0x7f38240d4230 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f382001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c401185a37ca718e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:36.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3828d60640 1 -- 172.21.15.82:0/519403718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200b0070 con 0x7f382415da80 2024-04-25T10:08:36.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3828d60640 1 -- 172.21.15.82:0/519403718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38200ab600 con 0x7f382415da80 2024-04-25T10:08:36.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f3828d60640 1 -- 172.21.15.82:0/519403718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200ab9d0 con 0x7f382415da80 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 -- 172.21.15.82:0/519403718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f382415da80 msgr2=0x7f38240d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.980+0000 7f382bfc6640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f382415da80 0x7f38240d4230 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f3820009870 tx=0x7f382001c520 comp rx=0 tx=0).stop 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/519403718 shutdown_connections 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38240d0620 0x7f38240ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- 172.21.15.82:0/519403718 >> v2:172.21.15.196:3300/0 conn(0x7f38240d1cd0 0x7f38240d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- 172.21.15.82:0/519403718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f382415da80 0x7f38240d4230 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/519403718 >> 172.21.15.82:0/519403718 conn(0x7f382408b000 msgr2=0x7f38240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/519403718 shutdown_connections 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/519403718 wait complete. 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 Processor -- start 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- start start 2024-04-25T10:08:36.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d1cd0 0x7f382415f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f382415da80 0x7f3824157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:36.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382406ca50 con 0x7f38240d0620 2024-04-25T10:08:36.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382406c8d0 con 0x7f38240d1cd0 2024-04-25T10:08:36.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382406cbd0 con 0x7f382415da80 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41202/0 (socket says 172.21.15.82:41202) 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 -- 172.21.15.82:0/2384165713 learned_addr learned my addr 172.21.15.82:0/2384165713 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382a563640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f382415da80 0x7f3824157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829561640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d1cd0 0x7f382415f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:36.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 -- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f382415da80 msgr2=0x7f3824157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f382415da80 0x7f3824157950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 -- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d1cd0 msgr2=0x7f382415f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d1cd0 0x7f382415f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 -- 172.21.15.82:0/2384165713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3824158bb0 con 0x7f38240d0620 2024-04-25T10:08:37.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f3829d62640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f38200ae040 tx=0x7f38200192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200b8070 con 0x7f38240d0620 2024-04-25T10:08:37.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38200b4410 con 0x7f38240d0620 2024-04-25T10:08:37.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3824158310 con 0x7f38240d0620 2024-04-25T10:08:37.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38200b0040 con 0x7f38240d0620 2024-04-25T10:08:37.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.984+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38241588d0 con 0x7f38240d0620 2024-04-25T10:08:37.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382404efa0 con 0x7f38240d0620 2024-04-25T10:08:37.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38200bb070 con 0x7f38240d0620 2024-04-25T10:08:37.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f381a7fc640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 0x7f3808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f382011ee00 con 0x7f38240d0620 2024-04-25T10:08:37.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f3829561640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 0x7f3808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.988+0000 7f3829561640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 0x7f3808069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3824161060 tx=0x7f3814008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:36.992+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3820016110 con 0x7f38240d0620 2024-04-25T10:08:37.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3824156e80 con 0x7f38240d0620 2024-04-25T10:08:37.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f381a7fc640 1 -- 172.21.15.82:0/2384165713 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f38200efcc0 con 0x7f38240d0620 2024-04-25T10:08:37.178 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:37.178 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:08:00.987254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:37.178 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-25T10:08:37.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 msgr2=0x7f3808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 0x7f3808069420 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f3824161060 tx=0x7f3814008040 comp rx=0 tx=0).stop 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 msgr2=0x7f3824151e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f38200ae040 tx=0x7f38200192c0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 shutdown_connections 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3808066f60 0x7f3808069420 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f382415da80 0x7f3824157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38240d1cd0 0x7f382415f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 --2- 172.21.15.82:0/2384165713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38240d0620 0x7f3824151e80 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.172+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 >> 172.21.15.82:0/2384165713 conn(0x7f382408b000 msgr2=0x7f38240b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.176+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 shutdown_connections 2024-04-25T10:08:37.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.176+0000 7f382bfc6640 1 -- 172.21.15.82:0/2384165713 wait complete. 2024-04-25T10:08:37.195 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:37.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.296+0000 7f012a8cd640 1 Processor -- start 2024-04-25T10:08:37.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.296+0000 7f012a8cd640 1 -- start start 2024-04-25T10:08:37.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f012a8cd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 0x7f01240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f012a8cd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01240d1cd0 0x7f01240d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f012a8cd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012406be60 con 0x7f01240d0620 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f012a8cd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012406c440 con 0x7f01240d1cd0 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f012a8cd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012415de60 con 0x7f012415da80 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01240d1cd0 0x7f01240d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0128e6a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f01240d0620 0x7f01240ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43180/0 (socket says 172.21.15.82:43180) 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0128e6a640 1 -- 172.21.15.82:0/4078199571 learned_addr learned my addr 172.21.15.82:0/4078199571 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0123fff640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 0x7f01240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 -- 172.21.15.82:0/4078199571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 msgr2=0x7f01240d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 0x7f01240d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 -- 172.21.15.82:0/4078199571 >> v1:172.21.15.82:6789/0 conn(0x7f01240d0620 legacy=0x7f01240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 -- 172.21.15.82:0/4078199571 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240d2450 con 0x7f01240d1cd0 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0122ffd640 1 -- 172.21.15.82:0/4078199571 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301366033 0 0) 0x7f012406be60 con 0x7f01240d0620 2024-04-25T10:08:37.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0123fff640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 0x7f01240d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:37.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f01237fe640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01240d00e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fae12691c192e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0122ffd640 1 -- 172.21.15.82:0/4078199571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180b2070 con 0x7f01240d1cd0 2024-04-25T10:08:37.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0122ffd640 1 -- 172.21.15.82:0/4078199571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180ad640 con 0x7f01240d1cd0 2024-04-25T10:08:37.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.300+0000 7f0122ffd640 1 -- 172.21.15.82:0/4078199571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180ada10 con 0x7f01240d1cd0 2024-04-25T10:08:37.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/4078199571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 msgr2=0x7f01240d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01240d00e0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f01180097a0 tx=0x7f011801c480 comp rx=0 tx=0).stop 2024-04-25T10:08:37.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/4078199571 shutdown_connections 2024-04-25T10:08:37.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01240d00e0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- 172.21.15.82:0/4078199571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f012415da80 0x7f01240d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/4078199571 >> 172.21.15.82:0/4078199571 conn(0x7f012408b000 msgr2=0x7f01240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/4078199571 shutdown_connections 2024-04-25T10:08:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/4078199571 wait complete. 2024-04-25T10:08:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 Processor -- start 2024-04-25T10:08:37.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- start start 2024-04-25T10:08:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01240d0620 0x7f01240b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012415da80 0x7f01240b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012416a730 con 0x7f01240d1cd0 2024-04-25T10:08:37.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012416a5b0 con 0x7f012415da80 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012416a8b0 con 0x7f01240d0620 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41220/0 (socket says 172.21.15.82:41220) 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 -- 172.21.15.82:0/1667002891 learned_addr learned my addr 172.21.15.82:0/1667002891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f0123fff640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01240d0620 0x7f01240b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f0128e6a640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012415da80 0x7f01240b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 -- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01240d0620 msgr2=0x7f01240b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01240d0620 0x7f01240b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 -- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012415da80 msgr2=0x7f01240b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012415da80 0x7f01240b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 -- 172.21.15.82:0/1667002891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01240b40a0 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f01237fe640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f01180b0040 tx=0x7f0118019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180b2040 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01180bc070 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0124155d50 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01180b6650 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.304+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0124156310 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f012404efa0 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01180b67f0 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f0120ff9640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 0x7f0110069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f011811f070 con 0x7f01240d1cd0 2024-04-25T10:08:37.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f0123fff640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 0x7f0110069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.308+0000 7f0123fff640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 0x7f0110069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.312+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01180c6020 con 0x7f01240d1cd0 2024-04-25T10:08:37.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.460+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01240008e0 con 0x7f01240d1cd0 2024-04-25T10:08:37.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.460+0000 7f0120ff9640 1 -- 172.21.15.82:0/1667002891 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f0118016070 con 0x7f01240d1cd0 2024-04-25T10:08:37.467 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:37.467 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 msgr2=0x7f0110069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 0x7f0110069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0114006fd0 tx=0x7f0114008040 comp rx=0 tx=0).stop 2024-04-25T10:08:37.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 msgr2=0x7f01241529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f01180b0040 tx=0x7f0118019220 comp rx=0 tx=0).stop 2024-04-25T10:08:37.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 shutdown_connections 2024-04-25T10:08:37.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0110066f60 0x7f0110069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f012415da80 0x7f01240b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01240d1cd0 0x7f01241529b0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 --2- 172.21.15.82:0/1667002891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01240d0620 0x7f01240b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 >> 172.21.15.82:0/1667002891 conn(0x7f012408b000 msgr2=0x7f0124160980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 shutdown_connections 2024-04-25T10:08:37.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.464+0000 7f012a8cd640 1 -- 172.21.15.82:0/1667002891 wait complete. 2024-04-25T10:08:37.484 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:37.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:37.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.600+0000 7f1fff402640 1 Processor -- start 2024-04-25T10:08:37.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.600+0000 7f1fff402640 1 -- start start 2024-04-25T10:08:37.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ff815b950 0x7f1ff809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cf420 0x7f1ff80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ff80cfb80 0x7f1ff80d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff815bd30 con 0x7f1ff80cfb80 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80cc0d0 con 0x7f1ff815b950 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1fff402640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff809ab10 con 0x7f1ff80cf420 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ff80cfb80 0x7f1ff80d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd19e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1ff815b950 0x7f1ff809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1ff80cfb80 0x7f1ff80d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41224/0 (socket says 172.21.15.82:41224) 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 -- 172.21.15.82:0/1653863801 learned_addr learned my addr 172.21.15.82:0/1653863801 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffc99d640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cf420 0x7f1ff80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 -- 172.21.15.82:0/1653863801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cf420 msgr2=0x7f1ff80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cf420 0x7f1ff80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 -- 172.21.15.82:0/1653863801 >> v2:172.21.15.196:3300/0 conn(0x7f1ff815b950 msgr2=0x7f1ff809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/1653863801 >> v2:172.21.15.196:3300/0 conn(0x7f1ff815b950 0x7f1ff809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 -- 172.21.15.82:0/1653863801 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80cc370 con 0x7f1ff80cfb80 2024-04-25T10:08:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff80cfb80 0x7f1ff80d0010 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8072480 tx=0x7f1ff001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b93a51206264b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1feffff640 1 -- 172.21.15.82:0/1653863801 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00b1070 con 0x7f1ff80cfb80 2024-04-25T10:08:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1feffff640 1 -- 172.21.15.82:0/1653863801 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff00ac630 con 0x7f1ff80cfb80 2024-04-25T10:08:37.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.604+0000 7f1feffff640 1 -- 172.21.15.82:0/1653863801 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00a8050 con 0x7f1ff80cfb80 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/1653863801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff80cfb80 msgr2=0x7f1ff80d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff80cfb80 0x7f1ff80d0010 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8072480 tx=0x7f1ff001c520 comp rx=0 tx=0).stop 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/1653863801 shutdown_connections 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff80cfb80 0x7f1ff80d0010 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- 172.21.15.82:0/1653863801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cf420 0x7f1ff80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- 172.21.15.82:0/1653863801 >> v2:172.21.15.196:3300/0 conn(0x7f1ff815b950 0x7f1ff809a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/1653863801 >> 172.21.15.82:0/1653863801 conn(0x7f1ff80b30b0 msgr2=0x7f1ff80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/1653863801 shutdown_connections 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/1653863801 wait complete. 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 Processor -- start 2024-04-25T10:08:37.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- start start 2024-04-25T10:08:37.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ff80cf420 0x7f1ff80c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cfb80 0x7f1ff80cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80cd390 con 0x7f1ff815b950 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff806faa0 con 0x7f1ff80cf420 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80c7f40 con 0x7f1ff80cfb80 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41228/0 (socket says 172.21.15.82:41228) 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 -- 172.21.15.82:0/2682690524 learned_addr learned my addr 172.21.15.82:0/2682690524 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffc99d640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cfb80 0x7f1ff80cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd19e640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ff80cf420 0x7f1ff80c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 -- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cfb80 msgr2=0x7f1ff80cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cfb80 0x7f1ff80cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 -- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ff80cf420 msgr2=0x7f1ff80c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ff80cf420 0x7f1ff80c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 -- 172.21.15.82:0/2682690524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80cb3a0 con 0x7f1ff815b950 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1ffd99f640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00af040 tx=0x7f1ff00192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00bc070 con 0x7f1ff815b950 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff00b7440 con 0x7f1ff815b950 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff80b7020 con 0x7f1ff815b950 2024-04-25T10:08:37.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff00b1070 con 0x7f1ff815b950 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.608+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff814a210 con 0x7f1ff815b950 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff804efa0 con 0x7f1ff815b950 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ff00bf070 con 0x7f1ff815b950 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1fedffb640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 0x7f1fdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1ff011ebd0 con 0x7f1ff815b950 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1ffd19e640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 0x7f1fdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.612+0000 7f1ffd19e640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 0x7f1fdc069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80d03f0 tx=0x7f1ff400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.616+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ff0016110 con 0x7f1ff815b950 2024-04-25T10:08:37.694 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:37.697 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:37.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.808+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ff80d13a0 con 0x7f1ff815b950 2024-04-25T10:08:37.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fedffb640 1 -- 172.21.15.82:0/2682690524 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f1ff00efa90 con 0x7f1ff815b950 2024-04-25T10:08:37.815 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:37.816 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:08:00.987254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:37.816 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 msgr2=0x7f1fdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 0x7f1fdc069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80d03f0 tx=0x7f1ff400b040 comp rx=0 tx=0).stop 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 msgr2=0x7f1ff8159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f1ff00af040 tx=0x7f1ff00192c0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 shutdown_connections 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1fdc066f60 0x7f1fdc069420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1ff80cfb80 0x7f1ff80cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1ff815b950 0x7f1ff8159980 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 --2- 172.21.15.82:0/2682690524 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1ff80cf420 0x7f1ff80c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 >> 172.21.15.82:0/2682690524 conn(0x7f1ff80b30b0 msgr2=0x7f1ff806fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 shutdown_connections 2024-04-25T10:08:37.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.812+0000 7f1fff402640 1 -- 172.21.15.82:0/2682690524 wait complete. 2024-04-25T10:08:37.834 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.944+0000 7fbbad5ed640 1 Processor -- start 2024-04-25T10:08:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.944+0000 7fbbad5ed640 1 -- start start 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.944+0000 7fbbad5ed640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbba815da80 0x7fbba80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.944+0000 7fbbad5ed640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbba80d1cd0 0x7fbba80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbbad5ed640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba806be60 con 0x7fbba80d0620 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbbad5ed640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80d0a70 con 0x7fbba80d1cd0 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbbad5ed640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba806c440 con 0x7fbba815da80 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbba80d1cd0 0x7fbba80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbba80d0620 0x7fbba80ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43186/0 (socket says 172.21.15.82:43186) 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba77fe640 1 -- 172.21.15.82:0/3826815998 learned_addr learned my addr 172.21.15.82:0/3826815998 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba6ffd640 1 --2- 172.21.15.82:0/3826815998 >> v2:172.21.15.196:3300/0 conn(0x7fbba815da80 0x7fbba80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 -- 172.21.15.82:0/3826815998 >> v2:172.21.15.196:3300/0 conn(0x7fbba815da80 msgr2=0x7fbba80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 --2- 172.21.15.82:0/3826815998 >> v2:172.21.15.196:3300/0 conn(0x7fbba815da80 0x7fbba80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 -- 172.21.15.82:0/3826815998 >> v1:172.21.15.82:6789/0 conn(0x7fbba80d0620 legacy=0x7fbba80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 -- 172.21.15.82:0/3826815998 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba815de60 con 0x7fbba80d1cd0 2024-04-25T10:08:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba5ffb640 1 -- 172.21.15.82:0/3826815998 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345091940 0 0) 0x7fbba806be60 con 0x7fbba80d0620 2024-04-25T10:08:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba67fc640 1 --2- 172.21.15.82:0/3826815998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80d00e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0097a0 tx=0x7fbb9c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369f7ec367fb35f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba5ffb640 1 -- 172.21.15.82:0/3826815998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0b0070 con 0x7fbba80d1cd0 2024-04-25T10:08:37.954 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba5ffb640 1 -- 172.21.15.82:0/3826815998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c0ab600 con 0x7fbba80d1cd0 2024-04-25T10:08:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbba5ffb640 1 -- 172.21.15.82:0/3826815998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0ab9d0 con 0x7fbba80d1cd0 2024-04-25T10:08:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbbad5ed640 1 -- 172.21.15.82:0/3826815998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 msgr2=0x7fbba80d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.948+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/3826815998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80d00e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0097a0 tx=0x7fbb9c01c440 comp rx=0 tx=0).stop 2024-04-25T10:08:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/3826815998 shutdown_connections 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/3826815998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80d00e0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/3826815998 >> v2:172.21.15.196:3300/0 conn(0x7fbba815da80 0x7fbba80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/3826815998 >> 172.21.15.82:0/3826815998 conn(0x7fbba808b000 msgr2=0x7fbba80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/3826815998 shutdown_connections 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/3826815998 wait complete. 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 Processor -- start 2024-04-25T10:08:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- start start 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbba80d0620 0x7fbba81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba815da80 0x7fbba80b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba816a730 con 0x7fbba80d1cd0 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba816a5b0 con 0x7fbba815da80 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba816a8b0 con 0x7fbba80d0620 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41258/0 (socket says 172.21.15.82:41258) 2024-04-25T10:08:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 -- 172.21.15.82:0/2421237015 learned_addr learned my addr 172.21.15.82:0/2421237015 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 -- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbba80d0620 msgr2=0x7fbba81553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbba80d0620 0x7fbba81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 -- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba815da80 msgr2=0x7fbba80b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba815da80 0x7fbba80b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 -- 172.21.15.82:0/2421237015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8157580 con 0x7fbba80d1cd0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbba67fc640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ae040 tx=0x7fbb9c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0b0050 con 0x7fbba80d1cd0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c0b1040 con 0x7fbba80d1cd0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c005bc0 con 0x7fbba80d1cd0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba8157860 con 0x7fbba80d1cd0 2024-04-25T10:08:37.959 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba8157e20 con 0x7fbba80d1cd0 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.952+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba804efa0 con 0x7fbba80d1cd0 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.956+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb9c0be060 con 0x7fbba80d1cd0 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.956+0000 7fbb8f7fe640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 0x7fbb94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.956+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbb9c11ebf0 con 0x7fbba80d1cd0 2024-04-25T10:08:37.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.956+0000 7fbba6ffd640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 0x7fbb94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.956+0000 7fbba6ffd640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 0x7fbb94069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800b850 tx=0x7fbb98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:37.960+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb9c016030 con 0x7fbba80d1cd0 2024-04-25T10:08:38.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.148+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbba80b2c50 con 0x7fbba80d1cd0 2024-04-25T10:08:38.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.148+0000 7fbb8f7fe640 1 -- 172.21.15.82:0/2421237015 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fbb9c0efab0 con 0x7fbba80d1cd0 2024-04-25T10:08:38.154 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:38.154 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:46.428439+0000","modified":"2024-04-25T10:08:00.987254+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7260":{"gid":7260,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":16,"addr":"172.21.15.82:6837/3956208581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3956208581},{"type":"v1","addr":"172.21.15.82:6837","nonce":3956208581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:38.154 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 49 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 msgr2=0x7fbb94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 0x7fbb94069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800b850 tx=0x7fbb98008040 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 msgr2=0x7fbba80b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0ae040 tx=0x7fbb9c01ac80 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 shutdown_connections 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb94066f60 0x7fbb94069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbba815da80 0x7fbba80b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbba80d1cd0 0x7fbba80b8220 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 --2- 172.21.15.82:0/2421237015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbba80d0620 0x7fbba81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 >> 172.21.15.82:0/2421237015 conn(0x7fbba808b000 msgr2=0x7fbba80b4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 shutdown_connections 2024-04-25T10:08:38.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.152+0000 7fbbad5ed640 1 -- 172.21.15.82:0/2421237015 wait complete. 2024-04-25T10:08:38.171 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:38.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.284+0000 7f43bae07640 1 Processor -- start 2024-04-25T10:08:38.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.284+0000 7f43bae07640 1 -- start start 2024-04-25T10:08:38.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43bae07640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43b4144ca0 0x7f43b406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43bae07640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b40ccc70 con 0x7f43b406ec60 2024-04-25T10:08:38.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43bae07640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b406dc40 con 0x7f43b40ce510 2024-04-25T10:08:38.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43bae07640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b40cc2b0 con 0x7f43b4144ca0 2024-04-25T10:08:38.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f43b4144ca0 0x7f43b406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b93a4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f43b406ec60 0x7f43b4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43200/0 (socket says 172.21.15.82:43200) 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b93a4640 1 -- 172.21.15.82:0/1165526118 learned_addr learned my addr 172.21.15.82:0/1165526118 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b3fff640 1 -- 172.21.15.82:0/1165526118 >> v1:172.21.15.196:6789/0 conn(0x7f43b40ce510 legacy=0x7f43b40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b3fff640 1 -- 172.21.15.82:0/1165526118 >> v1:172.21.15.82:6789/0 conn(0x7f43b406ec60 legacy=0x7f43b4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b3fff640 1 -- 172.21.15.82:0/1165526118 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b406df20 con 0x7f43b4144ca0 2024-04-25T10:08:38.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b37fe640 1 -- 172.21.15.82:0/1165526118 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850954071 0 0) 0x7f43b40ccc70 con 0x7f43b406ec60 2024-04-25T10:08:38.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b3fff640 1 --2- 172.21.15.82:0/1165526118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b406e720 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f439c0097c0 tx=0x7f439c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5289300b0f3edf70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b37fe640 1 -- 172.21.15.82:0/1165526118 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0b5070 con 0x7f43b4144ca0 2024-04-25T10:08:38.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b37fe640 1 -- 172.21.15.82:0/1165526118 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439c0a8950 con 0x7f43b4144ca0 2024-04-25T10:08:38.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.288+0000 7f43b37fe640 1 -- 172.21.15.82:0/1165526118 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c0a8d20 con 0x7f43b4144ca0 2024-04-25T10:08:38.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1165526118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 msgr2=0x7f43b406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 --2- 172.21.15.82:0/1165526118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b406e720 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f439c0097c0 tx=0x7f439c01ec50 comp rx=0 tx=0).stop 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1165526118 shutdown_connections 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 --2- 172.21.15.82:0/1165526118 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b406e720 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1165526118 >> 172.21.15.82:0/1165526118 conn(0x7f43b40b11a0 msgr2=0x7f43b40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1165526118 shutdown_connections 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1165526118 wait complete. 2024-04-25T10:08:38.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 Processor -- start 2024-04-25T10:08:38.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- start start 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43b406ec60 0x7f43b413ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 0x7f43b415aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4142d90 con 0x7f43b4144ca0 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b406e290 con 0x7f43b406ec60 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b4142f10 con 0x7f43b40ce510 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41270/0 (socket says 172.21.15.82:41270) 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 -- 172.21.15.82:0/1765202652 learned_addr learned my addr 172.21.15.82:0/1765202652 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b8ba3640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43b406ec60 0x7f43b413ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b3fff640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 0x7f43b415aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 -- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 msgr2=0x7f43b415aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 0x7f43b415aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 -- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43b406ec60 msgr2=0x7f43b413ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43b406ec60 0x7f43b413ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 -- 172.21.15.82:0/1765202652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b40b2640 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b3fff640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 0x7f43b415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b93a4640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c7a0 tx=0x7f43a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4018070 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400a640 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b40b2920 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400aab0 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b40b2ee0 con 0x7f43b4144ca0 2024-04-25T10:08:38.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.292+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b404efa0 con 0x7f43b4144ca0 2024-04-25T10:08:38.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.296+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f43a401b070 con 0x7f43b4144ca0 2024-04-25T10:08:38.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.296+0000 7f43b17fa640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 0x7f4394069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.296+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f43a4084830 con 0x7f43b4144ca0 2024-04-25T10:08:38.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.296+0000 7f43b8ba3640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 0x7f4394069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.296+0000 7f43b8ba3640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 0x7f4394069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f43ac004e30 tx=0x7f43ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.300+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f43a4055660 con 0x7f43b4144ca0 2024-04-25T10:08:38.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.460+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43b40ae4b0 con 0x7f43b4144ca0 2024-04-25T10:08:38.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.460+0000 7f43b17fa640 1 -- 172.21.15.82:0/1765202652 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+17084 (secure 0 0 0) 0x7f43a4054f60 con 0x7f43b4144ca0 2024-04-25T10:08:38.465 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:38.466 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:07:09.947257+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:44.746226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:45.417744+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 msgr2=0x7f4394069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 0x7f4394069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f43ac004e30 tx=0x7f43ac009040 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 msgr2=0x7f43b415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f43a400c7a0 tx=0x7f43a400cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 shutdown_connections 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4394066f60 0x7f4394069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f43b40ce510 0x7f43b415aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f43b4144ca0 0x7f43b415afc0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 --2- 172.21.15.82:0/1765202652 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f43b406ec60 0x7f43b413ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 >> 172.21.15.82:0/1765202652 conn(0x7f43b40b11a0 msgr2=0x7f43b40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 shutdown_connections 2024-04-25T10:08:38.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.464+0000 7f43bae07640 1 -- 172.21.15.82:0/1765202652 wait complete. 2024-04-25T10:08:38.484 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 Processor -- start 2024-04-25T10:08:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 -- start start 2024-04-25T10:08:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 0x7fe9980d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9980d07f0 0x7fe99806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99806be60 con 0x7fe9980d1cd0 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99806c440 con 0x7fe9980d07f0 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99c8f7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99815de60 con 0x7fe99815da80 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9980d07f0 0x7fe99806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9980d07f0 0x7fe99806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41274/0 (socket says 172.21.15.82:41274) 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 -- 172.21.15.82:0/812405425 learned_addr learned my addr 172.21.15.82:0/812405425 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99659c640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 0x7fe9980d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 -- 172.21.15.82:0/812405425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 msgr2=0x7fe9980d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 0x7fe9980d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 -- 172.21.15.82:0/812405425 >> v1:172.21.15.196:6789/0 conn(0x7fe9980d1cd0 legacy=0x7fe9980d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe996d9d640 1 -- 172.21.15.82:0/812405425 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980d2450 con 0x7fe9980d07f0 2024-04-25T10:08:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.596+0000 7fe99659c640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 0x7fe9980d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:38.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe996d9d640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe99806c8d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe990009e50 tx=0x7fe99001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e41a7d1b8105e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe995d9b640 1 -- 172.21.15.82:0/812405425 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900a5800 con 0x7fe9980d07f0 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe995d9b640 1 -- 172.21.15.82:0/812405425 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9900a59a0 con 0x7fe9980d07f0 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe995d9b640 1 -- 172.21.15.82:0/812405425 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9900a5d70 con 0x7fe9980d07f0 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- 172.21.15.82:0/812405425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 msgr2=0x7fe99806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe99806c8d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe990009e50 tx=0x7fe99001ca00 comp rx=0 tx=0).stop 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- 172.21.15.82:0/812405425 shutdown_connections 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe99806c8d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/812405425 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe99815da80 0x7fe9980d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- 172.21.15.82:0/812405425 >> 172.21.15.82:0/812405425 conn(0x7fe99808b000 msgr2=0x7fe9980d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- 172.21.15.82:0/812405425 shutdown_connections 2024-04-25T10:08:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- 172.21.15.82:0/812405425 wait complete. 2024-04-25T10:08:38.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 Processor -- start 2024-04-25T10:08:38.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.600+0000 7fe99c8f7640 1 -- start start 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9980d1cd0 0x7fe99815f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe99815da80 0x7fe99815bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99806d7a0 con 0x7fe9980d07f0 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99806d620 con 0x7fe99815da80 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99806d920 con 0x7fe9980d1cd0 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41278/0 (socket says 172.21.15.82:41278) 2024-04-25T10:08:38.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 -- 172.21.15.82:0/1333331048 learned_addr learned my addr 172.21.15.82:0/1333331048 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe996d9d640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe99815da80 0x7fe99815bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98dd9b640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9980d1cd0 0x7fe99815f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 -- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9980d1cd0 msgr2=0x7fe99815f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9980d1cd0 0x7fe99815f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 -- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe99815da80 msgr2=0x7fe99815bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe99815da80 0x7fe99815bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 -- 172.21.15.82:0/1333331048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9980b0850 con 0x7fe9980d07f0 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99659c640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe98800d830 tx=0x7fe98800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988018020 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9880096c0 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe998160140 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe988009ab0 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe998160700 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99804efa0 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe98801b020 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 0x7fe978069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98dd9b640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 0x7fe978069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe988085970 con 0x7fe9980d07f0 2024-04-25T10:08:38.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.604+0000 7fe98dd9b640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 0x7fe978069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe9980b0ad0 tx=0x7fe980009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.608+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe988022020 con 0x7fe9980d07f0 2024-04-25T10:08:38.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.808+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe998159410 con 0x7fe9980d07f0 2024-04-25T10:08:38.814 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:08:38.808+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:38.817 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:38.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.812+0000 7fe98f7fe640 1 -- 172.21.15.82:0/1333331048 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe988056710 con 0x7fe9980d07f0 2024-04-25T10:08:38.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.816+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 msgr2=0x7fe978069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.816+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 0x7fe978069450 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe9980b0ad0 tx=0x7fe980009040 comp rx=0 tx=0).stop 2024-04-25T10:08:38.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.816+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 msgr2=0x7fe998156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.816+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe98800d830 tx=0x7fe98800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 shutdown_connections 2024-04-25T10:08:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe978066f90 0x7fe978069450 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9980d1cd0 0x7fe99815f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe99815da80 0x7fe99815bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 --2- 172.21.15.82:0/1333331048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9980d07f0 0x7fe998156d40 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 >> 172.21.15.82:0/1333331048 conn(0x7fe99808b000 msgr2=0x7fe9980737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 shutdown_connections 2024-04-25T10:08:38.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.820+0000 7fe99c8f7640 1 -- 172.21.15.82:0/1333331048 wait complete. 2024-04-25T10:08:38.827 INFO:tasks.ceph.mds.a.smithi082.stderr: -4486> 2024-04-25T10:07:15.961+0000 7f19b316f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:38.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:38.869 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:08:38.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.960+0000 7f19db5f0640 1 Processor -- start 2024-04-25T10:08:38.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.960+0000 7f19db5f0640 1 -- start start 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19db5f0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f19d40cf590 0x7f19d40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19db5f0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40cbd20 con 0x7f19d4072000 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19db5f0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d40cc000 con 0x7f19d415b6b0 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19db5f0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d415ba90 con 0x7f19d40cf590 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d9b8d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f19d4072000 0x7f19d4072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43208/0 (socket says 172.21.15.82:43208) 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d9b8d640 1 -- 172.21.15.82:0/1223931043 learned_addr learned my addr 172.21.15.82:0/1223931043 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1223931043 >> v2:172.21.15.196:3300/0 conn(0x7f19d40cf590 0x7f19d40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47907931 0 0) 0x7f19d40cbd20 con 0x7f19d4072000 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19c0003530 con 0x7f19d4072000 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3222202026 0 0) 0x7f19c0003530 con 0x7f19d4072000 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 >> v2:172.21.15.196:3300/0 conn(0x7f19d40cf590 msgr2=0x7f19d40cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 --2- 172.21.15.82:0/1223931043 >> v2:172.21.15.196:3300/0 conn(0x7f19d40cf590 0x7f19d40cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 >> v1:172.21.15.196:6789/0 conn(0x7f19d415b6b0 legacy=0x7f19d40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40cc2e0 con 0x7f19d4072000 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f19cc001a00 con 0x7f19d4072000 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 >> v1:172.21.15.82:6789/0 conn(0x7f19d4072000 legacy=0x7f19d4072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c0003a30 0x7f19c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19c00051d0 0x7f19c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415ba90 con 0x7f19c0004810 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c000c100 con 0x7f19c0003a30 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c000c300 con 0x7f19c00051d0 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d9b8d640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19c00051d0 0x7f19c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 -- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19c00051d0 msgr2=0x7f19c000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19c00051d0 0x7f19c000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 -- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c0003a30 msgr2=0x7f19c0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19c0003a30 0x7f19c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d8b8b640 1 -- 172.21.15.82:0/1223931043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c0003530 con 0x7f19c0004810 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d9b8d640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19c00051d0 0x7f19c000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:38.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.964+0000 7f19d9b8d640 1 -- 172.21.15.82:0/1223931043 reap_dead start 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19c0004c90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009ae0 tx=0x7f19d001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67aafab1f5b74009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00b2070 con 0x7f19c0004810 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d00a5660 con 0x7f19c0004810 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19cbfff640 1 -- 172.21.15.82:0/1223931043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d00096b0 con 0x7f19c0004810 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 msgr2=0x7f19c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19c0004c90 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009ae0 tx=0x7f19d001ca70 comp rx=0 tx=0).stop 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- 172.21.15.82:0/1223931043 shutdown_connections 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1223931043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19c0004c90 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- 172.21.15.82:0/1223931043 >> 172.21.15.82:0/1223931043 conn(0x7f19d406ae60 msgr2=0x7f19d406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- 172.21.15.82:0/1223931043 shutdown_connections 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- 172.21.15.82:0/1223931043 wait complete. 2024-04-25T10:08:38.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 Processor -- start 2024-04-25T10:08:38.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.968+0000 7f19db5f0640 1 -- start start 2024-04-25T10:08:38.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d41f44f0 0x7f19d415cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d406e0d0 0x7f19d406e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cba30 con 0x7f19c0004810 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cc8a0 con 0x7f19d406e0d0 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cadb0 con 0x7f19d41f44f0 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41310/0 (socket says 172.21.15.82:41310) 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 -- 172.21.15.82:0/1624438740 learned_addr learned my addr 172.21.15.82:0/1624438740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 -- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d41f44f0 msgr2=0x7f19d415cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d41f44f0 0x7f19d415cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 -- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d406e0d0 msgr2=0x7f19d406e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d406e0d0 0x7f19d406e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:38.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 -- 172.21.15.82:0/1624438740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d41f4970 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d938c640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c4d0 tx=0x7f19c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c40052d0 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c4005490 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d415d290 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c4012850 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d415d810 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d404efa0 con 0x7f19c0004810 2024-04-25T10:08:38.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19c40129f0 con 0x7f19c0004810 2024-04-25T10:08:38.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 0x7f19b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:38.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f19c4084eb0 con 0x7f19c0004810 2024-04-25T10:08:38.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 0x7f19b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:38.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.972+0000 7f19d8b8b640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 0x7f19b8069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f19d00af000 tx=0x7f19d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:38.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:38.976+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19c4055c80 con 0x7f19c0004810 2024-04-25T10:08:39.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.172+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f19d40b03c0 con 0x7f19c0004810 2024-04-25T10:08:39.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.816+0000 7f19c9ffb640 1 -- 172.21.15.82:0/1624438740 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f19c4055580 con 0x7f19c0004810 2024-04-25T10:08:39.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 msgr2=0x7f19b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 0x7f19b8069450 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f19d00af000 tx=0x7f19d0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:39.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 msgr2=0x7f19d40b1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:39.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f19c400c4d0 tx=0x7f19c400c990 comp rx=0 tx=0).stop 2024-04-25T10:08:39.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 shutdown_connections 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19b8066f90 0x7f19b8069450 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f19d41f44f0 0x7f19d415cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f19d406e0d0 0x7f19d406e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 --2- 172.21.15.82:0/1624438740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f19c0004810 0x7f19d40b1ba0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 >> 172.21.15.82:0/1624438740 conn(0x7f19d406ae60 msgr2=0x7f19d41561c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 shutdown_connections 2024-04-25T10:08:39.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:39.820+0000 7f19db5f0640 1 -- 172.21.15.82:0/1624438740 wait complete. 2024-04-25T10:08:39.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:40.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 Processor -- start 2024-04-25T10:08:40.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 -- start start 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99640ae6c0 0x7f99640b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99640cf510 0x7f99640cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996415e0f0 con 0x7f996415a3c0 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99640cfe80 con 0x7f99640cf510 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996a301640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99640b24a0 con 0x7f99640ae6c0 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996889e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99640cf510 0x7f99640cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f9963fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f996415a3c0 0x7f996415a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43212/0 (socket says 172.21.15.82:43212) 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f9963fff640 1 -- 172.21.15.82:0/141690580 learned_addr learned my addr 172.21.15.82:0/141690580 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f99637fe640 1 --2- 172.21.15.82:0/141690580 >> v2:172.21.15.196:3300/0 conn(0x7f99640ae6c0 0x7f99640b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996889e640 1 -- 172.21.15.82:0/141690580 >> v2:172.21.15.196:3300/0 conn(0x7f99640ae6c0 msgr2=0x7f99640b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996889e640 1 --2- 172.21.15.82:0/141690580 >> v2:172.21.15.196:3300/0 conn(0x7f99640ae6c0 0x7f99640b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996889e640 1 -- 172.21.15.82:0/141690580 >> v1:172.21.15.82:6789/0 conn(0x7f996415a3c0 legacy=0x7f996415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f996889e640 1 -- 172.21.15.82:0/141690580 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996415b0b0 con 0x7f99640cf510 2024-04-25T10:08:40.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.012+0000 7f9962ffd640 1 -- 172.21.15.82:0/141690580 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280385356 0 0) 0x7f996415e0f0 con 0x7f996415a3c0 2024-04-25T10:08:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996889e640 1 --2- 172.21.15.82:0/141690580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640cf510 0x7f99640cf940 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f99640aedb0 tx=0x7f995c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b19f5cb3e669e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f9962ffd640 1 -- 172.21.15.82:0/141690580 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0b2070 con 0x7f99640cf510 2024-04-25T10:08:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f9962ffd640 1 -- 172.21.15.82:0/141690580 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c0a57f0 con 0x7f99640cf510 2024-04-25T10:08:40.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f9962ffd640 1 -- 172.21.15.82:0/141690580 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c0a5bc0 con 0x7f99640cf510 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- 172.21.15.82:0/141690580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640cf510 msgr2=0x7f99640cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 --2- 172.21.15.82:0/141690580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640cf510 0x7f99640cf940 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f99640aedb0 tx=0x7f995c01c930 comp rx=0 tx=0).stop 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- 172.21.15.82:0/141690580 shutdown_connections 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 --2- 172.21.15.82:0/141690580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640cf510 0x7f99640cf940 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 --2- 172.21.15.82:0/141690580 >> v2:172.21.15.196:3300/0 conn(0x7f99640ae6c0 0x7f99640b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- 172.21.15.82:0/141690580 >> 172.21.15.82:0/141690580 conn(0x7f996406ae60 msgr2=0x7f996406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- 172.21.15.82:0/141690580 shutdown_connections 2024-04-25T10:08:40.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- 172.21.15.82:0/141690580 wait complete. 2024-04-25T10:08:40.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 Processor -- start 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.016+0000 7f996a301640 1 -- start start 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99640cf510 0x7f9964072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 0x7f99640c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640b2e60 con 0x7f99640ae6c0 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99640b2ce0 con 0x7f996415a3c0 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99641673f0 con 0x7f99640cf510 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41328/0 (socket says 172.21.15.82:41328) 2024-04-25T10:08:40.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 -- 172.21.15.82:0/1663068506 learned_addr learned my addr 172.21.15.82:0/1663068506 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996889e640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 0x7f99640c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f99637fe640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99640cf510 0x7f9964072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 -- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99640cf510 msgr2=0x7f9964072e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99640cf510 0x7f9964072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 -- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 msgr2=0x7f99640c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 0x7f99640c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 -- 172.21.15.82:0/1663068506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996415cd70 con 0x7f99640ae6c0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996889e640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 0x7f99640c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9963fff640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f995400c480 tx=0x7f995400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954018070 con 0x7f99640ae6c0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9954007500 con 0x7f99640ae6c0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996415d050 con 0x7f99640ae6c0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9954004a30 con 0x7f99640ae6c0 2024-04-25T10:08:40.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996415d6a0 con 0x7f99640ae6c0 2024-04-25T10:08:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9964067c30 con 0x7f99640ae6c0 2024-04-25T10:08:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f995401b070 con 0x7f99640ae6c0 2024-04-25T10:08:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 0x7f9950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f99540845b0 con 0x7f99640ae6c0 2024-04-25T10:08:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f99637fe640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 0x7f9950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.020+0000 7f99637fe640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 0x7f9950069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99640ca910 tx=0x7f9958008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.024+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9954055410 con 0x7f99640ae6c0 2024-04-25T10:08:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.188+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f996406edf0 con 0x7f99640ae6c0 2024-04-25T10:08:40.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.824+0000 7f9960ff9640 1 -- 172.21.15.82:0/1663068506 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9954054d10 con 0x7f99640ae6c0 2024-04-25T10:08:40.829 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 msgr2=0x7f9950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 0x7f9950069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f99640ca910 tx=0x7f9958008040 comp rx=0 tx=0).stop 2024-04-25T10:08:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 msgr2=0x7f99640b0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f995400c480 tx=0x7f995400c940 comp rx=0 tx=0).stop 2024-04-25T10:08:40.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 shutdown_connections 2024-04-25T10:08:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9950066f60 0x7f9950069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99640cf510 0x7f9964072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f996415a3c0 0x7f99640c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 --2- 172.21.15.82:0/1663068506 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99640ae6c0 0x7f99640b0210 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 >> 172.21.15.82:0/1663068506 conn(0x7f996406ae60 msgr2=0x7f996406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 shutdown_connections 2024-04-25T10:08:40.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.828+0000 7f996a301640 1 -- 172.21.15.82:0/1663068506 wait complete. 2024-04-25T10:08:40.859 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:40.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 Processor -- start 2024-04-25T10:08:40.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 -- start start 2024-04-25T10:08:40.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f466015b710 0x7f46600cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466006f0d0 con 0x7f4660099a70 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46600cffa0 con 0x7f466015b710 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f46666d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466006f6e0 con 0x7f466009a610 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f466015b710 0x7f46600cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4660099a70 0x7f4660099f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43218/0 (socket says 172.21.15.82:43218) 2024-04-25T10:08:40.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465f7fe640 1 -- 172.21.15.82:0/50080898 learned_addr learned my addr 172.21.15.82:0/50080898 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f4664c73640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465ffff640 1 -- 172.21.15.82:0/50080898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 msgr2=0x7f466006fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465ffff640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466006fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465ffff640 1 -- 172.21.15.82:0/50080898 >> v1:172.21.15.82:6789/0 conn(0x7f4660099a70 legacy=0x7f4660099f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465ffff640 1 -- 172.21.15.82:0/50080898 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46600d01d0 con 0x7f466015b710 2024-04-25T10:08:40.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.988+0000 7f465effd640 1 -- 172.21.15.82:0/50080898 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892540136 0 0) 0x7f466006f0d0 con 0x7f4660099a70 2024-04-25T10:08:40.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f465ffff640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f46600cfa60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f46500097c0 tx=0x7f465001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc65d67dd270e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:40.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f465effd640 1 -- 172.21.15.82:0/50080898 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500b0070 con 0x7f466015b710 2024-04-25T10:08:40.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f465effd640 1 -- 172.21.15.82:0/50080898 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4650004230 con 0x7f466015b710 2024-04-25T10:08:40.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f465effd640 1 -- 172.21.15.82:0/50080898 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650003620 con 0x7f466015b710 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- 172.21.15.82:0/50080898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 msgr2=0x7f46600cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f46600cfa60 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f46500097c0 tx=0x7f465001c4a0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- 172.21.15.82:0/50080898 shutdown_connections 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466006fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 --2- 172.21.15.82:0/50080898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f46600cfa60 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- 172.21.15.82:0/50080898 >> 172.21.15.82:0/50080898 conn(0x7f46600b2f70 msgr2=0x7f46600b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- 172.21.15.82:0/50080898 shutdown_connections 2024-04-25T10:08:40.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- 172.21.15.82:0/50080898 wait complete. 2024-04-25T10:08:41.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 Processor -- start 2024-04-25T10:08:41.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.992+0000 7f46666d6640 1 -- start start 2024-04-25T10:08:41.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660099a70 0x7f46600b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4660070a40 con 0x7f466015b710 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600708c0 con 0x7f4660099a70 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4660070bc0 con 0x7f466009a610 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41346/0 (socket says 172.21.15.82:41346) 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 -- 172.21.15.82:0/2610995031 learned_addr learned my addr 172.21.15.82:0/2610995031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:41.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465f7fe640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465ffff640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660099a70 0x7f46600b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 -- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 msgr2=0x7f466015bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466015bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 -- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660099a70 msgr2=0x7f46600b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660099a70 0x7f46600b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 -- 172.21.15.82:0/2610995031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466015e3e0 con 0x7f466015b710 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f4664c73640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f465800d5d0 tx=0x7f465800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580125f0 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4658019070 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466015c2a0 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46580129f0 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466015e960 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660067c30 con 0x7f466015b710 2024-04-25T10:08:41.006 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4658012b90 con 0x7f466015b710 2024-04-25T10:08:41.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 0x7f4640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f4658085470 con 0x7f466015b710 2024-04-25T10:08:41.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465ffff640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 0x7f4640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:40.996+0000 7f465ffff640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 0x7f4640069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46500ad000 tx=0x7f4650008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.000+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4658056490 con 0x7f466015b710 2024-04-25T10:08:41.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.164+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f466015ac00 con 0x7f466015b710 2024-04-25T10:08:41.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.832+0000 7f465cff9640 1 -- 172.21.15.82:0/2610995031 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4658055d90 con 0x7f466015b710 2024-04-25T10:08:41.837 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:41.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 msgr2=0x7f4640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 0x7f4640069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46500ad000 tx=0x7f4650008040 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 msgr2=0x7f4660154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f465800d5d0 tx=0x7f465800da90 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 shutdown_connections 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4640066f60 0x7f4640069420 secure :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46500ad000 tx=0x7f4650008040 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f466009a610 0x7f466015bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f466015b710 0x7f4660154f90 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 --2- 172.21.15.82:0/2610995031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4660099a70 0x7f46600b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 >> 172.21.15.82:0/2610995031 conn(0x7f46600b2f70 msgr2=0x7f46600cd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 shutdown_connections 2024-04-25T10:08:41.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.836+0000 7f46666d6640 1 -- 172.21.15.82:0/2610995031 wait complete. 2024-04-25T10:08:41.866 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:41.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 Processor -- start 2024-04-25T10:08:41.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 -- start start 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7b40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b415ff50 con 0x7fe7b40cf590 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b40cbd20 con 0x7fe7b4073010 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7bc203640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b415ba90 con 0x7fe7b415b6b0 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7b979e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe7b40cf590 0x7fe7b415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43230/0 (socket says 172.21.15.82:43230) 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7b979e640 1 -- 172.21.15.82:0/1074199519 learned_addr learned my addr 172.21.15.82:0/1074199519 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825817814 0 0) 0x7fe7b415ff50 con 0x7fe7b40cf590 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a0003530 con 0x7fe7b40cf590 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.984+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7b40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2973201832 0 0) 0x7fe7a0003530 con 0x7fe7b40cf590 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 msgr2=0x7fe7b40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7b40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 >> v1:172.21.15.196:6789/0 conn(0x7fe7b4073010 legacy=0x7fe7b415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40cc000 con 0x7fe7b40cf590 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe7a4001680 con 0x7fe7b40cf590 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 >> v1:172.21.15.82:6789/0 conn(0x7fe7b40cf590 legacy=0x7fe7b415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0003a60 0x7fe7a0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a0004da0 0x7fe7a0005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b415ba90 con 0x7fe7a0004da0 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000bb90 con 0x7fe7a0003a60 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000bd90 con 0x7fe7b415b6b0 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a0004da0 0x7fe7a0005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0003a60 0x7fe7a0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b979e640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 -- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 msgr2=0x7fe7a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b415b6b0 0x7fe7a0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 -- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0003a60 msgr2=0x7fe7a0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:41.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7a0003a60 0x7fe7a0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:41.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 -- 172.21.15.82:0/1074199519 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0003530 con 0x7fe7a0004da0 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a0004da0 0x7fe7a0005220 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac001e40 tx=0x7fe7ac01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc41690063a77088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac001a90 con 0x7fe7a0004da0 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac003b80 con 0x7fe7a0004da0 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7b8f9d640 1 -- 172.21.15.82:0/1074199519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac005080 con 0x7fe7a0004da0 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 -- 172.21.15.82:0/1074199519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a0004da0 msgr2=0x7fe7a0005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:41.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 --2- 172.21.15.82:0/1074199519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7a0004da0 0x7fe7a0005220 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac001e40 tx=0x7fe7ac01c760 comp rx=0 tx=0).stop 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7ba7a0640 1 -- 172.21.15.82:0/1074199519 reap_dead start 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 -- 172.21.15.82:0/1074199519 shutdown_connections 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 -- 172.21.15.82:0/1074199519 >> 172.21.15.82:0/1074199519 conn(0x7fe7b406ae60 msgr2=0x7fe7b406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 -- 172.21.15.82:0/1074199519 shutdown_connections 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.988+0000 7fe7bc203640 1 -- 172.21.15.82:0/1074199519 wait complete. 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 Processor -- start 2024-04-25T10:08:41.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- start start 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 0x7fe7b41f19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 0x7fe7b40b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40cc8a0 con 0x7fe7b40ad570 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40cc5c0 con 0x7fe7b40c55e0 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40cba30 con 0x7fe7b40af650 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41368/0 (socket says 172.21.15.82:41368) 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 -- 172.21.15.82:0/2409718864 learned_addr learned my addr 172.21.15.82:0/2409718864 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 0x7fe7b40b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 0x7fe7b41f19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 -- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 msgr2=0x7fe7b41f19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 0x7fe7b41f19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 -- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 msgr2=0x7fe7b40b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 0x7fe7b40b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 -- 172.21.15.82:0/2409718864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b415cf00 con 0x7fe7b40ad570 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7ba7a0640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 0x7fe7b40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 0x7fe7b41f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7b979e640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d7a0 tx=0x7fe7a400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4019070 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a40074e0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b415d1e0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a4004a30 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b415d7a0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.992+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b404efa0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.996+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7a4004bd0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.996+0000 7fe7aa7fc640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 0x7fe798069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.996+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe7a40847a0 con 0x7fe7b40ad570 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.996+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 0x7fe798069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:41.996+0000 7fe7b9f9f640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 0x7fe798069450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b8c0 tx=0x7fe7b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.000+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7a4055960 con 0x7fe7b40ad570 2024-04-25T10:08:42.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.160+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7b40cab60 con 0x7fe7b40ad570 2024-04-25T10:08:42.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7aa7fc640 1 -- 172.21.15.82:0/2409718864 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7a4055260 con 0x7fe7b40ad570 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 msgr2=0x7fe798069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 0x7fe798069450 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000b8c0 tx=0x7fe7b0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 msgr2=0x7fe7b40b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe7a400d7a0 tx=0x7fe7a400dc60 comp rx=0 tx=0).stop 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 shutdown_connections 2024-04-25T10:08:42.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe798066f90 0x7fe798069450 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7b40c55e0 0x7fe7b40b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7b40ad570 0x7fe7b40b0f80 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 --2- 172.21.15.82:0/2409718864 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7b40af650 0x7fe7b41f19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 >> 172.21.15.82:0/2409718864 conn(0x7fe7b406ae60 msgr2=0x7fe7b40ace60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 shutdown_connections 2024-04-25T10:08:42.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.164+0000 7fe7bc203640 1 -- 172.21.15.82:0/2409718864 wait complete. 2024-04-25T10:08:42.188 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T10:08:42.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 Processor -- start 2024-04-25T10:08:42.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 -- start start 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff38c0b2c30 0x7ff38c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0cbec0 con 0x7ff38c09a7b0 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0cc1a0 con 0x7ff38c09a180 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff391862640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c0cc480 con 0x7ff38c0b2c30 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff38a7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff38c09a7b0 0x7ff38c099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43234/0 (socket says 172.21.15.82:43234) 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff38a7fc640 1 -- 172.21.15.82:0/1965916694 learned_addr learned my addr 172.21.15.82:0/1965916694 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:42.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff38affd640 1 --2- 172.21.15.82:0/1965916694 >> v2:172.21.15.196:3300/0 conn(0x7ff38c0b2c30 0x7ff38c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718251162 0 0) 0x7ff38c0cbec0 con 0x7ff38c09a7b0 2024-04-25T10:08:42.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff374003530 con 0x7ff38c09a7b0 2024-04-25T10:08:42.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214338628 0 0) 0x7ff374003530 con 0x7ff38c09a7b0 2024-04-25T10:08:42.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 >> v2:172.21.15.196:3300/0 conn(0x7ff38c0b2c30 msgr2=0x7ff38c0b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 --2- 172.21.15.82:0/1965916694 >> v2:172.21.15.196:3300/0 conn(0x7ff38c0b2c30 0x7ff38c0b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 >> v1:172.21.15.196:6789/0 conn(0x7ff38c09a180 legacy=0x7ff38c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0cc760 con 0x7ff38c09a7b0 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff380001680 con 0x7ff38c09a7b0 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 >> v1:172.21.15.82:6789/0 conn(0x7ff38c09a7b0 legacy=0x7ff38c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff374003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff374004810 0x7ff374004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff374005220 0x7ff37400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cc480 con 0x7ff374003a80 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c150 con 0x7ff374004810 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400c350 con 0x7ff374005220 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.308+0000 7ff38affd640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff374003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 -- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff374005220 msgr2=0x7ff37400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff374005220 0x7ff37400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 -- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff374004810 msgr2=0x7ff374004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff374004810 0x7ff374004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 -- 172.21.15.82:0/1965916694 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff374003530 con 0x7ff374003a80 2024-04-25T10:08:42.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38b7fe640 1 -- 172.21.15.82:0/1965916694 reap_dead start 2024-04-25T10:08:42.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff38affd640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff374003ea0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009fc0 tx=0x7ff37c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dda262e62998746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0b2070 con 0x7ff374003a80 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c0a5b90 con 0x7ff374003a80 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff389ffb640 1 -- 172.21.15.82:0/1965916694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c009c10 con 0x7ff374003a80 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 -- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 msgr2=0x7ff374003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff374003ea0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff37c009fc0 tx=0x7ff37c01cc90 comp rx=0 tx=0).stop 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 -- 172.21.15.82:0/1965916694 shutdown_connections 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 --2- 172.21.15.82:0/1965916694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff374003ea0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 -- 172.21.15.82:0/1965916694 >> 172.21.15.82:0/1965916694 conn(0x7ff38c0b3010 msgr2=0x7ff38c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 -- 172.21.15.82:0/1965916694 shutdown_connections 2024-04-25T10:08:42.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.312+0000 7ff391862640 1 -- 172.21.15.82:0/1965916694 wait complete. 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 Processor -- start 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- start start 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 0x7ff38c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 0x7ff38c0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c06c380 con 0x7ff374003a80 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c06c200 con 0x7ff38c15abd0 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c09ab90 con 0x7ff38c0c65e0 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41382/0 (socket says 172.21.15.82:41382) 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 -- 172.21.15.82:0/3579438633 learned_addr learned my addr 172.21.15.82:0/3579438633 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38a7fc640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 0x7ff38c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38b7fe640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 0x7ff38c0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 -- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 msgr2=0x7ff38c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 0x7ff38c0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 -- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 msgr2=0x7ff38c0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 0x7ff38c0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 -- 172.21.15.82:0/3579438633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0b6210 con 0x7ff374003a80 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38a7fc640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 0x7ff38c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38b7fe640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 0x7ff38c0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff38affd640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff37c01c700 tx=0x7ff37c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0ba070 con 0x7ff374003a80 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c0ada20 con 0x7ff374003a80 2024-04-25T10:08:42.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c0c5480 con 0x7ff374003a80 2024-04-25T10:08:42.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c0b2040 con 0x7ff374003a80 2024-04-25T10:08:42.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c0b51f0 con 0x7ff374003a80 2024-04-25T10:08:42.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.316+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c04efa0 con 0x7ff374003a80 2024-04-25T10:08:42.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.320+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff37c0b6410 con 0x7ff374003a80 2024-04-25T10:08:42.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.320+0000 7ff373fff640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 0x7ff378069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.320+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff37c11f480 con 0x7ff374003a80 2024-04-25T10:08:42.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.320+0000 7ff38a7fc640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 0x7ff378069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.320+0000 7ff38a7fc640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 0x7ff378069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff38c050240 tx=0x7ff380009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.324+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff38c0b6210 con 0x7ff374003a80 2024-04-25T10:08:42.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.544+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7ff38c06ba80 con 0x7ff374003a80 2024-04-25T10:08:42.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff373fff640 1 -- 172.21.15.82:0/3579438633 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v324) v1 ==== 166+0+0 (secure 0 0 0) 0x7ff37c0169d0 con 0x7ff374003a80 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 msgr2=0x7ff378069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 0x7ff378069450 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff38c050240 tx=0x7ff380009040 comp rx=0 tx=0).stop 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 msgr2=0x7ff38c0b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7ff37c01c700 tx=0x7ff37c009750 comp rx=0 tx=0).stop 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 shutdown_connections 2024-04-25T10:08:42.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff378066f90 0x7ff378069450 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff38c0c65e0 0x7ff38c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff38c15abd0 0x7ff38c0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 --2- 172.21.15.82:0/3579438633 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff374003a80 0x7ff38c0b5cd0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 >> 172.21.15.82:0/3579438633 conn(0x7ff38c0b3010 msgr2=0x7ff38c15cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 shutdown_connections 2024-04-25T10:08:42.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.836+0000 7ff391862640 1 -- 172.21.15.82:0/3579438633 wait complete. 2024-04-25T10:08:42.863 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:08:42.863 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T10:08:42.863 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:08:42.863 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:42.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.980+0000 7f8f0721a640 1 Processor -- start 2024-04-25T10:08:42.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.980+0000 7f8f0721a640 1 -- start start 2024-04-25T10:08:42.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f0721a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8f0006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f0721a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f000ccc70 con 0x7f8f0006ec60 2024-04-25T10:08:42.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f0721a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0006dc40 con 0x7f8f000ce510 2024-04-25T10:08:42.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f0721a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000cc2b0 con 0x7f8f00144ca0 2024-04-25T10:08:42.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8f0006ec60 0x7f8f00142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43250/0 (socket says 172.21.15.82:43250) 2024-04-25T10:08:42.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 -- 172.21.15.82:0/3093307880 learned_addr learned my addr 172.21.15.82:0/3093307880 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8f0006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112430544 0 0) 0x7f8f000ccc70 con 0x7f8f0006ec60 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ee8003530 con 0x7f8f0006ec60 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783408586 0 0) 0x7f8ee8003530 con 0x7f8f0006ec60 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 msgr2=0x7f8f0006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8f0006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 >> v1:172.21.15.196:6789/0 conn(0x7f8f000ce510 legacy=0x7f8f000ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0006df20 con 0x7f8f0006ec60 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8ef8001a00 con 0x7f8f0006ec60 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 >> v1:172.21.15.82:6789/0 conn(0x7f8f0006ec60 legacy=0x7f8f00142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee8003a60 0x7f8ee8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8ee8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee8004d50 0x7f8ee80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f000cc2b0 con 0x7f8ee8004d50 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee800bb70 con 0x7f8ee8003a60 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee800bd70 con 0x7f8f00144ca0 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee8004d50 0x7f8ee80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f04fb6640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee8003a60 0x7f8ee8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8ee8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 -- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 msgr2=0x7f8ee8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f00144ca0 0x7f8ee8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 -- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee8003a60 msgr2=0x7f8ee8003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ee8003a60 0x7f8ee8003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.984+0000 7f8f057b7640 1 -- 172.21.15.82:0/3093307880 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8003530 con 0x7f8ee8004d50 2024-04-25T10:08:42.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f057b7640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee8004d50 0x7f8ee80051d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8002240 tx=0x7f8ef801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d86ff785de5aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef800a4a0 con 0x7f8ee8004d50 2024-04-25T10:08:42.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef80ad650 con 0x7f8ee8004d50 2024-04-25T10:08:42.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8ef77fe640 1 -- 172.21.15.82:0/3093307880 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef80ada40 con 0x7f8ee8004d50 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- 172.21.15.82:0/3093307880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee8004d50 msgr2=0x7f8ee80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 --2- 172.21.15.82:0/3093307880 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ee8004d50 0x7f8ee80051d0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8002240 tx=0x7f8ef801d300 comp rx=0 tx=0).stop 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f057b7640 1 -- 172.21.15.82:0/3093307880 reap_dead start 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- 172.21.15.82:0/3093307880 shutdown_connections 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- 172.21.15.82:0/3093307880 >> 172.21.15.82:0/3093307880 conn(0x7f8f000b11a0 msgr2=0x7f8f000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- 172.21.15.82:0/3093307880 shutdown_connections 2024-04-25T10:08:42.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- 172.21.15.82:0/3093307880 wait complete. 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 Processor -- start 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- start start 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 0x7f8f0014da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f0014fad0 0x7f8f0014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f00142d90 con 0x7f8f000a9d30 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0006e290 con 0x7f8f0015abb0 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f0721a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f00142f10 con 0x7f8f0014fad0 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f04fb6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f04fb6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41412/0 (socket says 172.21.15.82:41412) 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.988+0000 7f8f04fb6640 1 -- 172.21.15.82:0/1583469987 learned_addr learned my addr 172.21.15.82:0/1583469987 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f057b7640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f0014fad0 0x7f8f0014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 0x7f8f0014da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 -- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f0014fad0 msgr2=0x7f8f0014ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f0014fad0 0x7f8f0014ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 -- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 msgr2=0x7f8f0014da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 0x7f8f0014da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 -- 172.21.15.82:0/1583469987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0014e3f0 con 0x7f8f000a9d30 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 0x7f8f0014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f04fb6640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e5b0 tx=0x7f8efc00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0052e0 con 0x7f8f000a9d30 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc018070 con 0x7f8f000a9d30 2024-04-25T10:08:42.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0014e6d0 con 0x7f8f000a9d30 2024-04-25T10:08:42.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc003840 con 0x7f8f000a9d30 2024-04-25T10:08:42.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0014ec90 con 0x7f8f000a9d30 2024-04-25T10:08:42.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0004efa0 con 0x7f8f000a9d30 2024-04-25T10:08:42.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8efc0039e0 con 0x7f8f000a9d30 2024-04-25T10:08:43.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 0x7f8eec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8efc084d30 con 0x7f8f000a9d30 2024-04-25T10:08:43.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 0x7f8eec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.992+0000 7f8ef7fff640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 0x7f8eec069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f001f4b60 tx=0x7f8ef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:42.996+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8efc00ce60 con 0x7f8f000a9d30 2024-04-25T10:08:43.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.156+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f000b1cc0 con 0x7f8f000a9d30 2024-04-25T10:08:43.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8ef57fa640 1 -- 172.21.15.82:0/1583469987 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+13434 (secure 0 0 0) 0x7f8efc005480 con 0x7f8f000a9d30 2024-04-25T10:08:43.164 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:43.165 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:41.829283+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/3956208581":"2024-04-26T10:08:38.814564+0000","172.21.15.82:6836/3956208581":"2024-04-26T10:08:38.814564+0000","172.21.15.82:6837/910515233":"2024-04-26T10:07:09.947207+0000","172.21.15.82:6836/910515233":"2024-04-26T10:07:09.947207+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 msgr2=0x7f8eec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 0x7f8eec069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f8f001f4b60 tx=0x7f8ef0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 msgr2=0x7f8f0014f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00e5b0 tx=0x7f8efc00ea70 comp rx=0 tx=0).stop 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 shutdown_connections 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8eec066f60 0x7f8eec069420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8f0014fad0 0x7f8f0014ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8f0015abb0 0x7f8f0014da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 --2- 172.21.15.82:0/1583469987 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8f000a9d30 0x7f8f0014f490 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 >> 172.21.15.82:0/1583469987 conn(0x7f8f000b11a0 msgr2=0x7f8f00071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 shutdown_connections 2024-04-25T10:08:43.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.160+0000 7f8f0721a640 1 -- 172.21.15.82:0/1583469987 wait complete. 2024-04-25T10:08:43.184 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:43.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:43.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 Processor -- start 2024-04-25T10:08:43.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 -- start start 2024-04-25T10:08:43.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7c0c06bca0 0x7f7c0c0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c0c066e30 0x7f7c0c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c0c06ec30 con 0x7f7c0c0af370 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c0669a0 con 0x7f7c0c066e30 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c14353640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0c0cd5f0 con 0x7f7c0c06bca0 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c0c066e30 0x7f7c0c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c0c066e30 0x7f7c0c06c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41428/0 (socket says 172.21.15.82:41428) 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 -- 172.21.15.82:0/3791206943 learned_addr learned my addr 172.21.15.82:0/3791206943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c128f0640 1 --1- 172.21.15.82:0/3791206943 >> v1:172.21.15.82:6789/0 conn(0x7f7c0c0af370 0x7f7c0c06d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43258/0 (socket says 172.21.15.82:43258) 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c120ef640 1 --2- 172.21.15.82:0/3791206943 >> v2:172.21.15.196:3300/0 conn(0x7f7c0c06bca0 0x7f7c0c0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 -- 172.21.15.82:0/3791206943 >> v2:172.21.15.196:3300/0 conn(0x7f7c0c06bca0 msgr2=0x7f7c0c0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 --2- 172.21.15.82:0/3791206943 >> v2:172.21.15.196:3300/0 conn(0x7f7c0c06bca0 0x7f7c0c0cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 -- 172.21.15.82:0/3791206943 >> v1:172.21.15.82:6789/0 conn(0x7f7c0c0af370 legacy=0x7f7c0c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c118ee640 1 -- 172.21.15.82:0/3791206943 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c06c5c0 con 0x7f7c0c066e30 2024-04-25T10:08:43.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.300+0000 7f7c110ed640 1 -- 172.21.15.82:0/3791206943 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914464877 0 0) 0x7f7c0c06ec30 con 0x7f7c0c0af370 2024-04-25T10:08:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c118ee640 1 --2- 172.21.15.82:0/3791206943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c066e30 0x7f7c0c06c080 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0b5ef0 tx=0x7f7c0801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8b002837443a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c110ed640 1 -- 172.21.15.82:0/3791206943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080b0070 con 0x7f7c0c066e30 2024-04-25T10:08:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c110ed640 1 -- 172.21.15.82:0/3791206943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c080ab600 con 0x7f7c0c066e30 2024-04-25T10:08:43.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c110ed640 1 -- 172.21.15.82:0/3791206943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080ab9d0 con 0x7f7c0c066e30 2024-04-25T10:08:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- 172.21.15.82:0/3791206943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c066e30 msgr2=0x7f7c0c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- 172.21.15.82:0/3791206943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c066e30 0x7f7c0c06c080 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c0b5ef0 tx=0x7f7c0801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- 172.21.15.82:0/3791206943 shutdown_connections 2024-04-25T10:08:43.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- 172.21.15.82:0/3791206943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c066e30 0x7f7c0c06c080 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- 172.21.15.82:0/3791206943 >> v2:172.21.15.196:3300/0 conn(0x7f7c0c06bca0 0x7f7c0c0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- 172.21.15.82:0/3791206943 >> 172.21.15.82:0/3791206943 conn(0x7f7c0c071810 msgr2=0x7f7c0c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- 172.21.15.82:0/3791206943 shutdown_connections 2024-04-25T10:08:43.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- 172.21.15.82:0/3791206943 wait complete. 2024-04-25T10:08:43.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 Processor -- start 2024-04-25T10:08:43.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- start start 2024-04-25T10:08:43.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 0x7f7c0c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c0c0af370 0x7f7c0c1571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15cc80 con 0x7f7c0c06bca0 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c15cb00 con 0x7f7c0c066e30 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c14353640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0c168630 con 0x7f7c0c0af370 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c118ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c118ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41442/0 (socket says 172.21.15.82:41442) 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.304+0000 7f7c118ee640 1 -- 172.21.15.82:0/2972814700 learned_addr learned my addr 172.21.15.82:0/2972814700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c120ef640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 0x7f7c0c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c128f0640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c0c0af370 0x7f7c0c1571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 -- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c0c0af370 msgr2=0x7f7c0c1571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c0c0af370 0x7f7c0c1571a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 -- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 msgr2=0x7f7c0c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 0x7f7c0c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 -- 172.21.15.82:0/2972814700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0c0b0890 con 0x7f7c0c06bca0 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c120ef640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 0x7f7c0c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c118ee640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7c0801a000 tx=0x7f7c08019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080b0040 con 0x7f7c0c06bca0 2024-04-25T10:08:43.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c080b4420 con 0x7f7c0c06bca0 2024-04-25T10:08:43.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0c0b0b70 con 0x7f7c0c06bca0 2024-04-25T10:08:43.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080b4810 con 0x7f7c0c06bca0 2024-04-25T10:08:43.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c0c153700 con 0x7f7c0c06bca0 2024-04-25T10:08:43.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0c06c230 con 0x7f7c0c06bca0 2024-04-25T10:08:43.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c080b49b0 con 0x7f7c0c06bca0 2024-04-25T10:08:43.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 0x7f7bf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7c0811f430 con 0x7f7c0c06bca0 2024-04-25T10:08:43.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c120ef640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 0x7f7bf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.308+0000 7f7c120ef640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 0x7f7bf0069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097a0 tx=0x7f7bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.312+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c08016110 con 0x7f7c0c06bca0 2024-04-25T10:08:43.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.460+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c0c072400 con 0x7f7c0c06bca0 2024-04-25T10:08:43.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.460+0000 7f7c02ffd640 1 -- 172.21.15.82:0/2972814700 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c080f05b0 con 0x7f7c0c06bca0 2024-04-25T10:08:43.467 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:08:43.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 msgr2=0x7f7bf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 0x7f7bf0069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0097a0 tx=0x7f7bfc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 msgr2=0x7f7c0c13b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f7c0801a000 tx=0x7f7c08019170 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 shutdown_connections 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7bf0066f90 0x7f7bf0069450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c0c0af370 0x7f7c0c1571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c0c06bca0 0x7f7c0c13b3c0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 --2- 172.21.15.82:0/2972814700 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c0c066e30 0x7f7c0c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 >> 172.21.15.82:0/2972814700 conn(0x7f7c0c071810 msgr2=0x7f7c0c159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 shutdown_connections 2024-04-25T10:08:43.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.464+0000 7f7c14353640 1 -- 172.21.15.82:0/2972814700 wait complete. 2024-04-25T10:08:43.485 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:08:43.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 Processor -- start 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 -- start start 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcee80c44f0 0x7fcee8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 0x7fcee80c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcee809a150 con 0x7fcee80cc0c0 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcee80cbc30 con 0x7fcee80c44f0 2024-04-25T10:08:43.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceefe50640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee809a430 con 0x7fcee80b3b30 2024-04-25T10:08:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceed3eb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 0x7fcee80c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceedbec640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcee80c44f0 0x7fcee8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceed3eb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 0x7fcee80c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:38992/0 (socket says 172.21.15.82:38992) 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceed3eb640 1 -- 172.21.15.82:0/7480969 learned_addr learned my addr 172.21.15.82:0/7480969 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceedbec640 1 -- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 msgr2=0x7fcee80c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceedbec640 1 --2- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 0x7fcee80c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceedbec640 1 -- 172.21.15.82:0/7480969 >> v1:172.21.15.196:6789/0 conn(0x7fcee80cc0c0 legacy=0x7fcee80cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.588+0000 7fceedbec640 1 -- 172.21.15.82:0/7480969 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee809a710 con 0x7fcee80c44f0 2024-04-25T10:08:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceedbec640 1 --2- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80c44f0 0x7fcee8099a70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fced80097c0 tx=0x7fced801c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f9c75b43375c5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceecbea640 1 -- 172.21.15.82:0/7480969 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80b0070 con 0x7fcee80c44f0 2024-04-25T10:08:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceecbea640 1 -- 172.21.15.82:0/7480969 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced80ab600 con 0x7fcee80c44f0 2024-04-25T10:08:43.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceecbea640 1 -- 172.21.15.82:0/7480969 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced80a4e60 con 0x7fcee80c44f0 2024-04-25T10:08:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80c44f0 msgr2=0x7fcee8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80c44f0 0x7fcee8099a70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fced80097c0 tx=0x7fced801c380 comp rx=0 tx=0).stop 2024-04-25T10:08:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- 172.21.15.82:0/7480969 shutdown_connections 2024-04-25T10:08:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80b3b30 0x7fcee80c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- 172.21.15.82:0/7480969 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80c44f0 0x7fcee8099a70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- 172.21.15.82:0/7480969 >> 172.21.15.82:0/7480969 conn(0x7fcee80b3010 msgr2=0x7fcee80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- 172.21.15.82:0/7480969 shutdown_connections 2024-04-25T10:08:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- 172.21.15.82:0/7480969 wait complete. 2024-04-25T10:08:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 Processor -- start 2024-04-25T10:08:43.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- start start 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 0x7fcee815a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80c44f0 0x7fcee8152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee816e9f0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee816e870 con 0x7fcee80b3b30 2024-04-25T10:08:43.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceefe50640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcee816eb70 con 0x7fcee80c44f0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.592+0000 7fceee3ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41450/0 (socket says 172.21.15.82:41450) 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 -- 172.21.15.82:0/458593950 learned_addr learned my addr 172.21.15.82:0/458593950 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceedbec640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 0x7fcee815a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceed3eb640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80c44f0 0x7fcee8152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 -- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80c44f0 msgr2=0x7fcee8152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80c44f0 0x7fcee8152380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 -- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 msgr2=0x7fcee815a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 0x7fcee815a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 -- 172.21.15.82:0/458593950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee815b560 con 0x7fcee80cc0c0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceedbec640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 0x7fcee815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceee3ed640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fcee000c430 tx=0x7fcee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0005220 con 0x7fcee80cc0c0 2024-04-25T10:08:43.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcee00053c0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee815b840 con 0x7fcee80cc0c0 2024-04-25T10:08:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcee0007270 con 0x7fcee80cc0c0 2024-04-25T10:08:43.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee80c9cf0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee804efa0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcee0019020 con 0x7fcee80cc0c0 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 0x7fced4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcee0084220 con 0x7fcee80cc0c0 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceedbec640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 0x7fced4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.596+0000 7fceedbec640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 0x7fced4069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fced80ad000 tx=0x7fced80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.600+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcee00553e0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.800+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fcee806ba80 con 0x7fcee80cc0c0 2024-04-25T10:08:43.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.840+0000 7fcee67fc640 1 -- 172.21.15.82:0/458593950 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v325) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcee0054ce0 con 0x7fcee80cc0c0 2024-04-25T10:08:43.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 msgr2=0x7fced4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 0x7fced4069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fced80ad000 tx=0x7fced80a6000 comp rx=0 tx=0).stop 2024-04-25T10:08:43.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 msgr2=0x7fcee8151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fcee000c430 tx=0x7fcee000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 shutdown_connections 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fced4066f60 0x7fced4069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcee80c44f0 0x7fcee8152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcee80cc0c0 0x7fcee8151720 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 --2- 172.21.15.82:0/458593950 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcee80b3b30 0x7fcee815a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 >> 172.21.15.82:0/458593950 conn(0x7fcee80b3010 msgr2=0x7fcee80b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 shutdown_connections 2024-04-25T10:08:43.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.844+0000 7fceefe50640 1 -- 172.21.15.82:0/458593950 wait complete. 2024-04-25T10:08:43.867 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:08:43.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 Processor -- start 2024-04-25T10:08:43.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 -- start start 2024-04-25T10:08:43.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f499c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f499c0d07f0 0x7f499c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c06be60 con 0x7f499c0d1cd0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c06c440 con 0x7f499c0d07f0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f49a2123640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15de60 con 0x7f499c15da80 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f499c0d1cd0 0x7f499c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43266/0 (socket says 172.21.15.82:43266) 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499affd640 1 -- 172.21.15.82:0/2950384209 learned_addr learned my addr 172.21.15.82:0/2950384209 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 --2- 172.21.15.82:0/2950384209 >> v2:172.21.15.196:3300/0 conn(0x7f499c0d07f0 0x7f499c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499b7fe640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f499c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935177526 0 0) 0x7f499c06be60 con 0x7f499c0d1cd0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4984003530 con 0x7f499c0d1cd0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3834165693 0 0) 0x7f4984003530 con 0x7f499c0d1cd0 2024-04-25T10:08:43.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 msgr2=0x7f499c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f499c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 >> v2:172.21.15.196:3300/0 conn(0x7f499c0d07f0 msgr2=0x7f499c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 --2- 172.21.15.82:0/2950384209 >> v2:172.21.15.196:3300/0 conn(0x7f499c0d07f0 0x7f499c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c0d2450 con 0x7f499c0d1cd0 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f49900016d0 con 0x7f499c0d1cd0 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 >> v1:172.21.15.82:6789/0 conn(0x7f499c0d1cd0 legacy=0x7f499c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4984003a30 0x7f4984003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f4984004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4984004d50 0x7f49840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c06c440 con 0x7f4984004d50 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15de60 con 0x7f4984003a30 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498400b9d0 con 0x7f499c15da80 2024-04-25T10:08:43.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4984004d50 0x7f49840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499b7fe640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4984003a30 0x7f4984003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499affd640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f4984004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 -- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 msgr2=0x7f4984004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f4984004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 -- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4984003a30 msgr2=0x7f4984003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4984003a30 0x7f4984003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499bfff640 1 -- 172.21.15.82:0/2950384209 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4984003530 con 0x7f4984004d50 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499b7fe640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4984003a30 0x7f4984003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.984+0000 7f499affd640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c15da80 0x7f4984004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:43.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f499bfff640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4984004d50 0x7f49840051d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4994009f90 tx=0x7f499401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd01769ce24c42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940b2070 con 0x7f4984004d50 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49940a5c90 con 0x7f4984004d50 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f499a7fc640 1 -- 172.21.15.82:0/2950384209 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4994009c10 con 0x7f4984004d50 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- 172.21.15.82:0/2950384209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4984004d50 msgr2=0x7f49840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 --2- 172.21.15.82:0/2950384209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4984004d50 0x7f49840051d0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f4994009f90 tx=0x7f499401cc80 comp rx=0 tx=0).stop 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f499bfff640 1 -- 172.21.15.82:0/2950384209 reap_dead start 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- 172.21.15.82:0/2950384209 shutdown_connections 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- 172.21.15.82:0/2950384209 >> 172.21.15.82:0/2950384209 conn(0x7f499c08b000 msgr2=0x7f499c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- 172.21.15.82:0/2950384209 shutdown_connections 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- 172.21.15.82:0/2950384209 wait complete. 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 Processor -- start 2024-04-25T10:08:43.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.988+0000 7f49a2123640 1 -- start start 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f499c0cbb70 0x7f499c1f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c0ccf90 0x7f499c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0d26f0 con 0x7f499c15a3f0 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0d20f0 con 0x7f499c0cbb70 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0d2870 con 0x7f499c0ccf90 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41464/0 (socket says 172.21.15.82:41464) 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 -- 172.21.15.82:0/3235191466 learned_addr learned my addr 172.21.15.82:0/3235191466 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499b7fe640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f499c0cbb70 0x7f499c1f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499affd640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c0ccf90 0x7f499c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 -- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c0ccf90 msgr2=0x7f499c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c0ccf90 0x7f499c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 -- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f499c0cbb70 msgr2=0x7f499c1f3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f499c0cbb70 0x7f499c1f3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 -- 172.21.15.82:0/3235191466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c158780 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499bfff640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f4994009f90 tx=0x7f4994009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940bd070 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49940a5a90 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940b2040 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c158a60 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c159020 con 0x7f499c15a3f0 2024-04-25T10:08:44.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04efa0 con 0x7f499c15a3f0 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49940c0070 con 0x7f499c15a3f0 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 0x7f4988069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f499411e970 con 0x7f499c15a3f0 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499b7fe640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 0x7f4988069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.992+0000 7f499b7fe640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 0x7f4988069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f499c0ca770 tx=0x7f498c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:43.996+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4994016990 con 0x7f499c15a3f0 2024-04-25T10:08:44.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.156+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f499c0cf670 con 0x7f499c15a3f0 2024-04-25T10:08:44.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.160+0000 7f4983fff640 1 -- 172.21.15.82:0/3235191466 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f49940ef4e0 con 0x7f499c15a3f0 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 msgr2=0x7f4988069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 0x7f4988069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f499c0ca770 tx=0x7f498c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 msgr2=0x7f499c15a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f4994009f90 tx=0x7f4994009750 comp rx=0 tx=0).stop 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 shutdown_connections 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4988066f60 0x7f4988069420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f499c0ccf90 0x7f499c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f499c15a3f0 0x7f499c15a860 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 --2- 172.21.15.82:0/3235191466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f499c0cbb70 0x7f499c1f3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 >> 172.21.15.82:0/3235191466 conn(0x7f499c08b000 msgr2=0x7f499c160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 shutdown_connections 2024-04-25T10:08:44.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.164+0000 7f49a2123640 1 -- 172.21.15.82:0/3235191466 wait complete. 2024-04-25T10:08:44.186 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:44.186 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:44.257 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:44.305 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:44.305 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:08:44.305 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:44.310 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:44.311 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:08:44.311 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:44.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:44.356 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:08:44.356 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:44.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:44.361 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:08:44.362 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:44.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:44.404 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:08:44.404 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.524+0000 7f8abf64b640 1 Processor -- start 2024-04-25T10:08:44.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 -- start start 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ab8159ea0 0x7f8ab80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab815ae30 con 0x7f8ab815a280 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab80b2990 con 0x7f8ab8159ea0 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abf64b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab815b110 con 0x7f8ab8071fd0 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ab8159ea0 0x7f8ab80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ab8159ea0 0x7f8ab80b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41480/0 (socket says 172.21.15.82:41480) 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 -- 172.21.15.82:0/1886480292 learned_addr learned my addr 172.21.15.82:0/1886480292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 -- 172.21.15.82:0/1886480292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 msgr2=0x7f8ab815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abdbe8640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 -- 172.21.15.82:0/1886480292 >> v1:172.21.15.196:6789/0 conn(0x7f8ab815a280 legacy=0x7f8ab80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.528+0000 7f8abd3e7640 1 -- 172.21.15.82:0/1886480292 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80b2620 con 0x7f8ab8159ea0 2024-04-25T10:08:44.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab8159ea0 0x7f8ab80b2e20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e50 tx=0x7f8aa801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18e6239af6e9470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8aaffff640 1 -- 172.21.15.82:0/1886480292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80a5810 con 0x7f8ab8159ea0 2024-04-25T10:08:44.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8aaffff640 1 -- 172.21.15.82:0/1886480292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa80a59b0 con 0x7f8ab8159ea0 2024-04-25T10:08:44.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8aaffff640 1 -- 172.21.15.82:0/1886480292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80a5d80 con 0x7f8ab8159ea0 2024-04-25T10:08:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1886480292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab8159ea0 msgr2=0x7f8ab80b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab8159ea0 0x7f8ab80b2e20 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e50 tx=0x7f8aa801ca30 comp rx=0 tx=0).stop 2024-04-25T10:08:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1886480292 shutdown_connections 2024-04-25T10:08:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1886480292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab8159ea0 0x7f8ab80b2e20 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1886480292 >> 172.21.15.82:0/1886480292 conn(0x7f8ab80b11a0 msgr2=0x7f8ab80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1886480292 shutdown_connections 2024-04-25T10:08:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1886480292 wait complete. 2024-04-25T10:08:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 Processor -- start 2024-04-25T10:08:44.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- start start 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab8150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ab8159ea0 0x7f8ab814c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8166d40 con 0x7f8ab815a280 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8166bc0 con 0x7f8ab8159ea0 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8166ec0 con 0x7f8ab8071fd0 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41494/0 (socket says 172.21.15.82:41494) 2024-04-25T10:08:44.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 -- 172.21.15.82:0/1110593791 learned_addr learned my addr 172.21.15.82:0/1110593791 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab8150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abcbe6640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ab8159ea0 0x7f8ab814c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 -- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 msgr2=0x7f8ab8150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab8150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 -- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ab8159ea0 msgr2=0x7f8ab814c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ab8159ea0 0x7f8ab814c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 -- 172.21.15.82:0/1110593791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab814ae60 con 0x7f8ab815a280 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab8150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abdbe8640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c820 tx=0x7f8ab400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4015070 con 0x7f8ab815a280 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab4004220 con 0x7f8ab815a280 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.532+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab814b140 con 0x7f8ab815a280 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab400ce10 con 0x7f8ab815a280 2024-04-25T10:08:44.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab8150e90 con 0x7f8ab815a280 2024-04-25T10:08:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab8151090 con 0x7f8ab815a280 2024-04-25T10:08:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ab4007830 con 0x7f8ab815a280 2024-04-25T10:08:44.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8aadffb640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 0x7f8aa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8ab4084e90 con 0x7f8ab815a280 2024-04-25T10:08:44.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 0x7f8aa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.536+0000 7f8abd3e7640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 0x7f8aa4069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ae000 tx=0x7f8aa80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.540+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ab8151090 con 0x7f8ab815a280 2024-04-25T10:08:44.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.720+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ab804efa0 con 0x7f8ab815a280 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8aadffb640 1 -- 172.21.15.82:0/1110593791 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f8ab4055ff0 con 0x7f8ab815a280 2024-04-25T10:08:44.727 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:44.728 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5231,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/2893791389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":2893791389},{"type":"v1","addr":"172.21.15.196:6833","nonce":2893791389}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":5740,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3684224535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3684224535},{"type":"v1","addr":"172.21.15.196:6835","nonce":3684224535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6963,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/3881433208","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":3881433208},{"type":"v1","addr":"172.21.15.82:6835","nonce":3881433208}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7713,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/2711277075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2711277075},{"type":"v1","addr":"172.21.15.82:6837","nonce":2711277075}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2024-04-25T10:08:44.728 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 51 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 msgr2=0x7f8aa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 0x7f8aa4069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80ae000 tx=0x7f8aa80a7070 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 msgr2=0x7f8ab814ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f8ab400c820 tx=0x7f8ab400cce0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 shutdown_connections 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8aa4066f60 0x7f8aa4069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ab815a280 0x7f8ab814ce60 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ab8159ea0 0x7f8ab814c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 --2- 172.21.15.82:0/1110593791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ab8071fd0 0x7f8ab8150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 >> 172.21.15.82:0/1110593791 conn(0x7f8ab80b11a0 msgr2=0x7f8ab80cf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 shutdown_connections 2024-04-25T10:08:44.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.724+0000 7f8abf64b640 1 -- 172.21.15.82:0/1110593791 wait complete. 2024-04-25T10:08:44.745 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:44.745 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:44.745 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:44.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:44.746 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:44.746 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:44.746 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:44.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:44.746 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:44.746 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:44.747 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:44.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:44.747 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:44.747 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:44.747 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:44.747 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:44.758 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:44.759 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:44.759 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:44.760 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:44.761 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:44.761 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:44.762 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:44.762 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:44.801 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:44.801 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:44.802 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:44.802 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:44.809 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:44.809 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:44.809 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:44.810 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:44.810 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:44.858 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:08:44.860 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:08:44.886 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:08:44.886 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:08:44.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 Processor -- start 2024-04-25T10:08:44.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 -- start start 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb120066e30 0x7fb12006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12006ec30 con 0x7fb1200af370 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb12006c5c0 con 0x7fb12006bca0 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb1270d8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1200669a0 con 0x7fb120066e30 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb120066e30 0x7fb12006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb125675640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb1200af370 0x7fb1200ceaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42220/0 (socket says 172.21.15.82:42220) 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb120066e30 0x7fb12006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49490/0 (socket says 172.21.15.82:49490) 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb125675640 1 -- 172.21.15.82:0/3399207099 learned_addr learned my addr 172.21.15.82:0/3399207099 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:44.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11ffff640 1 -- 172.21.15.82:0/3399207099 >> v1:172.21.15.196:6789/0 conn(0x7fb12006bca0 legacy=0x7fb1200cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11ffff640 1 -- 172.21.15.82:0/3399207099 >> v1:172.21.15.82:6789/0 conn(0x7fb1200af370 legacy=0x7fb1200ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11ffff640 1 -- 172.21.15.82:0/3399207099 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12006f230 con 0x7fb120066e30 2024-04-25T10:08:44.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.916+0000 7fb11f7fe640 1 -- 172.21.15.82:0/3399207099 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201749958 0 0) 0x7fb12006ec30 con 0x7fb1200af370 2024-04-25T10:08:44.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb11ffff640 1 --2- 172.21.15.82:0/3399207099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120066e30 0x7fb12006c080 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0097c0 tx=0x7fb10c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f630693bf0b9c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb11f7fe640 1 -- 172.21.15.82:0/3399207099 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0b0070 con 0x7fb120066e30 2024-04-25T10:08:44.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb11f7fe640 1 -- 172.21.15.82:0/3399207099 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c0ab600 con 0x7fb120066e30 2024-04-25T10:08:44.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb11f7fe640 1 -- 172.21.15.82:0/3399207099 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0ab9d0 con 0x7fb120066e30 2024-04-25T10:08:44.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- 172.21.15.82:0/3399207099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120066e30 msgr2=0x7fb12006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 --2- 172.21.15.82:0/3399207099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120066e30 0x7fb12006c080 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0097c0 tx=0x7fb10c01c350 comp rx=0 tx=0).stop 2024-04-25T10:08:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- 172.21.15.82:0/3399207099 shutdown_connections 2024-04-25T10:08:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 --2- 172.21.15.82:0/3399207099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb120066e30 0x7fb12006c080 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- 172.21.15.82:0/3399207099 >> 172.21.15.82:0/3399207099 conn(0x7fb120071810 msgr2=0x7fb120071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- 172.21.15.82:0/3399207099 shutdown_connections 2024-04-25T10:08:44.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- 172.21.15.82:0/3399207099 wait complete. 2024-04-25T10:08:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 Processor -- start 2024-04-25T10:08:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.920+0000 7fb1270d8640 1 -- start start 2024-04-25T10:08:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 0x7fb12014d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1200af370 0x7fb120152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1201697c0 con 0x7fb12006bca0 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb120169640 con 0x7fb1200af370 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb120169940 con 0x7fb120066e30 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49506/0 (socket says 172.21.15.82:49506) 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 -- 172.21.15.82:0/483437064 learned_addr learned my addr 172.21.15.82:0/483437064 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb125675640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1200af370 0x7fb120152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb124e74640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 0x7fb12014d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 -- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 msgr2=0x7fb12014d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 0x7fb12014d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 -- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1200af370 msgr2=0x7fb120152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:44.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1200af370 0x7fb120152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 -- 172.21.15.82:0/483437064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12014daa0 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb124e74640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 0x7fb12014d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11ffff640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb10c018f60 tx=0x7fb10c0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0b8020 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c0b4630 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0b0040 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb12014dd80 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1200b30f0 con 0x7fb12006bca0 2024-04-25T10:08:44.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb12004efa0 con 0x7fb12006bca0 2024-04-25T10:08:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb10c0bb020 con 0x7fb12006bca0 2024-04-25T10:08:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 0x7fb104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb10c11eb90 con 0x7fb12006bca0 2024-04-25T10:08:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb124e74640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 0x7fb104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:44.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.924+0000 7fb124e74640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 0x7fb104069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb114000f70 tx=0x7fb114009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:44.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:44.928+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb10c0c2030 con 0x7fb12006bca0 2024-04-25T10:08:45.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.084+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb120072910 con 0x7fb12006bca0 2024-04-25T10:08:45.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.860+0000 7fb11d7fa640 1 -- 172.21.15.82:0/483437064 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb10c016090 con 0x7fb12006bca0 2024-04-25T10:08:45.866 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:08:45.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 msgr2=0x7fb104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 0x7fb104069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb114000f70 tx=0x7fb114009040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 msgr2=0x7fb120159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fb10c018f60 tx=0x7fb10c0190f0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 shutdown_connections 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb104066f60 0x7fb104069420 secure :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fb114000f70 tx=0x7fb114009040 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1200af370 0x7fb120152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb12006bca0 0x7fb120159720 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 --2- 172.21.15.82:0/483437064 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb120066e30 0x7fb12014d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 >> 172.21.15.82:0/483437064 conn(0x7fb120071810 msgr2=0x7fb1200af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 shutdown_connections 2024-04-25T10:08:45.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:45.864+0000 7fb1270d8640 1 -- 172.21.15.82:0/483437064 wait complete. 2024-04-25T10:08:45.893 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:46.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.004+0000 7fbc7e571640 1 Processor -- start 2024-04-25T10:08:46.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.004+0000 7fbc7e571640 1 -- start start 2024-04-25T10:08:46.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7e571640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbc7815a280 0x7fbc7815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7e571640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc780718c0 0x7fbc78071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7e571640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc7815ae30 con 0x7fbc78159ea0 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7e571640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc780b2990 con 0x7fbc780718c0 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7e571640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc780b3530 con 0x7fbc7815a280 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc780718c0 0x7fbc78071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbc780718c0 0x7fbc78071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49512/0 (socket says 172.21.15.82:49512) 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 -- 172.21.15.82:0/2742377872 learned_addr learned my addr 172.21.15.82:0/2742377872 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc777fe640 1 --2- 172.21.15.82:0/2742377872 >> v2:172.21.15.196:3300/0 conn(0x7fbc7815a280 0x7fbc7815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 -- 172.21.15.82:0/2742377872 >> v2:172.21.15.196:3300/0 conn(0x7fbc7815a280 msgr2=0x7fbc7815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 --2- 172.21.15.82:0/2742377872 >> v2:172.21.15.196:3300/0 conn(0x7fbc7815a280 0x7fbc7815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 -- 172.21.15.82:0/2742377872 >> v1:172.21.15.196:6789/0 conn(0x7fbc78159ea0 legacy=0x7fbc780b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 -- 172.21.15.82:0/2742377872 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc7815b110 con 0x7fbc780718c0 2024-04-25T10:08:46.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc7cb0e640 1 --2- 172.21.15.82:0/2742377872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc78071d50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbc780728e0 tx=0x7fbc6801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9210a3b93c99a17e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc76ffd640 1 -- 172.21.15.82:0/2742377872 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680ad070 con 0x7fbc780718c0 2024-04-25T10:08:46.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc76ffd640 1 -- 172.21.15.82:0/2742377872 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc680a7b10 con 0x7fbc780718c0 2024-04-25T10:08:46.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.008+0000 7fbc76ffd640 1 -- 172.21.15.82:0/2742377872 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc680b1400 con 0x7fbc780718c0 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2742377872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 msgr2=0x7fbc78071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2742377872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc78071d50 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fbc780728e0 tx=0x7fbc6801ebb0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2742377872 shutdown_connections 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2742377872 >> v2:172.21.15.196:3300/0 conn(0x7fbc7815a280 0x7fbc7815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2742377872 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc78071d50 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2742377872 >> 172.21.15.82:0/2742377872 conn(0x7fbc780b11a0 msgr2=0x7fbc780b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2742377872 shutdown_connections 2024-04-25T10:08:46.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2742377872 wait complete. 2024-04-25T10:08:46.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 Processor -- start 2024-04-25T10:08:46.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- start start 2024-04-25T10:08:46.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc78159ea0 0x7fbc781522a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc7815a280 0x7fbc781527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7806d420 con 0x7fbc780718c0 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7806d2a0 con 0x7fbc78159ea0 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7806d5a0 con 0x7fbc7815a280 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49522/0 (socket says 172.21.15.82:49522) 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 -- 172.21.15.82:0/2714102225 learned_addr learned my addr 172.21.15.82:0/2714102225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc777fe640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc78159ea0 0x7fbc781522a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 -- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc7815a280 msgr2=0x7fbc781527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc7815a280 0x7fbc781527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 -- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc78159ea0 msgr2=0x7fbc781522a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc78159ea0 0x7fbc781522a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 -- 172.21.15.82:0/2714102225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78150bd0 con 0x7fbc780718c0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc77fff640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c4c0 tx=0x7fbc70001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70017070 con 0x7fbc780718c0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc70003600 con 0x7fbc780718c0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc78150eb0 con 0x7fbc780718c0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc700039f0 con 0x7fbc780718c0 2024-04-25T10:08:46.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.012+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc7814d520 con 0x7fbc780718c0 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc7804efa0 con 0x7fbc780718c0 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc7001a070 con 0x7fbc780718c0 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc74ff9640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 0x7fbc58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbc70084150 con 0x7fbc780718c0 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc777fe640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 0x7fbc58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.016+0000 7fbc777fe640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 0x7fbc58069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbc781f5b20 tx=0x7fbc60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.020+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc700553b0 con 0x7fbc780718c0 2024-04-25T10:08:46.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.220+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbc78073d90 con 0x7fbc780718c0 2024-04-25T10:08:46.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.220+0000 7fbc74ff9640 1 -- 172.21.15.82:0/2714102225 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbc70054cb0 con 0x7fbc780718c0 2024-04-25T10:08:46.226 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:46.226 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:08:46.226 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.227 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:08:46.228 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.229 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.230 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.231 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.232 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:08:46.233 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.234 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.235 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:08:46.236 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:08:46.237 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.238 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.239 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 msgr2=0x7fbc58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 0x7fbc58069420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbc781f5b20 tx=0x7fbc60008040 comp rx=0 tx=0).stop 2024-04-25T10:08:46.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 msgr2=0x7fbc7814cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c4c0 tx=0x7fbc70001e80 comp rx=0 tx=0).stop 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 shutdown_connections 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbc58066f60 0x7fbc58069420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbc7815a280 0x7fbc781527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbc78159ea0 0x7fbc781522a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 --2- 172.21.15.82:0/2714102225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbc780718c0 0x7fbc7814cfe0 secure :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fbc7000c4c0 tx=0x7fbc70001e80 comp rx=0 tx=0).stop 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 >> 172.21.15.82:0/2714102225 conn(0x7fbc780b11a0 msgr2=0x7fbc7806d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 shutdown_connections 2024-04-25T10:08:46.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.224+0000 7fbc7e571640 1 -- 172.21.15.82:0/2714102225 wait complete. 2024-04-25T10:08:46.244 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:46.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.356+0000 7efeea392640 1 Processor -- start 2024-04-25T10:08:46.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efeea392640 1 -- start start 2024-04-25T10:08:46.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efeea392640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efee406c3b0 0x7efee415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efeea392640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee4066c30 con 0x7efee40b2c30 2024-04-25T10:08:46.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efeea392640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee40af660 con 0x7efee40af280 2024-04-25T10:08:46.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efeea392640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee406daa0 con 0x7efee406c3b0 2024-04-25T10:08:46.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7efee40b2c30 0x7efee406df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42234/0 (socket says 172.21.15.82:42234) 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 -- 172.21.15.82:0/3045535316 learned_addr learned my addr 172.21.15.82:0/3045535316 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee37fe640 1 --2- 172.21.15.82:0/3045535316 >> v2:172.21.15.196:3300/0 conn(0x7efee406c3b0 0x7efee415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361884683 0 0) 0x7efee4066c30 con 0x7efee40b2c30 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efed0003530 con 0x7efee40b2c30 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623252567 0 0) 0x7efed0003530 con 0x7efee40b2c30 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 >> v2:172.21.15.196:3300/0 conn(0x7efee406c3b0 msgr2=0x7efee415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 --2- 172.21.15.82:0/3045535316 >> v2:172.21.15.196:3300/0 conn(0x7efee406c3b0 0x7efee415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 >> v1:172.21.15.196:6789/0 conn(0x7efee40af280 legacy=0x7efee406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee415e640 con 0x7efee40b2c30 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7efecc001680 con 0x7efee40b2c30 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 >> v1:172.21.15.82:6789/0 conn(0x7efee40b2c30 legacy=0x7efee406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed0004810 0x7efed0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 0x7efed000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed000c0d0 con 0x7efed00051d0 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed000c280 con 0x7efed0004810 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed000c480 con 0x7efed0003a30 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 0x7efed000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee37fe640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed0004810 0x7efed0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee3fff640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 -- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed0003a30 msgr2=0x7efed0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 -- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed0004810 msgr2=0x7efed0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efed0004810 0x7efed0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee892f640 1 -- 172.21.15.82:0/3045535316 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed0003530 con 0x7efed00051d0 2024-04-25T10:08:46.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.360+0000 7efee3fff640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed0003a30 0x7efed0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee892f640 1 -- 172.21.15.82:0/3045535316 reap_dead start 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee892f640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 0x7efed000b9d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c210 tx=0x7efecc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900b41dce5f7802e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc007d20 con 0x7efed00051d0 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc0a9070 con 0x7efed00051d0 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee2ffd640 1 -- 172.21.15.82:0/3045535316 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0a5d80 con 0x7efed00051d0 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 msgr2=0x7efed000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 0x7efed000b9d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c210 tx=0x7efecc01cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- 172.21.15.82:0/3045535316 shutdown_connections 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 --2- 172.21.15.82:0/3045535316 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efed00051d0 0x7efed000b9d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- 172.21.15.82:0/3045535316 >> 172.21.15.82:0/3045535316 conn(0x7efee4071810 msgr2=0x7efee4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- 172.21.15.82:0/3045535316 shutdown_connections 2024-04-25T10:08:46.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- 172.21.15.82:0/3045535316 wait complete. 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 Processor -- start 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- start start 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed00051d0 0x7efee41454d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4150b20 0x7efee4150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee406bca0 con 0x7efee414fd90 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee415ec00 con 0x7efee4150b20 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efeea392640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee406be20 con 0x7efed00051d0 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee892f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee892f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49546/0 (socket says 172.21.15.82:49546) 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee892f640 1 -- 172.21.15.82:0/3035028071 learned_addr learned my addr 172.21.15.82:0/3035028071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.364+0000 7efee37fe640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4150b20 0x7efee4150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 -- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed00051d0 msgr2=0x7efee41454d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed00051d0 0x7efee41454d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 -- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4150b20 msgr2=0x7efee4150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4150b20 0x7efee4150fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 -- 172.21.15.82:0/3035028071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee41516e0 con 0x7efee414fd90 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee892f640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c210 tx=0x7efed0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc007d20 con 0x7efee414fd90 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efecc0b2020 con 0x7efee414fd90 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee41519c0 con 0x7efee414fd90 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efecc0049f0 con 0x7efee414fd90 2024-04-25T10:08:46.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee414d640 con 0x7efee414fd90 2024-04-25T10:08:46.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee404efa0 con 0x7efee414fd90 2024-04-25T10:08:46.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efecc004b90 con 0x7efee414fd90 2024-04-25T10:08:46.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 0x7efec4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efecc11e3d0 con 0x7efee414fd90 2024-04-25T10:08:46.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee3fff640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 0x7efec4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.368+0000 7efee3fff640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 0x7efec4069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efedc0012f0 tx=0x7efedc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.372+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efecc0bc030 con 0x7efee414fd90 2024-04-25T10:08:46.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.532+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efee40727c0 con 0x7efee414fd90 2024-04-25T10:08:46.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.532+0000 7efee0ff9640 1 -- 172.21.15.82:0/3035028071 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+13177 (secure 0 0 0) 0x7efecc016970 con 0x7efee414fd90 2024-04-25T10:08:46.538 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:46.538 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":87,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:45.876563+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:46.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 msgr2=0x7efec4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 0x7efec4069420 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7efedc0012f0 tx=0x7efedc009040 comp rx=0 tx=0).stop 2024-04-25T10:08:46.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 msgr2=0x7efee4150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7efecc00c210 tx=0x7efed0005840 comp rx=0 tx=0).stop 2024-04-25T10:08:46.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 shutdown_connections 2024-04-25T10:08:46.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efec4066f60 0x7efec4069420 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efee414fd90 0x7efee4150210 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efee4150b20 0x7efee4150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 --2- 172.21.15.82:0/3035028071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efed00051d0 0x7efee41454d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 >> 172.21.15.82:0/3035028071 conn(0x7efee4071810 msgr2=0x7efee40acd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 shutdown_connections 2024-04-25T10:08:46.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.536+0000 7efeea392640 1 -- 172.21.15.82:0/3035028071 wait complete. 2024-04-25T10:08:46.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:46.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 Processor -- start 2024-04-25T10:08:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 -- start start 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2ebc0cfca0 0x7f2ebc072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ebc0cbd20 con 0x7f2ebc15b800 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cc930 con 0x7f2ebc15b420 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec26be640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cc000 con 0x7f2ebc0cfca0 2024-04-25T10:08:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49548/0 (socket says 172.21.15.82:49548) 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 -- 172.21.15.82:0/1297667369 learned_addr learned my addr 172.21.15.82:0/1297667369 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 -- 172.21.15.82:0/1297667369 >> v2:172.21.15.196:3300/0 conn(0x7f2ebc0cfca0 msgr2=0x7f2ebc072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ec0c5b640 1 --2- 172.21.15.82:0/1297667369 >> v2:172.21.15.196:3300/0 conn(0x7f2ebc0cfca0 0x7f2ebc072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 --2- 172.21.15.82:0/1297667369 >> v2:172.21.15.196:3300/0 conn(0x7f2ebc0cfca0 0x7f2ebc072000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 -- 172.21.15.82:0/1297667369 >> v1:172.21.15.196:6789/0 conn(0x7f2ebc15b800 legacy=0x7f2ebc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 -- 172.21.15.82:0/1297667369 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0cc2e0 con 0x7f2ebc15b420 2024-04-25T10:08:46.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebbfff640 1 --2- 172.21.15.82:0/1297667369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0712a0 tx=0x7f2eac01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c18805d28c65898 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.672+0000 7f2ebaffd640 1 -- 172.21.15.82:0/1297667369 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0ab650 con 0x7f2ebc15b420 2024-04-25T10:08:46.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ebaffd640 1 -- 172.21.15.82:0/1297667369 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eac0b0070 con 0x7f2ebc15b420 2024-04-25T10:08:46.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ebaffd640 1 -- 172.21.15.82:0/1297667369 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0aba10 con 0x7f2ebc15b420 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1297667369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b420 msgr2=0x7f2ebc0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1297667369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc0712a0 tx=0x7f2eac01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1297667369 shutdown_connections 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1297667369 >> v2:172.21.15.196:3300/0 conn(0x7f2ebc0cfca0 0x7f2ebc072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1297667369 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b420 0x7f2ebc0ca3c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1297667369 >> 172.21.15.82:0/1297667369 conn(0x7f2ebc06ae60 msgr2=0x7f2ebc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1297667369 shutdown_connections 2024-04-25T10:08:46.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1297667369 wait complete. 2024-04-25T10:08:46.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 Processor -- start 2024-04-25T10:08:46.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- start start 2024-04-25T10:08:46.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ebc0cfca0 0x7f2ebc0c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ebc15b420 0x7f2ebc096470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc0caf30 con 0x7f2ebc15b800 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc0cadb0 con 0x7f2ebc15b420 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ebc072d50 con 0x7f2ebc0cfca0 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49556/0 (socket says 172.21.15.82:49556) 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 -- 172.21.15.82:0/1882809653 learned_addr learned my addr 172.21.15.82:0/1882809653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ebbfff640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ebc0cfca0 0x7f2ebc0c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ebb7fe640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ebc15b420 0x7f2ebc096470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 -- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ebc0cfca0 msgr2=0x7f2ebc0c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ebc0cfca0 0x7f2ebc0c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 -- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ebc15b420 msgr2=0x7f2ebc096470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ebc15b420 0x7f2ebc096470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 -- 172.21.15.82:0/1882809653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ebc0c2cf0 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec0c5b640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400cd70 tx=0x7f2eb400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4003ba0 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb4003d40 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ebc0c2f90 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4004240 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.676+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ebc0c3550 con 0x7f2ebc15b800 2024-04-25T10:08:46.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ebc0c3750 con 0x7f2ebc15b800 2024-04-25T10:08:46.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2eb40043e0 con 0x7f2ebc15b800 2024-04-25T10:08:46.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2eb8ff9640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 0x7f2ea8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2ebbfff640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 0x7f2ea8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2eb4085930 con 0x7f2ebc15b800 2024-04-25T10:08:46.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.680+0000 7f2ebbfff640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 0x7f2ea8069450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0af000 tx=0x7f2eac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.684+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ebc0c3750 con 0x7f2ebc15b800 2024-04-25T10:08:46.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.832+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ebc04efa0 con 0x7f2ebc15b800 2024-04-25T10:08:46.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.832+0000 7f2eb8ff9640 1 -- 172.21.15.82:0/1882809653 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2eb401d340 con 0x7f2ebc15b800 2024-04-25T10:08:46.838 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:08:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 msgr2=0x7f2ea8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 0x7f2ea8069450 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2eac0af000 tx=0x7f2eac008040 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 msgr2=0x7f2ebc0969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400cd70 tx=0x7f2eb400a660 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 shutdown_connections 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2ea8066f90 0x7f2ea8069450 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2ebc15b800 0x7f2ebc0969b0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2ebc15b420 0x7f2ebc096470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 --2- 172.21.15.82:0/1882809653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2ebc0cfca0 0x7f2ebc0c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 >> 172.21.15.82:0/1882809653 conn(0x7f2ebc06ae60 msgr2=0x7f2ebc06e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 shutdown_connections 2024-04-25T10:08:46.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.836+0000 7f2ec26be640 1 -- 172.21.15.82:0/1882809653 wait complete. 2024-04-25T10:08:46.856 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:46.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 Processor -- start 2024-04-25T10:08:46.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 -- start start 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf815da80 0x7ffaf80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 0x7ffaf806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf806be60 con 0x7ffaf80d1cd0 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf806c440 con 0x7ffaf815da80 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffb00164640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf815de60 con 0x7ffaf80d07f0 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf815da80 0x7ffaf80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf815da80 0x7ffaf80d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49572/0 (socket says 172.21.15.82:49572) 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 -- 172.21.15.82:0/68521802 learned_addr learned my addr 172.21.15.82:0/68521802 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafe701640 1 --2- 172.21.15.82:0/68521802 >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 0x7ffaf806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 -- 172.21.15.82:0/68521802 >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 msgr2=0x7ffaf806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 --2- 172.21.15.82:0/68521802 >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 0x7ffaf806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 -- 172.21.15.82:0/68521802 >> v1:172.21.15.196:6789/0 conn(0x7ffaf80d1cd0 legacy=0x7ffaf80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:46.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafdf00640 1 -- 172.21.15.82:0/68521802 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf80d2450 con 0x7ffaf815da80 2024-04-25T10:08:46.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.956+0000 7ffafe701640 1 --2- 172.21.15.82:0/68521802 >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 0x7ffaf806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:46.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafdf00640 1 --2- 172.21.15.82:0/68521802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf80d4230 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb74759c27dc96bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafcefe640 1 -- 172.21.15.82:0/68521802 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40ab650 con 0x7ffaf815da80 2024-04-25T10:08:46.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafcefe640 1 -- 172.21.15.82:0/68521802 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf40b0070 con 0x7ffaf815da80 2024-04-25T10:08:46.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafcefe640 1 -- 172.21.15.82:0/68521802 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40aba10 con 0x7ffaf815da80 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- 172.21.15.82:0/68521802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 msgr2=0x7ffaf80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- 172.21.15.82:0/68521802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf80d4230 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009870 tx=0x7ffaf401c4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- 172.21.15.82:0/68521802 shutdown_connections 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- 172.21.15.82:0/68521802 >> v2:172.21.15.196:3300/0 conn(0x7ffaf80d07f0 0x7ffaf806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- 172.21.15.82:0/68521802 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf80d4230 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- 172.21.15.82:0/68521802 >> 172.21.15.82:0/68521802 conn(0x7ffaf808b000 msgr2=0x7ffaf80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:46.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- 172.21.15.82:0/68521802 shutdown_connections 2024-04-25T10:08:46.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- 172.21.15.82:0/68521802 wait complete. 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 Processor -- start 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- start start 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf80d07f0 0x7ffaf8153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf80d1cd0 0x7ffaf8153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf806d7a0 con 0x7ffaf815da80 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf806d620 con 0x7ffaf80d1cd0 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffb00164640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf806d920 con 0x7ffaf80d07f0 2024-04-25T10:08:46.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49582/0 (socket says 172.21.15.82:49582) 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 -- 172.21.15.82:0/1597646960 learned_addr learned my addr 172.21.15.82:0/1597646960 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafdf00640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf80d07f0 0x7ffaf8153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafd6ff640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf80d1cd0 0x7ffaf8153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 -- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf80d07f0 msgr2=0x7ffaf8153500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf80d07f0 0x7ffaf8153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 -- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf80d1cd0 msgr2=0x7ffaf8153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf80d1cd0 0x7ffaf8153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.960+0000 7ffafe701640 1 -- 172.21.15.82:0/1597646960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf8153a40 con 0x7ffaf815da80 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffafe701640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000cef0 tx=0x7ffaf000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf001a070 con 0x7ffaf815da80 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf000d860 con 0x7ffaf815da80 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf8154ef0 con 0x7ffaf815da80 2024-04-25T10:08:46.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf000dc50 con 0x7ffaf815da80 2024-04-25T10:08:46.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf80c99c0 con 0x7ffaf815da80 2024-04-25T10:08:46.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf804efa0 con 0x7ffaf815da80 2024-04-25T10:08:46.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffaf001d070 con 0x7ffaf815da80 2024-04-25T10:08:46.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:46.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffafdf00640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:46.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffaf00856c0 con 0x7ffaf815da80 2024-04-25T10:08:46.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.964+0000 7ffafdf00640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40af000 tx=0x7ffaf4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:46.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:46.968+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffaf8153a40 con 0x7ffaf815da80 2024-04-25T10:08:47.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffaf8073830 con 0x7ffaf815da80 2024-04-25T10:08:47.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffaee7fc640 1 -- 172.21.15.82:0/1597646960 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffaf0056950 con 0x7ffaf815da80 2024-04-25T10:08:47.120 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:08:47.121 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 msgr2=0x7ffadc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40af000 tx=0x7ffaf4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 msgr2=0x7ffaf8154180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000cef0 tx=0x7ffaf000a710 comp rx=0 tx=0).stop 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 shutdown_connections 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf815da80 0x7ffaf8154180 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf80d1cd0 0x7ffaf8153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 --2- 172.21.15.82:0/1597646960 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf80d07f0 0x7ffaf8153500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 >> 172.21.15.82:0/1597646960 conn(0x7ffaf808b000 msgr2=0x7ffaf80ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 shutdown_connections 2024-04-25T10:08:47.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.116+0000 7ffb00164640 1 -- 172.21.15.82:0/1597646960 wait complete. 2024-04-25T10:08:47.139 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:47.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:47.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 Processor -- start 2024-04-25T10:08:47.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 -- start start 2024-04-25T10:08:47.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f78400cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78400ac170 0x7f78400ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78400b28c0 con 0x7f78400adc70 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400cfe90 con 0x7f78400ac170 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f78482f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784015bb20 con 0x7f78400cf510 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7846094640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78400adc70 0x7f784006e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42240/0 (socket says 172.21.15.82:42240) 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7846094640 1 -- 172.21.15.82:0/422517796 learned_addr learned my addr 172.21.15.82:0/422517796 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7846895640 1 --2- 172.21.15.82:0/422517796 >> v2:172.21.15.196:3300/0 conn(0x7f78400ac170 0x7f78400ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845893640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f78400cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218567125 0 0) 0x7f78400b28c0 con 0x7f78400adc70 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782c003530 con 0x7f78400adc70 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3381812480 0 0) 0x7f782c003530 con 0x7f78400adc70 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 msgr2=0x7f78400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:47.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f78400cf950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 >> v2:172.21.15.196:3300/0 conn(0x7f78400ac170 msgr2=0x7f78400ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 --2- 172.21.15.82:0/422517796 >> v2:172.21.15.196:3300/0 conn(0x7f78400ac170 0x7f78400ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400b2b60 con 0x7f78400adc70 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7838001a00 con 0x7f78400adc70 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 >> v1:172.21.15.82:6789/0 conn(0x7f78400adc70 legacy=0x7f784006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f782c0039c0 0x7f782c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f782c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782c004d50 0x7f782c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400cfe90 con 0x7f782c004d50 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784015bb20 con 0x7f782c0039c0 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c00b9d0 con 0x7f78400cf510 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.252+0000 7f7846895640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782c004d50 0x7f782c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7845893640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f782c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 -- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 msgr2=0x7f782c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846094640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f782c0039c0 0x7f782c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78400cf510 0x7f782c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 -- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f782c0039c0 msgr2=0x7f782c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f782c0039c0 0x7f782c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 -- 172.21.15.82:0/422517796 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c003530 con 0x7f782c004d50 2024-04-25T10:08:47.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782c004d50 0x7f782c0051d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7834009ae0 tx=0x7f783401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed04056e11f4a10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340b2070 con 0x7f782c004d50 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78340a56a0 con 0x7f782c004d50 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7845092640 1 -- 172.21.15.82:0/422517796 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340096b0 con 0x7f782c004d50 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 -- 172.21.15.82:0/422517796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782c004d50 msgr2=0x7f782c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 --2- 172.21.15.82:0/422517796 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f782c004d50 0x7f782c0051d0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7834009ae0 tx=0x7f783401cab0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f7846895640 1 -- 172.21.15.82:0/422517796 reap_dead start 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 -- 172.21.15.82:0/422517796 shutdown_connections 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 -- 172.21.15.82:0/422517796 >> 172.21.15.82:0/422517796 conn(0x7f784006ae60 msgr2=0x7f784006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 -- 172.21.15.82:0/422517796 shutdown_connections 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.256+0000 7f78482f8640 1 -- 172.21.15.82:0/422517796 wait complete. 2024-04-25T10:08:47.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 Processor -- start 2024-04-25T10:08:47.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- start start 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 0x7f78400c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78400b02b0 0x7f78400b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b3400 con 0x7f7840159c10 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b3120 con 0x7f78400b02b0 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400b36e0 con 0x7f782c004d50 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49590/0 (socket says 172.21.15.82:49590) 2024-04-25T10:08:47.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 -- 172.21.15.82:0/3391450977 learned_addr learned my addr 172.21.15.82:0/3391450977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846094640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 0x7f78400c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7845893640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78400b02b0 0x7f78400b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 -- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 msgr2=0x7f78400c4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 0x7f78400c4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 -- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78400b02b0 msgr2=0x7f78400b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78400b02b0 0x7f78400b0730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 -- 172.21.15.82:0/3391450977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784015c0a0 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846094640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 0x7f78400c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846895640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7834009ae0 tx=0x7f782c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340b2040 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834004020 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f784015c380 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340096b0 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f784015c940 con 0x7f7840159c10 2024-04-25T10:08:47.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7840067c30 con 0x7f7840159c10 2024-04-25T10:08:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78340bf060 con 0x7f7840159c10 2024-04-25T10:08:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 0x7f7824069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f783411ea60 con 0x7f7840159c10 2024-04-25T10:08:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.260+0000 7f7846094640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 0x7f7824069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:47.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.264+0000 7f7846094640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 0x7f7824069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7838007c50 tx=0x7f7838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:47.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.268+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78340167c0 con 0x7f7840159c10 2024-04-25T10:08:47.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.424+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f784006d760 con 0x7f7840159c10 2024-04-25T10:08:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.868+0000 7f7832ffd640 1 -- 172.21.15.82:0/3391450977 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) v1 ==== 138+0+0 (secure 0 0 0) 0x7f78340efb90 con 0x7f7840159c10 2024-04-25T10:08:47.875 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 msgr2=0x7f7824069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 0x7f7824069450 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7838007c50 tx=0x7f7838009040 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 msgr2=0x7f784015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f7834009ae0 tx=0x7f782c005840 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 shutdown_connections 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7824066f90 0x7f7824069450 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7840159c10 0x7f784015a090 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78400b02b0 0x7f78400b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 --2- 172.21.15.82:0/3391450977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f782c004d50 0x7f78400c4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 >> 172.21.15.82:0/3391450977 conn(0x7f784006ae60 msgr2=0x7f78400b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 shutdown_connections 2024-04-25T10:08:47.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:47.872+0000 7f78482f8640 1 -- 172.21.15.82:0/3391450977 wait complete. 2024-04-25T10:08:47.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 Processor -- start 2024-04-25T10:08:48.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 -- start start 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3580cf590 0x7fd3580cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 0x7fd358072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3580cbd20 con 0x7fd35815b6b0 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd35815ba90 con 0x7fd3580cf590 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35c985640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3580cc000 con 0x7fd358072000 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd355d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3580cf590 0x7fd3580cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd35815b6b0 0x7fd3580ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42256/0 (socket says 172.21.15.82:42256) 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35659c640 1 -- 172.21.15.82:0/1163000052 learned_addr learned my addr 172.21.15.82:0/1163000052 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd356d9d640 1 --2- 172.21.15.82:0/1163000052 >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 0x7fd358072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd355d9b640 1 -- 172.21.15.82:0/1163000052 >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 msgr2=0x7fd358072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd355d9b640 1 --2- 172.21.15.82:0/1163000052 >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 0x7fd358072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd355d9b640 1 -- 172.21.15.82:0/1163000052 >> v1:172.21.15.82:6789/0 conn(0x7fd35815b6b0 legacy=0x7fd3580ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd35559a640 1 -- 172.21.15.82:0/1163000052 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700197308 0 0) 0x7fd3580cbd20 con 0x7fd35815b6b0 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd355d9b640 1 -- 172.21.15.82:0/1163000052 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580cc2e0 con 0x7fd3580cf590 2024-04-25T10:08:48.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.032+0000 7fd356d9d640 1 --2- 172.21.15.82:0/1163000052 >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 0x7fd358072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd355d9b640 1 --2- 172.21.15.82:0/1163000052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3580cf590 0x7fd3580cfa20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd35806f000 tx=0x7fd34c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87a63da747c7714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35559a640 1 -- 172.21.15.82:0/1163000052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0b0070 con 0x7fd3580cf590 2024-04-25T10:08:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35559a640 1 -- 172.21.15.82:0/1163000052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34c0ab600 con 0x7fd3580cf590 2024-04-25T10:08:48.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35559a640 1 -- 172.21.15.82:0/1163000052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0ab9d0 con 0x7fd3580cf590 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- 172.21.15.82:0/1163000052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3580cf590 msgr2=0x7fd3580cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- 172.21.15.82:0/1163000052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3580cf590 0x7fd3580cfa20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd35806f000 tx=0x7fd34c01c480 comp rx=0 tx=0).stop 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- 172.21.15.82:0/1163000052 shutdown_connections 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- 172.21.15.82:0/1163000052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3580cf590 0x7fd3580cfa20 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- 172.21.15.82:0/1163000052 >> v2:172.21.15.196:3300/0 conn(0x7fd358072000 0x7fd358072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- 172.21.15.82:0/1163000052 >> 172.21.15.82:0/1163000052 conn(0x7fd35806ae60 msgr2=0x7fd35806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- 172.21.15.82:0/1163000052 shutdown_connections 2024-04-25T10:08:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- 172.21.15.82:0/1163000052 wait complete. 2024-04-25T10:08:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 Processor -- start 2024-04-25T10:08:48.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- start start 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3580cf590 0x7fd358098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 0x7fd3581543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580caf30 con 0x7fd358072000 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3580cadb0 con 0x7fd35815b6b0 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35c985640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3581682a0 con 0x7fd3580cf590 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.036+0000 7fd35659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49606/0 (socket says 172.21.15.82:49606) 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 -- 172.21.15.82:0/1535586415 learned_addr learned my addr 172.21.15.82:0/1535586415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:48.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd356d9d640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 0x7fd3581543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd355d9b640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3580cf590 0x7fd358098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 -- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3580cf590 msgr2=0x7fd358098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3580cf590 0x7fd358098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 -- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 msgr2=0x7fd3581543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 0x7fd3581543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 -- 172.21.15.82:0/1535586415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35806d490 con 0x7fd358072000 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd356d9d640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 0x7fd3581543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35659c640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c480 tx=0x7fd34800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348018070 con 0x7fd358072000 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd348007500 con 0x7fd358072000 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3580cd0b0 con 0x7fd358072000 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd348004a30 con 0x7fd358072000 2024-04-25T10:08:48.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3580cd670 con 0x7fd358072000 2024-04-25T10:08:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd35804efa0 con 0x7fd358072000 2024-04-25T10:08:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd34801b070 con 0x7fd358072000 2024-04-25T10:08:48.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 0x7fd338069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd355d9b640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 0x7fd338069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd3480842b0 con 0x7fd358072000 2024-04-25T10:08:48.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.040+0000 7fd355d9b640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 0x7fd338069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0ad000 tx=0x7fd34c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:48.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.044+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3480553a0 con 0x7fd358072000 2024-04-25T10:08:48.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.204+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd35815fc90 con 0x7fd358072000 2024-04-25T10:08:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.880+0000 7fd346ffd640 1 -- 172.21.15.82:0/1535586415 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd348054ca0 con 0x7fd358072000 2024-04-25T10:08:48.887 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:08:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 msgr2=0x7fd338069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 0x7fd338069420 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0ad000 tx=0x7fd34c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 msgr2=0x7fd35806cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd34800c480 tx=0x7fd34800c940 comp rx=0 tx=0).stop 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 shutdown_connections 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd338066f60 0x7fd338069420 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3580cf590 0x7fd358098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd35815b6b0 0x7fd3581543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 --2- 172.21.15.82:0/1535586415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd358072000 0x7fd35806cf50 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 >> 172.21.15.82:0/1535586415 conn(0x7fd35806ae60 msgr2=0x7fd3580b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 shutdown_connections 2024-04-25T10:08:48.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:48.884+0000 7fd35c985640 1 -- 172.21.15.82:0/1535586415 wait complete. 2024-04-25T10:08:48.924 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:49.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.036+0000 7fecdb8ee640 1 Processor -- start 2024-04-25T10:08:49.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.036+0000 7fecdb8ee640 1 -- start start 2024-04-25T10:08:49.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecdb8ee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fecd415b6b0 0x7fecd40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecdb8ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecd40cf590 0x7fecd40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecdb8ee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cbd20 con 0x7fecd415f130 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecdb8ee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd415ba90 con 0x7fecd40cf590 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecdb8ee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd40cc000 con 0x7fecd415b6b0 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecd40cf590 0x7fecd40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fecd40cf590 0x7fecd40cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49616/0 (socket says 172.21.15.82:49616) 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 -- 172.21.15.82:0/703052233 learned_addr learned my addr 172.21.15.82:0/703052233 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd968a640 1 --2- 172.21.15.82:0/703052233 >> v2:172.21.15.196:3300/0 conn(0x7fecd415b6b0 0x7fecd40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 -- 172.21.15.82:0/703052233 >> v2:172.21.15.196:3300/0 conn(0x7fecd415b6b0 msgr2=0x7fecd40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 --2- 172.21.15.82:0/703052233 >> v2:172.21.15.196:3300/0 conn(0x7fecd415b6b0 0x7fecd40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 -- 172.21.15.82:0/703052233 >> v1:172.21.15.196:6789/0 conn(0x7fecd415f130 legacy=0x7fecd415f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:49.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 -- 172.21.15.82:0/703052233 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40cc2e0 con 0x7fecd40cf590 2024-04-25T10:08:49.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7fecd8e89640 1 --2- 172.21.15.82:0/703052233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd40cfa20 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fecd40d0180 tx=0x7fecd001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=702998ecb548f528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7feccbfff640 1 -- 172.21.15.82:0/703052233 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00b1070 con 0x7fecd40cf590 2024-04-25T10:08:49.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7feccbfff640 1 -- 172.21.15.82:0/703052233 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd00ac630 con 0x7fecd40cf590 2024-04-25T10:08:49.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.040+0000 7feccbfff640 1 -- 172.21.15.82:0/703052233 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00a8050 con 0x7fecd40cf590 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/703052233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 msgr2=0x7fecd40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/703052233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd40cfa20 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fecd40d0180 tx=0x7fecd001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/703052233 shutdown_connections 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/703052233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd40cfa20 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/703052233 >> v2:172.21.15.196:3300/0 conn(0x7fecd415b6b0 0x7fecd40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/703052233 >> 172.21.15.82:0/703052233 conn(0x7fecd406ae60 msgr2=0x7fecd406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/703052233 shutdown_connections 2024-04-25T10:08:49.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/703052233 wait complete. 2024-04-25T10:08:49.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 Processor -- start 2024-04-25T10:08:49.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- start start 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 0x7fecd41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecd415f130 0x7fecd4159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40cadb0 con 0x7fecd40cf590 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40ca900 con 0x7fecd415b6b0 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40caf30 con 0x7fecd415f130 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49630/0 (socket says 172.21.15.82:49630) 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 -- 172.21.15.82:0/3226101642 learned_addr learned my addr 172.21.15.82:0/3226101642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd8e89640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 0x7fecd41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd9e8b640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecd415f130 0x7fecd4159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 -- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecd415f130 msgr2=0x7fecd4159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecd415f130 0x7fecd4159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 -- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 msgr2=0x7fecd41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 0x7fecd41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 -- 172.21.15.82:0/3226101642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd4158f00 con 0x7fecd40cf590 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd8e89640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 0x7fecd41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecd968a640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c7f0 tx=0x7fecc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4015070 con 0x7fecd40cf590 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc4004240 con 0x7fecd40cf590 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd41591e0 con 0x7fecd40cf590 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4003620 con 0x7fecd40cf590 2024-04-25T10:08:49.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.044+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd41597a0 con 0x7fecd40cf590 2024-04-25T10:08:49.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd404efa0 con 0x7fecd40cf590 2024-04-25T10:08:49.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecc40039d0 con 0x7fecd40cf590 2024-04-25T10:08:49.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecc9ffb640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 0x7fecb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:49.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fecc4085220 con 0x7fecd40cf590 2024-04-25T10:08:49.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecd8e89640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 0x7fecb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:49.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.048+0000 7fecd8e89640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 0x7fecb8069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ae000 tx=0x7fecd00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:49.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.052+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecc4056100 con 0x7fecd40cf590 2024-04-25T10:08:49.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.232+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fecd415aec0 con 0x7fecd40cf590 2024-04-25T10:08:49.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.884+0000 7fecc9ffb640 1 -- 172.21.15.82:0/3226101642 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v54) v1 ==== 174+0+0 (secure 0 0 0) 0x7fecc4055a00 con 0x7fecd40cf590 2024-04-25T10:08:49.890 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T10:08:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 msgr2=0x7fecb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 0x7fecb8069420 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fecd00ae000 tx=0x7fecd00a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:49.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 msgr2=0x7fecd415a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fecc400c7f0 tx=0x7fecc400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 shutdown_connections 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecb8066f60 0x7fecb8069420 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecd415f130 0x7fecd4159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecd415b6b0 0x7fecd41561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 --2- 172.21.15.82:0/3226101642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecd40cf590 0x7fecd415a980 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 >> 172.21.15.82:0/3226101642 conn(0x7fecd406ae60 msgr2=0x7fecd406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:49.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 shutdown_connections 2024-04-25T10:08:49.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:49.888+0000 7fecdb8ee640 1 -- 172.21.15.82:0/3226101642 wait complete. 2024-04-25T10:08:49.926 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:50.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 Processor -- start 2024-04-25T10:08:50.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 -- start start 2024-04-25T10:08:50.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45100ce510 0x7f45100ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f45101451f0 0x7f4510142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45100cee30 con 0x7f4510144ca0 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45100cc2b0 con 0x7f45101451f0 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f451873e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451006e290 con 0x7f45100ce510 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f45101451f0 0x7f4510142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f45101451f0 0x7f4510142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49636/0 (socket says 172.21.15.82:49636) 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 -- 172.21.15.82:0/2764073002 learned_addr learned my addr 172.21.15.82:0/2764073002 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f45164da640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45100ce510 0x7f45100ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 -- 172.21.15.82:0/2764073002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45100ce510 msgr2=0x7f45100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45100ce510 0x7f45100ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 -- 172.21.15.82:0/2764073002 >> v1:172.21.15.196:6789/0 conn(0x7f4510144ca0 legacy=0x7f451006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.044+0000 7f4516cdb640 1 -- 172.21.15.82:0/2764073002 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45100ccc70 con 0x7f45101451f0 2024-04-25T10:08:50.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f4516cdb640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f4510142680 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4508009e50 tx=0x7f450801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5603f1fce22b88d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f45154d8640 1 -- 172.21.15.82:0/2764073002 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080b4070 con 0x7f45101451f0 2024-04-25T10:08:50.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f45154d8640 1 -- 172.21.15.82:0/2764073002 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45080af400 con 0x7f45101451f0 2024-04-25T10:08:50.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f45154d8640 1 -- 172.21.15.82:0/2764073002 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080af7d0 con 0x7f45101451f0 2024-04-25T10:08:50.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- 172.21.15.82:0/2764073002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 msgr2=0x7f4510142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f4510142680 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4508009e50 tx=0x7f450801f1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- 172.21.15.82:0/2764073002 shutdown_connections 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f4510142680 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 --2- 172.21.15.82:0/2764073002 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45100ce510 0x7f45100ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- 172.21.15.82:0/2764073002 >> 172.21.15.82:0/2764073002 conn(0x7f45100b11a0 msgr2=0x7f45100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- 172.21.15.82:0/2764073002 shutdown_connections 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- 172.21.15.82:0/2764073002 wait complete. 2024-04-25T10:08:50.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 Processor -- start 2024-04-25T10:08:50.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.048+0000 7f451873e640 1 -- start start 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45100ce510 0x7f4510144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4510144ca0 0x7f4510135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451006ee30 con 0x7f45101451f0 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451006df20 con 0x7f45100ce510 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45100cc900 con 0x7f4510144ca0 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49642/0 (socket says 172.21.15.82:49642) 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 -- 172.21.15.82:0/4107541133 learned_addr learned my addr 172.21.15.82:0/4107541133 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f45164da640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45100ce510 0x7f4510144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4515cd9640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4510144ca0 0x7f4510135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 -- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4510144ca0 msgr2=0x7f4510135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4510144ca0 0x7f4510135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 -- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45100ce510 msgr2=0x7f4510144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45100ce510 0x7f4510144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 -- 172.21.15.82:0/4107541133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45101e2250 con 0x7f45101451f0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4516cdb640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4508009e50 tx=0x7f450801bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080b4040 con 0x7f45101451f0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45080afe50 con 0x7f45101451f0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45101e2520 con 0x7f45101451f0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508009510 con 0x7f45101451f0 2024-04-25T10:08:50.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45101e2ae0 con 0x7f45101451f0 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451004efa0 con 0x7f45101451f0 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45080c3020 con 0x7f45101451f0 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 0x7f44f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4508122f40 con 0x7f45101451f0 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f45164da640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 0x7f44f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.052+0000 7f45164da640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 0x7f44f4069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a970 tx=0x7f450c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.056+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45080ca020 con 0x7f45101451f0 2024-04-25T10:08:50.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.216+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45100cd8e0 con 0x7f45101451f0 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.220+0000 7f4506ffd640 1 -- 172.21.15.82:0/4107541133 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16910 (secure 0 0 0) 0x7f4508018d40 con 0x7f45101451f0 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:50.226 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:49.239548+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:50.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 msgr2=0x7f44f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 0x7f44f4069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f450c00a970 tx=0x7f450c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 msgr2=0x7f45101e1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4508009e50 tx=0x7f450801bf80 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 shutdown_connections 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44f4066f60 0x7f44f4069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4510144ca0 0x7f4510135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45101451f0 0x7f45101e1d10 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 --2- 172.21.15.82:0/4107541133 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f45100ce510 0x7f4510144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 >> 172.21.15.82:0/4107541133 conn(0x7f45100b11a0 msgr2=0x7f45100c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 shutdown_connections 2024-04-25T10:08:50.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.224+0000 7f451873e640 1 -- 172.21.15.82:0/4107541133 wait complete. 2024-04-25T10:08:50.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:50.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.364+0000 7f725ef62640 1 Processor -- start 2024-04-25T10:08:50.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.364+0000 7f725ef62640 1 -- start start 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f725815b950 0x7f725809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f72580cf420 0x7f72580cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 0x7f72580cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72580d0510 con 0x7f725815b950 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725815bd30 con 0x7f72580cf420 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ef62640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cc0d0 con 0x7f72580cfb80 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f725815b950 0x7f725809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f725815b950 0x7f725809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49648/0 (socket says 172.21.15.82:49648) 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 -- 172.21.15.82:0/234396330 learned_addr learned my addr 172.21.15.82:0/234396330 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f7257fff640 1 --2- 172.21.15.82:0/234396330 >> v2:172.21.15.196:3300/0 conn(0x7f72580cf420 0x7f72580cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725d4ff640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 0x7f72580cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 -- 172.21.15.82:0/234396330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 msgr2=0x7f72580cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 0x7f72580cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 -- 172.21.15.82:0/234396330 >> v2:172.21.15.196:3300/0 conn(0x7f72580cf420 msgr2=0x7f72580cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 --2- 172.21.15.82:0/234396330 >> v2:172.21.15.196:3300/0 conn(0x7f72580cf420 0x7f72580cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 -- 172.21.15.82:0/234396330 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725809ab10 con 0x7f725815b950 2024-04-25T10:08:50.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725d4ff640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 0x7f72580cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:50.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f725ccfe640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725815b950 0x7f725809a480 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7258072710 tx=0x7f724c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caafdf26160a5361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f72577fe640 1 -- 172.21.15.82:0/234396330 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0aa610 con 0x7f725815b950 2024-04-25T10:08:50.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f72577fe640 1 -- 172.21.15.82:0/234396330 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c0af070 con 0x7f725815b950 2024-04-25T10:08:50.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.368+0000 7f72577fe640 1 -- 172.21.15.82:0/234396330 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0aa9d0 con 0x7f725815b950 2024-04-25T10:08:50.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/234396330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725815b950 msgr2=0x7f725809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725815b950 0x7f725809a480 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f7258072710 tx=0x7f724c01c4f0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/234396330 shutdown_connections 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cfb80 0x7f72580cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- 172.21.15.82:0/234396330 >> v2:172.21.15.196:3300/0 conn(0x7f72580cf420 0x7f72580cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- 172.21.15.82:0/234396330 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f725815b950 0x7f725809a480 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/234396330 >> 172.21.15.82:0/234396330 conn(0x7f72580b30b0 msgr2=0x7f72580b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/234396330 shutdown_connections 2024-04-25T10:08:50.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/234396330 wait complete. 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 Processor -- start 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- start start 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf420 0x7f72581544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815b950 0x7f72580caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cf9b0 con 0x7f72580cfb80 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cf830 con 0x7f725815b950 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cd390 con 0x7f72580cf420 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49650/0 (socket says 172.21.15.82:49650) 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 -- 172.21.15.82:0/1873698441 learned_addr learned my addr 172.21.15.82:0/1873698441 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ccfe640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf420 0x7f72581544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725d4ff640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815b950 0x7f72580caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 -- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf420 msgr2=0x7f72581544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf420 0x7f72581544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 -- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815b950 msgr2=0x7f72580caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815b950 0x7f72580caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 -- 172.21.15.82:0/1873698441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580c9ec0 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f7257fff640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f724400d5f0 tx=0x7f724400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440125f0 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7244019070 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72580ca160 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440129b0 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72580caa00 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.372+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f725804efa0 con 0x7f72580cfb80 2024-04-25T10:08:50.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.376+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7244012b50 con 0x7f72580cfb80 2024-04-25T10:08:50.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.376+0000 7f72557fa640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.376+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7244085610 con 0x7f72580cfb80 2024-04-25T10:08:50.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.376+0000 7f725ccfe640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.376+0000 7f725ccfe640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f724c0ae000 tx=0x7f724c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.380+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7244056490 con 0x7f72580cfb80 2024-04-25T10:08:50.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.540+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72580d13a0 con 0x7f72580cfb80 2024-04-25T10:08:50.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.540+0000 7f72557fa640 1 -- 172.21.15.82:0/1873698441 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16910 (secure 0 0 0) 0x7f7244055d90 con 0x7f72580cfb80 2024-04-25T10:08:50.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:50.547 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":90,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:49.239548+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 msgr2=0x7f723c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f724c0ae000 tx=0x7f724c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 msgr2=0x7f72580ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f724400d5f0 tx=0x7f724400dab0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 shutdown_connections 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815b950 0x7f72580caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580cfb80 0x7f72580ce2a0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 --2- 172.21.15.82:0/1873698441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf420 0x7f72581544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 >> 172.21.15.82:0/1873698441 conn(0x7f72580b30b0 msgr2=0x7f72580b0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 shutdown_connections 2024-04-25T10:08:50.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.544+0000 7f725ef62640 1 -- 172.21.15.82:0/1873698441 wait complete. 2024-04-25T10:08:50.566 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:08:50.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 Processor -- start 2024-04-25T10:08:50.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 -- start start 2024-04-25T10:08:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d70066a80 0x7f8d7006c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8d7015ddc0 0x7f8d700cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700ca3e0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7006c630 con 0x7f8d7015ddc0 2024-04-25T10:08:50.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d7492c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7006b6a0 con 0x7f8d70066a80 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8d7015ddc0 0x7f8d700cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8d7015d9e0 0x7f8d700cbd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42272/0 (socket says 172.21.15.82:42272) 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8d7015ddc0 0x7f8d700cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49658/0 (socket says 172.21.15.82:49658) 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6e59c640 1 -- 172.21.15.82:0/103653939 learned_addr learned my addr 172.21.15.82:0/103653939 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d67fff640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d70066a80 0x7f8d7006c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 -- 172.21.15.82:0/103653939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d70066a80 msgr2=0x7f8d7006c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d70066a80 0x7f8d7006c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 -- 172.21.15.82:0/103653939 >> v1:172.21.15.82:6789/0 conn(0x7f8d7015d9e0 legacy=0x7f8d700cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6ed9d640 1 -- 172.21.15.82:0/103653939 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700ca6c0 con 0x7f8d7015ddc0 2024-04-25T10:08:50.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.688+0000 7f8d6dd9b640 1 -- 172.21.15.82:0/103653939 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074348958 0 0) 0x7f8d700ca3e0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6ed9d640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015ddc0 0x7f8d700cefc0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009d90 tx=0x7f8d6801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71b5e786f44c7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6dd9b640 1 -- 172.21.15.82:0/103653939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680b2070 con 0x7f8d7015ddc0 2024-04-25T10:08:50.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6dd9b640 1 -- 172.21.15.82:0/103653939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d680a5830 con 0x7f8d7015ddc0 2024-04-25T10:08:50.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6dd9b640 1 -- 172.21.15.82:0/103653939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680a5c00 con 0x7f8d7015ddc0 2024-04-25T10:08:50.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/103653939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015ddc0 msgr2=0x7f8d700cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015ddc0 0x7f8d700cefc0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8d68009d90 tx=0x7f8d6801c970 comp rx=0 tx=0).stop 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/103653939 shutdown_connections 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015ddc0 0x7f8d700cefc0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- 172.21.15.82:0/103653939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d70066a80 0x7f8d7006c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/103653939 >> 172.21.15.82:0/103653939 conn(0x7f8d700cc670 msgr2=0x7f8d700cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/103653939 shutdown_connections 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/103653939 wait complete. 2024-04-25T10:08:50.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 Processor -- start 2024-04-25T10:08:50.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- start start 2024-04-25T10:08:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 0x7f8d7009e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d7015ddc0 0x7f8d7009fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7006e3b0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7006e230 con 0x7f8d70066a80 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d700cf910 con 0x7f8d7015ddc0 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49666/0 (socket says 172.21.15.82:49666) 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 -- 172.21.15.82:0/3046905850 learned_addr learned my addr 172.21.15.82:0/3046905850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6e59c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 0x7f8d7009e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 -- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d7015ddc0 msgr2=0x7f8d7009fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d7015ddc0 0x7f8d7009fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 -- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 msgr2=0x7f8d7009e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 0x7f8d7009e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 -- 172.21.15.82:0/3046905850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700bf680 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d6e59c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 0x7f8d7009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d67fff640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800d800 tx=0x7f8d5800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800b6c0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d58019070 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d700a0680 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.692+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d5800bae0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d700a0c40 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d7004efa0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d5800bc80 con 0x7f8d7015d9e0 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d66ffd640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 0x7f8d50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d58085670 con 0x7f8d7015d9e0 2024-04-25T10:08:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d6e59c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 0x7f8d50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.696+0000 7f8d6e59c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 0x7f8d50069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000f70 tx=0x7f8d60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.700+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d580564d0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.884+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8d70073d90 con 0x7f8d7015d9e0 2024-04-25T10:08:50.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d66ffd640 1 -- 172.21.15.82:0/3046905850 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8d58055dd0 con 0x7f8d7015d9e0 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 msgr2=0x7f8d50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 0x7f8d50069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f8d60000f70 tx=0x7f8d60009040 comp rx=0 tx=0).stop 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 msgr2=0x7f8d7015b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800d800 tx=0x7f8d5800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 shutdown_connections 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d50066f60 0x7f8d50069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d7015ddc0 0x7f8d7009fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d7015d9e0 0x7f8d7015b000 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 --2- 172.21.15.82:0/3046905850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d70066a80 0x7f8d7009e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:50.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 >> 172.21.15.82:0/3046905850 conn(0x7f8d700cc670 msgr2=0x7f8d700cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:50.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 shutdown_connections 2024-04-25T10:08:50.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:50.924+0000 7f8d7492c640 1 -- 172.21.15.82:0/3046905850 wait complete. 2024-04-25T10:08:50.947 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:08:50.947 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:08:51.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.064+0000 7f6b70a23640 1 Processor -- start 2024-04-25T10:08:51.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.064+0000 7f6b70a23640 1 -- start start 2024-04-25T10:08:51.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b70a23640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 0x7f6b6c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b70a23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b70a23640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b6c0d2720 con 0x7f6b6c0cf3c0 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b70a23640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d2290 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b70a23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c0d1680 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b6a59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 0x7f6b6c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:41744/0 (socket says 172.21.15.82:41744) 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b6a59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 0x7f6b6c0cee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:40104/0 (socket says 172.21.15.82:40104) 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 -- 172.21.15.82:0/1366151728 learned_addr learned my addr 172.21.15.82:0/1366151728 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:51.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b6ad9d640 1 --1- 172.21.15.82:0/1366151728 >> v1:172.21.15.196:6789/0 conn(0x7f6b6c0cf3c0 0x7f6b6c1615d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:49120/0 (socket says 172.21.15.82:49120) 2024-04-25T10:08:51.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 -- 172.21.15.82:0/1366151728 >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 msgr2=0x7f6b6c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:51.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 --2- 172.21.15.82:0/1366151728 >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 0x7f6b6c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 -- 172.21.15.82:0/1366151728 >> v1:172.21.15.196:6789/0 conn(0x7f6b6c0cf3c0 legacy=0x7f6b6c1615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.068+0000 7f6b69d9b640 1 -- 172.21.15.82:0/1366151728 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0d1960 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b69d9b640 1 --2- 172.21.15.82:0/1366151728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0d2410 tx=0x7f6b6001c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9deaf167df746d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b6959a640 1 -- 172.21.15.82:0/1366151728 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600a57c0 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b6959a640 1 -- 172.21.15.82:0/1366151728 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600a5960 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b6959a640 1 -- 172.21.15.82:0/1366151728 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600a5d30 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- 172.21.15.82:0/1366151728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 msgr2=0x7f6b6c0b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- 172.21.15.82:0/1366151728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0d2410 tx=0x7f6b6001c9e0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- 172.21.15.82:0/1366151728 shutdown_connections 2024-04-25T10:08:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- 172.21.15.82:0/1366151728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0b5270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- 172.21.15.82:0/1366151728 >> v2:172.21.15.196:3300/0 conn(0x7f6b6c15d8d0 0x7f6b6c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- 172.21.15.82:0/1366151728 >> 172.21.15.82:0/1366151728 conn(0x7f6b6c08b000 msgr2=0x7f6b6c0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- 172.21.15.82:0/1366151728 shutdown_connections 2024-04-25T10:08:51.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- 172.21.15.82:0/1366151728 wait complete. 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 Processor -- start 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- start start 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b6c0cf3c0 0x7f6b6c1555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c16aaa0 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c16a920 con 0x7f6b6c0cf3c0 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b70a23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6c16ac20 con 0x7f6b6c15dcb0 2024-04-25T10:08:51.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b69d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b69d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49680/0 (socket says 172.21.15.82:49680) 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.072+0000 7f6b69d9b640 1 -- 172.21.15.82:0/2192393262 learned_addr learned my addr 172.21.15.82:0/2192393262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b6ad9d640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b6a59c640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b6c0cf3c0 0x7f6b6c1555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 -- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 msgr2=0x7f6b6c0ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0ca830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 -- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b6c0cf3c0 msgr2=0x7f6b6c1555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b6c0cf3c0 0x7f6b6c1555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 -- 172.21.15.82:0/2192393262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c0c8180 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b69d9b640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009040 tx=0x7f6b60019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600b8440 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600b8600 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6c0c8460 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600b89f0 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6c0c8a20 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6c04efa0 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b600c1020 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b6011f710 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.084 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b6a59c640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:51.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.076+0000 7f6b6a59c640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 0x7f6b4c069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009b70 tx=0x7f6b64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.080+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b600c8020 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.272+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6b6c0c9f90 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b5affd640 1 -- 172.21.15.82:0/2192393262 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6b60016600 con 0x7f6b6c15d8d0 2024-04-25T10:08:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 msgr2=0x7f6b4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 0x7f6b4c069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009b70 tx=0x7f6b64008040 comp rx=0 tx=0).stop 2024-04-25T10:08:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 msgr2=0x7f6b6c155c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:51.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6b60009040 tx=0x7f6b60019780 comp rx=0 tx=0).stop 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 shutdown_connections 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b6c15dcb0 0x7f6b6c0ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b6c15d8d0 0x7f6b6c155c80 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.940+0000 7f6b70a23640 1 --2- 172.21.15.82:0/2192393262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b6c0cf3c0 0x7f6b6c1555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.944+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 >> 172.21.15.82:0/2192393262 conn(0x7f6b6c08b000 msgr2=0x7f6b6c0b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.944+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 shutdown_connections 2024-04-25T10:08:51.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:51.944+0000 7f6b70a23640 1 -- 172.21.15.82:0/2192393262 wait complete. 2024-04-25T10:08:51.969 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:08:51.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.084+0000 7f7f8d544640 1 Processor -- start 2024-04-25T10:08:52.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.084+0000 7f7f8d544640 1 -- start start 2024-04-25T10:08:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f88159ea0 0x7f7f880b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f8815a280 0x7f7f8815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 0x7f7f88071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f880b2990 con 0x7f7f88159ea0 2024-04-25T10:08:52.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8815ae30 con 0x7f7f880718c0 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f8d544640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8815b110 con 0x7f7f8815a280 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f88159ea0 0x7f7f880b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f88159ea0 0x7f7f880b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49682/0 (socket says 172.21.15.82:49682) 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 -- 172.21.15.82:0/1806213739 learned_addr learned my addr 172.21.15.82:0/1806213739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 -- 172.21.15.82:0/1806213739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f8815a280 msgr2=0x7f7f8815a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f867fc640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f8815a280 0x7f7f8815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f877fe640 1 --2- 172.21.15.82:0/1806213739 >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 0x7f7f88071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f8815a280 0x7f7f8815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 -- 172.21.15.82:0/1806213739 >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 msgr2=0x7f7f88071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1806213739 >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 0x7f7f88071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 -- 172.21.15.82:0/1806213739 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880b2620 con 0x7f7f88159ea0 2024-04-25T10:08:52.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f877fe640 1 --2- 172.21.15.82:0/1806213739 >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 0x7f7f88071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f88159ea0 0x7f7f880b2e20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7f880aed30 tx=0x7f7f7801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9363a88c9a4d85e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f85ffb640 1 -- 172.21.15.82:0/1806213739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780b0070 con 0x7f7f88159ea0 2024-04-25T10:08:52.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f85ffb640 1 -- 172.21.15.82:0/1806213739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f780ab600 con 0x7f7f88159ea0 2024-04-25T10:08:52.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.088+0000 7f7f85ffb640 1 -- 172.21.15.82:0/1806213739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780ab9d0 con 0x7f7f88159ea0 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1806213739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f88159ea0 msgr2=0x7f7f880b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f88159ea0 0x7f7f880b2e20 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7f880aed30 tx=0x7f7f7801c520 comp rx=0 tx=0).stop 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1806213739 shutdown_connections 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1806213739 >> v2:172.21.15.196:3300/0 conn(0x7f7f880718c0 0x7f7f88071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f8815a280 0x7f7f8815a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1806213739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f88159ea0 0x7f7f880b2e20 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1806213739 >> 172.21.15.82:0/1806213739 conn(0x7f7f880b11a0 msgr2=0x7f7f880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1806213739 shutdown_connections 2024-04-25T10:08:52.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1806213739 wait complete. 2024-04-25T10:08:52.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 Processor -- start 2024-04-25T10:08:52.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- start start 2024-04-25T10:08:52.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 0x7f7f880ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f88159ea0 0x7f7f88151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8806d420 con 0x7f7f8815a280 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8806d2a0 con 0x7f7f88159ea0 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8806d5a0 con 0x7f7f880718c0 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49694/0 (socket says 172.21.15.82:49694) 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 -- 172.21.15.82:0/1404749441 learned_addr learned my addr 172.21.15.82:0/1404749441 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 0x7f7f880ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f867fc640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f88159ea0 0x7f7f88151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 -- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 msgr2=0x7f7f880ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 0x7f7f880ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 -- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f88159ea0 msgr2=0x7f7f88151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f88159ea0 0x7f7f88151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 -- 172.21.15.82:0/1404749441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8813aca0 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 0x7f7f880ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f877fe640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000cef0 tx=0x7f7f8000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8001a070 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8000d840 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f88151bc0 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8000dc30 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.092+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8813b240 con 0x7f7f8815a280 2024-04-25T10:08:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8804efa0 con 0x7f7f8815a280 2024-04-25T10:08:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f8001d070 con 0x7f7f8815a280 2024-04-25T10:08:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f6f7fe640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 0x7f7f74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7f80085cb0 con 0x7f7f8815a280 2024-04-25T10:08:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 0x7f7f74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.096+0000 7f7f86ffd640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 0x7f7f74069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ad000 tx=0x7f7f78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.100+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f80024020 con 0x7f7f8815a280 2024-04-25T10:08:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.292+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f88073d90 con 0x7f7f8815a280 2024-04-25T10:08:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.292+0000 7f7f6f7fe640 1 -- 172.21.15.82:0/1404749441 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7f80056ca0 con 0x7f7f8815a280 2024-04-25T10:08:52.299 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:52.299 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:08:51.280173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2865291678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2865291678},{"type":"v1","addr":"172.21.15.82:6837","nonce":2865291678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:08:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 56 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 msgr2=0x7f7f74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 0x7f7f74069450 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7f780ad000 tx=0x7f7f78008040 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 msgr2=0x7f7f88152420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7f8000cef0 tx=0x7f7f8000a710 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 shutdown_connections 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f74066f90 0x7f7f74069450 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f8815a280 0x7f7f88152420 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f88159ea0 0x7f7f88151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 --2- 172.21.15.82:0/1404749441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f880718c0 0x7f7f880ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 >> 172.21.15.82:0/1404749441 conn(0x7f7f880b11a0 msgr2=0x7f7f88154330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 shutdown_connections 2024-04-25T10:08:52.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.296+0000 7f7f8d544640 1 -- 172.21.15.82:0/1404749441 wait complete. 2024-04-25T10:08:52.317 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.428+0000 7f61e8424640 1 Processor -- start 2024-04-25T10:08:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.428+0000 7f61e8424640 1 -- start start 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e8424640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61e00d07f0 0x7f61e00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e8424640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f61e00d1cd0 0x7f61e006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e8424640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e006be60 con 0x7f61e015da80 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e8424640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e006c440 con 0x7f61e00d07f0 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e8424640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e015de60 con 0x7f61e00d1cd0 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61e00d07f0 0x7f61e00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f61e00d07f0 0x7f61e00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49698/0 (socket says 172.21.15.82:49698) 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 -- 172.21.15.82:0/3272107952 learned_addr learned my addr 172.21.15.82:0/3272107952 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e69c1640 1 --2- 172.21.15.82:0/3272107952 >> v2:172.21.15.196:3300/0 conn(0x7f61e00d1cd0 0x7f61e006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 -- 172.21.15.82:0/3272107952 >> v2:172.21.15.196:3300/0 conn(0x7f61e00d1cd0 msgr2=0x7f61e006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 --2- 172.21.15.82:0/3272107952 >> v2:172.21.15.196:3300/0 conn(0x7f61e00d1cd0 0x7f61e006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 -- 172.21.15.82:0/3272107952 >> v1:172.21.15.196:6789/0 conn(0x7f61e015da80 legacy=0x7f61e00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 -- 172.21.15.82:0/3272107952 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00d2450 con 0x7f61e00d07f0 2024-04-25T10:08:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e59bf640 1 --2- 172.21.15.82:0/3272107952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d07f0 0x7f61e00d4230 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f61e006de30 tx=0x7f61dc01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6133a7fe07e7355f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e51be640 1 -- 172.21.15.82:0/3272107952 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0b1070 con 0x7f61e00d07f0 2024-04-25T10:08:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e51be640 1 -- 172.21.15.82:0/3272107952 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc0a8050 con 0x7f61e00d07f0 2024-04-25T10:08:52.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.432+0000 7f61e51be640 1 -- 172.21.15.82:0/3272107952 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0ac860 con 0x7f61e00d07f0 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/3272107952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d07f0 msgr2=0x7f61e00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- 172.21.15.82:0/3272107952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d07f0 0x7f61e00d4230 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f61e006de30 tx=0x7f61dc01c4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/3272107952 shutdown_connections 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- 172.21.15.82:0/3272107952 >> v2:172.21.15.196:3300/0 conn(0x7f61e00d1cd0 0x7f61e006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- 172.21.15.82:0/3272107952 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d07f0 0x7f61e00d4230 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/3272107952 >> 172.21.15.82:0/3272107952 conn(0x7f61e008b000 msgr2=0x7f61e00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/3272107952 shutdown_connections 2024-04-25T10:08:52.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/3272107952 wait complete. 2024-04-25T10:08:52.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 Processor -- start 2024-04-25T10:08:52.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- start start 2024-04-25T10:08:52.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 0x7f61e014d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 0x7f61e015e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e006d7a0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e006d620 con 0x7f61e015da80 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e006d920 con 0x7f61e00d07f0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49712/0 (socket says 172.21.15.82:49712) 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 -- 172.21.15.82:0/2278639824 learned_addr learned my addr 172.21.15.82:0/2278639824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e69c1640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 0x7f61e015e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 -- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 msgr2=0x7f61e014d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e61c0640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 0x7f61e014d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 0x7f61e014d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 -- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 msgr2=0x7f61e015e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 0x7f61e015e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 -- 172.21.15.82:0/2278639824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e015b2c0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e69c1640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 0x7f61e015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e61c0640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 0x7f61e014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e59bf640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f61dc01ab60 tx=0x7f61dc019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0b1040 con 0x7f61e00d1cd0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc005ba0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e015b5a0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc003720 con 0x7f61e00d1cd0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.436+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e015bb60 con 0x7f61e00d1cd0 2024-04-25T10:08:52.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e004efa0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61dc0bc060 con 0x7f61e00d1cd0 2024-04-25T10:08:52.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61d6ffd640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 0x7f61c4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f61dc11e3f0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61e61c0640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 0x7f61c4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.440+0000 7f61e61c0640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 0x7f61c4069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f61d000a850 tx=0x7f61d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.444+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61dc0160d0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.600+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61e00cf460 con 0x7f61e00d1cd0 2024-04-25T10:08:52.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61d6ffd640 1 -- 172.21.15.82:0/2278639824 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16897 (secure 0 0 0) 0x7f61dc0ef1e0 con 0x7f61e00d1cd0 2024-04-25T10:08:52.608 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:52.609 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:50.889383+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:52.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 msgr2=0x7f61c4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 0x7f61c4069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f61d000a850 tx=0x7f61d0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:52.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 msgr2=0x7f61e01607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f61dc01ab60 tx=0x7f61dc019250 comp rx=0 tx=0).stop 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 shutdown_connections 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f61c4066f90 0x7f61c4069450 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f61e015da80 0x7f61e015e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f61e00d1cd0 0x7f61e01607e0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 --2- 172.21.15.82:0/2278639824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f61e00d07f0 0x7f61e014d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 >> 172.21.15.82:0/2278639824 conn(0x7f61e008b000 msgr2=0x7f61e00ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 shutdown_connections 2024-04-25T10:08:52.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.604+0000 7f61e8424640 1 -- 172.21.15.82:0/2278639824 wait complete. 2024-04-25T10:08:52.627 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:08:52.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:52.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.728+0000 7f3b304c3640 1 Processor -- start 2024-04-25T10:08:52.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.728+0000 7f3b304c3640 1 -- start start 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b304c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d07f0 0x7f3b280d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b304c3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b280d1cd0 0x7f3b2806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b304c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2806be60 con 0x7f3b2815da80 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b304c3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2806c440 con 0x7f3b280d1cd0 2024-04-25T10:08:52.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b304c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2815de60 con 0x7f3b280d07f0 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b280d1cd0 0x7f3b2806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3b280d1cd0 0x7f3b2806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49718/0 (socket says 172.21.15.82:49718) 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 -- 172.21.15.82:0/2907713791 learned_addr learned my addr 172.21.15.82:0/2907713791 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 -- 172.21.15.82:0/2907713791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d07f0 msgr2=0x7f3b280d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2da5e640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d07f0 0x7f3b280d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d07f0 0x7f3b280d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 -- 172.21.15.82:0/2907713791 >> v1:172.21.15.196:6789/0 conn(0x7f3b2815da80 legacy=0x7f3b280d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:52.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.732+0000 7f3b2ea60640 1 -- 172.21.15.82:0/2907713791 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280d2450 con 0x7f3b280d1cd0 2024-04-25T10:08:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b2ea60640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b280d1cd0 0x7f3b2806c8d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009e50 tx=0x7f3b2001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80bba32e3ac004e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b2d25d640 1 -- 172.21.15.82:0/2907713791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200a5820 con 0x7f3b280d1cd0 2024-04-25T10:08:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b2d25d640 1 -- 172.21.15.82:0/2907713791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b200a59c0 con 0x7f3b280d1cd0 2024-04-25T10:08:52.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b2d25d640 1 -- 172.21.15.82:0/2907713791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200a5d90 con 0x7f3b280d1cd0 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 -- 172.21.15.82:0/2907713791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b280d1cd0 msgr2=0x7f3b2806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b280d1cd0 0x7f3b2806c8d0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009e50 tx=0x7f3b2001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 -- 172.21.15.82:0/2907713791 shutdown_connections 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d07f0 0x7f3b280d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 --2- 172.21.15.82:0/2907713791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b280d1cd0 0x7f3b2806c8d0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 -- 172.21.15.82:0/2907713791 >> 172.21.15.82:0/2907713791 conn(0x7f3b2808b000 msgr2=0x7f3b280d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 -- 172.21.15.82:0/2907713791 shutdown_connections 2024-04-25T10:08:52.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 -- 172.21.15.82:0/2907713791 wait complete. 2024-04-25T10:08:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.736+0000 7f3b304c3640 1 Processor -- start 2024-04-25T10:08:52.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- start start 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 0x7f3b28153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d1cd0 0x7f3b28154210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806d7a0 con 0x7f3b2815da80 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806d620 con 0x7f3b280d07f0 2024-04-25T10:08:52.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2806d920 con 0x7f3b280d1cd0 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49726/0 (socket says 172.21.15.82:49726) 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 -- 172.21.15.82:0/1706450920 learned_addr learned my addr 172.21.15.82:0/1706450920 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2da5e640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d1cd0 0x7f3b28154210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2e25f640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 0x7f3b28153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 -- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d1cd0 msgr2=0x7f3b28154210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d1cd0 0x7f3b28154210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 -- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 msgr2=0x7f3b28153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 0x7f3b28153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 -- 172.21.15.82:0/1706450920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28158af0 con 0x7f3b2815da80 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2e25f640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 0x7f3b28153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:52.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b2ea60640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af040 tx=0x7f3b20019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200b8880 con 0x7f3b2815da80 2024-04-25T10:08:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b200be070 con 0x7f3b2815da80 2024-04-25T10:08:52.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b200b8c80 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b28158da0 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2815e320 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2804efa0 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.740+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b200c3460 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.744+0000 7f3b1effd640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 0x7f3b0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.744+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3b20120670 con 0x7f3b2815da80 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.744+0000 7f3b2e25f640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 0x7f3b0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:52.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.744+0000 7f3b2e25f640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 0x7f3b0c069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400a850 tx=0x7f3b24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:52.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.748+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b200165e0 con 0x7f3b2815da80 2024-04-25T10:08:52.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.892+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b2815e4f0 con 0x7f3b0c066f60 2024-04-25T10:08:52.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.908+0000 7f3b1effd640 1 -- 172.21.15.82:0/1706450920 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246766 (secure 0 0 0) 0x7f3b2815e4f0 con 0x7f3b0c066f60 2024-04-25T10:08:52.915 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:52.916 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 msgr2=0x7f3b0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 0x7f3b0c069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400a850 tx=0x7f3b24008040 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 msgr2=0x7f3b28154750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f3b200af040 tx=0x7f3b20019760 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 shutdown_connections 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3b0c066f60 0x7f3b0c069420 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3b280d1cd0 0x7f3b28154210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3b2815da80 0x7f3b28154750 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 --2- 172.21.15.82:0/1706450920 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3b280d07f0 0x7f3b28153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 >> 172.21.15.82:0/1706450920 conn(0x7f3b2808b000 msgr2=0x7f3b2815cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:52.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 shutdown_connections 2024-04-25T10:08:52.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:52.912+0000 7f3b304c3640 1 -- 172.21.15.82:0/1706450920 wait complete. 2024-04-25T10:08:52.931 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-25T10:08:51.388587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259204,"kb_used_data":9736,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715516,"statfs":{"total":773094113280,"available":772828688384,"internally_reserved":0,"allocated":9969664,"data_stored":5203416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897534+0000","last_change":"2024-04-25T10:08:48.897534+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897534+0000","last_undegraded":"2024-04-25T10:08:48.897534+0000","last_fullsized":"2024-04-25T10:08:48.897534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900991+0000","last_change":"2024-04-25T10:08:48.900991+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900991+0000","last_undegraded":"2024-04-25T10:08:48.900991+0000","last_fullsized":"2024-04-25T10:08:48.900991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897938+0000","last_change":"2024-04-25T10:08:48.897938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897938+0000","last_undegraded":"2024-04-25T10:08:48.897938+0000","last_fullsized":"2024-04-25T10:08:48.897938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.909082+0000","last_change":"2024-04-25T10:08:48.909082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.909082+0000","last_undegraded":"2024-04-25T10:08:48.909082+0000","last_fullsized":"2024-04-25T10:08:48.909082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894412+0000","last_change":"2024-04-25T10:08:48.894412+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894412+0000","last_undegraded":"2024-04-25T10:08:48.894412+0000","last_fullsized":"2024-04-25T10:08:48.894412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896657+0000","last_change":"2024-04-25T10:08:48.896657+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896657+0000","last_undegraded":"2024-04-25T10:08:48.896657+0000","last_fullsized":"2024-04-25T10:08:48.896657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900586+0000","last_change":"2024-04-25T10:08:48.900586+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900586+0000","last_undegraded":"2024-04-25T10:08:48.900586+0000","last_fullsized":"2024-04-25T10:08:48.900586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899208+0000","last_change":"2024-04-25T10:08:48.899208+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899208+0000","last_undegraded":"2024-04-25T10:08:48.899208+0000","last_fullsized":"2024-04-25T10:08:48.899208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897320+0000","last_change":"2024-04-25T10:08:48.897320+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897320+0000","last_undegraded":"2024-04-25T10:08:48.897320+0000","last_fullsized":"2024-04-25T10:08:48.897320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898252+0000","last_change":"2024-04-25T10:08:48.898252+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898252+0000","last_undegraded":"2024-04-25T10:08:48.898252+0000","last_fullsized":"2024-04-25T10:08:48.898252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.912082+0000","last_change":"2024-04-25T10:08:48.912082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.912082+0000","last_undegraded":"2024-04-25T10:08:48.912082+0000","last_fullsized":"2024-04-25T10:08:48.912082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895959+0000","last_change":"2024-04-25T10:08:48.895959+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895959+0000","last_undegraded":"2024-04-25T10:08:48.895959+0000","last_fullsized":"2024-04-25T10:08:48.895959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903583+0000","last_change":"2024-04-25T10:08:48.903583+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903583+0000","last_undegraded":"2024-04-25T10:08:48.903583+0000","last_fullsized":"2024-04-25T10:08:48.903583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896282+0000","last_change":"2024-04-25T10:08:48.896282+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896282+0000","last_undegraded":"2024-04-25T10:08:48.896282+0000","last_fullsized":"2024-04-25T10:08:48.896282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902220+0000","last_change":"2024-04-25T10:08:48.902220+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902220+0000","last_undegraded":"2024-04-25T10:08:48.902220+0000","last_fullsized":"2024-04-25T10:08:48.902220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897711+0000","last_change":"2024-04-25T10:08:48.897711+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897711+0000","last_undegraded":"2024-04-25T10:08:48.897711+0000","last_fullsized":"2024-04-25T10:08:48.897711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896879+0000","last_change":"2024-04-25T10:08:48.896879+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896879+0000","last_undegraded":"2024-04-25T10:08:48.896879+0000","last_fullsized":"2024-04-25T10:08:48.896879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899121+0000","last_change":"2024-04-25T10:08:48.899121+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899121+0000","last_undegraded":"2024-04-25T10:08:48.899121+0000","last_fullsized":"2024-04-25T10:08:48.899121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899094+0000","last_change":"2024-04-25T10:08:48.899094+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899094+0000","last_undegraded":"2024-04-25T10:08:48.899094+0000","last_fullsized":"2024-04-25T10:08:48.899094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.908676+0000","last_change":"2024-04-25T10:08:48.908676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.908676+0000","last_undegraded":"2024-04-25T10:08:48.908676+0000","last_fullsized":"2024-04-25T10:08:48.908676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895275+0000","last_change":"2024-04-25T10:08:48.895275+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895275+0000","last_undegraded":"2024-04-25T10:08:48.895275+0000","last_fullsized":"2024-04-25T10:08:48.895275+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895212+0000","last_change":"2024-04-25T10:08:48.895212+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895212+0000","last_undegraded":"2024-04-25T10:08:48.895212+0000","last_fullsized":"2024-04-25T10:08:48.895212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895119+0000","last_change":"2024-04-25T10:08:48.895119+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895119+0000","last_undegraded":"2024-04-25T10:08:48.895119+0000","last_fullsized":"2024-04-25T10:08:48.895119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897114+0000","last_change":"2024-04-25T10:08:48.897114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897114+0000","last_undegraded":"2024-04-25T10:08:48.897114+0000","last_fullsized":"2024-04-25T10:08:48.897114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902873+0000","last_change":"2024-04-25T10:08:48.902873+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902873+0000","last_undegraded":"2024-04-25T10:08:48.902873+0000","last_fullsized":"2024-04-25T10:08:48.902873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896098+0000","last_change":"2024-04-25T10:08:48.896098+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896098+0000","last_undegraded":"2024-04-25T10:08:48.896098+0000","last_fullsized":"2024-04-25T10:08:48.896098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897466+0000","last_change":"2024-04-25T10:08:48.897466+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897466+0000","last_undegraded":"2024-04-25T10:08:48.897466+0000","last_fullsized":"2024-04-25T10:08:48.897466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902313+0000","last_change":"2024-04-25T10:08:48.902313+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902313+0000","last_undegraded":"2024-04-25T10:08:48.902313+0000","last_fullsized":"2024-04-25T10:08:48.902313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900867+0000","last_change":"2024-04-25T10:08:48.900867+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900867+0000","last_undegraded":"2024-04-25T10:08:48.900867+0000","last_fullsized":"2024-04-25T10:08:48.900867+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894114+0000","last_change":"2024-04-25T10:08:48.894114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894114+0000","last_undegraded":"2024-04-25T10:08:48.894114+0000","last_fullsized":"2024-04-25T10:08:48.894114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900938+0000","last_change":"2024-04-25T10:08:48.900938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900938+0000","last_undegraded":"2024-04-25T10:08:48.900938+0000","last_fullsized":"2024-04-25T10:08:48.900938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897207+0000","last_change":"2024-04-25T10:08:48.897207+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897207+0000","last_undegraded":"2024-04-25T10:08:48.897207+0000","last_fullsized":"2024-04-25T10:08:48.897207+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895876+0000","last_change":"2024-04-25T10:08:48.895876+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895876+0000","last_undegraded":"2024-04-25T10:08:48.895876+0000","last_fullsized":"2024-04-25T10:08:48.895876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922842+0000","last_change":"2024-04-25T10:08:48.913258+0000","last_active":"2024-04-25T10:08:48.922842+0000","last_peered":"2024-04-25T10:08:48.922842+0000","last_clean":"2024-04-25T10:08:48.922842+0000","last_became_active":"2024-04-25T10:08:48.912653+0000","last_became_peered":"2024-04-25T10:08:48.912653+0000","last_unstale":"2024-04-25T10:08:48.922842+0000","last_undegraded":"2024-04-25T10:08:48.922842+0000","last_fullsized":"2024-04-25T10:08:48.922842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:37.862500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895907+0000","last_change":"2024-04-25T10:08:48.895907+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895907+0000","last_undegraded":"2024-04-25T10:08:48.895907+0000","last_fullsized":"2024-04-25T10:08:48.895907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925620+0000","last_change":"2024-04-25T10:08:48.920404+0000","last_active":"2024-04-25T10:08:48.925620+0000","last_peered":"2024-04-25T10:08:48.925620+0000","last_clean":"2024-04-25T10:08:48.925620+0000","last_became_active":"2024-04-25T10:08:48.919687+0000","last_became_peered":"2024-04-25T10:08:48.919687+0000","last_unstale":"2024-04-25T10:08:48.925620+0000","last_undegraded":"2024-04-25T10:08:48.925620+0000","last_fullsized":"2024-04-25T10:08:48.925620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:08.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901697+0000","last_change":"2024-04-25T10:08:48.901697+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901697+0000","last_undegraded":"2024-04-25T10:08:48.901697+0000","last_fullsized":"2024-04-25T10:08:48.901697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924120+0000","last_change":"2024-04-25T10:08:48.921772+0000","last_active":"2024-04-25T10:08:48.924120+0000","last_peered":"2024-04-25T10:08:48.924120+0000","last_clean":"2024-04-25T10:08:48.924120+0000","last_became_active":"2024-04-25T10:08:48.921226+0000","last_became_peered":"2024-04-25T10:08:48.921226+0000","last_unstale":"2024-04-25T10:08:48.924120+0000","last_undegraded":"2024-04-25T10:08:48.924120+0000","last_fullsized":"2024-04-25T10:08:48.924120+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:45.783541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896461+0000","last_change":"2024-04-25T10:08:48.896461+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896461+0000","last_undegraded":"2024-04-25T10:08:48.896461+0000","last_fullsized":"2024-04-25T10:08:48.896461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927024+0000","last_change":"2024-04-25T10:08:48.920430+0000","last_active":"2024-04-25T10:08:48.927024+0000","last_peered":"2024-04-25T10:08:48.927024+0000","last_clean":"2024-04-25T10:08:48.927024+0000","last_became_active":"2024-04-25T10:08:48.919636+0000","last_became_peered":"2024-04-25T10:08:48.919636+0000","last_unstale":"2024-04-25T10:08:48.927024+0000","last_undegraded":"2024-04-25T10:08:48.927024+0000","last_fullsized":"2024-04-25T10:08:48.927024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:35.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898138+0000","last_change":"2024-04-25T10:08:48.898138+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898138+0000","last_undegraded":"2024-04-25T10:08:48.898138+0000","last_fullsized":"2024-04-25T10:08:48.898138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.920870+0000","last_change":"2024-04-25T10:08:48.913325+0000","last_active":"2024-04-25T10:08:48.920870+0000","last_peered":"2024-04-25T10:08:48.920870+0000","last_clean":"2024-04-25T10:08:48.920870+0000","last_became_active":"2024-04-25T10:08:48.912648+0000","last_became_peered":"2024-04-25T10:08:48.912648+0000","last_unstale":"2024-04-25T10:08:48.920870+0000","last_undegraded":"2024-04-25T10:08:48.920870+0000","last_fullsized":"2024-04-25T10:08:48.920870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:19.606750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893958+0000","last_change":"2024-04-25T10:08:48.893958+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893958+0000","last_undegraded":"2024-04-25T10:08:48.893958+0000","last_fullsized":"2024-04-25T10:08:48.893958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.926830+0000","last_change":"2024-04-25T10:08:48.921408+0000","last_active":"2024-04-25T10:08:48.926830+0000","last_peered":"2024-04-25T10:08:48.926830+0000","last_clean":"2024-04-25T10:08:48.926830+0000","last_became_active":"2024-04-25T10:08:48.921179+0000","last_became_peered":"2024-04-25T10:08:48.921179+0000","last_unstale":"2024-04-25T10:08:48.926830+0000","last_undegraded":"2024-04-25T10:08:48.926830+0000","last_fullsized":"2024-04-25T10:08:48.926830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:09.513075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896117+0000","last_change":"2024-04-25T10:08:48.896117+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896117+0000","last_undegraded":"2024-04-25T10:08:48.896117+0000","last_fullsized":"2024-04-25T10:08:48.896117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925432+0000","last_change":"2024-04-25T10:08:48.923790+0000","last_active":"2024-04-25T10:08:48.925432+0000","last_peered":"2024-04-25T10:08:48.925432+0000","last_clean":"2024-04-25T10:08:48.925432+0000","last_became_active":"2024-04-25T10:08:48.923543+0000","last_became_peered":"2024-04-25T10:08:48.923543+0000","last_unstale":"2024-04-25T10:08:48.925432+0000","last_undegraded":"2024-04-25T10:08:48.925432+0000","last_fullsized":"2024-04-25T10:08:48.925432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903018+0000","last_change":"2024-04-25T10:08:48.903018+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903018+0000","last_undegraded":"2024-04-25T10:08:48.903018+0000","last_fullsized":"2024-04-25T10:08:48.903018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923152+0000","last_change":"2024-04-25T10:08:48.915851+0000","last_active":"2024-04-25T10:08:48.923152+0000","last_peered":"2024-04-25T10:08:48.923152+0000","last_clean":"2024-04-25T10:08:48.923152+0000","last_became_active":"2024-04-25T10:08:48.915212+0000","last_became_peered":"2024-04-25T10:08:48.915212+0000","last_unstale":"2024-04-25T10:08:48.923152+0000","last_undegraded":"2024-04-25T10:08:48.923152+0000","last_fullsized":"2024-04-25T10:08:48.923152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.518129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894554+0000","last_change":"2024-04-25T10:08:48.894554+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894554+0000","last_undegraded":"2024-04-25T10:08:48.894554+0000","last_fullsized":"2024-04-25T10:08:48.894554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919005+0000","last_change":"2024-04-25T10:08:48.908751+0000","last_active":"2024-04-25T10:08:48.919005+0000","last_peered":"2024-04-25T10:08:48.919005+0000","last_clean":"2024-04-25T10:08:48.919005+0000","last_became_active":"2024-04-25T10:08:48.908243+0000","last_became_peered":"2024-04-25T10:08:48.908243+0000","last_unstale":"2024-04-25T10:08:48.919005+0000","last_undegraded":"2024-04-25T10:08:48.919005+0000","last_fullsized":"2024-04-25T10:08:48.919005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:43.075381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901140+0000","last_change":"2024-04-25T10:08:48.901140+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901140+0000","last_undegraded":"2024-04-25T10:08:48.901140+0000","last_fullsized":"2024-04-25T10:08:48.901140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919327+0000","last_change":"2024-04-25T10:08:48.917822+0000","last_active":"2024-04-25T10:08:48.919327+0000","last_peered":"2024-04-25T10:08:48.919327+0000","last_clean":"2024-04-25T10:08:48.919327+0000","last_became_active":"2024-04-25T10:08:48.917540+0000","last_became_peered":"2024-04-25T10:08:48.917540+0000","last_unstale":"2024-04-25T10:08:48.919327+0000","last_undegraded":"2024-04-25T10:08:48.919327+0000","last_fullsized":"2024-04-25T10:08:48.919327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898751+0000","last_change":"2024-04-25T10:08:48.898751+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898751+0000","last_undegraded":"2024-04-25T10:08:48.898751+0000","last_fullsized":"2024-04-25T10:08:48.898751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932807+0000","last_change":"2024-04-25T10:08:48.930283+0000","last_active":"2024-04-25T10:08:48.932807+0000","last_peered":"2024-04-25T10:08:48.932807+0000","last_clean":"2024-04-25T10:08:48.932807+0000","last_became_active":"2024-04-25T10:08:48.929026+0000","last_became_peered":"2024-04-25T10:08:48.929026+0000","last_unstale":"2024-04-25T10:08:48.932807+0000","last_undegraded":"2024-04-25T10:08:48.932807+0000","last_fullsized":"2024-04-25T10:08:48.932807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:30.464303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902636+0000","last_change":"2024-04-25T10:08:48.902636+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902636+0000","last_undegraded":"2024-04-25T10:08:48.902636+0000","last_fullsized":"2024-04-25T10:08:48.902636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927628+0000","last_change":"2024-04-25T10:08:48.926872+0000","last_active":"2024-04-25T10:08:48.927628+0000","last_peered":"2024-04-25T10:08:48.927628+0000","last_clean":"2024-04-25T10:08:48.927628+0000","last_became_active":"2024-04-25T10:08:48.926660+0000","last_became_peered":"2024-04-25T10:08:48.926660+0000","last_unstale":"2024-04-25T10:08:48.927628+0000","last_undegraded":"2024-04-25T10:08:48.927628+0000","last_fullsized":"2024-04-25T10:08:48.927628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.544601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896769+0000","last_change":"2024-04-25T10:08:48.896769+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896769+0000","last_undegraded":"2024-04-25T10:08:48.896769+0000","last_fullsized":"2024-04-25T10:08:48.896769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923043+0000","last_change":"2024-04-25T10:08:48.916082+0000","last_active":"2024-04-25T10:08:48.923043+0000","last_peered":"2024-04-25T10:08:48.923043+0000","last_clean":"2024-04-25T10:08:48.923043+0000","last_became_active":"2024-04-25T10:08:48.915311+0000","last_became_peered":"2024-04-25T10:08:48.915311+0000","last_unstale":"2024-04-25T10:08:48.923043+0000","last_undegraded":"2024-04-25T10:08:48.923043+0000","last_fullsized":"2024-04-25T10:08:48.923043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.892575+0000","last_change":"2024-04-25T10:08:48.892575+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.892575+0000","last_undegraded":"2024-04-25T10:08:48.892575+0000","last_fullsized":"2024-04-25T10:08:48.892575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927634+0000","last_change":"2024-04-25T10:08:48.927015+0000","last_active":"2024-04-25T10:08:48.927634+0000","last_peered":"2024-04-25T10:08:48.927634+0000","last_clean":"2024-04-25T10:08:48.927634+0000","last_became_active":"2024-04-25T10:08:48.926431+0000","last_became_peered":"2024-04-25T10:08:48.926431+0000","last_unstale":"2024-04-25T10:08:48.927634+0000","last_undegraded":"2024-04-25T10:08:48.927634+0000","last_fullsized":"2024-04-25T10:08:48.927634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:25.811654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900421+0000","last_change":"2024-04-25T10:08:48.900421+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900421+0000","last_undegraded":"2024-04-25T10:08:48.900421+0000","last_fullsized":"2024-04-25T10:08:48.900421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925675+0000","last_change":"2024-04-25T10:08:48.920708+0000","last_active":"2024-04-25T10:08:48.925675+0000","last_peered":"2024-04-25T10:08:48.925675+0000","last_clean":"2024-04-25T10:08:48.925675+0000","last_became_active":"2024-04-25T10:08:48.920393+0000","last_became_peered":"2024-04-25T10:08:48.920393+0000","last_unstale":"2024-04-25T10:08:48.925675+0000","last_undegraded":"2024-04-25T10:08:48.925675+0000","last_fullsized":"2024-04-25T10:08:48.925675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:14.265209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896676+0000","last_change":"2024-04-25T10:08:48.896676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896676+0000","last_undegraded":"2024-04-25T10:08:48.896676+0000","last_fullsized":"2024-04-25T10:08:48.896676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.918051+0000","last_change":"2024-04-25T10:08:48.912297+0000","last_active":"2024-04-25T10:08:48.918051+0000","last_peered":"2024-04-25T10:08:48.918051+0000","last_clean":"2024-04-25T10:08:48.918051+0000","last_became_active":"2024-04-25T10:08:48.911555+0000","last_became_peered":"2024-04-25T10:08:48.911555+0000","last_unstale":"2024-04-25T10:08:48.918051+0000","last_undegraded":"2024-04-25T10:08:48.918051+0000","last_fullsized":"2024-04-25T10:08:48.918051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.083371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895443+0000","last_change":"2024-04-25T10:08:48.895443+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895443+0000","last_undegraded":"2024-04-25T10:08:48.895443+0000","last_fullsized":"2024-04-25T10:08:48.895443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.928484+0000","last_change":"2024-04-25T10:08:48.923258+0000","last_active":"2024-04-25T10:08:48.928484+0000","last_peered":"2024-04-25T10:08:48.928484+0000","last_clean":"2024-04-25T10:08:48.928484+0000","last_became_active":"2024-04-25T10:08:48.922652+0000","last_became_peered":"2024-04-25T10:08:48.922652+0000","last_unstale":"2024-04-25T10:08:48.928484+0000","last_undegraded":"2024-04-25T10:08:48.928484+0000","last_fullsized":"2024-04-25T10:08:48.928484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894725+0000","last_change":"2024-04-25T10:08:48.894725+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894725+0000","last_undegraded":"2024-04-25T10:08:48.894725+0000","last_fullsized":"2024-04-25T10:08:48.894725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929141+0000","last_change":"2024-04-25T10:08:48.924659+0000","last_active":"2024-04-25T10:08:48.929141+0000","last_peered":"2024-04-25T10:08:48.929141+0000","last_clean":"2024-04-25T10:08:48.929141+0000","last_became_active":"2024-04-25T10:08:48.923481+0000","last_became_peered":"2024-04-25T10:08:48.923481+0000","last_unstale":"2024-04-25T10:08:48.929141+0000","last_undegraded":"2024-04-25T10:08:48.929141+0000","last_fullsized":"2024-04-25T10:08:48.929141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:56.284437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895174+0000","last_change":"2024-04-25T10:08:48.895174+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895174+0000","last_undegraded":"2024-04-25T10:08:48.895174+0000","last_fullsized":"2024-04-25T10:08:48.895174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925786+0000","last_change":"2024-04-25T10:08:48.921337+0000","last_active":"2024-04-25T10:08:48.925786+0000","last_peered":"2024-04-25T10:08:48.925786+0000","last_clean":"2024-04-25T10:08:48.925786+0000","last_became_active":"2024-04-25T10:08:48.918289+0000","last_became_peered":"2024-04-25T10:08:48.918289+0000","last_unstale":"2024-04-25T10:08:48.925786+0000","last_undegraded":"2024-04-25T10:08:48.925786+0000","last_fullsized":"2024-04-25T10:08:48.925786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.100052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894501+0000","last_change":"2024-04-25T10:08:48.894501+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894501+0000","last_undegraded":"2024-04-25T10:08:48.894501+0000","last_fullsized":"2024-04-25T10:08:48.894501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.911288+0000","last_change":"2024-04-25T10:08:48.906519+0000","last_active":"2024-04-25T10:08:48.911288+0000","last_peered":"2024-04-25T10:08:48.911288+0000","last_clean":"2024-04-25T10:08:48.911288+0000","last_became_active":"2024-04-25T10:08:48.905583+0000","last_became_peered":"2024-04-25T10:08:48.905583+0000","last_unstale":"2024-04-25T10:08:48.911288+0000","last_undegraded":"2024-04-25T10:08:48.911288+0000","last_fullsized":"2024-04-25T10:08:48.911288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:40.496209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900102+0000","last_change":"2024-04-25T10:08:48.900102+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900102+0000","last_undegraded":"2024-04-25T10:08:48.900102+0000","last_fullsized":"2024-04-25T10:08:48.900102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923192+0000","last_change":"2024-04-25T10:08:48.920400+0000","last_active":"2024-04-25T10:08:48.923192+0000","last_peered":"2024-04-25T10:08:48.923192+0000","last_clean":"2024-04-25T10:08:48.923192+0000","last_became_active":"2024-04-25T10:08:48.920006+0000","last_became_peered":"2024-04-25T10:08:48.920006+0000","last_unstale":"2024-04-25T10:08:48.923192+0000","last_undegraded":"2024-04-25T10:08:48.923192+0000","last_fullsized":"2024-04-25T10:08:48.923192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899888+0000","last_change":"2024-04-25T10:08:48.899888+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899888+0000","last_undegraded":"2024-04-25T10:08:48.899888+0000","last_fullsized":"2024-04-25T10:08:48.899888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921292+0000","last_change":"2024-04-25T10:08:48.908793+0000","last_active":"2024-04-25T10:08:48.921292+0000","last_peered":"2024-04-25T10:08:48.921292+0000","last_clean":"2024-04-25T10:08:48.921292+0000","last_became_active":"2024-04-25T10:08:48.908121+0000","last_became_peered":"2024-04-25T10:08:48.908121+0000","last_unstale":"2024-04-25T10:08:48.921292+0000","last_undegraded":"2024-04-25T10:08:48.921292+0000","last_fullsized":"2024-04-25T10:08:48.921292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:54.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894144+0000","last_change":"2024-04-25T10:08:48.894144+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894144+0000","last_undegraded":"2024-04-25T10:08:48.894144+0000","last_fullsized":"2024-04-25T10:08:48.894144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.916943+0000","last_change":"2024-04-25T10:08:48.907802+0000","last_active":"2024-04-25T10:08:48.916943+0000","last_peered":"2024-04-25T10:08:48.916943+0000","last_clean":"2024-04-25T10:08:48.916943+0000","last_became_active":"2024-04-25T10:08:48.907484+0000","last_became_peered":"2024-04-25T10:08:48.907484+0000","last_unstale":"2024-04-25T10:08:48.916943+0000","last_undegraded":"2024-04-25T10:08:48.916943+0000","last_fullsized":"2024-04-25T10:08:48.916943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.725069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901281+0000","last_change":"2024-04-25T10:08:48.901281+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901281+0000","last_undegraded":"2024-04-25T10:08:48.901281+0000","last_fullsized":"2024-04-25T10:08:48.901281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929174+0000","last_change":"2024-04-25T10:08:48.925017+0000","last_active":"2024-04-25T10:08:48.929174+0000","last_peered":"2024-04-25T10:08:48.929174+0000","last_clean":"2024-04-25T10:08:48.929174+0000","last_became_active":"2024-04-25T10:08:48.924231+0000","last_became_peered":"2024-04-25T10:08:48.924231+0000","last_unstale":"2024-04-25T10:08:48.929174+0000","last_undegraded":"2024-04-25T10:08:48.929174+0000","last_fullsized":"2024-04-25T10:08:48.929174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:31.040785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893835+0000","last_change":"2024-04-25T10:08:48.893835+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893835+0000","last_undegraded":"2024-04-25T10:08:48.893835+0000","last_fullsized":"2024-04-25T10:08:48.893835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.931898+0000","last_change":"2024-04-25T10:08:48.927255+0000","last_active":"2024-04-25T10:08:48.931898+0000","last_peered":"2024-04-25T10:08:48.931898+0000","last_clean":"2024-04-25T10:08:48.931898+0000","last_became_active":"2024-04-25T10:08:48.926623+0000","last_became_peered":"2024-04-25T10:08:48.926623+0000","last_unstale":"2024-04-25T10:08:48.931898+0000","last_undegraded":"2024-04-25T10:08:48.931898+0000","last_fullsized":"2024-04-25T10:08:48.931898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.736381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894241+0000","last_change":"2024-04-25T10:08:48.894241+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894241+0000","last_undegraded":"2024-04-25T10:08:48.894241+0000","last_fullsized":"2024-04-25T10:08:48.894241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932774+0000","last_change":"2024-04-25T10:08:48.929895+0000","last_active":"2024-04-25T10:08:48.932774+0000","last_peered":"2024-04-25T10:08:48.932774+0000","last_clean":"2024-04-25T10:08:48.932774+0000","last_became_active":"2024-04-25T10:08:48.928316+0000","last_became_peered":"2024-04-25T10:08:48.928316+0000","last_unstale":"2024-04-25T10:08:48.932774+0000","last_undegraded":"2024-04-25T10:08:48.932774+0000","last_fullsized":"2024-04-25T10:08:48.932774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:38.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902043+0000","last_change":"2024-04-25T10:08:48.902043+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902043+0000","last_undegraded":"2024-04-25T10:08:48.902043+0000","last_fullsized":"2024-04-25T10:08:48.902043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922564+0000","last_change":"2024-04-25T10:08:48.913827+0000","last_active":"2024-04-25T10:08:48.922564+0000","last_peered":"2024-04-25T10:08:48.922564+0000","last_clean":"2024-04-25T10:08:48.922564+0000","last_became_active":"2024-04-25T10:08:48.912878+0000","last_became_peered":"2024-04-25T10:08:48.912878+0000","last_unstale":"2024-04-25T10:08:48.922564+0000","last_undegraded":"2024-04-25T10:08:48.922564+0000","last_fullsized":"2024-04-25T10:08:48.922564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:15.708904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901260+0000","last_change":"2024-04-25T10:08:48.901260+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901260+0000","last_undegraded":"2024-04-25T10:08:48.901260+0000","last_fullsized":"2024-04-25T10:08:48.901260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927592+0000","last_change":"2024-04-25T10:08:48.926824+0000","last_active":"2024-04-25T10:08:48.927592+0000","last_peered":"2024-04-25T10:08:48.927592+0000","last_clean":"2024-04-25T10:08:48.927592+0000","last_became_active":"2024-04-25T10:08:48.926515+0000","last_became_peered":"2024-04-25T10:08:48.926515+0000","last_unstale":"2024-04-25T10:08:48.927592+0000","last_undegraded":"2024-04-25T10:08:48.927592+0000","last_fullsized":"2024-04-25T10:08:48.927592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:21.486349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896505+0000","last_change":"2024-04-25T10:08:48.896505+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896505+0000","last_undegraded":"2024-04-25T10:08:48.896505+0000","last_fullsized":"2024-04-25T10:08:48.896505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921407+0000","last_change":"2024-04-25T10:08:48.912204+0000","last_active":"2024-04-25T10:08:48.921407+0000","last_peered":"2024-04-25T10:08:48.921407+0000","last_clean":"2024-04-25T10:08:48.921407+0000","last_became_active":"2024-04-25T10:08:48.911509+0000","last_became_peered":"2024-04-25T10:08:48.911509+0000","last_unstale":"2024-04-25T10:08:48.921407+0000","last_undegraded":"2024-04-25T10:08:48.921407+0000","last_fullsized":"2024-04-25T10:08:48.921407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.689628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893499+0000","last_change":"2024-04-25T10:08:48.893499+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893499+0000","last_undegraded":"2024-04-25T10:08:48.893499+0000","last_fullsized":"2024-04-25T10:08:48.893499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921699+0000","last_change":"2024-04-25T10:08:48.912652+0000","last_active":"2024-04-25T10:08:48.921699+0000","last_peered":"2024-04-25T10:08:48.921699+0000","last_clean":"2024-04-25T10:08:48.921699+0000","last_became_active":"2024-04-25T10:08:48.911838+0000","last_became_peered":"2024-04-25T10:08:48.911838+0000","last_unstale":"2024-04-25T10:08:48.921699+0000","last_undegraded":"2024-04-25T10:08:48.921699+0000","last_fullsized":"2024-04-25T10:08:48.921699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:37.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893632+0000","last_change":"2024-04-25T10:08:48.893632+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893632+0000","last_undegraded":"2024-04-25T10:08:48.893632+0000","last_fullsized":"2024-04-25T10:08:48.893632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924165+0000","last_change":"2024-04-25T10:08:48.920751+0000","last_active":"2024-04-25T10:08:48.924165+0000","last_peered":"2024-04-25T10:08:48.924165+0000","last_clean":"2024-04-25T10:08:48.924165+0000","last_became_active":"2024-04-25T10:08:48.920001+0000","last_became_peered":"2024-04-25T10:08:48.920001+0000","last_unstale":"2024-04-25T10:08:48.924165+0000","last_undegraded":"2024-04-25T10:08:48.924165+0000","last_fullsized":"2024-04-25T10:08:48.924165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902674+0000","last_change":"2024-04-25T10:08:48.902674+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902674+0000","last_undegraded":"2024-04-25T10:08:48.902674+0000","last_fullsized":"2024-04-25T10:08:48.902674+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927324+0000","last_change":"2024-04-25T10:08:48.924246+0000","last_active":"2024-04-25T10:08:48.927324+0000","last_peered":"2024-04-25T10:08:48.927324+0000","last_clean":"2024-04-25T10:08:48.927324+0000","last_became_active":"2024-04-25T10:08:48.923370+0000","last_became_peered":"2024-04-25T10:08:48.923370+0000","last_unstale":"2024-04-25T10:08:48.927324+0000","last_undegraded":"2024-04-25T10:08:48.927324+0000","last_fullsized":"2024-04-25T10:08:48.927324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:01.982597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.905354+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:08:48.905354+0000","last_peered":"2024-04-25T10:08:48.905354+0000","last_clean":"2024-04-25T10:08:48.905354+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:08:48.905354+0000","last_undegraded":"2024-04-25T10:08:48.905354+0000","last_fullsized":"2024-04-25T10:08:48.905354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:52.936 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:53.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 Processor -- start 2024-04-25T10:08:53.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 -- start start 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 0x7fbaa40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaa406be60 con 0x7fbaa40d0620 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa40d0a70 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa94af640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaa406c440 con 0x7fbaa415da80 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49734/0 (socket says 172.21.15.82:49734) 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 -- 172.21.15.82:0/588878229 learned_addr learned my addr 172.21.15.82:0/588878229 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/588878229 >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 0x7fbaa40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 -- 172.21.15.82:0/588878229 >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 msgr2=0x7fbaa40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 --2- 172.21.15.82:0/588878229 >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 0x7fbaa40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 -- 172.21.15.82:0/588878229 >> v1:172.21.15.196:6789/0 conn(0x7fbaa40d0620 legacy=0x7fbaa40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 -- 172.21.15.82:0/588878229 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa415de60 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/588878229 >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 0x7fbaa40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fba9a7fc640 1 --2- 172.21.15.82:0/588878229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fba8c006fd0 tx=0x7fba8c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dd5b2d17a6c746 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.088+0000 7fbaa27fc640 1 -- 172.21.15.82:0/588878229 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0b0070 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa27fc640 1 -- 172.21.15.82:0/588878229 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c004230 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa27fc640 1 -- 172.21.15.82:0/588878229 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c003620 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- 172.21.15.82:0/588878229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 msgr2=0x7fbaa40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- 172.21.15.82:0/588878229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7fba8c006fd0 tx=0x7fba8c01c4c0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- 172.21.15.82:0/588878229 shutdown_connections 2024-04-25T10:08:53.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- 172.21.15.82:0/588878229 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa40d00e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- 172.21.15.82:0/588878229 >> v2:172.21.15.196:3300/0 conn(0x7fbaa415da80 0x7fbaa40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- 172.21.15.82:0/588878229 >> 172.21.15.82:0/588878229 conn(0x7fbaa408b000 msgr2=0x7fbaa40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- 172.21.15.82:0/588878229 shutdown_connections 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- 172.21.15.82:0/588878229 wait complete. 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 Processor -- start 2024-04-25T10:08:53.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- start start 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 0x7fbaa40c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 0x7fbaa415c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa416a730 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa416a5b0 con 0x7fbaa40d0620 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa94af640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa416a8b0 con 0x7fbaa415da80 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49738/0 (socket says 172.21.15.82:49738) 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 -- 172.21.15.82:0/861425145 learned_addr learned my addr 172.21.15.82:0/861425145 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 0x7fbaa40c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa37fe640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 0x7fbaa415c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 -- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 msgr2=0x7fbaa415c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 0x7fbaa415c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 -- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 msgr2=0x7fbaa40c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 0x7fbaa40c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 -- 172.21.15.82:0/861425145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaa40b6a00 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 0x7fbaa40c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fbaa37fe640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 0x7fbaa415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.092+0000 7fba9a7fc640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0ae040 tx=0x7fba8c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0bb070 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c0043e0 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa40ccfb0 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c0b0040 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa40cd570 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa404efa0 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba8c0be070 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 0x7fba84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 0x7fba84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fba8c11e1e0 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.096+0000 7fbaa2ffd640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 0x7fba84069450 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fba94008890 tx=0x7fba94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.100+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba8c0160b0 con 0x7fbaa40d1cd0 2024-04-25T10:08:53.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.252+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbaa40b2c00 con 0x7fba84066f90 2024-04-25T10:08:53.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.264+0000 7fba9bfff640 1 -- 172.21.15.82:0/861425145 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246766 (secure 0 0 0) 0x7fbaa40b2c00 con 0x7fba84066f90 2024-04-25T10:08:53.274 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:08:53.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 msgr2=0x7fba84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 0x7fba84069450 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fba94008890 tx=0x7fba94008040 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 msgr2=0x7fbaa415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fba8c0ae040 tx=0x7fba8c019260 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 shutdown_connections 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba84066f90 0x7fba84069450 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbaa415da80 0x7fbaa415c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbaa40d1cd0 0x7fbaa415b620 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 --2- 172.21.15.82:0/861425145 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbaa40d0620 0x7fbaa40c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 >> 172.21.15.82:0/861425145 conn(0x7fbaa408b000 msgr2=0x7fbaa4160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 shutdown_connections 2024-04-25T10:08:53.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.268+0000 7fbaa94af640 1 -- 172.21.15.82:0/861425145 wait complete. 2024-04-25T10:08:53.289 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-25T10:08:51.388587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259204,"kb_used_data":9736,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715516,"statfs":{"total":773094113280,"available":772828688384,"internally_reserved":0,"allocated":9969664,"data_stored":5203416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001086"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897534+0000","last_change":"2024-04-25T10:08:48.897534+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897534+0000","last_undegraded":"2024-04-25T10:08:48.897534+0000","last_fullsized":"2024-04-25T10:08:48.897534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900991+0000","last_change":"2024-04-25T10:08:48.900991+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900991+0000","last_undegraded":"2024-04-25T10:08:48.900991+0000","last_fullsized":"2024-04-25T10:08:48.900991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897938+0000","last_change":"2024-04-25T10:08:48.897938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897938+0000","last_undegraded":"2024-04-25T10:08:48.897938+0000","last_fullsized":"2024-04-25T10:08:48.897938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.909082+0000","last_change":"2024-04-25T10:08:48.909082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.909082+0000","last_undegraded":"2024-04-25T10:08:48.909082+0000","last_fullsized":"2024-04-25T10:08:48.909082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894412+0000","last_change":"2024-04-25T10:08:48.894412+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894412+0000","last_undegraded":"2024-04-25T10:08:48.894412+0000","last_fullsized":"2024-04-25T10:08:48.894412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896657+0000","last_change":"2024-04-25T10:08:48.896657+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896657+0000","last_undegraded":"2024-04-25T10:08:48.896657+0000","last_fullsized":"2024-04-25T10:08:48.896657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900586+0000","last_change":"2024-04-25T10:08:48.900586+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900586+0000","last_undegraded":"2024-04-25T10:08:48.900586+0000","last_fullsized":"2024-04-25T10:08:48.900586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899208+0000","last_change":"2024-04-25T10:08:48.899208+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899208+0000","last_undegraded":"2024-04-25T10:08:48.899208+0000","last_fullsized":"2024-04-25T10:08:48.899208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897320+0000","last_change":"2024-04-25T10:08:48.897320+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897320+0000","last_undegraded":"2024-04-25T10:08:48.897320+0000","last_fullsized":"2024-04-25T10:08:48.897320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898252+0000","last_change":"2024-04-25T10:08:48.898252+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898252+0000","last_undegraded":"2024-04-25T10:08:48.898252+0000","last_fullsized":"2024-04-25T10:08:48.898252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.912082+0000","last_change":"2024-04-25T10:08:48.912082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.912082+0000","last_undegraded":"2024-04-25T10:08:48.912082+0000","last_fullsized":"2024-04-25T10:08:48.912082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895959+0000","last_change":"2024-04-25T10:08:48.895959+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895959+0000","last_undegraded":"2024-04-25T10:08:48.895959+0000","last_fullsized":"2024-04-25T10:08:48.895959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903583+0000","last_change":"2024-04-25T10:08:48.903583+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903583+0000","last_undegraded":"2024-04-25T10:08:48.903583+0000","last_fullsized":"2024-04-25T10:08:48.903583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896282+0000","last_change":"2024-04-25T10:08:48.896282+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896282+0000","last_undegraded":"2024-04-25T10:08:48.896282+0000","last_fullsized":"2024-04-25T10:08:48.896282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902220+0000","last_change":"2024-04-25T10:08:48.902220+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902220+0000","last_undegraded":"2024-04-25T10:08:48.902220+0000","last_fullsized":"2024-04-25T10:08:48.902220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897711+0000","last_change":"2024-04-25T10:08:48.897711+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897711+0000","last_undegraded":"2024-04-25T10:08:48.897711+0000","last_fullsized":"2024-04-25T10:08:48.897711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896879+0000","last_change":"2024-04-25T10:08:48.896879+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896879+0000","last_undegraded":"2024-04-25T10:08:48.896879+0000","last_fullsized":"2024-04-25T10:08:48.896879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899121+0000","last_change":"2024-04-25T10:08:48.899121+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899121+0000","last_undegraded":"2024-04-25T10:08:48.899121+0000","last_fullsized":"2024-04-25T10:08:48.899121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899094+0000","last_change":"2024-04-25T10:08:48.899094+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899094+0000","last_undegraded":"2024-04-25T10:08:48.899094+0000","last_fullsized":"2024-04-25T10:08:48.899094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.908676+0000","last_change":"2024-04-25T10:08:48.908676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.908676+0000","last_undegraded":"2024-04-25T10:08:48.908676+0000","last_fullsized":"2024-04-25T10:08:48.908676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895275+0000","last_change":"2024-04-25T10:08:48.895275+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895275+0000","last_undegraded":"2024-04-25T10:08:48.895275+0000","last_fullsized":"2024-04-25T10:08:48.895275+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895212+0000","last_change":"2024-04-25T10:08:48.895212+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895212+0000","last_undegraded":"2024-04-25T10:08:48.895212+0000","last_fullsized":"2024-04-25T10:08:48.895212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895119+0000","last_change":"2024-04-25T10:08:48.895119+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895119+0000","last_undegraded":"2024-04-25T10:08:48.895119+0000","last_fullsized":"2024-04-25T10:08:48.895119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897114+0000","last_change":"2024-04-25T10:08:48.897114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897114+0000","last_undegraded":"2024-04-25T10:08:48.897114+0000","last_fullsized":"2024-04-25T10:08:48.897114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902873+0000","last_change":"2024-04-25T10:08:48.902873+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902873+0000","last_undegraded":"2024-04-25T10:08:48.902873+0000","last_fullsized":"2024-04-25T10:08:48.902873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896098+0000","last_change":"2024-04-25T10:08:48.896098+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896098+0000","last_undegraded":"2024-04-25T10:08:48.896098+0000","last_fullsized":"2024-04-25T10:08:48.896098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897466+0000","last_change":"2024-04-25T10:08:48.897466+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897466+0000","last_undegraded":"2024-04-25T10:08:48.897466+0000","last_fullsized":"2024-04-25T10:08:48.897466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902313+0000","last_change":"2024-04-25T10:08:48.902313+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902313+0000","last_undegraded":"2024-04-25T10:08:48.902313+0000","last_fullsized":"2024-04-25T10:08:48.902313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900867+0000","last_change":"2024-04-25T10:08:48.900867+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900867+0000","last_undegraded":"2024-04-25T10:08:48.900867+0000","last_fullsized":"2024-04-25T10:08:48.900867+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894114+0000","last_change":"2024-04-25T10:08:48.894114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894114+0000","last_undegraded":"2024-04-25T10:08:48.894114+0000","last_fullsized":"2024-04-25T10:08:48.894114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900938+0000","last_change":"2024-04-25T10:08:48.900938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900938+0000","last_undegraded":"2024-04-25T10:08:48.900938+0000","last_fullsized":"2024-04-25T10:08:48.900938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897207+0000","last_change":"2024-04-25T10:08:48.897207+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897207+0000","last_undegraded":"2024-04-25T10:08:48.897207+0000","last_fullsized":"2024-04-25T10:08:48.897207+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895876+0000","last_change":"2024-04-25T10:08:48.895876+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895876+0000","last_undegraded":"2024-04-25T10:08:48.895876+0000","last_fullsized":"2024-04-25T10:08:48.895876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922842+0000","last_change":"2024-04-25T10:08:48.913258+0000","last_active":"2024-04-25T10:08:48.922842+0000","last_peered":"2024-04-25T10:08:48.922842+0000","last_clean":"2024-04-25T10:08:48.922842+0000","last_became_active":"2024-04-25T10:08:48.912653+0000","last_became_peered":"2024-04-25T10:08:48.912653+0000","last_unstale":"2024-04-25T10:08:48.922842+0000","last_undegraded":"2024-04-25T10:08:48.922842+0000","last_fullsized":"2024-04-25T10:08:48.922842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:37.862500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895907+0000","last_change":"2024-04-25T10:08:48.895907+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895907+0000","last_undegraded":"2024-04-25T10:08:48.895907+0000","last_fullsized":"2024-04-25T10:08:48.895907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925620+0000","last_change":"2024-04-25T10:08:48.920404+0000","last_active":"2024-04-25T10:08:48.925620+0000","last_peered":"2024-04-25T10:08:48.925620+0000","last_clean":"2024-04-25T10:08:48.925620+0000","last_became_active":"2024-04-25T10:08:48.919687+0000","last_became_peered":"2024-04-25T10:08:48.919687+0000","last_unstale":"2024-04-25T10:08:48.925620+0000","last_undegraded":"2024-04-25T10:08:48.925620+0000","last_fullsized":"2024-04-25T10:08:48.925620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:08.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901697+0000","last_change":"2024-04-25T10:08:48.901697+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901697+0000","last_undegraded":"2024-04-25T10:08:48.901697+0000","last_fullsized":"2024-04-25T10:08:48.901697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924120+0000","last_change":"2024-04-25T10:08:48.921772+0000","last_active":"2024-04-25T10:08:48.924120+0000","last_peered":"2024-04-25T10:08:48.924120+0000","last_clean":"2024-04-25T10:08:48.924120+0000","last_became_active":"2024-04-25T10:08:48.921226+0000","last_became_peered":"2024-04-25T10:08:48.921226+0000","last_unstale":"2024-04-25T10:08:48.924120+0000","last_undegraded":"2024-04-25T10:08:48.924120+0000","last_fullsized":"2024-04-25T10:08:48.924120+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:45.783541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896461+0000","last_change":"2024-04-25T10:08:48.896461+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896461+0000","last_undegraded":"2024-04-25T10:08:48.896461+0000","last_fullsized":"2024-04-25T10:08:48.896461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927024+0000","last_change":"2024-04-25T10:08:48.920430+0000","last_active":"2024-04-25T10:08:48.927024+0000","last_peered":"2024-04-25T10:08:48.927024+0000","last_clean":"2024-04-25T10:08:48.927024+0000","last_became_active":"2024-04-25T10:08:48.919636+0000","last_became_peered":"2024-04-25T10:08:48.919636+0000","last_unstale":"2024-04-25T10:08:48.927024+0000","last_undegraded":"2024-04-25T10:08:48.927024+0000","last_fullsized":"2024-04-25T10:08:48.927024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:35.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898138+0000","last_change":"2024-04-25T10:08:48.898138+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898138+0000","last_undegraded":"2024-04-25T10:08:48.898138+0000","last_fullsized":"2024-04-25T10:08:48.898138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.920870+0000","last_change":"2024-04-25T10:08:48.913325+0000","last_active":"2024-04-25T10:08:48.920870+0000","last_peered":"2024-04-25T10:08:48.920870+0000","last_clean":"2024-04-25T10:08:48.920870+0000","last_became_active":"2024-04-25T10:08:48.912648+0000","last_became_peered":"2024-04-25T10:08:48.912648+0000","last_unstale":"2024-04-25T10:08:48.920870+0000","last_undegraded":"2024-04-25T10:08:48.920870+0000","last_fullsized":"2024-04-25T10:08:48.920870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:19.606750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893958+0000","last_change":"2024-04-25T10:08:48.893958+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893958+0000","last_undegraded":"2024-04-25T10:08:48.893958+0000","last_fullsized":"2024-04-25T10:08:48.893958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.926830+0000","last_change":"2024-04-25T10:08:48.921408+0000","last_active":"2024-04-25T10:08:48.926830+0000","last_peered":"2024-04-25T10:08:48.926830+0000","last_clean":"2024-04-25T10:08:48.926830+0000","last_became_active":"2024-04-25T10:08:48.921179+0000","last_became_peered":"2024-04-25T10:08:48.921179+0000","last_unstale":"2024-04-25T10:08:48.926830+0000","last_undegraded":"2024-04-25T10:08:48.926830+0000","last_fullsized":"2024-04-25T10:08:48.926830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:09.513075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896117+0000","last_change":"2024-04-25T10:08:48.896117+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896117+0000","last_undegraded":"2024-04-25T10:08:48.896117+0000","last_fullsized":"2024-04-25T10:08:48.896117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925432+0000","last_change":"2024-04-25T10:08:48.923790+0000","last_active":"2024-04-25T10:08:48.925432+0000","last_peered":"2024-04-25T10:08:48.925432+0000","last_clean":"2024-04-25T10:08:48.925432+0000","last_became_active":"2024-04-25T10:08:48.923543+0000","last_became_peered":"2024-04-25T10:08:48.923543+0000","last_unstale":"2024-04-25T10:08:48.925432+0000","last_undegraded":"2024-04-25T10:08:48.925432+0000","last_fullsized":"2024-04-25T10:08:48.925432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903018+0000","last_change":"2024-04-25T10:08:48.903018+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903018+0000","last_undegraded":"2024-04-25T10:08:48.903018+0000","last_fullsized":"2024-04-25T10:08:48.903018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923152+0000","last_change":"2024-04-25T10:08:48.915851+0000","last_active":"2024-04-25T10:08:48.923152+0000","last_peered":"2024-04-25T10:08:48.923152+0000","last_clean":"2024-04-25T10:08:48.923152+0000","last_became_active":"2024-04-25T10:08:48.915212+0000","last_became_peered":"2024-04-25T10:08:48.915212+0000","last_unstale":"2024-04-25T10:08:48.923152+0000","last_undegraded":"2024-04-25T10:08:48.923152+0000","last_fullsized":"2024-04-25T10:08:48.923152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.518129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894554+0000","last_change":"2024-04-25T10:08:48.894554+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894554+0000","last_undegraded":"2024-04-25T10:08:48.894554+0000","last_fullsized":"2024-04-25T10:08:48.894554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919005+0000","last_change":"2024-04-25T10:08:48.908751+0000","last_active":"2024-04-25T10:08:48.919005+0000","last_peered":"2024-04-25T10:08:48.919005+0000","last_clean":"2024-04-25T10:08:48.919005+0000","last_became_active":"2024-04-25T10:08:48.908243+0000","last_became_peered":"2024-04-25T10:08:48.908243+0000","last_unstale":"2024-04-25T10:08:48.919005+0000","last_undegraded":"2024-04-25T10:08:48.919005+0000","last_fullsized":"2024-04-25T10:08:48.919005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:43.075381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901140+0000","last_change":"2024-04-25T10:08:48.901140+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901140+0000","last_undegraded":"2024-04-25T10:08:48.901140+0000","last_fullsized":"2024-04-25T10:08:48.901140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919327+0000","last_change":"2024-04-25T10:08:48.917822+0000","last_active":"2024-04-25T10:08:48.919327+0000","last_peered":"2024-04-25T10:08:48.919327+0000","last_clean":"2024-04-25T10:08:48.919327+0000","last_became_active":"2024-04-25T10:08:48.917540+0000","last_became_peered":"2024-04-25T10:08:48.917540+0000","last_unstale":"2024-04-25T10:08:48.919327+0000","last_undegraded":"2024-04-25T10:08:48.919327+0000","last_fullsized":"2024-04-25T10:08:48.919327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898751+0000","last_change":"2024-04-25T10:08:48.898751+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898751+0000","last_undegraded":"2024-04-25T10:08:48.898751+0000","last_fullsized":"2024-04-25T10:08:48.898751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932807+0000","last_change":"2024-04-25T10:08:48.930283+0000","last_active":"2024-04-25T10:08:48.932807+0000","last_peered":"2024-04-25T10:08:48.932807+0000","last_clean":"2024-04-25T10:08:48.932807+0000","last_became_active":"2024-04-25T10:08:48.929026+0000","last_became_peered":"2024-04-25T10:08:48.929026+0000","last_unstale":"2024-04-25T10:08:48.932807+0000","last_undegraded":"2024-04-25T10:08:48.932807+0000","last_fullsized":"2024-04-25T10:08:48.932807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:30.464303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902636+0000","last_change":"2024-04-25T10:08:48.902636+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902636+0000","last_undegraded":"2024-04-25T10:08:48.902636+0000","last_fullsized":"2024-04-25T10:08:48.902636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927628+0000","last_change":"2024-04-25T10:08:48.926872+0000","last_active":"2024-04-25T10:08:48.927628+0000","last_peered":"2024-04-25T10:08:48.927628+0000","last_clean":"2024-04-25T10:08:48.927628+0000","last_became_active":"2024-04-25T10:08:48.926660+0000","last_became_peered":"2024-04-25T10:08:48.926660+0000","last_unstale":"2024-04-25T10:08:48.927628+0000","last_undegraded":"2024-04-25T10:08:48.927628+0000","last_fullsized":"2024-04-25T10:08:48.927628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.544601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896769+0000","last_change":"2024-04-25T10:08:48.896769+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896769+0000","last_undegraded":"2024-04-25T10:08:48.896769+0000","last_fullsized":"2024-04-25T10:08:48.896769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923043+0000","last_change":"2024-04-25T10:08:48.916082+0000","last_active":"2024-04-25T10:08:48.923043+0000","last_peered":"2024-04-25T10:08:48.923043+0000","last_clean":"2024-04-25T10:08:48.923043+0000","last_became_active":"2024-04-25T10:08:48.915311+0000","last_became_peered":"2024-04-25T10:08:48.915311+0000","last_unstale":"2024-04-25T10:08:48.923043+0000","last_undegraded":"2024-04-25T10:08:48.923043+0000","last_fullsized":"2024-04-25T10:08:48.923043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.892575+0000","last_change":"2024-04-25T10:08:48.892575+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.892575+0000","last_undegraded":"2024-04-25T10:08:48.892575+0000","last_fullsized":"2024-04-25T10:08:48.892575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927634+0000","last_change":"2024-04-25T10:08:48.927015+0000","last_active":"2024-04-25T10:08:48.927634+0000","last_peered":"2024-04-25T10:08:48.927634+0000","last_clean":"2024-04-25T10:08:48.927634+0000","last_became_active":"2024-04-25T10:08:48.926431+0000","last_became_peered":"2024-04-25T10:08:48.926431+0000","last_unstale":"2024-04-25T10:08:48.927634+0000","last_undegraded":"2024-04-25T10:08:48.927634+0000","last_fullsized":"2024-04-25T10:08:48.927634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:25.811654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900421+0000","last_change":"2024-04-25T10:08:48.900421+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900421+0000","last_undegraded":"2024-04-25T10:08:48.900421+0000","last_fullsized":"2024-04-25T10:08:48.900421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925675+0000","last_change":"2024-04-25T10:08:48.920708+0000","last_active":"2024-04-25T10:08:48.925675+0000","last_peered":"2024-04-25T10:08:48.925675+0000","last_clean":"2024-04-25T10:08:48.925675+0000","last_became_active":"2024-04-25T10:08:48.920393+0000","last_became_peered":"2024-04-25T10:08:48.920393+0000","last_unstale":"2024-04-25T10:08:48.925675+0000","last_undegraded":"2024-04-25T10:08:48.925675+0000","last_fullsized":"2024-04-25T10:08:48.925675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:14.265209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896676+0000","last_change":"2024-04-25T10:08:48.896676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896676+0000","last_undegraded":"2024-04-25T10:08:48.896676+0000","last_fullsized":"2024-04-25T10:08:48.896676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.918051+0000","last_change":"2024-04-25T10:08:48.912297+0000","last_active":"2024-04-25T10:08:48.918051+0000","last_peered":"2024-04-25T10:08:48.918051+0000","last_clean":"2024-04-25T10:08:48.918051+0000","last_became_active":"2024-04-25T10:08:48.911555+0000","last_became_peered":"2024-04-25T10:08:48.911555+0000","last_unstale":"2024-04-25T10:08:48.918051+0000","last_undegraded":"2024-04-25T10:08:48.918051+0000","last_fullsized":"2024-04-25T10:08:48.918051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.083371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895443+0000","last_change":"2024-04-25T10:08:48.895443+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895443+0000","last_undegraded":"2024-04-25T10:08:48.895443+0000","last_fullsized":"2024-04-25T10:08:48.895443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.928484+0000","last_change":"2024-04-25T10:08:48.923258+0000","last_active":"2024-04-25T10:08:48.928484+0000","last_peered":"2024-04-25T10:08:48.928484+0000","last_clean":"2024-04-25T10:08:48.928484+0000","last_became_active":"2024-04-25T10:08:48.922652+0000","last_became_peered":"2024-04-25T10:08:48.922652+0000","last_unstale":"2024-04-25T10:08:48.928484+0000","last_undegraded":"2024-04-25T10:08:48.928484+0000","last_fullsized":"2024-04-25T10:08:48.928484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894725+0000","last_change":"2024-04-25T10:08:48.894725+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894725+0000","last_undegraded":"2024-04-25T10:08:48.894725+0000","last_fullsized":"2024-04-25T10:08:48.894725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929141+0000","last_change":"2024-04-25T10:08:48.924659+0000","last_active":"2024-04-25T10:08:48.929141+0000","last_peered":"2024-04-25T10:08:48.929141+0000","last_clean":"2024-04-25T10:08:48.929141+0000","last_became_active":"2024-04-25T10:08:48.923481+0000","last_became_peered":"2024-04-25T10:08:48.923481+0000","last_unstale":"2024-04-25T10:08:48.929141+0000","last_undegraded":"2024-04-25T10:08:48.929141+0000","last_fullsized":"2024-04-25T10:08:48.929141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:56.284437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895174+0000","last_change":"2024-04-25T10:08:48.895174+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895174+0000","last_undegraded":"2024-04-25T10:08:48.895174+0000","last_fullsized":"2024-04-25T10:08:48.895174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925786+0000","last_change":"2024-04-25T10:08:48.921337+0000","last_active":"2024-04-25T10:08:48.925786+0000","last_peered":"2024-04-25T10:08:48.925786+0000","last_clean":"2024-04-25T10:08:48.925786+0000","last_became_active":"2024-04-25T10:08:48.918289+0000","last_became_peered":"2024-04-25T10:08:48.918289+0000","last_unstale":"2024-04-25T10:08:48.925786+0000","last_undegraded":"2024-04-25T10:08:48.925786+0000","last_fullsized":"2024-04-25T10:08:48.925786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.100052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894501+0000","last_change":"2024-04-25T10:08:48.894501+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894501+0000","last_undegraded":"2024-04-25T10:08:48.894501+0000","last_fullsized":"2024-04-25T10:08:48.894501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.911288+0000","last_change":"2024-04-25T10:08:48.906519+0000","last_active":"2024-04-25T10:08:48.911288+0000","last_peered":"2024-04-25T10:08:48.911288+0000","last_clean":"2024-04-25T10:08:48.911288+0000","last_became_active":"2024-04-25T10:08:48.905583+0000","last_became_peered":"2024-04-25T10:08:48.905583+0000","last_unstale":"2024-04-25T10:08:48.911288+0000","last_undegraded":"2024-04-25T10:08:48.911288+0000","last_fullsized":"2024-04-25T10:08:48.911288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:40.496209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900102+0000","last_change":"2024-04-25T10:08:48.900102+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900102+0000","last_undegraded":"2024-04-25T10:08:48.900102+0000","last_fullsized":"2024-04-25T10:08:48.900102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923192+0000","last_change":"2024-04-25T10:08:48.920400+0000","last_active":"2024-04-25T10:08:48.923192+0000","last_peered":"2024-04-25T10:08:48.923192+0000","last_clean":"2024-04-25T10:08:48.923192+0000","last_became_active":"2024-04-25T10:08:48.920006+0000","last_became_peered":"2024-04-25T10:08:48.920006+0000","last_unstale":"2024-04-25T10:08:48.923192+0000","last_undegraded":"2024-04-25T10:08:48.923192+0000","last_fullsized":"2024-04-25T10:08:48.923192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899888+0000","last_change":"2024-04-25T10:08:48.899888+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899888+0000","last_undegraded":"2024-04-25T10:08:48.899888+0000","last_fullsized":"2024-04-25T10:08:48.899888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921292+0000","last_change":"2024-04-25T10:08:48.908793+0000","last_active":"2024-04-25T10:08:48.921292+0000","last_peered":"2024-04-25T10:08:48.921292+0000","last_clean":"2024-04-25T10:08:48.921292+0000","last_became_active":"2024-04-25T10:08:48.908121+0000","last_became_peered":"2024-04-25T10:08:48.908121+0000","last_unstale":"2024-04-25T10:08:48.921292+0000","last_undegraded":"2024-04-25T10:08:48.921292+0000","last_fullsized":"2024-04-25T10:08:48.921292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:54.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894144+0000","last_change":"2024-04-25T10:08:48.894144+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894144+0000","last_undegraded":"2024-04-25T10:08:48.894144+0000","last_fullsized":"2024-04-25T10:08:48.894144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.916943+0000","last_change":"2024-04-25T10:08:48.907802+0000","last_active":"2024-04-25T10:08:48.916943+0000","last_peered":"2024-04-25T10:08:48.916943+0000","last_clean":"2024-04-25T10:08:48.916943+0000","last_became_active":"2024-04-25T10:08:48.907484+0000","last_became_peered":"2024-04-25T10:08:48.907484+0000","last_unstale":"2024-04-25T10:08:48.916943+0000","last_undegraded":"2024-04-25T10:08:48.916943+0000","last_fullsized":"2024-04-25T10:08:48.916943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.725069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901281+0000","last_change":"2024-04-25T10:08:48.901281+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901281+0000","last_undegraded":"2024-04-25T10:08:48.901281+0000","last_fullsized":"2024-04-25T10:08:48.901281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929174+0000","last_change":"2024-04-25T10:08:48.925017+0000","last_active":"2024-04-25T10:08:48.929174+0000","last_peered":"2024-04-25T10:08:48.929174+0000","last_clean":"2024-04-25T10:08:48.929174+0000","last_became_active":"2024-04-25T10:08:48.924231+0000","last_became_peered":"2024-04-25T10:08:48.924231+0000","last_unstale":"2024-04-25T10:08:48.929174+0000","last_undegraded":"2024-04-25T10:08:48.929174+0000","last_fullsized":"2024-04-25T10:08:48.929174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:31.040785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893835+0000","last_change":"2024-04-25T10:08:48.893835+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893835+0000","last_undegraded":"2024-04-25T10:08:48.893835+0000","last_fullsized":"2024-04-25T10:08:48.893835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.931898+0000","last_change":"2024-04-25T10:08:48.927255+0000","last_active":"2024-04-25T10:08:48.931898+0000","last_peered":"2024-04-25T10:08:48.931898+0000","last_clean":"2024-04-25T10:08:48.931898+0000","last_became_active":"2024-04-25T10:08:48.926623+0000","last_became_peered":"2024-04-25T10:08:48.926623+0000","last_unstale":"2024-04-25T10:08:48.931898+0000","last_undegraded":"2024-04-25T10:08:48.931898+0000","last_fullsized":"2024-04-25T10:08:48.931898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.736381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894241+0000","last_change":"2024-04-25T10:08:48.894241+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894241+0000","last_undegraded":"2024-04-25T10:08:48.894241+0000","last_fullsized":"2024-04-25T10:08:48.894241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932774+0000","last_change":"2024-04-25T10:08:48.929895+0000","last_active":"2024-04-25T10:08:48.932774+0000","last_peered":"2024-04-25T10:08:48.932774+0000","last_clean":"2024-04-25T10:08:48.932774+0000","last_became_active":"2024-04-25T10:08:48.928316+0000","last_became_peered":"2024-04-25T10:08:48.928316+0000","last_unstale":"2024-04-25T10:08:48.932774+0000","last_undegraded":"2024-04-25T10:08:48.932774+0000","last_fullsized":"2024-04-25T10:08:48.932774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:38.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902043+0000","last_change":"2024-04-25T10:08:48.902043+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902043+0000","last_undegraded":"2024-04-25T10:08:48.902043+0000","last_fullsized":"2024-04-25T10:08:48.902043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922564+0000","last_change":"2024-04-25T10:08:48.913827+0000","last_active":"2024-04-25T10:08:48.922564+0000","last_peered":"2024-04-25T10:08:48.922564+0000","last_clean":"2024-04-25T10:08:48.922564+0000","last_became_active":"2024-04-25T10:08:48.912878+0000","last_became_peered":"2024-04-25T10:08:48.912878+0000","last_unstale":"2024-04-25T10:08:48.922564+0000","last_undegraded":"2024-04-25T10:08:48.922564+0000","last_fullsized":"2024-04-25T10:08:48.922564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:15.708904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901260+0000","last_change":"2024-04-25T10:08:48.901260+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901260+0000","last_undegraded":"2024-04-25T10:08:48.901260+0000","last_fullsized":"2024-04-25T10:08:48.901260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927592+0000","last_change":"2024-04-25T10:08:48.926824+0000","last_active":"2024-04-25T10:08:48.927592+0000","last_peered":"2024-04-25T10:08:48.927592+0000","last_clean":"2024-04-25T10:08:48.927592+0000","last_became_active":"2024-04-25T10:08:48.926515+0000","last_became_peered":"2024-04-25T10:08:48.926515+0000","last_unstale":"2024-04-25T10:08:48.927592+0000","last_undegraded":"2024-04-25T10:08:48.927592+0000","last_fullsized":"2024-04-25T10:08:48.927592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:21.486349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896505+0000","last_change":"2024-04-25T10:08:48.896505+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896505+0000","last_undegraded":"2024-04-25T10:08:48.896505+0000","last_fullsized":"2024-04-25T10:08:48.896505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921407+0000","last_change":"2024-04-25T10:08:48.912204+0000","last_active":"2024-04-25T10:08:48.921407+0000","last_peered":"2024-04-25T10:08:48.921407+0000","last_clean":"2024-04-25T10:08:48.921407+0000","last_became_active":"2024-04-25T10:08:48.911509+0000","last_became_peered":"2024-04-25T10:08:48.911509+0000","last_unstale":"2024-04-25T10:08:48.921407+0000","last_undegraded":"2024-04-25T10:08:48.921407+0000","last_fullsized":"2024-04-25T10:08:48.921407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.689628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893499+0000","last_change":"2024-04-25T10:08:48.893499+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893499+0000","last_undegraded":"2024-04-25T10:08:48.893499+0000","last_fullsized":"2024-04-25T10:08:48.893499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921699+0000","last_change":"2024-04-25T10:08:48.912652+0000","last_active":"2024-04-25T10:08:48.921699+0000","last_peered":"2024-04-25T10:08:48.921699+0000","last_clean":"2024-04-25T10:08:48.921699+0000","last_became_active":"2024-04-25T10:08:48.911838+0000","last_became_peered":"2024-04-25T10:08:48.911838+0000","last_unstale":"2024-04-25T10:08:48.921699+0000","last_undegraded":"2024-04-25T10:08:48.921699+0000","last_fullsized":"2024-04-25T10:08:48.921699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:37.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893632+0000","last_change":"2024-04-25T10:08:48.893632+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893632+0000","last_undegraded":"2024-04-25T10:08:48.893632+0000","last_fullsized":"2024-04-25T10:08:48.893632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924165+0000","last_change":"2024-04-25T10:08:48.920751+0000","last_active":"2024-04-25T10:08:48.924165+0000","last_peered":"2024-04-25T10:08:48.924165+0000","last_clean":"2024-04-25T10:08:48.924165+0000","last_became_active":"2024-04-25T10:08:48.920001+0000","last_became_peered":"2024-04-25T10:08:48.920001+0000","last_unstale":"2024-04-25T10:08:48.924165+0000","last_undegraded":"2024-04-25T10:08:48.924165+0000","last_fullsized":"2024-04-25T10:08:48.924165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902674+0000","last_change":"2024-04-25T10:08:48.902674+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902674+0000","last_undegraded":"2024-04-25T10:08:48.902674+0000","last_fullsized":"2024-04-25T10:08:48.902674+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927324+0000","last_change":"2024-04-25T10:08:48.924246+0000","last_active":"2024-04-25T10:08:48.927324+0000","last_peered":"2024-04-25T10:08:48.927324+0000","last_clean":"2024-04-25T10:08:48.927324+0000","last_became_active":"2024-04-25T10:08:48.923370+0000","last_became_peered":"2024-04-25T10:08:48.923370+0000","last_unstale":"2024-04-25T10:08:48.927324+0000","last_undegraded":"2024-04-25T10:08:48.927324+0000","last_fullsized":"2024-04-25T10:08:48.927324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:01.982597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.905354+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:08:48.905354+0000","last_peered":"2024-04-25T10:08:48.905354+0000","last_clean":"2024-04-25T10:08:48.905354+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:08:48.905354+0000","last_undegraded":"2024-04-25T10:08:48.905354+0000","last_fullsized":"2024-04-25T10:08:48.905354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:53.293 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.448+0000 7f4288f13640 1 Processor -- start 2024-04-25T10:08:53.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 -- start start 2024-04-25T10:08:53.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4284144ca0 0x7f428406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f428406ec60 0x7f4284142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42840ccc70 con 0x7f42840ce510 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840cc2b0 con 0x7f4284144ca0 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4288f13640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428406e290 con 0x7f428406ec60 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4284144ca0 0x7f428406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4284144ca0 0x7f428406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49746/0 (socket says 172.21.15.82:49746) 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 -- 172.21.15.82:0/2603521086 learned_addr learned my addr 172.21.15.82:0/2603521086 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4282d9d640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f428406ec60 0x7f4284142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 -- 172.21.15.82:0/2603521086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f428406ec60 msgr2=0x7f4284142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f428406ec60 0x7f4284142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 -- 172.21.15.82:0/2603521086 >> v1:172.21.15.196:6789/0 conn(0x7f42840ce510 legacy=0x7f42840ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 -- 172.21.15.82:0/2603521086 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428406dc40 con 0x7f4284144ca0 2024-04-25T10:08:53.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f4281d9b640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4284144ca0 0x7f428406e720 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9c220cc08e8aac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f428159a640 1 -- 172.21.15.82:0/2603521086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0b5070 con 0x7f4284144ca0 2024-04-25T10:08:53.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f428159a640 1 -- 172.21.15.82:0/2603521086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c0a89d0 con 0x7f4284144ca0 2024-04-25T10:08:53.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.452+0000 7f428159a640 1 -- 172.21.15.82:0/2603521086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c0a8da0 con 0x7f4284144ca0 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/2603521086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4284144ca0 msgr2=0x7f428406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4284144ca0 0x7f428406e720 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f426c009870 tx=0x7f426c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/2603521086 shutdown_connections 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f428406ec60 0x7f4284142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- 172.21.15.82:0/2603521086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4284144ca0 0x7f428406e720 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/2603521086 >> 172.21.15.82:0/2603521086 conn(0x7f42840b11a0 msgr2=0x7f42840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/2603521086 shutdown_connections 2024-04-25T10:08:53.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/2603521086 wait complete. 2024-04-25T10:08:53.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 Processor -- start 2024-04-25T10:08:53.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- start start 2024-04-25T10:08:53.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42840ce510 0x7f42841f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4284144ca0 0x7f42840c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42841451f0 con 0x7f428406ec60 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42840cc900 con 0x7f4284144ca0 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4284145370 con 0x7f42840ce510 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49762/0 (socket says 172.21.15.82:49762) 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 -- 172.21.15.82:0/3210404054 learned_addr learned my addr 172.21.15.82:0/3210404054 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4282d9d640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4284144ca0 0x7f42840c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4281d9b640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42840ce510 0x7f42841f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 -- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42840ce510 msgr2=0x7f42841f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42840ce510 0x7f42841f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 -- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4284144ca0 msgr2=0x7f42840c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4284144ca0 0x7f42840c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 -- 172.21.15.82:0/3210404054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42841f5ca0 con 0x7f428406ec60 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f428259c640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c450 tx=0x7f427c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c011790 con 0x7f428406ec60 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c017070 con 0x7f428406ec60 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42840a8b90 con 0x7f428406ec60 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c011b90 con 0x7f428406ec60 2024-04-25T10:08:53.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.456+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42841e2140 con 0x7f428406ec60 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428404efa0 con 0x7f428406ec60 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f427c011d30 con 0x7f428406ec60 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f427affd640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 0x7f4264069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f427c084f60 con 0x7f428406ec60 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f4281d9b640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 0x7f4264069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.460+0000 7f4281d9b640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 0x7f4264069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b2000 tx=0x7f426c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.464+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f427c055e70 con 0x7f428406ec60 2024-04-25T10:08:53.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.604+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42840b0f60 con 0x7f4264066f60 2024-04-25T10:08:53.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.612+0000 7f427affd640 1 -- 172.21.15.82:0/3210404054 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246766 (secure 0 0 0) 0x7f42840b0f60 con 0x7f4264066f60 2024-04-25T10:08:53.620 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:53.621 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:08:53.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 msgr2=0x7f4264069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 0x7f4264069420 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f426c0b2000 tx=0x7f426c008040 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 msgr2=0x7f4284143540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c450 tx=0x7f427c00c910 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 shutdown_connections 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4264066f60 0x7f4264069420 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f42840ce510 0x7f42841f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4284144ca0 0x7f42840c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 --2- 172.21.15.82:0/3210404054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f428406ec60 0x7f4284143540 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 >> 172.21.15.82:0/3210404054 conn(0x7f42840b11a0 msgr2=0x7f42840c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 shutdown_connections 2024-04-25T10:08:53.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.616+0000 7f4288f13640 1 -- 172.21.15.82:0/3210404054 wait complete. 2024-04-25T10:08:53.637 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-25T10:08:53.389073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259204,"kb_used_data":9736,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715516,"statfs":{"total":773094113280,"available":772828688384,"internally_reserved":0,"allocated":9969664,"data_stored":5203416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.513569"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897534+0000","last_change":"2024-04-25T10:08:48.897534+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897534+0000","last_undegraded":"2024-04-25T10:08:48.897534+0000","last_fullsized":"2024-04-25T10:08:48.897534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900991+0000","last_change":"2024-04-25T10:08:48.900991+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900991+0000","last_undegraded":"2024-04-25T10:08:48.900991+0000","last_fullsized":"2024-04-25T10:08:48.900991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897938+0000","last_change":"2024-04-25T10:08:48.897938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897938+0000","last_undegraded":"2024-04-25T10:08:48.897938+0000","last_fullsized":"2024-04-25T10:08:48.897938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.909082+0000","last_change":"2024-04-25T10:08:48.909082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.909082+0000","last_undegraded":"2024-04-25T10:08:48.909082+0000","last_fullsized":"2024-04-25T10:08:48.909082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894412+0000","last_change":"2024-04-25T10:08:48.894412+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894412+0000","last_undegraded":"2024-04-25T10:08:48.894412+0000","last_fullsized":"2024-04-25T10:08:48.894412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896657+0000","last_change":"2024-04-25T10:08:48.896657+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896657+0000","last_undegraded":"2024-04-25T10:08:48.896657+0000","last_fullsized":"2024-04-25T10:08:48.896657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900586+0000","last_change":"2024-04-25T10:08:48.900586+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900586+0000","last_undegraded":"2024-04-25T10:08:48.900586+0000","last_fullsized":"2024-04-25T10:08:48.900586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899208+0000","last_change":"2024-04-25T10:08:48.899208+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899208+0000","last_undegraded":"2024-04-25T10:08:48.899208+0000","last_fullsized":"2024-04-25T10:08:48.899208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897320+0000","last_change":"2024-04-25T10:08:48.897320+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897320+0000","last_undegraded":"2024-04-25T10:08:48.897320+0000","last_fullsized":"2024-04-25T10:08:48.897320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898252+0000","last_change":"2024-04-25T10:08:48.898252+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898252+0000","last_undegraded":"2024-04-25T10:08:48.898252+0000","last_fullsized":"2024-04-25T10:08:48.898252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.912082+0000","last_change":"2024-04-25T10:08:48.912082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.912082+0000","last_undegraded":"2024-04-25T10:08:48.912082+0000","last_fullsized":"2024-04-25T10:08:48.912082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895959+0000","last_change":"2024-04-25T10:08:48.895959+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895959+0000","last_undegraded":"2024-04-25T10:08:48.895959+0000","last_fullsized":"2024-04-25T10:08:48.895959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903583+0000","last_change":"2024-04-25T10:08:48.903583+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903583+0000","last_undegraded":"2024-04-25T10:08:48.903583+0000","last_fullsized":"2024-04-25T10:08:48.903583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896282+0000","last_change":"2024-04-25T10:08:48.896282+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896282+0000","last_undegraded":"2024-04-25T10:08:48.896282+0000","last_fullsized":"2024-04-25T10:08:48.896282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902220+0000","last_change":"2024-04-25T10:08:48.902220+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902220+0000","last_undegraded":"2024-04-25T10:08:48.902220+0000","last_fullsized":"2024-04-25T10:08:48.902220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897711+0000","last_change":"2024-04-25T10:08:48.897711+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897711+0000","last_undegraded":"2024-04-25T10:08:48.897711+0000","last_fullsized":"2024-04-25T10:08:48.897711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896879+0000","last_change":"2024-04-25T10:08:48.896879+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896879+0000","last_undegraded":"2024-04-25T10:08:48.896879+0000","last_fullsized":"2024-04-25T10:08:48.896879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899121+0000","last_change":"2024-04-25T10:08:48.899121+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899121+0000","last_undegraded":"2024-04-25T10:08:48.899121+0000","last_fullsized":"2024-04-25T10:08:48.899121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899094+0000","last_change":"2024-04-25T10:08:48.899094+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899094+0000","last_undegraded":"2024-04-25T10:08:48.899094+0000","last_fullsized":"2024-04-25T10:08:48.899094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.908676+0000","last_change":"2024-04-25T10:08:48.908676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.908676+0000","last_undegraded":"2024-04-25T10:08:48.908676+0000","last_fullsized":"2024-04-25T10:08:48.908676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895275+0000","last_change":"2024-04-25T10:08:48.895275+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895275+0000","last_undegraded":"2024-04-25T10:08:48.895275+0000","last_fullsized":"2024-04-25T10:08:48.895275+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895212+0000","last_change":"2024-04-25T10:08:48.895212+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895212+0000","last_undegraded":"2024-04-25T10:08:48.895212+0000","last_fullsized":"2024-04-25T10:08:48.895212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895119+0000","last_change":"2024-04-25T10:08:48.895119+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895119+0000","last_undegraded":"2024-04-25T10:08:48.895119+0000","last_fullsized":"2024-04-25T10:08:48.895119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897114+0000","last_change":"2024-04-25T10:08:48.897114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897114+0000","last_undegraded":"2024-04-25T10:08:48.897114+0000","last_fullsized":"2024-04-25T10:08:48.897114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902873+0000","last_change":"2024-04-25T10:08:48.902873+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902873+0000","last_undegraded":"2024-04-25T10:08:48.902873+0000","last_fullsized":"2024-04-25T10:08:48.902873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896098+0000","last_change":"2024-04-25T10:08:48.896098+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896098+0000","last_undegraded":"2024-04-25T10:08:48.896098+0000","last_fullsized":"2024-04-25T10:08:48.896098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897466+0000","last_change":"2024-04-25T10:08:48.897466+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897466+0000","last_undegraded":"2024-04-25T10:08:48.897466+0000","last_fullsized":"2024-04-25T10:08:48.897466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902313+0000","last_change":"2024-04-25T10:08:48.902313+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902313+0000","last_undegraded":"2024-04-25T10:08:48.902313+0000","last_fullsized":"2024-04-25T10:08:48.902313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900867+0000","last_change":"2024-04-25T10:08:48.900867+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900867+0000","last_undegraded":"2024-04-25T10:08:48.900867+0000","last_fullsized":"2024-04-25T10:08:48.900867+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894114+0000","last_change":"2024-04-25T10:08:48.894114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894114+0000","last_undegraded":"2024-04-25T10:08:48.894114+0000","last_fullsized":"2024-04-25T10:08:48.894114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900938+0000","last_change":"2024-04-25T10:08:48.900938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900938+0000","last_undegraded":"2024-04-25T10:08:48.900938+0000","last_fullsized":"2024-04-25T10:08:48.900938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897207+0000","last_change":"2024-04-25T10:08:48.897207+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897207+0000","last_undegraded":"2024-04-25T10:08:48.897207+0000","last_fullsized":"2024-04-25T10:08:48.897207+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895876+0000","last_change":"2024-04-25T10:08:48.895876+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895876+0000","last_undegraded":"2024-04-25T10:08:48.895876+0000","last_fullsized":"2024-04-25T10:08:48.895876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922842+0000","last_change":"2024-04-25T10:08:48.913258+0000","last_active":"2024-04-25T10:08:48.922842+0000","last_peered":"2024-04-25T10:08:48.922842+0000","last_clean":"2024-04-25T10:08:48.922842+0000","last_became_active":"2024-04-25T10:08:48.912653+0000","last_became_peered":"2024-04-25T10:08:48.912653+0000","last_unstale":"2024-04-25T10:08:48.922842+0000","last_undegraded":"2024-04-25T10:08:48.922842+0000","last_fullsized":"2024-04-25T10:08:48.922842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:37.862500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895907+0000","last_change":"2024-04-25T10:08:48.895907+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895907+0000","last_undegraded":"2024-04-25T10:08:48.895907+0000","last_fullsized":"2024-04-25T10:08:48.895907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925620+0000","last_change":"2024-04-25T10:08:48.920404+0000","last_active":"2024-04-25T10:08:48.925620+0000","last_peered":"2024-04-25T10:08:48.925620+0000","last_clean":"2024-04-25T10:08:48.925620+0000","last_became_active":"2024-04-25T10:08:48.919687+0000","last_became_peered":"2024-04-25T10:08:48.919687+0000","last_unstale":"2024-04-25T10:08:48.925620+0000","last_undegraded":"2024-04-25T10:08:48.925620+0000","last_fullsized":"2024-04-25T10:08:48.925620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:08.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901697+0000","last_change":"2024-04-25T10:08:48.901697+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901697+0000","last_undegraded":"2024-04-25T10:08:48.901697+0000","last_fullsized":"2024-04-25T10:08:48.901697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924120+0000","last_change":"2024-04-25T10:08:48.921772+0000","last_active":"2024-04-25T10:08:48.924120+0000","last_peered":"2024-04-25T10:08:48.924120+0000","last_clean":"2024-04-25T10:08:48.924120+0000","last_became_active":"2024-04-25T10:08:48.921226+0000","last_became_peered":"2024-04-25T10:08:48.921226+0000","last_unstale":"2024-04-25T10:08:48.924120+0000","last_undegraded":"2024-04-25T10:08:48.924120+0000","last_fullsized":"2024-04-25T10:08:48.924120+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:45.783541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896461+0000","last_change":"2024-04-25T10:08:48.896461+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896461+0000","last_undegraded":"2024-04-25T10:08:48.896461+0000","last_fullsized":"2024-04-25T10:08:48.896461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927024+0000","last_change":"2024-04-25T10:08:48.920430+0000","last_active":"2024-04-25T10:08:48.927024+0000","last_peered":"2024-04-25T10:08:48.927024+0000","last_clean":"2024-04-25T10:08:48.927024+0000","last_became_active":"2024-04-25T10:08:48.919636+0000","last_became_peered":"2024-04-25T10:08:48.919636+0000","last_unstale":"2024-04-25T10:08:48.927024+0000","last_undegraded":"2024-04-25T10:08:48.927024+0000","last_fullsized":"2024-04-25T10:08:48.927024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:35.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898138+0000","last_change":"2024-04-25T10:08:48.898138+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898138+0000","last_undegraded":"2024-04-25T10:08:48.898138+0000","last_fullsized":"2024-04-25T10:08:48.898138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.920870+0000","last_change":"2024-04-25T10:08:48.913325+0000","last_active":"2024-04-25T10:08:48.920870+0000","last_peered":"2024-04-25T10:08:48.920870+0000","last_clean":"2024-04-25T10:08:48.920870+0000","last_became_active":"2024-04-25T10:08:48.912648+0000","last_became_peered":"2024-04-25T10:08:48.912648+0000","last_unstale":"2024-04-25T10:08:48.920870+0000","last_undegraded":"2024-04-25T10:08:48.920870+0000","last_fullsized":"2024-04-25T10:08:48.920870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:19.606750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893958+0000","last_change":"2024-04-25T10:08:48.893958+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893958+0000","last_undegraded":"2024-04-25T10:08:48.893958+0000","last_fullsized":"2024-04-25T10:08:48.893958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.926830+0000","last_change":"2024-04-25T10:08:48.921408+0000","last_active":"2024-04-25T10:08:48.926830+0000","last_peered":"2024-04-25T10:08:48.926830+0000","last_clean":"2024-04-25T10:08:48.926830+0000","last_became_active":"2024-04-25T10:08:48.921179+0000","last_became_peered":"2024-04-25T10:08:48.921179+0000","last_unstale":"2024-04-25T10:08:48.926830+0000","last_undegraded":"2024-04-25T10:08:48.926830+0000","last_fullsized":"2024-04-25T10:08:48.926830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:09.513075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896117+0000","last_change":"2024-04-25T10:08:48.896117+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896117+0000","last_undegraded":"2024-04-25T10:08:48.896117+0000","last_fullsized":"2024-04-25T10:08:48.896117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925432+0000","last_change":"2024-04-25T10:08:48.923790+0000","last_active":"2024-04-25T10:08:48.925432+0000","last_peered":"2024-04-25T10:08:48.925432+0000","last_clean":"2024-04-25T10:08:48.925432+0000","last_became_active":"2024-04-25T10:08:48.923543+0000","last_became_peered":"2024-04-25T10:08:48.923543+0000","last_unstale":"2024-04-25T10:08:48.925432+0000","last_undegraded":"2024-04-25T10:08:48.925432+0000","last_fullsized":"2024-04-25T10:08:48.925432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903018+0000","last_change":"2024-04-25T10:08:48.903018+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903018+0000","last_undegraded":"2024-04-25T10:08:48.903018+0000","last_fullsized":"2024-04-25T10:08:48.903018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923152+0000","last_change":"2024-04-25T10:08:48.915851+0000","last_active":"2024-04-25T10:08:48.923152+0000","last_peered":"2024-04-25T10:08:48.923152+0000","last_clean":"2024-04-25T10:08:48.923152+0000","last_became_active":"2024-04-25T10:08:48.915212+0000","last_became_peered":"2024-04-25T10:08:48.915212+0000","last_unstale":"2024-04-25T10:08:48.923152+0000","last_undegraded":"2024-04-25T10:08:48.923152+0000","last_fullsized":"2024-04-25T10:08:48.923152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.518129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894554+0000","last_change":"2024-04-25T10:08:48.894554+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894554+0000","last_undegraded":"2024-04-25T10:08:48.894554+0000","last_fullsized":"2024-04-25T10:08:48.894554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919005+0000","last_change":"2024-04-25T10:08:48.908751+0000","last_active":"2024-04-25T10:08:48.919005+0000","last_peered":"2024-04-25T10:08:48.919005+0000","last_clean":"2024-04-25T10:08:48.919005+0000","last_became_active":"2024-04-25T10:08:48.908243+0000","last_became_peered":"2024-04-25T10:08:48.908243+0000","last_unstale":"2024-04-25T10:08:48.919005+0000","last_undegraded":"2024-04-25T10:08:48.919005+0000","last_fullsized":"2024-04-25T10:08:48.919005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:43.075381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901140+0000","last_change":"2024-04-25T10:08:48.901140+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901140+0000","last_undegraded":"2024-04-25T10:08:48.901140+0000","last_fullsized":"2024-04-25T10:08:48.901140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919327+0000","last_change":"2024-04-25T10:08:48.917822+0000","last_active":"2024-04-25T10:08:48.919327+0000","last_peered":"2024-04-25T10:08:48.919327+0000","last_clean":"2024-04-25T10:08:48.919327+0000","last_became_active":"2024-04-25T10:08:48.917540+0000","last_became_peered":"2024-04-25T10:08:48.917540+0000","last_unstale":"2024-04-25T10:08:48.919327+0000","last_undegraded":"2024-04-25T10:08:48.919327+0000","last_fullsized":"2024-04-25T10:08:48.919327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898751+0000","last_change":"2024-04-25T10:08:48.898751+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898751+0000","last_undegraded":"2024-04-25T10:08:48.898751+0000","last_fullsized":"2024-04-25T10:08:48.898751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932807+0000","last_change":"2024-04-25T10:08:48.930283+0000","last_active":"2024-04-25T10:08:48.932807+0000","last_peered":"2024-04-25T10:08:48.932807+0000","last_clean":"2024-04-25T10:08:48.932807+0000","last_became_active":"2024-04-25T10:08:48.929026+0000","last_became_peered":"2024-04-25T10:08:48.929026+0000","last_unstale":"2024-04-25T10:08:48.932807+0000","last_undegraded":"2024-04-25T10:08:48.932807+0000","last_fullsized":"2024-04-25T10:08:48.932807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:30.464303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902636+0000","last_change":"2024-04-25T10:08:48.902636+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902636+0000","last_undegraded":"2024-04-25T10:08:48.902636+0000","last_fullsized":"2024-04-25T10:08:48.902636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927628+0000","last_change":"2024-04-25T10:08:48.926872+0000","last_active":"2024-04-25T10:08:48.927628+0000","last_peered":"2024-04-25T10:08:48.927628+0000","last_clean":"2024-04-25T10:08:48.927628+0000","last_became_active":"2024-04-25T10:08:48.926660+0000","last_became_peered":"2024-04-25T10:08:48.926660+0000","last_unstale":"2024-04-25T10:08:48.927628+0000","last_undegraded":"2024-04-25T10:08:48.927628+0000","last_fullsized":"2024-04-25T10:08:48.927628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.544601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896769+0000","last_change":"2024-04-25T10:08:48.896769+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896769+0000","last_undegraded":"2024-04-25T10:08:48.896769+0000","last_fullsized":"2024-04-25T10:08:48.896769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923043+0000","last_change":"2024-04-25T10:08:48.916082+0000","last_active":"2024-04-25T10:08:48.923043+0000","last_peered":"2024-04-25T10:08:48.923043+0000","last_clean":"2024-04-25T10:08:48.923043+0000","last_became_active":"2024-04-25T10:08:48.915311+0000","last_became_peered":"2024-04-25T10:08:48.915311+0000","last_unstale":"2024-04-25T10:08:48.923043+0000","last_undegraded":"2024-04-25T10:08:48.923043+0000","last_fullsized":"2024-04-25T10:08:48.923043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.892575+0000","last_change":"2024-04-25T10:08:48.892575+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.892575+0000","last_undegraded":"2024-04-25T10:08:48.892575+0000","last_fullsized":"2024-04-25T10:08:48.892575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927634+0000","last_change":"2024-04-25T10:08:48.927015+0000","last_active":"2024-04-25T10:08:48.927634+0000","last_peered":"2024-04-25T10:08:48.927634+0000","last_clean":"2024-04-25T10:08:48.927634+0000","last_became_active":"2024-04-25T10:08:48.926431+0000","last_became_peered":"2024-04-25T10:08:48.926431+0000","last_unstale":"2024-04-25T10:08:48.927634+0000","last_undegraded":"2024-04-25T10:08:48.927634+0000","last_fullsized":"2024-04-25T10:08:48.927634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:25.811654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900421+0000","last_change":"2024-04-25T10:08:48.900421+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900421+0000","last_undegraded":"2024-04-25T10:08:48.900421+0000","last_fullsized":"2024-04-25T10:08:48.900421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925675+0000","last_change":"2024-04-25T10:08:48.920708+0000","last_active":"2024-04-25T10:08:48.925675+0000","last_peered":"2024-04-25T10:08:48.925675+0000","last_clean":"2024-04-25T10:08:48.925675+0000","last_became_active":"2024-04-25T10:08:48.920393+0000","last_became_peered":"2024-04-25T10:08:48.920393+0000","last_unstale":"2024-04-25T10:08:48.925675+0000","last_undegraded":"2024-04-25T10:08:48.925675+0000","last_fullsized":"2024-04-25T10:08:48.925675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:14.265209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896676+0000","last_change":"2024-04-25T10:08:48.896676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896676+0000","last_undegraded":"2024-04-25T10:08:48.896676+0000","last_fullsized":"2024-04-25T10:08:48.896676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.918051+0000","last_change":"2024-04-25T10:08:48.912297+0000","last_active":"2024-04-25T10:08:48.918051+0000","last_peered":"2024-04-25T10:08:48.918051+0000","last_clean":"2024-04-25T10:08:48.918051+0000","last_became_active":"2024-04-25T10:08:48.911555+0000","last_became_peered":"2024-04-25T10:08:48.911555+0000","last_unstale":"2024-04-25T10:08:48.918051+0000","last_undegraded":"2024-04-25T10:08:48.918051+0000","last_fullsized":"2024-04-25T10:08:48.918051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.083371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895443+0000","last_change":"2024-04-25T10:08:48.895443+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895443+0000","last_undegraded":"2024-04-25T10:08:48.895443+0000","last_fullsized":"2024-04-25T10:08:48.895443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.928484+0000","last_change":"2024-04-25T10:08:48.923258+0000","last_active":"2024-04-25T10:08:48.928484+0000","last_peered":"2024-04-25T10:08:48.928484+0000","last_clean":"2024-04-25T10:08:48.928484+0000","last_became_active":"2024-04-25T10:08:48.922652+0000","last_became_peered":"2024-04-25T10:08:48.922652+0000","last_unstale":"2024-04-25T10:08:48.928484+0000","last_undegraded":"2024-04-25T10:08:48.928484+0000","last_fullsized":"2024-04-25T10:08:48.928484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894725+0000","last_change":"2024-04-25T10:08:48.894725+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894725+0000","last_undegraded":"2024-04-25T10:08:48.894725+0000","last_fullsized":"2024-04-25T10:08:48.894725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929141+0000","last_change":"2024-04-25T10:08:48.924659+0000","last_active":"2024-04-25T10:08:48.929141+0000","last_peered":"2024-04-25T10:08:48.929141+0000","last_clean":"2024-04-25T10:08:48.929141+0000","last_became_active":"2024-04-25T10:08:48.923481+0000","last_became_peered":"2024-04-25T10:08:48.923481+0000","last_unstale":"2024-04-25T10:08:48.929141+0000","last_undegraded":"2024-04-25T10:08:48.929141+0000","last_fullsized":"2024-04-25T10:08:48.929141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:56.284437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895174+0000","last_change":"2024-04-25T10:08:48.895174+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895174+0000","last_undegraded":"2024-04-25T10:08:48.895174+0000","last_fullsized":"2024-04-25T10:08:48.895174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925786+0000","last_change":"2024-04-25T10:08:48.921337+0000","last_active":"2024-04-25T10:08:48.925786+0000","last_peered":"2024-04-25T10:08:48.925786+0000","last_clean":"2024-04-25T10:08:48.925786+0000","last_became_active":"2024-04-25T10:08:48.918289+0000","last_became_peered":"2024-04-25T10:08:48.918289+0000","last_unstale":"2024-04-25T10:08:48.925786+0000","last_undegraded":"2024-04-25T10:08:48.925786+0000","last_fullsized":"2024-04-25T10:08:48.925786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.100052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894501+0000","last_change":"2024-04-25T10:08:48.894501+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894501+0000","last_undegraded":"2024-04-25T10:08:48.894501+0000","last_fullsized":"2024-04-25T10:08:48.894501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.911288+0000","last_change":"2024-04-25T10:08:48.906519+0000","last_active":"2024-04-25T10:08:48.911288+0000","last_peered":"2024-04-25T10:08:48.911288+0000","last_clean":"2024-04-25T10:08:48.911288+0000","last_became_active":"2024-04-25T10:08:48.905583+0000","last_became_peered":"2024-04-25T10:08:48.905583+0000","last_unstale":"2024-04-25T10:08:48.911288+0000","last_undegraded":"2024-04-25T10:08:48.911288+0000","last_fullsized":"2024-04-25T10:08:48.911288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:40.496209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900102+0000","last_change":"2024-04-25T10:08:48.900102+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900102+0000","last_undegraded":"2024-04-25T10:08:48.900102+0000","last_fullsized":"2024-04-25T10:08:48.900102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923192+0000","last_change":"2024-04-25T10:08:48.920400+0000","last_active":"2024-04-25T10:08:48.923192+0000","last_peered":"2024-04-25T10:08:48.923192+0000","last_clean":"2024-04-25T10:08:48.923192+0000","last_became_active":"2024-04-25T10:08:48.920006+0000","last_became_peered":"2024-04-25T10:08:48.920006+0000","last_unstale":"2024-04-25T10:08:48.923192+0000","last_undegraded":"2024-04-25T10:08:48.923192+0000","last_fullsized":"2024-04-25T10:08:48.923192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899888+0000","last_change":"2024-04-25T10:08:48.899888+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899888+0000","last_undegraded":"2024-04-25T10:08:48.899888+0000","last_fullsized":"2024-04-25T10:08:48.899888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921292+0000","last_change":"2024-04-25T10:08:48.908793+0000","last_active":"2024-04-25T10:08:48.921292+0000","last_peered":"2024-04-25T10:08:48.921292+0000","last_clean":"2024-04-25T10:08:48.921292+0000","last_became_active":"2024-04-25T10:08:48.908121+0000","last_became_peered":"2024-04-25T10:08:48.908121+0000","last_unstale":"2024-04-25T10:08:48.921292+0000","last_undegraded":"2024-04-25T10:08:48.921292+0000","last_fullsized":"2024-04-25T10:08:48.921292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:54.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894144+0000","last_change":"2024-04-25T10:08:48.894144+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894144+0000","last_undegraded":"2024-04-25T10:08:48.894144+0000","last_fullsized":"2024-04-25T10:08:48.894144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.916943+0000","last_change":"2024-04-25T10:08:48.907802+0000","last_active":"2024-04-25T10:08:48.916943+0000","last_peered":"2024-04-25T10:08:48.916943+0000","last_clean":"2024-04-25T10:08:48.916943+0000","last_became_active":"2024-04-25T10:08:48.907484+0000","last_became_peered":"2024-04-25T10:08:48.907484+0000","last_unstale":"2024-04-25T10:08:48.916943+0000","last_undegraded":"2024-04-25T10:08:48.916943+0000","last_fullsized":"2024-04-25T10:08:48.916943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.725069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901281+0000","last_change":"2024-04-25T10:08:48.901281+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901281+0000","last_undegraded":"2024-04-25T10:08:48.901281+0000","last_fullsized":"2024-04-25T10:08:48.901281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929174+0000","last_change":"2024-04-25T10:08:48.925017+0000","last_active":"2024-04-25T10:08:48.929174+0000","last_peered":"2024-04-25T10:08:48.929174+0000","last_clean":"2024-04-25T10:08:48.929174+0000","last_became_active":"2024-04-25T10:08:48.924231+0000","last_became_peered":"2024-04-25T10:08:48.924231+0000","last_unstale":"2024-04-25T10:08:48.929174+0000","last_undegraded":"2024-04-25T10:08:48.929174+0000","last_fullsized":"2024-04-25T10:08:48.929174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:31.040785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893835+0000","last_change":"2024-04-25T10:08:48.893835+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893835+0000","last_undegraded":"2024-04-25T10:08:48.893835+0000","last_fullsized":"2024-04-25T10:08:48.893835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.931898+0000","last_change":"2024-04-25T10:08:48.927255+0000","last_active":"2024-04-25T10:08:48.931898+0000","last_peered":"2024-04-25T10:08:48.931898+0000","last_clean":"2024-04-25T10:08:48.931898+0000","last_became_active":"2024-04-25T10:08:48.926623+0000","last_became_peered":"2024-04-25T10:08:48.926623+0000","last_unstale":"2024-04-25T10:08:48.931898+0000","last_undegraded":"2024-04-25T10:08:48.931898+0000","last_fullsized":"2024-04-25T10:08:48.931898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.736381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894241+0000","last_change":"2024-04-25T10:08:48.894241+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894241+0000","last_undegraded":"2024-04-25T10:08:48.894241+0000","last_fullsized":"2024-04-25T10:08:48.894241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932774+0000","last_change":"2024-04-25T10:08:48.929895+0000","last_active":"2024-04-25T10:08:48.932774+0000","last_peered":"2024-04-25T10:08:48.932774+0000","last_clean":"2024-04-25T10:08:48.932774+0000","last_became_active":"2024-04-25T10:08:48.928316+0000","last_became_peered":"2024-04-25T10:08:48.928316+0000","last_unstale":"2024-04-25T10:08:48.932774+0000","last_undegraded":"2024-04-25T10:08:48.932774+0000","last_fullsized":"2024-04-25T10:08:48.932774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:38.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902043+0000","last_change":"2024-04-25T10:08:48.902043+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902043+0000","last_undegraded":"2024-04-25T10:08:48.902043+0000","last_fullsized":"2024-04-25T10:08:48.902043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922564+0000","last_change":"2024-04-25T10:08:48.913827+0000","last_active":"2024-04-25T10:08:48.922564+0000","last_peered":"2024-04-25T10:08:48.922564+0000","last_clean":"2024-04-25T10:08:48.922564+0000","last_became_active":"2024-04-25T10:08:48.912878+0000","last_became_peered":"2024-04-25T10:08:48.912878+0000","last_unstale":"2024-04-25T10:08:48.922564+0000","last_undegraded":"2024-04-25T10:08:48.922564+0000","last_fullsized":"2024-04-25T10:08:48.922564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:15.708904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901260+0000","last_change":"2024-04-25T10:08:48.901260+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901260+0000","last_undegraded":"2024-04-25T10:08:48.901260+0000","last_fullsized":"2024-04-25T10:08:48.901260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927592+0000","last_change":"2024-04-25T10:08:48.926824+0000","last_active":"2024-04-25T10:08:48.927592+0000","last_peered":"2024-04-25T10:08:48.927592+0000","last_clean":"2024-04-25T10:08:48.927592+0000","last_became_active":"2024-04-25T10:08:48.926515+0000","last_became_peered":"2024-04-25T10:08:48.926515+0000","last_unstale":"2024-04-25T10:08:48.927592+0000","last_undegraded":"2024-04-25T10:08:48.927592+0000","last_fullsized":"2024-04-25T10:08:48.927592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:21.486349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896505+0000","last_change":"2024-04-25T10:08:48.896505+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896505+0000","last_undegraded":"2024-04-25T10:08:48.896505+0000","last_fullsized":"2024-04-25T10:08:48.896505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921407+0000","last_change":"2024-04-25T10:08:48.912204+0000","last_active":"2024-04-25T10:08:48.921407+0000","last_peered":"2024-04-25T10:08:48.921407+0000","last_clean":"2024-04-25T10:08:48.921407+0000","last_became_active":"2024-04-25T10:08:48.911509+0000","last_became_peered":"2024-04-25T10:08:48.911509+0000","last_unstale":"2024-04-25T10:08:48.921407+0000","last_undegraded":"2024-04-25T10:08:48.921407+0000","last_fullsized":"2024-04-25T10:08:48.921407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.689628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893499+0000","last_change":"2024-04-25T10:08:48.893499+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893499+0000","last_undegraded":"2024-04-25T10:08:48.893499+0000","last_fullsized":"2024-04-25T10:08:48.893499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921699+0000","last_change":"2024-04-25T10:08:48.912652+0000","last_active":"2024-04-25T10:08:48.921699+0000","last_peered":"2024-04-25T10:08:48.921699+0000","last_clean":"2024-04-25T10:08:48.921699+0000","last_became_active":"2024-04-25T10:08:48.911838+0000","last_became_peered":"2024-04-25T10:08:48.911838+0000","last_unstale":"2024-04-25T10:08:48.921699+0000","last_undegraded":"2024-04-25T10:08:48.921699+0000","last_fullsized":"2024-04-25T10:08:48.921699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:37.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893632+0000","last_change":"2024-04-25T10:08:48.893632+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893632+0000","last_undegraded":"2024-04-25T10:08:48.893632+0000","last_fullsized":"2024-04-25T10:08:48.893632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924165+0000","last_change":"2024-04-25T10:08:48.920751+0000","last_active":"2024-04-25T10:08:48.924165+0000","last_peered":"2024-04-25T10:08:48.924165+0000","last_clean":"2024-04-25T10:08:48.924165+0000","last_became_active":"2024-04-25T10:08:48.920001+0000","last_became_peered":"2024-04-25T10:08:48.920001+0000","last_unstale":"2024-04-25T10:08:48.924165+0000","last_undegraded":"2024-04-25T10:08:48.924165+0000","last_fullsized":"2024-04-25T10:08:48.924165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902674+0000","last_change":"2024-04-25T10:08:48.902674+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902674+0000","last_undegraded":"2024-04-25T10:08:48.902674+0000","last_fullsized":"2024-04-25T10:08:48.902674+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927324+0000","last_change":"2024-04-25T10:08:48.924246+0000","last_active":"2024-04-25T10:08:48.927324+0000","last_peered":"2024-04-25T10:08:48.927324+0000","last_clean":"2024-04-25T10:08:48.927324+0000","last_became_active":"2024-04-25T10:08:48.923370+0000","last_became_peered":"2024-04-25T10:08:48.923370+0000","last_unstale":"2024-04-25T10:08:48.927324+0000","last_undegraded":"2024-04-25T10:08:48.927324+0000","last_fullsized":"2024-04-25T10:08:48.927324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:01.982597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.905354+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:08:48.905354+0000","last_peered":"2024-04-25T10:08:48.905354+0000","last_clean":"2024-04-25T10:08:48.905354+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:08:48.905354+0000","last_undegraded":"2024-04-25T10:08:48.905354+0000","last_fullsized":"2024-04-25T10:08:48.905354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:53.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-25T10:08:53.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897534+0000', 'last_change': '2024-04-25T10:08:48.897534+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897534+0000', 'last_undegraded': '2024-04-25T10:08:48.897534+0000', 'last_fullsized': '2024-04-25T10:08:48.897534+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900991+0000', 'last_change': '2024-04-25T10:08:48.900991+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900991+0000', 'last_undegraded': '2024-04-25T10:08:48.900991+0000', 'last_fullsized': '2024-04-25T10:08:48.900991+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897938+0000', 'last_change': '2024-04-25T10:08:48.897938+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897938+0000', 'last_undegraded': '2024-04-25T10:08:48.897938+0000', 'last_fullsized': '2024-04-25T10:08:48.897938+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.909082+0000', 'last_change': '2024-04-25T10:08:48.909082+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.909082+0000', 'last_undegraded': '2024-04-25T10:08:48.909082+0000', 'last_fullsized': '2024-04-25T10:08:48.909082+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894412+0000', 'last_change': '2024-04-25T10:08:48.894412+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894412+0000', 'last_undegraded': '2024-04-25T10:08:48.894412+0000', 'last_fullsized': '2024-04-25T10:08:48.894412+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T10:08:53.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896657+0000', 'last_change': '2024-04-25T10:08:48.896657+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896657+0000', 'last_undegraded': '2024-04-25T10:08:48.896657+0000', 'last_fullsized': '2024-04-25T10:08:48.896657+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900586+0000', 'last_change': '2024-04-25T10:08:48.900586+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900586+0000', 'last_undegraded': '2024-04-25T10:08:48.900586+0000', 'last_fullsized': '2024-04-25T10:08:48.900586+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.899208+0000', 'last_change': '2024-04-25T10:08:48.899208+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.899208+0000', 'last_undegraded': '2024-04-25T10:08:48.899208+0000', 'last_fullsized': '2024-04-25T10:08:48.899208+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897320+0000', 'last_change': '2024-04-25T10:08:48.897320+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897320+0000', 'last_undegraded': '2024-04-25T10:08:48.897320+0000', 'last_fullsized': '2024-04-25T10:08:48.897320+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.898252+0000', 'last_change': '2024-04-25T10:08:48.898252+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.898252+0000', 'last_undegraded': '2024-04-25T10:08:48.898252+0000', 'last_fullsized': '2024-04-25T10:08:48.898252+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.912082+0000', 'last_change': '2024-04-25T10:08:48.912082+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.912082+0000', 'last_undegraded': '2024-04-25T10:08:48.912082+0000', 'last_fullsized': '2024-04-25T10:08:48.912082+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895959+0000', 'last_change': '2024-04-25T10:08:48.895959+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895959+0000', 'last_undegraded': '2024-04-25T10:08:48.895959+0000', 'last_fullsized': '2024-04-25T10:08:48.895959+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.903583+0000', 'last_change': '2024-04-25T10:08:48.903583+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.903583+0000', 'last_undegraded': '2024-04-25T10:08:48.903583+0000', 'last_fullsized': '2024-04-25T10:08:48.903583+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896282+0000', 'last_change': '2024-04-25T10:08:48.896282+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896282+0000', 'last_undegraded': '2024-04-25T10:08:48.896282+0000', 'last_fullsized': '2024-04-25T10:08:48.896282+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902220+0000', 'last_change': '2024-04-25T10:08:48.902220+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902220+0000', 'last_undegraded': '2024-04-25T10:08:48.902220+0000', 'last_fullsized': '2024-04-25T10:08:48.902220+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897711+0000', 'last_change': '2024-04-25T10:08:48.897711+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897711+0000', 'last_undegraded': '2024-04-25T10:08:48.897711+0000', 'last_fullsized': '2024-04-25T10:08:48.897711+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896879+0000', 'last_change': '2024-04-25T10:08:48.896879+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896879+0000', 'last_undegraded': '2024-04-25T10:08:48.896879+0000', 'last_fullsized': '2024-04-25T10:08:48.896879+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.899121+0000', 'last_change': '2024-04-25T10:08:48.899121+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.899121+0000', 'last_undegraded': '2024-04-25T10:08:48.899121+0000', 'last_fullsized': '2024-04-25T10:08:48.899121+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.899094+0000', 'last_change': '2024-04-25T10:08:48.899094+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.899094+0000', 'last_undegraded': '2024-04-25T10:08:48.899094+0000', 'last_fullsized': '2024-04-25T10:08:48.899094+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-25T10:08:53.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.908676+0000', 'last_change': '2024-04-25T10:08:48.908676+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.908676+0000', 'last_undegraded': '2024-04-25T10:08:48.908676+0000', 'last_fullsized': '2024-04-25T10:08:48.908676+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895275+0000', 'last_change': '2024-04-25T10:08:48.895275+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895275+0000', 'last_undegraded': '2024-04-25T10:08:48.895275+0000', 'last_fullsized': '2024-04-25T10:08:48.895275+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895212+0000', 'last_change': '2024-04-25T10:08:48.895212+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895212+0000', 'last_undegraded': '2024-04-25T10:08:48.895212+0000', 'last_fullsized': '2024-04-25T10:08:48.895212+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895119+0000', 'last_change': '2024-04-25T10:08:48.895119+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895119+0000', 'last_undegraded': '2024-04-25T10:08:48.895119+0000', 'last_fullsized': '2024-04-25T10:08:48.895119+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897114+0000', 'last_change': '2024-04-25T10:08:48.897114+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897114+0000', 'last_undegraded': '2024-04-25T10:08:48.897114+0000', 'last_fullsized': '2024-04-25T10:08:48.897114+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902873+0000', 'last_change': '2024-04-25T10:08:48.902873+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902873+0000', 'last_undegraded': '2024-04-25T10:08:48.902873+0000', 'last_fullsized': '2024-04-25T10:08:48.902873+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896098+0000', 'last_change': '2024-04-25T10:08:48.896098+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896098+0000', 'last_undegraded': '2024-04-25T10:08:48.896098+0000', 'last_fullsized': '2024-04-25T10:08:48.896098+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897466+0000', 'last_change': '2024-04-25T10:08:48.897466+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897466+0000', 'last_undegraded': '2024-04-25T10:08:48.897466+0000', 'last_fullsized': '2024-04-25T10:08:48.897466+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902313+0000', 'last_change': '2024-04-25T10:08:48.902313+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902313+0000', 'last_undegraded': '2024-04-25T10:08:48.902313+0000', 'last_fullsized': '2024-04-25T10:08:48.902313+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900867+0000', 'last_change': '2024-04-25T10:08:48.900867+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900867+0000', 'last_undegraded': '2024-04-25T10:08:48.900867+0000', 'last_fullsized': '2024-04-25T10:08:48.900867+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894114+0000', 'last_change': '2024-04-25T10:08:48.894114+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894114+0000', 'last_undegraded': '2024-04-25T10:08:48.894114+0000', 'last_fullsized': '2024-04-25T10:08:48.894114+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T10:08:53.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900938+0000', 'last_change': '2024-04-25T10:08:48.900938+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900938+0000', 'last_undegraded': '2024-04-25T10:08:48.900938+0000', 'last_fullsized': '2024-04-25T10:08:48.900938+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.897207+0000', 'last_change': '2024-04-25T10:08:48.897207+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.897207+0000', 'last_undegraded': '2024-04-25T10:08:48.897207+0000', 'last_fullsized': '2024-04-25T10:08:48.897207+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895876+0000', 'last_change': '2024-04-25T10:08:48.895876+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895876+0000', 'last_undegraded': '2024-04-25T10:08:48.895876+0000', 'last_fullsized': '2024-04-25T10:08:48.895876+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895907+0000', 'last_change': '2024-04-25T10:08:48.895907+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895907+0000', 'last_undegraded': '2024-04-25T10:08:48.895907+0000', 'last_fullsized': '2024-04-25T10:08:48.895907+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.901697+0000', 'last_change': '2024-04-25T10:08:48.901697+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.901697+0000', 'last_undegraded': '2024-04-25T10:08:48.901697+0000', 'last_fullsized': '2024-04-25T10:08:48.901697+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896461+0000', 'last_change': '2024-04-25T10:08:48.896461+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896461+0000', 'last_undegraded': '2024-04-25T10:08:48.896461+0000', 'last_fullsized': '2024-04-25T10:08:48.896461+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.898138+0000', 'last_change': '2024-04-25T10:08:48.898138+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.898138+0000', 'last_undegraded': '2024-04-25T10:08:48.898138+0000', 'last_fullsized': '2024-04-25T10:08:48.898138+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.893958+0000', 'last_change': '2024-04-25T10:08:48.893958+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.893958+0000', 'last_undegraded': '2024-04-25T10:08:48.893958+0000', 'last_fullsized': '2024-04-25T10:08:48.893958+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896117+0000', 'last_change': '2024-04-25T10:08:48.896117+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896117+0000', 'last_undegraded': '2024-04-25T10:08:48.896117+0000', 'last_fullsized': '2024-04-25T10:08:48.896117+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.903018+0000', 'last_change': '2024-04-25T10:08:48.903018+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.903018+0000', 'last_undegraded': '2024-04-25T10:08:48.903018+0000', 'last_fullsized': '2024-04-25T10:08:48.903018+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894554+0000', 'last_change': '2024-04-25T10:08:48.894554+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894554+0000', 'last_undegraded': '2024-04-25T10:08:48.894554+0000', 'last_fullsized': '2024-04-25T10:08:48.894554+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T10:08:53.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.901140+0000', 'last_change': '2024-04-25T10:08:48.901140+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.901140+0000', 'last_undegraded': '2024-04-25T10:08:48.901140+0000', 'last_fullsized': '2024-04-25T10:08:48.901140+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.898751+0000', 'last_change': '2024-04-25T10:08:48.898751+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.898751+0000', 'last_undegraded': '2024-04-25T10:08:48.898751+0000', 'last_fullsized': '2024-04-25T10:08:48.898751+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902636+0000', 'last_change': '2024-04-25T10:08:48.902636+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902636+0000', 'last_undegraded': '2024-04-25T10:08:48.902636+0000', 'last_fullsized': '2024-04-25T10:08:48.902636+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896769+0000', 'last_change': '2024-04-25T10:08:48.896769+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896769+0000', 'last_undegraded': '2024-04-25T10:08:48.896769+0000', 'last_fullsized': '2024-04-25T10:08:48.896769+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.892575+0000', 'last_change': '2024-04-25T10:08:48.892575+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.892575+0000', 'last_undegraded': '2024-04-25T10:08:48.892575+0000', 'last_fullsized': '2024-04-25T10:08:48.892575+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900421+0000', 'last_change': '2024-04-25T10:08:48.900421+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900421+0000', 'last_undegraded': '2024-04-25T10:08:48.900421+0000', 'last_fullsized': '2024-04-25T10:08:48.900421+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T10:08:53.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896676+0000', 'last_change': '2024-04-25T10:08:48.896676+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896676+0000', 'last_undegraded': '2024-04-25T10:08:48.896676+0000', 'last_fullsized': '2024-04-25T10:08:48.896676+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895443+0000', 'last_change': '2024-04-25T10:08:48.895443+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895443+0000', 'last_undegraded': '2024-04-25T10:08:48.895443+0000', 'last_fullsized': '2024-04-25T10:08:48.895443+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894725+0000', 'last_change': '2024-04-25T10:08:48.894725+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894725+0000', 'last_undegraded': '2024-04-25T10:08:48.894725+0000', 'last_fullsized': '2024-04-25T10:08:48.894725+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.895174+0000', 'last_change': '2024-04-25T10:08:48.895174+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.895174+0000', 'last_undegraded': '2024-04-25T10:08:48.895174+0000', 'last_fullsized': '2024-04-25T10:08:48.895174+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894501+0000', 'last_change': '2024-04-25T10:08:48.894501+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894501+0000', 'last_undegraded': '2024-04-25T10:08:48.894501+0000', 'last_fullsized': '2024-04-25T10:08:48.894501+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T10:08:53.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.900102+0000', 'last_change': '2024-04-25T10:08:48.900102+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.900102+0000', 'last_undegraded': '2024-04-25T10:08:48.900102+0000', 'last_fullsized': '2024-04-25T10:08:48.900102+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.899888+0000', 'last_change': '2024-04-25T10:08:48.899888+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.899888+0000', 'last_undegraded': '2024-04-25T10:08:48.899888+0000', 'last_fullsized': '2024-04-25T10:08:48.899888+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894144+0000', 'last_change': '2024-04-25T10:08:48.894144+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894144+0000', 'last_undegraded': '2024-04-25T10:08:48.894144+0000', 'last_fullsized': '2024-04-25T10:08:48.894144+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.901281+0000', 'last_change': '2024-04-25T10:08:48.901281+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.901281+0000', 'last_undegraded': '2024-04-25T10:08:48.901281+0000', 'last_fullsized': '2024-04-25T10:08:48.901281+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.893835+0000', 'last_change': '2024-04-25T10:08:48.893835+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.893835+0000', 'last_undegraded': '2024-04-25T10:08:48.893835+0000', 'last_fullsized': '2024-04-25T10:08:48.893835+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T10:08:53.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.894241+0000', 'last_change': '2024-04-25T10:08:48.894241+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.894241+0000', 'last_undegraded': '2024-04-25T10:08:48.894241+0000', 'last_fullsized': '2024-04-25T10:08:48.894241+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902043+0000', 'last_change': '2024-04-25T10:08:48.902043+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902043+0000', 'last_undegraded': '2024-04-25T10:08:48.902043+0000', 'last_fullsized': '2024-04-25T10:08:48.902043+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.901260+0000', 'last_change': '2024-04-25T10:08:48.901260+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.901260+0000', 'last_undegraded': '2024-04-25T10:08:48.901260+0000', 'last_fullsized': '2024-04-25T10:08:48.901260+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.896505+0000', 'last_change': '2024-04-25T10:08:48.896505+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.896505+0000', 'last_undegraded': '2024-04-25T10:08:48.896505+0000', 'last_fullsized': '2024-04-25T10:08:48.896505+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.893499+0000', 'last_change': '2024-04-25T10:08:48.893499+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.893499+0000', 'last_undegraded': '2024-04-25T10:08:48.893499+0000', 'last_fullsized': '2024-04-25T10:08:48.893499+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.893632+0000', 'last_change': '2024-04-25T10:08:48.893632+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.893632+0000', 'last_undegraded': '2024-04-25T10:08:48.893632+0000', 'last_fullsized': '2024-04-25T10:08:48.893632+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T10:08:53.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 89, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:08:48.902674+0000', 'last_change': '2024-04-25T10:08:48.902674+0000', 'last_active': '2024-04-25T10:08:48.875175+0000', 'last_peered': '2024-04-25T10:08:48.875175+0000', 'last_clean': '2024-04-25T10:08:48.875175+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:08:48.902674+0000', 'last_undegraded': '2024-04-25T10:08:48.902674+0000', 'last_fullsized': '2024-04-25T10:08:48.902674+0000', 'mapping_epoch': 89, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 89, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'last_clean_scrub_stamp': '2024-04-25T10:08:48.875175+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:08:53.653 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:08:53.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 Processor -- start 2024-04-25T10:08:53.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 -- start start 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffbc80cf560 0x7ffbc80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbc8070fa0 0x7ffbc80cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc815bbb0 con 0x7ffbc806e680 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc80ace70 con 0x7ffbc8070fa0 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbccb97640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc80cfe80 con 0x7ffbc80cf560 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbc8070fa0 0x7ffbc80cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbc8070fa0 0x7ffbc80cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49772/0 (socket says 172.21.15.82:49772) 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 -- 172.21.15.82:0/3791406781 learned_addr learned my addr 172.21.15.82:0/3791406781 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc659c640 1 --2- 172.21.15.82:0/3791406781 >> v2:172.21.15.196:3300/0 conn(0x7ffbc80cf560 0x7ffbc80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 -- 172.21.15.82:0/3791406781 >> v2:172.21.15.196:3300/0 conn(0x7ffbc80cf560 msgr2=0x7ffbc80b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 --2- 172.21.15.82:0/3791406781 >> v2:172.21.15.196:3300/0 conn(0x7ffbc80cf560 0x7ffbc80b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 -- 172.21.15.82:0/3791406781 >> v1:172.21.15.196:6789/0 conn(0x7ffbc806e680 legacy=0x7ffbc80ad300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:53.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.792+0000 7ffbc5d9b640 1 -- 172.21.15.82:0/3791406781 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc80ac280 con 0x7ffbc8070fa0 2024-04-25T10:08:53.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbc5d9b640 1 --2- 172.21.15.82:0/3791406781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80cf940 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80cf230 tx=0x7ffbbc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eff466776d2a80c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbc559a640 1 -- 172.21.15.82:0/3791406781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0b1070 con 0x7ffbc8070fa0 2024-04-25T10:08:53.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbc559a640 1 -- 172.21.15.82:0/3791406781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc0ac630 con 0x7ffbc8070fa0 2024-04-25T10:08:53.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbc559a640 1 -- 172.21.15.82:0/3791406781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc0a8050 con 0x7ffbc8070fa0 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 -- 172.21.15.82:0/3791406781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc8070fa0 msgr2=0x7ffbc80cf940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 --2- 172.21.15.82:0/3791406781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80cf940 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80cf230 tx=0x7ffbbc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 -- 172.21.15.82:0/3791406781 shutdown_connections 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 --2- 172.21.15.82:0/3791406781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80cf940 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 --2- 172.21.15.82:0/3791406781 >> v2:172.21.15.196:3300/0 conn(0x7ffbc80cf560 0x7ffbc80b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 -- 172.21.15.82:0/3791406781 >> 172.21.15.82:0/3791406781 conn(0x7ffbc806ae60 msgr2=0x7ffbc806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:53.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 -- 172.21.15.82:0/3791406781 shutdown_connections 2024-04-25T10:08:53.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.796+0000 7ffbccb97640 1 -- 172.21.15.82:0/3791406781 wait complete. 2024-04-25T10:08:53.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 Processor -- start 2024-04-25T10:08:53.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- start start 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbc806e680 0x7ffbc815e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8073310 con 0x7ffbc80cf560 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc8073190 con 0x7ffbc8070fa0 2024-04-25T10:08:53.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc806eaa0 con 0x7ffbc806e680 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49786/0 (socket says 172.21.15.82:49786) 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 -- 172.21.15.82:0/660388797 learned_addr learned my addr 172.21.15.82:0/660388797 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc5d9b640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc659c640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbc806e680 0x7ffbc815e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 -- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbc806e680 msgr2=0x7ffbc815e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbc806e680 0x7ffbc815e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 -- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 msgr2=0x7ffbc80b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 -- 172.21.15.82:0/660388797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8098d10 con 0x7ffbc80cf560 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc5d9b640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc6d9d640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b990 tx=0x7ffbc000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0019070 con 0x7ffbc80cf560 2024-04-25T10:08:53.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc000cba0 con 0x7ffbc80cf560 2024-04-25T10:08:53.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc815d2d0 con 0x7ffbc80cf560 2024-04-25T10:08:53.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0014440 con 0x7ffbc80cf560 2024-04-25T10:08:53.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc815d890 con 0x7ffbc80cf560 2024-04-25T10:08:53.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc804efa0 con 0x7ffbc80cf560 2024-04-25T10:08:53.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbc001c070 con 0x7ffbc80cf560 2024-04-25T10:08:53.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 0x7ffba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:53.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffbc0084f60 con 0x7ffbc80cf560 2024-04-25T10:08:53.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.800+0000 7ffbc659c640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 0x7ffba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:53.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.804+0000 7ffbc659c640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 0x7ffba8069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b800 tx=0x7ffbb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:53.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:53.804+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffbc0055de0 con 0x7ffbc80cf560 2024-04-25T10:08:54.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.024+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ffbc806d440 con 0x7ffbc80cf560 2024-04-25T10:08:54.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.024+0000 7ffbb6ffd640 1 -- 172.21.15.82:0/660388797 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1035 (secure 0 0 0) 0x7ffbc00556e0 con 0x7ffbc80cf560 2024-04-25T10:08:54.030 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:54.030 INFO:teuthology.orchestra.run.smithi082.stdout:{"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":431,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039313,"num_in_osds":8,"osd_in_since":1714039304,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265424896,"bytes_avail":772828688384,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":56,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"c","status":"up:active","gid":7794}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T10:08:39.385378+0000","services":{}},"progress_events":{}} 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 msgr2=0x7ffba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 0x7ffba8069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b800 tx=0x7ffbb8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 msgr2=0x7ffbc81581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7ffbc000b990 tx=0x7ffbc000be50 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 shutdown_connections 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffba8066f60 0x7ffba8069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbc80cf560 0x7ffbc81581a0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbc8070fa0 0x7ffbc80b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 --2- 172.21.15.82:0/660388797 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbc806e680 0x7ffbc815e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 >> 172.21.15.82:0/660388797 conn(0x7ffbc806ae60 msgr2=0x7ffbc815ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 shutdown_connections 2024-04-25T10:08:54.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.028+0000 7ffbccb97640 1 -- 172.21.15.82:0/660388797 wait complete. 2024-04-25T10:08:54.048 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:08:54.048 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:54.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.164+0000 7f23b62c3640 1 Processor -- start 2024-04-25T10:08:54.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.164+0000 7f23b62c3640 1 -- start start 2024-04-25T10:08:54.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b62c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b62c3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f23b00b3f50 0x7f23b00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b62c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b00b2090 con 0x7f23b00ce430 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b62c3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b015f200 con 0x7f23b00b3f50 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b62c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00b43e0 con 0x7f23b015a700 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23affff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23af7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f23b00b3f50 0x7f23b00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23affff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:41826/0 (socket says 172.21.15.82:41826) 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23affff640 1 -- 172.21.15.82:0/70877119 learned_addr learned my addr 172.21.15.82:0/70877119 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:54.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23b4860640 1 --1- 172.21.15.82:0/70877119 >> v1:172.21.15.196:6789/0 conn(0x7f23b00ce430 0x7f23b00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:49162/0 (socket says 172.21.15.82:49162) 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23af7fe640 1 -- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 msgr2=0x7f23b015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23af7fe640 1 --2- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23af7fe640 1 -- 172.21.15.82:0/70877119 >> v1:172.21.15.196:6789/0 conn(0x7f23b00ce430 legacy=0x7f23b00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.168+0000 7f23af7fe640 1 -- 172.21.15.82:0/70877119 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00b46c0 con 0x7f23b00b3f50 2024-04-25T10:08:54.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23af7fe640 1 --2- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b00b3f50 0x7f23b00cdef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097c0 tx=0x7f23a401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c540c044a71dfc1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23aeffd640 1 -- 172.21.15.82:0/70877119 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40b0070 con 0x7f23b00b3f50 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23aeffd640 1 -- 172.21.15.82:0/70877119 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a4004230 con 0x7f23b00b3f50 2024-04-25T10:08:54.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23aeffd640 1 -- 172.21.15.82:0/70877119 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4003620 con 0x7f23b00b3f50 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b00b3f50 msgr2=0x7f23b00cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b00b3f50 0x7f23b00cdef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f23a40097c0 tx=0x7f23a401c350 comp rx=0 tx=0).stop 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- 172.21.15.82:0/70877119 shutdown_connections 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b00b3f50 0x7f23b00cdef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- 172.21.15.82:0/70877119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- 172.21.15.82:0/70877119 >> 172.21.15.82:0/70877119 conn(0x7f23b00b11a0 msgr2=0x7f23b00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- 172.21.15.82:0/70877119 shutdown_connections 2024-04-25T10:08:54.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- 172.21.15.82:0/70877119 wait complete. 2024-04-25T10:08:54.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 Processor -- start 2024-04-25T10:08:54.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- start start 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b014f6d0 0x7f23b014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00cd490 con 0x7f23b00ce430 2024-04-25T10:08:54.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00cd310 con 0x7f23b014f6d0 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23b62c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00b2bb0 con 0x7f23b015a700 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.172+0000 7f23affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:49802/0 (socket says 172.21.15.82:49802) 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 -- 172.21.15.82:0/3923538681 learned_addr learned my addr 172.21.15.82:0/3923538681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23b4860640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b014f6d0 0x7f23b014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23af7fe640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 -- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 msgr2=0x7f23b0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 -- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b014f6d0 msgr2=0x7f23b014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b014f6d0 0x7f23b014b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 -- 172.21.15.82:0/3923538681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0166610 con 0x7f23b00ce430 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23af7fe640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23affff640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f23a000e890 tx=0x7f23a000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0017070 con 0x7f23b00ce430 2024-04-25T10:08:54.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a000c940 con 0x7f23b00ce430 2024-04-25T10:08:54.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01668f0 con 0x7f23b00ce430 2024-04-25T10:08:54.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000cd30 con 0x7f23b00ce430 2024-04-25T10:08:54.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b0166eb0 con 0x7f23b00ce430 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b004efa0 con 0x7f23b00ce430 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a001a070 con 0x7f23b00ce430 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f23a0085410 con 0x7f23b00ce430 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23af7fe640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:54.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.176+0000 7f23af7fe640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 0x7f239c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ad000 tx=0x7f23a40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:54.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.180+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23a0055c40 con 0x7f23b00ce430 2024-04-25T10:08:54.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.336+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23b006f260 con 0x7f239c066f60 2024-04-25T10:08:54.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.352+0000 7f23acff9640 1 -- 172.21.15.82:0/3923538681 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246766 (secure 0 0 0) 0x7f23a40f7400 con 0x7f239c066f60 2024-04-25T10:08:54.360 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:54.361 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:08:54.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 msgr2=0x7f239c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 0x7f239c069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f23a40ad000 tx=0x7f23a40a6000 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 msgr2=0x7f23b014f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f23a000e890 tx=0x7f23a000ed50 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 shutdown_connections 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f239c066f60 0x7f239c069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f23b015a700 0x7f23b0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f23b014f6d0 0x7f23b014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 --2- 172.21.15.82:0/3923538681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f23b00ce430 0x7f23b014f190 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 >> 172.21.15.82:0/3923538681 conn(0x7f23b00b11a0 msgr2=0x7f23b006e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 shutdown_connections 2024-04-25T10:08:54.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:54.356+0000 7f23b62c3640 1 -- 172.21.15.82:0/3923538681 wait complete. 2024-04-25T10:08:54.379 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-25T10:08:53.389073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259204,"kb_used_data":9736,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715516,"statfs":{"total":773094113280,"available":772828688384,"internally_reserved":0,"allocated":9969664,"data_stored":5203416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.513569"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897534+0000","last_change":"2024-04-25T10:08:48.897534+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897534+0000","last_undegraded":"2024-04-25T10:08:48.897534+0000","last_fullsized":"2024-04-25T10:08:48.897534+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900991+0000","last_change":"2024-04-25T10:08:48.900991+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900991+0000","last_undegraded":"2024-04-25T10:08:48.900991+0000","last_fullsized":"2024-04-25T10:08:48.900991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897938+0000","last_change":"2024-04-25T10:08:48.897938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897938+0000","last_undegraded":"2024-04-25T10:08:48.897938+0000","last_fullsized":"2024-04-25T10:08:48.897938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.909082+0000","last_change":"2024-04-25T10:08:48.909082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.909082+0000","last_undegraded":"2024-04-25T10:08:48.909082+0000","last_fullsized":"2024-04-25T10:08:48.909082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894412+0000","last_change":"2024-04-25T10:08:48.894412+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894412+0000","last_undegraded":"2024-04-25T10:08:48.894412+0000","last_fullsized":"2024-04-25T10:08:48.894412+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896657+0000","last_change":"2024-04-25T10:08:48.896657+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896657+0000","last_undegraded":"2024-04-25T10:08:48.896657+0000","last_fullsized":"2024-04-25T10:08:48.896657+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900586+0000","last_change":"2024-04-25T10:08:48.900586+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900586+0000","last_undegraded":"2024-04-25T10:08:48.900586+0000","last_fullsized":"2024-04-25T10:08:48.900586+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899208+0000","last_change":"2024-04-25T10:08:48.899208+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899208+0000","last_undegraded":"2024-04-25T10:08:48.899208+0000","last_fullsized":"2024-04-25T10:08:48.899208+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897320+0000","last_change":"2024-04-25T10:08:48.897320+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897320+0000","last_undegraded":"2024-04-25T10:08:48.897320+0000","last_fullsized":"2024-04-25T10:08:48.897320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898252+0000","last_change":"2024-04-25T10:08:48.898252+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898252+0000","last_undegraded":"2024-04-25T10:08:48.898252+0000","last_fullsized":"2024-04-25T10:08:48.898252+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.912082+0000","last_change":"2024-04-25T10:08:48.912082+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.912082+0000","last_undegraded":"2024-04-25T10:08:48.912082+0000","last_fullsized":"2024-04-25T10:08:48.912082+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895959+0000","last_change":"2024-04-25T10:08:48.895959+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895959+0000","last_undegraded":"2024-04-25T10:08:48.895959+0000","last_fullsized":"2024-04-25T10:08:48.895959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903583+0000","last_change":"2024-04-25T10:08:48.903583+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903583+0000","last_undegraded":"2024-04-25T10:08:48.903583+0000","last_fullsized":"2024-04-25T10:08:48.903583+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896282+0000","last_change":"2024-04-25T10:08:48.896282+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896282+0000","last_undegraded":"2024-04-25T10:08:48.896282+0000","last_fullsized":"2024-04-25T10:08:48.896282+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902220+0000","last_change":"2024-04-25T10:08:48.902220+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902220+0000","last_undegraded":"2024-04-25T10:08:48.902220+0000","last_fullsized":"2024-04-25T10:08:48.902220+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897711+0000","last_change":"2024-04-25T10:08:48.897711+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897711+0000","last_undegraded":"2024-04-25T10:08:48.897711+0000","last_fullsized":"2024-04-25T10:08:48.897711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896879+0000","last_change":"2024-04-25T10:08:48.896879+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896879+0000","last_undegraded":"2024-04-25T10:08:48.896879+0000","last_fullsized":"2024-04-25T10:08:48.896879+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899121+0000","last_change":"2024-04-25T10:08:48.899121+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899121+0000","last_undegraded":"2024-04-25T10:08:48.899121+0000","last_fullsized":"2024-04-25T10:08:48.899121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899094+0000","last_change":"2024-04-25T10:08:48.899094+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899094+0000","last_undegraded":"2024-04-25T10:08:48.899094+0000","last_fullsized":"2024-04-25T10:08:48.899094+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.908676+0000","last_change":"2024-04-25T10:08:48.908676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.908676+0000","last_undegraded":"2024-04-25T10:08:48.908676+0000","last_fullsized":"2024-04-25T10:08:48.908676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895275+0000","last_change":"2024-04-25T10:08:48.895275+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895275+0000","last_undegraded":"2024-04-25T10:08:48.895275+0000","last_fullsized":"2024-04-25T10:08:48.895275+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895212+0000","last_change":"2024-04-25T10:08:48.895212+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895212+0000","last_undegraded":"2024-04-25T10:08:48.895212+0000","last_fullsized":"2024-04-25T10:08:48.895212+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895119+0000","last_change":"2024-04-25T10:08:48.895119+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895119+0000","last_undegraded":"2024-04-25T10:08:48.895119+0000","last_fullsized":"2024-04-25T10:08:48.895119+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897114+0000","last_change":"2024-04-25T10:08:48.897114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897114+0000","last_undegraded":"2024-04-25T10:08:48.897114+0000","last_fullsized":"2024-04-25T10:08:48.897114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902873+0000","last_change":"2024-04-25T10:08:48.902873+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902873+0000","last_undegraded":"2024-04-25T10:08:48.902873+0000","last_fullsized":"2024-04-25T10:08:48.902873+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896098+0000","last_change":"2024-04-25T10:08:48.896098+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896098+0000","last_undegraded":"2024-04-25T10:08:48.896098+0000","last_fullsized":"2024-04-25T10:08:48.896098+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897466+0000","last_change":"2024-04-25T10:08:48.897466+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897466+0000","last_undegraded":"2024-04-25T10:08:48.897466+0000","last_fullsized":"2024-04-25T10:08:48.897466+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902313+0000","last_change":"2024-04-25T10:08:48.902313+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902313+0000","last_undegraded":"2024-04-25T10:08:48.902313+0000","last_fullsized":"2024-04-25T10:08:48.902313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900867+0000","last_change":"2024-04-25T10:08:48.900867+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900867+0000","last_undegraded":"2024-04-25T10:08:48.900867+0000","last_fullsized":"2024-04-25T10:08:48.900867+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894114+0000","last_change":"2024-04-25T10:08:48.894114+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894114+0000","last_undegraded":"2024-04-25T10:08:48.894114+0000","last_fullsized":"2024-04-25T10:08:48.894114+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900938+0000","last_change":"2024-04-25T10:08:48.900938+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900938+0000","last_undegraded":"2024-04-25T10:08:48.900938+0000","last_fullsized":"2024-04-25T10:08:48.900938+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.897207+0000","last_change":"2024-04-25T10:08:48.897207+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.897207+0000","last_undegraded":"2024-04-25T10:08:48.897207+0000","last_fullsized":"2024-04-25T10:08:48.897207+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895876+0000","last_change":"2024-04-25T10:08:48.895876+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895876+0000","last_undegraded":"2024-04-25T10:08:48.895876+0000","last_fullsized":"2024-04-25T10:08:48.895876+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922842+0000","last_change":"2024-04-25T10:08:48.913258+0000","last_active":"2024-04-25T10:08:48.922842+0000","last_peered":"2024-04-25T10:08:48.922842+0000","last_clean":"2024-04-25T10:08:48.922842+0000","last_became_active":"2024-04-25T10:08:48.912653+0000","last_became_peered":"2024-04-25T10:08:48.912653+0000","last_unstale":"2024-04-25T10:08:48.922842+0000","last_undegraded":"2024-04-25T10:08:48.922842+0000","last_fullsized":"2024-04-25T10:08:48.922842+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:37.862500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895907+0000","last_change":"2024-04-25T10:08:48.895907+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895907+0000","last_undegraded":"2024-04-25T10:08:48.895907+0000","last_fullsized":"2024-04-25T10:08:48.895907+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925620+0000","last_change":"2024-04-25T10:08:48.920404+0000","last_active":"2024-04-25T10:08:48.925620+0000","last_peered":"2024-04-25T10:08:48.925620+0000","last_clean":"2024-04-25T10:08:48.925620+0000","last_became_active":"2024-04-25T10:08:48.919687+0000","last_became_peered":"2024-04-25T10:08:48.919687+0000","last_unstale":"2024-04-25T10:08:48.925620+0000","last_undegraded":"2024-04-25T10:08:48.925620+0000","last_fullsized":"2024-04-25T10:08:48.925620+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:08.719823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901697+0000","last_change":"2024-04-25T10:08:48.901697+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901697+0000","last_undegraded":"2024-04-25T10:08:48.901697+0000","last_fullsized":"2024-04-25T10:08:48.901697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924120+0000","last_change":"2024-04-25T10:08:48.921772+0000","last_active":"2024-04-25T10:08:48.924120+0000","last_peered":"2024-04-25T10:08:48.924120+0000","last_clean":"2024-04-25T10:08:48.924120+0000","last_became_active":"2024-04-25T10:08:48.921226+0000","last_became_peered":"2024-04-25T10:08:48.921226+0000","last_unstale":"2024-04-25T10:08:48.924120+0000","last_undegraded":"2024-04-25T10:08:48.924120+0000","last_fullsized":"2024-04-25T10:08:48.924120+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:45.783541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896461+0000","last_change":"2024-04-25T10:08:48.896461+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896461+0000","last_undegraded":"2024-04-25T10:08:48.896461+0000","last_fullsized":"2024-04-25T10:08:48.896461+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927024+0000","last_change":"2024-04-25T10:08:48.920430+0000","last_active":"2024-04-25T10:08:48.927024+0000","last_peered":"2024-04-25T10:08:48.927024+0000","last_clean":"2024-04-25T10:08:48.927024+0000","last_became_active":"2024-04-25T10:08:48.919636+0000","last_became_peered":"2024-04-25T10:08:48.919636+0000","last_unstale":"2024-04-25T10:08:48.927024+0000","last_undegraded":"2024-04-25T10:08:48.927024+0000","last_fullsized":"2024-04-25T10:08:48.927024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:35.519198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898138+0000","last_change":"2024-04-25T10:08:48.898138+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898138+0000","last_undegraded":"2024-04-25T10:08:48.898138+0000","last_fullsized":"2024-04-25T10:08:48.898138+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.920870+0000","last_change":"2024-04-25T10:08:48.913325+0000","last_active":"2024-04-25T10:08:48.920870+0000","last_peered":"2024-04-25T10:08:48.920870+0000","last_clean":"2024-04-25T10:08:48.920870+0000","last_became_active":"2024-04-25T10:08:48.912648+0000","last_became_peered":"2024-04-25T10:08:48.912648+0000","last_unstale":"2024-04-25T10:08:48.920870+0000","last_undegraded":"2024-04-25T10:08:48.920870+0000","last_fullsized":"2024-04-25T10:08:48.920870+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:19.606750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893958+0000","last_change":"2024-04-25T10:08:48.893958+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893958+0000","last_undegraded":"2024-04-25T10:08:48.893958+0000","last_fullsized":"2024-04-25T10:08:48.893958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.926830+0000","last_change":"2024-04-25T10:08:48.921408+0000","last_active":"2024-04-25T10:08:48.926830+0000","last_peered":"2024-04-25T10:08:48.926830+0000","last_clean":"2024-04-25T10:08:48.926830+0000","last_became_active":"2024-04-25T10:08:48.921179+0000","last_became_peered":"2024-04-25T10:08:48.921179+0000","last_unstale":"2024-04-25T10:08:48.926830+0000","last_undegraded":"2024-04-25T10:08:48.926830+0000","last_fullsized":"2024-04-25T10:08:48.926830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:09.513075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896117+0000","last_change":"2024-04-25T10:08:48.896117+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896117+0000","last_undegraded":"2024-04-25T10:08:48.896117+0000","last_fullsized":"2024-04-25T10:08:48.896117+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925432+0000","last_change":"2024-04-25T10:08:48.923790+0000","last_active":"2024-04-25T10:08:48.925432+0000","last_peered":"2024-04-25T10:08:48.925432+0000","last_clean":"2024-04-25T10:08:48.925432+0000","last_became_active":"2024-04-25T10:08:48.923543+0000","last_became_peered":"2024-04-25T10:08:48.923543+0000","last_unstale":"2024-04-25T10:08:48.925432+0000","last_undegraded":"2024-04-25T10:08:48.925432+0000","last_fullsized":"2024-04-25T10:08:48.925432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.903018+0000","last_change":"2024-04-25T10:08:48.903018+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.903018+0000","last_undegraded":"2024-04-25T10:08:48.903018+0000","last_fullsized":"2024-04-25T10:08:48.903018+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923152+0000","last_change":"2024-04-25T10:08:48.915851+0000","last_active":"2024-04-25T10:08:48.923152+0000","last_peered":"2024-04-25T10:08:48.923152+0000","last_clean":"2024-04-25T10:08:48.923152+0000","last_became_active":"2024-04-25T10:08:48.915212+0000","last_became_peered":"2024-04-25T10:08:48.915212+0000","last_unstale":"2024-04-25T10:08:48.923152+0000","last_undegraded":"2024-04-25T10:08:48.923152+0000","last_fullsized":"2024-04-25T10:08:48.923152+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.518129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894554+0000","last_change":"2024-04-25T10:08:48.894554+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894554+0000","last_undegraded":"2024-04-25T10:08:48.894554+0000","last_fullsized":"2024-04-25T10:08:48.894554+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919005+0000","last_change":"2024-04-25T10:08:48.908751+0000","last_active":"2024-04-25T10:08:48.919005+0000","last_peered":"2024-04-25T10:08:48.919005+0000","last_clean":"2024-04-25T10:08:48.919005+0000","last_became_active":"2024-04-25T10:08:48.908243+0000","last_became_peered":"2024-04-25T10:08:48.908243+0000","last_unstale":"2024-04-25T10:08:48.919005+0000","last_undegraded":"2024-04-25T10:08:48.919005+0000","last_fullsized":"2024-04-25T10:08:48.919005+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:43.075381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901140+0000","last_change":"2024-04-25T10:08:48.901140+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901140+0000","last_undegraded":"2024-04-25T10:08:48.901140+0000","last_fullsized":"2024-04-25T10:08:48.901140+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.919327+0000","last_change":"2024-04-25T10:08:48.917822+0000","last_active":"2024-04-25T10:08:48.919327+0000","last_peered":"2024-04-25T10:08:48.919327+0000","last_clean":"2024-04-25T10:08:48.919327+0000","last_became_active":"2024-04-25T10:08:48.917540+0000","last_became_peered":"2024-04-25T10:08:48.917540+0000","last_unstale":"2024-04-25T10:08:48.919327+0000","last_undegraded":"2024-04-25T10:08:48.919327+0000","last_fullsized":"2024-04-25T10:08:48.919327+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.898751+0000","last_change":"2024-04-25T10:08:48.898751+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.898751+0000","last_undegraded":"2024-04-25T10:08:48.898751+0000","last_fullsized":"2024-04-25T10:08:48.898751+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932807+0000","last_change":"2024-04-25T10:08:48.930283+0000","last_active":"2024-04-25T10:08:48.932807+0000","last_peered":"2024-04-25T10:08:48.932807+0000","last_clean":"2024-04-25T10:08:48.932807+0000","last_became_active":"2024-04-25T10:08:48.929026+0000","last_became_peered":"2024-04-25T10:08:48.929026+0000","last_unstale":"2024-04-25T10:08:48.932807+0000","last_undegraded":"2024-04-25T10:08:48.932807+0000","last_fullsized":"2024-04-25T10:08:48.932807+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:30.464303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902636+0000","last_change":"2024-04-25T10:08:48.902636+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902636+0000","last_undegraded":"2024-04-25T10:08:48.902636+0000","last_fullsized":"2024-04-25T10:08:48.902636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927628+0000","last_change":"2024-04-25T10:08:48.926872+0000","last_active":"2024-04-25T10:08:48.927628+0000","last_peered":"2024-04-25T10:08:48.927628+0000","last_clean":"2024-04-25T10:08:48.927628+0000","last_became_active":"2024-04-25T10:08:48.926660+0000","last_became_peered":"2024-04-25T10:08:48.926660+0000","last_unstale":"2024-04-25T10:08:48.927628+0000","last_undegraded":"2024-04-25T10:08:48.927628+0000","last_fullsized":"2024-04-25T10:08:48.927628+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.544601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896769+0000","last_change":"2024-04-25T10:08:48.896769+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896769+0000","last_undegraded":"2024-04-25T10:08:48.896769+0000","last_fullsized":"2024-04-25T10:08:48.896769+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923043+0000","last_change":"2024-04-25T10:08:48.916082+0000","last_active":"2024-04-25T10:08:48.923043+0000","last_peered":"2024-04-25T10:08:48.923043+0000","last_clean":"2024-04-25T10:08:48.923043+0000","last_became_active":"2024-04-25T10:08:48.915311+0000","last_became_peered":"2024-04-25T10:08:48.915311+0000","last_unstale":"2024-04-25T10:08:48.923043+0000","last_undegraded":"2024-04-25T10:08:48.923043+0000","last_fullsized":"2024-04-25T10:08:48.923043+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.892575+0000","last_change":"2024-04-25T10:08:48.892575+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.892575+0000","last_undegraded":"2024-04-25T10:08:48.892575+0000","last_fullsized":"2024-04-25T10:08:48.892575+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927634+0000","last_change":"2024-04-25T10:08:48.927015+0000","last_active":"2024-04-25T10:08:48.927634+0000","last_peered":"2024-04-25T10:08:48.927634+0000","last_clean":"2024-04-25T10:08:48.927634+0000","last_became_active":"2024-04-25T10:08:48.926431+0000","last_became_peered":"2024-04-25T10:08:48.926431+0000","last_unstale":"2024-04-25T10:08:48.927634+0000","last_undegraded":"2024-04-25T10:08:48.927634+0000","last_fullsized":"2024-04-25T10:08:48.927634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:25.811654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900421+0000","last_change":"2024-04-25T10:08:48.900421+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900421+0000","last_undegraded":"2024-04-25T10:08:48.900421+0000","last_fullsized":"2024-04-25T10:08:48.900421+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925675+0000","last_change":"2024-04-25T10:08:48.920708+0000","last_active":"2024-04-25T10:08:48.925675+0000","last_peered":"2024-04-25T10:08:48.925675+0000","last_clean":"2024-04-25T10:08:48.925675+0000","last_became_active":"2024-04-25T10:08:48.920393+0000","last_became_peered":"2024-04-25T10:08:48.920393+0000","last_unstale":"2024-04-25T10:08:48.925675+0000","last_undegraded":"2024-04-25T10:08:48.925675+0000","last_fullsized":"2024-04-25T10:08:48.925675+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:14.265209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896676+0000","last_change":"2024-04-25T10:08:48.896676+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896676+0000","last_undegraded":"2024-04-25T10:08:48.896676+0000","last_fullsized":"2024-04-25T10:08:48.896676+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.918051+0000","last_change":"2024-04-25T10:08:48.912297+0000","last_active":"2024-04-25T10:08:48.918051+0000","last_peered":"2024-04-25T10:08:48.918051+0000","last_clean":"2024-04-25T10:08:48.918051+0000","last_became_active":"2024-04-25T10:08:48.911555+0000","last_became_peered":"2024-04-25T10:08:48.911555+0000","last_unstale":"2024-04-25T10:08:48.918051+0000","last_undegraded":"2024-04-25T10:08:48.918051+0000","last_fullsized":"2024-04-25T10:08:48.918051+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.083371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895443+0000","last_change":"2024-04-25T10:08:48.895443+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895443+0000","last_undegraded":"2024-04-25T10:08:48.895443+0000","last_fullsized":"2024-04-25T10:08:48.895443+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.928484+0000","last_change":"2024-04-25T10:08:48.923258+0000","last_active":"2024-04-25T10:08:48.928484+0000","last_peered":"2024-04-25T10:08:48.928484+0000","last_clean":"2024-04-25T10:08:48.928484+0000","last_became_active":"2024-04-25T10:08:48.922652+0000","last_became_peered":"2024-04-25T10:08:48.922652+0000","last_unstale":"2024-04-25T10:08:48.928484+0000","last_undegraded":"2024-04-25T10:08:48.928484+0000","last_fullsized":"2024-04-25T10:08:48.928484+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.636102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894725+0000","last_change":"2024-04-25T10:08:48.894725+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894725+0000","last_undegraded":"2024-04-25T10:08:48.894725+0000","last_fullsized":"2024-04-25T10:08:48.894725+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929141+0000","last_change":"2024-04-25T10:08:48.924659+0000","last_active":"2024-04-25T10:08:48.929141+0000","last_peered":"2024-04-25T10:08:48.929141+0000","last_clean":"2024-04-25T10:08:48.929141+0000","last_became_active":"2024-04-25T10:08:48.923481+0000","last_became_peered":"2024-04-25T10:08:48.923481+0000","last_unstale":"2024-04-25T10:08:48.929141+0000","last_undegraded":"2024-04-25T10:08:48.929141+0000","last_fullsized":"2024-04-25T10:08:48.929141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:56.284437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.895174+0000","last_change":"2024-04-25T10:08:48.895174+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.895174+0000","last_undegraded":"2024-04-25T10:08:48.895174+0000","last_fullsized":"2024-04-25T10:08:48.895174+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.925786+0000","last_change":"2024-04-25T10:08:48.921337+0000","last_active":"2024-04-25T10:08:48.925786+0000","last_peered":"2024-04-25T10:08:48.925786+0000","last_clean":"2024-04-25T10:08:48.925786+0000","last_became_active":"2024-04-25T10:08:48.918289+0000","last_became_peered":"2024-04-25T10:08:48.918289+0000","last_unstale":"2024-04-25T10:08:48.925786+0000","last_undegraded":"2024-04-25T10:08:48.925786+0000","last_fullsized":"2024-04-25T10:08:48.925786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:08.100052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894501+0000","last_change":"2024-04-25T10:08:48.894501+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894501+0000","last_undegraded":"2024-04-25T10:08:48.894501+0000","last_fullsized":"2024-04-25T10:08:48.894501+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.911288+0000","last_change":"2024-04-25T10:08:48.906519+0000","last_active":"2024-04-25T10:08:48.911288+0000","last_peered":"2024-04-25T10:08:48.911288+0000","last_clean":"2024-04-25T10:08:48.911288+0000","last_became_active":"2024-04-25T10:08:48.905583+0000","last_became_peered":"2024-04-25T10:08:48.905583+0000","last_unstale":"2024-04-25T10:08:48.911288+0000","last_undegraded":"2024-04-25T10:08:48.911288+0000","last_fullsized":"2024-04-25T10:08:48.911288+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:40.496209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.900102+0000","last_change":"2024-04-25T10:08:48.900102+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.900102+0000","last_undegraded":"2024-04-25T10:08:48.900102+0000","last_fullsized":"2024-04-25T10:08:48.900102+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.923192+0000","last_change":"2024-04-25T10:08:48.920400+0000","last_active":"2024-04-25T10:08:48.923192+0000","last_peered":"2024-04-25T10:08:48.923192+0000","last_clean":"2024-04-25T10:08:48.923192+0000","last_became_active":"2024-04-25T10:08:48.920006+0000","last_became_peered":"2024-04-25T10:08:48.920006+0000","last_unstale":"2024-04-25T10:08:48.923192+0000","last_undegraded":"2024-04-25T10:08:48.923192+0000","last_fullsized":"2024-04-25T10:08:48.923192+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:41.413680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.899888+0000","last_change":"2024-04-25T10:08:48.899888+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.899888+0000","last_undegraded":"2024-04-25T10:08:48.899888+0000","last_fullsized":"2024-04-25T10:08:48.899888+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921292+0000","last_change":"2024-04-25T10:08:48.908793+0000","last_active":"2024-04-25T10:08:48.921292+0000","last_peered":"2024-04-25T10:08:48.921292+0000","last_clean":"2024-04-25T10:08:48.921292+0000","last_became_active":"2024-04-25T10:08:48.908121+0000","last_became_peered":"2024-04-25T10:08:48.908121+0000","last_unstale":"2024-04-25T10:08:48.921292+0000","last_undegraded":"2024-04-25T10:08:48.921292+0000","last_fullsized":"2024-04-25T10:08:48.921292+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:54.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894144+0000","last_change":"2024-04-25T10:08:48.894144+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894144+0000","last_undegraded":"2024-04-25T10:08:48.894144+0000","last_fullsized":"2024-04-25T10:08:48.894144+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.916943+0000","last_change":"2024-04-25T10:08:48.907802+0000","last_active":"2024-04-25T10:08:48.916943+0000","last_peered":"2024-04-25T10:08:48.916943+0000","last_clean":"2024-04-25T10:08:48.916943+0000","last_became_active":"2024-04-25T10:08:48.907484+0000","last_became_peered":"2024-04-25T10:08:48.907484+0000","last_unstale":"2024-04-25T10:08:48.916943+0000","last_undegraded":"2024-04-25T10:08:48.916943+0000","last_fullsized":"2024-04-25T10:08:48.916943+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:13.725069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901281+0000","last_change":"2024-04-25T10:08:48.901281+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901281+0000","last_undegraded":"2024-04-25T10:08:48.901281+0000","last_fullsized":"2024-04-25T10:08:48.901281+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.929174+0000","last_change":"2024-04-25T10:08:48.925017+0000","last_active":"2024-04-25T10:08:48.929174+0000","last_peered":"2024-04-25T10:08:48.929174+0000","last_clean":"2024-04-25T10:08:48.929174+0000","last_became_active":"2024-04-25T10:08:48.924231+0000","last_became_peered":"2024-04-25T10:08:48.924231+0000","last_unstale":"2024-04-25T10:08:48.929174+0000","last_undegraded":"2024-04-25T10:08:48.929174+0000","last_fullsized":"2024-04-25T10:08:48.929174+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:31.040785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893835+0000","last_change":"2024-04-25T10:08:48.893835+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893835+0000","last_undegraded":"2024-04-25T10:08:48.893835+0000","last_fullsized":"2024-04-25T10:08:48.893835+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.931898+0000","last_change":"2024-04-25T10:08:48.927255+0000","last_active":"2024-04-25T10:08:48.931898+0000","last_peered":"2024-04-25T10:08:48.931898+0000","last_clean":"2024-04-25T10:08:48.931898+0000","last_became_active":"2024-04-25T10:08:48.926623+0000","last_became_peered":"2024-04-25T10:08:48.926623+0000","last_unstale":"2024-04-25T10:08:48.931898+0000","last_undegraded":"2024-04-25T10:08:48.931898+0000","last_fullsized":"2024-04-25T10:08:48.931898+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.736381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.894241+0000","last_change":"2024-04-25T10:08:48.894241+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.894241+0000","last_undegraded":"2024-04-25T10:08:48.894241+0000","last_fullsized":"2024-04-25T10:08:48.894241+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.932774+0000","last_change":"2024-04-25T10:08:48.929895+0000","last_active":"2024-04-25T10:08:48.932774+0000","last_peered":"2024-04-25T10:08:48.932774+0000","last_clean":"2024-04-25T10:08:48.932774+0000","last_became_active":"2024-04-25T10:08:48.928316+0000","last_became_peered":"2024-04-25T10:08:48.928316+0000","last_unstale":"2024-04-25T10:08:48.932774+0000","last_undegraded":"2024-04-25T10:08:48.932774+0000","last_fullsized":"2024-04-25T10:08:48.932774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:38.056270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902043+0000","last_change":"2024-04-25T10:08:48.902043+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902043+0000","last_undegraded":"2024-04-25T10:08:48.902043+0000","last_fullsized":"2024-04-25T10:08:48.902043+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.922564+0000","last_change":"2024-04-25T10:08:48.913827+0000","last_active":"2024-04-25T10:08:48.922564+0000","last_peered":"2024-04-25T10:08:48.922564+0000","last_clean":"2024-04-25T10:08:48.922564+0000","last_became_active":"2024-04-25T10:08:48.912878+0000","last_became_peered":"2024-04-25T10:08:48.912878+0000","last_unstale":"2024-04-25T10:08:48.922564+0000","last_undegraded":"2024-04-25T10:08:48.922564+0000","last_fullsized":"2024-04-25T10:08:48.922564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:15.708904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.901260+0000","last_change":"2024-04-25T10:08:48.901260+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.901260+0000","last_undegraded":"2024-04-25T10:08:48.901260+0000","last_fullsized":"2024-04-25T10:08:48.901260+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927592+0000","last_change":"2024-04-25T10:08:48.926824+0000","last_active":"2024-04-25T10:08:48.927592+0000","last_peered":"2024-04-25T10:08:48.927592+0000","last_clean":"2024-04-25T10:08:48.927592+0000","last_became_active":"2024-04-25T10:08:48.926515+0000","last_became_peered":"2024-04-25T10:08:48.926515+0000","last_unstale":"2024-04-25T10:08:48.927592+0000","last_undegraded":"2024-04-25T10:08:48.927592+0000","last_fullsized":"2024-04-25T10:08:48.927592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:21.486349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.896505+0000","last_change":"2024-04-25T10:08:48.896505+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.896505+0000","last_undegraded":"2024-04-25T10:08:48.896505+0000","last_fullsized":"2024-04-25T10:08:48.896505+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921407+0000","last_change":"2024-04-25T10:08:48.912204+0000","last_active":"2024-04-25T10:08:48.921407+0000","last_peered":"2024-04-25T10:08:48.921407+0000","last_clean":"2024-04-25T10:08:48.921407+0000","last_became_active":"2024-04-25T10:08:48.911509+0000","last_became_peered":"2024-04-25T10:08:48.911509+0000","last_unstale":"2024-04-25T10:08:48.921407+0000","last_undegraded":"2024-04-25T10:08:48.921407+0000","last_fullsized":"2024-04-25T10:08:48.921407+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:30.689628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893499+0000","last_change":"2024-04-25T10:08:48.893499+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893499+0000","last_undegraded":"2024-04-25T10:08:48.893499+0000","last_fullsized":"2024-04-25T10:08:48.893499+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.921699+0000","last_change":"2024-04-25T10:08:48.912652+0000","last_active":"2024-04-25T10:08:48.921699+0000","last_peered":"2024-04-25T10:08:48.921699+0000","last_clean":"2024-04-25T10:08:48.921699+0000","last_became_active":"2024-04-25T10:08:48.911838+0000","last_became_peered":"2024-04-25T10:08:48.911838+0000","last_unstale":"2024-04-25T10:08:48.921699+0000","last_undegraded":"2024-04-25T10:08:48.921699+0000","last_fullsized":"2024-04-25T10:08:48.921699+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:37.621879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.893632+0000","last_change":"2024-04-25T10:08:48.893632+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.893632+0000","last_undegraded":"2024-04-25T10:08:48.893632+0000","last_fullsized":"2024-04-25T10:08:48.893632+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.924165+0000","last_change":"2024-04-25T10:08:48.920751+0000","last_active":"2024-04-25T10:08:48.924165+0000","last_peered":"2024-04-25T10:08:48.924165+0000","last_clean":"2024-04-25T10:08:48.924165+0000","last_became_active":"2024-04-25T10:08:48.920001+0000","last_became_peered":"2024-04-25T10:08:48.920001+0000","last_unstale":"2024-04-25T10:08:48.924165+0000","last_undegraded":"2024-04-25T10:08:48.924165+0000","last_fullsized":"2024-04-25T10:08:48.924165+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":89,"state":"creating+peering","last_fresh":"2024-04-25T10:08:48.902674+0000","last_change":"2024-04-25T10:08:48.902674+0000","last_active":"2024-04-25T10:08:48.875175+0000","last_peered":"2024-04-25T10:08:48.875175+0000","last_clean":"2024-04-25T10:08:48.875175+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:08:48.902674+0000","last_undegraded":"2024-04-25T10:08:48.902674+0000","last_fullsized":"2024-04-25T10:08:48.902674+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.927324+0000","last_change":"2024-04-25T10:08:48.924246+0000","last_active":"2024-04-25T10:08:48.927324+0000","last_peered":"2024-04-25T10:08:48.927324+0000","last_clean":"2024-04-25T10:08:48.927324+0000","last_became_active":"2024-04-25T10:08:48.923370+0000","last_became_peered":"2024-04-25T10:08:48.923370+0000","last_unstale":"2024-04-25T10:08:48.927324+0000","last_undegraded":"2024-04-25T10:08:48.927324+0000","last_fullsized":"2024-04-25T10:08:48.927324+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:01.982597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":298,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-25T10:08:48.905354+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:08:48.905354+0000","last_peered":"2024-04-25T10:08:48.905354+0000","last_clean":"2024-04-25T10:08:48.905354+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:08:48.905354+0000","last_undegraded":"2024-04-25T10:08:48.905354+0000","last_fullsized":"2024-04-25T10:08:48.905354+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574934,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574936,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574934,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574936,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1097728,"data_stored":502835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32832,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339008,"statfs":{"total":96636764160,"available":96603144192,"internally_reserved":0,"allocated":1691648,"data_stored":1093203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:55.873 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:08:55.868+0000 7ff1bef4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:57.385 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:57.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.512+0000 7f26a3dbc640 1 Processor -- start 2024-04-25T10:08:57.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.512+0000 7f26a3dbc640 1 -- start start 2024-04-25T10:08:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a3dbc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f269c15da10 0x7f269c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a3dbc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a3dbc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f269c0adbd0 con 0x7f269c071e90 2024-04-25T10:08:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a3dbc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f269c0ae860 con 0x7f269c15da10 2024-04-25T10:08:57.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a3dbc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c15e210 con 0x7f269c0b24a0 2024-04-25T10:08:57.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1b58640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f269c15da10 0x7f269c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a2359640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f269c071e90 0x7f269c1614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:32816/0 (socket says 172.21.15.82:32816) 2024-04-25T10:08:57.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:32930/0 (socket says 172.21.15.82:32930) 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a2359640 1 -- 172.21.15.82:0/3991581898 learned_addr learned my addr 172.21.15.82:0/3991581898 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a0b56640 1 -- 172.21.15.82:0/3991581898 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795078019 0 0) 0x7f269c0adbd0 con 0x7f269c071e90 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a0b56640 1 -- 172.21.15.82:0/3991581898 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2688003530 con 0x7f269c071e90 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 -- 172.21.15.82:0/3991581898 >> v2:172.21.15.196:3300/0 conn(0x7f269c15da10 msgr2=0x7f269c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 --2- 172.21.15.82:0/3991581898 >> v2:172.21.15.196:3300/0 conn(0x7f269c15da10 0x7f269c071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 -- 172.21.15.82:0/3991581898 >> v1:172.21.15.196:6789/0 conn(0x7f269c071e90 legacy=0x7f269c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.516+0000 7f26a1357640 1 -- 172.21.15.82:0/3991581898 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c15e4f0 con 0x7f269c0b24a0 2024-04-25T10:08:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a1357640 1 --2- 172.21.15.82:0/3991581898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097c0 tx=0x7f268c01c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c0b770df9751b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a0b56640 1 -- 172.21.15.82:0/3991581898 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0b0070 con 0x7f269c0b24a0 2024-04-25T10:08:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a0b56640 1 -- 172.21.15.82:0/3991581898 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268c0ab600 con 0x7f269c0b24a0 2024-04-25T10:08:57.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a0b56640 1 -- 172.21.15.82:0/3991581898 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268c0ab9d0 con 0x7f269c0b24a0 2024-04-25T10:08:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- 172.21.15.82:0/3991581898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 msgr2=0x7f269c071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/3991581898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097c0 tx=0x7f268c01c380 comp rx=0 tx=0).stop 2024-04-25T10:08:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- 172.21.15.82:0/3991581898 shutdown_connections 2024-04-25T10:08:57.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/3991581898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c0b24a0 0x7f269c071570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/3991581898 >> v2:172.21.15.196:3300/0 conn(0x7f269c15da10 0x7f269c071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- 172.21.15.82:0/3991581898 >> 172.21.15.82:0/3991581898 conn(0x7f269c0b11a0 msgr2=0x7f269c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- 172.21.15.82:0/3991581898 shutdown_connections 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- 172.21.15.82:0/3991581898 wait complete. 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 Processor -- start 2024-04-25T10:08:57.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 -- start start 2024-04-25T10:08:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c071e90 0x7f269c1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f269c0b24a0 0x7f269c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.520+0000 7f26a3dbc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c16aa10 con 0x7f269c15da10 2024-04-25T10:08:57.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c16a890 con 0x7f269c0b24a0 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c16ab90 con 0x7f269c071e90 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33430/0 (socket says 172.21.15.82:33430) 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 -- 172.21.15.82:0/2088454412 learned_addr learned my addr 172.21.15.82:0/2088454412 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a1b58640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c071e90 0x7f269c1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a1357640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f269c0b24a0 0x7f269c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 -- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c071e90 msgr2=0x7f269c1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c071e90 0x7f269c1589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 -- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f269c0b24a0 msgr2=0x7f269c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f269c0b24a0 0x7f269c160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 -- 172.21.15.82:0/2088454412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c157c30 con 0x7f269c15da10 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a2359640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f269400cb40 tx=0x7f269400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694017070 con 0x7f269c15da10 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269400aca0 con 0x7f269c15da10 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c157f10 con 0x7f269c15da10 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694012620 con 0x7f269c15da10 2024-04-25T10:08:57.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c1584d0 con 0x7f269c15da10 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c04efa0 con 0x7f269c15da10 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f269401a070 con 0x7f269c15da10 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2694084e90 con 0x7f269c15da10 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a1b58640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.524+0000 7f26a1b58640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 0x7f2680069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ac0 tx=0x7f2698009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.528+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2694055750 con 0x7f269c15da10 2024-04-25T10:08:57.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.684+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f269c168660 con 0x7f2680066f60 2024-04-25T10:08:57.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.700+0000 7f26927fc640 1 -- 172.21.15.82:0/2088454412 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252083 (secure 0 0 0) 0x7f269c168660 con 0x7f2680066f60 2024-04-25T10:08:57.709 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:57.711 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:08:57.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 msgr2=0x7f2680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 0x7f2680069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2698009ac0 tx=0x7f2698009190 comp rx=0 tx=0).stop 2024-04-25T10:08:57.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 msgr2=0x7f269c15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f269400cb40 tx=0x7f269400a060 comp rx=0 tx=0).stop 2024-04-25T10:08:57.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 shutdown_connections 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2680066f60 0x7f2680069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f269c15da10 0x7f269c15f910 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f269c0b24a0 0x7f269c160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 --2- 172.21.15.82:0/2088454412 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f269c071e90 0x7f269c1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 >> 172.21.15.82:0/2088454412 conn(0x7f269c0b11a0 msgr2=0x7f269c0b0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 shutdown_connections 2024-04-25T10:08:57.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.704+0000 7f26a3dbc640 1 -- 172.21.15.82:0/2088454412 wait complete. 2024-04-25T10:08:57.728 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-04-25T10:08:57.390475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259456,"kb_used_data":9992,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715264,"statfs":{"total":773094113280,"available":772828430336,"internally_reserved":0,"allocated":10231808,"data_stored":5319108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002489"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024933+0000","last_change":"2024-04-25T10:08:49.928391+0000","last_active":"2024-04-25T10:08:51.024933+0000","last_peered":"2024-04-25T10:08:51.024933+0000","last_clean":"2024-04-25T10:08:51.024933+0000","last_became_active":"2024-04-25T10:08:49.928131+0000","last_became_peered":"2024-04-25T10:08:49.928131+0000","last_unstale":"2024-04-25T10:08:51.024933+0000","last_undegraded":"2024-04-25T10:08:51.024933+0000","last_fullsized":"2024-04-25T10:08:51.024933+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:00.845737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025386+0000","last_change":"2024-04-25T10:08:49.931688+0000","last_active":"2024-04-25T10:08:51.025386+0000","last_peered":"2024-04-25T10:08:51.025386+0000","last_clean":"2024-04-25T10:08:51.025386+0000","last_became_active":"2024-04-25T10:08:49.930778+0000","last_became_peered":"2024-04-25T10:08:49.930778+0000","last_unstale":"2024-04-25T10:08:51.025386+0000","last_undegraded":"2024-04-25T10:08:51.025386+0000","last_fullsized":"2024-04-25T10:08:51.025386+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:14.046715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024326+0000","last_change":"2024-04-25T10:08:49.934644+0000","last_active":"2024-04-25T10:08:51.024326+0000","last_peered":"2024-04-25T10:08:51.024326+0000","last_clean":"2024-04-25T10:08:51.024326+0000","last_became_active":"2024-04-25T10:08:49.934310+0000","last_became_peered":"2024-04-25T10:08:49.934310+0000","last_unstale":"2024-04-25T10:08:51.024326+0000","last_undegraded":"2024-04-25T10:08:51.024326+0000","last_fullsized":"2024-04-25T10:08:51.024326+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.999703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026008+0000","last_change":"2024-04-25T10:08:49.927564+0000","last_active":"2024-04-25T10:08:51.026008+0000","last_peered":"2024-04-25T10:08:51.026008+0000","last_clean":"2024-04-25T10:08:51.026008+0000","last_became_active":"2024-04-25T10:08:49.927308+0000","last_became_peered":"2024-04-25T10:08:49.927308+0000","last_unstale":"2024-04-25T10:08:51.026008+0000","last_undegraded":"2024-04-25T10:08:51.026008+0000","last_fullsized":"2024-04-25T10:08:51.026008+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:26.689138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897783+0000","last_change":"2024-04-25T10:08:49.933855+0000","last_active":"2024-04-25T10:08:50.897783+0000","last_peered":"2024-04-25T10:08:50.897783+0000","last_clean":"2024-04-25T10:08:50.897783+0000","last_became_active":"2024-04-25T10:08:49.932706+0000","last_became_peered":"2024-04-25T10:08:49.932706+0000","last_unstale":"2024-04-25T10:08:50.897783+0000","last_undegraded":"2024-04-25T10:08:50.897783+0000","last_fullsized":"2024-04-25T10:08:50.897783+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.701952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023616+0000","last_change":"2024-04-25T10:08:49.926752+0000","last_active":"2024-04-25T10:08:51.023616+0000","last_peered":"2024-04-25T10:08:51.023616+0000","last_clean":"2024-04-25T10:08:51.023616+0000","last_became_active":"2024-04-25T10:08:49.924347+0000","last_became_peered":"2024-04-25T10:08:49.924347+0000","last_unstale":"2024-04-25T10:08:51.023616+0000","last_undegraded":"2024-04-25T10:08:51.023616+0000","last_fullsized":"2024-04-25T10:08:51.023616+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:48.404071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024004+0000","last_change":"2024-04-25T10:08:49.929193+0000","last_active":"2024-04-25T10:08:51.024004+0000","last_peered":"2024-04-25T10:08:51.024004+0000","last_clean":"2024-04-25T10:08:51.024004+0000","last_became_active":"2024-04-25T10:08:49.928409+0000","last_became_peered":"2024-04-25T10:08:49.928409+0000","last_unstale":"2024-04-25T10:08:51.024004+0000","last_undegraded":"2024-04-25T10:08:51.024004+0000","last_fullsized":"2024-04-25T10:08:51.024004+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:01.703063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025518+0000","last_change":"2024-04-25T10:08:49.929653+0000","last_active":"2024-04-25T10:08:51.025518+0000","last_peered":"2024-04-25T10:08:51.025518+0000","last_clean":"2024-04-25T10:08:51.025518+0000","last_became_active":"2024-04-25T10:08:49.929378+0000","last_became_peered":"2024-04-25T10:08:49.929378+0000","last_unstale":"2024-04-25T10:08:51.025518+0000","last_undegraded":"2024-04-25T10:08:51.025518+0000","last_fullsized":"2024-04-25T10:08:51.025518+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:28.921193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896365+0000","last_change":"2024-04-25T10:08:49.949453+0000","last_active":"2024-04-25T10:08:50.896365+0000","last_peered":"2024-04-25T10:08:50.896365+0000","last_clean":"2024-04-25T10:08:50.896365+0000","last_became_active":"2024-04-25T10:08:49.948958+0000","last_became_peered":"2024-04-25T10:08:49.948958+0000","last_unstale":"2024-04-25T10:08:50.896365+0000","last_undegraded":"2024-04-25T10:08:50.896365+0000","last_fullsized":"2024-04-25T10:08:50.896365+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:23.850503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894597+0000","last_change":"2024-04-25T10:08:49.916829+0000","last_active":"2024-04-25T10:08:50.894597+0000","last_peered":"2024-04-25T10:08:50.894597+0000","last_clean":"2024-04-25T10:08:50.894597+0000","last_became_active":"2024-04-25T10:08:49.915668+0000","last_became_peered":"2024-04-25T10:08:49.915668+0000","last_unstale":"2024-04-25T10:08:50.894597+0000","last_undegraded":"2024-04-25T10:08:50.894597+0000","last_fullsized":"2024-04-25T10:08:50.894597+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:14.399799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896261+0000","last_change":"2024-04-25T10:08:49.947951+0000","last_active":"2024-04-25T10:08:50.896261+0000","last_peered":"2024-04-25T10:08:50.896261+0000","last_clean":"2024-04-25T10:08:50.896261+0000","last_became_active":"2024-04-25T10:08:49.947185+0000","last_became_peered":"2024-04-25T10:08:49.947185+0000","last_unstale":"2024-04-25T10:08:50.896261+0000","last_undegraded":"2024-04-25T10:08:50.896261+0000","last_fullsized":"2024-04-25T10:08:50.896261+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:57.177294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894560+0000","last_change":"2024-04-25T10:08:49.911228+0000","last_active":"2024-04-25T10:08:50.894560+0000","last_peered":"2024-04-25T10:08:50.894560+0000","last_clean":"2024-04-25T10:08:50.894560+0000","last_became_active":"2024-04-25T10:08:49.910934+0000","last_became_peered":"2024-04-25T10:08:49.910934+0000","last_unstale":"2024-04-25T10:08:50.894560+0000","last_undegraded":"2024-04-25T10:08:50.894560+0000","last_fullsized":"2024-04-25T10:08:50.894560+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:18.416494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025991+0000","last_change":"2024-04-25T10:08:49.940515+0000","last_active":"2024-04-25T10:08:51.025991+0000","last_peered":"2024-04-25T10:08:51.025991+0000","last_clean":"2024-04-25T10:08:51.025991+0000","last_became_active":"2024-04-25T10:08:49.940268+0000","last_became_peered":"2024-04-25T10:08:49.940268+0000","last_unstale":"2024-04-25T10:08:51.025991+0000","last_undegraded":"2024-04-25T10:08:51.025991+0000","last_fullsized":"2024-04-25T10:08:51.025991+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:55.794724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.895912+0000","last_change":"2024-04-25T10:08:49.917909+0000","last_active":"2024-04-25T10:08:50.895912+0000","last_peered":"2024-04-25T10:08:50.895912+0000","last_clean":"2024-04-25T10:08:50.895912+0000","last_became_active":"2024-04-25T10:08:49.917210+0000","last_became_peered":"2024-04-25T10:08:49.917210+0000","last_unstale":"2024-04-25T10:08:50.895912+0000","last_undegraded":"2024-04-25T10:08:50.895912+0000","last_fullsized":"2024-04-25T10:08:50.895912+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:01.702943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025590+0000","last_change":"2024-04-25T10:08:49.911241+0000","last_active":"2024-04-25T10:08:51.025590+0000","last_peered":"2024-04-25T10:08:51.025590+0000","last_clean":"2024-04-25T10:08:51.025590+0000","last_became_active":"2024-04-25T10:08:49.909789+0000","last_became_peered":"2024-04-25T10:08:49.909789+0000","last_unstale":"2024-04-25T10:08:51.025590+0000","last_undegraded":"2024-04-25T10:08:51.025590+0000","last_fullsized":"2024-04-25T10:08:51.025590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:41.508533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023533+0000","last_change":"2024-04-25T10:08:49.932459+0000","last_active":"2024-04-25T10:08:51.023533+0000","last_peered":"2024-04-25T10:08:51.023533+0000","last_clean":"2024-04-25T10:08:51.023533+0000","last_became_active":"2024-04-25T10:08:49.928084+0000","last_became_peered":"2024-04-25T10:08:49.928084+0000","last_unstale":"2024-04-25T10:08:51.023533+0000","last_undegraded":"2024-04-25T10:08:51.023533+0000","last_fullsized":"2024-04-25T10:08:51.023533+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:20.303866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024423+0000","last_change":"2024-04-25T10:08:49.924351+0000","last_active":"2024-04-25T10:08:51.024423+0000","last_peered":"2024-04-25T10:08:51.024423+0000","last_clean":"2024-04-25T10:08:51.024423+0000","last_became_active":"2024-04-25T10:08:49.923957+0000","last_became_peered":"2024-04-25T10:08:49.923957+0000","last_unstale":"2024-04-25T10:08:51.024423+0000","last_undegraded":"2024-04-25T10:08:51.024423+0000","last_fullsized":"2024-04-25T10:08:51.024423+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:10.647770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024615+0000","last_change":"2024-04-25T10:08:49.931996+0000","last_active":"2024-04-25T10:08:51.024615+0000","last_peered":"2024-04-25T10:08:51.024615+0000","last_clean":"2024-04-25T10:08:51.024615+0000","last_became_active":"2024-04-25T10:08:49.931723+0000","last_became_peered":"2024-04-25T10:08:49.931723+0000","last_unstale":"2024-04-25T10:08:51.024615+0000","last_undegraded":"2024-04-25T10:08:51.024615+0000","last_fullsized":"2024-04-25T10:08:51.024615+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:02:14.046715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896681+0000","last_change":"2024-04-25T10:08:49.916056+0000","last_active":"2024-04-25T10:08:50.896681+0000","last_peered":"2024-04-25T10:08:50.896681+0000","last_clean":"2024-04-25T10:08:50.896681+0000","last_became_active":"2024-04-25T10:08:49.915316+0000","last_became_peered":"2024-04-25T10:08:49.915316+0000","last_unstale":"2024-04-25T10:08:50.896681+0000","last_undegraded":"2024-04-25T10:08:50.896681+0000","last_fullsized":"2024-04-25T10:08:50.896681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:32:41.508533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026198+0000","last_change":"2024-04-25T10:08:49.911840+0000","last_active":"2024-04-25T10:08:51.026198+0000","last_peered":"2024-04-25T10:08:51.026198+0000","last_clean":"2024-04-25T10:08:51.026198+0000","last_became_active":"2024-04-25T10:08:49.910898+0000","last_became_peered":"2024-04-25T10:08:49.910898+0000","last_unstale":"2024-04-25T10:08:51.026198+0000","last_undegraded":"2024-04-25T10:08:51.026198+0000","last_fullsized":"2024-04-25T10:08:51.026198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:48.530453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897420+0000","last_change":"2024-04-25T10:08:49.933338+0000","last_active":"2024-04-25T10:08:50.897420+0000","last_peered":"2024-04-25T10:08:50.897420+0000","last_clean":"2024-04-25T10:08:50.897420+0000","last_became_active":"2024-04-25T10:08:49.931755+0000","last_became_peered":"2024-04-25T10:08:49.931755+0000","last_unstale":"2024-04-25T10:08:50.897420+0000","last_undegraded":"2024-04-25T10:08:50.897420+0000","last_fullsized":"2024-04-25T10:08:50.897420+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:55.794724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896319+0000","last_change":"2024-04-25T10:08:49.917704+0000","last_active":"2024-04-25T10:08:50.896319+0000","last_peered":"2024-04-25T10:08:50.896319+0000","last_clean":"2024-04-25T10:08:50.896319+0000","last_became_active":"2024-04-25T10:08:49.917001+0000","last_became_peered":"2024-04-25T10:08:49.917001+0000","last_unstale":"2024-04-25T10:08:50.896319+0000","last_undegraded":"2024-04-25T10:08:50.896319+0000","last_fullsized":"2024-04-25T10:08:50.896319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:20.651561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896060+0000","last_change":"2024-04-25T10:08:49.937375+0000","last_active":"2024-04-25T10:08:50.896060+0000","last_peered":"2024-04-25T10:08:50.896060+0000","last_clean":"2024-04-25T10:08:50.896060+0000","last_became_active":"2024-04-25T10:08:49.935340+0000","last_became_peered":"2024-04-25T10:08:49.935340+0000","last_unstale":"2024-04-25T10:08:50.896060+0000","last_undegraded":"2024-04-25T10:08:50.896060+0000","last_fullsized":"2024-04-25T10:08:50.896060+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:26.689138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023776+0000","last_change":"2024-04-25T10:08:49.933841+0000","last_active":"2024-04-25T10:08:51.023776+0000","last_peered":"2024-04-25T10:08:51.023776+0000","last_clean":"2024-04-25T10:08:51.023776+0000","last_became_active":"2024-04-25T10:08:49.933562+0000","last_became_peered":"2024-04-25T10:08:49.933562+0000","last_unstale":"2024-04-25T10:08:51.023776+0000","last_undegraded":"2024-04-25T10:08:51.023776+0000","last_fullsized":"2024-04-25T10:08:51.023776+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:24.287707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024963+0000","last_change":"2024-04-25T10:08:49.943758+0000","last_active":"2024-04-25T10:08:51.024963+0000","last_peered":"2024-04-25T10:08:51.024963+0000","last_clean":"2024-04-25T10:08:51.024963+0000","last_became_active":"2024-04-25T10:08:49.943054+0000","last_became_peered":"2024-04-25T10:08:49.943054+0000","last_unstale":"2024-04-25T10:08:51.024963+0000","last_undegraded":"2024-04-25T10:08:51.024963+0000","last_fullsized":"2024-04-25T10:08:51.024963+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:24.287707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896177+0000","last_change":"2024-04-25T10:08:49.924855+0000","last_active":"2024-04-25T10:08:50.896177+0000","last_peered":"2024-04-25T10:08:50.896177+0000","last_clean":"2024-04-25T10:08:50.896177+0000","last_became_active":"2024-04-25T10:08:49.921302+0000","last_became_peered":"2024-04-25T10:08:49.921302+0000","last_unstale":"2024-04-25T10:08:50.896177+0000","last_undegraded":"2024-04-25T10:08:50.896177+0000","last_fullsized":"2024-04-25T10:08:50.896177+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:59.474156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894464+0000","last_change":"2024-04-25T10:08:49.929345+0000","last_active":"2024-04-25T10:08:50.894464+0000","last_peered":"2024-04-25T10:08:50.894464+0000","last_clean":"2024-04-25T10:08:50.894464+0000","last_became_active":"2024-04-25T10:08:49.929033+0000","last_became_peered":"2024-04-25T10:08:49.929033+0000","last_unstale":"2024-04-25T10:08:50.894464+0000","last_undegraded":"2024-04-25T10:08:50.894464+0000","last_fullsized":"2024-04-25T10:08:50.894464+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:41.363738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026258+0000","last_change":"2024-04-25T10:08:49.934068+0000","last_active":"2024-04-25T10:08:51.026258+0000","last_peered":"2024-04-25T10:08:51.026258+0000","last_clean":"2024-04-25T10:08:51.026258+0000","last_became_active":"2024-04-25T10:08:49.933795+0000","last_became_peered":"2024-04-25T10:08:49.933795+0000","last_unstale":"2024-04-25T10:08:51.026258+0000","last_undegraded":"2024-04-25T10:08:51.026258+0000","last_fullsized":"2024-04-25T10:08:51.026258+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:15.446181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024698+0000","last_change":"2024-04-25T10:08:49.929735+0000","last_active":"2024-04-25T10:08:51.024698+0000","last_peered":"2024-04-25T10:08:51.024698+0000","last_clean":"2024-04-25T10:08:51.024698+0000","last_became_active":"2024-04-25T10:08:49.928894+0000","last_became_peered":"2024-04-25T10:08:49.928894+0000","last_unstale":"2024-04-25T10:08:51.024698+0000","last_undegraded":"2024-04-25T10:08:51.024698+0000","last_fullsized":"2024-04-25T10:08:51.024698+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:13.673484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896694+0000","last_change":"2024-04-25T10:08:49.947889+0000","last_active":"2024-04-25T10:08:50.896694+0000","last_peered":"2024-04-25T10:08:50.896694+0000","last_clean":"2024-04-25T10:08:50.896694+0000","last_became_active":"2024-04-25T10:08:49.947379+0000","last_became_peered":"2024-04-25T10:08:49.947379+0000","last_unstale":"2024-04-25T10:08:50.896694+0000","last_undegraded":"2024-04-25T10:08:50.896694+0000","last_fullsized":"2024-04-25T10:08:50.896694+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.087705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024218+0000","last_change":"2024-04-25T10:08:49.934628+0000","last_active":"2024-04-25T10:08:51.024218+0000","last_peered":"2024-04-25T10:08:51.024218+0000","last_clean":"2024-04-25T10:08:51.024218+0000","last_became_active":"2024-04-25T10:08:49.933850+0000","last_became_peered":"2024-04-25T10:08:49.933850+0000","last_unstale":"2024-04-25T10:08:51.024218+0000","last_undegraded":"2024-04-25T10:08:51.024218+0000","last_fullsized":"2024-04-25T10:08:51.024218+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:31.708204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025456+0000","last_change":"2024-04-25T10:08:49.926578+0000","last_active":"2024-04-25T10:08:51.025456+0000","last_peered":"2024-04-25T10:08:51.025456+0000","last_clean":"2024-04-25T10:08:51.025456+0000","last_became_active":"2024-04-25T10:08:49.924237+0000","last_became_peered":"2024-04-25T10:08:49.924237+0000","last_unstale":"2024-04-25T10:08:51.025456+0000","last_undegraded":"2024-04-25T10:08:51.025456+0000","last_fullsized":"2024-04-25T10:08:51.025456+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:19.683781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023681+0000","last_change":"2024-04-25T10:08:49.950353+0000","last_active":"2024-04-25T10:08:51.023681+0000","last_peered":"2024-04-25T10:08:51.023681+0000","last_clean":"2024-04-25T10:08:51.023681+0000","last_became_active":"2024-04-25T10:08:49.950150+0000","last_became_peered":"2024-04-25T10:08:49.950150+0000","last_unstale":"2024-04-25T10:08:51.023681+0000","last_undegraded":"2024-04-25T10:08:51.023681+0000","last_fullsized":"2024-04-25T10:08:51.023681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:47.481013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.899023+0000","last_change":"2024-04-25T10:08:48.913258+0000","last_active":"2024-04-25T10:08:50.899023+0000","last_peered":"2024-04-25T10:08:50.899023+0000","last_clean":"2024-04-25T10:08:50.899023+0000","last_became_active":"2024-04-25T10:08:48.912653+0000","last_became_peered":"2024-04-25T10:08:48.912653+0000","last_unstale":"2024-04-25T10:08:50.899023+0000","last_undegraded":"2024-04-25T10:08:50.899023+0000","last_fullsized":"2024-04-25T10:08:50.899023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:19.639236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024462+0000","last_change":"2024-04-25T10:08:49.924091+0000","last_active":"2024-04-25T10:08:51.024462+0000","last_peered":"2024-04-25T10:08:51.024462+0000","last_clean":"2024-04-25T10:08:51.024462+0000","last_became_active":"2024-04-25T10:08:49.916341+0000","last_became_peered":"2024-04-25T10:08:49.916341+0000","last_unstale":"2024-04-25T10:08:51.024462+0000","last_undegraded":"2024-04-25T10:08:51.024462+0000","last_fullsized":"2024-04-25T10:08:51.024462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:25.974390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896832+0000","last_change":"2024-04-25T10:08:48.920404+0000","last_active":"2024-04-25T10:08:50.896832+0000","last_peered":"2024-04-25T10:08:50.896832+0000","last_clean":"2024-04-25T10:08:50.896832+0000","last_became_active":"2024-04-25T10:08:48.919687+0000","last_became_peered":"2024-04-25T10:08:48.919687+0000","last_unstale":"2024-04-25T10:08:50.896832+0000","last_undegraded":"2024-04-25T10:08:50.896832+0000","last_fullsized":"2024-04-25T10:08:50.896832+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:40.598579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024958+0000","last_change":"2024-04-25T10:08:49.929324+0000","last_active":"2024-04-25T10:08:51.024958+0000","last_peered":"2024-04-25T10:08:51.024958+0000","last_clean":"2024-04-25T10:08:51.024958+0000","last_became_active":"2024-04-25T10:08:49.928505+0000","last_became_peered":"2024-04-25T10:08:49.928505+0000","last_unstale":"2024-04-25T10:08:51.024958+0000","last_undegraded":"2024-04-25T10:08:51.024958+0000","last_fullsized":"2024-04-25T10:08:51.024958+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:12.302044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024767+0000","last_change":"2024-04-25T10:08:48.921772+0000","last_active":"2024-04-25T10:08:51.024767+0000","last_peered":"2024-04-25T10:08:51.024767+0000","last_clean":"2024-04-25T10:08:51.024767+0000","last_became_active":"2024-04-25T10:08:48.921226+0000","last_became_peered":"2024-04-25T10:08:48.921226+0000","last_unstale":"2024-04-25T10:08:51.024767+0000","last_undegraded":"2024-04-25T10:08:51.024767+0000","last_fullsized":"2024-04-25T10:08:51.024767+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:17.934641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024618+0000","last_change":"2024-04-25T10:08:49.942909+0000","last_active":"2024-04-25T10:08:51.024618+0000","last_peered":"2024-04-25T10:08:51.024618+0000","last_clean":"2024-04-25T10:08:51.024618+0000","last_became_active":"2024-04-25T10:08:49.942675+0000","last_became_peered":"2024-04-25T10:08:49.942675+0000","last_unstale":"2024-04-25T10:08:51.024618+0000","last_undegraded":"2024-04-25T10:08:51.024618+0000","last_fullsized":"2024-04-25T10:08:51.024618+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:50.736917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897643+0000","last_change":"2024-04-25T10:08:48.920430+0000","last_active":"2024-04-25T10:08:50.897643+0000","last_peered":"2024-04-25T10:08:50.897643+0000","last_clean":"2024-04-25T10:08:50.897643+0000","last_became_active":"2024-04-25T10:08:48.919636+0000","last_became_peered":"2024-04-25T10:08:48.919636+0000","last_unstale":"2024-04-25T10:08:50.897643+0000","last_undegraded":"2024-04-25T10:08:50.897643+0000","last_fullsized":"2024-04-25T10:08:50.897643+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:00.690619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025517+0000","last_change":"2024-04-25T10:08:49.922040+0000","last_active":"2024-04-25T10:08:51.025517+0000","last_peered":"2024-04-25T10:08:51.025517+0000","last_clean":"2024-04-25T10:08:51.025517+0000","last_became_active":"2024-04-25T10:08:49.921741+0000","last_became_peered":"2024-04-25T10:08:49.921741+0000","last_unstale":"2024-04-25T10:08:51.025517+0000","last_undegraded":"2024-04-25T10:08:51.025517+0000","last_fullsized":"2024-04-25T10:08:51.025517+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:23.850503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.898148+0000","last_change":"2024-04-25T10:08:48.913325+0000","last_active":"2024-04-25T10:08:50.898148+0000","last_peered":"2024-04-25T10:08:50.898148+0000","last_clean":"2024-04-25T10:08:50.898148+0000","last_became_active":"2024-04-25T10:08:48.912648+0000","last_became_peered":"2024-04-25T10:08:48.912648+0000","last_unstale":"2024-04-25T10:08:50.898148+0000","last_undegraded":"2024-04-25T10:08:50.898148+0000","last_fullsized":"2024-04-25T10:08:50.898148+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:00.006302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896453+0000","last_change":"2024-04-25T10:08:49.936814+0000","last_active":"2024-04-25T10:08:50.896453+0000","last_peered":"2024-04-25T10:08:50.896453+0000","last_clean":"2024-04-25T10:08:50.896453+0000","last_became_active":"2024-04-25T10:08:49.935981+0000","last_became_peered":"2024-04-25T10:08:49.935981+0000","last_unstale":"2024-04-25T10:08:50.896453+0000","last_undegraded":"2024-04-25T10:08:50.896453+0000","last_fullsized":"2024-04-25T10:08:50.896453+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:15.446181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026131+0000","last_change":"2024-04-25T10:08:48.921408+0000","last_active":"2024-04-25T10:08:51.026131+0000","last_peered":"2024-04-25T10:08:51.026131+0000","last_clean":"2024-04-25T10:08:51.026131+0000","last_became_active":"2024-04-25T10:08:48.921179+0000","last_became_peered":"2024-04-25T10:08:48.921179+0000","last_unstale":"2024-04-25T10:08:51.026131+0000","last_undegraded":"2024-04-25T10:08:51.026131+0000","last_fullsized":"2024-04-25T10:08:51.026131+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:28.513972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023795+0000","last_change":"2024-04-25T10:08:49.927055+0000","last_active":"2024-04-25T10:08:51.023795+0000","last_peered":"2024-04-25T10:08:51.023795+0000","last_clean":"2024-04-25T10:08:51.023795+0000","last_became_active":"2024-04-25T10:08:49.924405+0000","last_became_peered":"2024-04-25T10:08:49.924405+0000","last_unstale":"2024-04-25T10:08:51.023795+0000","last_undegraded":"2024-04-25T10:08:51.023795+0000","last_fullsized":"2024-04-25T10:08:51.023795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:10.261301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023971+0000","last_change":"2024-04-25T10:08:48.923790+0000","last_active":"2024-04-25T10:08:51.023971+0000","last_peered":"2024-04-25T10:08:51.023971+0000","last_clean":"2024-04-25T10:08:51.023971+0000","last_became_active":"2024-04-25T10:08:48.923543+0000","last_became_peered":"2024-04-25T10:08:48.923543+0000","last_unstale":"2024-04-25T10:08:51.023971+0000","last_undegraded":"2024-04-25T10:08:51.023971+0000","last_fullsized":"2024-04-25T10:08:51.023971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:45.783541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025995+0000","last_change":"2024-04-25T10:08:49.927347+0000","last_active":"2024-04-25T10:08:51.025995+0000","last_peered":"2024-04-25T10:08:51.025995+0000","last_clean":"2024-04-25T10:08:51.025995+0000","last_became_active":"2024-04-25T10:08:49.927022+0000","last_became_peered":"2024-04-25T10:08:49.927022+0000","last_unstale":"2024-04-25T10:08:51.025995+0000","last_undegraded":"2024-04-25T10:08:51.025995+0000","last_fullsized":"2024-04-25T10:08:51.025995+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:52:31.701952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024200+0000","last_change":"2024-04-25T10:08:48.915851+0000","last_active":"2024-04-25T10:08:51.024200+0000","last_peered":"2024-04-25T10:08:51.024200+0000","last_clean":"2024-04-25T10:08:51.024200+0000","last_became_active":"2024-04-25T10:08:48.915212+0000","last_became_peered":"2024-04-25T10:08:48.915212+0000","last_unstale":"2024-04-25T10:08:51.024200+0000","last_undegraded":"2024-04-25T10:08:51.024200+0000","last_fullsized":"2024-04-25T10:08:51.024200+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:09.544601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897987+0000","last_change":"2024-04-25T10:08:49.916249+0000","last_active":"2024-04-25T10:08:50.897987+0000","last_peered":"2024-04-25T10:08:50.897987+0000","last_clean":"2024-04-25T10:08:50.897987+0000","last_became_active":"2024-04-25T10:08:49.915587+0000","last_became_peered":"2024-04-25T10:08:49.915587+0000","last_unstale":"2024-04-25T10:08:50.897987+0000","last_undegraded":"2024-04-25T10:08:50.897987+0000","last_fullsized":"2024-04-25T10:08:50.897987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:26.689138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025442+0000","last_change":"2024-04-25T10:08:48.908751+0000","last_active":"2024-04-25T10:08:51.025442+0000","last_peered":"2024-04-25T10:08:51.025442+0000","last_clean":"2024-04-25T10:08:51.025442+0000","last_became_active":"2024-04-25T10:08:48.908243+0000","last_became_peered":"2024-04-25T10:08:48.908243+0000","last_unstale":"2024-04-25T10:08:51.025442+0000","last_undegraded":"2024-04-25T10:08:51.025442+0000","last_fullsized":"2024-04-25T10:08:51.025442+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:37.820265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024502+0000","last_change":"2024-04-25T10:08:49.949584+0000","last_active":"2024-04-25T10:08:51.024502+0000","last_peered":"2024-04-25T10:08:51.024502+0000","last_clean":"2024-04-25T10:08:51.024502+0000","last_became_active":"2024-04-25T10:08:49.948930+0000","last_became_peered":"2024-04-25T10:08:49.948930+0000","last_unstale":"2024-04-25T10:08:51.024502+0000","last_undegraded":"2024-04-25T10:08:51.024502+0000","last_fullsized":"2024-04-25T10:08:51.024502+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:53.481085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024911+0000","last_change":"2024-04-25T10:08:48.917822+0000","last_active":"2024-04-25T10:08:51.024911+0000","last_peered":"2024-04-25T10:08:51.024911+0000","last_clean":"2024-04-25T10:08:51.024911+0000","last_became_active":"2024-04-25T10:08:48.917540+0000","last_became_peered":"2024-04-25T10:08:48.917540+0000","last_unstale":"2024-04-25T10:08:51.024911+0000","last_undegraded":"2024-04-25T10:08:51.024911+0000","last_fullsized":"2024-04-25T10:08:51.024911+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:56.284437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024800+0000","last_change":"2024-04-25T10:08:49.917169+0000","last_active":"2024-04-25T10:08:51.024800+0000","last_peered":"2024-04-25T10:08:51.024800+0000","last_clean":"2024-04-25T10:08:51.024800+0000","last_became_active":"2024-04-25T10:08:49.916903+0000","last_became_peered":"2024-04-25T10:08:49.916903+0000","last_unstale":"2024-04-25T10:08:51.024800+0000","last_undegraded":"2024-04-25T10:08:51.024800+0000","last_fullsized":"2024-04-25T10:08:51.024800+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:21.707833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897059+0000","last_change":"2024-04-25T10:08:48.930283+0000","last_active":"2024-04-25T10:08:50.897059+0000","last_peered":"2024-04-25T10:08:50.897059+0000","last_clean":"2024-04-25T10:08:50.897059+0000","last_became_active":"2024-04-25T10:08:48.929026+0000","last_became_peered":"2024-04-25T10:08:48.929026+0000","last_unstale":"2024-04-25T10:08:50.897059+0000","last_undegraded":"2024-04-25T10:08:50.897059+0000","last_fullsized":"2024-04-25T10:08:50.897059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:46.128263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026391+0000","last_change":"2024-04-25T10:08:49.942396+0000","last_active":"2024-04-25T10:08:51.026391+0000","last_peered":"2024-04-25T10:08:51.026391+0000","last_clean":"2024-04-25T10:08:51.026391+0000","last_became_active":"2024-04-25T10:08:49.942210+0000","last_became_peered":"2024-04-25T10:08:49.942210+0000","last_unstale":"2024-04-25T10:08:51.026391+0000","last_undegraded":"2024-04-25T10:08:51.026391+0000","last_fullsized":"2024-04-25T10:08:51.026391+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:38.832589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024033+0000","last_change":"2024-04-25T10:08:48.926872+0000","last_active":"2024-04-25T10:08:51.024033+0000","last_peered":"2024-04-25T10:08:51.024033+0000","last_clean":"2024-04-25T10:08:51.024033+0000","last_became_active":"2024-04-25T10:08:48.926660+0000","last_became_peered":"2024-04-25T10:08:48.926660+0000","last_unstale":"2024-04-25T10:08:51.024033+0000","last_undegraded":"2024-04-25T10:08:51.024033+0000","last_fullsized":"2024-04-25T10:08:51.024033+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:31.987379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025648+0000","last_change":"2024-04-25T10:08:49.929823+0000","last_active":"2024-04-25T10:08:51.025648+0000","last_peered":"2024-04-25T10:08:51.025648+0000","last_clean":"2024-04-25T10:08:51.025648+0000","last_became_active":"2024-04-25T10:08:49.929499+0000","last_became_peered":"2024-04-25T10:08:49.929499+0000","last_unstale":"2024-04-25T10:08:51.025648+0000","last_undegraded":"2024-04-25T10:08:51.025648+0000","last_fullsized":"2024-04-25T10:08:51.025648+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:44.087705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024867+0000","last_change":"2024-04-25T10:08:48.916082+0000","last_active":"2024-04-25T10:08:51.024867+0000","last_peered":"2024-04-25T10:08:51.024867+0000","last_clean":"2024-04-25T10:08:51.024867+0000","last_became_active":"2024-04-25T10:08:48.915311+0000","last_became_peered":"2024-04-25T10:08:48.915311+0000","last_unstale":"2024-04-25T10:08:51.024867+0000","last_undegraded":"2024-04-25T10:08:51.024867+0000","last_fullsized":"2024-04-25T10:08:51.024867+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:23.275383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.895682+0000","last_change":"2024-04-25T10:08:49.943731+0000","last_active":"2024-04-25T10:08:50.895682+0000","last_peered":"2024-04-25T10:08:50.895682+0000","last_clean":"2024-04-25T10:08:50.895682+0000","last_became_active":"2024-04-25T10:08:49.943041+0000","last_became_peered":"2024-04-25T10:08:49.943041+0000","last_unstale":"2024-04-25T10:08:50.895682+0000","last_undegraded":"2024-04-25T10:08:50.895682+0000","last_fullsized":"2024-04-25T10:08:50.895682+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:20.648426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023888+0000","last_change":"2024-04-25T10:08:48.927015+0000","last_active":"2024-04-25T10:08:51.023888+0000","last_peered":"2024-04-25T10:08:51.023888+0000","last_clean":"2024-04-25T10:08:51.023888+0000","last_became_active":"2024-04-25T10:08:48.926431+0000","last_became_peered":"2024-04-25T10:08:48.926431+0000","last_unstale":"2024-04-25T10:08:51.023888+0000","last_undegraded":"2024-04-25T10:08:51.023888+0000","last_fullsized":"2024-04-25T10:08:51.023888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:30.695879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894267+0000","last_change":"2024-04-25T10:08:49.912237+0000","last_active":"2024-04-25T10:08:50.894267+0000","last_peered":"2024-04-25T10:08:50.894267+0000","last_clean":"2024-04-25T10:08:50.894267+0000","last_became_active":"2024-04-25T10:08:49.911785+0000","last_became_peered":"2024-04-25T10:08:49.911785+0000","last_unstale":"2024-04-25T10:08:50.894267+0000","last_undegraded":"2024-04-25T10:08:50.894267+0000","last_fullsized":"2024-04-25T10:08:50.894267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:15.406529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025731+0000","last_change":"2024-04-25T10:08:48.920708+0000","last_active":"2024-04-25T10:08:51.025731+0000","last_peered":"2024-04-25T10:08:51.025731+0000","last_clean":"2024-04-25T10:08:51.025731+0000","last_became_active":"2024-04-25T10:08:48.920393+0000","last_became_peered":"2024-04-25T10:08:48.920393+0000","last_unstale":"2024-04-25T10:08:51.025731+0000","last_undegraded":"2024-04-25T10:08:51.025731+0000","last_fullsized":"2024-04-25T10:08:51.025731+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:57.163320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894652+0000","last_change":"2024-04-25T10:08:49.919035+0000","last_active":"2024-04-25T10:08:50.894652+0000","last_peered":"2024-04-25T10:08:50.894652+0000","last_clean":"2024-04-25T10:08:50.894652+0000","last_became_active":"2024-04-25T10:08:49.918460+0000","last_became_peered":"2024-04-25T10:08:49.918460+0000","last_unstale":"2024-04-25T10:08:50.894652+0000","last_undegraded":"2024-04-25T10:08:50.894652+0000","last_fullsized":"2024-04-25T10:08:50.894652+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:17.723566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025660+0000","last_change":"2024-04-25T10:08:48.912297+0000","last_active":"2024-04-25T10:08:51.025660+0000","last_peered":"2024-04-25T10:08:51.025660+0000","last_clean":"2024-04-25T10:08:51.025660+0000","last_became_active":"2024-04-25T10:08:48.911555+0000","last_became_peered":"2024-04-25T10:08:48.911555+0000","last_unstale":"2024-04-25T10:08:51.025660+0000","last_undegraded":"2024-04-25T10:08:51.025660+0000","last_fullsized":"2024-04-25T10:08:51.025660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:39.472790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023898+0000","last_change":"2024-04-25T10:08:49.950578+0000","last_active":"2024-04-25T10:08:51.023898+0000","last_peered":"2024-04-25T10:08:51.023898+0000","last_clean":"2024-04-25T10:08:51.023898+0000","last_became_active":"2024-04-25T10:08:49.950276+0000","last_became_peered":"2024-04-25T10:08:49.950276+0000","last_unstale":"2024-04-25T10:08:51.023898+0000","last_undegraded":"2024-04-25T10:08:51.023898+0000","last_fullsized":"2024-04-25T10:08:51.023898+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:46.324741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.899296+0000","last_change":"2024-04-25T10:08:48.923258+0000","last_active":"2024-04-25T10:08:50.899296+0000","last_peered":"2024-04-25T10:08:50.899296+0000","last_clean":"2024-04-25T10:08:50.899296+0000","last_became_active":"2024-04-25T10:08:48.922652+0000","last_became_peered":"2024-04-25T10:08:48.922652+0000","last_unstale":"2024-04-25T10:08:50.899296+0000","last_undegraded":"2024-04-25T10:08:50.899296+0000","last_fullsized":"2024-04-25T10:08:50.899296+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:22.838179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.898719+0000","last_change":"2024-04-25T10:08:49.933047+0000","last_active":"2024-04-25T10:08:50.898719+0000","last_peered":"2024-04-25T10:08:50.898719+0000","last_clean":"2024-04-25T10:08:50.898719+0000","last_became_active":"2024-04-25T10:08:49.932331+0000","last_became_peered":"2024-04-25T10:08:49.932331+0000","last_unstale":"2024-04-25T10:08:50.898719+0000","last_undegraded":"2024-04-25T10:08:50.898719+0000","last_fullsized":"2024-04-25T10:08:50.898719+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:46.795865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025117+0000","last_change":"2024-04-25T10:08:48.924659+0000","last_active":"2024-04-25T10:08:51.025117+0000","last_peered":"2024-04-25T10:08:51.025117+0000","last_clean":"2024-04-25T10:08:51.025117+0000","last_became_active":"2024-04-25T10:08:48.923481+0000","last_became_peered":"2024-04-25T10:08:48.923481+0000","last_unstale":"2024-04-25T10:08:51.025117+0000","last_undegraded":"2024-04-25T10:08:51.025117+0000","last_fullsized":"2024-04-25T10:08:51.025117+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:25.811654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896574+0000","last_change":"2024-04-25T10:08:49.925371+0000","last_active":"2024-04-25T10:08:50.896574+0000","last_peered":"2024-04-25T10:08:50.896574+0000","last_clean":"2024-04-25T10:08:50.896574+0000","last_became_active":"2024-04-25T10:08:49.922133+0000","last_became_peered":"2024-04-25T10:08:49.922133+0000","last_unstale":"2024-04-25T10:08:50.896574+0000","last_undegraded":"2024-04-25T10:08:50.896574+0000","last_fullsized":"2024-04-25T10:08:50.896574+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:28.921193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897219+0000","last_change":"2024-04-25T10:08:48.921337+0000","last_active":"2024-04-25T10:08:50.897219+0000","last_peered":"2024-04-25T10:08:50.897219+0000","last_clean":"2024-04-25T10:08:50.897219+0000","last_became_active":"2024-04-25T10:08:48.918289+0000","last_became_peered":"2024-04-25T10:08:48.918289+0000","last_unstale":"2024-04-25T10:08:50.897219+0000","last_undegraded":"2024-04-25T10:08:50.897219+0000","last_fullsized":"2024-04-25T10:08:50.897219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:29.713431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024677+0000","last_change":"2024-04-25T10:08:49.932889+0000","last_active":"2024-04-25T10:08:51.024677+0000","last_peered":"2024-04-25T10:08:51.024677+0000","last_clean":"2024-04-25T10:08:51.024677+0000","last_became_active":"2024-04-25T10:08:49.932500+0000","last_became_peered":"2024-04-25T10:08:49.932500+0000","last_unstale":"2024-04-25T10:08:51.024677+0000","last_undegraded":"2024-04-25T10:08:51.024677+0000","last_fullsized":"2024-04-25T10:08:51.024677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:44.271269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024199+0000","last_change":"2024-04-25T10:08:48.906519+0000","last_active":"2024-04-25T10:08:51.024199+0000","last_peered":"2024-04-25T10:08:51.024199+0000","last_clean":"2024-04-25T10:08:51.024199+0000","last_became_active":"2024-04-25T10:08:48.905583+0000","last_became_peered":"2024-04-25T10:08:48.905583+0000","last_unstale":"2024-04-25T10:08:51.024199+0000","last_undegraded":"2024-04-25T10:08:51.024199+0000","last_fullsized":"2024-04-25T10:08:51.024199+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:14.433856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024460+0000","last_change":"2024-04-25T10:08:49.939208+0000","last_active":"2024-04-25T10:08:51.024460+0000","last_peered":"2024-04-25T10:08:51.024460+0000","last_clean":"2024-04-25T10:08:51.024460+0000","last_became_active":"2024-04-25T10:08:49.932965+0000","last_became_peered":"2024-04-25T10:08:49.932965+0000","last_unstale":"2024-04-25T10:08:51.024460+0000","last_undegraded":"2024-04-25T10:08:51.024460+0000","last_fullsized":"2024-04-25T10:08:51.024460+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:26.823978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894314+0000","last_change":"2024-04-25T10:08:48.920400+0000","last_active":"2024-04-25T10:08:50.894314+0000","last_peered":"2024-04-25T10:08:50.894314+0000","last_clean":"2024-04-25T10:08:50.894314+0000","last_became_active":"2024-04-25T10:08:48.920006+0000","last_became_peered":"2024-04-25T10:08:48.920006+0000","last_unstale":"2024-04-25T10:08:50.894314+0000","last_undegraded":"2024-04-25T10:08:50.894314+0000","last_fullsized":"2024-04-25T10:08:50.894314+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:17.404869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024268+0000","last_change":"2024-04-25T10:08:49.932407+0000","last_active":"2024-04-25T10:08:51.024268+0000","last_peered":"2024-04-25T10:08:51.024268+0000","last_clean":"2024-04-25T10:08:51.024268+0000","last_became_active":"2024-04-25T10:08:49.931550+0000","last_became_peered":"2024-04-25T10:08:49.931550+0000","last_unstale":"2024-04-25T10:08:51.024268+0000","last_undegraded":"2024-04-25T10:08:51.024268+0000","last_fullsized":"2024-04-25T10:08:51.024268+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:10.556925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024711+0000","last_change":"2024-04-25T10:08:48.908793+0000","last_active":"2024-04-25T10:08:51.024711+0000","last_peered":"2024-04-25T10:08:51.024711+0000","last_clean":"2024-04-25T10:08:51.024711+0000","last_became_active":"2024-04-25T10:08:48.908121+0000","last_became_peered":"2024-04-25T10:08:48.908121+0000","last_unstale":"2024-04-25T10:08:51.024711+0000","last_undegraded":"2024-04-25T10:08:51.024711+0000","last_fullsized":"2024-04-25T10:08:51.024711+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:25.676813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025329+0000","last_change":"2024-04-25T10:08:49.924003+0000","last_active":"2024-04-25T10:08:51.025329+0000","last_peered":"2024-04-25T10:08:51.025329+0000","last_clean":"2024-04-25T10:08:51.025329+0000","last_became_active":"2024-04-25T10:08:49.916376+0000","last_became_peered":"2024-04-25T10:08:49.916376+0000","last_unstale":"2024-04-25T10:08:51.025329+0000","last_undegraded":"2024-04-25T10:08:51.025329+0000","last_fullsized":"2024-04-25T10:08:51.025329+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:28.685721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894391+0000","last_change":"2024-04-25T10:08:48.907802+0000","last_active":"2024-04-25T10:08:50.894391+0000","last_peered":"2024-04-25T10:08:50.894391+0000","last_clean":"2024-04-25T10:08:50.894391+0000","last_became_active":"2024-04-25T10:08:48.907484+0000","last_became_peered":"2024-04-25T10:08:48.907484+0000","last_unstale":"2024-04-25T10:08:50.894391+0000","last_undegraded":"2024-04-25T10:08:50.894391+0000","last_fullsized":"2024-04-25T10:08:50.894391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:53.392534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897244+0000","last_change":"2024-04-25T10:08:49.935472+0000","last_active":"2024-04-25T10:08:50.897244+0000","last_peered":"2024-04-25T10:08:50.897244+0000","last_clean":"2024-04-25T10:08:50.897244+0000","last_became_active":"2024-04-25T10:08:49.934581+0000","last_became_peered":"2024-04-25T10:08:49.934581+0000","last_unstale":"2024-04-25T10:08:50.897244+0000","last_undegraded":"2024-04-25T10:08:50.897244+0000","last_fullsized":"2024-04-25T10:08:50.897244+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:38.832589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025591+0000","last_change":"2024-04-25T10:08:48.925017+0000","last_active":"2024-04-25T10:08:51.025591+0000","last_peered":"2024-04-25T10:08:51.025591+0000","last_clean":"2024-04-25T10:08:51.025591+0000","last_became_active":"2024-04-25T10:08:48.924231+0000","last_became_peered":"2024-04-25T10:08:48.924231+0000","last_unstale":"2024-04-25T10:08:51.025591+0000","last_undegraded":"2024-04-25T10:08:51.025591+0000","last_fullsized":"2024-04-25T10:08:51.025591+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:21.486349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024702+0000","last_change":"2024-04-25T10:08:49.950419+0000","last_active":"2024-04-25T10:08:51.024702+0000","last_peered":"2024-04-25T10:08:51.024702+0000","last_clean":"2024-04-25T10:08:51.024702+0000","last_became_active":"2024-04-25T10:08:49.950087+0000","last_became_peered":"2024-04-25T10:08:49.950087+0000","last_unstale":"2024-04-25T10:08:51.024702+0000","last_undegraded":"2024-04-25T10:08:51.024702+0000","last_fullsized":"2024-04-25T10:08:51.024702+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:47.656140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897300+0000","last_change":"2024-04-25T10:08:48.927255+0000","last_active":"2024-04-25T10:08:50.897300+0000","last_peered":"2024-04-25T10:08:50.897300+0000","last_clean":"2024-04-25T10:08:50.897300+0000","last_became_active":"2024-04-25T10:08:48.926623+0000","last_became_peered":"2024-04-25T10:08:48.926623+0000","last_unstale":"2024-04-25T10:08:50.897300+0000","last_undegraded":"2024-04-25T10:08:50.897300+0000","last_fullsized":"2024-04-25T10:08:50.897300+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:31.607079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024188+0000","last_change":"2024-04-25T10:08:49.924001+0000","last_active":"2024-04-25T10:08:51.024188+0000","last_peered":"2024-04-25T10:08:51.024188+0000","last_clean":"2024-04-25T10:08:51.024188+0000","last_became_active":"2024-04-25T10:08:49.923543+0000","last_became_peered":"2024-04-25T10:08:49.923543+0000","last_unstale":"2024-04-25T10:08:51.024188+0000","last_undegraded":"2024-04-25T10:08:51.024188+0000","last_fullsized":"2024-04-25T10:08:51.024188+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:53.439235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897364+0000","last_change":"2024-04-25T10:08:48.929895+0000","last_active":"2024-04-25T10:08:50.897364+0000","last_peered":"2024-04-25T10:08:50.897364+0000","last_clean":"2024-04-25T10:08:50.897364+0000","last_became_active":"2024-04-25T10:08:48.928316+0000","last_became_peered":"2024-04-25T10:08:48.928316+0000","last_unstale":"2024-04-25T10:08:50.897364+0000","last_undegraded":"2024-04-25T10:08:50.897364+0000","last_fullsized":"2024-04-25T10:08:50.897364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:41.083371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025229+0000","last_change":"2024-04-25T10:08:49.950248+0000","last_active":"2024-04-25T10:08:51.025229+0000","last_peered":"2024-04-25T10:08:51.025229+0000","last_clean":"2024-04-25T10:08:51.025229+0000","last_became_active":"2024-04-25T10:08:49.949446+0000","last_became_peered":"2024-04-25T10:08:49.949446+0000","last_unstale":"2024-04-25T10:08:51.025229+0000","last_undegraded":"2024-04-25T10:08:51.025229+0000","last_fullsized":"2024-04-25T10:08:51.025229+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:32.999703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.897901+0000","last_change":"2024-04-25T10:08:48.913827+0000","last_active":"2024-04-25T10:08:50.897901+0000","last_peered":"2024-04-25T10:08:50.897901+0000","last_clean":"2024-04-25T10:08:50.897901+0000","last_became_active":"2024-04-25T10:08:48.912878+0000","last_became_peered":"2024-04-25T10:08:48.912878+0000","last_unstale":"2024-04-25T10:08:50.897901+0000","last_undegraded":"2024-04-25T10:08:50.897901+0000","last_fullsized":"2024-04-25T10:08:50.897901+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:58.461832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.894638+0000","last_change":"2024-04-25T10:08:49.915877+0000","last_active":"2024-04-25T10:08:50.894638+0000","last_peered":"2024-04-25T10:08:50.894638+0000","last_clean":"2024-04-25T10:08:50.894638+0000","last_became_active":"2024-04-25T10:08:49.915210+0000","last_became_peered":"2024-04-25T10:08:49.915210+0000","last_unstale":"2024-04-25T10:08:50.894638+0000","last_undegraded":"2024-04-25T10:08:50.894638+0000","last_fullsized":"2024-04-25T10:08:50.894638+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:36.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024149+0000","last_change":"2024-04-25T10:08:48.926824+0000","last_active":"2024-04-25T10:08:51.024149+0000","last_peered":"2024-04-25T10:08:51.024149+0000","last_clean":"2024-04-25T10:08:51.024149+0000","last_became_active":"2024-04-25T10:08:48.926515+0000","last_became_peered":"2024-04-25T10:08:48.926515+0000","last_unstale":"2024-04-25T10:08:51.024149+0000","last_undegraded":"2024-04-25T10:08:51.024149+0000","last_fullsized":"2024-04-25T10:08:51.024149+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:23.275383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023859+0000","last_change":"2024-04-25T10:08:49.911297+0000","last_active":"2024-04-25T10:08:51.023859+0000","last_peered":"2024-04-25T10:08:51.023859+0000","last_clean":"2024-04-25T10:08:51.023859+0000","last_became_active":"2024-04-25T10:08:49.910937+0000","last_became_peered":"2024-04-25T10:08:49.910937+0000","last_unstale":"2024-04-25T10:08:51.023859+0000","last_undegraded":"2024-04-25T10:08:51.023859+0000","last_fullsized":"2024-04-25T10:08:51.023859+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:31.708204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024211+0000","last_change":"2024-04-25T10:08:48.912204+0000","last_active":"2024-04-25T10:08:51.024211+0000","last_peered":"2024-04-25T10:08:51.024211+0000","last_clean":"2024-04-25T10:08:51.024211+0000","last_became_active":"2024-04-25T10:08:48.911509+0000","last_became_peered":"2024-04-25T10:08:48.911509+0000","last_unstale":"2024-04-25T10:08:51.024211+0000","last_undegraded":"2024-04-25T10:08:51.024211+0000","last_fullsized":"2024-04-25T10:08:51.024211+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:47.518129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896159+0000","last_change":"2024-04-25T10:08:49.925037+0000","last_active":"2024-04-25T10:08:50.896159+0000","last_peered":"2024-04-25T10:08:50.896159+0000","last_clean":"2024-04-25T10:08:50.896159+0000","last_became_active":"2024-04-25T10:08:49.924298+0000","last_became_peered":"2024-04-25T10:08:49.924298+0000","last_unstale":"2024-04-25T10:08:50.896159+0000","last_undegraded":"2024-04-25T10:08:50.896159+0000","last_fullsized":"2024-04-25T10:08:50.896159+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:48.530453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:50.896683+0000","last_change":"2024-04-25T10:08:48.912652+0000","last_active":"2024-04-25T10:08:50.896683+0000","last_peered":"2024-04-25T10:08:50.896683+0000","last_clean":"2024-04-25T10:08:50.896683+0000","last_became_active":"2024-04-25T10:08:48.911838+0000","last_became_peered":"2024-04-25T10:08:48.911838+0000","last_unstale":"2024-04-25T10:08:50.896683+0000","last_undegraded":"2024-04-25T10:08:50.896683+0000","last_fullsized":"2024-04-25T10:08:50.896683+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:56.164970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024054+0000","last_change":"2024-04-25T10:08:49.923555+0000","last_active":"2024-04-25T10:08:51.024054+0000","last_peered":"2024-04-25T10:08:51.024054+0000","last_clean":"2024-04-25T10:08:51.024054+0000","last_became_active":"2024-04-25T10:08:49.916247+0000","last_became_peered":"2024-04-25T10:08:49.916247+0000","last_unstale":"2024-04-25T10:08:51.024054+0000","last_undegraded":"2024-04-25T10:08:51.024054+0000","last_fullsized":"2024-04-25T10:08:51.024054+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:55.902599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.023932+0000","last_change":"2024-04-25T10:08:48.920751+0000","last_active":"2024-04-25T10:08:51.023932+0000","last_peered":"2024-04-25T10:08:51.023932+0000","last_clean":"2024-04-25T10:08:51.023932+0000","last_became_active":"2024-04-25T10:08:48.920001+0000","last_became_peered":"2024-04-25T10:08:48.920001+0000","last_unstale":"2024-04-25T10:08:51.023932+0000","last_undegraded":"2024-04-25T10:08:51.023932+0000","last_fullsized":"2024-04-25T10:08:51.023932+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:45.160552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.025381+0000","last_change":"2024-04-25T10:08:49.929453+0000","last_active":"2024-04-25T10:08:51.025381+0000","last_peered":"2024-04-25T10:08:51.025381+0000","last_clean":"2024-04-25T10:08:51.025381+0000","last_became_active":"2024-04-25T10:08:49.928680+0000","last_became_peered":"2024-04-25T10:08:49.928680+0000","last_unstale":"2024-04-25T10:08:51.025381+0000","last_undegraded":"2024-04-25T10:08:51.025381+0000","last_fullsized":"2024-04-25T10:08:51.025381+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:48.875175+0000","last_clean_scrub_stamp":"2024-04-25T10:08:48.875175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:01.783225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.026311+0000","last_change":"2024-04-25T10:08:48.924246+0000","last_active":"2024-04-25T10:08:51.026311+0000","last_peered":"2024-04-25T10:08:51.026311+0000","last_clean":"2024-04-25T10:08:51.026311+0000","last_became_active":"2024-04-25T10:08:48.923370+0000","last_became_peered":"2024-04-25T10:08:48.923370+0000","last_unstale":"2024-04-25T10:08:51.026311+0000","last_undegraded":"2024-04-25T10:08:51.026311+0000","last_fullsized":"2024-04-25T10:08:51.026311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:47.862850+0000","last_clean_scrub_stamp":"2024-04-25T10:08:47.862850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:49.651863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":302,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T10:08:51.024468+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:08:51.024468+0000","last_peered":"2024-04-25T10:08:51.024468+0000","last_clean":"2024-04-25T10:08:51.024468+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:08:51.024468+0000","last_undegraded":"2024-04-25T10:08:51.024468+0000","last_fullsized":"2024-04-25T10:08:51.024468+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574935,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1724416,"data_stored":1107118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574936,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":518910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574937,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":516694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574935,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1130496,"data_stored":516854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574937,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1122304,"data_stored":516694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574936,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1122304,"data_stored":516694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1126400,"data_stored":516764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574934,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32888,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1740800,"data_stored":1109380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:57.732 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:08:57.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:08:57.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 Processor -- start 2024-04-25T10:08:57.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 -- start start 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f72f0159bd0 0x7f72f00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72f0159fb0 0x7f72f00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f00b26f0 0x7f72f00b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f0072780 con 0x7f72f0159fb0 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f00c9ef0 con 0x7f72f0159bd0 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72f4c10640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f0071b70 con 0x7f72f00b26f0 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72f0159fb0 0x7f72f00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72f0159fb0 0x7f72f00b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33446/0 (socket says 172.21.15.82:33446) 2024-04-25T10:08:57.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 -- 172.21.15.82:0/3252491799 learned_addr learned my addr 172.21.15.82:0/3252491799 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72ee59c640 1 --2- 172.21.15.82:0/3252491799 >> v2:172.21.15.196:3300/0 conn(0x7f72f0159bd0 0x7f72f00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72eed9d640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f00b26f0 0x7f72f00b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 -- 172.21.15.82:0/3252491799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f00b26f0 msgr2=0x7f72f00b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f00b26f0 0x7f72f00b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 -- 172.21.15.82:0/3252491799 >> v2:172.21.15.196:3300/0 conn(0x7f72f0159bd0 msgr2=0x7f72f00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 --2- 172.21.15.82:0/3252491799 >> v2:172.21.15.196:3300/0 conn(0x7f72f0159bd0 0x7f72f00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.896+0000 7f72edd9b640 1 -- 172.21.15.82:0/3252491799 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0071e50 con 0x7f72f0159fb0 2024-04-25T10:08:57.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72edd9b640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f0159fb0 0x7f72f00b21b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f72f00cf2c0 tx=0x7f72e401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766e2f9c9e391dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72ed59a640 1 -- 172.21.15.82:0/3252491799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40ab430 con 0x7f72f0159fb0 2024-04-25T10:08:57.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72ed59a640 1 -- 172.21.15.82:0/3252491799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e40b0070 con 0x7f72f0159fb0 2024-04-25T10:08:57.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72ed59a640 1 -- 172.21.15.82:0/3252491799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e40ab7f0 con 0x7f72f0159fb0 2024-04-25T10:08:57.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- 172.21.15.82:0/3252491799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f0159fb0 msgr2=0x7f72f00b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f0159fb0 0x7f72f00b21b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f72f00cf2c0 tx=0x7f72e401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- 172.21.15.82:0/3252491799 shutdown_connections 2024-04-25T10:08:57.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f00b26f0 0x7f72f00b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 --2- 172.21.15.82:0/3252491799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f0159fb0 0x7f72f00b21b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 --2- 172.21.15.82:0/3252491799 >> v2:172.21.15.196:3300/0 conn(0x7f72f0159bd0 0x7f72f00c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- 172.21.15.82:0/3252491799 >> 172.21.15.82:0/3252491799 conn(0x7f72f006ae60 msgr2=0x7f72f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- 172.21.15.82:0/3252491799 shutdown_connections 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- 172.21.15.82:0/3252491799 wait complete. 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 Processor -- start 2024-04-25T10:08:57.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.900+0000 7f72f4c10640 1 -- start start 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 0x7f72f01538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f0159fb0 0x7f72f01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00cb500 con 0x7f72f00b26f0 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00cb380 con 0x7f72f0159bd0 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f00cb680 con 0x7f72f0159fb0 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33456/0 (socket says 172.21.15.82:33456) 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 -- 172.21.15.82:0/2976451892 learned_addr learned my addr 172.21.15.82:0/2976451892 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:57.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 -- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f0159fb0 msgr2=0x7f72f01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72edd9b640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 0x7f72f01538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72eed9d640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f0159fb0 0x7f72f01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f0159fb0 0x7f72f01548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 -- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 msgr2=0x7f72f01538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 0x7f72f01538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 -- 172.21.15.82:0/2976451892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f0157b30 con 0x7f72f00b26f0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72edd9b640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 0x7f72f01538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72ee59c640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e930 tx=0x7f72e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0017070 con 0x7f72f00b26f0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e000ca00 con 0x7f72f00b26f0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e000cdb0 con 0x7f72f00b26f0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f0156e30 con 0x7f72f00b26f0 2024-04-25T10:08:57.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f01577e0 con 0x7f72f00b26f0 2024-04-25T10:08:57.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f004efa0 con 0x7f72f00b26f0 2024-04-25T10:08:57.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72e001a070 con 0x7f72f00b26f0 2024-04-25T10:08:57.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 0x7f72d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:57.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72e00854c0 con 0x7f72f00b26f0 2024-04-25T10:08:57.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72edd9b640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 0x7f72d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:57.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.904+0000 7f72edd9b640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 0x7f72d0069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72e40af000 tx=0x7f72e40a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:57.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:57.908+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72e0056490 con 0x7f72f00b26f0 2024-04-25T10:08:58.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.124+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f72f00ae440 con 0x7f72f00b26f0 2024-04-25T10:08:58.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.128+0000 7f72deffd640 1 -- 172.21.15.82:0/2976451892 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f72e0055d90 con 0x7f72f00b26f0 2024-04-25T10:08:58.134 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:08:58.134 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:08:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:08:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:08:58.135 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:58.135 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 msgr2=0x7f72d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 0x7f72d0069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72e40af000 tx=0x7f72e40a7000 comp rx=0 tx=0).stop 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 msgr2=0x7f72f00b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72e000e930 tx=0x7f72e000edf0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 shutdown_connections 2024-04-25T10:08:58.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f72d0066f60 0x7f72d0069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72f0159fb0 0x7f72f01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f72f0159bd0 0x7f72f01538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 --2- 172.21.15.82:0/2976451892 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72f00b26f0 0x7f72f00b04d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 >> 172.21.15.82:0/2976451892 conn(0x7f72f006ae60 msgr2=0x7f72f006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 shutdown_connections 2024-04-25T10:08:58.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.132+0000 7f72f4c10640 1 -- 172.21.15.82:0/2976451892 wait complete. 2024-04-25T10:08:58.151 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.260+0000 7f49c65f4640 1 Processor -- start 2024-04-25T10:08:58.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.260+0000 7f49c65f4640 1 -- start start 2024-04-25T10:08:58.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f49c00d07f0 0x7f49c00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f49c00d1cd0 0x7f49c006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c006be60 con 0x7f49c015da80 2024-04-25T10:08:58.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c006c440 con 0x7f49c00d1cd0 2024-04-25T10:08:58.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c015de60 con 0x7f49c00d07f0 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49bffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f49c015da80 0x7f49c00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36736/0 (socket says 172.21.15.82:36736) 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49bffff640 1 -- 172.21.15.82:0/16722607 learned_addr learned my addr 172.21.15.82:0/16722607 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 --2- 172.21.15.82:0/16722607 >> v2:172.21.15.82:3300/0 conn(0x7f49c00d1cd0 0x7f49c006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 -- 172.21.15.82:0/16722607 >> v2:172.21.15.196:3300/0 conn(0x7f49c00d07f0 msgr2=0x7f49c00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49bf7fe640 1 --2- 172.21.15.82:0/16722607 >> v2:172.21.15.196:3300/0 conn(0x7f49c00d07f0 0x7f49c00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 --2- 172.21.15.82:0/16722607 >> v2:172.21.15.196:3300/0 conn(0x7f49c00d07f0 0x7f49c00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49beffd640 1 -- 172.21.15.82:0/16722607 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199490923 0 0) 0x7f49c006be60 con 0x7f49c015da80 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 -- 172.21.15.82:0/16722607 >> v1:172.21.15.82:6789/0 conn(0x7f49c015da80 legacy=0x7f49c00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 -- 172.21.15.82:0/16722607 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00d2450 con 0x7f49c00d1cd0 2024-04-25T10:08:58.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c4b91640 1 --2- 172.21.15.82:0/16722607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c00d1cd0 0x7f49c006c8d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f49c008d880 tx=0x7f49b001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14c900bcdda3877 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49beffd640 1 -- 172.21.15.82:0/16722607 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00a7070 con 0x7f49c00d1cd0 2024-04-25T10:08:58.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49beffd640 1 -- 172.21.15.82:0/16722607 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b00b0070 con 0x7f49c00d1cd0 2024-04-25T10:08:58.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49beffd640 1 -- 172.21.15.82:0/16722607 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00ab890 con 0x7f49c00d1cd0 2024-04-25T10:08:58.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 -- 172.21.15.82:0/16722607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c00d1cd0 msgr2=0x7f49c006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.264+0000 7f49c65f4640 1 --2- 172.21.15.82:0/16722607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c00d1cd0 0x7f49c006c8d0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f49c008d880 tx=0x7f49b001c450 comp rx=0 tx=0).stop 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/16722607 shutdown_connections 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 --2- 172.21.15.82:0/16722607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c00d1cd0 0x7f49c006c8d0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 --2- 172.21.15.82:0/16722607 >> v2:172.21.15.196:3300/0 conn(0x7f49c00d07f0 0x7f49c00d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/16722607 >> 172.21.15.82:0/16722607 conn(0x7f49c008b000 msgr2=0x7f49c00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/16722607 shutdown_connections 2024-04-25T10:08:58.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/16722607 wait complete. 2024-04-25T10:08:58.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 Processor -- start 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- start start 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c00d07f0 0x7f49c00ca8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 0x7f49c00cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c006d7a0 con 0x7f49c015da80 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c006d620 con 0x7f49c00d1cd0 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c006d920 con 0x7f49c00d07f0 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33466/0 (socket says 172.21.15.82:33466) 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 -- 172.21.15.82:0/2206124416 learned_addr learned my addr 172.21.15.82:0/2206124416 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bffff640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c00d07f0 0x7f49c00ca8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bf7fe640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 0x7f49c00cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 -- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c00d07f0 msgr2=0x7f49c00ca8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c00d07f0 0x7f49c00ca8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 -- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 msgr2=0x7f49c00cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 0x7f49c00cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 -- 172.21.15.82:0/2206124416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00ccaa0 con 0x7f49c015da80 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bf7fe640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 0x7f49c00cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c4b91640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f49b0019060 tx=0x7f49b001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00bf610 con 0x7f49c015da80 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b00bf7b0 con 0x7f49c015da80 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00ccd80 con 0x7f49c015da80 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00bfba0 con 0x7f49c015da80 2024-04-25T10:08:58.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c00cd340 con 0x7f49c015da80 2024-04-25T10:08:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.268+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c004efa0 con 0x7f49c015da80 2024-04-25T10:08:58.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.272+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49b00bd020 con 0x7f49c015da80 2024-04-25T10:08:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.272+0000 7f49bcff9640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 0x7f49a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.272+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49b011fbd0 con 0x7f49c015da80 2024-04-25T10:08:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.272+0000 7f49bffff640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 0x7f49a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:58.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.272+0000 7f49bffff640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 0x7f49a0069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b8005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:58.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.276+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49b0016070 con 0x7f49c015da80 2024-04-25T10:08:58.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.460+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c00cdcb0 con 0x7f49c015da80 2024-04-25T10:08:58.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.460+0000 7f49bcff9640 1 -- 172.21.15.82:0/2206124416 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f49b00f0b70 con 0x7f49c015da80 2024-04-25T10:08:58.467 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:08:58.467 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:08:51.280173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2865291678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2865291678},{"type":"v1","addr":"172.21.15.82:6837","nonce":2865291678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:08:58.467 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 56 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 msgr2=0x7f49a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 0x7f49a0069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b8005b80 comp rx=0 tx=0).stop 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 msgr2=0x7f49c0152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f49b0019060 tx=0x7f49b001ac90 comp rx=0 tx=0).stop 2024-04-25T10:08:58.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 shutdown_connections 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f49a0066f60 0x7f49a0069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f49c015da80 0x7f49c0152120 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f49c00d1cd0 0x7f49c00cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 --2- 172.21.15.82:0/2206124416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f49c00d07f0 0x7f49c00ca8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 >> 172.21.15.82:0/2206124416 conn(0x7f49c008b000 msgr2=0x7f49c0159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 shutdown_connections 2024-04-25T10:08:58.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:58.464+0000 7f49c65f4640 1 -- 172.21.15.82:0/2206124416 wait complete. 2024-04-25T10:08:58.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:08:51.280173+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7794': {'gid': 7794, 'name': 'c', 'rank': 0, 'incarnation': 54, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/2865291678', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 2865291678}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2865291678}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:58.887 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:58.888 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:08:59.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 Processor -- start 2024-04-25T10:08:59.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 -- start start 2024-04-25T10:08:59.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8015da80 0x7efe800d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 0x7efe8006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8006be60 con 0x7efe800d1cd0 2024-04-25T10:08:59.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8006c440 con 0x7efe8015da80 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe868ad640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8015de60 con 0x7efe800d07f0 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8015da80 0x7efe800d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8015da80 0x7efe800d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33468/0 (socket says 172.21.15.82:33468) 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 -- 172.21.15.82:0/2454313700 learned_addr learned my addr 172.21.15.82:0/2454313700 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 -- 172.21.15.82:0/2454313700 >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 msgr2=0x7efe8006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2454313700 >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 0x7efe8006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2454313700 >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 0x7efe8006c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 -- 172.21.15.82:0/2454313700 >> v1:172.21.15.196:6789/0 conn(0x7efe800d1cd0 legacy=0x7efe800d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe7ffff640 1 -- 172.21.15.82:0/2454313700 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800d2450 con 0x7efe8015da80 2024-04-25T10:08:59.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.016+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2454313700 >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 0x7efe8006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2454313700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800d4230 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7122dfa99035b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe7f7fe640 1 -- 172.21.15.82:0/2454313700 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700ab650 con 0x7efe8015da80 2024-04-25T10:08:59.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe7f7fe640 1 -- 172.21.15.82:0/2454313700 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700b0070 con 0x7efe8015da80 2024-04-25T10:08:59.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe7f7fe640 1 -- 172.21.15.82:0/2454313700 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700aba10 con 0x7efe8015da80 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 -- 172.21.15.82:0/2454313700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 msgr2=0x7efe800d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 --2- 172.21.15.82:0/2454313700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800d4230 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 -- 172.21.15.82:0/2454313700 shutdown_connections 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 --2- 172.21.15.82:0/2454313700 >> v2:172.21.15.196:3300/0 conn(0x7efe800d07f0 0x7efe8006c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 --2- 172.21.15.82:0/2454313700 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800d4230 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 -- 172.21.15.82:0/2454313700 >> 172.21.15.82:0/2454313700 conn(0x7efe8008b000 msgr2=0x7efe800d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 -- 172.21.15.82:0/2454313700 shutdown_connections 2024-04-25T10:08:59.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.020+0000 7efe868ad640 1 -- 172.21.15.82:0/2454313700 wait complete. 2024-04-25T10:08:59.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 Processor -- start 2024-04-25T10:08:59.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- start start 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe800d07f0 0x7efe800b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 0x7efe801582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006d7a0 con 0x7efe8015da80 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006d620 con 0x7efe800d07f0 2024-04-25T10:08:59.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006d920 con 0x7efe800d1cd0 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33480/0 (socket says 172.21.15.82:33480) 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 -- 172.21.15.82:0/2348354452 learned_addr learned my addr 172.21.15.82:0/2348354452 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe77fff640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 0x7efe801582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe800d07f0 0x7efe800b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 -- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 msgr2=0x7efe801582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 0x7efe801582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 -- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe800d07f0 msgr2=0x7efe800b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe800d07f0 0x7efe800b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 -- 172.21.15.82:0/2348354452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe800b6a60 con 0x7efe8015da80 2024-04-25T10:08:59.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe77fff640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 0x7efe801582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efe7800cef0 tx=0x7efe7800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe7d7fa640 1 -- 172.21.15.82:0/2348354452 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7801a070 con 0x7efe8015da80 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe7d7fa640 1 -- 172.21.15.82:0/2348354452 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7800d860 con 0x7efe8015da80 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe800b6d40 con 0x7efe8015da80 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe7d7fa640 1 -- 172.21.15.82:0/2348354452 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800dc50 con 0x7efe8015da80 2024-04-25T10:08:59.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.024+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe800b7300 con 0x7efe8015da80 2024-04-25T10:08:59.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7d7fa640 1 -- 172.21.15.82:0/2348354452 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe7801d070 con 0x7efe8015da80 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7d7fa640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 0x7efe60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7d7fa640 1 -- 172.21.15.82:0/2348354452 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe78085b60 con 0x7efe8015da80 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 0x7efe60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 0x7efe60069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efe700af000 tx=0x7efe70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 Processor -- start 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 -- start start 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911279f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 0x560d91128310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d9100f5f0 con 0x560d91171920 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d90f76da0 con 0x560d91127f30 2024-04-25T10:08:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d90eff150 con 0x560d91172240 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33482/0 (socket says 172.21.15.82:33482) 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 -- 172.21.15.82:0/497498572 learned_addr learned my addr 172.21.15.82:0/497498572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe84e4a640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 0x560d91128310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe77fff640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911279f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 -- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 msgr2=0x560d911279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911279f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 -- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 msgr2=0x560d91128310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 0x560d91128310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 -- 172.21.15.82:0/497498572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d90f18ca0 con 0x560d91171920 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe77fff640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe84e4a640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 0x560d91128310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe7ffff640 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7efe7001a0e0 tx=0x7efe70004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cd92888d6b9ba0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.028+0000 7efe75ffb640 1 -- 172.21.15.82:0/497498572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700abdf0 con 0x560d91171920 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe75ffb640 1 -- 172.21.15.82:0/497498572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700b0040 con 0x560d91171920 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe75ffb640 1 -- 172.21.15.82:0/497498572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700b4ac0 con 0x560d91171920 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/497498572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 msgr2=0x560d91171d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7efe7001a0e0 tx=0x7efe70004740 comp rx=0 tx=0).stop 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/497498572 shutdown_connections 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911279f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91127f30 0x560d91128310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- 172.21.15.82:0/497498572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91171920 0x560d91171d00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/497498572 >> 172.21.15.82:0/497498572 conn(0x560d9112e2a0 msgr2=0x560d911262d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/497498572 shutdown_connections 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/497498572 wait complete. 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- start start 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 Processor -- start 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 0x560d91178e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d90f31990 con 0x560d91172240 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d90f72e40 con 0x560d91171920 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d90f02900 con 0x560d91127f30 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 10 client.0 _ll_register_callbacks cb 0x560d9102a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:59.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33488/0 (socket says 172.21.15.82:33488) 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 -- 172.21.15.82:0/2119955138 learned_addr learned my addr 172.21.15.82:0/2119955138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe77fff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 -- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 msgr2=0x560d91178e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 0x560d91178e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 0x560d91178e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 -- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 msgr2=0x560d911793a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911793a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 -- 172.21.15.82:0/2119955138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d90f2f630 con 0x560d91172240 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe77fff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911793a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 0x560d91178e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe84e4a640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7efe7805e4e0 tx=0x7efe7805a510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149f8f979c9a5f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7805e580 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe700b0040 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe7805e720 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe78074050 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 10 client.7953 fetch_fsmap learned FSMap version 56 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x560d90eff3a0 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700b0040 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe780741f0 con 0x560d91172240 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 10 client.7953 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 10 client.7953 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:08:59.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 20 client.7953 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9112d160 0x560d91182ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 4 client.7953 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 --> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d90f12a50 con 0x560d9112d160 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9112d160 0x560d91182ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe7ffff640 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9112d160 0x560d91182ec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da499b62c3eed71 server_cookie=5e22221dad549dd4 in_seq=0 out_seq=0 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.032+0000 7efe5effd640 10 client.7953 ms_handle_connect on v2:172.21.15.82:6836/2865291678 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe5effd640 1 -- 172.21.15.82:0/2119955138 <== mds.0 v2:172.21.15.82:6836/2865291678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d90f12a50 con 0x560d9112d160 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe5effd640 10 client.7953 handle_command_reply: tid=0 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 client.7953 shutdown 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 msgr2=0x560d9112baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7efe7805e4e0 tx=0x7efe7805a510 comp rx=0 tx=0).stop 2024-04-25T10:08:59.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 shutdown_connections 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9112d160 0x560d91182ec0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91172240 0x560d9112baa0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911793a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 --2- 172.21.15.82:0/2119955138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91127f30 0x560d91178e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 >> 172.21.15.82:0/2119955138 conn(0x560d9112e2a0 msgr2=0x560d911269e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe5effd640 0 client.7953 ms_handle_reset on v2:172.21.15.82:6836/2865291678 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 shutdown_connections 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 1 -- 172.21.15.82:0/2119955138 wait complete. 2024-04-25T10:08:59.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.036+0000 7efe89a1a000 20 client.7953 trim_cache size 0 max 16384 2024-04-25T10:08:59.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.048+0000 7efe89a1a000 1 Processor -- start 2024-04-25T10:08:59.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.048+0000 7efe89a1a000 1 -- start start 2024-04-25T10:08:59.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91171920 0x560d91170220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 0x560d9117ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe80202000 con 0x560d91127f30 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe800c7ba0 con 0x560d91172240 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe80201a80 con 0x560d91171920 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33496/0 (socket says 172.21.15.82:33496) 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/3336919426 learned_addr learned my addr 172.21.15.82:0/3336919426 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe84e4a640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 0x560d9117ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91171920 msgr2=0x560d91170220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe77fff640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91171920 0x560d91170220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91171920 0x560d91170220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 msgr2=0x560d9117ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 0x560d9117ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/3336919426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d91172070 con 0x560d91127f30 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe84e4a640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 0x560d9117ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7efe7001c990 tx=0x7efe70009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56dca75b056d5100 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe757fa640 1 -- 172.21.15.82:0/3336919426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700cde60 con 0x560d91127f30 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe757fa640 1 -- 172.21.15.82:0/3336919426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700b0070 con 0x560d91127f30 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe757fa640 1 -- 172.21.15.82:0/3336919426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700b4a40 con 0x560d91127f30 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- 172.21.15.82:0/3336919426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 msgr2=0x560d9116fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7efe7001c990 tx=0x7efe70009240 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- 172.21.15.82:0/3336919426 shutdown_connections 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91171920 0x560d91170220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91172240 0x560d9117ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- 172.21.15.82:0/3336919426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9116fce0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- 172.21.15.82:0/3336919426 >> 172.21.15.82:0/3336919426 conn(0x560d9112e2a0 msgr2=0x560d9112b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- 172.21.15.82:0/3336919426 shutdown_connections 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- 172.21.15.82:0/3336919426 wait complete. 2024-04-25T10:08:59.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- start start 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 Processor -- start 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d90f7b3d0 con 0x560d91127f30 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d91032d50 con 0x560d91171920 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d90f11980 con 0x560d91172240 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe89a1a000 10 client.0 _ll_register_callbacks cb 0x7efe700096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33500/0 (socket says 172.21.15.82:33500) 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/1620876355 learned_addr learned my addr 172.21.15.82:0/1620876355 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe84e4a640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe77fff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 msgr2=0x560d911cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 msgr2=0x560d911cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911cd7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 -- 172.21.15.82:0/1620876355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d90ff8600 con 0x560d91127f30 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe7ffff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7efe70000c10 tx=0x7efe700046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1ade1293685b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.052+0000 7efe5effd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700dd060 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe700c4040 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe700b0070 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe700e3050 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 10 client.7959 fetch_fsmap learned FSMap version 56 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x560d90f0b9b0 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700b0070 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe700e6020 con 0x560d91127f30 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 10 client.7959 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:08:59.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 10 client.7959 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 20 client.7959 populate_metadata read hostname 'smithi082' 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9117eca0 0x560d911ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 4 client.7959 mds_command: new command op to 7794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 --> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560d90f842e0 con 0x560d9117eca0 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe77fff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9117eca0 0x560d911ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 10 client.7959 ms_handle_connect on v2:172.21.15.82:6836/2865291678 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe77fff640 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9117eca0 0x560d911ce250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5addbe72bc8f4a0c server_cookie=e5dd1a9b968b015e in_seq=0 out_seq=0 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 1 -- 172.21.15.82:0/1620876355 <== mds.0 v2:172.21.15.82:6836/2865291678 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x560d90f842e0 con 0x560d9117eca0 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 10 client.7959 handle_command_reply: tid=0 2024-04-25T10:08:59.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 client.7959 shutdown 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 msgr2=0x560d9117e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7efe70000c10 tx=0x7efe700046d0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 shutdown_connections 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:6836/2865291678,v1:172.21.15.82:6837/2865291678] conn(0x560d9117eca0 0x560d911ce250 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560d91172240 0x560d911cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560d91171920 0x560d911cd7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 --2- 172.21.15.82:0/1620876355 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560d91127f30 0x560d9117e760 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe5effd640 0 client.7959 ms_handle_reset on v2:172.21.15.82:6836/2865291678 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 >> 172.21.15.82:0/1620876355 conn(0x560d9112e2a0 msgr2=0x560d91179a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 shutdown_connections 2024-04-25T10:08:59.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 1 -- 172.21.15.82:0/1620876355 wait complete. 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe89a1a000 20 client.7959 trim_cache size 0 max 16384 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7794, 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:08:59.072 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 9.1701433140000006, 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 56, 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 91, 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 14.198556807999999 2024-04-25T10:08:59.073 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:08:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 msgr2=0x7efe60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 0x7efe60069420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efe700af000 tx=0x7efe70008040 comp rx=0 tx=0).stop 2024-04-25T10:08:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 msgr2=0x7efe800b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7efe7800cef0 tx=0x7efe7800a710 comp rx=0 tx=0).stop 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 shutdown_connections 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe60066f60 0x7efe60069420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe800d1cd0 0x7efe801582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8015da80 0x7efe800b3f20 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 --2- 172.21.15.82:0/2348354452 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe800d07f0 0x7efe800b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 >> 172.21.15.82:0/2348354452 conn(0x7efe8008b000 msgr2=0x7efe800b0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 shutdown_connections 2024-04-25T10:08:59.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.056+0000 7efe868ad640 1 -- 172.21.15.82:0/2348354452 wait complete. 2024-04-25T10:08:59.077 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:59.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 Processor -- start 2024-04-25T10:08:59.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 -- start start 2024-04-25T10:08:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c7c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c06b5b0 con 0x7f5c7c0cc130 2024-04-25T10:08:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7c15f360 con 0x7f5c7c15ca90 2024-04-25T10:08:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c840db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c15ff70 con 0x7f5c7c06ac90 2024-04-25T10:08:59.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5c7c0cc130 0x7f5c7c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36752/0 (socket says 172.21.15.82:36752) 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 -- 172.21.15.82:0/461918819 learned_addr learned my addr 172.21.15.82:0/461918819 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c81e77640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c7c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178715461 0 0) 0x7f5c7c06b5b0 con 0x7f5c7c0cc130 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c68003530 con 0x7f5c7c0cc130 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226682836 0 0) 0x7f5c68003530 con 0x7f5c7c0cc130 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 msgr2=0x7f5c7c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c7c06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 >> v1:172.21.15.196:6789/0 conn(0x7f5c7c15ca90 legacy=0x7f5c7c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c15f640 con 0x7f5c7c0cc130 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5c74001a00 con 0x7f5c7c0cc130 2024-04-25T10:08:59.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 >> v1:172.21.15.82:6789/0 conn(0x7f5c7c0cc130 legacy=0x7f5c7c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c68003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c68003fe0 0x7f5c68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6800bb00 con 0x7f5c68004d50 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6800bcb0 con 0x7f5c68003fe0 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6800beb0 con 0x7f5c7c06ac90 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c81e77640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c68003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c81676640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c68003fe0 0x7f5c68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 -- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 msgr2=0x7f5c68003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06ac90 0x7f5c68003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 -- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c68003fe0 msgr2=0x7f5c68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c68003fe0 0x7f5c68004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.184+0000 7f5c82678640 1 -- 172.21.15.82:0/461918819 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68003530 con 0x7f5c68004d50 2024-04-25T10:08:59.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c82678640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c680051d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0cc560 tx=0x7f5c7401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c9e9664cd06232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400a4c0 con 0x7f5c68004d50 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c740ad630 con 0x7f5c68004d50 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c80e75640 1 -- 172.21.15.82:0/461918819 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c740ada20 con 0x7f5c68004d50 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- 172.21.15.82:0/461918819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 msgr2=0x7f5c680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 --2- 172.21.15.82:0/461918819 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c680051d0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0cc560 tx=0x7f5c7401d3c0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c82678640 1 -- 172.21.15.82:0/461918819 reap_dead start 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- 172.21.15.82:0/461918819 shutdown_connections 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- 172.21.15.82:0/461918819 >> 172.21.15.82:0/461918819 conn(0x7f5c7c099a70 msgr2=0x7f5c7c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- 172.21.15.82:0/461918819 shutdown_connections 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- 172.21.15.82:0/461918819 wait complete. 2024-04-25T10:08:59.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 Processor -- start 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- start start 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c06e5c0 0x7f5c7c14b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06fb40 0x7f5c7c0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c0d1320 con 0x7f5c68004d50 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.188+0000 7f5c840db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c15fc00 con 0x7f5c7c06e5c0 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c840db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c06eaa0 con 0x7f5c7c06fb40 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33524/0 (socket says 172.21.15.82:33524) 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 -- 172.21.15.82:0/2797825278 learned_addr learned my addr 172.21.15.82:0/2797825278 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c82678640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06fb40 0x7f5c7c0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81676640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c06e5c0 0x7f5c7c14b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 -- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06fb40 msgr2=0x7f5c7c0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06fb40 0x7f5c7c0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 -- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c06e5c0 msgr2=0x7f5c7c14b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c06e5c0 0x7f5c7c14b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 -- 172.21.15.82:0/2797825278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c06ed80 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81e77640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5c780115e0 tx=0x7f5c78011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800f8f0 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7800fa90 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c06f060 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800fe60 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c06f620 con 0x7f5c68004d50 2024-04-25T10:08:59.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c06b7c0 con 0x7f5c68004d50 2024-04-25T10:08:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c78015460 con 0x7f5c68004d50 2024-04-25T10:08:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 0x7f5c60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c78085ab0 con 0x7f5c68004d50 2024-04-25T10:08:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81676640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 0x7f5c60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:59.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.192+0000 7f5c81676640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 0x7f5c60069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c09e5b0 tx=0x7f5c6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:59.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.196+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c78019030 con 0x7f5c68004d50 2024-04-25T10:08:59.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.388+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5c7c0b6480 con 0x7f5c68004d50 2024-04-25T10:08:59.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.388+0000 7f5c727fc640 1 -- 172.21.15.82:0/2797825278 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5c780569c0 con 0x7f5c68004d50 2024-04-25T10:08:59.394 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 msgr2=0x7f5c60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 0x7f5c60069420 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c09e5b0 tx=0x7f5c6c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 msgr2=0x7f5c7c156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f5c780115e0 tx=0x7f5c78011aa0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 shutdown_connections 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5c60066f60 0x7f5c60069420 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5c7c06fb40 0x7f5c7c0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5c7c06e5c0 0x7f5c7c14b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 --2- 172.21.15.82:0/2797825278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5c68004d50 0x7f5c7c156ef0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 >> 172.21.15.82:0/2797825278 conn(0x7f5c7c099a70 msgr2=0x7f5c7c157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 shutdown_connections 2024-04-25T10:08:59.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:08:59.392+0000 7f5c840db640 1 -- 172.21.15.82:0/2797825278 wait complete. 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:59.412 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:59.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.413 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:08:59.418 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.419 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-04-25T10:08:59.420 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:59.421 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:08:59.422 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:59.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.423 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:59.476 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:08:59.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.477 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:08:59.528 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:08:59.528 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:59.528 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:59.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.529 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:08:59.530 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:08:59.530 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:08:59.530 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:08:59.530 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:08:59.530 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:08:59.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.604 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:08:59.652 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:59.652 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:59.652 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:59.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.652 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:08:59.706 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:08:59.706 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:59.706 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:59.751 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:59.751 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:59.752 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:59.800 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:08:59.858 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:08:59.905 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:08:59.906 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:59.907 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:59.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:59.908 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:08:59.956 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:08:59.956 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:09:00.001 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:09:00.013 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:09:00.064 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:00.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:00.066 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:00.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:00.067 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:09:00.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:09:00.072+0000 7fc85bb9d580 -1 init, newargv = 0x55944dd09500 newargc=15 2024-04-25T10:09:00.078 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[48335]: starting ceph client 2024-04-25T10:09:00.085 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[48335]: starting fuse 2024-04-25T10:09:01.185 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:09:01.202 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:01.249 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:09:01.249 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:09:01.249 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:09:01.249 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:09:01.249 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:09:01.250 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:01.251 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:09:01.252 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:01.253 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:01.253 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:01.253 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:01.253 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:01.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.254 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:09:01.300 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:09:01.300 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:01.301 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:09:01.302 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:09:01.303 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:01.303 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:09:01.303 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:09:01.422 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.48335.asok 2024-04-25T10:09:01.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:01.435 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48335.asok 2024-04-25T10:09:01.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48335.asok status 2024-04-25T10:09:01.593 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:01.593 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "48335", 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 7971, 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:09:01.594 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 7971 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1483010635 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1483010635 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.7971 192.168.0.1:0/1483010635", 2024-04-25T10:09:01.595 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/1483010635", 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 56, 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 91, 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:09:01.596 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:09:01.597 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:01.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.608 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:09:01.608 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:01.615 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:09:01.615 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:01.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.615 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:01.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.672 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T10:09:01.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.729 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T10:09:01.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.781 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T10:09:01.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:01.833 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T10:09:01.891 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:09:01.891 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:09:01.892 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:09:01.892 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:01.892 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T10:09:01.892 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:09:02.002 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:09:02.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:02.012 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:09:02.012 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:02.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:02.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 Processor -- start 2024-04-25T10:09:02.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 -- start start 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc95415da10 0x7fc954071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc954071e90 0x7fc954071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540adbd0 con 0x7fc9540b24a0 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9540ae860 con 0x7fc954071e90 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc958a9a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95415e210 con 0x7fc95415da10 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc954071e90 0x7fc954071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc954071e90 0x7fc954071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33528/0 (socket says 172.21.15.82:33528) 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 -- 172.21.15.82:0/544870296 learned_addr learned my addr 172.21.15.82:0/544870296 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc95259c640 1 --2- 172.21.15.82:0/544870296 >> v2:172.21.15.196:3300/0 conn(0x7fc95415da10 0x7fc954071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 -- 172.21.15.82:0/544870296 >> v2:172.21.15.196:3300/0 conn(0x7fc95415da10 msgr2=0x7fc954071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 --2- 172.21.15.82:0/544870296 >> v2:172.21.15.196:3300/0 conn(0x7fc95415da10 0x7fc954071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 -- 172.21.15.82:0/544870296 >> v1:172.21.15.196:6789/0 conn(0x7fc9540b24a0 legacy=0x7fc9541614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.236+0000 7fc952d9d640 1 -- 172.21.15.82:0/544870296 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95415e4f0 con 0x7fc954071e90 2024-04-25T10:09:02.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc952d9d640 1 --2- 172.21.15.82:0/544870296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc954071570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b43e0 tx=0x7fc94c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bb3d9e9aaa3cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95159a640 1 -- 172.21.15.82:0/544870296 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0a5560 con 0x7fc954071e90 2024-04-25T10:09:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95159a640 1 -- 172.21.15.82:0/544870296 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c0a5700 con 0x7fc954071e90 2024-04-25T10:09:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95159a640 1 -- 172.21.15.82:0/544870296 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0a5ad0 con 0x7fc954071e90 2024-04-25T10:09:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- 172.21.15.82:0/544870296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 msgr2=0x7fc954071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- 172.21.15.82:0/544870296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc954071570 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b43e0 tx=0x7fc94c01c870 comp rx=0 tx=0).stop 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- 172.21.15.82:0/544870296 shutdown_connections 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- 172.21.15.82:0/544870296 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc954071570 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- 172.21.15.82:0/544870296 >> v2:172.21.15.196:3300/0 conn(0x7fc95415da10 0x7fc954071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- 172.21.15.82:0/544870296 >> 172.21.15.82:0/544870296 conn(0x7fc9540b11a0 msgr2=0x7fc9540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- 172.21.15.82:0/544870296 shutdown_connections 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- 172.21.15.82:0/544870296 wait complete. 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 Processor -- start 2024-04-25T10:09:02.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- start start 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9540b24a0 0x7fc954153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 0x7fc9541503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95416aa10 con 0x7fc954071e90 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95416a890 con 0x7fc95415da10 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc958a9a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95416ab90 con 0x7fc9540b24a0 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33536/0 (socket says 172.21.15.82:33536) 2024-04-25T10:09:02.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95259c640 1 -- 172.21.15.82:0/2163654004 learned_addr learned my addr 172.21.15.82:0/2163654004 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc952d9d640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 0x7fc9541503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc951d9b640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9540b24a0 0x7fc954153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.240+0000 7fc95259c640 1 -- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9540b24a0 msgr2=0x7fc954153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc95259c640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9540b24a0 0x7fc954153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc95259c640 1 -- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 msgr2=0x7fc9541503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc95259c640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 0x7fc9541503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc95259c640 1 -- 172.21.15.82:0/2163654004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9541596e0 con 0x7fc954071e90 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc952d9d640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 0x7fc9541503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc95259c640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c7f0 tx=0x7fc94400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944015070 con 0x7fc954071e90 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944007820 con 0x7fc954071e90 2024-04-25T10:09:02.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9541599c0 con 0x7fc954071e90 2024-04-25T10:09:02.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944007c50 con 0x7fc954071e90 2024-04-25T10:09:02.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954159f80 con 0x7fc954071e90 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95415a180 con 0x7fc954071e90 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc944018070 con 0x7fc954071e90 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 0x7fc934069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc944085310 con 0x7fc954071e90 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc951d9b640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 0x7fc934069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.244+0000 7fc951d9b640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 0x7fc934069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc948004e30 tx=0x7fc948009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.248+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc95415a180 con 0x7fc954071e90 2024-04-25T10:09:02.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.408+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9540b04c0 con 0x7fc954071e90 2024-04-25T10:09:02.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.408+0000 7fc942ffd640 1 -- 172.21.15.82:0/2163654004 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fc944056190 con 0x7fc954071e90 2024-04-25T10:09:02.415 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:02.415 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:50.889383+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:02.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 msgr2=0x7fc934069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 0x7fc934069450 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc948004e30 tx=0x7fc948009040 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 msgr2=0x7fc9540afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fc94400c7f0 tx=0x7fc94400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 shutdown_connections 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc934066f90 0x7fc934069450 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc9540b24a0 0x7fc954153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc95415da10 0x7fc9541503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 --2- 172.21.15.82:0/2163654004 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc954071e90 0x7fc9540afee0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 >> 172.21.15.82:0/2163654004 conn(0x7fc9540b11a0 msgr2=0x7fc954157310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 shutdown_connections 2024-04-25T10:09:02.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.412+0000 7fc958a9a640 1 -- 172.21.15.82:0/2163654004 wait complete. 2024-04-25T10:09:02.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:02.433 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:02.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 Processor -- start 2024-04-25T10:09:02.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 -- start start 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fda380d4230 0x7fda380d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3806be60 con 0x7fda3815da80 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda380d4650 con 0x7fda380d07f0 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda3d696640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda380d2210 con 0x7fda380d4230 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda36ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fda3815da80 0x7fda380d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36764/0 (socket says 172.21.15.82:36764) 2024-04-25T10:09:02.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda377fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fda380d4230 0x7fda380d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda36ffd640 1 -- 172.21.15.82:0/878887762 learned_addr learned my addr 172.21.15.82:0/878887762 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:02.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda377fe640 1 -- 172.21.15.82:0/878887762 >> v1:172.21.15.196:6789/0 conn(0x7fda380d07f0 legacy=0x7fda3806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda377fe640 1 -- 172.21.15.82:0/878887762 >> v1:172.21.15.82:6789/0 conn(0x7fda3815da80 legacy=0x7fda380d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda35ffb640 1 -- 172.21.15.82:0/878887762 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380307279 0 0) 0x7fda3806be60 con 0x7fda3815da80 2024-04-25T10:09:02.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.556+0000 7fda377fe640 1 -- 172.21.15.82:0/878887762 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3815de60 con 0x7fda380d4230 2024-04-25T10:09:02.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda377fe640 1 --2- 172.21.15.82:0/878887762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda380d4230 0x7fda380d1cd0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fda30009d90 tx=0x7fda3001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fd1575dd270219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda35ffb640 1 -- 172.21.15.82:0/878887762 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300b2070 con 0x7fda380d4230 2024-04-25T10:09:02.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda35ffb640 1 -- 172.21.15.82:0/878887762 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda300a57f0 con 0x7fda380d4230 2024-04-25T10:09:02.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda35ffb640 1 -- 172.21.15.82:0/878887762 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300a5bc0 con 0x7fda380d4230 2024-04-25T10:09:02.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- 172.21.15.82:0/878887762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda380d4230 msgr2=0x7fda380d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 --2- 172.21.15.82:0/878887762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda380d4230 0x7fda380d1cd0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fda30009d90 tx=0x7fda3001c930 comp rx=0 tx=0).stop 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- 172.21.15.82:0/878887762 shutdown_connections 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 --2- 172.21.15.82:0/878887762 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda380d4230 0x7fda380d1cd0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- 172.21.15.82:0/878887762 >> 172.21.15.82:0/878887762 conn(0x7fda3808b000 msgr2=0x7fda380d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- 172.21.15.82:0/878887762 shutdown_connections 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- 172.21.15.82:0/878887762 wait complete. 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 Processor -- start 2024-04-25T10:09:02.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.560+0000 7fda3d696640 1 -- start start 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 0x7fda381551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda380d4230 0x7fda380b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3806dc00 con 0x7fda3815da80 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3806da80 con 0x7fda380d4230 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3816b5c0 con 0x7fda380d07f0 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33548/0 (socket says 172.21.15.82:33548) 2024-04-25T10:09:02.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 -- 172.21.15.82:0/146181450 learned_addr learned my addr 172.21.15.82:0/146181450 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda367fc640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda380d4230 0x7fda380b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda36ffd640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 0x7fda381551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 -- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 msgr2=0x7fda381551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 0x7fda381551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 -- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda380d4230 msgr2=0x7fda380b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda380d4230 0x7fda380b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 -- 172.21.15.82:0/146181450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda38155700 con 0x7fda3815da80 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda36ffd640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 0x7fda381551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda377fe640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fda300b0040 tx=0x7fda300196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300b2040 con 0x7fda3815da80 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda300bf070 con 0x7fda3815da80 2024-04-25T10:09:02.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda381586f0 con 0x7fda3815da80 2024-04-25T10:09:02.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300b8870 con 0x7fda3815da80 2024-04-25T10:09:02.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda38158cb0 con 0x7fda3815da80 2024-04-25T10:09:02.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3804efa0 con 0x7fda3815da80 2024-04-25T10:09:02.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda300b8a10 con 0x7fda3815da80 2024-04-25T10:09:02.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 0x7fda24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda30120300 con 0x7fda3815da80 2024-04-25T10:09:02.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda36ffd640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 0x7fda24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.564+0000 7fda36ffd640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 0x7fda24069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda28000f70 tx=0x7fda28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.568+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda38155700 con 0x7fda3815da80 2024-04-25T10:09:02.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fda380b9570 con 0x7fda3815da80 2024-04-25T10:09:02.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda1f7fe640 1 -- 172.21.15.82:0/146181450 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fda30016520 con 0x7fda3815da80 2024-04-25T10:09:02.733 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 msgr2=0x7fda24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 0x7fda24069420 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fda28000f70 tx=0x7fda28009040 comp rx=0 tx=0).stop 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 msgr2=0x7fda38160230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fda300b0040 tx=0x7fda300196d0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 shutdown_connections 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fda24066f60 0x7fda24069420 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fda3815da80 0x7fda38160230 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fda380d4230 0x7fda380b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 --2- 172.21.15.82:0/146181450 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fda380d07f0 0x7fda381551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 >> 172.21.15.82:0/146181450 conn(0x7fda3808b000 msgr2=0x7fda380b3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 shutdown_connections 2024-04-25T10:09:02.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.728+0000 7fda3d696640 1 -- 172.21.15.82:0/146181450 wait complete. 2024-04-25T10:09:02.752 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:02.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.904+0000 7f71d22fd640 1 Processor -- start 2024-04-25T10:09:02.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.904+0000 7f71d22fd640 1 -- start start 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d22fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71cc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d22fd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0ca4c0 con 0x7f71cc0cf590 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d22fd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0cd690 con 0x7f71cc0c9ef0 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d22fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0ca780 con 0x7f71cc15ecb0 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71cbfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f71cc0cf590 0x7f71cc0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36772/0 (socket says 172.21.15.82:36772) 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71cbfff640 1 -- 172.21.15.82:0/3048661571 learned_addr learned my addr 172.21.15.82:0/3048661571 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71cb7fe640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71cc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147617384 0 0) 0x7f71cc0ca4c0 con 0x7f71cc0cf590 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71b4003530 con 0x7f71cc0cf590 2024-04-25T10:09:02.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2498750 0 0) 0x7f71b4003530 con 0x7f71cc0cf590 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 msgr2=0x7f71cc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71cc0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 >> v1:172.21.15.196:6789/0 conn(0x7f71cc0c9ef0 legacy=0x7f71cc0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc0cd970 con 0x7f71cc0cf590 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f71bc001680 con 0x7f71cc0cf590 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 >> v1:172.21.15.82:6789/0 conn(0x7f71cc0cf590 legacy=0x7f71cc0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71b4003a60 0x7f71b4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b4004d50 0x7f71b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b400ba80 con 0x7f71b4004d50 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b400bc00 con 0x7f71b4003a60 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b400bdb0 con 0x7f71cc15ecb0 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b4004d50 0x7f71b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71cbfff640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71b4003a60 0x7f71b4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71cb7fe640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 -- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 msgr2=0x7f71b4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71cc15ecb0 0x7f71b4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 -- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71b4003a60 msgr2=0x7f71b4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71b4003a60 0x7f71b4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.908+0000 7f71d089a640 1 -- 172.21.15.82:0/3048661571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4003530 con 0x7f71b4004d50 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d089a640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b4004d50 0x7f71b40051d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f71c4001f60 tx=0x7f71c401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae039c0825f96f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40b2070 con 0x7f71b4004d50 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c40a5770 con 0x7f71b4004d50 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71caffd640 1 -- 172.21.15.82:0/3048661571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4001b60 con 0x7f71b4004d50 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- 172.21.15.82:0/3048661571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b4004d50 msgr2=0x7f71b40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3048661571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71b4004d50 0x7f71b40051d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f71c4001f60 tx=0x7f71c401c880 comp rx=0 tx=0).stop 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d089a640 1 -- 172.21.15.82:0/3048661571 reap_dead start 2024-04-25T10:09:02.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- 172.21.15.82:0/3048661571 shutdown_connections 2024-04-25T10:09:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- 172.21.15.82:0/3048661571 >> 172.21.15.82:0/3048661571 conn(0x7f71cc06ae60 msgr2=0x7f71cc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- 172.21.15.82:0/3048661571 shutdown_connections 2024-04-25T10:09:02.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- 172.21.15.82:0/3048661571 wait complete. 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 Processor -- start 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- start start 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71b4004d50 0x7f71cc0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 0x7f71cc0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0cd440 con 0x7f71cc1580f0 2024-04-25T10:09:02.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc15f0f0 con 0x7f71cc073090 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d22fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0cdce0 con 0x7f71b4004d50 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33560/0 (socket says 172.21.15.82:33560) 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 -- 172.21.15.82:0/3280436908 learned_addr learned my addr 172.21.15.82:0/3280436908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d089a640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 0x7f71cc0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cbfff640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71b4004d50 0x7f71cc0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 -- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71b4004d50 msgr2=0x7f71cc0c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71b4004d50 0x7f71cc0c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 -- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 msgr2=0x7f71cc0af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 0x7f71cc0af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71cb7fe640 1 -- 172.21.15.82:0/3280436908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc1555b0 con 0x7f71cc1580f0 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.912+0000 7f71d089a640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 0x7f71cc0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71cb7fe640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f71c0011610 tx=0x7f71c0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000f8f0 con 0x7f71cc1580f0 2024-04-25T10:09:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c000fa90 con 0x7f71cc1580f0 2024-04-25T10:09:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc155310 con 0x7f71cc1580f0 2024-04-25T10:09:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c000fe40 con 0x7f71cc1580f0 2024-04-25T10:09:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc0cbf80 con 0x7f71cc1580f0 2024-04-25T10:09:02.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc04efa0 con 0x7f71cc1580f0 2024-04-25T10:09:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71c0015460 con 0x7f71cc1580f0 2024-04-25T10:09:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 0x7f71b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71c0085a60 con 0x7f71cc1580f0 2024-04-25T10:09:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71cbfff640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 0x7f71b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:02.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.916+0000 7f71cbfff640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 0x7f71b8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0017e0 tx=0x7f71bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:02.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:02.920+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71c0019030 con 0x7f71cc1580f0 2024-04-25T10:09:03.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.076+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f71cc0b4170 con 0x7f71cc1580f0 2024-04-25T10:09:03.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.076+0000 7f71c8ff9640 1 -- 172.21.15.82:0/3280436908 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f71c0056a50 con 0x7f71cc1580f0 2024-04-25T10:09:03.083 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:09:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 msgr2=0x7f71b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 0x7f71b8069420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0017e0 tx=0x7f71bc009040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 msgr2=0x7f71cc0cc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f71c0011610 tx=0x7f71c0011ad0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 shutdown_connections 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f71b8066f60 0x7f71b8069420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f71cc073090 0x7f71cc0af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f71cc1580f0 0x7f71cc0cc4a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 --2- 172.21.15.82:0/3280436908 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f71b4004d50 0x7f71cc0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 >> 172.21.15.82:0/3280436908 conn(0x7f71cc06ae60 msgr2=0x7f71cc06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 shutdown_connections 2024-04-25T10:09:03.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.080+0000 7f71d22fd640 1 -- 172.21.15.82:0/3280436908 wait complete. 2024-04-25T10:09:03.102 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 Processor -- start 2024-04-25T10:09:03.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 -- start start 2024-04-25T10:09:03.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70e415b6b0 0x7f70e40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e415ff50 con 0x7f70e40cf590 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70e40cbd20 con 0x7f70e4073010 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e8a47640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70e415ba90 con 0x7f70e415b6b0 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f70e40cf590 0x7f70e415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36776/0 (socket says 172.21.15.82:36776) 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 -- 172.21.15.82:0/4197777850 learned_addr learned my addr 172.21.15.82:0/4197777850 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e259c640 1 --2- 172.21.15.82:0/4197777850 >> v2:172.21.15.196:3300/0 conn(0x7f70e415b6b0 0x7f70e40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4022841055 0 0) 0x7f70e415ff50 con 0x7f70e40cf590 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70cc003530 con 0x7f70e40cf590 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1857262174 0 0) 0x7f70cc003530 con 0x7f70e40cf590 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 >> v2:172.21.15.196:3300/0 conn(0x7f70e415b6b0 msgr2=0x7f70e40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 --2- 172.21.15.82:0/4197777850 >> v2:172.21.15.196:3300/0 conn(0x7f70e415b6b0 0x7f70e40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 >> v1:172.21.15.196:6789/0 conn(0x7f70e4073010 legacy=0x7f70e415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e40cc000 con 0x7f70e40cf590 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f70d8001680 con 0x7f70e40cf590 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 >> v1:172.21.15.82:6789/0 conn(0x7f70e40cf590 legacy=0x7f70e415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70cc003a30 0x7f70cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 0x7f70cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc005240 0x7f70cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e415ba90 con 0x7f70cc004810 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc00c1a0 con 0x7f70cc003a30 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70cc00c3a0 con 0x7f70cc005240 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 0x7f70cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e259c640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70cc003a30 0x7f70cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 -- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc005240 msgr2=0x7f70cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e2d9d640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc005240 0x7f70cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc005240 0x7f70cc00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 -- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70cc003a30 msgr2=0x7f70cc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70cc003a30 0x7f70cc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e1d9b640 1 -- 172.21.15.82:0/4197777850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc003530 con 0x7f70cc004810 2024-04-25T10:09:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e259c640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70cc003a30 0x7f70cc003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.233 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.224+0000 7f70e2d9d640 1 -- 172.21.15.82:0/4197777850 reap_dead start 2024-04-25T10:09:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 0x7f70cc004c90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c210 tx=0x7f70d801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a390e72071e3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007d40 con 0x7f70cc004810 2024-04-25T10:09:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d8005b00 con 0x7f70cc004810 2024-04-25T10:09:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e159a640 1 -- 172.21.15.82:0/4197777850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8007070 con 0x7f70cc004810 2024-04-25T10:09:03.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 msgr2=0x7f70cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 0x7f70cc004c90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c210 tx=0x7f70d801f3f0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- 172.21.15.82:0/4197777850 shutdown_connections 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 --2- 172.21.15.82:0/4197777850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70cc004810 0x7f70cc004c90 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- 172.21.15.82:0/4197777850 >> 172.21.15.82:0/4197777850 conn(0x7f70e406ae60 msgr2=0x7f70e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- 172.21.15.82:0/4197777850 shutdown_connections 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- 172.21.15.82:0/4197777850 wait complete. 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 Processor -- start 2024-04-25T10:09:03.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- start start 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc004810 0x7f70e40caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e4152720 0x7f70e4152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cc8a0 con 0x7f70e415a7f0 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cc5c0 con 0x7f70e4152720 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e8a47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e40cba30 con 0x7f70cc004810 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e1d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33590/0 (socket says 172.21.15.82:33590) 2024-04-25T10:09:03.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e1d9b640 1 -- 172.21.15.82:0/1752762338 learned_addr learned my addr 172.21.15.82:0/1752762338 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e2d9d640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e4152720 0x7f70e4152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.228+0000 7f70e259c640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc004810 0x7f70e40caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 -- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc004810 msgr2=0x7f70e40caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc004810 0x7f70e40caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 -- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e4152720 msgr2=0x7f70e4152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e4152720 0x7f70e4152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 -- 172.21.15.82:0/1752762338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e415fd20 con 0x7f70e415a7f0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e1d9b640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f70d801f220 tx=0x7f70cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d80049f0 con 0x7f70e415a7f0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d80b6070 con 0x7f70e415a7f0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e40ce0e0 con 0x7f70e415a7f0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d8004d90 con 0x7f70e415a7f0 2024-04-25T10:09:03.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e40ce6a0 con 0x7f70e415a7f0 2024-04-25T10:09:03.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e404efa0 con 0x7f70e415a7f0 2024-04-25T10:09:03.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70d80b8020 con 0x7f70e415a7f0 2024-04-25T10:09:03.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 0x7f70c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f70d8122520 con 0x7f70e415a7f0 2024-04-25T10:09:03.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e259c640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 0x7f70c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.232+0000 7f70e259c640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 0x7f70c4069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f70e4050240 tx=0x7f70d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.236+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70d8019100 con 0x7f70e415a7f0 2024-04-25T10:09:03.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.384+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f70e40b2290 con 0x7f70e415a7f0 2024-04-25T10:09:03.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.384+0000 7f70d2ffd640 1 -- 172.21.15.82:0/1752762338 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f70d80f3430 con 0x7f70e415a7f0 2024-04-25T10:09:03.390 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 msgr2=0x7f70c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 0x7f70c4069420 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f70e4050240 tx=0x7f70d4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 msgr2=0x7f70e40cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f70d801f220 tx=0x7f70cc005840 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 shutdown_connections 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70c4066f60 0x7f70c4069420 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70e4152720 0x7f70e4152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70e415a7f0 0x7f70e40cd7b0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 --2- 172.21.15.82:0/1752762338 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70cc004810 0x7f70e40caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 >> 172.21.15.82:0/1752762338 conn(0x7f70e406ae60 msgr2=0x7f70e40b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 shutdown_connections 2024-04-25T10:09:03.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.388+0000 7f70e8a47640 1 -- 172.21.15.82:0/1752762338 wait complete. 2024-04-25T10:09:03.408 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:03.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 Processor -- start 2024-04-25T10:09:03.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 -- start start 2024-04-25T10:09:03.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2580b3f50 0x7fd25815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2580b24a0 0x7fd2580b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2580b2090 con 0x7fd25815a700 2024-04-25T10:09:03.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25815f200 con 0x7fd2580b24a0 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25f367640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2580b43e0 con 0x7fd2580b3f50 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2580b24a0 0x7fd2580b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2580b24a0 0x7fd2580b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33598/0 (socket says 172.21.15.82:33598) 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 -- 172.21.15.82:0/2605508740 learned_addr learned my addr 172.21.15.82:0/2605508740 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25c902640 1 --2- 172.21.15.82:0/2605508740 >> v2:172.21.15.196:3300/0 conn(0x7fd2580b3f50 0x7fd25815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 -- 172.21.15.82:0/2605508740 >> v2:172.21.15.196:3300/0 conn(0x7fd2580b3f50 msgr2=0x7fd25815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 --2- 172.21.15.82:0/2605508740 >> v2:172.21.15.196:3300/0 conn(0x7fd2580b3f50 0x7fd25815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 -- 172.21.15.82:0/2605508740 >> v1:172.21.15.196:6789/0 conn(0x7fd25815a700 legacy=0x7fd2580cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.524+0000 7fd25d904640 1 -- 172.21.15.82:0/2605508740 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2580b46c0 con 0x7fd2580b24a0 2024-04-25T10:09:03.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25d904640 1 --2- 172.21.15.82:0/2605508740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2580b24a0 0x7fd2580b28f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd25806c980 tx=0x7fd25001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4baf885eb45714 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd24ffff640 1 -- 172.21.15.82:0/2605508740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500a5620 con 0x7fd2580b24a0 2024-04-25T10:09:03.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd24ffff640 1 -- 172.21.15.82:0/2605508740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500a57c0 con 0x7fd2580b24a0 2024-04-25T10:09:03.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd24ffff640 1 -- 172.21.15.82:0/2605508740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500a5b90 con 0x7fd2580b24a0 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 -- 172.21.15.82:0/2605508740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2580b24a0 msgr2=0x7fd2580b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 --2- 172.21.15.82:0/2605508740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2580b24a0 0x7fd2580b28f0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd25806c980 tx=0x7fd25001c870 comp rx=0 tx=0).stop 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 -- 172.21.15.82:0/2605508740 shutdown_connections 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 --2- 172.21.15.82:0/2605508740 >> v2:172.21.15.196:3300/0 conn(0x7fd2580b3f50 0x7fd25815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 --2- 172.21.15.82:0/2605508740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2580b24a0 0x7fd2580b28f0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 -- 172.21.15.82:0/2605508740 >> 172.21.15.82:0/2605508740 conn(0x7fd2580b11a0 msgr2=0x7fd2580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 -- 172.21.15.82:0/2605508740 shutdown_connections 2024-04-25T10:09:03.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.528+0000 7fd25f367640 1 -- 172.21.15.82:0/2605508740 wait complete. 2024-04-25T10:09:03.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 Processor -- start 2024-04-25T10:09:03.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- start start 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 0x7fd258159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd25815a700 0x7fd25815f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580cd310 con 0x7fd258164be0 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580ce600 con 0x7fd25815a700 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2580cd490 con 0x7fd2580b24a0 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33600/0 (socket says 172.21.15.82:33600) 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 -- 172.21.15.82:0/888246139 learned_addr learned my addr 172.21.15.82:0/888246139 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25c902640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd25815a700 0x7fd25815f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d103640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 0x7fd258159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 -- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 msgr2=0x7fd258159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 0x7fd258159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 -- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd25815a700 msgr2=0x7fd25815f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd25815a700 0x7fd25815f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 -- 172.21.15.82:0/888246139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd258144a60 con 0x7fd258164be0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d103640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 0x7fd258159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25d904640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd2500af040 tx=0x7fd250019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500bc020 con 0x7fd258164be0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2500b8670 con 0x7fd258164be0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd258144d40 con 0x7fd258164be0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250009510 con 0x7fd258164be0 2024-04-25T10:09:03.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2581455e0 con 0x7fd258164be0 2024-04-25T10:09:03.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd258144f10 con 0x7fd258164be0 2024-04-25T10:09:03.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.532+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2500bf020 con 0x7fd258164be0 2024-04-25T10:09:03.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.536+0000 7fd24dffb640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.536+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2501201d0 con 0x7fd258164be0 2024-04-25T10:09:03.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.536+0000 7fd25d103640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.536+0000 7fd25d103640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 0x7fd23c069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd254003e40 tx=0x7fd254009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.540+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2500c6030 con 0x7fd258164be0 2024-04-25T10:09:03.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd25804efa0 con 0x7fd258164be0 2024-04-25T10:09:03.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd24dffb640 1 -- 172.21.15.82:0/888246139 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd250016580 con 0x7fd258164be0 2024-04-25T10:09:03.717 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 msgr2=0x7fd23c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 0x7fd23c069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd254003e40 tx=0x7fd254009040 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 msgr2=0x7fd258145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd2500af040 tx=0x7fd250019610 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 shutdown_connections 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd23c066f60 0x7fd23c069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd258164be0 0x7fd258145b00 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd25815a700 0x7fd25815f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 --2- 172.21.15.82:0/888246139 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2580b24a0 0x7fd258159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 >> 172.21.15.82:0/888246139 conn(0x7fd2580b11a0 msgr2=0x7fd2580b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 shutdown_connections 2024-04-25T10:09:03.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.712+0000 7fd25f367640 1 -- 172.21.15.82:0/888246139 wait complete. 2024-04-25T10:09:03.735 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:09:03.735 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:09:03.741 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:09:03.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 Processor -- start 2024-04-25T10:09:03.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 -- start start 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 0x7fdcdc0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcdc0d04d0 con 0x7fdcdc15b950 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc15bd30 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdce0abe640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcdc0cc0d0 con 0x7fdcdc09a480 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33604/0 (socket says 172.21.15.82:33604) 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 -- 172.21.15.82:0/616668391 learned_addr learned my addr 172.21.15.82:0/616668391 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/616668391 >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 0x7fdcdc0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 -- 172.21.15.82:0/616668391 >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 msgr2=0x7fdcdc0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 --2- 172.21.15.82:0/616668391 >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 0x7fdcdc0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 -- 172.21.15.82:0/616668391 >> v1:172.21.15.196:6789/0 conn(0x7fdcdc15b950 legacy=0x7fdcdc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcdad9d640 1 -- 172.21.15.82:0/616668391 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc09ab10 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.904+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/616668391 >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 0x7fdcdc0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcdad9d640 1 --2- 172.21.15.82:0/616668391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc072710 tx=0x7fdcd401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5de2968c16c81b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcd959a640 1 -- 172.21.15.82:0/616668391 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40b0070 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcd959a640 1 -- 172.21.15.82:0/616668391 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd40ab600 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcd959a640 1 -- 172.21.15.82:0/616668391 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40ab9d0 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- 172.21.15.82:0/616668391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc0cfb80 msgr2=0x7fdcdc0cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- 172.21.15.82:0/616668391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc072710 tx=0x7fdcd401c300 comp rx=0 tx=0).stop 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- 172.21.15.82:0/616668391 shutdown_connections 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- 172.21.15.82:0/616668391 >> v2:172.21.15.196:3300/0 conn(0x7fdcdc09a480 0x7fdcdc0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- 172.21.15.82:0/616668391 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc0cff90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- 172.21.15.82:0/616668391 >> 172.21.15.82:0/616668391 conn(0x7fdcdc0b30b0 msgr2=0x7fdcdc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:03.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- 172.21.15.82:0/616668391 shutdown_connections 2024-04-25T10:09:03.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- 172.21.15.82:0/616668391 wait complete. 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 Processor -- start 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- start start 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 0x7fdcdc0af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc0cd390 con 0x7fdcdc09a480 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc06faa0 con 0x7fdcdc0cfb80 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdce0abe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcdc1537d0 con 0x7fdcdc15b950 2024-04-25T10:09:03.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcda59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33612/0 (socket says 172.21.15.82:33612) 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcda59c640 1 -- 172.21.15.82:0/1650326933 learned_addr learned my addr 172.21.15.82:0/1650326933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcdad9d640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 0x7fdcdc0af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcda59c640 1 -- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 msgr2=0x7fdcdc0af340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.908+0000 7fdcda59c640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 0x7fdcdc0af340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcda59c640 1 -- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 msgr2=0x7fdcdc159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcda59c640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcda59c640 1 -- 172.21.15.82:0/1650326933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc152f30 con 0x7fdcdc09a480 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcdad9d640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 0x7fdcdc0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcda59c640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00ba50 tx=0x7fdccc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc019070 con 0x7fdcdc09a480 2024-04-25T10:09:03.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc00cbc0 con 0x7fdcdc09a480 2024-04-25T10:09:03.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc014440 con 0x7fdcdc09a480 2024-04-25T10:09:03.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcdc153210 con 0x7fdcdc09a480 2024-04-25T10:09:03.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcdc152120 con 0x7fdcdc09a480 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcdc04efa0 con 0x7fdcdc09a480 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdccc01c070 con 0x7fdcdc09a480 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdccc084dc0 con 0x7fdcdc09a480 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:03.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.912+0000 7fdcd9d9b640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 0x7fdcbc069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc070b40 tx=0x7fdcc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:03.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:03.916+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdccc055680 con 0x7fdcdc09a480 2024-04-25T10:09:04.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcdc0d13a0 con 0x7fdcdc09a480 2024-04-25T10:09:04.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdcd2ffd640 1 -- 172.21.15.82:0/1650326933 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16897 (secure 0 0 0) 0x7fdccc059720 con 0x7fdcdc09a480 2024-04-25T10:09:04.077 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:04.077 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:08:50.889383+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 msgr2=0x7fdcbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 0x7fdcbc069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc070b40 tx=0x7fdcc4008040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 msgr2=0x7fdcdc151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fdccc00ba50 tx=0x7fdccc00bf10 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 shutdown_connections 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdcbc066f60 0x7fdcbc069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdcdc15b950 0x7fdcdc0af340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdcdc0cfb80 0x7fdcdc159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 --2- 172.21.15.82:0/1650326933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdcdc09a480 0x7fdcdc151be0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 >> 172.21.15.82:0/1650326933 conn(0x7fdcdc0b30b0 msgr2=0x7fdcdc0b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 shutdown_connections 2024-04-25T10:09:04.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.072+0000 7fdce0abe640 1 -- 172.21.15.82:0/1650326933 wait complete. 2024-04-25T10:09:04.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:09:04.095 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:09:04.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 Processor -- start 2024-04-25T10:09:04.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 -- start start 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2ec06b440 0x7fb2ec0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2ec06b820 0x7fb2ec0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ec0afb20 con 0x7fb2ec0b18f0 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0b06c0 con 0x7fb2ec06b820 2024-04-25T10:09:04.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f3bee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ec0afd90 con 0x7fb2ec06b440 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2ec06b820 0x7fb2ec0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2ec06b820 0x7fb2ec0b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33614/0 (socket says 172.21.15.82:33614) 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 -- 172.21.15.82:0/4291516679 learned_addr learned my addr 172.21.15.82:0/4291516679 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f198a640 1 --2- 172.21.15.82:0/4291516679 >> v2:172.21.15.196:3300/0 conn(0x7fb2ec06b440 0x7fb2ec0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 -- 172.21.15.82:0/4291516679 >> v2:172.21.15.196:3300/0 conn(0x7fb2ec06b440 msgr2=0x7fb2ec0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 --2- 172.21.15.82:0/4291516679 >> v2:172.21.15.196:3300/0 conn(0x7fb2ec06b440 0x7fb2ec0b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 -- 172.21.15.82:0/4291516679 >> v1:172.21.15.196:6789/0 conn(0x7fb2ec0b18f0 legacy=0x7fb2ec0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.212+0000 7fb2f1189640 1 -- 172.21.15.82:0/4291516679 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0b1d40 con 0x7fb2ec06b820 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f1189640 1 --2- 172.21.15.82:0/4291516679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b820 0x7fb2ec0b13b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0724c0 tx=0x7fb2dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707cfb2bf8a9da59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f0988640 1 -- 172.21.15.82:0/4291516679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0b0070 con 0x7fb2ec06b820 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f0988640 1 -- 172.21.15.82:0/4291516679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc0ab600 con 0x7fb2ec06b820 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f0988640 1 -- 172.21.15.82:0/4291516679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0ab9d0 con 0x7fb2ec06b820 2024-04-25T10:09:04.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- 172.21.15.82:0/4291516679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b820 msgr2=0x7fb2ec0b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/4291516679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b820 0x7fb2ec0b13b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0724c0 tx=0x7fb2dc01c520 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- 172.21.15.82:0/4291516679 shutdown_connections 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/4291516679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b820 0x7fb2ec0b13b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/4291516679 >> v2:172.21.15.196:3300/0 conn(0x7fb2ec06b440 0x7fb2ec0b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- 172.21.15.82:0/4291516679 >> 172.21.15.82:0/4291516679 conn(0x7fb2ec0b1f10 msgr2=0x7fb2ec0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- 172.21.15.82:0/4291516679 shutdown_connections 2024-04-25T10:09:04.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- 172.21.15.82:0/4291516679 wait complete. 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 Processor -- start 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 -- start start 2024-04-25T10:09:04.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec06b820 0x7fb2ec155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.216+0000 7fb2f3bee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 0x7fb2ec152320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0cd170 con 0x7fb2ec06b440 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec0ccff0 con 0x7fb2ec0b18f0 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec15d080 con 0x7fb2ec06b820 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33618/0 (socket says 172.21.15.82:33618) 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 -- 172.21.15.82:0/532593263 learned_addr learned my addr 172.21.15.82:0/532593263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 -- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec06b820 msgr2=0x7fb2ec155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec06b820 0x7fb2ec155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 -- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 msgr2=0x7fb2ec152320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f218b640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 0x7fb2ec152320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 0x7fb2ec152320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 -- 172.21.15.82:0/532593263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0c43e0 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f218b640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 0x7fb2ec152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f198a640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d800 tx=0x7fb2e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800b6c0 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8019070 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec0c46c0 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e800ba60 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec0c4c80 con 0x7fb2ec06b440 2024-04-25T10:09:04.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec04efa0 con 0x7fb2ec06b440 2024-04-25T10:09:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2e800bc00 con 0x7fb2ec06b440 2024-04-25T10:09:04.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2e80856a0 con 0x7fb2ec06b440 2024-04-25T10:09:04.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f1189640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.220+0000 7fb2f1189640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ad000 tx=0x7fb2dc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.224+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2e8056520 con 0x7fb2ec06b440 2024-04-25T10:09:04.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.384+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2ec0cc160 con 0x7fb2ec06b440 2024-04-25T10:09:04.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.384+0000 7fb2e27fc640 1 -- 172.21.15.82:0/532593263 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2e8055e20 con 0x7fb2ec06b440 2024-04-25T10:09:04.389 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 msgr2=0x7fb2d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0ad000 tx=0x7fb2dc004740 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 msgr2=0x7fb2ec0c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800d800 tx=0x7fb2e800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 shutdown_connections 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2ec06b820 0x7fb2ec155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2ec0b18f0 0x7fb2ec152320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 --2- 172.21.15.82:0/532593263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2ec06b440 0x7fb2ec0c3d70 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 >> 172.21.15.82:0/532593263 conn(0x7fb2ec0b1f10 msgr2=0x7fb2ec0adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 shutdown_connections 2024-04-25T10:09:04.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.388+0000 7fb2f3bee640 1 -- 172.21.15.82:0/532593263 wait complete. 2024-04-25T10:09:04.408 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 Processor -- start 2024-04-25T10:09:04.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 -- start start 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840cf590 0x7f78840cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840cbd20 con 0x7f788415b6b0 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840cc000 con 0x7f7884072000 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7888893640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788415ba90 con 0x7f78840cf590 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f788415b6b0 0x7f78840ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36782/0 (socket says 172.21.15.82:36782) 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 -- 172.21.15.82:0/328620216 learned_addr learned my addr 172.21.15.82:0/328620216 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f787bfff640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840cf590 0x7f78840cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851499884 0 0) 0x7f78840cbd20 con 0x7f788415b6b0 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7864003530 con 0x7f788415b6b0 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1794508170 0 0) 0x7f7864003530 con 0x7f788415b6b0 2024-04-25T10:09:04.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840cf590 msgr2=0x7f78840cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840cf590 0x7f78840cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 >> v1:172.21.15.196:6789/0 conn(0x7f7884072000 legacy=0x7f7884072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840cc2e0 con 0x7f788415b6b0 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7874001680 con 0x7f788415b6b0 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 >> v1:172.21.15.82:6789/0 conn(0x7f788415b6b0 legacy=0x7f78840ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864003a60 0x7f7864003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864004810 0x7f7864004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7864005220 0x7f786400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788415ba90 con 0x7f7864003a60 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786400c100 con 0x7f7864004810 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786400c300 con 0x7f7864005220 2024-04-25T10:09:04.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f787bfff640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840cf590 0x7f78840cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864003a60 0x7f7864003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f7882d9d640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7864005220 0x7f786400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 -- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7864005220 msgr2=0x7f786400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7864005220 0x7f786400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 -- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864004810 msgr2=0x7f7864004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864004810 0x7f7864004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.524+0000 7f788259c640 1 -- 172.21.15.82:0/328620216 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864003530 con 0x7f7864003a60 2024-04-25T10:09:04.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f788259c640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864003a60 0x7f7864003e80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f787400c210 tx=0x7f787401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962c7b1f2bdf1b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874007d40 con 0x7f7864003a60 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740a9070 con 0x7f7864003a60 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7881d9b640 1 -- 172.21.15.82:0/328620216 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740a5db0 con 0x7f7864003a60 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- 172.21.15.82:0/328620216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864003a60 msgr2=0x7f7864003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 --2- 172.21.15.82:0/328620216 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7864003a60 0x7f7864003e80 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f787400c210 tx=0x7f787401cc70 comp rx=0 tx=0).stop 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7882d9d640 1 -- 172.21.15.82:0/328620216 reap_dead start 2024-04-25T10:09:04.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- 172.21.15.82:0/328620216 shutdown_connections 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- 172.21.15.82:0/328620216 >> 172.21.15.82:0/328620216 conn(0x7f788406ae60 msgr2=0x7f788406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- 172.21.15.82:0/328620216 shutdown_connections 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- 172.21.15.82:0/328620216 wait complete. 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 Processor -- start 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.528+0000 7f7888893640 1 -- start start 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864003a60 0x7f78840b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840ad770 0x7f78840adc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840cff00 con 0x7f788414bb70 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840cfd80 con 0x7f7864003a60 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840cadb0 con 0x7f78840ad770 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33630/0 (socket says 172.21.15.82:33630) 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 -- 172.21.15.82:0/1314274830 learned_addr learned my addr 172.21.15.82:0/1314274830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f788259c640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864003a60 0x7f78840b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7882d9d640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840ad770 0x7f78840adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 -- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840ad770 msgr2=0x7f78840adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840ad770 0x7f78840adc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 -- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864003a60 msgr2=0x7f78840b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864003a60 0x7f78840b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 -- 172.21.15.82:0/1314274830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78840aeea0 con 0x7f788414bb70 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787bfff640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e560 tx=0x7f786c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c017070 con 0x7f788414bb70 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c013650 con 0x7f788414bb70 2024-04-25T10:09:04.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840af180 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00f040 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840b1bd0 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788404efa0 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f786c01a070 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f788259c640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f786c084ba0 con 0x7f788414bb70 2024-04-25T10:09:04.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.532+0000 7f788259c640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 0x7f7868069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f78740ab000 tx=0x7f78740b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.536+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f786c055a20 con 0x7f788414bb70 2024-04-25T10:09:04.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.688+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f788406edb0 con 0x7f788414bb70 2024-04-25T10:09:04.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.688+0000 7f787affd640 1 -- 172.21.15.82:0/1314274830 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f786c055320 con 0x7f788414bb70 2024-04-25T10:09:04.695 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 msgr2=0x7f7868069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 0x7f7868069450 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f78740ab000 tx=0x7f78740b1040 comp rx=0 tx=0).stop 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 msgr2=0x7f7884098a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f786c00e560 tx=0x7f786c00ea20 comp rx=0 tx=0).stop 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 shutdown_connections 2024-04-25T10:09:04.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78840ad770 0x7f78840adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788414bb70 0x7f7884098a40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 --2- 172.21.15.82:0/1314274830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7864003a60 0x7f78840b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 >> 172.21.15.82:0/1314274830 conn(0x7f788406ae60 msgr2=0x7f788406df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 shutdown_connections 2024-04-25T10:09:04.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.692+0000 7f7888893640 1 -- 172.21.15.82:0/1314274830 wait complete. 2024-04-25T10:09:04.712 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:09:04.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 Processor -- start 2024-04-25T10:09:04.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 -- start start 2024-04-25T10:09:04.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91cc0cf590 0x7f91cc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 0x7f91cc072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc0cbd20 con 0x7f91cc15b6b0 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc15ba90 con 0x7f91cc0cf590 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d3fc9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc0cc000 con 0x7f91cc072000 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91cc0cf590 0x7f91cc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f91cc0cf590 0x7f91cc0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54518/0 (socket says 172.21.15.82:54518) 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 -- 172.21.15.82:0/1490890602 learned_addr learned my addr 172.21.15.82:0/1490890602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d2566640 1 --2- 172.21.15.82:0/1490890602 >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 0x7f91cc072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 -- 172.21.15.82:0/1490890602 >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 msgr2=0x7f91cc072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 --2- 172.21.15.82:0/1490890602 >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 0x7f91cc072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 -- 172.21.15.82:0/1490890602 >> v1:172.21.15.196:6789/0 conn(0x7f91cc15b6b0 legacy=0x7f91cc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:04.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d1564640 1 -- 172.21.15.82:0/1490890602 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0cc2e0 con 0x7f91cc0cf590 2024-04-25T10:09:04.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.820+0000 7f91d2566640 1 --2- 172.21.15.82:0/1490890602 >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 0x7f91cc072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1564640 1 --2- 172.21.15.82:0/1490890602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc0cf590 0x7f91cc0cfa20 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f91cc06f000 tx=0x7f91bc01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861fab19f9b818a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d0d63640 1 -- 172.21.15.82:0/1490890602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0b1070 con 0x7f91cc0cf590 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d0d63640 1 -- 172.21.15.82:0/1490890602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc0ac630 con 0x7f91cc0cf590 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d0d63640 1 -- 172.21.15.82:0/1490890602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0a8050 con 0x7f91cc0cf590 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/1490890602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc0cf590 msgr2=0x7f91cc0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/1490890602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc0cf590 0x7f91cc0cfa20 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f91cc06f000 tx=0x7f91bc01c4e0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/1490890602 shutdown_connections 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/1490890602 >> v2:172.21.15.196:3300/0 conn(0x7f91cc072000 0x7f91cc072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/1490890602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc0cf590 0x7f91cc0cfa20 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/1490890602 >> 172.21.15.82:0/1490890602 conn(0x7f91cc06ae60 msgr2=0x7f91cc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/1490890602 shutdown_connections 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/1490890602 wait complete. 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 Processor -- start 2024-04-25T10:09:04.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- start start 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91cc0cf590 0x7f91cc15f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91cc15b6b0 0x7f91cc1522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0caf30 con 0x7f91cc072000 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc0cadb0 con 0x7f91cc15b6b0 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1682a0 con 0x7f91cc0cf590 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54530/0 (socket says 172.21.15.82:54530) 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 -- 172.21.15.82:0/2417430575 learned_addr learned my addr 172.21.15.82:0/2417430575 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 -- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91cc0cf590 msgr2=0x7f91cc15f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91cc0cf590 0x7f91cc15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 -- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91cc15b6b0 msgr2=0x7f91cc1522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91cc15b6b0 0x7f91cc1522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 -- 172.21.15.82:0/2417430575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0b23c0 con 0x7f91cc072000 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d1d65640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b440 tx=0x7f91c800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8015020 con 0x7f91cc072000 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8011890 con 0x7f91cc072000 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8011c80 con 0x7f91cc072000 2024-04-25T10:09:04.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc06e3f0 con 0x7f91cc072000 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.824+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc153180 con 0x7f91cc072000 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc04efa0 con 0x7f91cc072000 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91c8018020 con 0x7f91cc072000 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91c27fc640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 0x7f91b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91d1564640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 0x7f91b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91c8084ff0 con 0x7f91cc072000 2024-04-25T10:09:04.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.828+0000 7f91d1564640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 0x7f91b0069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0ae000 tx=0x7f91bc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:04.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.832+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91c801f030 con 0x7f91cc072000 2024-04-25T10:09:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f91cc153820 con 0x7f91cc072000 2024-04-25T10:09:04.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91c27fc640 1 -- 172.21.15.82:0/2417430575 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f91c800d040 con 0x7f91cc072000 2024-04-25T10:09:04.989 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:09:04.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 msgr2=0x7f91b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 0x7f91b0069420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f91bc0ae000 tx=0x7f91bc004740 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 msgr2=0x7f91cc15f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f91c800b440 tx=0x7f91c800b900 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 shutdown_connections 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f91b0066f60 0x7f91b0069420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91cc0cf590 0x7f91cc15f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f91cc15b6b0 0x7f91cc1522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 --2- 172.21.15.82:0/2417430575 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f91cc072000 0x7f91cc15f0c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.984+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 >> 172.21.15.82:0/2417430575 conn(0x7f91cc06ae60 msgr2=0x7f91cc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.988+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 shutdown_connections 2024-04-25T10:09:04.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:04.988+0000 7f91d3fc9640 1 -- 172.21.15.82:0/2417430575 wait complete. 2024-04-25T10:09:05.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:09:05.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 Processor -- start 2024-04-25T10:09:05.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 -- start start 2024-04-25T10:09:05.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cdc15a700 0x7f1cdc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 0x7f1cdc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0b2090 con 0x7f1cdc0b3f50 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15f200 con 0x7f1cdc15a700 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce2da5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b43e0 con 0x7f1cdc0b24a0 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cdc15a700 0x7f1cdc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cdc15a700 0x7f1cdc15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54538/0 (socket says 172.21.15.82:54538) 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 -- 172.21.15.82:0/1612705196 learned_addr learned my addr 172.21.15.82:0/1612705196 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce1342640 1 --2- 172.21.15.82:0/1612705196 >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 0x7f1cdc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 -- 172.21.15.82:0/1612705196 >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 msgr2=0x7f1cdc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 --2- 172.21.15.82:0/1612705196 >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 0x7f1cdc0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 -- 172.21.15.82:0/1612705196 >> v1:172.21.15.196:6789/0 conn(0x7f1cdc0b3f50 legacy=0x7f1cdc0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce0b41640 1 -- 172.21.15.82:0/1612705196 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0b46c0 con 0x7f1cdc15a700 2024-04-25T10:09:05.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.112+0000 7f1ce1342640 1 --2- 172.21.15.82:0/1612705196 >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 0x7f1cdc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:05.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce0b41640 1 --2- 172.21.15.82:0/1612705196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15a700 0x7f1cdc15ecc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06bd20 tx=0x7f1ccc01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb5c25018fcd0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/1612705196 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0b0070 con 0x7f1cdc15a700 2024-04-25T10:09:05.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/1612705196 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc0ab600 con 0x7f1cdc15a700 2024-04-25T10:09:05.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/1612705196 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc0ab9d0 con 0x7f1cdc15a700 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- 172.21.15.82:0/1612705196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15a700 msgr2=0x7f1cdc15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/1612705196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15a700 0x7f1cdc15ecc0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc06bd20 tx=0x7f1ccc01c3a0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- 172.21.15.82:0/1612705196 shutdown_connections 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/1612705196 >> v2:172.21.15.196:3300/0 conn(0x7f1cdc0b24a0 0x7f1cdc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/1612705196 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15a700 0x7f1cdc15ecc0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- 172.21.15.82:0/1612705196 >> 172.21.15.82:0/1612705196 conn(0x7f1cdc0b11a0 msgr2=0x7f1cdc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- 172.21.15.82:0/1612705196 shutdown_connections 2024-04-25T10:09:05.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- 172.21.15.82:0/1612705196 wait complete. 2024-04-25T10:09:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 Processor -- start 2024-04-25T10:09:05.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.116+0000 7f1ce2da5640 1 -- start start 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc0b24a0 0x7f1cdc13ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc15a700 0x7f1cdc1514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0cd310 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0ce600 con 0x7f1cdc15a700 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0cd490 con 0x7f1cdc0b24a0 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54542/0 (socket says 172.21.15.82:54542) 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 -- 172.21.15.82:0/3164486576 learned_addr learned my addr 172.21.15.82:0/3164486576 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 -- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc0b24a0 msgr2=0x7f1cdc13ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc0b24a0 0x7f1cdc13ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 -- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc15a700 msgr2=0x7f1cdc1514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cdbfff640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc15a700 0x7f1cdc1514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc15a700 0x7f1cdc1514f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 -- 172.21.15.82:0/3164486576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc1571f0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce1342640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ce60 tx=0x7f1cd400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400c2e0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd400ebf0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400a470 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc13efb0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc150450 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc04efa0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cd400a610 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 0x7f1cc80695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:05.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cd40875c0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce0b41640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 0x7f1cc80695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:05.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.120+0000 7f1ce0b41640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 0x7f1cc80695c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ad000 tx=0x7f1ccc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:05.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.124+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cdc13efb0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1cdc157a60 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1cd97fa640 1 -- 172.21.15.82:0/3164486576 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1cdc1571f0 con 0x7f1cdc13b9a0 2024-04-25T10:09:05.288 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 msgr2=0x7f1cc80695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 0x7f1cc80695c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0ad000 tx=0x7f1ccc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 msgr2=0x7f1cdc151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ce60 tx=0x7f1cd400e420 comp rx=0 tx=0).stop 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 shutdown_connections 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8067100 0x7f1cc80695c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc13b9a0 0x7f1cdc151a30 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc15a700 0x7f1cdc1514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 --2- 172.21.15.82:0/3164486576 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc0b24a0 0x7f1cdc13ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:05.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 >> 172.21.15.82:0/3164486576 conn(0x7f1cdc0b11a0 msgr2=0x7f1cdc0b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:05.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 shutdown_connections 2024-04-25T10:09:05.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:05.284+0000 7f1ce2da5640 1 -- 172.21.15.82:0/3164486576 wait complete. 2024-04-25T10:09:05.305 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:09:05.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:05.306 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:05.316 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[48335]: fuse finished with error 0 and tester_r 0 2024-04-25T10:09:05.319 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:09:07.981 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:07.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:11.421 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:09:11.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:11.421 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:09:11.429 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:11.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 Processor -- start 2024-04-25T10:09:11.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 -- start start 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f518c0b3f50 0x7f518c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f518c0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0b2090 con 0x7f518c15a700 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c15f200 con 0x7f518c0b3f50 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f51914cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0b43e0 con 0x7f518c0b24a0 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f518c15a700 0x7f518c0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55134/0 (socket says 172.21.15.82:55134) 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518affd640 1 -- 172.21.15.82:0/2546308991 learned_addr learned my addr 172.21.15.82:0/2546308991 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:11.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2546308991 >> v2:172.21.15.196:3300/0 conn(0x7f518c0b3f50 0x7f518c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518b7fe640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f518c0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009528460 0 0) 0x7f518c0b2090 con 0x7f518c15a700 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5174003530 con 0x7f518c15a700 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493212828 0 0) 0x7f5174003530 con 0x7f518c15a700 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 msgr2=0x7f518c0b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f518c0b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 >> v2:172.21.15.196:3300/0 conn(0x7f518c0b3f50 msgr2=0x7f518c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 --2- 172.21.15.82:0/2546308991 >> v2:172.21.15.196:3300/0 conn(0x7f518c0b3f50 0x7f518c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0b46c0 con 0x7f518c15a700 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f517c0016a0 con 0x7f518c15a700 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 >> v1:172.21.15.82:6789/0 conn(0x7f518c15a700 legacy=0x7f518c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f5174003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174003fb0 0x7f5174004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5174004d50 0x7f51740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c15f200 con 0x7f5174003fb0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0b43e0 con 0x7f5174004d50 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517400bb30 con 0x7f518c0b24a0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174003fb0 0x7f5174004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518affd640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f5174003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 -- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 msgr2=0x7f5174003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f5174003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518b7fe640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5174004d50 0x7f51740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 -- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5174004d50 msgr2=0x7f51740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5174004d50 0x7f51740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518a7fc640 1 -- 172.21.15.82:0/2546308991 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174003530 con 0x7f5174003fb0 2024-04-25T10:09:11.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.596+0000 7f518affd640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f518c0b24a0 0x7f5174003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174003fb0 0x7f5174004810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5180009670 tx=0x7f518001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8def784a11c90086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800b4070 con 0x7f5174003fb0 2024-04-25T10:09:11.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51800a78b0 con 0x7f5174003fb0 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f5189ffb640 1 -- 172.21.15.82:0/2546308991 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800096b0 con 0x7f5174003fb0 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- 172.21.15.82:0/2546308991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174003fb0 msgr2=0x7f5174004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 --2- 172.21.15.82:0/2546308991 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5174003fb0 0x7f5174004810 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f5180009670 tx=0x7f518001ecb0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f518b7fe640 1 -- 172.21.15.82:0/2546308991 reap_dead start 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- 172.21.15.82:0/2546308991 shutdown_connections 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- 172.21.15.82:0/2546308991 >> 172.21.15.82:0/2546308991 conn(0x7f518c0b11a0 msgr2=0x7f518c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- 172.21.15.82:0/2546308991 shutdown_connections 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- 172.21.15.82:0/2546308991 wait complete. 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 Processor -- start 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.600+0000 7f51914cc640 1 -- start start 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5174003fb0 0x7f518c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c0ad290 0x7f518c1629e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0b2c90 con 0x7f518c0a84d0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0ce600 con 0x7f518c0ad290 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cd310 con 0x7f5174003fb0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54552/0 (socket says 172.21.15.82:54552) 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 -- 172.21.15.82:0/2665040995 learned_addr learned my addr 172.21.15.82:0/2665040995 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518affd640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5174003fb0 0x7f518c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518b7fe640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c0ad290 0x7f518c1629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 -- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5174003fb0 msgr2=0x7f518c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5174003fb0 0x7f518c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 -- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c0ad290 msgr2=0x7f518c1629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c0ad290 0x7f518c1629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 -- 172.21.15.82:0/2665040995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0ad810 con 0x7f518c0a84d0 2024-04-25T10:09:11.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518a7fc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f51800b2040 tx=0x7f5174005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800be030 con 0x7f518c0a84d0 2024-04-25T10:09:11.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518000f000 con 0x7f518c0a84d0 2024-04-25T10:09:11.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c1604b0 con 0x7f518c0a84d0 2024-04-25T10:09:11.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51800b4040 con 0x7f518c0a84d0 2024-04-25T10:09:11.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c160a70 con 0x7f518c0a84d0 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c04efa0 con 0x7f518c0a84d0 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51800c1070 con 0x7f518c0a84d0 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 0x7f5178069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518affd640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 0x7f5178069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51801210d0 con 0x7f518c0a84d0 2024-04-25T10:09:11.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.604+0000 7f518affd640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 0x7f5178069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f517c001ec0 tx=0x7f517c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:11.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.608+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51800c8020 con 0x7f518c0a84d0 2024-04-25T10:09:11.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.804+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f518c06f260 con 0x7f518c0a84d0 2024-04-25T10:09:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.804+0000 7f51737fe640 1 -- 172.21.15.82:0/2665040995 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f51800189f0 con 0x7f518c0a84d0 2024-04-25T10:09:11.809 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:11.809 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:08:51.280173+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"c","rank":0,"incarnation":54,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/2865291678","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":2865291678},{"type":"v1","addr":"172.21.15.82:6837","nonce":2865291678}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 56 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 msgr2=0x7f5178069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 0x7f5178069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f517c001ec0 tx=0x7f517c009040 comp rx=0 tx=0).stop 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 msgr2=0x7f518c0acd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f51800b2040 tx=0x7f5174005840 comp rx=0 tx=0).stop 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 shutdown_connections 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5178066f90 0x7f5178069450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f518c0ad290 0x7f518c1629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f518c0a84d0 0x7f518c0acd50 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 --2- 172.21.15.82:0/2665040995 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5174003fb0 0x7f518c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 >> 172.21.15.82:0/2665040995 conn(0x7f518c0b11a0 msgr2=0x7f518c15cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 shutdown_connections 2024-04-25T10:09:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:11.808+0000 7f51914cc640 1 -- 172.21.15.82:0/2665040995 wait complete. 2024-04-25T10:09:11.828 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:09:11.978 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:09:11.978 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:09:11.979 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:09:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 Processor -- start 2024-04-25T10:09:12.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 -- start start 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b500ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b501451f0 0x7f6b50142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b500cee30 con 0x7f6b50144ca0 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b500cc2b0 con 0x7f6b501451f0 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b54937640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5006e290 con 0x7f6b500ce510 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b501451f0 0x7f6b50142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6b501451f0 0x7f6b50142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54558/0 (socket says 172.21.15.82:54558) 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 -- 172.21.15.82:0/2865613902 learned_addr learned my addr 172.21.15.82:0/2865613902 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b500ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 -- 172.21.15.82:0/2865613902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 msgr2=0x7f6b500ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b500ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 -- 172.21.15.82:0/2865613902 >> v1:172.21.15.196:6789/0 conn(0x7f6b50144ca0 legacy=0x7f6b5006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4ed9d640 1 -- 172.21.15.82:0/2865613902 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500ccc70 con 0x7f6b501451f0 2024-04-25T10:09:12.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.100+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b500ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4ed9d640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b501451f0 0x7f6b50142680 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009e50 tx=0x7f6b4801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d1903eb767bb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4d59a640 1 -- 172.21.15.82:0/2865613902 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480b4070 con 0x7f6b501451f0 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4d59a640 1 -- 172.21.15.82:0/2865613902 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b480af400 con 0x7f6b501451f0 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4d59a640 1 -- 172.21.15.82:0/2865613902 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b480af7d0 con 0x7f6b501451f0 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- 172.21.15.82:0/2865613902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b501451f0 msgr2=0x7f6b50142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b501451f0 0x7f6b50142680 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6b48009e50 tx=0x7f6b4801f1e0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- 172.21.15.82:0/2865613902 shutdown_connections 2024-04-25T10:09:12.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b501451f0 0x7f6b50142680 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- 172.21.15.82:0/2865613902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b500ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- 172.21.15.82:0/2865613902 >> 172.21.15.82:0/2865613902 conn(0x7f6b500b11a0 msgr2=0x7f6b500b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- 172.21.15.82:0/2865613902 shutdown_connections 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- 172.21.15.82:0/2865613902 wait complete. 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 Processor -- start 2024-04-25T10:09:12.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- start start 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b5013ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b501451f0 0x7f6b5013b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5006ee30 con 0x7f6b50144ca0 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5006df20 con 0x7f6b501451f0 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b54937640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b500cc900 con 0x7f6b500ce510 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54564/0 (socket says 172.21.15.82:54564) 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.104+0000 7f6b4dd9b640 1 -- 172.21.15.82:0/3093365170 learned_addr learned my addr 172.21.15.82:0/3093365170 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4ed9d640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b501451f0 0x7f6b5013b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b5013ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 -- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 msgr2=0x7f6b5013ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b5013ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 -- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b501451f0 msgr2=0x7f6b5013b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b501451f0 0x7f6b5013b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 -- 172.21.15.82:0/3093365170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b50159ef0 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b5013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4dd9b640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400c410 tx=0x7f6b4400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011790 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b44017070 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5015a190 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44011b70 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5015a750 con 0x7f6b50144ca0 2024-04-25T10:09:12.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5004efa0 con 0x7f6b50144ca0 2024-04-25T10:09:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b44011d10 con 0x7f6b50144ca0 2024-04-25T10:09:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b44084d70 con 0x7f6b50144ca0 2024-04-25T10:09:12.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.108+0000 7f6b4e59c640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 0x7f6b30069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b40008930 tx=0x7f6b40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.112+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b44021020 con 0x7f6b50144ca0 2024-04-25T10:09:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.300+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6b500b1990 con 0x7f6b50144ca0 2024-04-25T10:09:12.306 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:09:12.300+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:09:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.304+0000 7f6b3effd640 1 -- 172.21.15.82:0/3093365170 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v57) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6b44055c80 con 0x7f6b50144ca0 2024-04-25T10:09:12.308 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:09:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 msgr2=0x7f6b30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 0x7f6b30069420 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b40008930 tx=0x7f6b40008040 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 msgr2=0x7f6b50141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400c410 tx=0x7f6b4400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 shutdown_connections 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6b30066f60 0x7f6b30069420 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6b501451f0 0x7f6b5013b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6b50144ca0 0x7f6b50141ae0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 --2- 172.21.15.82:0/3093365170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6b500ce510 0x7f6b5013ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 >> 172.21.15.82:0/3093365170 conn(0x7f6b500b11a0 msgr2=0x7f6b50071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 shutdown_connections 2024-04-25T10:09:12.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.308+0000 7f6b54937640 1 -- 172.21.15.82:0/3093365170 wait complete. 2024-04-25T10:09:12.315 INFO:tasks.ceph.mds.c.smithi082.stderr: -3994> 2024-04-25T10:08:55.868+0000 7ff1bef4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:12.328 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T10:09:12.344 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:09:12.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 Processor -- start 2024-04-25T10:09:12.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- start start 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d770110 0x55fa2d819790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d7fe340 0x55fa2d7435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d78dae0 con 0x55fa2d7704f0 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d78d840 con 0x55fa2d7fe340 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d78dd80 con 0x55fa2d770110 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54576/0 (socket says 172.21.15.82:54576) 2024-04-25T10:09:12.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 -- 172.21.15.82:0/1473575441 learned_addr learned my addr 172.21.15.82:0/1473575441 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 -- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d770110 msgr2=0x55fa2d819790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d770110 0x55fa2d819790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 -- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d7fe340 msgr2=0x55fa2d7435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d7fe340 0x55fa2d7435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 -- 172.21.15.82:0/1473575441 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2d81af90 con 0x55fa2d7704f0 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009640 tx=0x7f19cc009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75c3cf57bdb5b87b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dadc8640 1 -- 172.21.15.82:0/1473575441 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc006600 con 0x55fa2d7704f0 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dadc8640 1 -- 172.21.15.82:0/1473575441 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc0067a0 con 0x55fa2d7704f0 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dadc8640 1 -- 172.21.15.82:0/1473575441 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc006b90 con 0x55fa2d7704f0 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- 172.21.15.82:0/1473575441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 msgr2=0x55fa2d743090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009640 tx=0x7f19cc009b00 comp rx=0 tx=0).stop 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- 172.21.15.82:0/1473575441 shutdown_connections 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d7fe340 0x55fa2d7435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7704f0 0x55fa2d743090 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- 172.21.15.82:0/1473575441 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d770110 0x55fa2d819790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- 172.21.15.82:0/1473575441 >> 172.21.15.82:0/1473575441 conn(0x55fa2d76ea80 msgr2=0x55fa2d788190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- 172.21.15.82:0/1473575441 shutdown_connections 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- 172.21.15.82:0/1473575441 wait complete. 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 Processor -- start 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- start start 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d770110 0x55fa2d820130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d7704f0 0x55fa2d746d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 0x55fa2d818bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d81b4d0 con 0x55fa2d7fe340 2024-04-25T10:09:12.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d81b230 con 0x55fa2d770110 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19deaad080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fa2d81b770 con 0x55fa2d7704f0 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 0x55fa2d818bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 0x55fa2d818bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54584/0 (socket says 172.21.15.82:54584) 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 -- 172.21.15.82:0/734141433 learned_addr learned my addr 172.21.15.82:0/734141433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dbdca640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d7704f0 0x55fa2d746d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dc5cb640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d770110 0x55fa2d820130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 -- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d7704f0 msgr2=0x55fa2d746d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55fa2d7704f0 0x55fa2d746d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 -- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d770110 msgr2=0x55fa2d820130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d770110 0x55fa2d820130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 -- 172.21.15.82:0/734141433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa2d820790 con 0x55fa2d7fe340 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dc5cb640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55fa2d770110 0x55fa2d820130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.356+0000 7f19dcdcc640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 0x55fa2d818bc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005f00 tx=0x7f19d4008430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 -- 172.21.15.82:0/734141433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4010640 con 0x55fa2d7fe340 2024-04-25T10:09:12.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 -- 172.21.15.82:0/734141433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4015070 con 0x55fa2d7fe340 2024-04-25T10:09:12.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fa2d820a30 con 0x55fa2d7fe340 2024-04-25T10:09:12.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 -- 172.21.15.82:0/734141433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4010a20 con 0x55fa2d7fe340 2024-04-25T10:09:12.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fa2d820f70 con 0x55fa2d7fe340 2024-04-25T10:09:12.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 -- 172.21.15.82:0/734141433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19d4010bc0 con 0x55fa2d7fe340 2024-04-25T10:09:12.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19c8065940 0x7f19c8067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19d9dc6640 1 -- 172.21.15.82:0/734141433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f19d4083c40 con 0x55fa2d7fe340 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19dc5cb640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19c8065940 0x7f19c8067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19deaad080 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55fa2d81ef10 0x55fa2d81f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e92) v9 -- 0x55fa2d81f830 con 0x55fa2d81ef10 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19dbdca640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55fa2d81ef10 0x55fa2d81f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19dc5cb640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19c8065940 0x7f19c8067e00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ac000 tx=0x7f19d0003030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.360+0000 7f19dbdca640 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55fa2d81ef10 0x55fa2d81f2f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19dbdca640 1 -- 172.21.15.82:0/734141433 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v92'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f19cc006b20 con 0x55fa2d81ef10 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55fa2d81ef10 msgr2=0x55fa2d81f2f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55fa2d81ef10 0x55fa2d81f2f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19c8065940 msgr2=0x7f19c8067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f19c8065940 0x7f19c8067e00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f19d00ac000 tx=0x7f19d0003030 comp rx=0 tx=0).stop 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 msgr2=0x55fa2d818bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 --2- 172.21.15.82:0/734141433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55fa2d7fe340 0x55fa2d818bc0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f19d4005f00 tx=0x7f19d4008430 comp rx=0 tx=0).stop 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19dcdcc640 1 -- 172.21.15.82:0/734141433 reap_dead start 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 shutdown_connections 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 >> 172.21.15.82:0/734141433 conn(0x55fa2d76ea80 msgr2=0x55fa2d7fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 shutdown_connections 2024-04-25T10:09:12.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.364+0000 7f19deaad080 1 -- 172.21.15.82:0/734141433 wait complete. 2024-04-25T10:09:12.373 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T10:09:12.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- start start 2024-04-25T10:09:12.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 Processor -- start 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023b89800 0x558023d43b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 0x558023d43600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024958180 con 0x558023b89000 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023be8900 con 0x558023b89400 2024-04-25T10:09:12.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558023be8480 con 0x558023b89800 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54598/0 (socket says 172.21.15.82:54598) 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 -- 172.21.15.82:0/111623017 learned_addr learned my addr 172.21.15.82:0/111623017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c608c4640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023b89800 0x558023d43b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c600c3640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 0x558023d43600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 -- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023b89800 msgr2=0x558023d43b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023b89800 0x558023d43b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 -- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 msgr2=0x558023d43600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 0x558023d43600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 -- 172.21.15.82:0/111623017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558024940b40 con 0x558023b89000 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c600c3640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 0x558023d43600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c610c5640 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5580249e9350 tx=0x558023c40ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e352432ecf62c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c5e0bf640 1 -- 172.21.15.82:0/111623017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbe4e0 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c5e0bf640 1 -- 172.21.15.82:0/111623017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558023bd21c0 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c5e0bf640 1 -- 172.21.15.82:0/111623017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbeb60 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558024940d20 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558024940f00 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 4 waiting for MDS map... 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c5e0bf640 1 -- 172.21.15.82:0/111623017 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x558024a9a600 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c5e0bf640 1 -- 172.21.15.82:0/111623017 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 57) v1 ==== 2723+0+0 (secure 0 0 0) 0x558023bdf180 con 0x558023b89000 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 4 Got MDS map 57 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 10 main: JournalTool::main 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 Processor -- start 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.444+0000 7f6c62105400 1 -- start start 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558023be8180 con 0x558024a9f400 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024958d80 con 0x558024a9ec00 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558024958f00 con 0x558024a9f000 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54608/0 (socket says 172.21.15.82:54608) 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/4228527054 learned_addr learned my addr 172.21.15.82:0/4228527054 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c608c4640 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 msgr2=0x558023d42580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9ec00 msgr2=0x558023d42b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/4228527054 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558024940d20 con 0x558024a9f400 2024-04-25T10:09:12.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c608c4640 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x558024aba690 tx=0x558024b100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464b2d50c9ec9978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5c0bb640 1 -- 172.21.15.82:0/4228527054 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbe4e0 con 0x558024a9f400 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5c0bb640 1 -- 172.21.15.82:0/4228527054 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558023bd2e00 con 0x558024a9f400 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5c0bb640 1 -- 172.21.15.82:0/4228527054 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbeb60 con 0x558024a9f400 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/4228527054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 msgr2=0x558023d44100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x558024aba690 tx=0x558024b100c0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/4228527054 shutdown_connections 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- 172.21.15.82:0/4228527054 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/4228527054 >> 172.21.15.82:0/4228527054 conn(0x558023b88c00 msgr2=0x558023bb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/4228527054 shutdown_connections 2024-04-25T10:09:12.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/4228527054 wait complete. 2024-04-25T10:09:12.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 Processor -- start 2024-04-25T10:09:12.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- start start 2024-04-25T10:09:12.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024959200 con 0x558024a9f400 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024959080 con 0x558024a9f000 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558024959380 con 0x558024a9ec00 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54610/0 (socket says 172.21.15.82:54610) 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/3777096059 learned_addr learned my addr 172.21.15.82:0/3777096059 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9ec00 msgr2=0x558023d42b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 msgr2=0x558023d42580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 -- 172.21.15.82:0/3777096059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558024941c20 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x558024abb350 tx=0x558024b10000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5a8b8640 1 -- 172.21.15.82:0/3777096059 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbe000 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5a8b8640 1 -- 172.21.15.82:0/3777096059 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558023bd2540 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558024941e00 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c5a8b8640 1 -- 172.21.15.82:0/3777096059 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023bbe4e0 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558024941c20 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.448+0000 7f6c62105400 4 main: JournalTool: resolving pool 13 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c5a8b8640 1 -- 172.21.15.82:0/3777096059 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x558024af6c00 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c5a8b8640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 0x558023f9e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c5a8b8640 1 -- 172.21.15.82:0/3777096059 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x558023eb0400 con 0x558024a9f400 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:09:12.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 4 main: Executing for rank 0 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 0x558023f9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 0x558023f9e000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x558023bd9a70 tx=0x55802494a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- start start 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 Processor -- start 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9fc00 0x558023d44680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023eb9000 0x558023d44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024959800 con 0x558023eb9400 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558024959980 con 0x558024a9fc00 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558024959b00 con 0x558023eb9000 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54618/0 (socket says 172.21.15.82:54618) 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 learned_addr learned my addr 172.21.15.82:0/3387532307 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023eb9000 0x558023d44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023eb9000 msgr2=0x558023d44c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9fc00 0x558023d44680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023eb9000 0x558023d44c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9fc00 msgr2=0x558023d44680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9fc00 0x558023d44680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558023ff6960 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x558023ef9320 tx=0x558024b11620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0e9134ee85806c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c578b2640 1 -- 172.21.15.82:0/3387532307 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023f35a00 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c578b2640 1 -- 172.21.15.82:0/3387532307 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558023bd2e00 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558023ff6b40 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558023ff6d20 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 4 waiting for MDS map... 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c578b2640 1 -- 172.21.15.82:0/3387532307 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558023f35ba0 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c578b2640 1 -- 172.21.15.82:0/3387532307 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x558024010600 con 0x558023eb9400 2024-04-25T10:09:12.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c578b2640 1 -- 172.21.15.82:0/3387532307 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 57) v1 ==== 2723+0+0 (secure 0 0 0) 0x55802401c000 con 0x558023eb9400 2024-04-25T10:09:12.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 4 Got MDS map 57 2024-04-25T10:09:12.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 0x558023d45700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) v9 -- 0x558023eb9c00 con 0x558023eb9800 2024-04-25T10:09:12.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.452+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 0x558023d45700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.456+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 0x558023d45700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=c41e9cf15b2c0a52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.460+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x558023d04000 con 0x558023eb9800 2024-04-25T10:09:12.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.460+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55802400f000 0x558023f9e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.460+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802400f400 con 0x55802400f000 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.460+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55802400f000 0x558023f9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.460+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55802400f000 0x558023f9e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=f638996ba2e22a45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c608c4640 1 -- 172.21.15.82:0/3387532307 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x558023d04500 con 0x55802400f000 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c560af640 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802402a000 con 0x55802400f000 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c560af640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55802402a400 0x558023fa0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c560af640 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802402a800 con 0x55802402a400 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c608c4640 1 -- 172.21.15.82:0/3387532307 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x558023d04500 con 0x55802400f000 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55802402a400 0x558023fa0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55802402a400 0x558023fa0c00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=c551ff2bff4aa19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stdout:old journal was 4209834~892 2024-04-25T10:09:12.469 INFO:teuthology.orchestra.run.smithi082.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-25T10:09:12.470 INFO:teuthology.orchestra.run.smithi082.stdout:writing journal head 2024-04-25T10:09:12.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x55802402a400 2024-04-25T10:09:12.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802400f800 con 0x55802400f000 2024-04-25T10:09:12.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c608c4640 1 -- 172.21.15.82:0/3387532307 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v92'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04500 con 0x55802400f000 2024-04-25T10:09:12.471 INFO:teuthology.orchestra.run.smithi082.stdout:writing EResetJournal entry 2024-04-25T10:09:12.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802400fc00 con 0x55802402a400 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55802402a000 0x558023f9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802402ac00 con 0x55802402a000 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802402b000 con 0x558023eb9800 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55802402b400 0x558023f9f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.464+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802402b800 con 0x55802402b400 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x55802400f400 con 0x55802402a400 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 0x558023f9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x558024963400 con 0x55802400f800 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55802402b400 0x558023f9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55802402a000 0x558023f9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c600c3640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55802402b400 0x558023f9f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d30e5fb25d236a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 0x558023f9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c608c4640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55802402a000 0x558023f9eb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=46e4c1458d07bde8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 0x558023f9f600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=7b15e19ac6c74e4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(9 200.00000005 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04780 con 0x55802402b400 2024-04-25T10:09:12.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(8 200.00000004 [delete] v92'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x558023eb9800 2024-04-25T10:09:12.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(6 200.00000002 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x55802402a400 2024-04-25T10:09:12.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(10 200.00000006 [delete] v92'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x55802402a400 2024-04-25T10:09:12.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.468+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(11 200.00000007 [delete] v92'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x55802400f800 2024-04-25T10:09:12.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.472+0000 7f6c608c4640 1 -- 172.21.15.82:0/3387532307 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(7 200.00000003 [delete] v92'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04500 con 0x55802402a000 2024-04-25T10:09:12.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.472+0000 7f6c560af640 1 -- 172.21.15.82:0/3387532307 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92) v9 -- 0x558023eb9c00 con 0x55802402a400 2024-04-25T10:09:12.481 INFO:teuthology.orchestra.run.smithi082.stdout:done 2024-04-25T10:09:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c610c5640 1 -- 172.21.15.82:0/3387532307 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v92'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x558023d04000 con 0x55802402a400 2024-04-25T10:09:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55802400f000 msgr2=0x558023f9e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55802400f000 0x558023f9e580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55802402b400 msgr2=0x558023f9f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55802402b400 0x558023f9f080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55802402a400 msgr2=0x558023fa0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55802402a400 0x558023fa0c00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55802402a000 msgr2=0x558023f9eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55802402a000 0x558023f9eb00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 msgr2=0x558023d45700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 0x558023d45700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c600c3640 1 -- 172.21.15.82:0/3387532307 reap_dead start 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 msgr2=0x558023f9f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 0x558023f9f600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 msgr2=0x558023d45180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x558023ef9320 tx=0x558024b11620 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 shutdown_connections 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55802400f800 0x558023f9f600 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x558023eb9800 0x558023d45700 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3387532307 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023eb9400 0x558023d45180 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 >> 172.21.15.82:0/3387532307 conn(0x558024a9f800 msgr2=0x558023ff2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 shutdown_connections 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3387532307 wait complete. 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 msgr2=0x558023f9e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 0x558023f9e000 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x558023bd9a70 tx=0x55802494a000 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 msgr2=0x558023d44100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x558024abb350 tx=0x558024b10000 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 shutdown_connections 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x558023eb8c00 0x558023f9e000 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558024a9f400 0x558023d44100 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558024a9f000 0x558023d42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/3777096059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558024a9ec00 0x558023d42b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 >> 172.21.15.82:0/3777096059 conn(0x558023b88c00 msgr2=0x558023bb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 shutdown_connections 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/3777096059 wait complete. 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 msgr2=0x558023d43080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x5580249e9350 tx=0x558023c40ba0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 shutdown_connections 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558023b89000 0x558023d43080 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558023b89400 0x558023d43600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 --2- 172.21.15.82:0/111623017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558023b89800 0x558023d43b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 >> 172.21.15.82:0/111623017 conn(0x558023b89c00 msgr2=0x558023bb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 shutdown_connections 2024-04-25T10:09:12.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.476+0000 7f6c62105400 1 -- 172.21.15.82:0/111623017 wait complete. 2024-04-25T10:09:12.487 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.114526, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T10:09:12.488 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:09:12.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.640+0000 7f10c57f1640 1 Processor -- start 2024-04-25T10:09:12.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.640+0000 7f10c57f1640 1 -- start start 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10c57f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 0x7f10c00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10c57f1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00cbd20 con 0x7f10c015b6b0 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10c57f1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c00cc000 con 0x7f10c0072000 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10c57f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c015ba90 con 0x7f10c00cf590 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f10c015b6b0 0x7f10c00ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55150/0 (socket says 172.21.15.82:55150) 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 -- 172.21.15.82:0/3876697439 learned_addr learned my addr 172.21.15.82:0/3876697439 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10be7fc640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 0x7f10c00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857193822 0 0) 0x7f10c00cbd20 con 0x7f10c015b6b0 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10ac003530 con 0x7f10c015b6b0 2024-04-25T10:09:12.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823847604 0 0) 0x7f10ac003530 con 0x7f10c015b6b0 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 msgr2=0x7f10c00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 0x7f10c00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 >> v1:172.21.15.196:6789/0 conn(0x7f10c0072000 legacy=0x7f10c0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c00cc2e0 con 0x7f10c015b6b0 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f10a8001680 con 0x7f10c015b6b0 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 >> v1:172.21.15.82:6789/0 conn(0x7f10c015b6b0 legacy=0x7f10c00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10ac003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 0x7f10ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ac004d50 0x7f10ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c015ba90 con 0x7f10ac003a60 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac00bae0 con 0x7f10ac004d50 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac00bce0 con 0x7f10c00cf590 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10ac003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bf7fe640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ac004d50 0x7f10ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 -- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 msgr2=0x7f10ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c00cf590 0x7f10ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 -- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ac004d50 msgr2=0x7f10ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ac004d50 0x7f10ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10beffd640 1 -- 172.21.15.82:0/3876697439 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac003530 con 0x7f10ac003a60 2024-04-25T10:09:12.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.644+0000 7f10bf7fe640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10ac004d50 0x7f10ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10ac003e80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10a8001ec0 tx=0x7f10a801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e19ddc18ff4d105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8007d40 con 0x7f10ac003a60 2024-04-25T10:09:12.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a80a9070 con 0x7f10ac003a60 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10bdffb640 1 -- 172.21.15.82:0/3876697439 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005d40 con 0x7f10ac003a60 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/3876697439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 msgr2=0x7f10ac003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 --2- 172.21.15.82:0/3876697439 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10ac003e80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10a8001ec0 tx=0x7f10a801cb70 comp rx=0 tx=0).stop 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10bf7fe640 1 -- 172.21.15.82:0/3876697439 reap_dead start 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/3876697439 shutdown_connections 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/3876697439 >> 172.21.15.82:0/3876697439 conn(0x7f10c006ae60 msgr2=0x7f10c006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/3876697439 shutdown_connections 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/3876697439 wait complete. 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 Processor -- start 2024-04-25T10:09:12.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- start start 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 0x7f10c015a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c0153f70 0x7f10c01543f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00cff00 con 0x7f10ac003a60 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00cfd80 con 0x7f10c0072500 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c00cadb0 con 0x7f10c0153f70 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54642/0 (socket says 172.21.15.82:54642) 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 -- 172.21.15.82:0/818172601 learned_addr learned my addr 172.21.15.82:0/818172601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:12.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10bf7fe640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c0153f70 0x7f10c01543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10be7fc640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 0x7f10c015a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 -- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c0153f70 msgr2=0x7f10c01543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c0153f70 0x7f10c01543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 -- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 msgr2=0x7f10c015a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 0x7f10c015a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 -- 172.21.15.82:0/818172601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c014df70 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10be7fc640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 0x7f10c015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10beffd640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f10a801c9a0 tx=0x7f10ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a8005d20 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a80a5850 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c014e250 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a80a5c40 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.648+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c014e810 con 0x7f10ac003a60 2024-04-25T10:09:12.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10c004efa0 con 0x7f10ac003a60 2024-04-25T10:09:12.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10a80047f0 con 0x7f10ac003a60 2024-04-25T10:09:12.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10b77fe640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 0x7f10a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:12.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f10a811e890 con 0x7f10ac003a60 2024-04-25T10:09:12.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10be7fc640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 0x7f10a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:12.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.652+0000 7f10be7fc640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 0x7f10a0069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10c015fa40 tx=0x7f10b800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:12.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.656+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10a80168b0 con 0x7f10ac003a60 2024-04-25T10:09:12.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:12.840+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f10c015f790 con 0x7f10ac003a60 2024-04-25T10:09:13.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10b77fe640 1 -- 172.21.15.82:0/818172601 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v58) v1 ==== 162+0+0 (secure 0 0 0) 0x7f10a80ef6c0 con 0x7f10ac003a60 2024-04-25T10:09:13.311 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 msgr2=0x7f10a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 0x7f10a0069450 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f10c015fa40 tx=0x7f10b800b040 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 msgr2=0x7f10c0098940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f10a801c9a0 tx=0x7f10ac005840 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 shutdown_connections 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f10a0066f90 0x7f10a0069450 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10c0153f70 0x7f10c01543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10c0072500 0x7f10c015a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 --2- 172.21.15.82:0/818172601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10ac003a60 0x7f10c0098940 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 >> 172.21.15.82:0/818172601 conn(0x7f10c006ae60 msgr2=0x7f10c006d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 shutdown_connections 2024-04-25T10:09:13.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.308+0000 7f10c57f1640 1 -- 172.21.15.82:0/818172601 wait complete. 2024-04-25T10:09:13.344 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:13.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.448+0000 7f2d31fc1640 1 Processor -- start 2024-04-25T10:09:13.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d31fc1640 1 -- start start 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d31fc1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2d2c0af280 0x7f2d2c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d31fc1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c066c30 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d31fc1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c0af660 con 0x7f2d2c0b2c30 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d31fc1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c06daa0 con 0x7f2d2c0af280 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2d2c06c3b0 0x7f2d2c06df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55164/0 (socket says 172.21.15.82:55164) 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 -- 172.21.15.82:0/1510932344 learned_addr learned my addr 172.21.15.82:0/1510932344 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:13.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2b7fe640 1 --2- 172.21.15.82:0/1510932344 >> v2:172.21.15.196:3300/0 conn(0x7f2d2c0af280 0x7f2d2c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228672007 0 0) 0x7f2d2c066c30 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d18003530 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2711877939 0 0) 0x7f2d18003530 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 >> v2:172.21.15.196:3300/0 conn(0x7f2d2c0af280 msgr2=0x7f2d2c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 --2- 172.21.15.82:0/1510932344 >> v2:172.21.15.196:3300/0 conn(0x7f2d2c0af280 0x7f2d2c15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 >> v1:172.21.15.196:6789/0 conn(0x7f2d2c0b2c30 legacy=0x7f2d2c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c15e640 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2d24001680 con 0x7f2d2c06c3b0 2024-04-25T10:09:13.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 >> v1:172.21.15.82:6789/0 conn(0x7f2d2c06c3b0 legacy=0x7f2d2c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d18003a30 0x7f2d18003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d18004810 0x7f2d18004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 0x7f2d1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800c0d0 con 0x7f2d180051d0 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800c280 con 0x7f2d18004810 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1800c480 con 0x7f2d18003a30 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 0x7f2d1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d22ffd640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d18004810 0x7f2d18004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 -- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d18003a30 msgr2=0x7f2d18003e50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:09:13.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d18003a30 0x7f2d18003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 -- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d18004810 msgr2=0x7f2d18004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d18004810 0x7f2d18004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.452+0000 7f2d2bfff640 1 -- 172.21.15.82:0/1510932344 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d18003530 con 0x7f2d180051d0 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2bfff640 1 -- 172.21.15.82:0/1510932344 reap_dead start 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2bfff640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 0x7f2d1800b9d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c210 tx=0x7f2d2401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726d932b25fd490c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d24007d20 con 0x7f2d180051d0 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d240a9070 con 0x7f2d180051d0 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2affd640 1 -- 172.21.15.82:0/1510932344 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d240a5d80 con 0x7f2d180051d0 2024-04-25T10:09:13.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 msgr2=0x7f2d1800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 0x7f2d1800b9d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2d2400c210 tx=0x7f2d2401cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- 172.21.15.82:0/1510932344 shutdown_connections 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/1510932344 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d180051d0 0x7f2d1800b9d0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- 172.21.15.82:0/1510932344 >> 172.21.15.82:0/1510932344 conn(0x7f2d2c071810 msgr2=0x7f2d2c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- 172.21.15.82:0/1510932344 shutdown_connections 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- 172.21.15.82:0/1510932344 wait complete. 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 Processor -- start 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- start start 2024-04-25T10:09:13.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d180051d0 0x7f2d2c0cf5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d2c156bf0 0x7f2d2c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06bca0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c15ec00 con 0x7f2d2c156bf0 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d31fc1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c06be20 con 0x7f2d180051d0 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:54652/0 (socket says 172.21.15.82:54652) 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 -- 172.21.15.82:0/3967285909 learned_addr learned my addr 172.21.15.82:0/3967285909 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2b7fe640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d180051d0 0x7f2d2c0cf5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d2bfff640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d2c156bf0 0x7f2d2c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 -- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d180051d0 msgr2=0x7f2d2c0cf5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d180051d0 0x7f2d2c0cf5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 -- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d2c156bf0 msgr2=0x7f2d2c1570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d2c156bf0 0x7f2d2c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.456+0000 7f2d22ffd640 1 -- 172.21.15.82:0/3967285909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c1f7220 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d22ffd640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e410 tx=0x7f2d1400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14019070 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140052e0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003820 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c1f7500 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c1f6ee0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c04efa0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d1401c070 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2d140846e0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d2b7fe640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:13.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.460+0000 7f2d2b7fe640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 0x7f2d0c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009fd0 tx=0x7f2d1c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:13.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.464+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d14054f50 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.652+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d2c0727c0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.652+0000 7f2d28ff9640 1 -- 172.21.15.82:0/3967285909 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f2d14058ff0 con 0x7f2d2c1f6a60 2024-04-25T10:09:13.657 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:13.658 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:13.315577+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:13.658 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 59 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 msgr2=0x7f2d0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 0x7f2d0c069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009fd0 tx=0x7f2d1c00b040 comp rx=0 tx=0).stop 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 msgr2=0x7f2d2c15aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e410 tx=0x7f2d1400e8d0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 shutdown_connections 2024-04-25T10:09:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2d0c066f60 0x7f2d0c069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2d2c156bf0 0x7f2d2c1570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2d2c1f6a60 0x7f2d2c15aa00 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 --2- 172.21.15.82:0/3967285909 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2d180051d0 0x7f2d2c0cf5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 >> 172.21.15.82:0/3967285909 conn(0x7f2d2c071810 msgr2=0x7f2d2c0afc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 shutdown_connections 2024-04-25T10:09:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:13.656+0000 7f2d31fc1640 1 -- 172.21.15.82:0/3967285909 wait complete. 2024-04-25T10:09:14.078 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:09:13.315577+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8052': {'gid': 8052, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/362560393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 362560393}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 362560393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:14.078 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8052:up:replay 2024-04-25T10:09:15.079 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:15.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.204+0000 7f181f4e8640 1 Processor -- start 2024-04-25T10:09:15.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.204+0000 7f181f4e8640 1 -- start start 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181f4e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18180ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181f4e8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180cee30 con 0x7f1818144ca0 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181f4e8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18180ccc70 con 0x7f18181451f0 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181f4e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cc2b0 con 0x7f18180ce510 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1818144ca0 0x7f181806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35954/0 (socket says 172.21.15.82:35954) 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 -- 172.21.15.82:0/2474160068 learned_addr learned my addr 172.21.15.82:0/2474160068 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181d284640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18180ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236940234 0 0) 0x7f18180cee30 con 0x7f1818144ca0 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1804003530 con 0x7f1818144ca0 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1677061915 0 0) 0x7f1804003530 con 0x7f1818144ca0 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 msgr2=0x7f18180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18180ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 >> v1:172.21.15.196:6789/0 conn(0x7f18181451f0 legacy=0x7f1818142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181806dc40 con 0x7f1818144ca0 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1808001680 con 0x7f1818144ca0 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 >> v1:172.21.15.82:6789/0 conn(0x7f1818144ca0 legacy=0x7f181806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804003a60 0x7f1804003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804004810 0x7f1804004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cc2b0 con 0x7f1804004810 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180400bb60 con 0x7f1804003a60 2024-04-25T10:09:15.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180400bd60 con 0x7f18180ce510 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804004810 0x7f1804004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181d284640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804003a60 0x7f1804003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181da85640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 -- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 msgr2=0x7f18040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f18180ce510 0x7f18040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 -- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804003a60 msgr2=0x7f1804003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804003a60 0x7f1804003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181ca83640 1 -- 172.21.15.82:0/2474160068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804003530 con 0x7f1804004810 2024-04-25T10:09:15.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.208+0000 7f181d284640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804003a60 0x7f1804003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:15.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181ca83640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804004810 0x7f1804004c90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f181806f4c0 tx=0x7f180801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8568c6a32e8dd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808007d40 con 0x7f1804004810 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18080a9070 con 0x7f1804004810 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f180ffff640 1 -- 172.21.15.82:0/2474160068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808005d40 con 0x7f1804004810 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- 172.21.15.82:0/2474160068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804004810 msgr2=0x7f1804004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2474160068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1804004810 0x7f1804004c90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f181806f4c0 tx=0x7f180801cc60 comp rx=0 tx=0).stop 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 -- 172.21.15.82:0/2474160068 reap_dead start 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- 172.21.15.82:0/2474160068 shutdown_connections 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- 172.21.15.82:0/2474160068 >> 172.21.15.82:0/2474160068 conn(0x7f18180b11a0 msgr2=0x7f18180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- 172.21.15.82:0/2474160068 shutdown_connections 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- 172.21.15.82:0/2474160068 wait complete. 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 Processor -- start 2024-04-25T10:09:15.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- start start 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804004810 0x7f18181e1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1818141cf0 0x7f18181e3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181806ee30 con 0x7f18181e40e0 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181806e290 con 0x7f1804004810 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181f4e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180cc900 con 0x7f1818141cf0 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47932/0 (socket says 172.21.15.82:47932) 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 -- 172.21.15.82:0/2311571484 learned_addr learned my addr 172.21.15.82:0/2311571484 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:15.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181d284640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804004810 0x7f18181e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 -- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1818141cf0 msgr2=0x7f18181e3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181ca83640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1818141cf0 0x7f18181e3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1818141cf0 0x7f18181e3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 -- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804004810 msgr2=0x7f18181e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804004810 0x7f18181e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.212+0000 7f181da85640 1 -- 172.21.15.82:0/2311571484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181812edf0 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181da85640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f181806cea0 tx=0x7f181000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810019070 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100052e0 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1818142170 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810003820 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181812f340 con 0x7f18181e40e0 2024-04-25T10:09:15.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181804efa0 con 0x7f18181e40e0 2024-04-25T10:09:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f181001c070 con 0x7f18181e40e0 2024-04-25T10:09:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 0x7f17fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f18100845e0 con 0x7f18181e40e0 2024-04-25T10:09:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181d284640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 0x7f17fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:15.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.216+0000 7f181d284640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 0x7f17fc069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1814007520 tx=0x7f181400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:15.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.220+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18100554a0 con 0x7f18181e40e0 2024-04-25T10:09:15.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.408+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18180b1cc0 con 0x7f18181e40e0 2024-04-25T10:09:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.408+0000 7f180dffb640 1 -- 172.21.15.82:0/2311571484 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f1810054da0 con 0x7f18181e40e0 2024-04-25T10:09:15.415 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:15.415 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:14.331234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:15.415 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 61 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 msgr2=0x7f17fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 0x7f17fc069420 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f1814007520 tx=0x7f181400b040 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 msgr2=0x7f18180ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f181806cea0 tx=0x7f181000e820 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 shutdown_connections 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f17fc066f60 0x7f17fc069420 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1818141cf0 0x7f18181e3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f18181e40e0 0x7f18180ca250 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 --2- 172.21.15.82:0/2311571484 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1804004810 0x7f18181e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:15.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 >> 172.21.15.82:0/2311571484 conn(0x7f18180b11a0 msgr2=0x7f1818142830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 shutdown_connections 2024-04-25T10:09:15.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:15.412+0000 7f181f4e8640 1 -- 172.21.15.82:0/2311571484 wait complete. 2024-04-25T10:09:15.837 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:09:14.331234+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8052': {'gid': 8052, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/362560393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 362560393}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 362560393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:15.837 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8052:up:rejoin 2024-04-25T10:09:16.838 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.968+0000 7fdc2bc51640 1 Processor -- start 2024-04-25T10:09:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.968+0000 7fdc2bc51640 1 -- start start 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2bc51640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdc240ce510 0x7fdc240ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2bc51640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240cee30 con 0x7fdc24144ca0 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2bc51640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240ccc70 con 0x7fdc241451f0 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2bc51640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240cc2b0 con 0x7fdc240ce510 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc291ec640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdc24144ca0 0x7fdc2406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35970/0 (socket says 172.21.15.82:35970) 2024-04-25T10:09:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc291ec640 1 -- 172.21.15.82:0/463026007 learned_addr learned my addr 172.21.15.82:0/463026007 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc299ed640 1 --2- 172.21.15.82:0/463026007 >> v2:172.21.15.196:3300/0 conn(0x7fdc240ce510 0x7fdc240ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234638994 0 0) 0x7fdc240cee30 con 0x7fdc24144ca0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc10003530 con 0x7fdc24144ca0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3196864118 0 0) 0x7fdc10003530 con 0x7fdc24144ca0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 >> v2:172.21.15.196:3300/0 conn(0x7fdc240ce510 msgr2=0x7fdc240ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 --2- 172.21.15.82:0/463026007 >> v2:172.21.15.196:3300/0 conn(0x7fdc240ce510 0x7fdc240ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 >> v1:172.21.15.196:6789/0 conn(0x7fdc241451f0 legacy=0x7fdc24142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2406dc40 con 0x7fdc24144ca0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fdc14001680 con 0x7fdc24144ca0 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 >> v1:172.21.15.82:6789/0 conn(0x7fdc24144ca0 legacy=0x7fdc2406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc10003a30 0x7fdc10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc1000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cc2b0 con 0x7fdc100051d0 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1000c0e0 con 0x7fdc10003a30 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1000c2e0 con 0x7fdc10004810 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc1000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc299ed640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc10003a30 0x7fdc10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc291ec640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 -- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc10004810 msgr2=0x7fdc10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 -- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc10003a30 msgr2=0x7fdc10003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc10003a30 0x7fdc10003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc2a1ee640 1 -- 172.21.15.82:0/463026007 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc10003530 con 0x7fdc100051d0 2024-04-25T10:09:16.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.972+0000 7fdc291ec640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc10004810 0x7fdc10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2a1ee640 1 -- 172.21.15.82:0/463026007 reap_dead start 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2a1ee640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc1000b9d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001e40 tx=0x7fdc1c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff9dabf792a927d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c001a90 con 0x7fdc100051d0 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c003b80 con 0x7fdc100051d0 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc289eb640 1 -- 172.21.15.82:0/463026007 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c005020 con 0x7fdc100051d0 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 msgr2=0x7fdc1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc1000b9d0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c001e40 tx=0x7fdc1c01c760 comp rx=0 tx=0).stop 2024-04-25T10:09:16.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- 172.21.15.82:0/463026007 shutdown_connections 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/463026007 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc1000b9d0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- 172.21.15.82:0/463026007 >> 172.21.15.82:0/463026007 conn(0x7fdc240b11a0 msgr2=0x7fdc240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- 172.21.15.82:0/463026007 shutdown_connections 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- 172.21.15.82:0/463026007 wait complete. 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 Processor -- start 2024-04-25T10:09:16.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- start start 2024-04-25T10:09:16.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc241f43b0 0x7fdc241e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc241e11d0 0x7fdc240b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cc900 con 0x7fdc100051d0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2406e290 con 0x7fdc241e11d0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc2bc51640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cca80 con 0x7fdc241f43b0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc299ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.976+0000 7fdc299ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47952/0 (socket says 172.21.15.82:47952) 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 -- 172.21.15.82:0/3538755610 learned_addr learned my addr 172.21.15.82:0/3538755610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc2a1ee640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc241e11d0 0x7fdc240b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc291ec640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc241f43b0 0x7fdc241e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 -- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc241f43b0 msgr2=0x7fdc241e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc241f43b0 0x7fdc241e0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 -- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc241e11d0 msgr2=0x7fdc240b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc241e11d0 0x7fdc240b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 -- 172.21.15.82:0/3538755610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc241e1750 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc299ed640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdc200119c0 tx=0x7fdc20011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2000fa90 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000fc30 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc241f38d0 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20014440 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc240ae020 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2404efa0 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc20012040 con 0x7fdc100051d0 2024-04-25T10:09:16.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 0x7fdc08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:16.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc291ec640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 0x7fdc08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:16.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdc20085750 con 0x7fdc100051d0 2024-04-25T10:09:16.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.980+0000 7fdc291ec640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 0x7fdc08069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc24144c50 tx=0x7fdc14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:16.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:16.984+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc20019030 con 0x7fdc100051d0 2024-04-25T10:09:17.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.172+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc240b1cc0 con 0x7fdc100051d0 2024-04-25T10:09:17.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc1a7fc640 1 -- 172.21.15.82:0/3538755610 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fdc20056580 con 0x7fdc100051d0 2024-04-25T10:09:17.180 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:17.180 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":62,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:15.334147+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:17.180 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 62 2024-04-25T10:09:17.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 msgr2=0x7fdc08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 0x7fdc08069450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fdc24144c50 tx=0x7fdc14009040 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 msgr2=0x7fdc241f3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdc200119c0 tx=0x7fdc20011e80 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 shutdown_connections 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc08066f90 0x7fdc08069450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc241f43b0 0x7fdc241e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc241e11d0 0x7fdc240b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 --2- 172.21.15.82:0/3538755610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc100051d0 0x7fdc241f3020 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 >> 172.21.15.82:0/3538755610 conn(0x7fdc240b11a0 msgr2=0x7fdc241e30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 shutdown_connections 2024-04-25T10:09:17.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.176+0000 7fdc2bc51640 1 -- 172.21.15.82:0/3538755610 wait complete. 2024-04-25T10:09:17.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:09:15.334147+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 92, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8052}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8052': {'gid': 8052, 'name': 'c', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/362560393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 362560393}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 362560393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:09:17.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:09:17.601 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:09:17.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 Processor -- start 2024-04-25T10:09:17.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 -- start start 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ff0071030 0x7f5ff0161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 0x7f5ff00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff00b29e0 con 0x7f5ff0071410 2024-04-25T10:09:17.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff0162680 con 0x7f5ff0071030 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff74f5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff0162960 con 0x7f5ff0071e90 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ff0071030 0x7f5ff0161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5ff0071030 0x7f5ff0161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47964/0 (socket says 172.21.15.82:47964) 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 -- 172.21.15.82:0/2821246068 learned_addr learned my addr 172.21.15.82:0/2821246068 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/2821246068 >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 0x7f5ff00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 -- 172.21.15.82:0/2821246068 >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 msgr2=0x7f5ff00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2821246068 >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 0x7f5ff00b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 -- 172.21.15.82:0/2821246068 >> v1:172.21.15.196:6789/0 conn(0x7f5ff0071410 legacy=0x7f5ff0165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff5291640 1 -- 172.21.15.82:0/2821246068 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff0162c40 con 0x7f5ff0071030 2024-04-25T10:09:17.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.728+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/2821246068 >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 0x7f5ff00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2821246068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071030 0x7f5ff0161fb0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b1130 tx=0x7f5fe801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617de36e99c6f5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5fe7fff640 1 -- 172.21.15.82:0/2821246068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80a56f0 con 0x7f5ff0071030 2024-04-25T10:09:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5fe7fff640 1 -- 172.21.15.82:0/2821246068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe80a5890 con 0x7f5ff0071030 2024-04-25T10:09:17.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5fe7fff640 1 -- 172.21.15.82:0/2821246068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80a5c60 con 0x7f5ff0071030 2024-04-25T10:09:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 -- 172.21.15.82:0/2821246068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071030 msgr2=0x7f5ff0161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/2821246068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071030 0x7f5ff0161fb0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00b1130 tx=0x7f5fe801c920 comp rx=0 tx=0).stop 2024-04-25T10:09:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 -- 172.21.15.82:0/2821246068 shutdown_connections 2024-04-25T10:09:17.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/2821246068 >> v2:172.21.15.196:3300/0 conn(0x7f5ff0071e90 0x7f5ff00b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/2821246068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071030 0x7f5ff0161fb0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 -- 172.21.15.82:0/2821246068 >> 172.21.15.82:0/2821246068 conn(0x7f5ff00b11a0 msgr2=0x7f5ff00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 -- 172.21.15.82:0/2821246068 shutdown_connections 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.732+0000 7f5ff74f5640 1 -- 172.21.15.82:0/2821246068 wait complete. 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 Processor -- start 2024-04-25T10:09:17.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- start start 2024-04-25T10:09:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff0071030 0x7f5ff01546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ff0071410 0x7f5ff01572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff016f050 con 0x7f5ff0071e90 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff016eed0 con 0x7f5ff0071030 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff016f1d0 con 0x7f5ff0071410 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47974/0 (socket says 172.21.15.82:47974) 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 -- 172.21.15.82:0/242505602 learned_addr learned my addr 172.21.15.82:0/242505602 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5291640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff0071030 0x7f5ff01546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ff0071410 0x7f5ff01572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 -- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ff0071410 msgr2=0x7f5ff01572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ff0071410 0x7f5ff01572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 -- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff0071030 msgr2=0x7f5ff01546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff0071030 0x7f5ff01546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 -- 172.21.15.82:0/242505602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff014c360 con 0x7f5ff0071e90 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00c3c0 tx=0x7f5fec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5fe5ffb640 1 -- 172.21.15.82:0/242505602 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec011790 con 0x7f5ff0071e90 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5fe5ffb640 1 -- 172.21.15.82:0/242505602 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec017070 con 0x7f5ff0071e90 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff014c640 con 0x7f5ff0071e90 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5fe5ffb640 1 -- 172.21.15.82:0/242505602 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec011b70 con 0x7f5ff0071e90 2024-04-25T10:09:17.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.736+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff014cc00 con 0x7f5ff0071e90 2024-04-25T10:09:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fe5ffb640 1 -- 172.21.15.82:0/242505602 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5fec011d10 con 0x7f5ff0071e90 2024-04-25T10:09:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fe5ffb640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 0x7f5fd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fe5ffb640 1 -- 172.21.15.82:0/242505602 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5fec085030 con 0x7f5ff0071e90 2024-04-25T10:09:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5291640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 0x7f5fd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5291640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 0x7f5fd4069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80ae000 tx=0x7f5fe80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 Processor -- start 2024-04-25T10:09:17.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 -- start start 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f337b5f0 con 0x55b9f3493f30 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f32e2da0 con 0x55b9f34dd920 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9f326b150 con 0x55b9f34de240 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47988/0 (socket says 172.21.15.82:47988) 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 -- 172.21.15.82:0/1686471567 learned_addr learned my addr 172.21.15.82:0/1686471567 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5291640 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 -- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 msgr2=0x55b9f34939f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34939f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 -- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 msgr2=0x55b9f34ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 -- 172.21.15.82:0/1686471567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9f3284ca0 con 0x55b9f3493f30 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5fec05d940 tx=0x7f5fec0553d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf824d43093d1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fdaffd640 1 -- 172.21.15.82:0/1686471567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec06d070 con 0x55b9f3493f30 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fdaffd640 1 -- 172.21.15.82:0/1686471567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec05de70 con 0x55b9f3493f30 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.740+0000 7f5fdaffd640 1 -- 172.21.15.82:0/1686471567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec06d210 con 0x55b9f3493f30 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/1686471567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 msgr2=0x55b9f3494310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5fec05d940 tx=0x7f5fec0553d0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/1686471567 shutdown_connections 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34939f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f3494310 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- 172.21.15.82:0/1686471567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/1686471567 >> 172.21.15.82:0/1686471567 conn(0x55b9f349a2a0 msgr2=0x55b9f34922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/1686471567 shutdown_connections 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/1686471567 wait complete. 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- start start 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 Processor -- start 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f34e4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f3497aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f329d990 con 0x55b9f34dd920 2024-04-25T10:09:17.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f32dee40 con 0x55b9f34de240 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9f326e900 con 0x55b9f3493f30 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 10 client.0 _ll_register_callbacks cb 0x55b9f33961c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:47992/0 (socket says 172.21.15.82:47992) 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 -- 172.21.15.82:0/2576431370 learned_addr learned my addr 172.21.15.82:0/2576431370 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f3497aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f34e4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 -- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 msgr2=0x55b9f34e4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f34e4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 -- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 msgr2=0x55b9f3497aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f3497aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 -- 172.21.15.82:0/2576431370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9f329b630 con 0x55b9f34dd920 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f3497aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0010580 tx=0x7f5fe0010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5d98b718377e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe001a020 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5fe0016690 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fec05de70 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe0024050 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 10 client.8112 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55b9f326b3a0 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec05de70 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fe0027020 con 0x55b9f34dd920 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 10 client.8112 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:17.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 10 client.8112 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 20 client.8112 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499160 0x55b9f34eec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 4 client.8112 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b9f327ea50 con 0x55b9f3499160 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499160 0x55b9f34eec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499160 0x55b9f34eec70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37f706fe95fcfdc5 server_cookie=1c5e357268956959 in_seq=0 out_seq=0 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.744+0000 7f5fcffff640 10 client.8112 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5fcffff640 1 -- 172.21.15.82:0/2576431370 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b9f327ea50 con 0x55b9f3499160 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5fcffff640 10 client.8112 handle_command_reply: tid=0 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 client.8112 shutdown 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 msgr2=0x55b9f34e51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0010580 tx=0x7f5fe0010a40 comp rx=0 tx=0).stop 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 shutdown_connections 2024-04-25T10:09:17.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499160 0x55b9f34eec70 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f3497aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f34e51c0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5fcffff640 0 client.8112 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2576431370 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f34e4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 >> 172.21.15.82:0/2576431370 conn(0x55b9f349a2a0 msgr2=0x55b9f34929e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 shutdown_connections 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 1 -- 172.21.15.82:0/2576431370 wait complete. 2024-04-25T10:09:17.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.748+0000 7f5ffa662000 20 client.8112 trim_cache size 0 max 16384 2024-04-25T10:09:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 Processor -- start 2024-04-25T10:09:17.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 -- start start 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34e7880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0206bf0 con 0x55b9f3493f30 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0156ca0 con 0x55b9f34dd920 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.760+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0153b70 con 0x55b9f34de240 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48008/0 (socket says 172.21.15.82:48008) 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 -- 172.21.15.82:0/2253307411 learned_addr learned my addr 172.21.15.82:0/2253307411 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34e7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 -- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 msgr2=0x55b9f34e7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34e7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 -- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 msgr2=0x55b9f34dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 -- 172.21.15.82:0/2253307411 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b9f34de070 con 0x55b9f3493f30 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5291640 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f90 tx=0x7f5fe8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d31135ccd19f88b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fda7fc640 1 -- 172.21.15.82:0/2253307411 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80ce070 con 0x55b9f3493f30 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fda7fc640 1 -- 172.21.15.82:0/2253307411 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe80c8e50 con 0x55b9f3493f30 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fda7fc640 1 -- 172.21.15.82:0/2253307411 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80c8350 con 0x55b9f3493f30 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/2253307411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 msgr2=0x55b9f34dc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8000f90 tx=0x7f5fe8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/2253307411 shutdown_connections 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f34de240 0x55b9f34e7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34dd920 0x55b9f34dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- 172.21.15.82:0/2253307411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f3493f30 0x55b9f34dc1b0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/2253307411 >> 172.21.15.82:0/2253307411 conn(0x55b9f349a2a0 msgr2=0x55b9f3497220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/2253307411 shutdown_connections 2024-04-25T10:09:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/2253307411 wait complete. 2024-04-25T10:09:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- start start 2024-04-25T10:09:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 Processor -- start 2024-04-25T10:09:17.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f353f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f34f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f32e73d0 con 0x55b9f34dd920 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b9f339ed50 con 0x55b9f34de240 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b9f327d980 con 0x55b9f3493f30 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 10 client.0 _ll_register_callbacks cb 0x7f5fec051a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48010/0 (socket says 172.21.15.82:48010) 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 -- 172.21.15.82:0/3730303516 learned_addr learned my addr 172.21.15.82:0/3730303516 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f34f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 -- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 msgr2=0x55b9f353f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f353f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 -- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 msgr2=0x55b9f34f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f34f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 -- 172.21.15.82:0/3730303516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b9f3364600 con 0x55b9f34dd920 2024-04-25T10:09:17.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff5a92640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f34f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ff4a90640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001ec0 tx=0x7f5fe0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3873decc9c81c9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe001a040 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5fe0029c80 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fe80c8e50 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fe000edb0 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 10 client.8118 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55b9f32779b0 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe80c8e50 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5fe0027070 con 0x55b9f34dd920 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 10 client.8118 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 10 client.8118 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 20 client.8118 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499e00 0x55b9f34f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 4 client.8118 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.764+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b9f32f02e0 con 0x55b9f3499e00 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff5291640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499e00 0x55b9f34f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff5291640 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499e00 0x55b9f34f8c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdbf4d893dfc9612 server_cookie=c268ef8a64ec33cb in_seq=0 out_seq=0 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5fcffff640 10 client.8118 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5fcffff640 1 -- 172.21.15.82:0/3730303516 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b9f32f02e0 con 0x55b9f3499e00 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5fcffff640 10 client.8118 handle_command_reply: tid=0 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 client.8118 shutdown 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 msgr2=0x55b9f353f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0001ec0 tx=0x7f5fe0004740 comp rx=0 tx=0).stop 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 shutdown_connections 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55b9f3499e00 0x55b9f34f8c30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b9f34de240 0x55b9f34f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5fcffff640 0 client.8118 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b9f34dd920 0x55b9f353f720 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 --2- 172.21.15.82:0/3730303516 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b9f3493f30 0x55b9f353f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 >> 172.21.15.82:0/3730303516 conn(0x55b9f349a2a0 msgr2=0x55b9f34e5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 shutdown_connections 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 1 -- 172.21.15.82:0/3730303516 wait complete. 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ffa662000 20 client.8118 trim_cache size 0 max 16384 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:09:17.778 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8052, 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4502682629999999, 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 62, 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 92, 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 92, 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 5.4268329209999999 2024-04-25T10:09:17.779 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 msgr2=0x7f5fd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 0x7f5fd4069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80ae000 tx=0x7f5fe80a7070 comp rx=0 tx=0).stop 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 msgr2=0x7f5ff0157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00c3c0 tx=0x7f5fec00c880 comp rx=0 tx=0).stop 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 shutdown_connections 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5fd4066f60 0x7f5fd4069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5ff0071410 0x7f5ff01572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5ff0071e90 0x7f5ff0157830 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 --2- 172.21.15.82:0/242505602 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5ff0071030 0x7f5ff01546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 >> 172.21.15.82:0/242505602 conn(0x7f5ff00b11a0 msgr2=0x7f5ff0163d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 shutdown_connections 2024-04-25T10:09:17.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.768+0000 7f5ff74f5640 1 -- 172.21.15.82:0/242505602 wait complete. 2024-04-25T10:09:17.788 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:09:17.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 Processor -- start 2024-04-25T10:09:17.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 -- start start 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f01a006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f01a00af370 0x7f01a00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01a006ec30 con 0x7f01a006bca0 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01a00669a0 con 0x7f01a00af370 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f01a4c22640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a006c5c0 con 0x7f01a0066e30 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019e59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f01a006bca0 0x7f01a00cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:35982/0 (socket says 172.21.15.82:35982) 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019e59c640 1 -- 172.21.15.82:0/3682920854 learned_addr learned my addr 172.21.15.82:0/3682920854 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f01a006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019ed9d640 1 --2- 172.21.15.82:0/3682920854 >> v2:172.21.15.196:3300/0 conn(0x7f01a00af370 0x7f01a00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919301404 0 0) 0x7f01a006ec30 con 0x7f01a006bca0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018c003530 con 0x7f01a006bca0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 385572991 0 0) 0x7f018c003530 con 0x7f01a006bca0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 msgr2=0x7f01a006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f01a006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 >> v2:172.21.15.196:3300/0 conn(0x7f01a00af370 msgr2=0x7f01a00ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 --2- 172.21.15.82:0/3682920854 >> v2:172.21.15.196:3300/0 conn(0x7f01a00af370 0x7f01a00ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a006f230 con 0x7f01a006bca0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019ed9d640 1 --2- 172.21.15.82:0/3682920854 >> v2:172.21.15.196:3300/0 conn(0x7f01a00af370 0x7f01a00ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0198001680 con 0x7f01a006bca0 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 >> v1:172.21.15.82:6789/0 conn(0x7f01a006bca0 legacy=0x7f01a00cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018c003a30 0x7f018c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f018c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f018c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a00669a0 con 0x7f018c004810 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a006c5c0 con 0x7f018c003a30 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00bb00 con 0x7f01a0066e30 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f018c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019e59c640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018c003a30 0x7f018c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019ed9d640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f018c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 -- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 msgr2=0x7f018c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a0066e30 0x7f018c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 -- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018c003a30 msgr2=0x7f018c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018c003a30 0x7f018c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019dd9b640 1 -- 172.21.15.82:0/3682920854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c003530 con 0x7f018c004810 2024-04-25T10:09:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.896+0000 7f019e59c640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f018c003a30 0x7f018c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f019dd9b640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f018c004c90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0188009ab0 tx=0x7f018801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda39a7f295d0922 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880b2070 con 0x7f018c004810 2024-04-25T10:09:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01880a55f0 con 0x7f018c004810 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f019d59a640 1 -- 172.21.15.82:0/3682920854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01880096b0 con 0x7f018c004810 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- 172.21.15.82:0/3682920854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 msgr2=0x7f018c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 --2- 172.21.15.82:0/3682920854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f018c004c90 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f0188009ab0 tx=0x7f018801ca00 comp rx=0 tx=0).stop 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f019ed9d640 1 -- 172.21.15.82:0/3682920854 reap_dead start 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- 172.21.15.82:0/3682920854 shutdown_connections 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- 172.21.15.82:0/3682920854 >> 172.21.15.82:0/3682920854 conn(0x7f01a0071810 msgr2=0x7f01a0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- 172.21.15.82:0/3682920854 shutdown_connections 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- 172.21.15.82:0/3682920854 wait complete. 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 Processor -- start 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- start start 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a01f7740 0x7f01a0156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 0x7f01a014a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a01667a0 con 0x7f018c004810 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0166620 con 0x7f01a0158af0 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.900+0000 7f01a4c22640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a0166920 con 0x7f01a01f7740 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48034/0 (socket says 172.21.15.82:48034) 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 -- 172.21.15.82:0/941055397 learned_addr learned my addr 172.21.15.82:0/941055397 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019ed9d640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 0x7f01a014a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019dd9b640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a01f7740 0x7f01a0156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 -- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a01f7740 msgr2=0x7f01a0156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a01f7740 0x7f01a0156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 -- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 msgr2=0x7f01a014a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 0x7f01a014a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 -- 172.21.15.82:0/941055397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a0165790 con 0x7f018c004810 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019ed9d640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 0x7f01a014a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019e59c640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f019800d7f0 tx=0x7f019800dcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198019070 con 0x7f018c004810 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01980074e0 con 0x7f018c004810 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0198004a30 con 0x7f018c004810 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a0157870 con 0x7f018c004810 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a0159f80 con 0x7f018c004810 2024-04-25T10:09:17.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a004efa0 con 0x7f018c004810 2024-04-25T10:09:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0198004bd0 con 0x7f018c004810 2024-04-25T10:09:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 0x7f0180069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019dd9b640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 0x7f0180069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0198084ac0 con 0x7f018c004810 2024-04-25T10:09:17.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.904+0000 7f019dd9b640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 0x7f0180069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f01880af000 tx=0x7f01880046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:17.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:17.908+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f01980558f0 con 0x7f018c004810 2024-04-25T10:09:18.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.092+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01a00ae7f0 con 0x7f018c004810 2024-04-25T10:09:18.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.092+0000 7f0196ffd640 1 -- 172.21.15.82:0/941055397 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7f01980551f0 con 0x7f018c004810 2024-04-25T10:09:18.099 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 msgr2=0x7f0180069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 0x7f0180069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f01880af000 tx=0x7f01880046d0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 msgr2=0x7f01a0157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f019800d7f0 tx=0x7f019800dcb0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 shutdown_connections 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0180066f90 0x7f0180069450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01a01f7740 0x7f01a0156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01a0158af0 0x7f01a014a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 --2- 172.21.15.82:0/941055397 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f018c004810 0x7f01a0157330 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 >> 172.21.15.82:0/941055397 conn(0x7f01a0071810 msgr2=0x7f01a0158120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 shutdown_connections 2024-04-25T10:09:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:18.096+0000 7f01a4c22640 1 -- 172.21.15.82:0/941055397 wait complete. 2024-04-25T10:09:18.116 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:09:18.240 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:09:18.241 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:18.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.242 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:09:18.248 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:09:18.248 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:09:18.248 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:09:18.248 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:09:18.249 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42111sec preferred_lft 42111sec 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:18.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:09:18.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.251 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:09:18.306 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:09:18.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.308 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:09:18.323 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:09:18.320+0000 7f33f5da6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:09:18.333 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:09:18.333 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:09:18.334 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:09:18.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:09:18.335 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:09:18.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.408 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:09:18.456 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:09:18.456 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:09:18.457 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:18.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.457 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:09:18.512 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:09:18.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:18.513 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:18.560 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:09:18.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:09:18.560 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:18.607 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:09:18.665 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:18.713 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:09:18.714 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:09:18.715 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:18.716 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:18.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.717 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:09:18.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:09:18.764 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:09:18.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:09:18.822 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:09:18.870 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:18.871 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:18.872 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:18.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:18.873 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:09:18.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:09:18.884+0000 7fc0e9aee580 -1 init, newargv = 0x565040f40270 newargc=15 2024-04-25T10:09:18.890 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[49021]: starting ceph client 2024-04-25T10:09:18.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[49021]: starting fuse 2024-04-25T10:09:19.995 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:09:20.012 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:09:20.062 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:09:20.063 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:09:20.064 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:09:20.065 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:20.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.067 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:09:20.116 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:09:20.116 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:09:20.116 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:09:20.116 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:09:20.117 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:09:20.118 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:09:20.242 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.49021.asok 2024-04-25T10:09:20.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:20.254 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49021.asok 2024-04-25T10:09:20.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.255 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49021.asok status 2024-04-25T10:09:20.412 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:20.412 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:09:20.412 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:09:20.412 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "49021", 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:09:20.413 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8130, 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:09:20.414 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 8130 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3348601749 2024-04-25T10:09:20.415 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3348601749 2024-04-25T10:09:20.416 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.8130 192.168.0.1:0/3348601749", 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/3348601749", 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 62, 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 92, 2024-04-25T10:09:20.417 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:09:20.418 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:09:20.418 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:09:20.418 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:20.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.426 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:09:20.426 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:20.432 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:09:20.432 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:20.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.432 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:09:20.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.488 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T10:09:20.541 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:09:20.536+0000 7f33f1d9e640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:09:20.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:09:20.543 INFO:teuthology.orchestra.run.smithi082.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T10:09:20.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.544 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T10:09:20.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 Processor -- start 2024-04-25T10:09:20.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 -- start start 2024-04-25T10:09:20.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 0x7f01f40d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01f40cbad0 0x7f01f4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f406f1d0 con 0x7f01f409a970 2024-04-25T10:09:20.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f406f4b0 con 0x7f01f40cbad0 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f9d83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f406f790 con 0x7f01f415d960 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01f40cbad0 0x7f01f4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f01f40cbad0 0x7f01f4099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48040/0 (socket says 172.21.15.82:48040) 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 -- 172.21.15.82:0/1484434356 learned_addr learned my addr 172.21.15.82:0/1484434356 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f37fe640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 0x7f01f40d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 -- 172.21.15.82:0/1484434356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 msgr2=0x7f01f40d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 0x7f01f40d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 -- 172.21.15.82:0/1484434356 >> v1:172.21.15.196:6789/0 conn(0x7f01f409a970 legacy=0x7f01f40ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f2ffd640 1 -- 172.21.15.82:0/1484434356 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f406fa70 con 0x7f01f40cbad0 2024-04-25T10:09:20.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.720+0000 7f01f37fe640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 0x7f01f40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f40cbad0 0x7f01f4099a70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8ee2872b3bf049 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f27fc640 1 -- 172.21.15.82:0/1484434356 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b1070 con 0x7f01f40cbad0 2024-04-25T10:09:20.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f27fc640 1 -- 172.21.15.82:0/1484434356 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40ac620 con 0x7f01f40cbad0 2024-04-25T10:09:20.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f27fc640 1 -- 172.21.15.82:0/1484434356 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40ac9f0 con 0x7f01f40cbad0 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- 172.21.15.82:0/1484434356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f40cbad0 msgr2=0x7f01f4099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f40cbad0 0x7f01f4099a70 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009870 tx=0x7f01e401c560 comp rx=0 tx=0).stop 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- 172.21.15.82:0/1484434356 shutdown_connections 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f40cbad0 0x7f01f4099a70 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 --2- 172.21.15.82:0/1484434356 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f415d960 0x7f01f40d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- 172.21.15.82:0/1484434356 >> 172.21.15.82:0/1484434356 conn(0x7f01f40b2f70 msgr2=0x7f01f40b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- 172.21.15.82:0/1484434356 shutdown_connections 2024-04-25T10:09:20.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- 172.21.15.82:0/1484434356 wait complete. 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 Processor -- start 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.724+0000 7f01f9d83640 1 -- start start 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01f409a970 0x7f01f414a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f40cbad0 0x7f01f41546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40701e0 con 0x7f01f415d960 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40d0ae0 con 0x7f01f409a970 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4070360 con 0x7f01f40cbad0 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48046/0 (socket says 172.21.15.82:48046) 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 -- 172.21.15.82:0/2190191659 learned_addr learned my addr 172.21.15.82:0/2190191659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f37fe640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01f409a970 0x7f01f414a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f40cbad0 0x7f01f41546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 -- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f40cbad0 msgr2=0x7f01f41546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f40cbad0 0x7f01f41546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 -- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01f409a970 msgr2=0x7f01f414a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01f409a970 0x7f01f414a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 -- 172.21.15.82:0/2190191659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f414a760 con 0x7f01f415d960 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f3fff640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c770 tx=0x7f01ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 -- 172.21.15.82:0/2190191659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003b80 con 0x7f01f415d960 2024-04-25T10:09:20.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 -- 172.21.15.82:0/2190191659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec003d20 con 0x7f01f415d960 2024-04-25T10:09:20.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 -- 172.21.15.82:0/2190191659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00aa90 con 0x7f01f415d960 2024-04-25T10:09:20.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f4152fc0 con 0x7f01f415d960 2024-04-25T10:09:20.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f40c54a0 con 0x7f01f415d960 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 -- 172.21.15.82:0/2190191659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f01ec00ac30 con 0x7f01f415d960 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01ebfff640 1 -- 172.21.15.82:0/2190191659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01ec085100 con 0x7f01f415d960 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f37fe640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01f37fe640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 0x7f01d4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01fcef0000 1 Processor -- start 2024-04-25T10:09:20.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.728+0000 7f01fcef0000 1 -- start start 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583de20ffd0 con 0x5583de40cad0 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583de198170 con 0x5583de40c1b0 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583de20c070 con 0x5583de3f0f10 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48052/0 (socket says 172.21.15.82:48052) 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 -- 172.21.15.82:0/4019596947 learned_addr learned my addr 172.21.15.82:0/4019596947 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f37fe640 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f3fff640 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 -- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 msgr2=0x5583de3f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 -- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 msgr2=0x5583de40c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 -- 172.21.15.82:0/4019596947 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583de256d20 con 0x5583de40cad0 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f01e401a190 tx=0x7f01e40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87a2688d50b0f9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01e97fa640 1 -- 172.21.15.82:0/4019596947 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40bc070 con 0x5583de40cad0 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01e97fa640 1 -- 172.21.15.82:0/4019596947 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40ac880 con 0x5583de40cad0 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01e97fa640 1 -- 172.21.15.82:0/4019596947 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b1040 con 0x5583de40cad0 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- 172.21.15.82:0/4019596947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 msgr2=0x5583de3f09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f01e401a190 tx=0x7f01e40a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- 172.21.15.82:0/4019596947 shutdown_connections 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de40cad0 0x5583de3f09d0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- 172.21.15.82:0/4019596947 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- 172.21.15.82:0/4019596947 >> 172.21.15.82:0/4019596947 conn(0x5583de3c8710 msgr2=0x5583de3c8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- 172.21.15.82:0/4019596947 shutdown_connections 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- 172.21.15.82:0/4019596947 wait complete. 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- start start 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 Processor -- start 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de413e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 0x5583de3f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583de2cdc10 con 0x5583de3f0f10 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583de1ca680 con 0x5583de40c1b0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583de214600 con 0x5583de40cad0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01fcef0000 10 client.0 _ll_register_callbacks cb 0x5583de2c5080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48054/0 (socket says 172.21.15.82:48054) 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f37fe640 1 -- 172.21.15.82:0/450296427 learned_addr learned my addr 172.21.15.82:0/450296427 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de413e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f3fff640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 0x5583de3f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.732+0000 7f01f37fe640 1 -- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 msgr2=0x5583de3f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f37fe640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 0x5583de3f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f37fe640 1 -- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 msgr2=0x5583de413e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f37fe640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de413e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f37fe640 1 -- 172.21.15.82:0/450296427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583de1e7d30 con 0x5583de3f0f10 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f3fff640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 0x5583de3f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f37fe640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0137d0 tx=0x7f01dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc848030cbe8efba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc024020 con 0x5583de3f0f10 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01dc014980 con 0x5583de3f0f10 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e40ac880 con 0x5583de3f0f10 2024-04-25T10:09:20.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01dc02e050 con 0x5583de3f0f10 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 10 client.8142 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5583de1983c0 con 0x5583de3f0f10 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e40ac880 con 0x5583de3f0f10 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01dc031020 con 0x5583de3f0f10 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 10 client.8142 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 10 client.8142 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 20 client.8142 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de3f2090 0x5583de3f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 4 client.8142 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5583de1abe00 con 0x5583de3f2090 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de3f2090 0x5583de3f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 10 client.8142 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de3f2090 0x5583de3f24a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1017de42e03ee1 server_cookie=7d7347d3c1160835 in_seq=0 out_seq=0 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 1 -- 172.21.15.82:0/450296427 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5583de1abe00 con 0x5583de3f2090 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 10 client.8142 handle_command_reply: tid=0 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 client.8142 shutdown 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 msgr2=0x5583de413930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0137d0 tx=0x7f01dc004740 comp rx=0 tx=0).stop 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 shutdown_connections 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de3f2090 0x5583de3f24a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de40cad0 0x5583de3f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01d27fc640 0 client.8142 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de413e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 --2- 172.21.15.82:0/450296427 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de413930 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 >> 172.21.15.82:0/450296427 conn(0x5583de3c8710 msgr2=0x5583de3c0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 shutdown_connections 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 1 -- 172.21.15.82:0/450296427 wait complete. 2024-04-25T10:09:20.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.736+0000 7f01fcef0000 20 client.8142 trim_cache size 0 max 16384 2024-04-25T10:09:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 Processor -- start 2024-04-25T10:09:20.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- start start 2024-04-25T10:09:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3c8710 0x5583de40a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4203620 con 0x5583de3f0f10 2024-04-25T10:09:20.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f42027f0 con 0x5583de40c1b0 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40c6b50 con 0x5583de3c8710 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48066/0 (socket says 172.21.15.82:48066) 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 -- 172.21.15.82:0/2160710785 learned_addr learned my addr 172.21.15.82:0/2160710785 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 -- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3c8710 msgr2=0x5583de40a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3c8710 0x5583de40a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f3fff640 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 -- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 msgr2=0x5583de40afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 -- 172.21.15.82:0/2160710785 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583de2920f0 con 0x5583de3f0f10 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f01e4008010 tx=0x7f01e4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac16f9e18086a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01e8ff9640 1 -- 172.21.15.82:0/2160710785 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40bc040 con 0x5583de3f0f10 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01e8ff9640 1 -- 172.21.15.82:0/2160710785 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4006a30 con 0x5583de3f0f10 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01e8ff9640 1 -- 172.21.15.82:0/2160710785 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40b1040 con 0x5583de3f0f10 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- 172.21.15.82:0/2160710785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 msgr2=0x5583de40aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f01e4008010 tx=0x7f01e4004740 comp rx=0 tx=0).stop 2024-04-25T10:09:20.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- 172.21.15.82:0/2160710785 shutdown_connections 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de40afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3f0f10 0x5583de40aa70 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 --2- 172.21.15.82:0/2160710785 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3c8710 0x5583de40a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- 172.21.15.82:0/2160710785 >> 172.21.15.82:0/2160710785 conn(0x5583de419d20 msgr2=0x5583de3f05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- 172.21.15.82:0/2160710785 shutdown_connections 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.752+0000 7f01fcef0000 1 -- 172.21.15.82:0/2160710785 wait complete. 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- start start 2024-04-25T10:09:20.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 Processor -- start 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3c2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de432800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583de2cdc10 con 0x5583de3c8710 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f42022c0 con 0x5583de40c1b0 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583de1d3860 con 0x5583de3f0f10 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 10 client.0 _ll_register_callbacks cb 0x5583de1adb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48076/0 (socket says 172.21.15.82:48076) 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 -- 172.21.15.82:0/626493259 learned_addr learned my addr 172.21.15.82:0/626493259 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f3fff640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de432800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 -- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 msgr2=0x5583de3c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 -- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 msgr2=0x5583de432800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de432800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 -- 172.21.15.82:0/626493259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583de1a4d20 con 0x5583de3c8710 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f37fe640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f01dc02c040 tx=0x7f01dc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715c6b7c0fb9425e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc037060 con 0x5583de3c8710 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01dc0149d0 con 0x5583de3c8710 2024-04-25T10:09:20.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01e4006a30 con 0x5583de3c8710 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01dc03d050 con 0x5583de3c8710 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 10 client.8148 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x5583de1b70c0 con 0x5583de3c8710 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4006a30 con 0x5583de3c8710 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f01dc040020 con 0x5583de3c8710 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 10 client.8148 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 10 client.8148 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 20 client.8148 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de40cad0 0x5583de432d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 4 client.8148 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x5583de21d510 con 0x5583de40cad0 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de40cad0 0x5583de432d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 10 client.8148 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01f2ffd640 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de40cad0 0x5583de432d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7641e1e7e5dbdbe server_cookie=59c9489074524df7 in_seq=0 out_seq=0 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 1 -- 172.21.15.82:0/626493259 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5583de21d510 con 0x5583de40cad0 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 10 client.8148 handle_command_reply: tid=0 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 client.8148 shutdown 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 msgr2=0x5583de3c2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f01dc02c040 tx=0x7f01dc0046d0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 shutdown_connections 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x5583de40cad0 0x5583de432d40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5583de3f0f10 0x5583de3c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5583de40c1b0 0x5583de432800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 --2- 172.21.15.82:0/626493259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5583de3c8710 0x5583de3c2020 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01d27fc640 0 client.8148 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 >> 172.21.15.82:0/626493259 conn(0x5583de419d20 msgr2=0x5583de3c6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.770 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f33f1d9e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0) n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x56527a957b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:09:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 shutdown_connections 2024-04-25T10:09:20.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 1 -- 172.21.15.82:0/626493259 wait complete. 2024-04-25T10:09:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.756+0000 7f01fcef0000 20 client.8148 trim_cache size 0 max 16384 2024-04-25T10:09:20.771 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:20.771 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:09:20.771 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "1bc50045-e6e9-4be8-9b29-9349eba1f715", 2024-04-25T10:09:20.771 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:09:20.772 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 msgr2=0x7f01d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 0x7f01d4069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f01dc006fd0 tx=0x7f01dc008040 comp rx=0 tx=0).stop 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 msgr2=0x7f01f4154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00c770 tx=0x7f01ec00cc30 comp rx=0 tx=0).stop 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 shutdown_connections 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f01d4066f60 0x7f01d4069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f01f40cbad0 0x7f01f41546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f01f415d960 0x7f01f4154c10 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 --2- 172.21.15.82:0/2190191659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f01f409a970 0x7f01f414a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 >> 172.21.15.82:0/2190191659 conn(0x7f01f40b2f70 msgr2=0x7f01f406cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 shutdown_connections 2024-04-25T10:09:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.760+0000 7f01f9d83640 1 -- 172.21.15.82:0/2190191659 wait complete. 2024-04-25T10:09:20.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:20.780 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:20.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.888+0000 7f101b61c640 1 Processor -- start 2024-04-25T10:09:20.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.888+0000 7f101b61c640 1 -- start start 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140cf590 0x7f101415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f10140cf970 0x7f10140c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10140ca4c0 con 0x7f10140c9ef0 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140ca780 con 0x7f10140cf970 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10140cd440 con 0x7f10140cf590 2024-04-25T10:09:20.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f10140cf970 0x7f10140c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f10140cf970 0x7f10140c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48082/0 (socket says 172.21.15.82:48082) 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 -- 172.21.15.82:0/3301479174 learned_addr learned my addr 172.21.15.82:0/3301479174 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f10193b8640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140cf590 0x7f101415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 -- 172.21.15.82:0/3301479174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140cf590 msgr2=0x7f101415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140cf590 0x7f101415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 -- 172.21.15.82:0/3301479174 >> v1:172.21.15.196:6789/0 conn(0x7f10140c9ef0 legacy=0x7f10140b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 -- 172.21.15.82:0/3301479174 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140cd690 con 0x7f10140cf970 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f1018bb7640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf970 0x7f10140c99b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1004009870 tx=0x7f100401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632c7f9ef02cad49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f100bfff640 1 -- 172.21.15.82:0/3301479174 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040b0070 con 0x7f10140cf970 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f100bfff640 1 -- 172.21.15.82:0/3301479174 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10040ab600 con 0x7f10140cf970 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f100bfff640 1 -- 172.21.15.82:0/3301479174 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040ab9d0 con 0x7f10140cf970 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- 172.21.15.82:0/3301479174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf970 msgr2=0x7f10140c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf970 0x7f10140c99b0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f1004009870 tx=0x7f100401c520 comp rx=0 tx=0).stop 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- 172.21.15.82:0/3301479174 shutdown_connections 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf970 0x7f10140c99b0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 --2- 172.21.15.82:0/3301479174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140cf590 0x7f101415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.892+0000 7f101b61c640 1 -- 172.21.15.82:0/3301479174 >> 172.21.15.82:0/3301479174 conn(0x7f101406ae60 msgr2=0x7f101406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- 172.21.15.82:0/3301479174 shutdown_connections 2024-04-25T10:09:20.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- 172.21.15.82:0/3301479174 wait complete. 2024-04-25T10:09:20.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 Processor -- start 2024-04-25T10:09:20.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- start start 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140c9ef0 0x7f10140c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10140cf970 0x7f10141555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1014167660 con 0x7f10140cf590 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141674e0 con 0x7f10140cf970 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10141677e0 con 0x7f10140c9ef0 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48090/0 (socket says 172.21.15.82:48090) 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 -- 172.21.15.82:0/856160861 learned_addr learned my addr 172.21.15.82:0/856160861 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 -- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140c9ef0 msgr2=0x7f10140c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140c9ef0 0x7f10140c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 -- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10140cf970 msgr2=0x7f10141555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10140cf970 0x7f10141555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 -- 172.21.15.82:0/856160861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014156040 con 0x7f10140cf590 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1018bb7640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f10040ae040 tx=0x7f10040192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1009ffb640 1 -- 172.21.15.82:0/856160861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040b0040 con 0x7f10140cf590 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1009ffb640 1 -- 172.21.15.82:0/856160861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10040b4640 con 0x7f10140cf590 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10141569a0 con 0x7f10140cf590 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f1009ffb640 1 -- 172.21.15.82:0/856160861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10040b4a30 con 0x7f10140cf590 2024-04-25T10:09:20.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.896+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1014156f60 con 0x7f10140cf590 2024-04-25T10:09:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f1009ffb640 1 -- 172.21.15.82:0/856160861 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10040ab4c0 con 0x7f10140cf590 2024-04-25T10:09:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f1009ffb640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 0x7f0ff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 0x7f0ff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f1009ffb640 1 -- 172.21.15.82:0/856160861 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f100411ef30 con 0x7f10140cf590 2024-04-25T10:09:20.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 0x7f0ff8069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f10140c5b20 tx=0x7f1010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 Processor -- start 2024-04-25T10:09:20.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 -- start start 2024-04-25T10:09:20.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475ddde750 0x56475dd93f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475dd94890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475dbdcff0 con 0x56475dddde30 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475db65150 con 0x56475ddde750 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56475dbd9090 con 0x56475dd944b0 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48104/0 (socket says 172.21.15.82:48104) 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 -- 172.21.15.82:0/2314184034 learned_addr learned my addr 172.21.15.82:0/2314184034 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 -- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 msgr2=0x56475dd94890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475dd94890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 -- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475ddde750 msgr2=0x56475dd93f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475ddde750 0x56475dd93f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 -- 172.21.15.82:0/2314184034 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56475dd730d0 con 0x56475dddde30 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f10193b8640 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f101000ca90 tx=0x7f101000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167a01216dd8d0e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f0ffeffd640 1 -- 172.21.15.82:0/2314184034 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010023070 con 0x56475dddde30 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f0ffeffd640 1 -- 172.21.15.82:0/2314184034 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010013db0 con 0x56475dddde30 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.900+0000 7f0ffeffd640 1 -- 172.21.15.82:0/2314184034 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101001e600 con 0x56475dddde30 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/2314184034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 msgr2=0x56475ddde210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f101000ca90 tx=0x7f101000f000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/2314184034 shutdown_connections 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475dd94890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475ddde750 0x56475dd93f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- 172.21.15.82:0/2314184034 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddde210 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/2314184034 >> 172.21.15.82:0/2314184034 conn(0x56475dd8c020 msgr2=0x56475dd8e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/2314184034 shutdown_connections 2024-04-25T10:09:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/2314184034 wait complete. 2024-04-25T10:09:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- start start 2024-04-25T10:09:20.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 Processor -- start 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd944b0 0x56475dddd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dddde30 0x56475dd9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475dbe1620 con 0x56475ddde750 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475db97690 con 0x56475dd944b0 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56475db65a40 con 0x56475dddde30 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 10 client.0 _ll_register_callbacks cb 0x56475dd7ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48106/0 (socket says 172.21.15.82:48106) 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 -- 172.21.15.82:0/3354171608 learned_addr learned my addr 172.21.15.82:0/3354171608 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1018bb7640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dddde30 0x56475dd9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f10193b8640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd944b0 0x56475dddd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 -- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dddde30 msgr2=0x56475dd9a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dddde30 0x56475dd9a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 -- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd944b0 msgr2=0x56475dddd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd944b0 0x56475dddd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 -- 172.21.15.82:0/3354171608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56475dc8a1c0 con 0x56475ddde750 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f1019bb9640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c410 tx=0x7f100c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62953130d74db6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:20.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c011790 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f100c011930 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1010013db0 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f100c01f050 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 10 client.8160 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x56475dc86170 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c017070 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f100c011b70 con 0x56475ddde750 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 10 client.8160 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 10 client.8160 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 20 client.8160 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475dd9b5e0 0x56475ddf00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 4 client.8160 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:20.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1010013db0 con 0x56475dd9b5e0 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f10193b8640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475dd9b5e0 0x56475ddf00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f10193b8640 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475dd9b5e0 0x56475ddf00f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6778f6e4cf777bd server_cookie=78f7e236506b0b95 in_seq=0 out_seq=0 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.904+0000 7f0ff3fff640 10 client.8160 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f0ff3fff640 1 -- 172.21.15.82:0/3354171608 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f1010013db0 con 0x56475dd9b5e0 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f0ff3fff640 10 client.8160 handle_command_reply: tid=0 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 client.8160 shutdown 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 msgr2=0x56475dd9a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f100c00c410 tx=0x7f100c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 shutdown_connections 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475dd9b5e0 0x56475ddf00f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dddde30 0x56475dd9a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f0ff3fff640 0 client.8160 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475ddde750 0x56475dd9a8a0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 --2- 172.21.15.82:0/3354171608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd944b0 0x56475dddd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 >> 172.21.15.82:0/3354171608 conn(0x56475dd8c020 msgr2=0x56475dd95120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 shutdown_connections 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 1 -- 172.21.15.82:0/3354171608 wait complete. 2024-04-25T10:09:20.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.908+0000 7f101e789000 20 client.8160 trim_cache size 0 max 16384 2024-04-25T10:09:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.920+0000 7f101e789000 1 Processor -- start 2024-04-25T10:09:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- start start 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd8c020 0x56475ddbcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddbbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10141fef00 con 0x56475dddde30 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101415d120 con 0x56475dd8c020 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10140ccbe0 con 0x56475dd944b0 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48120/0 (socket says 172.21.15.82:48120) 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 -- 172.21.15.82:0/1881662966 learned_addr learned my addr 172.21.15.82:0/1881662966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1018bb7640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddbbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 -- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 msgr2=0x56475ddbbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddbbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 -- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd8c020 msgr2=0x56475ddbcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f10193b8640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd8c020 0x56475ddbcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd8c020 0x56475ddbcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 -- 172.21.15.82:0/1881662966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56475ddbf980 con 0x56475dddde30 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1018bb7640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddbbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f1019bb9640 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f100c003d00 tx=0x7f100c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64d3f3f39b48f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f0ffe7fc640 1 -- 172.21.15.82:0/1881662966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c027020 con 0x56475dddde30 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f0ffe7fc640 1 -- 172.21.15.82:0/1881662966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c017070 con 0x56475dddde30 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f0ffe7fc640 1 -- 172.21.15.82:0/1881662966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0119d0 con 0x56475dddde30 2024-04-25T10:09:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- 172.21.15.82:0/1881662966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 msgr2=0x56475ddbc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f100c003d00 tx=0x7f100c011000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- 172.21.15.82:0/1881662966 shutdown_connections 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddbbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dddde30 0x56475ddbc000 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- 172.21.15.82:0/1881662966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dd8c020 0x56475ddbcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- 172.21.15.82:0/1881662966 >> 172.21.15.82:0/1881662966 conn(0x56475dd95080 msgr2=0x56475dd8e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- 172.21.15.82:0/1881662966 shutdown_connections 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- 172.21.15.82:0/1881662966 wait complete. 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- start start 2024-04-25T10:09:20.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 Processor -- start 2024-04-25T10:09:20.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddf1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 0x56475ddfbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475db77e00 con 0x56475dd8c020 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101415cb20 con 0x56475dddde30 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56475dbdc300 con 0x56475dd944b0 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f101e789000 10 client.0 _ll_register_callbacks cb 0x56475dd5b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f10193b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f10193b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:48122/0 (socket says 172.21.15.82:48122) 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.924+0000 7f10193b8640 1 -- 172.21.15.82:0/2428457470 learned_addr learned my addr 172.21.15.82:0/2428457470 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f1019bb9640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 0x56475ddfbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 -- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 msgr2=0x56475ddf1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f1018bb7640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddf1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddf1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 -- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 msgr2=0x56475ddfbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 0x56475ddfbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 -- 172.21.15.82:0/2428457470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56475dc23de0 con 0x56475dd8c020 2024-04-25T10:09:20.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f1019bb9640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 0x56475ddfbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f10193b8640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f101000ca30 tx=0x7f1010011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ae3690b449586d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010023040 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1010013d90 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f100c017070 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101003c050 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 10 client.8166 fetch_fsmap learned FSMap version 62 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x56475db7e8e0 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c017070 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 62) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f101003f020 con 0x56475dd8c020 2024-04-25T10:09:20.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 10 client.8166 fetch_fsmap finished waiting for FSMap version 62 2024-04-25T10:09:20.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 10 client.8166 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:20.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 20 client.8166 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:20.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475ddde750 0x56475ddfc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:20.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 4 client.8166 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56475db78c60 con 0x56475ddde750 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f1018bb7640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475ddde750 0x56475ddfc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 10 client.8166 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f1018bb7640 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475ddde750 0x56475ddfc0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd375a6885466afc server_cookie=8438dbb750e34d4 in_seq=0 out_seq=0 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 1 -- 172.21.15.82:0/2428457470 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x56475db78c60 con 0x56475ddde750 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 10 client.8166 handle_command_reply: tid=0 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 client.8166 shutdown 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 msgr2=0x56475ddf15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f101000ca30 tx=0x7f1010011000 comp rx=0 tx=0).stop 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 shutdown_connections 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56475ddde750 0x56475ddfc0c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56475dd944b0 0x56475ddf1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56475dddde30 0x56475ddfbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 --2- 172.21.15.82:0/2428457470 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56475dd8c020 0x56475ddf15a0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f0ff3fff640 0 client.8166 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 >> 172.21.15.82:0/2428457470 conn(0x56475dd95080 msgr2=0x56475dd8eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 shutdown_connections 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 1 -- 172.21.15.82:0/2428457470 wait complete. 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101e789000 20 client.8166 trim_cache size 0 max 16384 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:20.944 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: "1bc50045-e6e9-4be8-9b29-9349eba1f715": { 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir", 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "1bc50045-e6e9-4be8-9b29-9349eba1f715", 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive" 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:20.945 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 msgr2=0x7f0ff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 0x7f0ff8069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f10140c5b20 tx=0x7f1010008040 comp rx=0 tx=0).stop 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 msgr2=0x7f1014157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f10040ae040 tx=0x7f10040192c0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 shutdown_connections 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0ff8066f60 0x7f0ff8069420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f10140cf970 0x7f10141555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f10140cf590 0x7f1014157760 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 --2- 172.21.15.82:0/856160861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f10140c9ef0 0x7f10140c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:20.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 >> 172.21.15.82:0/856160861 conn(0x7f101406ae60 msgr2=0x7f101406d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:20.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 shutdown_connections 2024-04-25T10:09:20.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:20.928+0000 7f101b61c640 1 -- 172.21.15.82:0/856160861 wait complete. 2024-04-25T10:09:20.949 INFO:tasks.cephfs.filesystem:scrub status for tag:1bc50045-e6e9-4be8-9b29-9349eba1f715 - {'path': '/dir', 'tag': '1bc50045-e6e9-4be8-9b29-9349eba1f715', 'options': 'recursive'} 2024-04-25T10:09:23.464 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:09:23.460+0000 7ff48c596640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:09:38.021 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:38.024 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:50.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:50.950 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 Processor -- start 2024-04-25T10:09:51.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 -- start start 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9cc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff9cc15ca90 0x7ff9cc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9cc06b5b0 con 0x7ff9cc0cc130 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9cc15ff70 con 0x7ff9cc15ca90 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d3829640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc0d0300 con 0x7ff9cc06ac90 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d1dc6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff9cc0cc130 0x7ff9cc0ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58068/0 (socket says 172.21.15.82:58068) 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.083+0000 7ff9d1dc6640 1 -- 172.21.15.82:0/4264214834 learned_addr learned my addr 172.21.15.82:0/4264214834 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4264214834 >> v2:172.21.15.196:3300/0 conn(0x7ff9cc15ca90 0x7ff9cc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9cc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1543364097 0 0) 0x7ff9cc06b5b0 con 0x7ff9cc0cc130 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b8003530 con 0x7ff9cc0cc130 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 288578020 0 0) 0x7ff9b8003530 con 0x7ff9cc0cc130 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 msgr2=0x7ff9cc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9cc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 >> v2:172.21.15.196:3300/0 conn(0x7ff9cc15ca90 msgr2=0x7ff9cc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 --2- 172.21.15.82:0/4264214834 >> v2:172.21.15.196:3300/0 conn(0x7ff9cc15ca90 0x7ff9cc0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc15f360 con 0x7ff9cc0cc130 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff9bc001680 con 0x7ff9cc0cc130 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 >> v1:172.21.15.82:6789/0 conn(0x7ff9cc0cc130 legacy=0x7ff9cc0ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b8003a30 0x7ff9b8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8004d50 0x7ff9b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b800b9d0 con 0x7ff9b8003a30 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b800bb50 con 0x7ff9b8004d50 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b800bcd0 con 0x7ff9cc06ac90 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b8003a30 0x7ff9b8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8004d50 0x7ff9b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 msgr2=0x7ff9b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8004d50 msgr2=0x7ff9b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9b8004d50 0x7ff9b80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4264214834 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8003530 con 0x7ff9b8003a30 2024-04-25T10:09:51.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9cc06ac90 0x7ff9b8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b8003a30 0x7ff9b8003e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009fc0 tx=0x7ff9c401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab66912d082ebcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b2070 con 0x7ff9b8003a30 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40a5c40 con 0x7ff9b8003a30 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9c3fff640 1 -- 172.21.15.82:0/4264214834 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4009c10 con 0x7ff9b8003a30 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d3829640 1 -- 172.21.15.82:0/4264214834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b8003a30 msgr2=0x7ff9b8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d3829640 1 --2- 172.21.15.82:0/4264214834 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9b8003a30 0x7ff9b8003e50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009fc0 tx=0x7ff9c401cc50 comp rx=0 tx=0).stop 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.087+0000 7ff9d1dc6640 1 -- 172.21.15.82:0/4264214834 reap_dead start 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/4264214834 shutdown_connections 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/4264214834 >> 172.21.15.82:0/4264214834 conn(0x7ff9cc099a70 msgr2=0x7ff9cc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/4264214834 shutdown_connections 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/4264214834 wait complete. 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 Processor -- start 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- start start 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b8003a30 0x7ff9cc14b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 0x7ff9cc09d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc159750 con 0x7ff9cc162630 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc1595d0 con 0x7ff9cc09d410 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc166650 con 0x7ff9b8003a30 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58478/0 (socket says 172.21.15.82:58478) 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/3287773507 learned_addr learned my addr 172.21.15.82:0/3287773507 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b8003a30 0x7ff9cc14b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b8003a30 msgr2=0x7ff9cc14b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 0x7ff9cc09d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b8003a30 0x7ff9cc14b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 msgr2=0x7ff9cc09d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 0x7ff9cc09d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/3287773507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9cc0b5870 con 0x7ff9cc162630 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 0x7ff9cc09d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8010980 tx=0x7ff9c8010e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9c1ffb640 1 -- 172.21.15.82:0/3287773507 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800f040 con 0x7ff9cc162630 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9c1ffb640 1 -- 172.21.15.82:0/3287773507 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c800ec20 con 0x7ff9cc162630 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9cc0c46c0 con 0x7ff9cc162630 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9c1ffb640 1 -- 172.21.15.82:0/3287773507 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8004020 con 0x7ff9cc162630 2024-04-25T10:09:51.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.091+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9cc0c4c80 con 0x7ff9cc162630 2024-04-25T10:09:51.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9c1ffb640 1 -- 172.21.15.82:0/3287773507 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9c80041c0 con 0x7ff9cc162630 2024-04-25T10:09:51.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9c1ffb640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 0x7ff9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9c1ffb640 1 -- 172.21.15.82:0/3287773507 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff9c8085600 con 0x7ff9cc162630 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 0x7ff9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 0x7ff9b0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40af000 tx=0x7ff9c40a7040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 Processor -- start 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 -- start start 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e543cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e543d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5289ff0 con 0x55c5e54869d0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5212150 con 0x55c5e54872f0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5286090 con 0x55c5e543d0a0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58494/0 (socket says 172.21.15.82:58494) 2024-04-25T10:09:51.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 -- 172.21.15.82:0/3249195997 learned_addr learned my addr 172.21.15.82:0/3249195997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e543cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e543d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 -- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 msgr2=0x55c5e543d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e543d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 -- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 msgr2=0x55c5e543cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e543cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 -- 172.21.15.82:0/3249195997 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e54200d0 con 0x55c5e54869d0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e543cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401ca80 tx=0x7ff9c401b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f794c308b0f363b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9b6ffd640 1 -- 172.21.15.82:0/3249195997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40ad910 con 0x55c5e54869d0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9b6ffd640 1 -- 172.21.15.82:0/3249195997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40c8070 con 0x55c5e54869d0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.095+0000 7ff9b6ffd640 1 -- 172.21.15.82:0/3249195997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40adcf0 con 0x55c5e54869d0 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/3249195997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 msgr2=0x55c5e5486db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401ca80 tx=0x7ff9c401b490 comp rx=0 tx=0).stop 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/3249195997 shutdown_connections 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e543d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e543cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- 172.21.15.82:0/3249195997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e5486db0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/3249195997 >> 172.21.15.82:0/3249195997 conn(0x55c5e5439020 msgr2=0x55c5e543b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/3249195997 shutdown_connections 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/3249195997 wait complete. 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- start start 2024-04-25T10:09:51.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 Processor -- start 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e546bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e546c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e524e250 con 0x55c5e543d0a0 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5244690 con 0x55c5e54869d0 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5212a40 con 0x55c5e54872f0 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 10 client.0 _ll_register_callbacks cb 0x55c5e5427e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58502/0 (socket says 172.21.15.82:58502) 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 -- 172.21.15.82:0/1930072322 learned_addr learned my addr 172.21.15.82:0/1930072322 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e546bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e546c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 -- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 msgr2=0x55c5e546c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e546c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 -- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 msgr2=0x55c5e546bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e546bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 -- 172.21.15.82:0/1930072322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5e53371c0 con 0x55c5e543d0a0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e546c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401ca80 tx=0x7ff9b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4a08343b99fbc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40add60 con 0x55c5e543d0a0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff9c40d7cf0 con 0x55c5e543d0a0 2024-04-25T10:09:51.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c40c8070 con 0x55c5e543d0a0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c40de050 con 0x55c5e543d0a0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 10 client.8178 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55c5e5333170 con 0x55c5e543d0a0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c40c8070 con 0x55c5e543d0a0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9c40e1020 con 0x55c5e543d0a0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 10 client.8178 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 10 client.8178 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 20 client.8178 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e5496c80 0x55c5e5499070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 4 client.8178 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5e5225c60 con 0x55c5e5496c80 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e5496c80 0x55c5e5499070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9abfff640 10 client.8178 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.099+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e5496c80 0x55c5e5499070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1cee8e0c72b1eb server_cookie=7c1ffa62be60d12f in_seq=0 out_seq=0 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9abfff640 1 -- 172.21.15.82:0/1930072322 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5e5225c60 con 0x55c5e5496c80 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9abfff640 10 client.8178 handle_command_reply: tid=0 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 client.8178 shutdown 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 msgr2=0x55c5e546b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401ca80 tx=0x7ff9b8005840 comp rx=0 tx=0).stop 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 shutdown_connections 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e5496c80 0x55c5e5499070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e546c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e546bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 --2- 172.21.15.82:0/1930072322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e546b7f0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9abfff640 0 client.8178 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 >> 172.21.15.82:0/1930072322 conn(0x55c5e5439020 msgr2=0x55c5e543bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 shutdown_connections 2024-04-25T10:09:51.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 1 -- 172.21.15.82:0/1930072322 wait complete. 2024-04-25T10:09:51.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.103+0000 7ff9d6996000 20 client.8178 trim_cache size 0 max 16384 2024-04-25T10:09:51.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.115+0000 7ff9d6996000 1 Processor -- start 2024-04-25T10:09:51.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.115+0000 7ff9d6996000 1 -- start start 2024-04-25T10:09:51.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e54e3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e54e3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc201ec0 con 0x55c5e54869d0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc2010c0 con 0x55c5e54872f0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc2004c0 con 0x55c5e543d0a0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58510/0 (socket says 172.21.15.82:58510) 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/2557425373 learned_addr learned my addr 172.21.15.82:0/2557425373 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e54e3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 msgr2=0x55c5e54e3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e54e3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 msgr2=0x55c5e54e3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e54e3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 -- 172.21.15.82:0/2557425373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5e52d0de0 con 0x55c5e54869d0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8055ce0 tx=0x7ff9c8083840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e1c4cdf0fba7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9b67fc640 1 -- 172.21.15.82:0/2557425373 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8021040 con 0x55c5e54869d0 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9b67fc640 1 -- 172.21.15.82:0/2557425373 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c806d070 con 0x55c5e54869d0 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9b67fc640 1 -- 172.21.15.82:0/2557425373 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c805e460 con 0x55c5e54869d0 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- 172.21.15.82:0/2557425373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 msgr2=0x55c5e54e3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8055ce0 tx=0x7ff9c8083840 comp rx=0 tx=0).stop 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- 172.21.15.82:0/2557425373 shutdown_connections 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54872f0 0x55c5e54e3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e54869d0 0x55c5e54e3840 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- 172.21.15.82:0/2557425373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e543d0a0 0x55c5e54e3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- 172.21.15.82:0/2557425373 >> 172.21.15.82:0/2557425373 conn(0x55c5e5439020 msgr2=0x55c5e5493fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- 172.21.15.82:0/2557425373 shutdown_connections 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- 172.21.15.82:0/2557425373 wait complete. 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- start start 2024-04-25T10:09:51.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 Processor -- start 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e5499cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e549a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5224e00 con 0x55c5e543d0a0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9cc200ac0 con 0x55c5e54869d0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5e5289300 con 0x55c5e54872f0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d6996000 10 client.0 _ll_register_callbacks cb 0x55c5e5408040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58526/0 (socket says 172.21.15.82:58526) 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4006630850 learned_addr learned my addr 172.21.15.82:0/4006630850 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e549a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e5499cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 msgr2=0x55c5e549a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e549a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 msgr2=0x55c5e5499cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e5499cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d15c5640 1 -- 172.21.15.82:0/4006630850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5e522b8e0 con 0x55c5e543d0a0 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.119+0000 7ff9d1dc6640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e549a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.128 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d15c5640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40b9740 tx=0x7ff9c4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd849aa243b0e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40e7060 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff9c40adc20 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c806d070 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c40b9770 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 10 client.8184 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55c5e52e38f0 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c806d070 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff9c40e1070 con 0x55c5e543d0a0 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 10 client.8184 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 10 client.8184 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 20 client.8184 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e54ecf50 0x55c5e54ef340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 4 client.8184 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c5e5297530 con 0x55c5e54ecf50 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e54ecf50 0x55c5e54ef340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 10 client.8184 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d0dc4640 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e54ecf50 0x55c5e54ef340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b36731b7aa61f14 server_cookie=5fb00b28be9a9d9a in_seq=0 out_seq=0 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 1 -- 172.21.15.82:0/4006630850 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x7ff9c8073050 con 0x55c5e54ecf50 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 10 client.8184 handle_command_reply: tid=0 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 client.8184 shutdown 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 msgr2=0x55c5e5499770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40b9740 tx=0x7ff9c4004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 shutdown_connections 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55c5e54ecf50 0x55c5e54ef340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c5e54872f0 0x55c5e549a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c5e54869d0 0x55c5e5499cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9abfff640 0 client.8184 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 --2- 172.21.15.82:0/4006630850 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c5e543d0a0 0x55c5e5499770 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 >> 172.21.15.82:0/4006630850 conn(0x55c5e5439020 msgr2=0x55c5e54850a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 shutdown_connections 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 1 -- 172.21.15.82:0/4006630850 wait complete. 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d6996000 20 client.8184 trim_cache size 0 max 16384 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:09:51.131 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 msgr2=0x7ff9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 0x7ff9b0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40af000 tx=0x7ff9c40a7040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 msgr2=0x7ff9cc162ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8010980 tx=0x7ff9c8010e40 comp rx=0 tx=0).stop 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 shutdown_connections 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9b0066f60 0x7ff9b0069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff9cc09d410 0x7ff9cc09d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff9cc162630 0x7ff9cc162ab0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 --2- 172.21.15.82:0/3287773507 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff9b8003a30 0x7ff9cc14b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 >> 172.21.15.82:0/3287773507 conn(0x7ff9cc099a70 msgr2=0x7ff9cc15d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 shutdown_connections 2024-04-25T10:09:51.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.123+0000 7ff9d3829640 1 -- 172.21.15.82:0/3287773507 wait complete. 2024-04-25T10:09:51.141 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:09:51.141 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:09:51.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.255+0000 7fd0d494d640 1 Processor -- start 2024-04-25T10:09:51.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0d494d640 1 -- start start 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0d494d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd0d015ecc0 0x7fd0d006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0d494d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00b2090 con 0x7fd0d015a700 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0d494d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00b43e0 con 0x7fd0d00b3f50 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0d494d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d00b46c0 con 0x7fd0d015ecc0 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ced9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd0d015ecc0 0x7fd0d006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ce59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd0d015a700 0x7fd0d00cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58070/0 (socket says 172.21.15.82:58070) 2024-04-25T10:09:51.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ce59c640 1 -- 172.21.15.82:0/4072207209 learned_addr learned my addr 172.21.15.82:0/4072207209 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:51.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0cdd9b640 1 -- 172.21.15.82:0/4072207209 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4283709873 0 0) 0x7fd0d00b2090 con 0x7fd0d015a700 2024-04-25T10:09:51.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0cdd9b640 1 -- 172.21.15.82:0/4072207209 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0b0003530 con 0x7fd0d015a700 2024-04-25T10:09:51.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ced9d640 1 -- 172.21.15.82:0/4072207209 >> v1:172.21.15.196:6789/0 conn(0x7fd0d00b3f50 legacy=0x7fd0d00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ced9d640 1 -- 172.21.15.82:0/4072207209 >> v1:172.21.15.82:6789/0 conn(0x7fd0d015a700 legacy=0x7fd0d00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ced9d640 1 -- 172.21.15.82:0/4072207209 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00cd6f0 con 0x7fd0d015ecc0 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.259+0000 7fd0ced9d640 1 --2- 172.21.15.82:0/4072207209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015ecc0 0x7fd0d006e1a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009d90 tx=0x7fd0c801c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc99501fb854fc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0cdd9b640 1 -- 172.21.15.82:0/4072207209 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80b2070 con 0x7fd0d015ecc0 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0cdd9b640 1 -- 172.21.15.82:0/4072207209 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c80a57f0 con 0x7fd0d015ecc0 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0cdd9b640 1 -- 172.21.15.82:0/4072207209 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80a5bc0 con 0x7fd0d015ecc0 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/4072207209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015ecc0 msgr2=0x7fd0d006e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 --2- 172.21.15.82:0/4072207209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015ecc0 0x7fd0d006e1a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009d90 tx=0x7fd0c801c820 comp rx=0 tx=0).stop 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/4072207209 shutdown_connections 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 --2- 172.21.15.82:0/4072207209 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015ecc0 0x7fd0d006e1a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/4072207209 >> 172.21.15.82:0/4072207209 conn(0x7fd0d00b11a0 msgr2=0x7fd0d00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/4072207209 shutdown_connections 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/4072207209 wait complete. 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 Processor -- start 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- start start 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0d015ecc0 0x7fd0d01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0d014eb90 0x7fd0d014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00cd310 con 0x7fd0d015a700 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00ce600 con 0x7fd0d015ecc0 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00cd490 con 0x7fd0d014eb90 2024-04-25T10:09:51.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58546/0 (socket says 172.21.15.82:58546) 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 -- 172.21.15.82:0/1862517256 learned_addr learned my addr 172.21.15.82:0/1862517256 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0d015ecc0 0x7fd0d01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ced9d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0d014eb90 0x7fd0d014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 -- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0d014eb90 msgr2=0x7fd0d014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0d014eb90 0x7fd0d014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 -- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0d015ecc0 msgr2=0x7fd0d01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0d015ecc0 0x7fd0d01585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 -- 172.21.15.82:0/1862517256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d00cba50 con 0x7fd0d015a700 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0ce59c640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c500 tx=0x7fd0c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0017070 con 0x7fd0d015a700 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c0005cf0 con 0x7fd0d015a700 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d015b0d0 con 0x7fd0d015a700 2024-04-25T10:09:51.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00072f0 con 0x7fd0d015a700 2024-04-25T10:09:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.263+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d015baf0 con 0x7fd0d015a700 2024-04-25T10:09:51.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d015bcf0 con 0x7fd0d015a700 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0c001a070 con 0x7fd0d015a700 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0c6ffd640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd0c0084cd0 con 0x7fd0d015a700 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.267+0000 7fd0c7fff640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 0x7fd0b4069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd0d01f9b30 tx=0x7fd0b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.271+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0c0021070 con 0x7fd0d015a700 2024-04-25T10:09:51.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.467+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0d006f260 con 0x7fd0d015a700 2024-04-25T10:09:51.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.467+0000 7fd0c6ffd640 1 -- 172.21.15.82:0/1862517256 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd0c0055cc0 con 0x7fd0d015a700 2024-04-25T10:09:51.470 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:51.470 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:23.329237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:09:51.470 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 63 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 msgr2=0x7fd0b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 0x7fd0b4069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd0d01f9b30 tx=0x7fd0b8009040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 msgr2=0x7fd0d00cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c500 tx=0x7fd0c000c9c0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 shutdown_connections 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0b4066f60 0x7fd0b4069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd0d014eb90 0x7fd0d014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd0d015ecc0 0x7fd0d01585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 --2- 172.21.15.82:0/1862517256 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd0d015a700 0x7fd0d00cb510 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 >> 172.21.15.82:0/1862517256 conn(0x7fd0d00b11a0 msgr2=0x7fd0d0071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 shutdown_connections 2024-04-25T10:09:51.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.471+0000 7fd0d494d640 1 -- 172.21.15.82:0/1862517256 wait complete. 2024-04-25T10:09:51.489 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:09:51.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 Processor -- start 2024-04-25T10:09:51.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 -- start start 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22bc15b6b0 0x7f22bc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22bc0cbd20 con 0x7f22bc0cf590 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22bc15ba90 con 0x7f22bc15b6b0 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c293c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0cc000 con 0x7f22bc073010 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22bbfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22bc15b6b0 0x7f22bc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:41432/0 (socket says 172.21.15.82:41432) 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22c0ed9640 1 -- 172.21.15.82:0/3485930276 learned_addr learned my addr 172.21.15.82:0/3485930276 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.607+0000 7f22bb7fe640 1 --1- 172.21.15.82:0/3485930276 >> v1:172.21.15.196:6789/0 conn(0x7f22bc0cf590 0x7f22bc15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:46834/0 (socket says 172.21.15.82:46834) 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c0ed9640 1 -- 172.21.15.82:0/3485930276 >> v2:172.21.15.196:3300/0 conn(0x7f22bc15b6b0 msgr2=0x7f22bc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/3485930276 >> v2:172.21.15.196:3300/0 conn(0x7f22bc15b6b0 0x7f22bc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c0ed9640 1 -- 172.21.15.82:0/3485930276 >> v1:172.21.15.196:6789/0 conn(0x7f22bc0cf590 legacy=0x7f22bc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c0ed9640 1 -- 172.21.15.82:0/3485930276 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0cc2e0 con 0x7f22bc073010 2024-04-25T10:09:51.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/3485930276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009e50 tx=0x7f22b401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a2df33654bef6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22baffd640 1 -- 172.21.15.82:0/3485930276 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a5700 con 0x7f22bc073010 2024-04-25T10:09:51.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22baffd640 1 -- 172.21.15.82:0/3485930276 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40a58a0 con 0x7f22bc073010 2024-04-25T10:09:51.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22baffd640 1 -- 172.21.15.82:0/3485930276 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a5c70 con 0x7f22bc073010 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 -- 172.21.15.82:0/3485930276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 msgr2=0x7f22bc0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 --2- 172.21.15.82:0/3485930276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009e50 tx=0x7f22b401ca30 comp rx=0 tx=0).stop 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 -- 172.21.15.82:0/3485930276 shutdown_connections 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 --2- 172.21.15.82:0/3485930276 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc073010 0x7f22bc0cfa20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 --2- 172.21.15.82:0/3485930276 >> v2:172.21.15.196:3300/0 conn(0x7f22bc15b6b0 0x7f22bc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 -- 172.21.15.82:0/3485930276 >> 172.21.15.82:0/3485930276 conn(0x7f22bc06ae60 msgr2=0x7f22bc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 -- 172.21.15.82:0/3485930276 shutdown_connections 2024-04-25T10:09:51.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.611+0000 7f22c293c640 1 -- 172.21.15.82:0/3485930276 wait complete. 2024-04-25T10:09:51.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 Processor -- start 2024-04-25T10:09:51.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- start start 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22bc0cf590 0x7f22bc164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 0x7f22bc164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0cadb0 con 0x7f22bc073010 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0ca900 con 0x7f22bc0cf590 2024-04-25T10:09:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0caf30 con 0x7f22bc15b6b0 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58560/0 (socket says 172.21.15.82:58560) 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 -- 172.21.15.82:0/3941041515 learned_addr learned my addr 172.21.15.82:0/3941041515 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 0x7f22bc164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22bc0cf590 0x7f22bc164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 -- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 msgr2=0x7f22bc164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 0x7f22bc164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 -- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22bc0cf590 msgr2=0x7f22bc164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22bc0cf590 0x7f22bc164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 -- 172.21.15.82:0/3941041515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0b3540 con 0x7f22bc073010 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 0x7f22bc164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e870 tx=0x7f22ac00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22b8ff9640 1 -- 172.21.15.82:0/3941041515 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac017070 con 0x7f22bc073010 2024-04-25T10:09:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22b8ff9640 1 -- 172.21.15.82:0/3941041515 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac00c940 con 0x7f22bc073010 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22bc152f40 con 0x7f22bc073010 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22b8ff9640 1 -- 172.21.15.82:0/3941041515 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00cd10 con 0x7f22bc073010 2024-04-25T10:09:51.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.615+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22bc098e60 con 0x7f22bc073010 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22b8ff9640 1 -- 172.21.15.82:0/3941041515 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22ac01a070 con 0x7f22bc073010 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22b8ff9640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 0x7f22a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22b8ff9640 1 -- 172.21.15.82:0/3941041515 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f22ac0855a0 con 0x7f22bc073010 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 0x7f22a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 0x7f22a8069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0c5eb0 tx=0x7f22b0007cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 Processor -- start 2024-04-25T10:09:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 -- start start 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb5a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb442ba0 con 0x55a9fb55b8e0 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb3a8090 con 0x55a9fb5a5c00 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb330160 con 0x55a9fb5a52e0 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58566/0 (socket says 172.21.15.82:58566) 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 -- 172.21.15.82:0/2835377345 learned_addr learned my addr 172.21.15.82:0/2835377345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22bbfff640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb5a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 -- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 msgr2=0x55a9fb5a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb5a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 -- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a5c00 msgr2=0x55a9fb55b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 -- 172.21.15.82:0/2835377345 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9fb451c20 con 0x55a9fb55b8e0 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22bbfff640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb5a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f22b401b0e0 tx=0x7f22b4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1b26b662a9fc68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22a1ffb640 1 -- 172.21.15.82:0/2835377345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a5700 con 0x55a9fb55b8e0 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22a1ffb640 1 -- 172.21.15.82:0/2835377345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40a58a0 con 0x55a9fb55b8e0 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22a1ffb640 1 -- 172.21.15.82:0/2835377345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40a5c50 con 0x55a9fb55b8e0 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 -- 172.21.15.82:0/2835377345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 msgr2=0x55a9fb55bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.619+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f22b401b0e0 tx=0x7f22b4004ac0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/2835377345 shutdown_connections 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55bcc0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/2835377345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb5a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/2835377345 >> 172.21.15.82:0/2835377345 conn(0x55a9fb557890 msgr2=0x55a9fb559c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/2835377345 shutdown_connections 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/2835377345 wait complete. 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- start start 2024-04-25T10:09:51.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 Processor -- start 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb5ac640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb593090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb3a4130 con 0x55a9fb5a52e0 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb36c130 con 0x55a9fb55b8e0 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb466300 con 0x55a9fb5a5c00 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 10 client.0 _ll_register_callbacks cb 0x55a9fb546bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58582/0 (socket says 172.21.15.82:58582) 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 -- 172.21.15.82:0/3359395 learned_addr learned my addr 172.21.15.82:0/3359395 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb5ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 -- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 msgr2=0x55a9fb593090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb593090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb593090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 -- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 msgr2=0x55a9fb5ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb5ac640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 -- 172.21.15.82:0/3359395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9fb385060 con 0x55a9fb5a52e0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f22b0001d30 tx=0x7f22b0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25464755bcd208bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b0027060 con 0x55a9fb5a52e0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f22b0010040 con 0x55a9fb5a52e0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b40a58a0 con 0x55a9fb5a52e0 2024-04-25T10:09:51.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22b00076f0 con 0x55a9fb5a52e0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 10 client.8202 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55a9fb3497c0 con 0x55a9fb5a52e0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40a58a0 con 0x55a9fb5a52e0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22b0004c80 con 0x55a9fb5a52e0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 10 client.8202 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 10 client.8202 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 20 client.8202 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb593e30 0x55a9fb594210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 4 client.8202 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9fb343cd0 con 0x55a9fb593e30 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb593e30 0x55a9fb594210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 10 client.8202 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f22bbfff640 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb593e30 0x55a9fb594210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b609d7e1aab730 server_cookie=7c4b2b9e6512163 in_seq=0 out_seq=0 2024-04-25T10:09:51.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 1 -- 172.21.15.82:0/3359395 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9fb343cd0 con 0x55a9fb593e30 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.623+0000 7f229affd640 10 client.8202 handle_command_reply: tid=0 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 client.8202 shutdown 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 msgr2=0x55a9fb5acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f22b0001d30 tx=0x7f22b0000bd0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 shutdown_connections 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb593e30 0x55a9fb594210 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb593090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a52e0 0x55a9fb5acb80 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f229affd640 0 client.8202 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/3359395 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb5ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 >> 172.21.15.82:0/3359395 conn(0x55a9fb557890 msgr2=0x55a9fb55a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 shutdown_connections 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 1 -- 172.21.15.82:0/3359395 wait complete. 2024-04-25T10:09:51.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.627+0000 7f22c5aa9000 20 client.8202 trim_cache size 0 max 16384 2024-04-25T10:09:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 Processor -- start 2024-04-25T10:09:51.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- start start 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb559c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc202520 con 0x55a9fb5a5c00 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc0ce190 con 0x55a9fb55b8e0 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22bc201f20 con 0x55a9fb5a52e0 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58598/0 (socket says 172.21.15.82:58598) 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 -- 172.21.15.82:0/4264365800 learned_addr learned my addr 172.21.15.82:0/4264365800 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 -- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 msgr2=0x55a9fb559c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb559c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 -- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 msgr2=0x55a9fb55aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 -- 172.21.15.82:0/4264365800 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9fb363fc0 con 0x55a9fb5a5c00 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c0ed9640 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f22b401b0e0 tx=0x7f22b40a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5755656e71983d9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22a17fa640 1 -- 172.21.15.82:0/4264365800 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40ba070 con 0x55a9fb5a5c00 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22a17fa640 1 -- 172.21.15.82:0/4264365800 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40c4020 con 0x55a9fb5a5c00 2024-04-25T10:09:51.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22a17fa640 1 -- 172.21.15.82:0/4264365800 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4003bc0 con 0x55a9fb5a5c00 2024-04-25T10:09:51.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- 172.21.15.82:0/4264365800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 msgr2=0x55a9fb55a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f22b401b0e0 tx=0x7f22b40a7000 comp rx=0 tx=0).stop 2024-04-25T10:09:51.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- 172.21.15.82:0/4264365800 shutdown_connections 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a52e0 0x55a9fb559c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb5a5c00 0x55a9fb55a1c0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/4264365800 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb55b8e0 0x55a9fb55aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- 172.21.15.82:0/4264365800 >> 172.21.15.82:0/4264365800 conn(0x55a9fb557890 msgr2=0x55a9fb5a4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- 172.21.15.82:0/4264365800 shutdown_connections 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- 172.21.15.82:0/4264365800 wait complete. 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 -- start start 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.639+0000 7f22c5aa9000 1 Processor -- start 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a52e0 0x55a9fb60e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb5a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb342e70 con 0x55a9fb55b8e0 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb330a50 con 0x55a9fb5a52e0 2024-04-25T10:09:51.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9fb3a73a0 con 0x55a9fb5a5c00 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 10 client.0 _ll_register_callbacks cb 0x55a9fb354240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58604/0 (socket says 172.21.15.82:58604) 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 -- 172.21.15.82:0/379665392 learned_addr learned my addr 172.21.15.82:0/379665392 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 -- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 msgr2=0x55a9fb5a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb5a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 -- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a52e0 msgr2=0x55a9fb60e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a52e0 0x55a9fb60e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 -- 172.21.15.82:0/379665392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9fb4019e0 con 0x55a9fb55b8e0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bbfff640 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0558c0 tx=0x7f22ac055c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2578b521999529eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac021040 con 0x55a9fb55b8e0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f22ac052230 con 0x55a9fb55b8e0 2024-04-25T10:09:51.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22b40c4020 con 0x55a9fb55b8e0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22ac05de40 con 0x55a9fb55b8e0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 10 client.8208 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55a9fb347630 con 0x55a9fb55b8e0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b40c4020 con 0x55a9fb55b8e0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f22ac069b80 con 0x55a9fb55b8e0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 10 client.8208 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 10 client.8208 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 20 client.8208 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb614d90 0x55a9fb617180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 4 client.8208 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a9fb3b55d0 con 0x55a9fb614d90 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb614d90 0x55a9fb617180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22bb7fe640 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb614d90 0x55a9fb617180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b8048a01840845a server_cookie=f700f9cfe62175a0 in_seq=0 out_seq=0 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 10 client.8208 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 1 -- 172.21.15.82:0/379665392 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+420 (crc 0 0 0) 0x55a9fb3b55d0 con 0x55a9fb614d90 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 10 client.8208 handle_command_reply: tid=0 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 client.8208 shutdown 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 msgr2=0x55a9fb60df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0558c0 tx=0x7f22ac055c80 comp rx=0 tx=0).stop 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 shutdown_connections 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55a9fb614d90 0x55a9fb617180 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a9fb5a5c00 0x55a9fb5a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a9fb5a52e0 0x55a9fb60e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 --2- 172.21.15.82:0/379665392 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a9fb55b8e0 0x55a9fb60df30 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f229affd640 0 client.8208 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:51.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 >> 172.21.15.82:0/379665392 conn(0x55a9fb557890 msgr2=0x55a9fb5a4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 shutdown_connections 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 1 -- 172.21.15.82:0/379665392 wait complete. 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c5aa9000 20 client.8208 trim_cache size 0 max 16384 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 1194646827, 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:09:51.651 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/testdir/hardlink" 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "dir_frag", 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 1736843379, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627776, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "frag": "*", 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir" 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 2714698994, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627776, 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir" 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:51.652 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:09:51.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 msgr2=0x7f22a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 0x7f22a8069420 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f22bc0c5eb0 tx=0x7f22b0007cd0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 msgr2=0x7f22bc098920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e870 tx=0x7f22ac00ed30 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 shutdown_connections 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f22a8066f60 0x7f22a8069420 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22bc15b6b0 0x7f22bc164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22bc0cf590 0x7f22bc164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 --2- 172.21.15.82:0/3941041515 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22bc073010 0x7f22bc098920 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 >> 172.21.15.82:0/3941041515 conn(0x7f22bc06ae60 msgr2=0x7f22bc0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 shutdown_connections 2024-04-25T10:09:51.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.643+0000 7f22c293c640 1 -- 172.21.15.82:0/3941041515 wait complete. 2024-04-25T10:09:51.662 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:09:51.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.771+0000 7fc7559c3640 1 Processor -- start 2024-04-25T10:09:51.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.771+0000 7fc7559c3640 1 -- start start 2024-04-25T10:09:51.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7559c3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc750161160 0x7fc7500d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7559c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7559c3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7500d1d40 con 0x7fc7500b3470 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7559c3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7500d2020 con 0x7fc750161160 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7559c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500d2300 con 0x7fc7500b2780 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7467fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74effd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc750161160 0x7fc7500d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7467fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:41484/0 (socket says 172.21.15.82:41484) 2024-04-25T10:09:51.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7467fc640 1 -- 172.21.15.82:0/3920341087 learned_addr learned my addr 172.21.15.82:0/3920341087 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74effd640 1 -- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 msgr2=0x7fc7500d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74effd640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74effd640 1 -- 172.21.15.82:0/3920341087 >> v1:172.21.15.196:6789/0 conn(0x7fc7500b3470 legacy=0x7fc7500b3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74e7fc640 1 -- 172.21.15.82:0/3920341087 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280807251 0 0) 0x7fc7500d1d40 con 0x7fc7500b3470 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc74effd640 1 -- 172.21.15.82:0/3920341087 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500d25e0 con 0x7fc750161160 2024-04-25T10:09:51.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.775+0000 7fc7467fc640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74effd640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc750161160 0x7fc7500d00c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc7400097c0 tx=0x7fc74001c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4c6e8e9eab28ccc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74e7fc640 1 -- 172.21.15.82:0/3920341087 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400ab430 con 0x7fc750161160 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74e7fc640 1 -- 172.21.15.82:0/3920341087 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7400b0070 con 0x7fc750161160 2024-04-25T10:09:51.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74e7fc640 1 -- 172.21.15.82:0/3920341087 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7400ab7f0 con 0x7fc750161160 2024-04-25T10:09:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc750161160 msgr2=0x7fc7500d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc750161160 0x7fc7500d00c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc7400097c0 tx=0x7fc74001c3d0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/3920341087 shutdown_connections 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc7500b2780 0x7fc7500d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- 172.21.15.82:0/3920341087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc750161160 0x7fc7500d00c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/3920341087 >> 172.21.15.82:0/3920341087 conn(0x7fc75008b000 msgr2=0x7fc7500d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/3920341087 shutdown_connections 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/3920341087 wait complete. 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 Processor -- start 2024-04-25T10:09:51.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- start start 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 0x7fc750154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 0x7fc7500ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500cf710 con 0x7fc7500b3470 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500cf590 con 0x7fc7500b2780 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500cf890 con 0x7fc750161160 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58618/0 (socket says 172.21.15.82:58618) 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 -- 172.21.15.82:0/1911788354 learned_addr learned my addr 172.21.15.82:0/1911788354 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74effd640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 0x7fc750154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74f7fe640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 0x7fc7500ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 -- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 msgr2=0x7fc7500ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 0x7fc7500ccae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 -- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 msgr2=0x7fc750154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 0x7fc750154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 -- 172.21.15.82:0/1911788354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc750155780 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74f7fe640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 0x7fc7500ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74effd640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 0x7fc750154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7467fc640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc73800e9a0 tx=0x7fc73800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738017070 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73800ca00 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc750155a60 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73800cdb0 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.779+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc750156020 con 0x7fc7500b3470 2024-04-25T10:09:51.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc75004efa0 con 0x7fc7500b3470 2024-04-25T10:09:51.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc73801a070 con 0x7fc7500b3470 2024-04-25T10:09:51.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc74cff9640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 0x7fc730069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:51.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc738085520 con 0x7fc7500b3470 2024-04-25T10:09:51.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc74effd640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 0x7fc730069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:51.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.783+0000 7fc74effd640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 0x7fc730069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7400af000 tx=0x7fc740008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:51.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.787+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7380562e0 con 0x7fc7500b3470 2024-04-25T10:09:51.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc7500d30c0 con 0x7fc7500b3470 2024-04-25T10:09:51.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc74cff9640 1 -- 172.21.15.82:0/1911788354 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fc738055be0 con 0x7fc7500b3470 2024-04-25T10:09:51.976 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:09:51.976 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T10:09:51.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 msgr2=0x7fc730069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 0x7fc730069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fc7400af000 tx=0x7fc740008040 comp rx=0 tx=0).stop 2024-04-25T10:09:51.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 msgr2=0x7fc7500c91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:51.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc73800e9a0 tx=0x7fc73800ee60 comp rx=0 tx=0).stop 2024-04-25T10:09:51.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 shutdown_connections 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc730066f90 0x7fc730069450 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc750161160 0x7fc7500ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7500b3470 0x7fc7500c91d0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 --2- 172.21.15.82:0/1911788354 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc7500b2780 0x7fc750154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 >> 172.21.15.82:0/1911788354 conn(0x7fc75008b000 msgr2=0x7fc75006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 shutdown_connections 2024-04-25T10:09:51.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:51.975+0000 7fc7559c3640 1 -- 172.21.15.82:0/1911788354 wait complete. 2024-04-25T10:09:51.993 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:09:51.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:51.993 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:09:52.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 Processor -- start 2024-04-25T10:09:52.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 -- start start 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7feed006f380 0x7feed006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed006ecb0 con 0x7feed015dc70 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed006ef50 con 0x7feed006e7f0 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feed60e1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed00cec00 con 0x7feed006f380 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7feed015dc70 0x7feed006e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58080/0 (socket says 172.21.15.82:58080) 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 -- 172.21.15.82:0/884942322 learned_addr learned my addr 172.21.15.82:0/884942322 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecffff640 1 --2- 172.21.15.82:0/884942322 >> v2:172.21.15.196:3300/0 conn(0x7feed006f380 0x7feed006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411907492 0 0) 0x7feed006ecb0 con 0x7feed015dc70 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb8003530 con 0x7feed015dc70 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3156339308 0 0) 0x7feeb8003530 con 0x7feed015dc70 2024-04-25T10:09:52.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 >> v2:172.21.15.196:3300/0 conn(0x7feed006f380 msgr2=0x7feed006fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 --2- 172.21.15.82:0/884942322 >> v2:172.21.15.196:3300/0 conn(0x7feed006f380 0x7feed006fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 >> v1:172.21.15.196:6789/0 conn(0x7feed006e7f0 legacy=0x7feed00aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00b2de0 con 0x7feed015dc70 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7feec80016a0 con 0x7feed015dc70 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 >> v1:172.21.15.82:6789/0 conn(0x7feed015dc70 legacy=0x7feed006e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 0x7feeb8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00cec00 con 0x7feeb8003a10 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb800c0d0 con 0x7feeb80051d0 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb800c2d0 con 0x7feeb8004810 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 0x7feeb8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.124 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feeceffd640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 -- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8004810 msgr2=0x7feeb8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 -- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb80051d0 msgr2=0x7feeb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecffff640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feeb80051d0 0x7feeb800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecffff640 1 -- 172.21.15.82:0/884942322 reap_dead start 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feecf7fe640 1 -- 172.21.15.82:0/884942322 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb8003530 con 0x7feeb8003a10 2024-04-25T10:09:52.125 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.119+0000 7feeceffd640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8004810 0x7feeb8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feecf7fe640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 0x7feeb8003e30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7feed006ee80 tx=0x7feec801cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f324d4a4a1797097 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8007c90 con 0x7feeb8003a10 2024-04-25T10:09:52.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec80a9070 con 0x7feeb8003a10 2024-04-25T10:09:52.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feece7fc640 1 -- 172.21.15.82:0/884942322 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8005d20 con 0x7feeb8003a10 2024-04-25T10:09:52.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 -- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 msgr2=0x7feeb8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 0x7feeb8003e30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7feed006ee80 tx=0x7feec801cae0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 -- 172.21.15.82:0/884942322 shutdown_connections 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 --2- 172.21.15.82:0/884942322 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feeb8003a10 0x7feeb8003e30 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 -- 172.21.15.82:0/884942322 >> 172.21.15.82:0/884942322 conn(0x7feed00b1f80 msgr2=0x7feed00b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 -- 172.21.15.82:0/884942322 shutdown_connections 2024-04-25T10:09:52.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.123+0000 7feed60e1640 1 -- 172.21.15.82:0/884942322 wait complete. 2024-04-25T10:09:52.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 Processor -- start 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- start start 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8003a10 0x7feed0154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed015cc70 0x7feed015d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00cf1f0 con 0x7feed0155160 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed00cf070 con 0x7feed015cc70 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed00ada70 con 0x7feeb8003a10 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58630/0 (socket says 172.21.15.82:58630) 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 -- 172.21.15.82:0/3399950915 learned_addr learned my addr 172.21.15.82:0/3399950915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feeceffd640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed015cc70 0x7feed015d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8003a10 0x7feed0154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.130 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 -- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8003a10 msgr2=0x7feed0154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8003a10 0x7feed0154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 -- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed015cc70 msgr2=0x7feed015d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed015cc70 0x7feed015d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 -- 172.21.15.82:0/3399950915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00c6a80 con 0x7feed0155160 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feecffff640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7feed00cd970 tx=0x7feec000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feeb7fff640 1 -- 172.21.15.82:0/3399950915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0016420 con 0x7feed0155160 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feeb7fff640 1 -- 172.21.15.82:0/3399950915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec001b070 con 0x7feed0155160 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feeb7fff640 1 -- 172.21.15.82:0/3399950915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0016820 con 0x7feed0155160 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed00c67a0 con 0x7feed0155160 2024-04-25T10:09:52.131 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed015c470 con 0x7feed0155160 2024-04-25T10:09:52.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.127+0000 7feeb7fff640 1 -- 172.21.15.82:0/3399950915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feec00169c0 con 0x7feed0155160 2024-04-25T10:09:52.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeb7fff640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 0x7feebc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 0x7feebc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeb7fff640 1 -- 172.21.15.82:0/3399950915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7feec0086170 con 0x7feed0155160 2024-04-25T10:09:52.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 0x7feebc069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feec80ab000 tx=0x7feec80b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 Processor -- start 2024-04-25T10:09:52.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- start start 2024-04-25T10:09:52.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffb0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffb05b0 0x56250ff65d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fe4cf40 con 0x56250ff66290 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fdb4070 con 0x56250ffb05b0 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56250fd3c1a0 con 0x56250ffafc90 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58640/0 (socket says 172.21.15.82:58640) 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 -- 172.21.15.82:0/1866743934 learned_addr learned my addr 172.21.15.82:0/1866743934 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeceffd640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffb05b0 0x56250ff65d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecf7fe640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffb0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 -- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 msgr2=0x56250ffb0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffb0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 -- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffb05b0 msgr2=0x56250ff65d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffb05b0 0x56250ff65d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 -- 172.21.15.82:0/1866743934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56250fd3a510 con 0x56250ff66290 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecf7fe640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffb0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feecffff640 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feec0084460 tx=0x7feec00567a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87886745fa7352d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeb57fa640 1 -- 172.21.15.82:0/1866743934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0082070 con 0x56250ff66290 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeb57fa640 1 -- 172.21.15.82:0/1866743934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec005a9f0 con 0x56250ff66290 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feeb57fa640 1 -- 172.21.15.82:0/1866743934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec005efb0 con 0x56250ff66290 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- 172.21.15.82:0/1866743934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 msgr2=0x56250ff66670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7feec0084460 tx=0x7feec00567a0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- 172.21.15.82:0/1866743934 shutdown_connections 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff66670 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffb05b0 0x56250ff65d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 --2- 172.21.15.82:0/1866743934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffb0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- 172.21.15.82:0/1866743934 >> 172.21.15.82:0/1866743934 conn(0x56250ff62240 msgr2=0x56250ff64630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- 172.21.15.82:0/1866743934 shutdown_connections 2024-04-25T10:09:52.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.131+0000 7feed924e000 1 -- 172.21.15.82:0/1866743934 wait complete. 2024-04-25T10:09:52.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- start start 2024-04-25T10:09:52.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 Processor -- start 2024-04-25T10:09:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x56250ff71370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ff718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fd6e6c0 con 0x56250ff66290 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fdb0110 con 0x56250ffafc90 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56250ff6f200 con 0x56250ffb05b0 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 10 client.0 _ll_register_callbacks cb 0x56250ff51570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58652/0 (socket says 172.21.15.82:58652) 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 -- 172.21.15.82:0/2420685082 learned_addr learned my addr 172.21.15.82:0/2420685082 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 -- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 msgr2=0x56250ff718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ff718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 -- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 msgr2=0x56250ff71370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x56250ff71370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 -- 172.21.15.82:0/2420685082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56250fd3c3b0 con 0x56250ff66290 2024-04-25T10:09:52.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feecf7fe640 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7feec80983c0 tx=0x7feec8019880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db17fc9f1b3815d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8004c80 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7feec8004e20 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec005a9f0 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec8007520 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 10 client.8223 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56250fe35e00 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec005a9f0 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feec80a56a0 con 0x56250ff66290 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 10 client.8223 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 10 client.8223 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 20 client.8223 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ff72150 0x56250ffc1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 4 client.8223 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56250fd4fd60 con 0x56250ff72150 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeceffd640 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ff72150 0x56250ffc1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 10 client.8223 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeceffd640 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ff72150 0x56250ffc1410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f1c69f1e86ecdb3 server_cookie=69f0e448e4750d04 in_seq=0 out_seq=0 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 1 -- 172.21.15.82:0/2420685082 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56250fd4fd60 con 0x56250ff72150 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 10 client.8223 handle_command_reply: tid=0 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 client.8223 shutdown 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 msgr2=0x56250ff70e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7feec80983c0 tx=0x7feec8019880 comp rx=0 tx=0).stop 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 shutdown_connections 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ff72150 0x56250ffc1410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ff718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x56250ff71370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feeae7fc640 0 client.8223 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 --2- 172.21.15.82:0/2420685082 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x56250ff70e30 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 >> 172.21.15.82:0/2420685082 conn(0x56250ff62240 msgr2=0x56250ff64ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 shutdown_connections 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 1 -- 172.21.15.82:0/2420685082 wait complete. 2024-04-25T10:09:52.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.135+0000 7feed924e000 20 client.8223 trim_cache size 0 max 16384 2024-04-25T10:09:52.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 Processor -- start 2024-04-25T10:09:52.154 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 -- start start 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 0x56250ffc3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffc3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0204260 con 0x56250ffb05b0 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0203420 con 0x56250ff66290 2024-04-25T10:09:52.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed0202910 con 0x56250ffafc90 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58664/0 (socket says 172.21.15.82:58664) 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 -- 172.21.15.82:0/3878591240 learned_addr learned my addr 172.21.15.82:0/3878591240 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 0x56250ffc3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 -- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 msgr2=0x56250ffc3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffc3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 -- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 msgr2=0x56250ffc3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 0x56250ffc3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 -- 172.21.15.82:0/3878591240 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56250fe0bc50 con 0x56250ffb05b0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 0x56250ffc3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.151+0000 7feecffff640 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7feec0084510 tx=0x7feec006b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a620e260d974ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeb4ff9640 1 -- 172.21.15.82:0/3878591240 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0082030 con 0x56250ffb05b0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeb4ff9640 1 -- 172.21.15.82:0/3878591240 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0073070 con 0x56250ffb05b0 2024-04-25T10:09:52.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeb4ff9640 1 -- 172.21.15.82:0/3878591240 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec0056670 con 0x56250ffb05b0 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3878591240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 msgr2=0x562510019ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7feec0084510 tx=0x7feec006b000 comp rx=0 tx=0).stop 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3878591240 shutdown_connections 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffafc90 0x56250ffc3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ffb05b0 0x562510019ae0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- 172.21.15.82:0/3878591240 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ff66290 0x56250ffc3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3878591240 >> 172.21.15.82:0/3878591240 conn(0x56250ff62240 msgr2=0x56250ff6ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3878591240 shutdown_connections 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3878591240 wait complete. 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- start start 2024-04-25T10:09:52.158 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 Processor -- start 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x562510018f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ffd46f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56250fdb3380 con 0x56250ff66290 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed0202e70 con 0x56250ffafc90 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56250fdb0110 con 0x56250ffb05b0 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 10 client.0 _ll_register_callbacks cb 0x7feec0052b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58668/0 (socket says 172.21.15.82:58668) 2024-04-25T10:09:52.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 -- 172.21.15.82:0/3138684824 learned_addr learned my addr 172.21.15.82:0/3138684824 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 -- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 msgr2=0x56250ffd46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeceffd640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x562510018f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ffd46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 -- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 msgr2=0x562510018f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x562510018f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 -- 172.21.15.82:0/3138684824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56250fd7c120 con 0x56250ff66290 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feecf7fe640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c10 tx=0x7feec8001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b5705c36f558a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80a5c90 con 0x56250ff66290 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7feec80d0d50 con 0x56250ff66290 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec0073070 con 0x56250ff66290 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec8004c80 con 0x56250ff66290 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 10 client.8229 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:52.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x56250fd5b170 con 0x56250ff66290 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec0073070 con 0x56250ff66290 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feec80ae460 con 0x56250ff66290 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 10 client.8229 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 10 client.8229 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 20 client.8229 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ffd5110 0x56250ffd5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 4 client.8229 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x56250fdc15b0 con 0x56250ffd5110 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeceffd640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ffd5110 0x56250ffd5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeae7fc640 10 client.8229 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.155+0000 7feeceffd640 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ffd5110 0x56250ffd5520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7894be4cf8dc80 server_cookie=f8996ea6dffd4269 in_seq=0 out_seq=0 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feeae7fc640 1 -- 172.21.15.82:0/3138684824 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56250fdc15b0 con 0x56250ffd5110 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feeae7fc640 10 client.8229 handle_command_reply: tid=0 2024-04-25T10:09:52.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 client.8229 shutdown 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 msgr2=0x562510018a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7feec8000c10 tx=0x7feec8001e10 comp rx=0 tx=0).stop 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 shutdown_connections 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x56250ffd5110 0x56250ffd5520 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56250ffb05b0 0x56250ffd46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feeae7fc640 0 client.8229 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56250ffafc90 0x562510018f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 --2- 172.21.15.82:0/3138684824 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56250ff66290 0x562510018a40 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 >> 172.21.15.82:0/3138684824 conn(0x56250ff62240 msgr2=0x56250ff64630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 shutdown_connections 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 1 -- 172.21.15.82:0/3138684824 wait complete. 2024-04-25T10:09:52.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed924e000 20 client.8229 trim_cache size 0 max 16384 2024-04-25T10:09:52.166 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:52.166 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:09:52.166 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "9b818353-73ef-44f8-bba2-ee61ce8d2148", 2024-04-25T10:09:52.166 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:09:52.166 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:52.166 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:09:52.159+0000 7f33f1d9e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)->f(v1 m2024-04-25T10:09:01.778099+0000) n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)->n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56527a957b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 msgr2=0x7feebc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 0x7feebc069450 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7feec80ab000 tx=0x7feec80b0040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 msgr2=0x7feed01555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7feed00cd970 tx=0x7feec000f520 comp rx=0 tx=0).stop 2024-04-25T10:09:52.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 shutdown_connections 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7feebc066f90 0x7feebc069450 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7feed0155160 0x7feed01555e0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7feed015cc70 0x7feed015d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 --2- 172.21.15.82:0/3399950915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7feeb8003a10 0x7feed0154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 >> 172.21.15.82:0/3399950915 conn(0x7feed00b1f80 msgr2=0x7feed00b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 shutdown_connections 2024-04-25T10:09:52.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.159+0000 7feed60e1640 1 -- 172.21.15.82:0/3399950915 wait complete. 2024-04-25T10:09:52.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:52.177 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:09:52.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.283+0000 7f37ccfee640 1 Processor -- start 2024-04-25T10:09:52.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.283+0000 7f37ccfee640 1 -- start start 2024-04-25T10:09:52.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.283+0000 7f37ccfee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f37c80cd220 0x7f37c80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c80b2740 0x7f37c80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37c80b1240 con 0x7f37c80b2bb0 2024-04-25T10:09:52.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c815f360 con 0x7f37c80b2740 2024-04-25T10:09:52.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c815f640 con 0x7f37c80cd220 2024-04-25T10:09:52.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c80b2740 0x7f37c80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37c80b2740 0x7f37c80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58676/0 (socket says 172.21.15.82:58676) 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 -- 172.21.15.82:0/606649362 learned_addr learned my addr 172.21.15.82:0/606649362 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 -- 172.21.15.82:0/606649362 >> v2:172.21.15.196:3300/0 conn(0x7f37c80cd220 msgr2=0x7f37c80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 --2- 172.21.15.82:0/606649362 >> v2:172.21.15.196:3300/0 conn(0x7f37c80cd220 0x7f37c80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 -- 172.21.15.82:0/606649362 >> v1:172.21.15.196:6789/0 conn(0x7f37c80b2bb0 legacy=0x7f37c8099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 -- 172.21.15.82:0/606649362 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c815f920 con 0x7f37c80b2740 2024-04-25T10:09:52.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37bffff640 1 --2- 172.21.15.82:0/606649362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80b2740 0x7f37c80cf900 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009870 tx=0x7f37c809abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebbf492a87bfa9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37c5d9b640 1 -- 172.21.15.82:0/606649362 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00b0070 con 0x7f37c80b2740 2024-04-25T10:09:52.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37c5d9b640 1 -- 172.21.15.82:0/606649362 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b00ab600 con 0x7f37c80b2740 2024-04-25T10:09:52.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37c5d9b640 1 -- 172.21.15.82:0/606649362 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00ab9d0 con 0x7f37c80b2740 2024-04-25T10:09:52.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- 172.21.15.82:0/606649362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80b2740 msgr2=0x7f37c80cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 --2- 172.21.15.82:0/606649362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80b2740 0x7f37c80cf900 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009870 tx=0x7f37c809abd0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- 172.21.15.82:0/606649362 shutdown_connections 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 --2- 172.21.15.82:0/606649362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80b2740 0x7f37c80cf900 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 --2- 172.21.15.82:0/606649362 >> v2:172.21.15.196:3300/0 conn(0x7f37c80cd220 0x7f37c80cc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- 172.21.15.82:0/606649362 >> 172.21.15.82:0/606649362 conn(0x7f37c80d15b0 msgr2=0x7f37c80d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- 172.21.15.82:0/606649362 shutdown_connections 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.287+0000 7f37ccfee640 1 -- 172.21.15.82:0/606649362 wait complete. 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 Processor -- start 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- start start 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37c80b2740 0x7f37c80c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c80b2bb0 0x7f37c815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8168880 con 0x7f37c80cd220 2024-04-25T10:09:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8168700 con 0x7f37c80b2bb0 2024-04-25T10:09:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8168a00 con 0x7f37c80b2740 2024-04-25T10:09:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58678/0 (socket says 172.21.15.82:58678) 2024-04-25T10:09:52.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 -- 172.21.15.82:0/736420721 learned_addr learned my addr 172.21.15.82:0/736420721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c659c640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37c80b2740 0x7f37c80c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 -- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37c80b2740 msgr2=0x7f37c80c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37bffff640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c80b2bb0 0x7f37c815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37c80b2740 0x7f37c80c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 -- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c80b2bb0 msgr2=0x7f37c815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c80b2bb0 0x7f37c815ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 -- 172.21.15.82:0/736420721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c815a9e0 con 0x7f37c80cd220 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37c6d9d640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c780 tx=0x7f37c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37beffd640 1 -- 172.21.15.82:0/736420721 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0017070 con 0x7f37c80cd220 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37beffd640 1 -- 172.21.15.82:0/736420721 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c0003db0 con 0x7f37c80cd220 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8159500 con 0x7f37c80cd220 2024-04-25T10:09:52.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37beffd640 1 -- 172.21.15.82:0/736420721 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00052b0 con 0x7f37c80cd220 2024-04-25T10:09:52.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c815af60 con 0x7f37c80cd220 2024-04-25T10:09:52.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.291+0000 7f37beffd640 1 -- 172.21.15.82:0/736420721 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37c001a070 con 0x7f37c80cd220 2024-04-25T10:09:52.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37beffd640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 0x7f37a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37beffd640 1 -- 172.21.15.82:0/736420721 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f37c0084800 con 0x7f37c80cd220 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 0x7f37a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 0x7f37a8069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f37b800a830 tx=0x7f37b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 Processor -- start 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- start start 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x556527157f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x556527158890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556526fa0ff0 con 0x5565271a1e30 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x556526f29150 con 0x5565271584b0 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x556526f9d090 con 0x5565271a2750 2024-04-25T10:09:52.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58690/0 (socket says 172.21.15.82:58690) 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 -- 172.21.15.82:0/2805382746 learned_addr learned my addr 172.21.15.82:0/2805382746 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 -- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 msgr2=0x556527157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x556527157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 -- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 msgr2=0x556527158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x556527158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 -- 172.21.15.82:0/2805382746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565271370d0 con 0x5565271a1e30 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37c659c640 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f37b800ca10 tx=0x7f37b800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cfb87eef76015d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37a7fff640 1 -- 172.21.15.82:0/2805382746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8023070 con 0x5565271a1e30 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37a7fff640 1 -- 172.21.15.82:0/2805382746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b8013db0 con 0x5565271a1e30 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37a7fff640 1 -- 172.21.15.82:0/2805382746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b801e620 con 0x5565271a1e30 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- 172.21.15.82:0/2805382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 msgr2=0x5565271a2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f37b800ca10 tx=0x7f37b800f000 comp rx=0 tx=0).stop 2024-04-25T10:09:52.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- 172.21.15.82:0/2805382746 shutdown_connections 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x556527157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x556527158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 --2- 172.21.15.82:0/2805382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271a2210 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- 172.21.15.82:0/2805382746 >> 172.21.15.82:0/2805382746 conn(0x556527150020 msgr2=0x556527152440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- 172.21.15.82:0/2805382746 shutdown_connections 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- 172.21.15.82:0/2805382746 wait complete. 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 -- start start 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.295+0000 7f37d015b000 1 Processor -- start 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a1e30 0x55652715e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x55652715e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556526fa5620 con 0x5565271584b0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x556526f5b690 con 0x5565271a1e30 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x556526f29a40 con 0x5565271a2750 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 10 client.0 _ll_register_callbacks cb 0x55652713ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58704/0 (socket says 172.21.15.82:58704) 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 -- 172.21.15.82:0/2444019219 learned_addr learned my addr 172.21.15.82:0/2444019219 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 -- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 msgr2=0x55652715e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x55652715e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 -- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a1e30 msgr2=0x55652715e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a1e30 0x55652715e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 -- 172.21.15.82:0/2444019219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55652704e1c0 con 0x5565271584b0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37c659c640 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f37b8021040 tx=0x7f37b80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b882428efdb934c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:52.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8023040 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37b8013db0 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f37b801e9b0 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b8036050 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 10 client.8241 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55652704a170 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b8013db0 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f37b8039020 con 0x5565271584b0 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 10 client.8241 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 10 client.8241 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:52.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 20 client.8241 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55652715f610 0x5565271b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 4 client.8241 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556526f3cc60 con 0x55652715f610 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37bffff640 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55652715f610 0x5565271b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37bffff640 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55652715f610 0x5565271b40f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc30a65e8b49a42 server_cookie=317e150853362406 in_seq=0 out_seq=0 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 10 client.8241 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 1 -- 172.21.15.82:0/2444019219 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556526f3cc60 con 0x55652715f610 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 10 client.8241 handle_command_reply: tid=0 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 client.8241 shutdown 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 msgr2=0x5565271a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f37b8021040 tx=0x7f37b80046d0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 shutdown_connections 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55652715f610 0x5565271b40f0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x55652715e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a1e30 0x55652715e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 --2- 172.21.15.82:0/2444019219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x5565271a1840 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37a57fa640 0 client.8241 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 >> 172.21.15.82:0/2444019219 conn(0x556527150020 msgr2=0x556527159120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 shutdown_connections 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 1 -- 172.21.15.82:0/2444019219 wait complete. 2024-04-25T10:09:52.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.299+0000 7f37d015b000 20 client.8241 trim_cache size 0 max 16384 2024-04-25T10:09:52.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 Processor -- start 2024-04-25T10:09:52.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- start start 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x5565271b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x5565271af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c81ff3a0 con 0x5565271a1e30 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c806d680 con 0x5565271584b0 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c81587f0 con 0x5565271a2750 2024-04-25T10:09:52.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58710/0 (socket says 172.21.15.82:58710) 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 -- 172.21.15.82:0/191702955 learned_addr learned my addr 172.21.15.82:0/191702955 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 -- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 msgr2=0x5565271af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x5565271af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 -- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 msgr2=0x5565271b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x5565271b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 -- 172.21.15.82:0/191702955 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556526fe7de0 con 0x5565271a1e30 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37bffff640 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f37b001a920 tx=0x7f37b001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1739fa6d65621b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37a77fe640 1 -- 172.21.15.82:0/191702955 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00b0040 con 0x5565271a1e30 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37a77fe640 1 -- 172.21.15.82:0/191702955 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b00ab740 con 0x5565271a1e30 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37a77fe640 1 -- 172.21.15.82:0/191702955 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00abb10 con 0x5565271a1e30 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- 172.21.15.82:0/191702955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 msgr2=0x5565271b6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f37b001a920 tx=0x7f37b001c270 comp rx=0 tx=0).stop 2024-04-25T10:09:52.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- 172.21.15.82:0/191702955 shutdown_connections 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a2750 0x5565271af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271a1e30 0x5565271b6660 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 --2- 172.21.15.82:0/191702955 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271584b0 0x5565271b6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- 172.21.15.82:0/191702955 >> 172.21.15.82:0/191702955 conn(0x556527150020 msgr2=0x5565271aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- 172.21.15.82:0/191702955 shutdown_connections 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.315+0000 7f37d015b000 1 -- 172.21.15.82:0/191702955 wait complete. 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- start start 2024-04-25T10:09:52.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 Processor -- start 2024-04-25T10:09:52.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a1e30 0x5565271579a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a2750 0x5565271b55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x556526f3be00 con 0x5565271584b0 2024-04-25T10:09:52.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c806d080 con 0x5565271a2750 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x556526fa0300 con 0x5565271a1e30 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 10 client.0 _ll_register_callbacks cb 0x55652711f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:58718/0 (socket says 172.21.15.82:58718) 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 -- 172.21.15.82:0/597144463 learned_addr learned my addr 172.21.15.82:0/597144463 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:09:52.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37bffff640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a1e30 0x5565271579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 -- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a1e30 msgr2=0x5565271579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a1e30 0x5565271579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c6d9d640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a2750 0x5565271b55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 -- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a2750 msgr2=0x5565271b55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a2750 0x5565271b55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 -- 172.21.15.82:0/597144463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556526f428e0 con 0x5565271584b0 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37c659c640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f37b801edc0 tx=0x7f37b80047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9d933cf1f196a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8023040 con 0x5565271584b0 2024-04-25T10:09:52.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f37b8031760 con 0x5565271584b0 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37b00ab740 con 0x5565271584b0 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b8044050 con 0x5565271584b0 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 10 client.8247 fetch_fsmap learned FSMap version 63 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x556526ffa8f0 con 0x5565271584b0 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b00ab740 con 0x5565271584b0 2024-04-25T10:09:52.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f37b8047020 con 0x5565271584b0 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 10 client.8247 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 10 client.8247 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 20 client.8247 populate_metadata read hostname 'smithi082' 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x556527152410 0x5565271527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 4 client.8247 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:09:52.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556526fae530 con 0x556527152410 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37bffff640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x556527152410 0x5565271527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 10 client.8247 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37bffff640 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x556527152410 0x5565271527f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a316b2c5420d306c server_cookie=a9324be72563e084 in_seq=0 out_seq=0 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 1 -- 172.21.15.82:0/597144463 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x556526fae530 con 0x556527152410 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 10 client.8247 handle_command_reply: tid=0 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 client.8247 shutdown 2024-04-25T10:09:52.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 msgr2=0x556527157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f37b801edc0 tx=0x7f37b80047e0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 shutdown_connections 2024-04-25T10:09:52.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x556527152410 0x5565271527f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565271a1e30 0x5565271579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37a57fa640 0 client.8247 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565271a2750 0x5565271b55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 --2- 172.21.15.82:0/597144463 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565271584b0 0x556527157460 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 >> 172.21.15.82:0/597144463 conn(0x556527150020 msgr2=0x5565271af420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 shutdown_connections 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 1 -- 172.21.15.82:0/597144463 wait complete. 2024-04-25T10:09:52.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37d015b000 20 client.8247 trim_cache size 0 max 16384 2024-04-25T10:09:52.336 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:09:52.336 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: "9b818353-73ef-44f8-bba2-ee61ce8d2148": { 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir", 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "9b818353-73ef-44f8-bba2-ee61ce8d2148", 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:09:52.337 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:52.338 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:09:52.338 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:09:52.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 msgr2=0x7f37a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 0x7f37a8069420 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f37b800a830 tx=0x7f37b8008040 comp rx=0 tx=0).stop 2024-04-25T10:09:52.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 msgr2=0x7f37c80c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:09:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f37c000c780 tx=0x7f37c000cc40 comp rx=0 tx=0).stop 2024-04-25T10:09:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 shutdown_connections 2024-04-25T10:09:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37a8066f60 0x7f37a8069420 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37c80cd220 0x7f37c80c5820 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37c80b2bb0 0x7f37c815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 --2- 172.21.15.82:0/736420721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37c80b2740 0x7f37c80c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:09:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 >> 172.21.15.82:0/736420721 conn(0x7f37c80d15b0 msgr2=0x7f37c815ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:09:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 shutdown_connections 2024-04-25T10:09:52.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:09:52.319+0000 7f37ccfee640 1 -- 172.21.15.82:0/736420721 wait complete. 2024-04-25T10:09:52.342 INFO:tasks.cephfs.filesystem:scrub status for tag:9b818353-73ef-44f8-bba2-ee61ce8d2148 - {'path': '/dir', 'tag': '9b818353-73ef-44f8-bba2-ee61ce8d2148', 'options': 'recursive,repair'} 2024-04-25T10:09:59.999 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:09:59.999+0000 7ff48c596640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-25T10:10:08.093 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:08.097 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:22.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:22.344 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:10:22.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 Processor -- start 2024-04-25T10:10:22.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 -- start start 2024-04-25T10:10:22.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fed180cf590 0x7fed180cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed18072000 0x7fed18072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180cbd20 con 0x7fed1815b6b0 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1815ba90 con 0x7fed18072000 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.474+0000 7fed203db640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180cc000 con 0x7fed180cf590 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e177640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fed1815b6b0 0x7fed180ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50124/0 (socket says 172.21.15.82:50124) 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 --2- 172.21.15.82:0/2628728361 >> v2:172.21.15.82:3300/0 conn(0x7fed18072000 0x7fed18072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e177640 1 -- 172.21.15.82:0/2628728361 learned_addr learned my addr 172.21.15.82:0/2628728361 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1d976640 1 --2- 172.21.15.82:0/2628728361 >> v2:172.21.15.196:3300/0 conn(0x7fed180cf590 0x7fed180cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 -- 172.21.15.82:0/2628728361 >> v2:172.21.15.196:3300/0 conn(0x7fed180cf590 msgr2=0x7fed180cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 --2- 172.21.15.82:0/2628728361 >> v2:172.21.15.196:3300/0 conn(0x7fed180cf590 0x7fed180cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 -- 172.21.15.82:0/2628728361 >> v1:172.21.15.82:6789/0 conn(0x7fed1815b6b0 legacy=0x7fed180ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 -- 172.21.15.82:0/2628728361 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180cc2e0 con 0x7fed18072000 2024-04-25T10:10:22.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1d175640 1 -- 172.21.15.82:0/2628728361 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2676104982 0 0) 0x7fed180cbd20 con 0x7fed1815b6b0 2024-04-25T10:10:22.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1e978640 1 --2- 172.21.15.82:0/2628728361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed18072000 0x7fed18072430 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fed180b4430 tx=0x7fed0801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d32e4713bebdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1d175640 1 -- 172.21.15.82:0/2628728361 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080b0070 con 0x7fed18072000 2024-04-25T10:10:22.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1d175640 1 -- 172.21.15.82:0/2628728361 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080ab600 con 0x7fed18072000 2024-04-25T10:10:22.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.478+0000 7fed1d175640 1 -- 172.21.15.82:0/2628728361 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080ab9d0 con 0x7fed18072000 2024-04-25T10:10:22.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2628728361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed18072000 msgr2=0x7fed18072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- 172.21.15.82:0/2628728361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed18072000 0x7fed18072430 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fed180b4430 tx=0x7fed0801c480 comp rx=0 tx=0).stop 2024-04-25T10:10:22.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2628728361 shutdown_connections 2024-04-25T10:10:22.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- 172.21.15.82:0/2628728361 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed18072000 0x7fed18072430 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- 172.21.15.82:0/2628728361 >> v2:172.21.15.196:3300/0 conn(0x7fed180cf590 0x7fed180cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2628728361 >> 172.21.15.82:0/2628728361 conn(0x7fed1806ae60 msgr2=0x7fed1806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2628728361 shutdown_connections 2024-04-25T10:10:22.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2628728361 wait complete. 2024-04-25T10:10:22.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 Processor -- start 2024-04-25T10:10:22.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- start start 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 0x7fed180988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 0x7fed18164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180caf30 con 0x7fed180cf590 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180cadb0 con 0x7fed18072000 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181682a0 con 0x7fed1815b6b0 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36050/0 (socket says 172.21.15.82:36050) 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 -- 172.21.15.82:0/2574162387 learned_addr learned my addr 172.21.15.82:0/2574162387 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1e177640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 0x7fed180988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1e978640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 0x7fed18164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 -- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 msgr2=0x7fed18164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 0x7fed18164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 -- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 msgr2=0x7fed180988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 0x7fed180988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 -- 172.21.15.82:0/2574162387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1806d500 con 0x7fed180cf590 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1e978640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 0x7fed18164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1e177640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 0x7fed180988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed1d976640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d830 tx=0x7fed0c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed16ffd640 1 -- 172.21.15.82:0/2574162387 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00b660 con 0x7fed180cf590 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed16ffd640 1 -- 172.21.15.82:0/2574162387 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c019070 con 0x7fed180cf590 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1806d7e0 con 0x7fed180cf590 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed16ffd640 1 -- 172.21.15.82:0/2574162387 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c00ba80 con 0x7fed180cf590 2024-04-25T10:10:22.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.482+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1806dda0 con 0x7fed180cf590 2024-04-25T10:10:22.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed16ffd640 1 -- 172.21.15.82:0/2574162387 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed0c01e460 con 0x7fed180cf590 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed16ffd640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 0x7fed04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed16ffd640 1 -- 172.21.15.82:0/2574162387 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fed0c0859d0 con 0x7fed180cf590 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 0x7fed04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 0x7fed04069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed10007fd0 tx=0x7fed10005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 Processor -- start 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 -- start start 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d10697ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d10697b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1067c7ff0 con 0x55d1069c49d0 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d106750150 con 0x55d1069c52f0 2024-04-25T10:10:22.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1067c4090 con 0x55d10697b0a0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36062/0 (socket says 172.21.15.82:36062) 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 -- 172.21.15.82:0/3801920476 learned_addr learned my addr 172.21.15.82:0/3801920476 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1d976640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d10697ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e978640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d10697b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 -- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 msgr2=0x55d10697b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d10697b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 -- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 msgr2=0x55d10697ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d10697ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1e177640 1 -- 172.21.15.82:0/3801920476 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d10695e0d0 con 0x55d1069c49d0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.486+0000 7fed1d976640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d10697ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fed10001dd0 tx=0x7fed10012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2bd05b864bda21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfffff640 1 -- 172.21.15.82:0/3801920476 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10015b40 con 0x55d1069c49d0 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfffff640 1 -- 172.21.15.82:0/3801920476 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10015ce0 con 0x55d1069c49d0 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfffff640 1 -- 172.21.15.82:0/3801920476 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10021650 con 0x55d1069c49d0 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/3801920476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 msgr2=0x55d1069c4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fed10001dd0 tx=0x7fed10012000 comp rx=0 tx=0).stop 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/3801920476 shutdown_connections 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d10697b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d10697ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- 172.21.15.82:0/3801920476 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c49d0 0x55d1069c4db0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/3801920476 >> 172.21.15.82:0/3801920476 conn(0x55d106977020 msgr2=0x55d106979440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/3801920476 shutdown_connections 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/3801920476 wait complete. 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- start start 2024-04-25T10:10:22.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 Processor -- start 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c49d0 0x55d1069a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d1069aa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d10678c250 con 0x55d10697b0a0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d106782690 con 0x55d1069c52f0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d106750a40 con 0x55d1069c49d0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 10 client.0 _ll_register_callbacks cb 0x55d106965e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36078/0 (socket says 172.21.15.82:36078) 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 -- 172.21.15.82:0/4115041565 learned_addr learned my addr 172.21.15.82:0/4115041565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1d976640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c49d0 0x55d1069a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 -- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c49d0 msgr2=0x55d1069a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e978640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d1069aa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c49d0 0x55d1069a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 -- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 msgr2=0x55d1069aa270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d1069aa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 -- 172.21.15.82:0/4115041565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1068751c0 con 0x55d10697b0a0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1e177640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fed100012f0 tx=0x7fed10001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4b92988de0b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10023820 con 0x55d10697b0a0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fed10010040 con 0x55d10697b0a0 2024-04-25T10:10:22.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed10015ce0 con 0x55d10697b0a0 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed10034050 con 0x55d10697b0a0 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 10 client.8262 fetch_fsmap learned FSMap version 63 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55d106871170 con 0x55d10697b0a0 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10015ce0 con 0x55d10697b0a0 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed10037020 con 0x55d10697b0a0 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 10 client.8262 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 10 client.8262 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 20 client.8262 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d4c80 0x55d1069d7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 4 client.8262 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:22.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d106763c60 con 0x55d1069d4c80 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.490+0000 7fed1d976640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d4c80 0x55d1069d7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fecfd7fa640 10 client.8262 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed1d976640 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d4c80 0x55d1069d7070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78a7bd80b679dd3 server_cookie=db79c1803f3a2243 in_seq=0 out_seq=0 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fecfd7fa640 1 -- 172.21.15.82:0/4115041565 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d106763c60 con 0x55d1069d4c80 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fecfd7fa640 10 client.8262 handle_command_reply: tid=0 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 client.8262 shutdown 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 msgr2=0x55d1069a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fed100012f0 tx=0x7fed10001260 comp rx=0 tx=0).stop 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 shutdown_connections 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d4c80 0x55d1069d7070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c49d0 0x55d1069a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c52f0 0x55d1069aa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 --2- 172.21.15.82:0/4115041565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069a97f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 >> 172.21.15.82:0/4115041565 conn(0x55d106977020 msgr2=0x55d106979ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fecfd7fa640 0 client.8262 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 shutdown_connections 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 1 -- 172.21.15.82:0/4115041565 wait complete. 2024-04-25T10:10:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.494+0000 7fed23548000 20 client.8262 trim_cache size 0 max 16384 2024-04-25T10:10:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 Processor -- start 2024-04-25T10:10:22.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- start start 2024-04-25T10:10:22.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d106a20ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d106a21420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed182032e0 con 0x55d1069c52f0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed182024a0 con 0x55d1069c49d0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed182018a0 con 0x55d10697b0a0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36080/0 (socket says 172.21.15.82:36080) 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 -- 172.21.15.82:0/1074847564 learned_addr learned my addr 172.21.15.82:0/1074847564 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1d976640 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d106a21420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 -- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 msgr2=0x55d106a20ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d106a20ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 -- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 msgr2=0x55d106a21420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d106a21420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 -- 172.21.15.82:0/1074847564 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d10680ede0 con 0x55d1069c52f0 2024-04-25T10:10:22.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1d976640 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d106a21420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed1e978640 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fed08008010 tx=0x7fed08004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf080fa59883785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fecff7fe640 1 -- 172.21.15.82:0/1074847564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080b0040 con 0x55d1069c52f0 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fecff7fe640 1 -- 172.21.15.82:0/1074847564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080ab680 con 0x55d1069c52f0 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fecff7fe640 1 -- 172.21.15.82:0/1074847564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed080aba70 con 0x55d1069c52f0 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- 172.21.15.82:0/1074847564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 msgr2=0x55d106a21960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fed08008010 tx=0x7fed08004740 comp rx=0 tx=0).stop 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- 172.21.15.82:0/1074847564 shutdown_connections 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d1069c52f0 0x55d106a21960 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d106a21420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 --2- 172.21.15.82:0/1074847564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d10697b0a0 0x55d106a20ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- 172.21.15.82:0/1074847564 >> 172.21.15.82:0/1074847564 conn(0x55d106977020 msgr2=0x55d1069c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- 172.21.15.82:0/1074847564 shutdown_connections 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- 172.21.15.82:0/1074847564 wait complete. 2024-04-25T10:10:22.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 -- start start 2024-04-25T10:10:22.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.510+0000 7fed23548000 1 Processor -- start 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d1069d78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c52f0 0x55d1069d7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d106762e00 con 0x55d10697b0a0 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed18201ea0 con 0x55d1069c49d0 2024-04-25T10:10:22.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1067c7300 con 0x55d1069c52f0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 10 client.0 _ll_register_callbacks cb 0x55d106946040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36088/0 (socket says 172.21.15.82:36088) 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 -- 172.21.15.82:0/324616372 learned_addr learned my addr 172.21.15.82:0/324616372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1d976640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d1069d78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 -- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c52f0 msgr2=0x55d1069d7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c52f0 0x55d1069d7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 -- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 msgr2=0x55d1069d78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d1069d78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 -- 172.21.15.82:0/324616372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1067698e0 con 0x55d10697b0a0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1e177640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fed100019b0 tx=0x7fed10001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b233d04c071310 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10015cc0 con 0x55d10697b0a0 2024-04-25T10:10:22.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fed10029070 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed080ab680 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed1003e050 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 10 client.8268 fetch_fsmap learned FSMap version 63 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55d1068218f0 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed080ab680 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fed10041020 con 0x55d10697b0a0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 10 client.8268 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 10 client.8268 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 20 client.8268 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d87f0 0x55d106a2cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 4 client.8268 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d1067d5530 con 0x55d1069d87f0 2024-04-25T10:10:22.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1d976640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d87f0 0x55d106a2cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 10 client.8268 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed1d976640 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d87f0 0x55d106a2cf60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0e9f9181e24613 server_cookie=e47f7dba4a344248 in_seq=0 out_seq=0 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 1 -- 172.21.15.82:0/324616372 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d1067d5530 con 0x55d1069d87f0 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 10 client.8268 handle_command_reply: tid=0 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 client.8268 shutdown 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 msgr2=0x55d1069d7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fed100019b0 tx=0x7fed10001410 comp rx=0 tx=0).stop 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 shutdown_connections 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55d1069d87f0 0x55d106a2cf60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d1069c52f0 0x55d1069d7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d1069c49d0 0x55d1069d78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 --2- 172.21.15.82:0/324616372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d10697b0a0 0x55d1069d7380 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 >> 172.21.15.82:0/324616372 conn(0x55d106977020 msgr2=0x55d1069c3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fecfd7fa640 0 client.8268 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 shutdown_connections 2024-04-25T10:10:22.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 1 -- 172.21.15.82:0/324616372 wait complete. 2024-04-25T10:10:22.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.514+0000 7fed23548000 20 client.8268 trim_cache size 0 max 16384 2024-04-25T10:10:22.524 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:10:22.524 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:10:22.524 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:10:22.524 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 msgr2=0x7fed04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 0x7fed04069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fed10007fd0 tx=0x7fed10005b80 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 msgr2=0x7fed180cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00d830 tx=0x7fed0c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 shutdown_connections 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed04066f60 0x7fed04069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed1815b6b0 0x7fed18164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed180cf590 0x7fed180cdda0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 --2- 172.21.15.82:0/2574162387 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed18072000 0x7fed180988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 >> 172.21.15.82:0/2574162387 conn(0x7fed1806ae60 msgr2=0x7fed18073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 shutdown_connections 2024-04-25T10:10:22.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.518+0000 7fed203db640 1 -- 172.21.15.82:0/2574162387 wait complete. 2024-04-25T10:10:22.537 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:10:22.537 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:22.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 Processor -- start 2024-04-25T10:10:22.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 -- start start 2024-04-25T10:10:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 0x7fc87815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc8780b2c30 0x7fc87806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc878066c30 con 0x7fc87806c1e0 2024-04-25T10:10:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87806daa0 con 0x7fc8780b2c30 2024-04-25T10:10:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87f91a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780af660 con 0x7fc8780af280 2024-04-25T10:10:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87deb7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc87806c1e0 0x7fc878161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50138/0 (socket says 172.21.15.82:50138) 2024-04-25T10:10:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87ceb5640 1 --2- 172.21.15.82:0/120875108 >> v2:172.21.15.82:3300/0 conn(0x7fc8780b2c30 0x7fc87806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87deb7640 1 -- 172.21.15.82:0/120875108 learned_addr learned my addr 172.21.15.82:0/120875108 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/120875108 >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 0x7fc87815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87ceb5640 1 -- 172.21.15.82:0/120875108 >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 msgr2=0x7fc87815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87ceb5640 1 --2- 172.21.15.82:0/120875108 >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 0x7fc87815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87ceb5640 1 -- 172.21.15.82:0/120875108 >> v1:172.21.15.82:6789/0 conn(0x7fc87806c1e0 legacy=0x7fc878161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc877fff640 1 -- 172.21.15.82:0/120875108 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298528643 0 0) 0x7fc878066c30 con 0x7fc87806c1e0 2024-04-25T10:10:22.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87ceb5640 1 -- 172.21.15.82:0/120875108 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87815e640 con 0x7fc8780b2c30 2024-04-25T10:10:22.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.646+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/120875108 >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 0x7fc87815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87ceb5640 1 --2- 172.21.15.82:0/120875108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87806bca0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc8780cfbf0 tx=0x7fc86c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8b163f209a7208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc877fff640 1 -- 172.21.15.82:0/120875108 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0b0070 con 0x7fc8780b2c30 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc877fff640 1 -- 172.21.15.82:0/120875108 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c0ab600 con 0x7fc8780b2c30 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc877fff640 1 -- 172.21.15.82:0/120875108 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0ab9d0 con 0x7fc8780b2c30 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/120875108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 msgr2=0x7fc87806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- 172.21.15.82:0/120875108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87806bca0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fc8780cfbf0 tx=0x7fc86c01c480 comp rx=0 tx=0).stop 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/120875108 shutdown_connections 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- 172.21.15.82:0/120875108 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87806bca0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- 172.21.15.82:0/120875108 >> v2:172.21.15.196:3300/0 conn(0x7fc8780af280 0x7fc87815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/120875108 >> 172.21.15.82:0/120875108 conn(0x7fc878071810 msgr2=0x7fc878071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/120875108 shutdown_connections 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/120875108 wait complete. 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 Processor -- start 2024-04-25T10:10:22.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- start start 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 0x7fc8781509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8780af280 0x7fc878166880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87816ac30 con 0x7fc8780b2c30 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87816aab0 con 0x7fc87806c1e0 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87816adb0 con 0x7fc8780af280 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36102/0 (socket says 172.21.15.82:36102) 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 -- 172.21.15.82:0/841958566 learned_addr learned my addr 172.21.15.82:0/841958566 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 0x7fc8781509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87ceb5640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8780af280 0x7fc878166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 -- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8780af280 msgr2=0x7fc878166880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8780af280 0x7fc878166880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 -- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 msgr2=0x7fc8781509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 0x7fc8781509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 -- 172.21.15.82:0/841958566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc878163810 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 0x7fc8781509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87deb7640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc870001570 tx=0x7fc87000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870018070 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000a640 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc878163250 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000aa30 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.650+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87815f8f0 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87804efa0 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc87001b070 con 0x7fc8780b2c30 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc875ffb640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 0x7fc864069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc8700848a0 con 0x7fc8780b2c30 2024-04-25T10:10:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 0x7fc864069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.654+0000 7fc87d6b6640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 0x7fc864069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc868008930 tx=0x7fc868008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.658+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc870055660 con 0x7fc8780b2c30 2024-04-25T10:10:22.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8780727c0 con 0x7fc8780b2c30 2024-04-25T10:10:22.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc875ffb640 1 -- 172.21.15.82:0/841958566 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc870054f60 con 0x7fc8780b2c30 2024-04-25T10:10:22.841 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:22.841 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:23.329237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:22.841 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 63 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 msgr2=0x7fc864069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 0x7fc864069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc868008930 tx=0x7fc868008040 comp rx=0 tx=0).stop 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 msgr2=0x7fc87815fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.838+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc870001570 tx=0x7fc87000cca0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 shutdown_connections 2024-04-25T10:10:22.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc864066f90 0x7fc864069450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8780af280 0x7fc878166880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8780b2c30 0x7fc87815fe10 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 --2- 172.21.15.82:0/841958566 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc87806c1e0 0x7fc8781509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 >> 172.21.15.82:0/841958566 conn(0x7fc878071810 msgr2=0x7fc87806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 shutdown_connections 2024-04-25T10:10:22.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.842+0000 7fc87f91a640 1 -- 172.21.15.82:0/841958566 wait complete. 2024-04-25T10:10:22.858 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:10:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 Processor -- start 2024-04-25T10:10:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 -- start start 2024-04-25T10:10:22.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40b3cf0 0x7f3ff40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ff415e6c0 0x7f3ff40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff40cbf10 con 0x7f3ff40cfad0 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff40cff10 con 0x7f3ff415e6c0 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.974+0000 7f3ffb491640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff415ead0 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff9a2e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3ff415e6c0 0x7f3ff40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff922d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40b3cf0 0x7f3ff40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff8a2c640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f3ff40cfad0 0x7f3ff415dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:49276/0 (socket says 172.21.15.82:49276) 2024-04-25T10:10:22.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/1044751205 learned_addr learned my addr 172.21.15.82:0/1044751205 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:22.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40b3cf0 msgr2=0x7f3ff40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40b3cf0 0x7f3ff40cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/1044751205 >> v1:172.21.15.196:6789/0 conn(0x7f3ff40cfad0 legacy=0x7f3ff415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.978+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/1044751205 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40cc1f0 con 0x7f3ff415e6c0 2024-04-25T10:10:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff40caaf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ecffdb7172538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3febfff640 1 -- 172.21.15.82:0/1044751205 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0b2070 con 0x7f3ff415e6c0 2024-04-25T10:10:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3febfff640 1 -- 172.21.15.82:0/1044751205 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec0a5830 con 0x7f3ff415e6c0 2024-04-25T10:10:22.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3febfff640 1 -- 172.21.15.82:0/1044751205 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0a5c00 con 0x7f3ff415e6c0 2024-04-25T10:10:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 msgr2=0x7f3ff40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff40caaf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec01c970 comp rx=0 tx=0).stop 2024-04-25T10:10:22.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- 172.21.15.82:0/1044751205 shutdown_connections 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff40caaf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- 172.21.15.82:0/1044751205 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40b3cf0 0x7f3ff40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- 172.21.15.82:0/1044751205 >> 172.21.15.82:0/1044751205 conn(0x7f3ff406ae60 msgr2=0x7f3ff406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- 172.21.15.82:0/1044751205 shutdown_connections 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- 172.21.15.82:0/1044751205 wait complete. 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 Processor -- start 2024-04-25T10:10:22.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- start start 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40cfad0 0x7f3ff4154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff4071fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40b4250 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40b40d0 con 0x7f3ff415e6c0 2024-04-25T10:10:22.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ffb491640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff40cb840 con 0x7f3ff40cfad0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ff922d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ff922d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36110/0 (socket says 172.21.15.82:36110) 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.982+0000 7f3ff922d640 1 -- 172.21.15.82:0/2388149757 learned_addr learned my addr 172.21.15.82:0/2388149757 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff4071fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40cfad0 0x7f3ff4154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 -- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40cfad0 msgr2=0x7f3ff4154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40cfad0 0x7f3ff4154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 -- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 msgr2=0x7f3ff4071fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff4071fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 -- 172.21.15.82:0/2388149757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff4098ed0 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e930 tx=0x7f3ff000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 -- 172.21.15.82:0/2388149757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0017070 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 -- 172.21.15.82:0/2388149757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000ca00 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff41557d0 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 -- 172.21.15.82:0/2388149757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000cdb0 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff4155d90 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 -- 172.21.15.82:0/2388149757 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ff001a070 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 0x7f3fd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3fe9ffb640 1 -- 172.21.15.82:0/2388149757 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3ff0085260 con 0x7f3ff40b3cf0 2024-04-25T10:10:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 0x7f3fd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 0x7f3fd8069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3ff415a030 tx=0x7f3fe4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ffe5fe000 1 Processor -- start 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.986+0000 7f3ffe5fe000 1 -- start start 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ec36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88e79cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88d60ba0 con 0x55cc88ec3c00 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88cc6090 con 0x55cc88ec32e0 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc88c4e160 con 0x55cc88e798e0 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36116/0 (socket says 172.21.15.82:36116) 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2429549152 learned_addr learned my addr 172.21.15.82:0/2429549152 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ec36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 msgr2=0x55cc88e79cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88e79cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 msgr2=0x55cc88ec36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ec36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2429549152 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc88d6fc20 con 0x55cc88ec3c00 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005c40 tx=0x7f3fe4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5decc2575065b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3fdeffd640 1 -- 172.21.15.82:0/2429549152 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400d900 con 0x55cc88ec3c00 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3fdeffd640 1 -- 172.21.15.82:0/2429549152 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400daa0 con 0x55cc88ec3c00 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3fdeffd640 1 -- 172.21.15.82:0/2429549152 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe400de90 con 0x55cc88ec3c00 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2429549152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 msgr2=0x55cc88e793a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:22.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4005c40 tx=0x7f3fe4010000 comp rx=0 tx=0).stop 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2429549152 shutdown_connections 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88e79cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88e793a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2429549152 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ec36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2429549152 >> 172.21.15.82:0/2429549152 conn(0x55cc88e75890 msgr2=0x55cc88e77c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2429549152 shutdown_connections 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2429549152 wait complete. 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- start start 2024-04-25T10:10:22.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 Processor -- start 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ecab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec3c00 0x55cc88eb1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88cc2130 con 0x55cc88e798e0 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88c8a130 con 0x55cc88ec32e0 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc88d84300 con 0x55cc88ec3c00 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ffe5fe000 10 client.0 _ll_register_callbacks cb 0x55cc88e64bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36122/0 (socket says 172.21.15.82:36122) 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 -- 172.21.15.82:0/3213478436 learned_addr learned my addr 172.21.15.82:0/3213478436 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ecab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 -- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec3c00 msgr2=0x55cc88eb1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec3c00 0x55cc88eb1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 -- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 msgr2=0x55cc88ecab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ecab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 -- 172.21.15.82:0/3213478436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc88ca3060 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ecab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3ff922d640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3ff005dbc0 tx=0x7f3ff00561a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9fa6aaac722d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.990+0000 7f3fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0080020 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fe400daa0 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3ff0054e30 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ff0021040 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 10 client.8286 fetch_fsmap learned FSMap version 63 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55cc88c677c0 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe400daa0 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ff0024060 con 0x55cc88e798e0 2024-04-25T10:10:23.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 10 client.8286 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 10 client.8286 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 20 client.8286 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88eb1e00 0x55cc88eb21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 4 client.8286 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc88c61cd0 con 0x55cc88eb1e00 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88eb1e00 0x55cc88eb21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 10 client.8286 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88eb1e00 0x55cc88eb21e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e66d8e16aba90f server_cookie=f91a4271719b1af6 in_seq=0 out_seq=0 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 1 -- 172.21.15.82:0/3213478436 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cc88c61cd0 con 0x55cc88eb1e00 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 10 client.8286 handle_command_reply: tid=0 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 client.8286 shutdown 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 msgr2=0x55cc88eca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f3ff005dbc0 tx=0x7f3ff00561a0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 shutdown_connections 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88eb1e00 0x55cc88eb21e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec3c00 0x55cc88eb1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec32e0 0x55cc88ecab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3fd3fff640 0 client.8286 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/3213478436 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88e798e0 0x55cc88eca640 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 >> 172.21.15.82:0/3213478436 conn(0x55cc88e75890 msgr2=0x55cc88e78510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 shutdown_connections 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/3213478436 wait complete. 2024-04-25T10:10:23.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:22.994+0000 7f3ffe5fe000 20 client.8286 trim_cache size 0 max 16384 2024-04-25T10:10:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 Processor -- start 2024-04-25T10:10:23.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- start start 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88e798e0 0x55cc88f1f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec32e0 0x55cc88f1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff42025f0 con 0x55cc88ec3c00 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff4158e40 con 0x55cc88e798e0 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff4202040 con 0x55cc88ec32e0 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36124/0 (socket says 172.21.15.82:36124) 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/2004262801 learned_addr learned my addr 172.21.15.82:0/2004262801 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec32e0 msgr2=0x55cc88f1fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec32e0 0x55cc88f1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88e798e0 msgr2=0x55cc88f1f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88e798e0 0x55cc88f1f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 -- 172.21.15.82:0/2004262801 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc88c81fc0 con 0x55cc88ec3c00 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ff9a2e640 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec01b1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a9258613191196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3fde7fc640 1 -- 172.21.15.82:0/2004262801 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0bd070 con 0x55cc88ec3c00 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3fde7fc640 1 -- 172.21.15.82:0/2004262801 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec0a5830 con 0x55cc88ec3c00 2024-04-25T10:10:23.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3fde7fc640 1 -- 172.21.15.82:0/2004262801 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0b2040 con 0x55cc88ec3c00 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2004262801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 msgr2=0x55cc88f1ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009d90 tx=0x7f3fec01b1b0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2004262801 shutdown_connections 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88ec32e0 0x55cc88f1fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec3c00 0x55cc88f1ff80 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2004262801 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88e798e0 0x55cc88f1f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2004262801 >> 172.21.15.82:0/2004262801 conn(0x55cc88e75890 msgr2=0x55cc88ec2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2004262801 shutdown_connections 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.006+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2004262801 wait complete. 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- start start 2024-04-25T10:10:23.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 Processor -- start 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88f23510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec3c00 0x55cc88ec79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88c60e70 con 0x55cc88ec32e0 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc88c4ea50 con 0x55cc88ec3c00 2024-04-25T10:10:23.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cc88cc53a0 con 0x55cc88e798e0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 10 client.0 _ll_register_callbacks cb 0x55cc88c72240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36132/0 (socket says 172.21.15.82:36132) 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2151594199 learned_addr learned my addr 172.21.15.82:0/2151594199 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 msgr2=0x55cc88f23510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88f23510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec3c00 msgr2=0x55cc88ec79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec3c00 0x55cc88ec79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 -- 172.21.15.82:0/2151594199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc88d1f9e0 con 0x55cc88ec32e0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff8a2c640 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4013000 tx=0x7f3fe4000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0934da9cb55e7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:23.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe402d7b0 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3fe4020450 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fec0a5830 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fe403d050 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 10 client.8292 fetch_fsmap learned FSMap version 63 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x55cc88c65630 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec0a5830 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 63) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3fe4040020 con 0x55cc88ec32e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 10 client.8292 fetch_fsmap finished waiting for FSMap version 63 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 10 client.8292 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 20 client.8292 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88ec82e0 0x55cc88ec86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 4 client.8292 mds_command: new command op to 8052 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 --> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55cc88cd35d0 con 0x55cc88ec82e0 2024-04-25T10:10:23.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88ec82e0 0x55cc88ec86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 10 client.8292 ms_handle_connect on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ff922d640 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88ec82e0 0x55cc88ec86c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a40158f818ed551b server_cookie=6b3470a352cdae53 in_seq=0 out_seq=0 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 1 -- 172.21.15.82:0/2151594199 <== mds.0 v2:172.21.15.82:6836/362560393 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x55cc88cd35d0 con 0x55cc88ec82e0 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 10 client.8292 handle_command_reply: tid=0 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 client.8292 shutdown 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 msgr2=0x55cc88f23a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4013000 tx=0x7f3fe4000ee0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 shutdown_connections 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:6836/362560393,v1:172.21.15.82:6837/362560393] conn(0x55cc88ec82e0 0x55cc88ec86c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55cc88ec3c00 0x55cc88ec79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55cc88ec32e0 0x55cc88f23a50 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 --2- 172.21.15.82:0/2151594199 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55cc88e798e0 0x55cc88f23510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 >> 172.21.15.82:0/2151594199 conn(0x55cc88e75890 msgr2=0x55cc88ec2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3fd3fff640 0 client.8292 ms_handle_reset on v2:172.21.15.82:6836/362560393 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 shutdown_connections 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 1 -- 172.21.15.82:0/2151594199 wait complete. 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffe5fe000 20 client.8292 trim_cache size 0 max 16384 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:10:23.021 INFO:teuthology.orchestra.run.smithi082.stdout: "damage_type": "backtrace", 2024-04-25T10:10:23.022 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 1194646827, 2024-04-25T10:10:23.022 INFO:teuthology.orchestra.run.smithi082.stdout: "ino": 1099511627777, 2024-04-25T10:10:23.022 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/testdir/hardlink" 2024-04-25T10:10:23.022 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:10:23.022 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 msgr2=0x7f3fd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 0x7f3fd8069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3ff415a030 tx=0x7f3fe4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 msgr2=0x7f3ff4098990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3ff000e930 tx=0x7f3ff000edf0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 shutdown_connections 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3fd8066f60 0x7f3fd8069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3ff40cfad0 0x7f3ff4154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3ff415e6c0 0x7f3ff4071fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 --2- 172.21.15.82:0/2388149757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3ff40b3cf0 0x7f3ff4098990 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 >> 172.21.15.82:0/2388149757 conn(0x7f3ff406ae60 msgr2=0x7f3ff406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 shutdown_connections 2024-04-25T10:10:23.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.010+0000 7f3ffb491640 1 -- 172.21.15.82:0/2388149757 wait complete. 2024-04-25T10:10:23.030 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:10:23.030 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:23.036 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:23.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 Processor -- start 2024-04-25T10:10:23.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 -- start start 2024-04-25T10:10:23.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4180b3f50 0x7fb41815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180b2090 con 0x7fb4180b24a0 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180b43e0 con 0x7fb41815a700 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb42026b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41815f200 con 0x7fb4180b3f50 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4180b3f50 0x7fb41815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4180b3f50 0x7fb41815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36134/0 (socket says 172.21.15.82:36134) 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 -- 172.21.15.82:0/2719830686 learned_addr learned my addr 172.21.15.82:0/2719830686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 -- 172.21.15.82:0/2719830686 >> v1:172.21.15.196:6789/0 conn(0x7fb41815a700 legacy=0x7fb4180cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 -- 172.21.15.82:0/2719830686 >> v1:172.21.15.82:6789/0 conn(0x7fb4180b24a0 legacy=0x7fb41806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 -- 172.21.15.82:0/2719830686 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4180b46c0 con 0x7fb4180b3f50 2024-04-25T10:10:23.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d806640 1 --2- 172.21.15.82:0/2719830686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4180b3f50 0x7fb41815ecc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb408009870 tx=0x7fb40801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0bc3e04aa5a190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.182+0000 7fb41d005640 1 -- 172.21.15.82:0/2719830686 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b0070 con 0x7fb4180b3f50 2024-04-25T10:10:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d005640 1 -- 172.21.15.82:0/2719830686 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080ab600 con 0x7fb4180b3f50 2024-04-25T10:10:23.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d005640 1 -- 172.21.15.82:0/2719830686 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080ab9d0 con 0x7fb4180b3f50 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2719830686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4180b3f50 msgr2=0x7fb41815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 --2- 172.21.15.82:0/2719830686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4180b3f50 0x7fb41815ecc0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fb408009870 tx=0x7fb40801c3d0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2719830686 shutdown_connections 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 --2- 172.21.15.82:0/2719830686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4180b3f50 0x7fb41815ecc0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2719830686 >> 172.21.15.82:0/2719830686 conn(0x7fb4180b11a0 msgr2=0x7fb4180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2719830686 shutdown_connections 2024-04-25T10:10:23.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2719830686 wait complete. 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 Processor -- start 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- start start 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4180b24a0 0x7fb4180a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4181532e0 0x7fb418151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cd310 con 0x7fb41815a700 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180ce600 con 0x7fb4181532e0 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cd490 con 0x7fb4180b24a0 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36150/0 (socket says 172.21.15.82:36150) 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 -- 172.21.15.82:0/2743058854 learned_addr learned my addr 172.21.15.82:0/2743058854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 -- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4180b24a0 msgr2=0x7fb4180a8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4180b24a0 0x7fb4180a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 -- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4181532e0 msgr2=0x7fb418151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4181532e0 0x7fb418151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 -- 172.21.15.82:0/2743058854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb418153810 con 0x7fb41815a700 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb41d806640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ae040 tx=0x7fb40801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080ba020 con 0x7fb41815a700 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4080b6640 con 0x7fb41815a700 2024-04-25T10:10:23.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb418153af0 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b0040 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.186+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4181540b0 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4181542f0 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4080bd020 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb40effd640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 0x7fb3fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb41e007640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 0x7fb3fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb40811f620 con 0x7fb41815a700 2024-04-25T10:10:23.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.190+0000 7fb41e007640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 0x7fb3fc069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb414007fd0 tx=0x7fb414007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.194+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4181542f0 con 0x7fb41815a700 2024-04-25T10:10:23.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.354+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb41804efa0 con 0x7fb41815a700 2024-04-25T10:10:23.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb40effd640 1 -- 172.21.15.82:0/2743058854 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17219 (secure 0 0 0) 0x7fb4080c4030 con 0x7fb41815a700 2024-04-25T10:10:23.360 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:23.360 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:09:50.537791+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:23.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 msgr2=0x7fb3fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 0x7fb3fc069420 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb414007fd0 tx=0x7fb414007d40 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 msgr2=0x7fb418152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ae040 tx=0x7fb40801ac10 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 shutdown_connections 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f60 0x7fb3fc069420 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4181532e0 0x7fb418151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb41815a700 0x7fb418152da0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 --2- 172.21.15.82:0/2743058854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4180b24a0 0x7fb4180a8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 >> 172.21.15.82:0/2743058854 conn(0x7fb4180b11a0 msgr2=0x7fb4180af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 shutdown_connections 2024-04-25T10:10:23.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.358+0000 7fb42026b640 1 -- 172.21.15.82:0/2743058854 wait complete. 2024-04-25T10:10:23.379 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:23.380 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:23.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 Processor -- start 2024-04-25T10:10:23.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 -- start start 2024-04-25T10:10:23.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f749c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f749c072000 0x7f749c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749c0cbd20 con 0x7f749c15b6b0 2024-04-25T10:10:23.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749c15ba90 con 0x7f749c072000 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f74a194c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0cff30 con 0x7f749c0cf590 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f749c15b6b0 0x7f749c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50152/0 (socket says 172.21.15.82:50152) 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749affd640 1 -- 172.21.15.82:0/1782444110 learned_addr learned my addr 172.21.15.82:0/1782444110 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 --2- 172.21.15.82:0/1782444110 >> v2:172.21.15.196:3300/0 conn(0x7f749c072000 0x7f749c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749a7fc640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f749c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.516 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2074452045 0 0) 0x7f749c0cbd20 con 0x7f749c15b6b0 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7484003530 con 0x7f749c15b6b0 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1494422226 0 0) 0x7f7484003530 con 0x7f749c15b6b0 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 msgr2=0x7f749c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f749c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 >> v2:172.21.15.196:3300/0 conn(0x7f749c072000 msgr2=0x7f749c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 --2- 172.21.15.82:0/1782444110 >> v2:172.21.15.196:3300/0 conn(0x7f749c072000 0x7f749c072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0cc000 con 0x7f749c15b6b0 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f748c001680 con 0x7f749c15b6b0 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 >> v1:172.21.15.82:6789/0 conn(0x7f749c15b6b0 legacy=0x7f749c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f7484003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484003fb0 0x7f7484004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7484004d50 0x7f74840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c15ba90 con 0x7f7484004d50 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748400b9d0 con 0x7f7484003fb0 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748400bb80 con 0x7f749c0cf590 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7484004d50 0x7f74840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749affd640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f7484003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 -- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 msgr2=0x7f7484003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749a7fc640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484003fb0 0x7f7484004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c0cf590 0x7f7484003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 -- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484003fb0 msgr2=0x7f7484004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7484003fb0 0x7f7484004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.502+0000 7f749b7fe640 1 -- 172.21.15.82:0/1782444110 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484003530 con 0x7f7484004d50 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749b7fe640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7484004d50 0x7f74840051d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7494009fc0 tx=0x7f749401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a43f916801e3940 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74940b2070 con 0x7f7484004d50 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74940a5cd0 con 0x7f7484004d50 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f7499ffb640 1 -- 172.21.15.82:0/1782444110 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494009c10 con 0x7f7484004d50 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- 172.21.15.82:0/1782444110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7484004d50 msgr2=0x7f74840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 --2- 172.21.15.82:0/1782444110 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7484004d50 0x7f74840051d0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f7494009fc0 tx=0x7f749401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749b7fe640 1 -- 172.21.15.82:0/1782444110 reap_dead start 2024-04-25T10:10:23.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- 172.21.15.82:0/1782444110 shutdown_connections 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- 172.21.15.82:0/1782444110 >> 172.21.15.82:0/1782444110 conn(0x7f749c06ae60 msgr2=0x7f749c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- 172.21.15.82:0/1782444110 shutdown_connections 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- 172.21.15.82:0/1782444110 wait complete. 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 Processor -- start 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- start start 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f749c098880 0x7f749c1548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c159d00 0x7f749c15a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0cc8a0 con 0x7f749c098c60 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0cc5c0 con 0x7f749c098880 2024-04-25T10:10:23.520 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f74a194c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749c0cba30 con 0x7f749c159d00 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36174/0 (socket says 172.21.15.82:36174) 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 -- 172.21.15.82:0/3740813428 learned_addr learned my addr 172.21.15.82:0/3740813428 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749affd640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f749c098880 0x7f749c1548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749b7fe640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c159d00 0x7f749c15a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 -- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c159d00 msgr2=0x7f749c15a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c159d00 0x7f749c15a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 -- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f749c098880 msgr2=0x7f749c1548f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f749c098880 0x7f749c1548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.506+0000 7f749a7fc640 1 -- 172.21.15.82:0/3740813428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749c0ad280 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f749a7fc640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f74900108f0 tx=0x7f7490010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000f040 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749000eb60 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f749c0ad560 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749001c400 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749c0adb20 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749c04efa0 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f749001c5a0 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 0x7f7488069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7490085980 con 0x7f749c098c60 2024-04-25T10:10:23.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f749affd640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 0x7f7488069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.510+0000 7f749affd640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 0x7f7488069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f749c14feb0 tx=0x7f748c000f00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.514+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7490018030 con 0x7f749c098c60 2024-04-25T10:10:23.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.666+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f749c15fa50 con 0x7f749c098c60 2024-04-25T10:10:23.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.666+0000 7f7483fff640 1 -- 172.21.15.82:0/3740813428 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7490056740 con 0x7f749c098c60 2024-04-25T10:10:23.671 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:10:23.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 msgr2=0x7f7488069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 0x7f7488069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f749c14feb0 tx=0x7f748c000f00 comp rx=0 tx=0).stop 2024-04-25T10:10:23.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 msgr2=0x7f749c0acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f74900108f0 tx=0x7f7490010db0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 shutdown_connections 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7488066f90 0x7f7488069450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f749c159d00 0x7f749c15a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f749c098c60 0x7f749c0acb80 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 --2- 172.21.15.82:0/3740813428 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f749c098880 0x7f749c1548f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 >> 172.21.15.82:0/3740813428 conn(0x7f749c06ae60 msgr2=0x7f749c0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 shutdown_connections 2024-04-25T10:10:23.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.670+0000 7f74a194c640 1 -- 172.21.15.82:0/3740813428 wait complete. 2024-04-25T10:10:23.688 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:23.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 Processor -- start 2024-04-25T10:10:23.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 -- start start 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa000b0a30 0x7ffa000cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa0015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000b0e10 con 0x7ffa0006f4a0 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000d0d90 con 0x7ffa000b0a30 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ffa057d9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000d0180 con 0x7ffa0009a180 2024-04-25T10:10:23.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9feffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffa000b0a30 0x7ffa000cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9fe7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffa0006f4a0 0x7ffa00099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50156/0 (socket says 172.21.15.82:50156) 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9fe7fc640 1 -- 172.21.15.82:0/1644967616 learned_addr learned my addr 172.21.15.82:0/1644967616 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9feffd640 1 -- 172.21.15.82:0/1644967616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 msgr2=0x7ffa0015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9feffd640 1 --2- 172.21.15.82:0/1644967616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa0015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9feffd640 1 -- 172.21.15.82:0/1644967616 >> v1:172.21.15.82:6789/0 conn(0x7ffa0006f4a0 legacy=0x7ffa00099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9feffd640 1 -- 172.21.15.82:0/1644967616 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000d0460 con 0x7ffa000b0a30 2024-04-25T10:10:23.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.810+0000 7ff9fdffb640 1 -- 172.21.15.82:0/1644967616 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619331757 0 0) 0x7ffa000b0e10 con 0x7ffa0006f4a0 2024-04-25T10:10:23.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ff9feffd640 1 --2- 172.21.15.82:0/1644967616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa000b0a30 0x7ffa000cf890 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00097c0 tx=0x7ff9f001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d68384b34fd10c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ff9fdffb640 1 -- 172.21.15.82:0/1644967616 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00a7070 con 0x7ffa000b0a30 2024-04-25T10:10:23.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ff9fdffb640 1 -- 172.21.15.82:0/1644967616 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f00b0070 con 0x7ffa000b0a30 2024-04-25T10:10:23.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ff9fdffb640 1 -- 172.21.15.82:0/1644967616 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00ab890 con 0x7ffa000b0a30 2024-04-25T10:10:23.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- 172.21.15.82:0/1644967616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa000b0a30 msgr2=0x7ffa000cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- 172.21.15.82:0/1644967616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa000b0a30 0x7ffa000cf890 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00097c0 tx=0x7ff9f001c470 comp rx=0 tx=0).stop 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- 172.21.15.82:0/1644967616 shutdown_connections 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- 172.21.15.82:0/1644967616 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa0015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- 172.21.15.82:0/1644967616 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa000b0a30 0x7ffa000cf890 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- 172.21.15.82:0/1644967616 >> 172.21.15.82:0/1644967616 conn(0x7ffa000b30b0 msgr2=0x7ffa000b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- 172.21.15.82:0/1644967616 shutdown_connections 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- 172.21.15.82:0/1644967616 wait complete. 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 Processor -- start 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- start start 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa000cad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 0x7ffa000cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cbd90 con 0x7ffa0006f4a0 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cbc10 con 0x7ffa000b0a30 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.814+0000 7ffa057d9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cbf10 con 0x7ffa0009a180 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36180/0 (socket says 172.21.15.82:36180) 2024-04-25T10:10:23.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 -- 172.21.15.82:0/4148986673 learned_addr learned my addr 172.21.15.82:0/4148986673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:23.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9fe7fc640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa000cad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9ff7fe640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 0x7ffa000cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 -- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 msgr2=0x7ffa000cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa000cad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 -- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 msgr2=0x7ffa000cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 0x7ffa000cc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 -- 172.21.15.82:0/4148986673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000c7760 con 0x7ffa0006f4a0 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9fe7fc640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa000cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9ff7fe640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 0x7ffa000cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9feffd640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00ad040 tx=0x7ff9f001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00bf610 con 0x7ffa0006f4a0 2024-04-25T10:10:23.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f00bf7f0 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa000c7a40 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00bfbe0 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001545f0 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000c7f60 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9f00bd020 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff9f011fd80 con 0x7ffa0006f4a0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9fe7fc640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:23.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.818+0000 7ff9fe7fc640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffa000c6600 tx=0x7ff9f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:23.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.822+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa000c7f60 con 0x7ffa0006f4a0 2024-04-25T10:10:23.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.970+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa0004efa0 con 0x7ffa0006f4a0 2024-04-25T10:10:23.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.970+0000 7ff9e77fe640 1 -- 172.21.15.82:0/4148986673 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9f0016090 con 0x7ffa0006f4a0 2024-04-25T10:10:23.974 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 msgr2=0x7ff9ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 0x7ff9ec069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffa000c6600 tx=0x7ff9f4009040 comp rx=0 tx=0).stop 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 msgr2=0x7ffa001540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00ad040 tx=0x7ff9f001acb0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 shutdown_connections 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff9ec066f60 0x7ff9ec069420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa0009a180 0x7ffa000cad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa000b0a30 0x7ffa000cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 --2- 172.21.15.82:0/4148986673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa0006f4a0 0x7ffa001540b0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:23.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 >> 172.21.15.82:0/4148986673 conn(0x7ffa000b30b0 msgr2=0x7ffa000b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:23.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 shutdown_connections 2024-04-25T10:10:23.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:23.974+0000 7ffa057d9640 1 -- 172.21.15.82:0/4148986673 wait complete. 2024-04-25T10:10:23.992 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:24.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 Processor -- start 2024-04-25T10:10:24.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 -- start start 2024-04-25T10:10:24.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08180cd0d0 0x7f0818099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081809a150 con 0x7f08180c44f0 2024-04-25T10:10:24.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081809a430 con 0x7f08180b3b30 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f081e846640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08180cbc30 con 0x7f08180cd0d0 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f08177fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f08180cd0d0 0x7f0818099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f0817fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f08180c44f0 0x7f08180c48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50162/0 (socket says 172.21.15.82:50162) 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f0817fff640 1 -- 172.21.15.82:0/3366646786 learned_addr learned my addr 172.21.15.82:0/3366646786 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f08177fe640 1 -- 172.21.15.82:0/3366646786 >> v1:172.21.15.196:6789/0 conn(0x7f08180b3b30 legacy=0x7f081816b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f08177fe640 1 -- 172.21.15.82:0/3366646786 >> v1:172.21.15.82:6789/0 conn(0x7f08180c44f0 legacy=0x7f08180c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.110+0000 7f08177fe640 1 -- 172.21.15.82:0/3366646786 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081809a710 con 0x7f08180cd0d0 2024-04-25T10:10:24.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f08177fe640 1 --2- 172.21.15.82:0/3366646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180cd0d0 0x7f0818099a70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f08000097c0 tx=0x7f080001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe6bf440c941c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f0816ffd640 1 -- 172.21.15.82:0/3366646786 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000b0070 con 0x7f08180cd0d0 2024-04-25T10:10:24.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f0816ffd640 1 -- 172.21.15.82:0/3366646786 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000ab600 con 0x7f08180cd0d0 2024-04-25T10:10:24.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f0816ffd640 1 -- 172.21.15.82:0/3366646786 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000ab9d0 con 0x7f08180cd0d0 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- 172.21.15.82:0/3366646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180cd0d0 msgr2=0x7f0818099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 --2- 172.21.15.82:0/3366646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180cd0d0 0x7f0818099a70 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f08000097c0 tx=0x7f080001c260 comp rx=0 tx=0).stop 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- 172.21.15.82:0/3366646786 shutdown_connections 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 --2- 172.21.15.82:0/3366646786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180cd0d0 0x7f0818099a70 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- 172.21.15.82:0/3366646786 >> 172.21.15.82:0/3366646786 conn(0x7f08180b3010 msgr2=0x7f08180b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- 172.21.15.82:0/3366646786 shutdown_connections 2024-04-25T10:10:24.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- 172.21.15.82:0/3366646786 wait complete. 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 Processor -- start 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.114+0000 7f081e846640 1 -- start start 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08180b3b30 0x7f08180c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 0x7f081815ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081816fb20 con 0x7f08180c44f0 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f081816f9a0 con 0x7f08180cd0d0 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f081816fca0 con 0x7f08180b3b30 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36196/0 (socket says 172.21.15.82:36196) 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 -- 172.21.15.82:0/2401580898 learned_addr learned my addr 172.21.15.82:0/2401580898 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081cde3640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 0x7f081815ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0817fff640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08180b3b30 0x7f08180c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 -- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08180b3b30 msgr2=0x7f08180c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08180b3b30 0x7f08180c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 -- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 msgr2=0x7f081815ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 0x7f081815ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 -- 172.21.15.82:0/2401580898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180c5080 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081cde3640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 0x7f081815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f08177fe640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0800019e90 tx=0x7f0800019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000b8020 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08000b4610 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08180c5360 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08000b0040 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08180c7be0 con 0x7f08180c44f0 2024-04-25T10:10:24.122 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081804efa0 con 0x7f08180c44f0 2024-04-25T10:10:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f08000bb020 con 0x7f08180c44f0 2024-04-25T10:10:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 0x7f07f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f080011ef00 con 0x7f08180c44f0 2024-04-25T10:10:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0817fff640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 0x7f07f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.123 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.118+0000 7f0817fff640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 0x7f07f8069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0808007fd0 tx=0x7f0808009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.126 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.122+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08000c2030 con 0x7f08180c44f0 2024-04-25T10:10:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.282+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f081806ba80 con 0x7f08180c44f0 2024-04-25T10:10:24.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.282+0000 7f0814ff9640 1 -- 172.21.15.82:0/2401580898 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0800015fb0 con 0x7f08180c44f0 2024-04-25T10:10:24.285 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.282+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 msgr2=0x7f07f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.282+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 0x7f07f8069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0808007fd0 tx=0x7f0808009040 comp rx=0 tx=0).stop 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 msgr2=0x7f08180c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0800019e90 tx=0x7f0800019000 comp rx=0 tx=0).stop 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 shutdown_connections 2024-04-25T10:10:24.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07f8066f60 0x7f07f8069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f08180cd0d0 0x7f081815ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f08180c44f0 0x7f08180c5e40 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 --2- 172.21.15.82:0/2401580898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f08180b3b30 0x7f08180c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 >> 172.21.15.82:0/2401580898 conn(0x7f08180b3010 msgr2=0x7f08180b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 shutdown_connections 2024-04-25T10:10:24.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.286+0000 7f081e846640 1 -- 172.21.15.82:0/2401580898 wait complete. 2024-04-25T10:10:24.303 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:24.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.406+0000 7fa66c266640 1 Processor -- start 2024-04-25T10:10:24.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 -- start start 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa66415a700 0x7fa66415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6640b3f50 0x7fa6640cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 0x7fa6640b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa66415f200 con 0x7fa6640b3f50 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6640b2090 con 0x7fa6640ce430 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66c266640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640b43e0 con 0x7fa66415a700 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6640b3f50 0x7fa6640cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6640b3f50 0x7fa6640cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36206/0 (socket says 172.21.15.82:36206) 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 -- 172.21.15.82:0/2721675877 learned_addr learned my addr 172.21.15.82:0/2721675877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66a803640 1 --2- 172.21.15.82:0/2721675877 >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 0x7fa6640b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66a002640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa66415a700 0x7fa66415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 -- 172.21.15.82:0/2721675877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa66415a700 msgr2=0x7fa66415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa66415a700 0x7fa66415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 -- 172.21.15.82:0/2721675877 >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 msgr2=0x7fa6640b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 --2- 172.21.15.82:0/2721675877 >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 0x7fa6640b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 -- 172.21.15.82:0/2721675877 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6640b46c0 con 0x7fa6640b3f50 2024-04-25T10:10:24.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa66a803640 1 --2- 172.21.15.82:0/2721675877 >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 0x7fa6640b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669801640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640b3f50 0x7fa6640cdef0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa66406ca20 tx=0x7fa65801eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14ed9a5fa2e5395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669000640 1 -- 172.21.15.82:0/2721675877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580b4070 con 0x7fa6640b3f50 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669000640 1 -- 172.21.15.82:0/2721675877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6580a7a40 con 0x7fa6640b3f50 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.410+0000 7fa669000640 1 -- 172.21.15.82:0/2721675877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6580a7e10 con 0x7fa6640b3f50 2024-04-25T10:10:24.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/2721675877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640b3f50 msgr2=0x7fa6640cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640b3f50 0x7fa6640cdef0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fa66406ca20 tx=0x7fa65801eaa0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/2721675877 shutdown_connections 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- 172.21.15.82:0/2721675877 >> v2:172.21.15.196:3300/0 conn(0x7fa6640ce430 0x7fa6640b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640b3f50 0x7fa6640cdef0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- 172.21.15.82:0/2721675877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa66415a700 0x7fa66415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/2721675877 >> 172.21.15.82:0/2721675877 conn(0x7fa6640b11a0 msgr2=0x7fa6640b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/2721675877 shutdown_connections 2024-04-25T10:10:24.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/2721675877 wait complete. 2024-04-25T10:10:24.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 Processor -- start 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- start start 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa66415a700 0x7fa664158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa664156ec0 0x7fa66413fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640cd490 con 0x7fa6640ce430 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6640cd310 con 0x7fa66415a700 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa664071840 con 0x7fa664156ec0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36208/0 (socket says 172.21.15.82:36208) 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 -- 172.21.15.82:0/3616832031 learned_addr learned my addr 172.21.15.82:0/3616832031 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a803640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa664156ec0 0x7fa66413fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa669801640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa66415a700 0x7fa664158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 -- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa664156ec0 msgr2=0x7fa66413fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa664156ec0 0x7fa66413fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 -- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa66415a700 msgr2=0x7fa664158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa66415a700 0x7fa664158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 -- 172.21.15.82:0/3616832031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6641575c0 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66a002640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00c7f0 tx=0x7fa65c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c015070 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa65c004240 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6641578a0 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa65c003620 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.414+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa664157e60 con 0x7fa6640ce430 2024-04-25T10:10:24.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa66404efa0 con 0x7fa6640ce430 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa65c0039d0 con 0x7fa6640ce430 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa6627fc640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 0x7fa650069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa65c085310 con 0x7fa6640ce430 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa669801640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 0x7fa650069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.418+0000 7fa669801640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 0x7fa650069450 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa6580b1000 tx=0x7fa6580a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.422+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa65c0560d0 con 0x7fa6640ce430 2024-04-25T10:10:24.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa6640ce210 con 0x7fa6640ce430 2024-04-25T10:10:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa6627fc640 1 -- 172.21.15.82:0/3616832031 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) v1 ==== 80+0+122 (secure 0 0 0) 0x7fa65c0559d0 con 0x7fa6640ce430 2024-04-25T10:10:24.581 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:10:24.581 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:10:24.581 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 msgr2=0x7fa650069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 0x7fa650069450 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fa6580b1000 tx=0x7fa6580a9070 comp rx=0 tx=0).stop 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 msgr2=0x7fa664159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00c7f0 tx=0x7fa65c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 shutdown_connections 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa650066f90 0x7fa650069450 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa664156ec0 0x7fa66413fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.578+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa66415a700 0x7fa664158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.582+0000 7fa66c266640 1 --2- 172.21.15.82:0/3616832031 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6640ce430 0x7fa664159fd0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.582+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 >> 172.21.15.82:0/3616832031 conn(0x7fa6640b11a0 msgr2=0x7fa6640af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.582+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 shutdown_connections 2024-04-25T10:10:24.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.582+0000 7fa66c266640 1 -- 172.21.15.82:0/3616832031 wait complete. 2024-04-25T10:10:24.599 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:10:24.599 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:10:24.605 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:10:24.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.758+0000 7f75cf1ee640 1 Processor -- start 2024-04-25T10:10:24.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.758+0000 7f75cf1ee640 1 -- start start 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cf1ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 0x7f75c80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cf1ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75c81451f0 0x7f75c8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cf1ee640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c80cee30 con 0x7f75c8144ca0 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cf1ee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c80cc2b0 con 0x7f75c81451f0 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cf1ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c806e290 con 0x7f75c80ce510 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75c81451f0 0x7f75c8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f75c81451f0 0x7f75c8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34776/0 (socket says 172.21.15.82:34776) 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 -- 172.21.15.82:0/2311957363 learned_addr learned my addr 172.21.15.82:0/2311957363 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75ccf8a640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 0x7f75c80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 -- 172.21.15.82:0/2311957363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 msgr2=0x7f75c80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 0x7f75c80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 -- 172.21.15.82:0/2311957363 >> v1:172.21.15.196:6789/0 conn(0x7f75c8144ca0 legacy=0x7f75c806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 -- 172.21.15.82:0/2311957363 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80ccc70 con 0x7f75c81451f0 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75ccf8a640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 0x7f75c80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75cd78b640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c81451f0 0x7f75c8142680 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98963ff0fe5c3ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.762+0000 7f75c77fe640 1 -- 172.21.15.82:0/2311957363 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00b4070 con 0x7f75c81451f0 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c77fe640 1 -- 172.21.15.82:0/2311957363 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c00af400 con 0x7f75c81451f0 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c77fe640 1 -- 172.21.15.82:0/2311957363 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c00af7d0 con 0x7f75c81451f0 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- 172.21.15.82:0/2311957363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c81451f0 msgr2=0x7f75c8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c81451f0 0x7f75c8142680 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009e50 tx=0x7f75c001f1e0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- 172.21.15.82:0/2311957363 shutdown_connections 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c81451f0 0x7f75c8142680 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/2311957363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c80ce510 0x7f75c80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- 172.21.15.82:0/2311957363 >> 172.21.15.82:0/2311957363 conn(0x7f75c80b11a0 msgr2=0x7f75c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- 172.21.15.82:0/2311957363 shutdown_connections 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- 172.21.15.82:0/2311957363 wait complete. 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 Processor -- start 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- start start 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75c80ce510 0x7f75c80a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c81451f0 0x7f75c81e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c806ee30 con 0x7f75c8144ca0 2024-04-25T10:10:24.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c806df20 con 0x7f75c80ce510 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75cf1ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c80cc900 con 0x7f75c81451f0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34792/0 (socket says 172.21.15.82:34792) 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 -- 172.21.15.82:0/1941278891 learned_addr learned my addr 172.21.15.82:0/1941278891 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75ccf8a640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75c80ce510 0x7f75c80a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 -- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c81451f0 msgr2=0x7f75c81e33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c81451f0 0x7f75c81e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 -- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75c80ce510 msgr2=0x7f75c80a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75c80ce510 0x7f75c80a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 -- 172.21.15.82:0/1941278891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8134be0 con 0x7f75c8144ca0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c7fff640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c410 tx=0x7f75bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.766+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc011790 con 0x7f75c8144ca0 2024-04-25T10:10:24.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc017070 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc011bb0 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c8134ec0 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c8135480 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c804efa0 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75bc01c460 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75c57fa640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 0x7f75b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f75bc085160 con 0x7f75c8144ca0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75ccf8a640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 0x7f75b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.770+0000 7f75ccf8a640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 0x7f75b4069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006fd0 tx=0x7f75b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:24.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.774+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75bc055f20 con 0x7f75c8144ca0 2024-04-25T10:10:24.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.926+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75c80b1cc0 con 0x7f75c8144ca0 2024-04-25T10:10:24.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.930+0000 7f75c57fa640 1 -- 172.21.15.82:0/1941278891 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+17219 (secure 0 0 0) 0x7f75bc055820 con 0x7f75c8144ca0 2024-04-25T10:10:24.933 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:24.934 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:09:50.537791+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:10:24.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 msgr2=0x7f75b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 0x7f75b4069420 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75b8006fd0 tx=0x7f75b8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 msgr2=0x7f75c81e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c410 tx=0x7f75bc00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 shutdown_connections 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f75b4066f60 0x7f75b4069420 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f75c81451f0 0x7f75c81e33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f75c8144ca0 0x7f75c81e2e90 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 --2- 172.21.15.82:0/1941278891 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f75c80ce510 0x7f75c80a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 >> 172.21.15.82:0/1941278891 conn(0x7f75c80b11a0 msgr2=0x7f75c80c7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 shutdown_connections 2024-04-25T10:10:24.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:24.934+0000 7f75cf1ee640 1 -- 172.21.15.82:0/1941278891 wait complete. 2024-04-25T10:10:24.951 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:10:24.951 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:10:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 Processor -- start 2024-04-25T10:10:25.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 -- start start 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb97c15ca90 0x7fb97c0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c15f360 con 0x7fb97c0cc130 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb97c15f640 con 0x7fb97c06ac90 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb982810640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb97c15ff70 con 0x7fb97c15ca90 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb980dad640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb97c0cc130 0x7fb97c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58438/0 (socket says 172.21.15.82:58438) 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb980dad640 1 -- 172.21.15.82:0/2850911358 learned_addr learned my addr 172.21.15.82:0/2850911358 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97b7fe640 1 --2- 172.21.15.82:0/2850911358 >> v2:172.21.15.196:3300/0 conn(0x7fb97c15ca90 0x7fb97c0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352791931 0 0) 0x7fb97c15f360 con 0x7fb97c0cc130 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb968003530 con 0x7fb97c0cc130 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 496116685 0 0) 0x7fb968003530 con 0x7fb97c0cc130 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 >> v2:172.21.15.196:3300/0 conn(0x7fb97c15ca90 msgr2=0x7fb97c0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 --2- 172.21.15.82:0/2850911358 >> v2:172.21.15.196:3300/0 conn(0x7fb97c15ca90 0x7fb97c0d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 >> v1:172.21.15.196:6789/0 conn(0x7fb97c06ac90 legacy=0x7fb97c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.062+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c15f920 con 0x7fb97c0cc130 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb974001680 con 0x7fb97c0cc130 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 >> v1:172.21.15.82:6789/0 conn(0x7fb97c0cc130 legacy=0x7fb97c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968003a30 0x7fb968003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb968004810 0x7fb968004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 0x7fb96800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800c0d0 con 0x7fb9680051d0 2024-04-25T10:10:25.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800c280 con 0x7fb968003a30 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb96800c480 con 0x7fb968004810 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 0x7fb96800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97bfff640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968003a30 0x7fb968003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97b7fe640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb968004810 0x7fb968004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 -- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb968004810 msgr2=0x7fb968004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb968004810 0x7fb968004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 -- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968003a30 msgr2=0x7fb968003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968003a30 0x7fb968003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 -- 172.21.15.82:0/2850911358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb968003530 con 0x7fb9680051d0 2024-04-25T10:10:25.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97bfff640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb968003a30 0x7fb968003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 -- 172.21.15.82:0/2850911358 reap_dead start 2024-04-25T10:10:25.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb980dad640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 0x7fb96800b9d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06bd70 tx=0x7fb97401cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a00be03257b697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb974007d20 con 0x7fb9680051d0 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9740a9070 con 0x7fb9680051d0 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb97affd640 1 -- 172.21.15.82:0/2850911358 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9740a5d60 con 0x7fb9680051d0 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 -- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 msgr2=0x7fb96800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 0x7fb96800b9d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fb97c06bd70 tx=0x7fb97401cc20 comp rx=0 tx=0).stop 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 -- 172.21.15.82:0/2850911358 shutdown_connections 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 --2- 172.21.15.82:0/2850911358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9680051d0 0x7fb96800b9d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 -- 172.21.15.82:0/2850911358 >> 172.21.15.82:0/2850911358 conn(0x7fb97c099a70 msgr2=0x7fb97c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 -- 172.21.15.82:0/2850911358 shutdown_connections 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.066+0000 7fb982810640 1 -- 172.21.15.82:0/2850911358 wait complete. 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 Processor -- start 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- start start 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9680051d0 0x7fb97c0cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb97c0d0280 0x7fb97c0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c159750 con 0x7fb97c0cfe20 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c1595d0 con 0x7fb97c0d0280 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c06b070 con 0x7fb9680051d0 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34814/0 (socket says 172.21.15.82:34814) 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 -- 172.21.15.82:0/1979620155 learned_addr learned my addr 172.21.15.82:0/1979620155 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97bfff640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9680051d0 0x7fb97c0cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb980dad640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb97c0d0280 0x7fb97c0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 -- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9680051d0 msgr2=0x7fb97c0cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9680051d0 0x7fb97c0cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 -- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb97c0d0280 msgr2=0x7fb97c0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb97c0d0280 0x7fb97c0b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 -- 172.21.15.82:0/1979620155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c0d07f0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb97b7fe640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00f3b0 tx=0x7fb96c00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c01c070 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96c017620 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c0d0ad0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96c00fea0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c0ce270 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c0d0f80 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb96c01f070 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.070+0000 7fb978ff9640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 0x7fb95c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.074+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb96c0854e0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.074+0000 7fb97bfff640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 0x7fb95c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.074+0000 7fb97bfff640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 0x7fb95c069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c10 tx=0x7fb964009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.078+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb96c012070 con 0x7fb97c0cfe20 2024-04-25T10:10:25.225 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.222+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb97c06b7c0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.222+0000 7fb978ff9640 1 -- 172.21.15.82:0/1979620155 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb96c055bc0 con 0x7fb97c0cfe20 2024-04-25T10:10:25.226 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:10:25.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 msgr2=0x7fb95c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 0x7fb95c069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb964000c10 tx=0x7fb964009040 comp rx=0 tx=0).stop 2024-04-25T10:10:25.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 msgr2=0x7fb97c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.228 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fb96c00f3b0 tx=0x7fb96c00f870 comp rx=0 tx=0).stop 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 shutdown_connections 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb95c066f60 0x7fb95c069420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb97c0d0280 0x7fb97c0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb97c0cfe20 0x7fb97c0b2030 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 --2- 172.21.15.82:0/1979620155 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9680051d0 0x7fb97c0cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 >> 172.21.15.82:0/1979620155 conn(0x7fb97c099a70 msgr2=0x7fb97c0addc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 shutdown_connections 2024-04-25T10:10:25.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.226+0000 7fb982810640 1 -- 172.21.15.82:0/1979620155 wait complete. 2024-04-25T10:10:25.243 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:10:25.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 Processor -- start 2024-04-25T10:10:25.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 -- start start 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6ddc0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6ddc072000 0x7f6ddc072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc0cbd20 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ddc15ba90 con 0x7f6ddc072000 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6de0d7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0cff30 con 0x7f6ddc0cf590 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dda59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6ddc15b6b0 0x7f6ddc0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58448/0 (socket says 172.21.15.82:58448) 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dda59c640 1 -- 172.21.15.82:0/1935493394 learned_addr learned my addr 172.21.15.82:0/1935493394 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225615513 0 0) 0x7f6ddc0cbd20 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dc4003530 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> v2:172.21.15.196:3300/0 conn(0x7f6ddc072000 0x7f6ddc072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd9d9b640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6ddc0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591924229 0 0) 0x7f6dc4003530 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 msgr2=0x7f6ddc0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6ddc0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 >> v2:172.21.15.196:3300/0 conn(0x7f6ddc072000 msgr2=0x7f6ddc072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 --2- 172.21.15.82:0/1935493394 >> v2:172.21.15.196:3300/0 conn(0x7f6ddc072000 0x7f6ddc072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc0cc000 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6dcc001680 con 0x7f6ddc15b6b0 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> v2:172.21.15.196:3300/0 conn(0x7f6ddc072000 0x7f6ddc072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 >> v1:172.21.15.82:6789/0 conn(0x7f6ddc15b6b0 legacy=0x7f6ddc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6dc4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc4004060 0x7f6dc4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc4004d50 0x7f6dc40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc15ba90 con 0x7f6dc4004d50 2024-04-25T10:10:25.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc400bae0 con 0x7f6dc4004060 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc400bce0 con 0x7f6ddc0cf590 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc4004d50 0x7f6dc40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dd9d9b640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc4004060 0x7f6dc4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6dda59c640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6dc4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 -- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 msgr2=0x7f6dc4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6ddc0cf590 0x7f6dc4003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 -- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc4004060 msgr2=0x7f6dc4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6dc4004060 0x7f6dc4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.354+0000 7f6ddad9d640 1 -- 172.21.15.82:0/1935493394 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc4003530 con 0x7f6dc4004d50 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc4004d50 0x7f6dc40051d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40073b0 tx=0x7f6dd401cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3589ca02feff9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd40b2070 con 0x7f6dc4004d50 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd40a5e20 con 0x7f6dc4004d50 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd959a640 1 -- 172.21.15.82:0/1935493394 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd4009c80 con 0x7f6dc4004d50 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- 172.21.15.82:0/1935493394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc4004d50 msgr2=0x7f6dc40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/1935493394 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6dc4004d50 0x7f6dc40051d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f6dd40073b0 tx=0x7f6dd401cf20 comp rx=0 tx=0).stop 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6ddad9d640 1 -- 172.21.15.82:0/1935493394 reap_dead start 2024-04-25T10:10:25.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- 172.21.15.82:0/1935493394 shutdown_connections 2024-04-25T10:10:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- 172.21.15.82:0/1935493394 >> 172.21.15.82:0/1935493394 conn(0x7f6ddc06ae60 msgr2=0x7f6ddc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- 172.21.15.82:0/1935493394 shutdown_connections 2024-04-25T10:10:25.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- 172.21.15.82:0/1935493394 wait complete. 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 Processor -- start 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- start start 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc4004d50 0x7f6ddc0c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ddc06e0f0 0x7f6ddc0ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0cc8a0 con 0x7f6ddc098ad0 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0cc5c0 con 0x7f6ddc06e0f0 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6de0d7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0cba30 con 0x7f6dc4004d50 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34844/0 (socket says 172.21.15.82:34844) 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dd9d9b640 1 -- 172.21.15.82:0/50842040 learned_addr learned my addr 172.21.15.82:0/50842040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6dda59c640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc4004d50 0x7f6ddc0c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.358+0000 7f6ddad9d640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ddc06e0f0 0x7f6ddc0ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 -- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc4004d50 msgr2=0x7f6ddc0c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc4004d50 0x7f6ddc0c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 -- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ddc06e0f0 msgr2=0x7f6ddc0ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ddc06e0f0 0x7f6ddc0ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 -- 172.21.15.82:0/50842040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc06e8d0 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dd9d9b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f5c0 tx=0x7f6dd000fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0018020 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd0014870 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc0c2fa0 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0014c40 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc0c3520 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc04efa0 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6dd001b020 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 0x7f6dbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6dd0085830 con 0x7f6ddc098ad0 2024-04-25T10:10:25.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dda59c640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 0x7f6dbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.362+0000 7f6dda59c640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 0x7f6dbc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.366+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6dd0022020 con 0x7f6ddc098ad0 2024-04-25T10:10:25.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.522+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6ddc0b30a0 con 0x7f6ddc098ad0 2024-04-25T10:10:25.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.522+0000 7f6dcaffd640 1 -- 172.21.15.82:0/50842040 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dd0056680 con 0x7f6ddc098ad0 2024-04-25T10:10:25.526 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:10:25.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 msgr2=0x7f6dbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 0x7f6dbc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc000f70 tx=0x7f6dcc009040 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 msgr2=0x7f6ddc0c3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000f5c0 tx=0x7f6dd000fa80 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 shutdown_connections 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6dbc066f60 0x7f6dbc069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6ddc06e0f0 0x7f6ddc0ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6ddc098ad0 0x7f6ddc0c3d20 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 --2- 172.21.15.82:0/50842040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6dc4004d50 0x7f6ddc0c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 >> 172.21.15.82:0/50842040 conn(0x7f6ddc06ae60 msgr2=0x7f6ddc15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 shutdown_connections 2024-04-25T10:10:25.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.526+0000 7f6de0d7b640 1 -- 172.21.15.82:0/50842040 wait complete. 2024-04-25T10:10:25.544 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:10:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 Processor -- start 2024-04-25T10:10:25.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 -- start start 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8815b420 0x7efe880ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 0x7efe880cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe880cbd20 con 0x7efe880cfad0 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880cff40 con 0x7efe8815b420 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe8d264640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880cc930 con 0x7efe8815b800 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8815b420 0x7efe880ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7efe8815b420 0x7efe880ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34852/0 (socket says 172.21.15.82:34852) 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 -- 172.21.15.82:0/1452814040 learned_addr learned my addr 172.21.15.82:0/1452814040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 -- 172.21.15.82:0/1452814040 >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 msgr2=0x7efe880cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe7ffff640 1 --2- 172.21.15.82:0/1452814040 >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 0x7efe880cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 --2- 172.21.15.82:0/1452814040 >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 0x7efe880cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 -- 172.21.15.82:0/1452814040 >> v1:172.21.15.196:6789/0 conn(0x7efe880cfad0 legacy=0x7efe8815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe86ffd640 1 -- 172.21.15.82:0/1452814040 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880cc000 con 0x7efe8815b420 2024-04-25T10:10:25.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.670+0000 7efe7ffff640 1 --2- 172.21.15.82:0/1452814040 >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 0x7efe880cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:25.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe86ffd640 1 --2- 172.21.15.82:0/1452814040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe880ca3c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7efe880b4430 tx=0x7efe7801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39cf96e0f0377ea3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe867fc640 1 -- 172.21.15.82:0/1452814040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe780aa610 con 0x7efe8815b420 2024-04-25T10:10:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe867fc640 1 -- 172.21.15.82:0/1452814040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe780af070 con 0x7efe8815b420 2024-04-25T10:10:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe867fc640 1 -- 172.21.15.82:0/1452814040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe780aa9d0 con 0x7efe8815b420 2024-04-25T10:10:25.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- 172.21.15.82:0/1452814040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 msgr2=0x7efe880ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- 172.21.15.82:0/1452814040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe880ca3c0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7efe880b4430 tx=0x7efe7801c4b0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- 172.21.15.82:0/1452814040 shutdown_connections 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- 172.21.15.82:0/1452814040 >> v2:172.21.15.196:3300/0 conn(0x7efe8815b800 0x7efe880cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- 172.21.15.82:0/1452814040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe880ca3c0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- 172.21.15.82:0/1452814040 >> 172.21.15.82:0/1452814040 conn(0x7efe8806ae60 msgr2=0x7efe8806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- 172.21.15.82:0/1452814040 shutdown_connections 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- 172.21.15.82:0/1452814040 wait complete. 2024-04-25T10:10:25.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 Processor -- start 2024-04-25T10:10:25.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- start start 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 0x7efe880cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe8815b800 0x7efe8815c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880caa80 con 0x7efe8815b420 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880ca900 con 0x7efe880cfad0 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe8d264640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe880cadb0 con 0x7efe8815b800 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34868/0 (socket says 172.21.15.82:34868) 2024-04-25T10:10:25.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.674+0000 7efe7ffff640 1 -- 172.21.15.82:0/585097495 learned_addr learned my addr 172.21.15.82:0/585097495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe86ffd640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 0x7efe880cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe877fe640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe8815b800 0x7efe8815c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 -- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe8815b800 msgr2=0x7efe8815c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe8815b800 0x7efe8815c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 -- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 msgr2=0x7efe880cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 0x7efe880cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 -- 172.21.15.82:0/585097495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880ca0f0 con 0x7efe8815b420 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe86ffd640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 0x7efe880cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7ffff640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c7f0 tx=0x7efe7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70015070 con 0x7efe8815b420 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe70007820 con 0x7efe8815b420 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8815e800 con 0x7efe8815b420 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70007c10 con 0x7efe8815b420 2024-04-25T10:10:25.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe88158050 con 0x7efe8815b420 2024-04-25T10:10:25.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8804efa0 con 0x7efe8815b420 2024-04-25T10:10:25.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe70018070 con 0x7efe8815b420 2024-04-25T10:10:25.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 0x7efe68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7efe70085890 con 0x7efe8815b420 2024-04-25T10:10:25.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe86ffd640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 0x7efe68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.678+0000 7efe86ffd640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 0x7efe68069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efe780ae000 tx=0x7efe78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.682+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe700566c0 con 0x7efe8815b420 2024-04-25T10:10:25.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.838+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe880b1f20 con 0x7efe8815b420 2024-04-25T10:10:25.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe7f7fe640 1 -- 172.21.15.82:0/585097495 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe70055fc0 con 0x7efe8815b420 2024-04-25T10:10:25.843 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 msgr2=0x7efe68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 0x7efe68069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efe780ae000 tx=0x7efe78008040 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 msgr2=0x7efe8815bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7efe7000c7f0 tx=0x7efe7000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 shutdown_connections 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efe68066f60 0x7efe68069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efe8815b800 0x7efe8815c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7efe8815b420 0x7efe8815bc10 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 --2- 172.21.15.82:0/585097495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7efe880cfad0 0x7efe880cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 >> 172.21.15.82:0/585097495 conn(0x7efe8806ae60 msgr2=0x7efe8806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 shutdown_connections 2024-04-25T10:10:25.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.842+0000 7efe8d264640 1 -- 172.21.15.82:0/585097495 wait complete. 2024-04-25T10:10:25.862 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:10:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.982+0000 7f0c74128640 1 Processor -- start 2024-04-25T10:10:25.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.982+0000 7f0c74128640 1 -- start start 2024-04-25T10:10:25.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c74128640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c6c0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c74128640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c162a40 con 0x7f0c6c0d0510 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c74128640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c0d2530 con 0x7f0c6c0d1fe0 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c74128640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0d2810 con 0x7f0c6c0d0130 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0c6c0d0510 0x7f0c6c162330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58464/0 (socket says 172.21.15.82:58464) 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 -- 172.21.15.82:0/1147055403 learned_addr learned my addr 172.21.15.82:0/1147055403 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c71ec4640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c6c0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612339350 0 0) 0x7f0c6c162a40 con 0x7f0c6c0d0510 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c58003530 con 0x7f0c6c0d0510 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4140561999 0 0) 0x7f0c58003530 con 0x7f0c6c0d0510 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 msgr2=0x7f0c6c0cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c6c0cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 >> v1:172.21.15.196:6789/0 conn(0x7f0c6c0d1fe0 legacy=0x7f0c6c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0b3160 con 0x7f0c6c0d0510 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f0c64001a00 con 0x7f0c6c0d0510 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 >> v1:172.21.15.82:6789/0 conn(0x7f0c6c0d0510 legacy=0x7f0c6c162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c58003a60 0x7f0c58003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c58004da0 0x7f0c58005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0d2810 con 0x7f0c58004da0 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5800bb90 con 0x7f0c58003a60 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5800bd90 con 0x7f0c6c0d0130 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c58004da0 0x7f0c58005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c71ec4640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c58003a60 0x7f0c58003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c716c3640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 -- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 msgr2=0x7f0c58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c0d0130 0x7f0c58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 -- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c58003a60 msgr2=0x7f0c58003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:25.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c58003a60 0x7f0c58003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 -- 172.21.15.82:0/1147055403 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58003530 con 0x7f0c58004da0 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c726c5640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c58004da0 0x7f0c58005220 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400a860 tx=0x7f0c6401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da32395f45e3e469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400a480 con 0x7f0c58004da0 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c640ad650 con 0x7f0c58004da0 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.986+0000 7f0c70ec2640 1 -- 172.21.15.82:0/1147055403 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640ada40 con 0x7f0c58004da0 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1147055403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c58004da0 msgr2=0x7f0c58005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 --2- 172.21.15.82:0/1147055403 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c58004da0 0x7f0c58005220 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400a860 tx=0x7f0c6401d380 comp rx=0 tx=0).stop 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c726c5640 1 -- 172.21.15.82:0/1147055403 reap_dead start 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1147055403 shutdown_connections 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1147055403 >> 172.21.15.82:0/1147055403 conn(0x7f0c6c08b000 msgr2=0x7f0c6c0d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:25.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1147055403 shutdown_connections 2024-04-25T10:10:25.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1147055403 wait complete. 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 Processor -- start 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- start start 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c0cc200 0x7f0c6c1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c06cf60 0x7f0c6c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0b4c50 con 0x7f0c6c1f3130 2024-04-25T10:10:25.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0b3720 con 0x7f0c6c0cc200 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c0b4dd0 con 0x7f0c6c06cf60 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34900/0 (socket says 172.21.15.82:34900) 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 -- 172.21.15.82:0/1465110431 learned_addr learned my addr 172.21.15.82:0/1465110431 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c716c3640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c0cc200 0x7f0c6c1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c726c5640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c06cf60 0x7f0c6c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 -- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c06cf60 msgr2=0x7f0c6c06d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c06cf60 0x7f0c6c06d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 -- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c0cc200 msgr2=0x7f0c6c1f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c0cc200 0x7f0c6c1f3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 -- 172.21.15.82:0/1465110431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0cc680 con 0x7f0c6c1f3130 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c71ec4640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800f5b0 tx=0x7f0c6800fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68019020 con 0x7f0c6c1f3130 2024-04-25T10:10:25.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c68014870 con 0x7f0c6c1f3130 2024-04-25T10:10:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68014c60 con 0x7f0c6c1f3130 2024-04-25T10:10:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c1f3c40 con 0x7f0c6c1f3130 2024-04-25T10:10:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.990+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c1f4200 con 0x7f0c6c1f3130 2024-04-25T10:10:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c06d920 con 0x7f0c6c1f3130 2024-04-25T10:10:25.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c6801c020 con 0x7f0c6c1f3130 2024-04-25T10:10:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c627fc640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 0x7f0c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f0c68085a60 con 0x7f0c6c1f3130 2024-04-25T10:10:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c716c3640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 0x7f0c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:25.998 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.994+0000 7f0c716c3640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 0x7f0c50069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0cd1d0 tx=0x7f0c5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:26.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:25.998+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c68023020 con 0x7f0c6c1f3130 2024-04-25T10:10:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.166+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c6c04efa0 con 0x7f0c6c1f3130 2024-04-25T10:10:26.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.166+0000 7f0c627fc640 1 -- 172.21.15.82:0/1465110431 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v94) v1 ==== 80+0+122 (secure 0 0 0) 0x7f0c68026350 con 0x7f0c6c1f3130 2024-04-25T10:10:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:10:26.171 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:10:26.171 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 msgr2=0x7f0c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 0x7f0c50069420 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c0cd1d0 tx=0x7f0c5c009040 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 msgr2=0x7f0c6c1f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f0c6800f5b0 tx=0x7f0c6800fa70 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 shutdown_connections 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c50066f60 0x7f0c50069420 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c6c06cf60 0x7f0c6c06d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c6c0cc200 0x7f0c6c1f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 --2- 172.21.15.82:0/1465110431 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c6c1f3130 0x7f0c6c1f4970 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:26.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 >> 172.21.15.82:0/1465110431 conn(0x7f0c6c08b000 msgr2=0x7f0c6c0b5530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:26.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 shutdown_connections 2024-04-25T10:10:26.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:26.170+0000 7f0c74128640 1 -- 172.21.15.82:0/1465110431 wait complete. 2024-04-25T10:10:26.189 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:10:26.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:26.189 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:26.199 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[49021]: fuse finished with error 0 and tester_r 0 2024-04-25T10:10:26.201 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:10:32.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:10:32.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:32.303 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:10:32.310 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:32.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.482+0000 7f0a22f2c640 1 Processor -- start 2024-04-25T10:10:32.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.482+0000 7f0a22f2c640 1 -- start start 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0a1c15da80 0x7f0a1c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c0d1cd0 0x7f0a1c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c06c440 con 0x7f0a1c0d0620 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c06be60 con 0x7f0a1c15da80 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a22f2c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c15de60 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34914/0 (socket says 172.21.15.82:34914) 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 -- 172.21.15.82:0/4207304522 learned_addr learned my addr 172.21.15.82:0/4207304522 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a20cc8640 1 --2- 172.21.15.82:0/4207304522 >> v2:172.21.15.196:3300/0 conn(0x7f0a1c15da80 0x7f0a1c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c0d1cd0 0x7f0a1c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 -- 172.21.15.82:0/4207304522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c0d1cd0 msgr2=0x7f0a1c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c0d1cd0 0x7f0a1c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 -- 172.21.15.82:0/4207304522 >> v2:172.21.15.196:3300/0 conn(0x7f0a1c15da80 msgr2=0x7f0a1c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 --2- 172.21.15.82:0/4207304522 >> v2:172.21.15.196:3300/0 conn(0x7f0a1c15da80 0x7f0a1c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 -- 172.21.15.82:0/4207304522 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c0d2450 con 0x7f0a1c0d0620 2024-04-25T10:10:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a214c9640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c08d0a0 tx=0x7f0a1401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7b3979f60bde55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a1b7fe640 1 -- 172.21.15.82:0/4207304522 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5690 con 0x7f0a1c0d0620 2024-04-25T10:10:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a1b7fe640 1 -- 172.21.15.82:0/4207304522 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140a5830 con 0x7f0a1c0d0620 2024-04-25T10:10:32.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.486+0000 7f0a1b7fe640 1 -- 172.21.15.82:0/4207304522 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a140a5c00 con 0x7f0a1c0d0620 2024-04-25T10:10:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- 172.21.15.82:0/4207304522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d0620 msgr2=0x7f0a1c0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c08d0a0 tx=0x7f0a1401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- 172.21.15.82:0/4207304522 shutdown_connections 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c0ce3b0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/4207304522 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c0d1cd0 0x7f0a1c0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/4207304522 >> v2:172.21.15.196:3300/0 conn(0x7f0a1c15da80 0x7f0a1c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- 172.21.15.82:0/4207304522 >> 172.21.15.82:0/4207304522 conn(0x7f0a1c08b000 msgr2=0x7f0a1c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- 172.21.15.82:0/4207304522 shutdown_connections 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- 172.21.15.82:0/4207304522 wait complete. 2024-04-25T10:10:32.492 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 Processor -- start 2024-04-25T10:10:32.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- start start 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c15da80 0x7f0a1c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c06ca30 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c06c8b0 con 0x7f0a1c0d0620 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a22f2c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c06cbb0 con 0x7f0a1c15da80 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34918/0 (socket says 172.21.15.82:34918) 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1924164624 learned_addr learned my addr 172.21.15.82:0/1924164624 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:32.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a20cc8640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a214c9640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c15da80 0x7f0a1c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c15da80 msgr2=0x7f0a1c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c15da80 0x7f0a1c0c6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a1c0d0620 msgr2=0x7f0a1c15ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c15ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 -- 172.21.15.82:0/1924164624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c15f170 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a1bfff640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c820 tx=0x7f0a1000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.490+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10015070 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10004220 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c15f450 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000ce10 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c15fa10 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c04efa0 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a10007830 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a197fa640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 0x7f0a08069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a20cc8640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 0x7f0a08069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f0a10085170 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.494+0000 7f0a20cc8640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 0x7f0a08069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b930 tx=0x7f0a0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.498+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a10055ea0 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.686+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a1c0b0360 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.686+0000 7f0a197fa640 1 -- 172.21.15.82:0/1924164624 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f0a100557a0 con 0x7f0a1c0d1cd0 2024-04-25T10:10:32.689 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:32.689 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:09:23.329237+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":92,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8052},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8052":{"gid":8052,"name":"c","rank":0,"incarnation":59,"state":"up:active","state_seq":6,"addr":"172.21.15.82:6837/362560393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":362560393},{"type":"v1","addr":"172.21.15.82:6837","nonce":362560393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:32.689 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 63 2024-04-25T10:10:32.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 msgr2=0x7f0a08069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 0x7f0a08069420 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b930 tx=0x7f0a0c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:32.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 msgr2=0x7f0a1c156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c820 tx=0x7f0a1000cce0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 shutdown_connections 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0a08066f60 0x7f0a08069420 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0a1c15da80 0x7f0a1c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0a1c0d1cd0 0x7f0a1c156db0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 --2- 172.21.15.82:0/1924164624 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0a1c0d0620 0x7f0a1c15ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 >> 172.21.15.82:0/1924164624 conn(0x7f0a1c08b000 msgr2=0x7f0a1c0cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 shutdown_connections 2024-04-25T10:10:32.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.690+0000 7f0a22f2c640 1 -- 172.21.15.82:0/1924164624 wait complete. 2024-04-25T10:10:32.707 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:10:32.842 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:10:32.842 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:10:32.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:10:32.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 Processor -- start 2024-04-25T10:10:32.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 -- start start 2024-04-25T10:10:32.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 0x7fcbc4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbc40ccc70 con 0x7fcbc40ce510 2024-04-25T10:10:32.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbc40cc2b0 con 0x7fcbc406ec60 2024-04-25T10:10:32.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc9c5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc406e290 con 0x7fcbc4144ca0 2024-04-25T10:10:32.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc3fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 0x7fcbc4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:39996/0 (socket says 172.21.15.82:39996) 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/2759684719 learned_addr learned my addr 172.21.15.82:0/2759684719 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc37fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fcbc40ce510 0x7fcbc40ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:46382/0 (socket says 172.21.15.82:46382) 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/2759684719 >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 msgr2=0x7fcbc4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/2759684719 >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 0x7fcbc4142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/2759684719 >> v1:172.21.15.196:6789/0 conn(0x7fcbc40ce510 legacy=0x7fcbc40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/2759684719 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc406dc40 con 0x7fcbc4144ca0 2024-04-25T10:10:32.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.962+0000 7fcbc3fff640 1 --2- 172.21.15.82:0/2759684719 >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 0x7fcbc4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:32.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/2759684719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb401ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500d534675787e97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc27fc640 1 -- 172.21.15.82:0/2759684719 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40b5070 con 0x7fcbc4144ca0 2024-04-25T10:10:32.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc27fc640 1 -- 172.21.15.82:0/2759684719 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40a8a50 con 0x7fcbc4144ca0 2024-04-25T10:10:32.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc27fc640 1 -- 172.21.15.82:0/2759684719 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40a8e20 con 0x7fcbc4144ca0 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/2759684719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 msgr2=0x7fcbc406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/2759684719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4009870 tx=0x7fcbb401ed50 comp rx=0 tx=0).stop 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/2759684719 shutdown_connections 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/2759684719 >> v2:172.21.15.196:3300/0 conn(0x7fcbc406ec60 0x7fcbc4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/2759684719 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc4144ca0 0x7fcbc406e720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/2759684719 >> 172.21.15.82:0/2759684719 conn(0x7fcbc40b11a0 msgr2=0x7fcbc40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/2759684719 shutdown_connections 2024-04-25T10:10:32.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/2759684719 wait complete. 2024-04-25T10:10:32.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 Processor -- start 2024-04-25T10:10:32.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.966+0000 7fcbc9c5e640 1 -- start start 2024-04-25T10:10:32.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc406ec60 0x7fcbc415af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4144ca0 0x7fcbc40aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc41451f0 con 0x7fcbc40ce510 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc40cc900 con 0x7fcbc4144ca0 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4145370 con 0x7fcbc406ec60 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34922/0 (socket says 172.21.15.82:34922) 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/3748420019 learned_addr learned my addr 172.21.15.82:0/3748420019 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc406ec60 0x7fcbc415af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc3fff640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4144ca0 0x7fcbc40aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc406ec60 msgr2=0x7fcbc415af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc406ec60 0x7fcbc415af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4144ca0 msgr2=0x7fcbc40aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4144ca0 0x7fcbc40aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 -- 172.21.15.82:0/3748420019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc40af290 con 0x7fcbc40ce510 2024-04-25T10:10:32.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc2ffd640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40b3040 tx=0x7fcbb401baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40c0070 con 0x7fcbc40ce510 2024-04-25T10:10:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb40b07e0 con 0x7fcbc40ce510 2024-04-25T10:10:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc40af570 con 0x7fcbc40ce510 2024-04-25T10:10:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb40b5040 con 0x7fcbc40ce510 2024-04-25T10:10:32.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc40afb30 con 0x7fcbc40ce510 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc404efa0 con 0x7fcbc40ce510 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbb40c3070 con 0x7fcbc40ce510 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.970+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcbb4121fe0 con 0x7fcbc40ce510 2024-04-25T10:10:32.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.974+0000 7fcbc37fe640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8000f70 tx=0x7fcbb8007cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:32.974+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbb40188b0 con 0x7fcbc40ce510 2024-04-25T10:10:33.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.170+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcbc406c030 con 0x7fcbc40ce510 2024-04-25T10:10:33.173 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:10:33.170+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:10:33.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.174+0000 7fcbabfff640 1 -- 172.21.15.82:0/3748420019 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcbb40f2da0 con 0x7fcbc40ce510 2024-04-25T10:10:33.179 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 msgr2=0x7fcbb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8000f70 tx=0x7fcbb8007cd0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 msgr2=0x7fcbc40b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcbb40b3040 tx=0x7fcbb401baf0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 shutdown_connections 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbb0066f60 0x7fcbb0069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbc4144ca0 0x7fcbc40aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbc40ce510 0x7fcbc40b0330 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 --2- 172.21.15.82:0/3748420019 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbc406ec60 0x7fcbc415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 >> 172.21.15.82:0/3748420019 conn(0x7fcbc40b11a0 msgr2=0x7fcbc40cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 shutdown_connections 2024-04-25T10:10:33.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.178+0000 7fcbc9c5e640 1 -- 172.21.15.82:0/3748420019 wait complete. 2024-04-25T10:10:33.187 INFO:tasks.ceph.mds.c.smithi082.stderr: -4775> 2024-04-25T10:09:18.320+0000 7f33f5da6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:33.187 INFO:tasks.ceph.mds.c.smithi082.stderr: -4388> 2024-04-25T10:09:20.536+0000 7f33f1d9e640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T10:10:33.188 INFO:tasks.ceph.mds.c.smithi082.stderr: -4226> 2024-04-25T10:09:20.756+0000 7f33f1d9e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0) n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x56527a957b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T10:10:33.190 INFO:tasks.ceph.mds.c.smithi082.stderr: -2491> 2024-04-25T10:09:52.159+0000 7f33f1d9e640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)->f(v1 m2024-04-25T10:09:01.778099+0000) n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)->n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x56527a957b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T10:09:01.778099+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T10:09:01.886871+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T10:10:33.201 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:10:33.220 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:10:33.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- start start 2024-04-25T10:10:33.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 Processor -- start 2024-04-25T10:10:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504eabf800 0x56504ec78580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 0x56504ec79080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba180 con 0x56504eabf400 2024-04-25T10:10:33.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba300 con 0x56504eabf000 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba480 con 0x56504eabf800 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34966/0 (socket says 172.21.15.82:34966) 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 -- 172.21.15.82:0/153422980 learned_addr learned my addr 172.21.15.82:0/153422980 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f9715eb8640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504eabf800 0x56504ec78580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97166b9640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 0x56504ec79080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 -- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504eabf800 msgr2=0x56504ec78580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504eabf800 0x56504ec78580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 -- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 msgr2=0x56504ec79080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 0x56504ec79080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 -- 172.21.15.82:0/153422980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56504eae41e0 con 0x56504eabf400 2024-04-25T10:10:33.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97166b9640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 0x56504ec79080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97156b7640 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x56504f9050e0 tx=0x56504f882000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407fc4cf06a61998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97136b3640 1 -- 172.21.15.82:0/153422980 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf49c0 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97136b3640 1 -- 172.21.15.82:0/153422980 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56504eb08e00 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97136b3640 1 -- 172.21.15.82:0/153422980 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf44e0 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56504eae52c0 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56504eae43c0 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 4 waiting for MDS map... 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97136b3640 1 -- 172.21.15.82:0/153422980 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x56504f9cc200 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97136b3640 1 -- 172.21.15.82:0/153422980 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 64) v1 ==== 2723+0+0 (secure 0 0 0) 0x56504eb15180 con 0x56504eabf400 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 4 Got MDS map 64 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 4 datascan.main: Using metadata pool output 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:10:33.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 Processor -- start 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.234+0000 7f97176f98c0 1 -- start start 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504f895800 0x56504ec79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504fa3a000 0x56504ec7a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba600 con 0x56504f895c00 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba780 con 0x56504f895800 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56504f9ba900 con 0x56504fa3a000 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34982/0 (socket says 172.21.15.82:34982) 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/885091722 learned_addr learned my addr 172.21.15.82:0/885091722 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504fa3a000 msgr2=0x56504ec7a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504fa3a000 0x56504ec7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504f895800 msgr2=0x56504ec79600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504f895800 0x56504ec79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/885091722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56504eae41e0 con 0x56504f895c00 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x56504f9cac00 tx=0x56504f9e8cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b30b9d9cb43e04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97116af640 1 -- 172.21.15.82:0/885091722 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf49c0 con 0x56504f895c00 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97116af640 1 -- 172.21.15.82:0/885091722 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56504eb08a80 con 0x56504f895c00 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97116af640 1 -- 172.21.15.82:0/885091722 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf44e0 con 0x56504f895c00 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/885091722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 msgr2=0x56504ec79b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x56504f9cac00 tx=0x56504f9e8cc0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/885091722 shutdown_connections 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504fa3a000 0x56504ec7a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895c00 0x56504ec79b80 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- 172.21.15.82:0/885091722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504f895800 0x56504ec79600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/885091722 >> 172.21.15.82:0/885091722 conn(0x56504eabec00 msgr2=0x56504eaed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/885091722 shutdown_connections 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/885091722 wait complete. 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 Processor -- start 2024-04-25T10:10:33.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- start start 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504f895c00 0x56504ec7b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504fa3b800 0x56504ec7a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9bbc80 con 0x56504f895800 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56504f9bbb00 con 0x56504fa3b800 2024-04-25T10:10:33.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56504f9bb980 con 0x56504f895c00 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34996/0 (socket says 172.21.15.82:34996) 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 learned_addr learned my addr 172.21.15.82:0/39430933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504f895c00 msgr2=0x56504ec7b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504f895c00 0x56504ec7b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504fa3b800 msgr2=0x56504ec7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504fa3b800 0x56504ec7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56504eae54a0 con 0x56504f895800 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x56504f9cb8c0 tx=0x56504f9e8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f970feac640 1 -- 172.21.15.82:0/39430933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf4000 con 0x56504f895800 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f970feac640 1 -- 172.21.15.82:0/39430933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56504eb09340 con 0x56504f895800 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56504eae5a40 con 0x56504f895800 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f970feac640 1 -- 172.21.15.82:0/39430933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56504eaf49c0 con 0x56504f895800 2024-04-25T10:10:33.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56504eae5c20 con 0x56504f895800 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.238+0000 7f97176f98c0 4 datascan.init: resolving metadata pool 13 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f970feac640 1 -- 172.21.15.82:0/39430933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x56504fa46c00 con 0x56504f895800 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f970feac640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56504fa44800 0x56504ec7b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f970feac640 1 -- 172.21.15.82:0/39430933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x56504fa3d400 con 0x56504f895800 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97176f98c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97176f98c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97176f98c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x56504fa3b400 0x56504ec7a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3b000 con 0x56504fa3b400 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56504fa44800 0x56504ec7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x56504fa3b400 0x56504ec7a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x56504fa3b400 0x56504ec7a100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56504fa44800 0x56504ec7b700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56504eb9b080 tx=0x56504eed40c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f9710eae640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x56504fa45800 0x56504eee6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa45c00 con 0x56504fa45800 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x56504fa45800 0x56504eee6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x56504fa45800 0x56504eee6000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97116af640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x56504eef4000 0x56504eee6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.242+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4400 con 0x56504eef4000 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x56504eef4000 0x56504eee6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x56504eef4000 0x56504eee6580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504fa45800 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4800 con 0x56504fa45800 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9710eae640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x56504eef5000 0x56504eee6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5400 con 0x56504eef5000 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x56504eef5000 0x56504eee6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.246+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x56504eef5000 0x56504eee6b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504eef4000 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4800 con 0x56504eef4000 2024-04-25T10:10:33.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(8 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97116af640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56504fa45400 0x56504eee7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa45c00 con 0x56504fa45400 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56504fa45400 0x56504eee7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56504fa45400 0x56504eee7600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45400 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5800 con 0x56504eef4000 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97116af640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x56504eef4400 0x56504eee7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.250+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504eef4400 2024-04-25T10:10:33.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x56504eef4400 0x56504eee7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x56504eef4400 0x56504eee7b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(11 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5c00 con 0x56504fa3b400 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4800 con 0x56504fa45800 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(13 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08000 con 0x56504eef4000 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5800 con 0x56504fa45800 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(15 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08400 con 0x56504eef4000 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09400 con 0x56504fa45400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08800 con 0x56504eef4400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.254+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09800 con 0x56504fa3b400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08c00 con 0x56504eef4400 2024-04-25T10:10:33.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f97116af640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x56504ef09c00 0x56504eee8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08000 con 0x56504ef09c00 2024-04-25T10:10:33.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x56504ef09c00 0x56504eee8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.258+0000 7f97156b7640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x56504ef09c00 0x56504eee8100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09000 con 0x56504eef4400 2024-04-25T10:10:33.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504fa45800 2024-04-25T10:10:33.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(23 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09400 con 0x56504fa45800 2024-04-25T10:10:33.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(24 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5400 con 0x56504fa45800 2024-04-25T10:10:33.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.262+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08400 con 0x56504eef4400 2024-04-25T10:10:33.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5c00 con 0x56504eef5000 2024-04-25T10:10:33.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(27 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4800 con 0x56504fa3b400 2024-04-25T10:10:33.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(28 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3b000 con 0x56504eef4400 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5800 con 0x56504eef5000 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97116af640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56504f895400 0x56504eee8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08800 con 0x56504f895400 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56504f895400 0x56504eee8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.266+0000 7f97166b9640 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56504f895400 0x56504eee8680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.270+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(31 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504f895400 2024-04-25T10:10:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.270+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504ef09c00 2024-04-25T10:10:33.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(32 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08c00 con 0x56504eef5000 2024-04-25T10:10:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(33 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa45c00 con 0x56504ef09c00 2024-04-25T10:10:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(34 [pgnls start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09800 con 0x56504eef4400 2024-04-25T10:10:33.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 7 ==== osd_op_reply(35 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08000 con 0x56504eef4000 2024-04-25T10:10:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(36 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1c000 con 0x56504ef09c00 2024-04-25T10:10:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(37 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08c00 con 0x56504fa45400 2024-04-25T10:10:33.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(38 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45400 2024-04-25T10:10:33.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1c400 con 0x56504fa45400 2024-04-25T10:10:33.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(39 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45400 2024-04-25T10:10:33.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.274+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5c00 con 0x56504f895400 2024-04-25T10:10:33.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(40 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504f895400 2024-04-25T10:10:33.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1c800 con 0x56504fa45800 2024-04-25T10:10:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 9 ==== osd_op_reply(41 [pgnls start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4800 con 0x56504fa3b400 2024-04-25T10:10:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 5 ==== osd_op_reply(42 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1cc00 con 0x56504f895400 2024-04-25T10:10:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(43 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504f895400 2024-04-25T10:10:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5800 con 0x56504eef4400 2024-04-25T10:10:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 8 ==== osd_op_reply(44 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1d000 con 0x56504fa45400 2024-04-25T10:10:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.6 v2:172.21.15.196:6801/1119498737 5 ==== osd_op_reply(45 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45400 2024-04-25T10:10:33.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef4c00 con 0x56504eef4400 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 9 ==== osd_op_reply(46 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1d400 con 0x56504fa45800 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 10 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.278+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09000 con 0x56504eef5000 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 5 ==== osd_op_reply(48 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1d800 con 0x56504fa45800 2024-04-25T10:10:33.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 11 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08400 con 0x56504f895400 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504f895400 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1dc00 con 0x56504fa3b400 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.2 v2:172.21.15.82:6803/3086763100 6 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504fa3b400 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09800 con 0x56504ef09c00 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(52 [pgnls start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5800 con 0x56504fa45800 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 12 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08000 con 0x56504eef4400 2024-04-25T10:10:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 10 ==== osd_op_reply(54 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5400 con 0x56504eef4000 2024-04-25T10:10:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09400 con 0x56504eef4400 2024-04-25T10:10:33.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 11 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.282+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504eef5c00 con 0x56504ef09c00 2024-04-25T10:10:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v91'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3ac00 con 0x56504ef09c00 2024-04-25T10:10:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97176f98c0 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T10:10:33.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3a800 con 0x56504ef09c00 2024-04-25T10:10:33.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(59 10000000001.00000000 [call] v95'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.286+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3a400 con 0x56504ef09c00 2024-04-25T10:10:33.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.5 v2:172.21.15.196:6800/1745719129 9 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504ef09c00 2024-04-25T10:10:33.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef1c400 con 0x56504eef4000 2024-04-25T10:10:33.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.1 v2:172.21.15.82:6816/1010291016 9 ==== osd_op_reply(61 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4000 2024-04-25T10:10:33.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08800 con 0x56504eef4400 2024-04-25T10:10:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 <== osd.4 v2:172.21.15.196:6816/2164040014 12 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce780 con 0x56504eef4400 2024-04-25T10:10:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09000 con 0x56504f895400 2024-04-25T10:10:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504f895400 2024-04-25T10:10:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef08c00 con 0x56504fa45800 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97116af640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504ef09800 con 0x56504fa45800 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97166b9640 1 -- 172.21.15.82:0/39430933 <== osd.0 v2:172.21.15.82:6800/2082960483 14 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce500 con 0x56504fa45800 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9710eae640 1 -- 172.21.15.82:0/39430933 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x56504fa3b000 con 0x56504eef5000 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97156b7640 1 -- 172.21.15.82:0/39430933 <== osd.3 v2:172.21.15.82:6824/3269667522 6 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56504ebce000 con 0x56504eef5000 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x56504fa45800 msgr2=0x56504eee6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x56504fa45800 0x56504eee6000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x56504eef4000 msgr2=0x56504eee6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x56504eef4000 0x56504eee6580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x56504fa3b400 msgr2=0x56504ec7a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x56504fa3b400 0x56504ec7a100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x56504eef5000 msgr2=0x56504eee6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x56504eef5000 0x56504eee6b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 reap_dead start 2024-04-25T10:10:33.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x56504eef4400 msgr2=0x56504eee7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x56504eef4400 0x56504eee7b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x56504ef09c00 msgr2=0x56504eee8100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x56504ef09c00 0x56504eee8100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56504fa45400 msgr2=0x56504eee7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56504fa45400 0x56504eee7600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56504f895400 msgr2=0x56504eee8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56504f895400 0x56504eee8680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56504fa44800 msgr2=0x56504ec7b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56504fa44800 0x56504ec7b700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56504eb9b080 tx=0x56504eed40c0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 msgr2=0x56504ec7ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f9715eb8640 1 -- 172.21.15.82:0/39430933 reap_dead start 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x56504f9cb8c0 tx=0x56504f9e8c00 comp rx=0 tx=0).stop 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 shutdown_connections 2024-04-25T10:10:33.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/39430933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504f895800 0x56504ec7ac00 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 >> 172.21.15.82:0/39430933 conn(0x56504eabec00 msgr2=0x56504eaed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 shutdown_connections 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/39430933 wait complete. 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 msgr2=0x56504ec78b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x56504f9050e0 tx=0x56504f882000 comp rx=0 tx=0).stop 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 shutdown_connections 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56504eabf000 0x56504ec79080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56504eabf400 0x56504ec78b00 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 --2- 172.21.15.82:0/153422980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56504eabf800 0x56504ec78580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 >> 172.21.15.82:0/153422980 conn(0x56504eabfc00 msgr2=0x56504eaed000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 shutdown_connections 2024-04-25T10:10:33.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.290+0000 7f97176f98c0 1 -- 172.21.15.82:0/153422980 wait complete. 2024-04-25T10:10:33.299 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.098317, result: 2024-04-25T10:10:33.299 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T10:10:33.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- start start 2024-04-25T10:10:33.375 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 Processor -- start 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f4ad1400 0x5601f4c8ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f4ad1000 0x5601f4c8b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc180 con 0x5601f4ad1800 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc300 con 0x5601f4ad1400 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc480 con 0x5601f4ad1000 2024-04-25T10:10:33.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35018/0 (socket says 172.21.15.82:35018) 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3359555653 learned_addr learned my addr 172.21.15.82:0/3359555653 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f4ad1000 msgr2=0x5601f4c8b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f4ad1000 0x5601f4c8b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f4ad1400 msgr2=0x5601f4c8ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f4ad1400 0x5601f4c8ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f4ad1400 0x5601f4c8ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3359555653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f4af61e0 con 0x5601f4ad1800 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x5601f59110e0 tx=0x5601f5894000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818e9cfbed9063a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dcfb0640 1 -- 172.21.15.82:0/3359555653 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b069c0 con 0x5601f4ad1800 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dcfb0640 1 -- 172.21.15.82:0/3359555653 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601f4b1ae00 con 0x5601f4ad1800 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dcfb0640 1 -- 172.21.15.82:0/3359555653 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b064e0 con 0x5601f4ad1800 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601f4af72c0 con 0x5601f4ad1800 2024-04-25T10:10:33.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5601f4af63c0 con 0x5601f4ad1800 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 4 waiting for MDS map... 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dcfb0640 1 -- 172.21.15.82:0/3359555653 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x5601f59da000 con 0x5601f4ad1800 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dcfb0640 1 -- 172.21.15.82:0/3359555653 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 64) v1 ==== 2723+0+0 (secure 0 0 0) 0x5601f4b27180 con 0x5601f4ad1800 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 4 Got MDS map 64 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 4 datascan.main: Using metadata pool output 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 Processor -- start 2024-04-25T10:10:33.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- start start 2024-04-25T10:10:33.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f58a1800 0x5601f4c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc600 con 0x5601f5a4a000 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc780 con 0x5601f58a1c00 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cc900 con 0x5601f58a1800 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35032/0 (socket says 172.21.15.82:35032) 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 -- 172.21.15.82:0/970395329 learned_addr learned my addr 172.21.15.82:0/970395329 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 -- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f58a1800 msgr2=0x5601f4c8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f58a1800 0x5601f4c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 -- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 msgr2=0x5601f4c8bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 -- 172.21.15.82:0/970395329 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f4af61e0 con 0x5601f5a4a000 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2defb4640 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x5601f59eac00 tx=0x5601f59f6cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84763753c87e0c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dafac640 1 -- 172.21.15.82:0/970395329 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b069c0 con 0x5601f5a4a000 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dafac640 1 -- 172.21.15.82:0/970395329 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601f4b1aa80 con 0x5601f5a4a000 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.374+0000 7fb2dafac640 1 -- 172.21.15.82:0/970395329 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b064e0 con 0x5601f5a4a000 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/970395329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 msgr2=0x5601f4c8c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x5601f59eac00 tx=0x5601f59f6cc0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/970395329 shutdown_connections 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f5a4a000 0x5601f4c8c100 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/970395329 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f58a1800 0x5601f4c8b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/970395329 >> 172.21.15.82:0/970395329 conn(0x5601f4ad0c00 msgr2=0x5601f4aff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/970395329 shutdown_connections 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/970395329 wait complete. 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 Processor -- start 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- start start 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f5a4b800 0x5601f4c8c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cdc80 con 0x5601f58a1800 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cdb00 con 0x5601f58a1c00 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f59cd980 con 0x5601f5a4b800 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35040/0 (socket says 172.21.15.82:35040) 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 learned_addr learned my addr 172.21.15.82:0/3060491413 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f5a4b800 0x5601f4c8c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f5a4b800 msgr2=0x5601f4c8c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f5a4b800 0x5601f4c8c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 msgr2=0x5601f4c8d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f58a1c00 0x5601f4c8d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601f4af74a0 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x5601f59eb860 tx=0x5601f5894300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2d97a9640 1 -- 172.21.15.82:0/3060491413 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b06b60 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2d97a9640 1 -- 172.21.15.82:0/3060491413 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601f4b1ac40 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5601f4af7a40 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2d97a9640 1 -- 172.21.15.82:0/3060491413 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5601f4b06340 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601f4af7c20 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2e0ff68c0 4 datascan.init: resolving metadata pool 13 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.378+0000 7fb2d97a9640 1 -- 172.21.15.82:0/3060491413 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5601f4dfac00 con 0x5601f58a1800 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2d97a9640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 0x5601f4c8d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2d97a9640 1 -- 172.21.15.82:0/3060491413 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x5601f5a59000 con 0x5601f58a1800 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 0x5601f4c8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5601f5a4b400 0x5601f4c8c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4b000 con 0x5601f5a4b400 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5601f5a4b400 0x5601f4c8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 0x5601f4c8d700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5601f4bad080 tx=0x5601f4ede480 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5601f5a4b400 0x5601f4c8c100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 0x5601f4c8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a800 con 0x5601f5a4ac00 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 0x5601f4c8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 0x5601f4c8bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5601f5a4a400 0x5601f4c8b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f5a4a400 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5601f5a4a400 0x5601f4c8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5601f5a4a400 0x5601f4c8b600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a52400 con 0x5601f5a4b400 2024-04-25T10:10:33.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dafac640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5601f5a52000 0x5601f4f00000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4bc00 con 0x5601f5a52000 2024-04-25T10:10:33.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.382+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5601f5a52000 0x5601f4f00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5601f5a52000 0x5601f4f00000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4b400 2024-04-25T10:10:33.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4b000 con 0x5601f5a4b400 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2da7ab640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5601f4eeac00 0x5601f4f00580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb000 con 0x5601f4eeac00 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5601f4eeac00 0x5601f4f00580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5601f4eeac00 0x5601f4f00580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f5a52000 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb400 con 0x5601f5a52000 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4ac00 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb800 con 0x5601f5a52000 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dafac640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5601f5a4bc00 0x5601f4f00b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4b000 con 0x5601f5a4bc00 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.386+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5601f5a4bc00 0x5601f4f00b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2df7b5640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5601f5a4bc00 0x5601f4f00b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f5a4a400 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a52400 con 0x5601f5a4b400 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb000 con 0x5601f5a52000 2024-04-25T10:10:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f5a4b400 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a52000 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb400 con 0x5601f5a4ac00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a800 con 0x5601f5a4bc00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb800 con 0x5601f5a4a400 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.390+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f5a4bc00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2dafac640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5601f4eeb000 0x5601f4f01080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f4eeb000 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5601f4eeb000 0x5601f4f01080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5601f4eeb000 0x5601f4f01080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4b000 con 0x5601f5a4bc00 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4b400 2024-04-25T10:10:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f5a4b400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a52400 con 0x5601f5a4b400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.394+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53400 con 0x5601f5a4bc00 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a800 con 0x5601f4eeac00 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53000 con 0x5601f5a4a400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f5a4bc00 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1e000 con 0x5601f4eeac00 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dafac640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 0x5601f4f01600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c800 con 0x5601f4f1c400 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 0x5601f4f01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dffb6640 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 0x5601f4f01600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.398+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1e400 con 0x5601f4eeb000 2024-04-25T10:10:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1cc00 con 0x5601f4eeac00 2024-04-25T10:10:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1e800 con 0x5601f4eeb000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d000 con 0x5601f5a4bc00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1ec00 con 0x5601f5a52000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d400 con 0x5601f4eeb000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1f000 con 0x5601f5a4ac00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d800 con 0x5601f5a4ac00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.402+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1f400 con 0x5601f4f1c400 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1dc00 con 0x5601f5a4b400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f5a4a400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1e400 con 0x5601f4f1c400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4bc00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f5a4ac00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.406+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f5a4bc00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d000 con 0x5601f5a4b400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb800 con 0x5601f4eeac00 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c800 con 0x5601f5a4b400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a52400 con 0x5601f4f1c400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1cc00 con 0x5601f5a4a400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53400 con 0x5601f4eeb000 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(54 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f5a4b400 2024-04-25T10:10:33.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53000 con 0x5601f5a4bc00 2024-04-25T10:10:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4b000 con 0x5601f5a52000 2024-04-25T10:10:33.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(57 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.410+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb800 con 0x5601f5a4bc00 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a800 con 0x5601f4eeb000 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f5a4a400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb400 con 0x5601f5a4b400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d800 con 0x5601f5a52000 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30000 con 0x5601f5a4b400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f5a4b400 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30400 con 0x5601f4eeac00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32000 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30800 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32400 con 0x5601f5a4ac00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.414+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30c00 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32800 con 0x5601f5a4bc00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31000 con 0x5601f5a4a400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32c00 con 0x5601f5a4b400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31400 con 0x5601f5a52000 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 13 ==== osd_op_reply(73 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33000 con 0x5601f5a4b400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31800 con 0x5601f5a52000 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33400 con 0x5601f5a4ac00 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31c00 con 0x5601f5a4bc00 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33800 con 0x5601f5a4a400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.418+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb400 con 0x5601f5a4bc00 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33c00 con 0x5601f4eeb000 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(80 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f5a4bc00 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32000 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32c00 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53000 con 0x5601f5a4bc00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32800 con 0x5601f4eeac00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d800 con 0x5601f5a4a400 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f5a4bc00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.422+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f4eeac00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f4f1c400 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1cc00 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(91 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb800 con 0x5601f4eeac00 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30000 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 9 ==== osd_op_reply(93 [pgnls-filter start_epoch 95 out=49b] v91'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f5a4bc00 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30800 con 0x5601f5a52000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 15 ==== osd_op_reply(95 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.426+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30c00 con 0x5601f5a4ac00 2024-04-25T10:10:33.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32400 con 0x5601f5a4ac00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31400 con 0x5601f4f1c400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33400 con 0x5601f5a4b400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 95 out=49b] v91'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1cc00 con 0x5601f5a4a400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32800 con 0x5601f4f1c400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.430+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53000 con 0x5601f5a4bc00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30400 con 0x5601f5a4ac00 2024-04-25T10:10:33.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33000 con 0x5601f5a4bc00 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a52400 con 0x5601f5a4b400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33c00 con 0x5601f4eeac00 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea800 con 0x5601f5a4b400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32000 con 0x5601f4f1c400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 95 out=49b] v91'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eeb400 con 0x5601f5a4a400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.2 v2:172.21.15.82:6803/3086763100 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 95 out=49b] v91'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4a400 2024-04-25T10:10:33.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f32c00 con 0x5601f4eeb000 2024-04-25T10:10:33.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 11 ==== osd_op_reply(111 [pgnls-filter start_epoch 95 out=49b] v91'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eebc00 con 0x5601f5a4b400 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.434+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33800 con 0x5601f5a4bc00 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4eea400 con 0x5601f5a52000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 16 ==== osd_op_reply(114 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31800 con 0x5601f5a4bc00 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 95 out=49b] v91'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a000 con 0x5601f4eeb000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 95 out=81b] v95'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 10 datascan.operator(): handling object 10000000001.0 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a4a800 con 0x5601f4eeb000 2024-04-25T10:10:33.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f5a53400 con 0x5601f5a52000 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.438+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d400 con 0x5601f5a4ac00 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.6 v2:172.21.15.196:6801/1119498737 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x5601f4be0500 con 0x5601f5a4ac00 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.get_frag_of: frag is * 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1dc00 con 0x5601f5a52000 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d000 con 0x5601f5a52000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.read_dentry: file_head not found in result 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c800 con 0x5601f5a52000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v95'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30c00 con 0x5601f4eeb000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.5 v2:172.21.15.196:6800/1745719129 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 95 out=49b] v91'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f4eeb000 2024-04-25T10:10:33.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1c000 con 0x5601f5a52000 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.1 v2:172.21.15.82:6816/1010291016 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a52000 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.442+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30000 con 0x5601f5a4bc00 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2df7b5640 1 -- 172.21.15.82:0/3060491413 <== osd.4 v2:172.21.15.196:6816/2164040014 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 95 out=49b] v91'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0780 con 0x5601f5a4bc00 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f1d800 con 0x5601f4f1c400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.7 v2:172.21.15.196:6822/2552340661 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 95 out=49b] v91'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4f1c400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f30800 con 0x5601f5a4b400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2da7ab640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f33400 con 0x5601f5a4b400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 <== osd.0 v2:172.21.15.82:6800/2082960483 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 95 out=49b] v91'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0000 con 0x5601f5a4b400 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2dafac640 1 -- 172.21.15.82:0/3060491413 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 95 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x5601f4f31000 con 0x5601f4eeac00 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2dffb6640 1 -- 172.21.15.82:0/3060491413 <== osd.3 v2:172.21.15.82:6824/3269667522 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 95 out=49b] v91'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5601f4be0500 con 0x5601f4eeac00 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5601f5a4b400 msgr2=0x5601f4c8c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5601f5a4b400 0x5601f4c8c100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5601f5a52000 msgr2=0x5601f4f00000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5601f5a52000 0x5601f4f00000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5601f5a4a400 msgr2=0x5601f4c8b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5601f5a4a400 0x5601f4c8b600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5601f4eeac00 msgr2=0x5601f4f00580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5601f4eeac00 0x5601f4f00580 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5601f5a4bc00 msgr2=0x5601f4f00b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5601f5a4bc00 0x5601f4f00b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5601f4eeb000 msgr2=0x5601f4f01080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5601f4eeb000 0x5601f4f01080 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 reap_dead start 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 msgr2=0x5601f4c8bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 0x5601f4c8bb80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 reap_dead start 2024-04-25T10:10:33.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2defb4640 1 -- 172.21.15.82:0/3060491413 reap_dead start 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 msgr2=0x5601f4f01600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 0x5601f4f01600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 msgr2=0x5601f4c8d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 0x5601f4c8d700 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x5601f4bad080 tx=0x5601f4ede480 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 msgr2=0x5601f4c8cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x5601f59eb860 tx=0x5601f5894300 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 shutdown_connections 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5601f5a53800 0x5601f4c8d700 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5601f4f1c400 0x5601f4f01600 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5601f5a4ac00 0x5601f4c8bb80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3060491413 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f58a1800 0x5601f4c8cc00 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 >> 172.21.15.82:0/3060491413 conn(0x5601f4ad0c00 msgr2=0x5601f4aff800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 shutdown_connections 2024-04-25T10:10:33.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3060491413 wait complete. 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 msgr2=0x5601f4c8a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x5601f59110e0 tx=0x5601f5894000 comp rx=0 tx=0).stop 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 shutdown_connections 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5601f4ad1000 0x5601f4c8b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5601f4ad1400 0x5601f4c8ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 --2- 172.21.15.82:0/3359555653 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5601f4ad1800 0x5601f4c8a580 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 >> 172.21.15.82:0/3359555653 conn(0x5601f4ad1c00 msgr2=0x5601f4aff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 shutdown_connections 2024-04-25T10:10:33.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.446+0000 7fb2e0ff68c0 1 -- 172.21.15.82:0/3359555653 wait complete. 2024-04-25T10:10:33.459 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.159572, result: 2024-04-25T10:10:33.459 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:10:33.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.526+0000 7fcd6f0b48c0 1 -- start start 2024-04-25T10:10:33.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.526+0000 7fcd6f0b48c0 1 Processor -- start 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a48b00400 0x563a48c27b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a48b01800 0x563a48c26b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48b02480 con 0x563a48b01c00 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48b02600 con 0x563a48b00400 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a48b02780 con 0x563a48b01800 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35066/0 (socket says 172.21.15.82:35066) 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/2679959945 learned_addr learned my addr 172.21.15.82:0/2679959945 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d873640 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a48b00400 0x563a48c27b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a48b01800 msgr2=0x563a48c26b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a48b01800 0x563a48c26b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a48b00400 msgr2=0x563a48c27b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a48b00400 0x563a48c27b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/2679959945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a48a921e0 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x563a498bfb30 tx=0x563a48b04540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16696d15ed4c2300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6b06e640 1 -- 172.21.15.82:0/2679959945 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa29c0 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6b06e640 1 -- 172.21.15.82:0/2679959945 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a48ab61c0 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6b06e640 1 -- 172.21.15.82:0/2679959945 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa2b60 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a48a932c0 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563a48a93e00 con 0x563a48b01c00 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 4 waiting for MDS map... 2024-04-25T10:10:33.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6b06e640 1 -- 172.21.15.82:0/2679959945 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x563a4998e400 con 0x563a48b01c00 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6b06e640 1 -- 172.21.15.82:0/2679959945 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 64) v1 ==== 2723+0+0 (secure 0 0 0) 0x563a48ac3180 con 0x563a48b01c00 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 4 Got MDS map 64 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 4 datascan.main: Using metadata pool output 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 Processor -- start 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- start start 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48acc180 con 0x563a49993400 2024-04-25T10:10:33.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48b02f00 con 0x563a49992c00 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a48b03080 con 0x563a49993000 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35076/0 (socket says 172.21.15.82:35076) 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/1431654234 learned_addr learned my addr 172.21.15.82:0/1431654234 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993000 msgr2=0x563a48c28680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 msgr2=0x563a48c28100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6e074640 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 -- 172.21.15.82:0/1431654234 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a48a921e0 con 0x563a49993400 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6d072640 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x563a499aaf60 tx=0x563a48b041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b1a5abcf2349bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6906a640 1 -- 172.21.15.82:0/1431654234 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa2340 con 0x563a49993400 2024-04-25T10:10:33.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6906a640 1 -- 172.21.15.82:0/1431654234 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a48ab6e00 con 0x563a49993400 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6906a640 1 -- 172.21.15.82:0/1431654234 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa29c0 con 0x563a49993400 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/1431654234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 msgr2=0x563a48c28c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x563a499aaf60 tx=0x563a48b041e0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/1431654234 shutdown_connections 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/1431654234 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/1431654234 >> 172.21.15.82:0/1431654234 conn(0x563a48b01000 msgr2=0x563a48d9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/1431654234 shutdown_connections 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.530+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/1431654234 wait complete. 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 Processor -- start 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- start start 2024-04-25T10:10:33.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48b03380 con 0x563a49993000 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a48b03200 con 0x563a49992c00 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a48b03500 con 0x563a49993400 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35080/0 (socket says 172.21.15.82:35080) 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 learned_addr learned my addr 172.21.15.82:0/423913868 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993400 msgr2=0x563a48c28c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a49993400 0x563a48c28c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 msgr2=0x563a48c28100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a48a932c0 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a49992c00 0x563a48c28100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x563a499abbc0 tx=0x563a499b6f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 -- 172.21.15.82:0/423913868 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa2340 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 -- 172.21.15.82:0/423913868 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563a48ab6a80 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 -- 172.21.15.82:0/423913868 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563a48aa29c0 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a48db4000 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a48db41e0 con 0x563a49993000 2024-04-25T10:10:33.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 4 datascan.init: resolving metadata pool 13 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 -- 172.21.15.82:0/423913868 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563a48da8c00 con 0x563a49993000 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 0x563a48c29700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd67867640 1 -- 172.21.15.82:0/423913868 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x563a48da7000 con 0x563a49993000 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 4 datascan.main: resolving metadata pool 13 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 0x563a48c29700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x563a49993800 0x563a48c29180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a49993c00 con 0x563a49993800 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 0x563a48c29700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x563a48d93290 tx=0x563a48b04120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.534+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x563a49993800 0x563a48c29180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x563a49993800 0x563a48c29180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(1 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x563a49993c00 0x563a48e9e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a49992800 con 0x563a49993c00 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x563a49993c00 0x563a48e9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x563a49993c00 0x563a48e9e000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(2 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x563a49992800 0x563a48c27600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48a6d400 con 0x563a49992800 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x563a49992800 0x563a48c27600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x563a49992800 0x563a48c27600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(3 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48a6d400 con 0x563a49993800 2024-04-25T10:10:33.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(4 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x563a48a6d400 0x563a48e9e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.538+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a49992400 con 0x563a48a6d400 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x563a48a6d400 0x563a48e9e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x563a48a6d400 0x563a48e9e580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(5 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x563a49992400 0x563a48e9eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf000 con 0x563a49992400 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x563a49992400 0x563a48e9eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x563a49992400 0x563a48e9eb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(6 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf000 con 0x563a49993c00 2024-04-25T10:10:33.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(7 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf000 con 0x563a48a6d400 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(8 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x563a48daf000 0x563a48e9f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf400 con 0x563a48daf000 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x563a48daf000 0x563a48e9f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.542+0000 7fcd6d873640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x563a48daf000 0x563a48e9f080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(9 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf400 con 0x563a49992400 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(10 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf400 con 0x563a49992800 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(11 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf400 con 0x563a49993c00 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(12 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x563a48daf400 0x563a48c26580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48a6d800 con 0x563a48daf400 2024-04-25T10:10:33.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x563a48daf400 0x563a48c26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x563a48daf400 0x563a48c26580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(13 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d072640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x563a48a6d800 0x563a48e9f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48a6d800 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x563a48a6d800 0x563a48e9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x563a48a6d800 0x563a48e9f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(14 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a48a6d800 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.546+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993800 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(15 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf400 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(16 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48a6d400 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(17 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993c00 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(18 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf000 2024-04-25T10:10:33.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(19 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48a6d800 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(20 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a48a6d800 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(21 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.550+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(22 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(23 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993800 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(24 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49992400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(25 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993800 2024-04-25T10:10:33.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(26 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf000 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(27 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf400 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(28 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993800 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 6 ==== osd_op_reply(29 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.554+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48daf000 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(30 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48a6d400 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(31 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a48a6d400 2024-04-25T10:10:33.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 5 ==== osd_op_reply(32 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0000 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0400 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0c00 con 0x563a49993c00 2024-04-25T10:10:33.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1000 con 0x563a49993c00 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1400 con 0x563a49992400 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:10:33.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.558+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0000 con 0x563a48a6d400 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0400 con 0x563a49992800 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:10:33.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0c00 con 0x563a48daf400 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1000 con 0x563a49993800 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:10:33.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:10:33.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:10:33.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.562+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1400 con 0x563a49993800 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1800 con 0x563a49993800 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:10:33.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:10:33.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0000 con 0x563a48a6d400 2024-04-25T10:10:33.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 11 ==== osd_op_reply(47 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993c00 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 10 ==== osd_op_reply(48 [pgnls start_epoch 95 out=177b] v95'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49992800 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(49 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 12 ==== osd_op_reply(50 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.566+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d400 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 8 ==== osd_op_reply(51 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49992400 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(52 [pgnls start_epoch 95 out=149b] v94'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993c00 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 11 ==== osd_op_reply(53 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d400 2024-04-25T10:10:33.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 9 ==== osd_op_reply(54 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf000 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(55 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49992400 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(56 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49992800 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 5 ==== osd_op_reply(57 [pgnls start_epoch 95 out=81b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993c00 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 12 ==== osd_op_reply(58 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.570+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 8 ==== osd_op_reply(59 [pgnls start_epoch 95 out=77b] v94'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d800 2024-04-25T10:10:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(60 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a48a6d800 2024-04-25T10:10:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 13 ==== osd_op_reply(61 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 9 ==== osd_op_reply(62 [pgnls start_epoch 95 out=49b] v92'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d400 2024-04-25T10:10:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 10 ==== osd_op_reply(63 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993c00 2024-04-25T10:10:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 13 ==== osd_op_reply(64 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf000 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(65 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d800 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(66 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a48a6d800 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 10 ==== osd_op_reply(67 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.574+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 11 ==== osd_op_reply(68 [pgnls start_epoch 95 out=97b] v94'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 12 ==== osd_op_reply(69 [pgnls start_epoch 95 out=74b] v90'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 14 ==== osd_op_reply(70 [pgnls start_epoch 95 out=73b] v94'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49992400 2024-04-25T10:10:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(71 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 15 ==== osd_op_reply(72 [pgnls start_epoch 95 out=98b] v94'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.581 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf000 2024-04-25T10:10:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(73 [pgnls start_epoch 95 out=76b] v94'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.578+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf400 2024-04-25T10:10:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 13 ==== osd_op_reply(74 [pgnls start_epoch 95 out=73b] v94'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 16 ==== osd_op_reply(75 [pgnls start_epoch 95 out=73b] v94'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48daf000 2024-04-25T10:10:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(76 [pgnls start_epoch 95 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563a48b20780 con 0x563a48daf000 2024-04-25T10:10:33.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d400 2024-04-25T10:10:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 11 ==== osd_op_reply(77 [pgnls start_epoch 95 out=73b] v90'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 95 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a48a6d400 2024-04-25T10:10:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 12 ==== osd_op_reply(78 [pgnls start_epoch 95 out=101b] v94'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:10:33.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.582+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0c00 con 0x563a49993c00 2024-04-25T10:10:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:10:33.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1000 con 0x563a49993c00 2024-04-25T10:10:33.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1c00 con 0x563a49993c00 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48dafc00 con 0x563a49993c00 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48daf800 con 0x563a49993c00 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ece000 con 0x563a49992400 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20500 con 0x563a49992400 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:10:33.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ece400 con 0x563a48a6d400 2024-04-25T10:10:33.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ece800 con 0x563a49992800 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 <== osd.2 v2:172.21.15.82:6803/3086763100 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x563a48b20780 con 0x563a49992800 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.586+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecec00 con 0x563a49993800 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecf000 con 0x563a48daf400 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:10:33.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:10:33.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:10:33.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:10:33.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecf400 con 0x563a49993800 2024-04-25T10:10:33.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecf800 con 0x563a49993800 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:10:33.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecfc00 con 0x563a49993800 2024-04-25T10:10:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.590+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ece400 con 0x563a48a6d400 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.6 v2:172.21.15.196:6801/1119498737 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x563a48b20000 con 0x563a48a6d400 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecec00 con 0x563a49993c00 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:10:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ecf000 con 0x563a49993c00 2024-04-25T10:10:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v95'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1400 con 0x563a49993c00 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 20 datascan.scan_links: first is now 2 2024-04-25T10:10:33.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.594+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec1800 con 0x563a49993c00 2024-04-25T10:10:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6e074640 1 -- 172.21.15.82:0/423913868 <== osd.1 v2:172.21.15.82:6816/1010291016 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v95'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563a48b20500 con 0x563a49993c00 2024-04-25T10:10:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T10:10:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 10 datascan.scan_links: updating inotable 2024-04-25T10:10:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0000 con 0x563a49993800 2024-04-25T10:10:33.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T10:10:33.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T10:10:33.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ec0800 con 0x563a49993800 2024-04-25T10:10:33.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.7 v2:172.21.15.196:6822/2552340661 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v95'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x563a48b20000 con 0x563a49993800 2024-04-25T10:10:33.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 10 datascan.scan_links: updating snaptable 2024-04-25T10:10:33.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.598+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e95) v9 -- 0x563a48ece000 con 0x563a48daf400 2024-04-25T10:10:33.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6d072640 1 -- 172.21.15.82:0/423913868 <== osd.4 v2:172.21.15.196:6816/2164040014 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x563a48b20000 con 0x563a48daf400 2024-04-25T10:10:33.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x563a49992400 msgr2=0x563a48e9eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x563a49992400 0x563a48e9eb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x563a49993c00 msgr2=0x563a48e9e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x563a49993c00 0x563a48e9e000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x563a49992800 msgr2=0x563a48c27600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x563a49992800 0x563a48c27600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x563a48a6d800 msgr2=0x563a48e9f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x563a48a6d800 0x563a48e9f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x563a48daf400 msgr2=0x563a48c26580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x563a48daf400 0x563a48c26580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x563a48daf000 msgr2=0x563a48e9f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x563a48daf000 0x563a48e9f080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 reap_dead start 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x563a48a6d400 msgr2=0x563a48e9e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x563a48a6d400 0x563a48e9e580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x563a49993800 msgr2=0x563a48c29180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x563a49993800 0x563a48c29180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 reap_dead start 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6d873640 1 -- 172.21.15.82:0/423913868 reap_dead start 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 msgr2=0x563a48c29700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 0x563a48c29700 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x563a48d93290 tx=0x563a48b04120 comp rx=0 tx=0).stop 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 msgr2=0x563a48c28680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x563a499abbc0 tx=0x563a499b6f00 comp rx=0 tx=0).stop 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 shutdown_connections 2024-04-25T10:10:33.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x563a48daec00 0x563a48c29700 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/423913868 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a49993000 0x563a48c28680 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 >> 172.21.15.82:0/423913868 conn(0x563a48b01000 msgr2=0x563a48d9c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 shutdown_connections 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/423913868 wait complete. 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 msgr2=0x563a48c27080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x563a498bfb30 tx=0x563a48b04540 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 shutdown_connections 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x563a48b01800 0x563a48c26b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x563a48b01c00 0x563a48c27080 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 --2- 172.21.15.82:0/2679959945 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x563a48b00400 0x563a48c27b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 >> 172.21.15.82:0/2679959945 conn(0x563a48a6cc00 msgr2=0x563a48a9a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 shutdown_connections 2024-04-25T10:10:33.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.602+0000 7fcd6f0b48c0 1 -- 172.21.15.82:0/2679959945 wait complete. 2024-04-25T10:10:33.609 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.149384, result: 2024-04-25T10:10:33.609 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:10:33.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 Processor -- start 2024-04-25T10:10:33.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 -- start start 2024-04-25T10:10:33.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00b5270 0x7f3aa00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa00b5650 0x7f3aa00d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3aa00d4440 0x7f3aa006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa006cc00 con 0x7f3aa00b5650 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa006bff0 con 0x7f3aa00d4440 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa86ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa006c2d0 con 0x7f3aa00b5270 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa00b5650 0x7f3aa00d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3aa00b5650 0x7f3aa00d3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35096/0 (socket says 172.21.15.82:35096) 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3291354998 learned_addr learned my addr 172.21.15.82:0/3291354998 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa6c8b640 1 --2- 172.21.15.82:0/3291354998 >> v2:172.21.15.196:3300/0 conn(0x7f3aa00d4440 0x7f3aa006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa648a640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00b5270 0x7f3aa00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3291354998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00b5270 msgr2=0x7f3aa00b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00b5270 0x7f3aa00b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3291354998 >> v2:172.21.15.196:3300/0 conn(0x7f3aa00d4440 msgr2=0x7f3aa006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:33.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3291354998 >> v2:172.21.15.196:3300/0 conn(0x7f3aa00d4440 0x7f3aa006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.770+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3291354998 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa006c5b0 con 0x7f3aa00b5650 2024-04-25T10:10:33.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa00d3f00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00cdcb0 tx=0x7f3a9001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7709ac515bfb4668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5488640 1 -- 172.21.15.82:0/3291354998 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900aa660 con 0x7f3aa00b5650 2024-04-25T10:10:33.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5488640 1 -- 172.21.15.82:0/3291354998 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a900af070 con 0x7f3aa00b5650 2024-04-25T10:10:33.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5488640 1 -- 172.21.15.82:0/3291354998 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900aaa20 con 0x7f3aa00b5650 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3291354998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 msgr2=0x7f3aa00d3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa00d3f00 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3aa00cdcb0 tx=0x7f3a9001c520 comp rx=0 tx=0).stop 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3291354998 shutdown_connections 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3291354998 >> v2:172.21.15.196:3300/0 conn(0x7f3aa00d4440 0x7f3aa006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa00d3f00 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3291354998 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00b5270 0x7f3aa00b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3291354998 >> 172.21.15.82:0/3291354998 conn(0x7f3aa008b000 msgr2=0x7f3aa00d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3291354998 shutdown_connections 2024-04-25T10:10:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3291354998 wait complete. 2024-04-25T10:10:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 Processor -- start 2024-04-25T10:10:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- start start 2024-04-25T10:10:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa00b5270 0x7f3aa00c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00d4440 0x7f3aa00b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015dfa0 con 0x7f3aa00b5650 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa015de20 con 0x7f3aa00b5270 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa86ee640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa00ce300 con 0x7f3aa00d4440 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35104/0 (socket says 172.21.15.82:35104) 2024-04-25T10:10:33.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3076883845 learned_addr learned my addr 172.21.15.82:0/3076883845 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa648a640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa00b5270 0x7f3aa00c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00d4440 msgr2=0x7f3aa00b8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00d4440 0x7f3aa00b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa00b5270 msgr2=0x7f3aa00c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa00b5270 0x7f3aa00c9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:33.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.774+0000 7f3aa5c89640 1 -- 172.21.15.82:0/3076883845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa00b63f0 con 0x7f3aa00b5650 2024-04-25T10:10:33.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa5c89640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f3a9001abd0 tx=0x7f3a9001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900bd3f0 con 0x7f3aa00b5650 2024-04-25T10:10:33.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a900bd590 con 0x7f3aa00b5650 2024-04-25T10:10:33.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a900bd940 con 0x7f3aa00b5650 2024-04-25T10:10:33.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa00ca8d0 con 0x7f3aa00b5650 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa00cae90 con 0x7f3aa00b5650 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa004efa0 con 0x7f3aa00b5650 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a900b33f0 con 0x7f3aa00b5650 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3a9011fb20 con 0x7f3aa00b5650 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa648a640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:33.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.778+0000 7f3aa648a640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006fd0 tx=0x7f3a9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:33.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.782+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a90016110 con 0x7f3aa00b5650 2024-04-25T10:10:33.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:33.982+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3aa00d1b20 con 0x7f3aa00b5650 2024-04-25T10:10:34.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.186+0000 7f3a96ffd640 1 -- 172.21.15.82:0/3076883845 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v65) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3a900f04c0 con 0x7f3aa00b5650 2024-04-25T10:10:34.190 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 msgr2=0x7f3a84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c006fd0 tx=0x7f3a9c008040 comp rx=0 tx=0).stop 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 msgr2=0x7f3aa0143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f3a9001abd0 tx=0x7f3a9001ad60 comp rx=0 tx=0).stop 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 shutdown_connections 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3a84066f60 0x7f3a84069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3aa00d4440 0x7f3aa00b8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3aa00b5650 0x7f3aa0143a40 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 --2- 172.21.15.82:0/3076883845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3aa00b5270 0x7f3aa00c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 >> 172.21.15.82:0/3076883845 conn(0x7f3aa008b000 msgr2=0x7f3aa00b4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 shutdown_connections 2024-04-25T10:10:34.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.190+0000 7f3aa86ee640 1 -- 172.21.15.82:0/3076883845 wait complete. 2024-04-25T10:10:34.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:34.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 Processor -- start 2024-04-25T10:10:34.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 -- start start 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd968144ca0 0x7fd96806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9680ccc70 con 0x7fd96806ec60 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd96806dc40 con 0x7fd9680ce510 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96fb39640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9680cc2b0 con 0x7fd968144ca0 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd96806ec60 0x7fd968142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58512/0 (socket says 172.21.15.82:58512) 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 -- 172.21.15.82:0/1571088526 learned_addr learned my addr 172.21.15.82:0/1571088526 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:34.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2537337913 0 0) 0x7fd9680ccc70 con 0x7fd96806ec60 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/1571088526 >> v2:172.21.15.196:3300/0 conn(0x7fd968144ca0 0x7fd96806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd954003530 con 0x7fd96806ec60 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 35137946 0 0) 0x7fd954003530 con 0x7fd96806ec60 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 >> v2:172.21.15.196:3300/0 conn(0x7fd968144ca0 msgr2=0x7fd96806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 --2- 172.21.15.82:0/1571088526 >> v2:172.21.15.196:3300/0 conn(0x7fd968144ca0 0x7fd96806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 >> v1:172.21.15.196:6789/0 conn(0x7fd9680ce510 legacy=0x7fd9680ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96806df20 con 0x7fd96806ec60 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fd960001a00 con 0x7fd96806ec60 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 >> v1:172.21.15.82:6789/0 conn(0x7fd96806ec60 legacy=0x7fd968142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd954003a30 0x7fd954003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd954004810 0x7fd954004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd95400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9680cc2b0 con 0x7fd9540051d0 2024-04-25T10:10:34.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95400c0e0 con 0x7fd954003a30 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95400c2e0 con 0x7fd954004810 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd95400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd954004810 0x7fd954004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 -- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd954004810 msgr2=0x7fd954004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd954004810 0x7fd954004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 -- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd954003a30 msgr2=0x7fd954003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96d8d5640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd954003a30 0x7fd954003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd954003a30 0x7fd954003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96e0d6640 1 -- 172.21.15.82:0/1571088526 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd954003530 con 0x7fd9540051d0 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.346+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd954004810 0x7fd954004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96e0d6640 1 -- 172.21.15.82:0/1571088526 reap_dead start 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96e0d6640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd95400b9d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd96000a850 tx=0x7fd96001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e4203ae672fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96000a4a0 con 0x7fd9540051d0 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9600ad650 con 0x7fd9540051d0 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96c8d3640 1 -- 172.21.15.82:0/1571088526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9600ada40 con 0x7fd9540051d0 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 msgr2=0x7fd95400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd95400b9d0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fd96000a850 tx=0x7fd96001d3a0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- 172.21.15.82:0/1571088526 shutdown_connections 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 --2- 172.21.15.82:0/1571088526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd95400b9d0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- 172.21.15.82:0/1571088526 >> 172.21.15.82:0/1571088526 conn(0x7fd9680b11a0 msgr2=0x7fd9680b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- 172.21.15.82:0/1571088526 shutdown_connections 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- 172.21.15.82:0/1571088526 wait complete. 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 Processor -- start 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- start start 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd96815b1a0 0x7fd9680c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9680c8470 0x7fd9680c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9681451f0 con 0x7fd9540051d0 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96806e290 con 0x7fd9680c8470 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96fb39640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd968145370 con 0x7fd96815b1a0 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96d8d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96d8d5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:35122/0 (socket says 172.21.15.82:35122) 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96d8d5640 1 -- 172.21.15.82:0/2806063898 learned_addr learned my addr 172.21.15.82:0/2806063898 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd96815b1a0 0x7fd9680c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.350+0000 7fd96e0d6640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9680c8470 0x7fd9680c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 -- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd96815b1a0 msgr2=0x7fd9680c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd96815b1a0 0x7fd9680c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 -- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9680c8470 msgr2=0x7fd9680c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9680c8470 0x7fd9680c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 -- 172.21.15.82:0/2806063898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96815b600 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d8d5640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d570 tx=0x7fd96400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964005480 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd964017070 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9680c5830 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964003820 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9680ac0b0 con 0x7fd9540051d0 2024-04-25T10:10:34.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96804efa0 con 0x7fd9540051d0 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9640039c0 con 0x7fd9540051d0 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 0x7fd94c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 0x7fd94c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd964085150 con 0x7fd9540051d0 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.354+0000 7fd96d0d4640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 0x7fd94c069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd96806e690 tx=0x7fd958008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:34.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.358+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd964019310 con 0x7fd9540051d0 2024-04-25T10:10:34.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.538+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9680b1cc0 con 0x7fd9540051d0 2024-04-25T10:10:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.538+0000 7fd95e7fc640 1 -- 172.21.15.82:0/2806063898 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd964055e10 con 0x7fd9540051d0 2024-04-25T10:10:34.541 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:34.541 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:34.194955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:34.541 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 66 2024-04-25T10:10:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 msgr2=0x7fd94c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 0x7fd94c069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd96806e690 tx=0x7fd958008040 comp rx=0 tx=0).stop 2024-04-25T10:10:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 msgr2=0x7fd9680ae3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fd96400d570 tx=0x7fd96400da30 comp rx=0 tx=0).stop 2024-04-25T10:10:34.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 shutdown_connections 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd94c066f60 0x7fd94c069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd96815b1a0 0x7fd9680c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9680c8470 0x7fd9680c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 --2- 172.21.15.82:0/2806063898 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9540051d0 0x7fd9680ae3c0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 >> 172.21.15.82:0/2806063898 conn(0x7fd9680b11a0 msgr2=0x7fd9680b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 shutdown_connections 2024-04-25T10:10:34.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:34.542+0000 7fd96fb39640 1 -- 172.21.15.82:0/2806063898 wait complete. 2024-04-25T10:10:34.961 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:10:34.194955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8376}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8376': {'gid': 8376, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/570372586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 570372586}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 570372586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:34.961 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8376:up:replay 2024-04-25T10:10:35.962 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:36.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 Processor -- start 2024-04-25T10:10:36.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 -- start start 2024-04-25T10:10:36.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1dc1451f0 0x7fb1dc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1dc0cee30 con 0x7fb1dc144ca0 2024-04-25T10:10:36.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1dc0cc2b0 con 0x7fb1dc1451f0 2024-04-25T10:10:36.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e43d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc06e290 con 0x7fb1dc0ce510 2024-04-25T10:10:36.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2971640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb1dc1451f0 0x7fb1dc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:55706/0 (socket says 172.21.15.82:55706) 2024-04-25T10:10:36.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 -- 172.21.15.82:0/3441161963 learned_addr learned my addr 172.21.15.82:0/3441161963 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:36.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e196f640 1 --1- 172.21.15.82:0/3441161963 >> v1:172.21.15.196:6789/0 conn(0x7fb1dc144ca0 0x7fb1dc06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:33868/0 (socket says 172.21.15.82:33868) 2024-04-25T10:10:36.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 -- 172.21.15.82:0/3441161963 >> v2:172.21.15.196:3300/0 conn(0x7fb1dc1451f0 msgr2=0x7fb1dc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:36.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 --2- 172.21.15.82:0/3441161963 >> v2:172.21.15.196:3300/0 conn(0x7fb1dc1451f0 0x7fb1dc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 -- 172.21.15.82:0/3441161963 >> v1:172.21.15.196:6789/0 conn(0x7fb1dc144ca0 legacy=0x7fb1dc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.090+0000 7fb1e2170640 1 -- 172.21.15.82:0/3441161963 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc0ccc70 con 0x7fb1dc0ce510 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e2170640 1 --2- 172.21.15.82:0/3441161963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009870 tx=0x7fb1cc01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb00461811f64e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e116e640 1 -- 172.21.15.82:0/3441161963 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b5070 con 0x7fb1dc0ce510 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e116e640 1 -- 172.21.15.82:0/3441161963 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0a8a50 con 0x7fb1dc0ce510 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e116e640 1 -- 172.21.15.82:0/3441161963 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0a8e20 con 0x7fb1dc0ce510 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- 172.21.15.82:0/3441161963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 msgr2=0x7fb1dc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/3441161963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009870 tx=0x7fb1cc01ed50 comp rx=0 tx=0).stop 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- 172.21.15.82:0/3441161963 shutdown_connections 2024-04-25T10:10:36.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/3441161963 >> v2:172.21.15.196:3300/0 conn(0x7fb1dc1451f0 0x7fb1dc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/3441161963 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc0ce510 0x7fb1dc0ce8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- 172.21.15.82:0/3441161963 >> 172.21.15.82:0/3441161963 conn(0x7fb1dc0b11a0 msgr2=0x7fb1dc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- 172.21.15.82:0/3441161963 shutdown_connections 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- 172.21.15.82:0/3441161963 wait complete. 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 Processor -- start 2024-04-25T10:10:36.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 -- start start 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.094+0000 7fb1e43d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1dc144ca0 0x7fb1dc12ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc1451f0 0x7fb1dc0c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc06ee30 con 0x7fb1dc0ce510 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc06df20 con 0x7fb1dc144ca0 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1dc0cc900 con 0x7fb1dc1451f0 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33022/0 (socket says 172.21.15.82:33022) 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 -- 172.21.15.82:0/1574246033 learned_addr learned my addr 172.21.15.82:0/1574246033 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:36.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e196f640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1dc144ca0 0x7fb1dc12ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2971640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc1451f0 0x7fb1dc0c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 -- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc1451f0 msgr2=0x7fb1dc0c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc1451f0 0x7fb1dc0c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 -- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1dc144ca0 msgr2=0x7fb1dc12ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1dc144ca0 0x7fb1dc12ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 -- 172.21.15.82:0/1574246033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc12e7b0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e2170640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01d400 tx=0x7fb1cc01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0c0070 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1cc0ae550 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1dc12bb00 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1cc0b5070 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1dc12c0c0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1dc04efa0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1cc0c3070 con 0x7fb1dc0ce510 2024-04-25T10:10:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 0x7fb1c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fb1cc122be0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e196f640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 0x7fb1c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:36.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.098+0000 7fb1e196f640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 0x7fb1c0069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc1f4570 tx=0x7fb1d8007cd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:36.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.102+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1dc12e7b0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.298+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1dc0008e0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.298+0000 7fb1d2ffd640 1 -- 172.21.15.82:0/1574246033 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fb1cc0188b0 con 0x7fb1dc0ce510 2024-04-25T10:10:36.301 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:36.302 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":68,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:35.204230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:36.302 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 68 2024-04-25T10:10:36.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 msgr2=0x7fb1c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 0x7fb1c0069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc1f4570 tx=0x7fb1d8007cd0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 msgr2=0x7fb1dc12e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:36.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc01d400 tx=0x7fb1cc01baf0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 shutdown_connections 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb1c0066f60 0x7fb1c0069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb1dc1451f0 0x7fb1dc0c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb1dc144ca0 0x7fb1dc12ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 --2- 172.21.15.82:0/1574246033 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb1dc0ce510 0x7fb1dc12e270 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 >> 172.21.15.82:0/1574246033 conn(0x7fb1dc0b11a0 msgr2=0x7fb1dc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 shutdown_connections 2024-04-25T10:10:36.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:36.302+0000 7fb1e43d4640 1 -- 172.21.15.82:0/1574246033 wait complete. 2024-04-25T10:10:36.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:10:35.204230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8376}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8376': {'gid': 8376, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/570372586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 570372586}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 570372586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:36.724 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8376:up:rejoin 2024-04-25T10:10:37.725 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:10:37.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 Processor -- start 2024-04-25T10:10:37.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 -- start start 2024-04-25T10:10:37.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b8015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5b800b24a0 0x7f5b800b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800b2090 con 0x7f5b800b3f50 2024-04-25T10:10:37.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b8015f200 con 0x7f5b800b24a0 2024-04-25T10:10:37.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b86f83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800b43e0 con 0x7f5b8015a700 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f5b800b3f50 0x7f5b800cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59634/0 (socket says 172.21.15.82:59634) 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7ffff640 1 -- 172.21.15.82:0/2299773862 learned_addr learned my addr 172.21.15.82:0/2299773862 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 --2- 172.21.15.82:0/2299773862 >> v2:172.21.15.196:3300/0 conn(0x7f5b800b24a0 0x7f5b800b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b84d1f640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b8015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809538153 0 0) 0x7f5b800b2090 con 0x7f5b800b3f50 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b68003530 con 0x7f5b800b3f50 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2003951435 0 0) 0x7f5b68003530 con 0x7f5b800b3f50 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 msgr2=0x7f5b8015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b8015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 >> v2:172.21.15.196:3300/0 conn(0x7f5b800b24a0 msgr2=0x7f5b800b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 --2- 172.21.15.82:0/2299773862 >> v2:172.21.15.196:3300/0 conn(0x7f5b800b24a0 0x7f5b800b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800b46c0 con 0x7f5b800b3f50 2024-04-25T10:10:37.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f5b70001680 con 0x7f5b800b3f50 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 >> v1:172.21.15.82:6789/0 conn(0x7f5b800b3f50 legacy=0x7f5b800cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8015f200 con 0x7f5b68004d50 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800b43e0 con 0x7f5b68003a30 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6800bad0 con 0x7f5b8015a700 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b84d1f640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b7ffff640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 -- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 msgr2=0x7f5b68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b8015a700 0x7f5b68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 -- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b68003a30 msgr2=0x7f5b68003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:37.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b68003a30 0x7f5b68003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.850+0000 7f5b85520640 1 -- 172.21.15.82:0/2299773862 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b68003530 con 0x7f5b68004d50 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b85520640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53b1f79a4350cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780b2070 con 0x7f5b68004d50 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b780a5b30 con 0x7f5b68004d50 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b7f7fe640 1 -- 172.21.15.82:0/2299773862 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78009bf0 con 0x7f5b68004d50 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- 172.21.15.82:0/2299773862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b68004d50 msgr2=0x7f5b680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 --2- 172.21.15.82:0/2299773862 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b68004d50 0x7f5b680051d0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009fd0 tx=0x7f5b7801cc80 comp rx=0 tx=0).stop 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b85520640 1 -- 172.21.15.82:0/2299773862 reap_dead start 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- 172.21.15.82:0/2299773862 shutdown_connections 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- 172.21.15.82:0/2299773862 >> 172.21.15.82:0/2299773862 conn(0x7f5b800b11a0 msgr2=0x7f5b800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- 172.21.15.82:0/2299773862 shutdown_connections 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- 172.21.15.82:0/2299773862 wait complete. 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 Processor -- start 2024-04-25T10:10:37.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.854+0000 7f5b86f83640 1 -- start start 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b801511f0 0x7f5b8015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b801515d0 0x7f5b80158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cd310 con 0x7f5b80151a70 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800ce600 con 0x7f5b801511f0 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cd490 con 0x7f5b801515d0 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33036/0 (socket says 172.21.15.82:33036) 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 -- 172.21.15.82:0/1418557561 learned_addr learned my addr 172.21.15.82:0/1418557561 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b84d1f640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b801511f0 0x7f5b8015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7ffff640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b801515d0 0x7f5b80158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 -- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b801515d0 msgr2=0x7f5b80158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b801515d0 0x7f5b80158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 -- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b801511f0 msgr2=0x7f5b8015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b801511f0 0x7f5b8015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 -- 172.21.15.82:0/1418557561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8015d3e0 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b85520640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5b78002240 tx=0x7f5b78009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b780b2040 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b780a5930 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8015db70 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78009bf0 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b80168290 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8004efa0 con 0x7f5b80151a70 2024-04-25T10:10:37.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b780c0030 con 0x7f5b80151a70 2024-04-25T10:10:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b84d1f640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5b7811f900 con 0x7f5b80151a70 2024-04-25T10:10:37.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.858+0000 7f5b84d1f640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 0x7f5b6c069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009ae0 tx=0x7f5b74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:37.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:37.862+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b780c7060 con 0x7f5b80151a70 2024-04-25T10:10:38.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b8006f260 con 0x7f5b80151a70 2024-04-25T10:10:38.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b7d7fa640 1 -- 172.21.15.82:0/1418557561 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f5b780169d0 con 0x7f5b80151a70 2024-04-25T10:10:38.065 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:10:38.065 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:10:38.065 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:10:38.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 msgr2=0x7f5b6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 0x7f5b6c069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009ae0 tx=0x7f5b74008040 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 msgr2=0x7f5b80158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5b78002240 tx=0x7f5b78009750 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 shutdown_connections 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5b801515d0 0x7f5b80158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5b80151a70 0x7f5b80158c50 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 --2- 172.21.15.82:0/1418557561 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5b801511f0 0x7f5b8015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 >> 172.21.15.82:0/1418557561 conn(0x7f5b800b11a0 msgr2=0x7f5b80164c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.062+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 shutdown_connections 2024-04-25T10:10:38.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.066+0000 7f5b86f83640 1 -- 172.21.15.82:0/1418557561 wait complete. 2024-04-25T10:10:38.219 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:38.221 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:38.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:49.240218+0000', 'modified': '2024-04-25T10:10:36.212278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 95, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8376}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8376': {'gid': 8376, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/570372586', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 570372586}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 570372586}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:10:38.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:10:38.487 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:10:38.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 Processor -- start 2024-04-25T10:10:38.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 -- start start 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 0x7f362006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f362006ec60 0x7f3620142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36200ccc70 con 0x7f36200ce510 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36200cc2b0 con 0x7f362006ec60 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3626bed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362006e290 con 0x7f3620144ca0 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f362518a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f362006ec60 0x7f3620142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3624989640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f36200ce510 0x7f36200ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59648/0 (socket says 172.21.15.82:59648) 2024-04-25T10:10:38.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f3624989640 1 -- 172.21.15.82:0/465597750 learned_addr learned my addr 172.21.15.82:0/465597750 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f361ffff640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 0x7f362006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f362518a640 1 -- 172.21.15.82:0/465597750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 msgr2=0x7f362006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f362518a640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 0x7f362006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f362518a640 1 -- 172.21.15.82:0/465597750 >> v1:172.21.15.82:6789/0 conn(0x7f36200ce510 legacy=0x7f36200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f362518a640 1 -- 172.21.15.82:0/465597750 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362006dc40 con 0x7f362006ec60 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f361f7fe640 1 -- 172.21.15.82:0/465597750 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846638113 0 0) 0x7f36200ccc70 con 0x7f36200ce510 2024-04-25T10:10:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.618+0000 7f361ffff640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 0x7f362006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f362518a640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f3620142680 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f3610009d90 tx=0x7f361001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35336336accca85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f361f7fe640 1 -- 172.21.15.82:0/465597750 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100b4070 con 0x7f362006ec60 2024-04-25T10:10:38.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f361f7fe640 1 -- 172.21.15.82:0/465597750 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100af600 con 0x7f362006ec60 2024-04-25T10:10:38.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f361f7fe640 1 -- 172.21.15.82:0/465597750 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100af9d0 con 0x7f362006ec60 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- 172.21.15.82:0/465597750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 msgr2=0x7f3620142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f3620142680 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f3610009d90 tx=0x7f361001f150 comp rx=0 tx=0).stop 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- 172.21.15.82:0/465597750 shutdown_connections 2024-04-25T10:10:38.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f3620142680 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- 172.21.15.82:0/465597750 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3620144ca0 0x7f362006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- 172.21.15.82:0/465597750 >> 172.21.15.82:0/465597750 conn(0x7f36200b11a0 msgr2=0x7f36200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- 172.21.15.82:0/465597750 shutdown_connections 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- 172.21.15.82:0/465597750 wait complete. 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 Processor -- start 2024-04-25T10:10:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- start start 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 0x7f362015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 0x7f36200b4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36201451f0 con 0x7f362006ec60 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36200cc900 con 0x7f3620144ca0 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3626bed640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3620145370 con 0x7f36200ce510 2024-04-25T10:10:38.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33052/0 (socket says 172.21.15.82:33052) 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 -- 172.21.15.82:0/1030909362 learned_addr learned my addr 172.21.15.82:0/1030909362 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f361ffff640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 0x7f362015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f362518a640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 0x7f36200b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 -- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 msgr2=0x7f362015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 0x7f362015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 -- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 msgr2=0x7f36200b4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 0x7f36200b4000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f3624989640 1 -- 172.21.15.82:0/1030909362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36200af4d0 con 0x7f362006ec60 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f362518a640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 0x7f36200b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.622+0000 7f361ffff640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 0x7f362015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3624989640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f361800c520 tx=0x7f361800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 -- 172.21.15.82:0/1030909362 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3618018070 con 0x7f362006ec60 2024-04-25T10:10:38.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 -- 172.21.15.82:0/1030909362 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36180074e0 con 0x7f362006ec60 2024-04-25T10:10:38.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362015b8c0 con 0x7f362006ec60 2024-04-25T10:10:38.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 -- 172.21.15.82:0/1030909362 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36180049f0 con 0x7f362006ec60 2024-04-25T10:10:38.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36200ae1d0 con 0x7f362006ec60 2024-04-25T10:10:38.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 -- 172.21.15.82:0/1030909362 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f361801b070 con 0x7f362006ec60 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 0x7f360c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361d7fa640 1 -- 172.21.15.82:0/1030909362 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3618084780 con 0x7f362006ec60 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361ffff640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 0x7f360c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f361ffff640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 0x7f360c069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f36201f6a80 tx=0x7f3614008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 Processor -- start 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 -- start start 2024-04-25T10:10:38.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 0x5565f999dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f983b5f0 con 0x5565f9953f30 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f97a2da0 con 0x5565f999d920 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565f972b150 con 0x5565f999e240 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.626+0000 7f362518a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33066/0 (socket says 172.21.15.82:33066) 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 -- 172.21.15.82:0/1612991729 learned_addr learned my addr 172.21.15.82:0/1612991729 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3624989640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 0x5565f999dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 -- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 msgr2=0x5565f99539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 -- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 msgr2=0x5565f999dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 0x5565f999dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 -- 172.21.15.82:0/1612991729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f9744ca0 con 0x5565f9953f30 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3624989640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 0x5565f999dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f36100b2040 tx=0x7f3610004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33be6c9f66a1bcfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f36067fc640 1 -- 172.21.15.82:0/1612991729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100b4040 con 0x5565f9953f30 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f36067fc640 1 -- 172.21.15.82:0/1612991729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100af890 con 0x5565f9953f30 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f36067fc640 1 -- 172.21.15.82:0/1612991729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100afc40 con 0x5565f9953f30 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/1612991729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 msgr2=0x5565f9954310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f36100b2040 tx=0x7f3610004ac0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/1612991729 shutdown_connections 2024-04-25T10:10:38.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f9953f30 0x5565f9954310 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1612991729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999d920 0x5565f999dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/1612991729 >> 172.21.15.82:0/1612991729 conn(0x5565f995a2a0 msgr2=0x5565f99522d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/1612991729 shutdown_connections 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/1612991729 wait complete. 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- start start 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 Processor -- start 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f9957aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f975d990 con 0x5565f999d920 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f979ee40 con 0x5565f9953f30 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565f972e900 con 0x5565f999e240 2024-04-25T10:10:38.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 10 client.0 _ll_register_callbacks cb 0x5565f98561c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33074/0 (socket says 172.21.15.82:33074) 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 -- 172.21.15.82:0/2718928051 learned_addr learned my addr 172.21.15.82:0/2718928051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f362518a640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f9957aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3624989640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 -- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 msgr2=0x5565f9957aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f9957aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 -- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 msgr2=0x5565f99a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 -- 172.21.15.82:0/2718928051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565f975b630 con 0x5565f999d920 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f361ffff640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f36140126f0 tx=0x7f3614004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402b8be7d52f2fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f35ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361401f610 con 0x5565f999d920 2024-04-25T10:10:38.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f361401f7b0 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36100af890 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f361402c050 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 10 client.8454 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.630+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5565f972b3a0 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36100af890 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f361402f020 con 0x5565f999d920 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 10 client.8454 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 10 client.8454 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 20 client.8454 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99584b0 0x5565f9958890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 4 client.8454 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565f973ea50 con 0x5565f99584b0 2024-04-25T10:10:38.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3624989640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99584b0 0x5565f9958890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 10 client.8454 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3624989640 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99584b0 0x5565f9958890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b511e29f7e25e787 server_cookie=d9e4257cbb253d20 in_seq=0 out_seq=0 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 1 -- 172.21.15.82:0/2718928051 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5565f973ea50 con 0x5565f99584b0 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 10 client.8454 handle_command_reply: tid=0 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 client.8454 shutdown 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 msgr2=0x5565f99a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f36140126f0 tx=0x7f3614004740 comp rx=0 tx=0).stop 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 shutdown_connections 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99584b0 0x5565f9958890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f9957aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99a51c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f35ff7fe640 0 client.8454 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 --2- 172.21.15.82:0/2718928051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 >> 172.21.15.82:0/2718928051 conn(0x5565f995a2a0 msgr2=0x5565f99529e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 shutdown_connections 2024-04-25T10:10:38.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 1 -- 172.21.15.82:0/2718928051 wait complete. 2024-04-25T10:10:38.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.634+0000 7f3629d5a000 20 client.8454 trim_cache size 0 max 16384 2024-04-25T10:10:38.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 Processor -- start 2024-04-25T10:10:38.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- start start 2024-04-25T10:10:38.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f9953f30 0x5565f999bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999e240 0x5565f99a7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36201ff410 con 0x5565f999d920 2024-04-25T10:10:38.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620143820 con 0x5565f999e240 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36201fee40 con 0x5565f9953f30 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33076/0 (socket says 172.21.15.82:33076) 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 -- 172.21.15.82:0/1480282363 learned_addr learned my addr 172.21.15.82:0/1480282363 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 -- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f9953f30 msgr2=0x5565f999bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3624989640 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f9953f30 0x5565f999bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f9953f30 0x5565f999bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 -- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999e240 msgr2=0x5565f99a7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999e240 0x5565f99a7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 -- 172.21.15.82:0/1480282363 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565f9824600 con 0x5565f999d920 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f361ffff640 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f36140082a0 tx=0x7f361401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b01bb92d1b5cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3605ffb640 1 -- 172.21.15.82:0/1480282363 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361401f610 con 0x5565f999d920 2024-04-25T10:10:38.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3605ffb640 1 -- 172.21.15.82:0/1480282363 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614024070 con 0x5565f999d920 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3605ffb640 1 -- 172.21.15.82:0/1480282363 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361401f9b0 con 0x5565f999d920 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- 172.21.15.82:0/1480282363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 msgr2=0x5565f999c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f36140082a0 tx=0x7f361401d000 comp rx=0 tx=0).stop 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- 172.21.15.82:0/1480282363 shutdown_connections 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f999e240 0x5565f99a7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f999c220 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 --2- 172.21.15.82:0/1480282363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f9953f30 0x5565f999bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- 172.21.15.82:0/1480282363 >> 172.21.15.82:0/1480282363 conn(0x5565f995a2a0 msgr2=0x5565f9957220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- 172.21.15.82:0/1480282363 shutdown_connections 2024-04-25T10:10:38.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- 172.21.15.82:0/1480282363 wait complete. 2024-04-25T10:10:38.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 -- start start 2024-04-25T10:10:38.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.650+0000 7f3629d5a000 1 Processor -- start 2024-04-25T10:10:38.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99b9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f97a73d0 con 0x5565f999d920 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565f985ed50 con 0x5565f9953f30 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565f973d980 con 0x5565f999e240 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 10 client.0 _ll_register_callbacks cb 0x7f36100b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33080/0 (socket says 172.21.15.82:33080) 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 -- 172.21.15.82:0/128705971 learned_addr learned my addr 172.21.15.82:0/128705971 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f362518a640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3624989640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 -- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 msgr2=0x5565f99b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 -- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 msgr2=0x5565f99aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 -- 172.21.15.82:0/128705971 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565f97379b0 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f362518a640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3624989640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f361ffff640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3614008010 tx=0x7f3614012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95dc33f78041114e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614040030 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f3614005c70 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3614024070 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3614049050 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 10 client.8460 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5565f974a0b0 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3614024070 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f361404c020 con 0x5565f999d920 2024-04-25T10:10:38.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 10 client.8460 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 10 client.8460 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 20 client.8460 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99b9db0 0x5565f99fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 4 client.8460 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565f97b02e0 con 0x5565f99b9db0 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3624989640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99b9db0 0x5565f99fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 10 client.8460 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3624989640 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99b9db0 0x5565f99fe3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c20b4a75de64d6c server_cookie=f028e9b6db6ffc61 in_seq=0 out_seq=0 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 1 -- 172.21.15.82:0/128705971 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5565f97b02e0 con 0x5565f99b9db0 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 10 client.8460 handle_command_reply: tid=0 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 client.8460 shutdown 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 msgr2=0x5565f99b9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f3614008010 tx=0x7f3614012000 comp rx=0 tx=0).stop 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 shutdown_connections 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x5565f99b9db0 0x5565f99fe3c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5565f999e240 0x5565f99b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f35ff7fe640 0 client.8460 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5565f999d920 0x5565f99b9330 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 --2- 172.21.15.82:0/128705971 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5565f9953f30 0x5565f99aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 >> 172.21.15.82:0/128705971 conn(0x5565f995a2a0 msgr2=0x5565f99a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 shutdown_connections 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 1 -- 172.21.15.82:0/128705971 wait complete. 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3629d5a000 20 client.8460 trim_cache size 0 max 16384 2024-04-25T10:10:38.663 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8376, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4573709890000002, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 69, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 95, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 5.4348486640000004 2024-04-25T10:10:38.664 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 msgr2=0x7f360c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 0x7f360c069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f36201f6a80 tx=0x7f3614008040 comp rx=0 tx=0).stop 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 msgr2=0x7f36200aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f361800c520 tx=0x7f361800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 shutdown_connections 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f360c066f60 0x7f360c069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f36200ce510 0x7f362015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3620144ca0 0x7f36200b4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 --2- 172.21.15.82:0/1030909362 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f362006ec60 0x7f36200aef90 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 >> 172.21.15.82:0/1030909362 conn(0x7f36200b11a0 msgr2=0x7f3620071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 shutdown_connections 2024-04-25T10:10:38.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.654+0000 7f3626bed640 1 -- 172.21.15.82:0/1030909362 wait complete. 2024-04-25T10:10:38.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:10:38.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.782+0000 7f054edc5640 1 Processor -- start 2024-04-25T10:10:38.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.782+0000 7f054edc5640 1 -- start start 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054edc5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f05480ce540 0x7f054815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054edc5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0548099df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054edc5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054806c570 con 0x7f054815d8d0 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054edc5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054806cea0 con 0x7f05480ce540 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054edc5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054806c850 con 0x7f054806d330 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f054815d8d0 0x7f05480cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59652/0 (socket says 172.21.15.82:59652) 2024-04-25T10:10:38.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 -- 172.21.15.82:0/1874287630 learned_addr learned my addr 172.21.15.82:0/1874287630 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f0547fff640 1 --2- 172.21.15.82:0/1874287630 >> v2:172.21.15.196:3300/0 conn(0x7f05480ce540 0x7f054815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054d362640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0548099df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175432276 0 0) 0x7f054806c570 con 0x7f054815d8d0 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0530003530 con 0x7f054815d8d0 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2323629767 0 0) 0x7f0530003530 con 0x7f054815d8d0 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 msgr2=0x7f0548099df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0548099df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 >> v2:172.21.15.196:3300/0 conn(0x7f05480ce540 msgr2=0x7f054815dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 --2- 172.21.15.82:0/1874287630 >> v2:172.21.15.196:3300/0 conn(0x7f05480ce540 0x7f054815dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054806cb30 con 0x7f054815d8d0 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f053c001680 con 0x7f054815d8d0 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 >> v1:172.21.15.82:6789/0 conn(0x7f054815d8d0 legacy=0x7f05480cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0530003a30 0x7f0530003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0530004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530004d50 0x7f05300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054806cea0 con 0x7f0530003a30 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054806c850 con 0x7f0530004d50 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053000b9d0 con 0x7f054806d330 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0530003a30 0x7f0530003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f0547fff640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0530004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054d362640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530004d50 0x7f05300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 -- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 msgr2=0x7f0530004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f054806d330 0x7f0530004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 -- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530004d50 msgr2=0x7f05300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0530004d50 0x7f05300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 -- 172.21.15.82:0/1874287630 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0530003530 con 0x7f0530003a30 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f054cb61640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0530003a30 0x7f0530003e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f053c00c210 tx=0x7f053c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b285bc6e8a17c7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c007d80 con 0x7f0530003a30 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053c0a9020 con 0x7f0530003a30 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.786+0000 7f05477fe640 1 -- 172.21.15.82:0/1874287630 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053c0a5ba0 con 0x7f0530003a30 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/1874287630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0530003a30 msgr2=0x7f0530003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 --2- 172.21.15.82:0/1874287630 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0530003a30 0x7f0530003e50 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f053c00c210 tx=0x7f053c01cc60 comp rx=0 tx=0).stop 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 -- 172.21.15.82:0/1874287630 reap_dead start 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/1874287630 shutdown_connections 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/1874287630 >> 172.21.15.82:0/1874287630 conn(0x7f05480d15b0 msgr2=0x7f05480d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/1874287630 shutdown_connections 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/1874287630 wait complete. 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 Processor -- start 2024-04-25T10:10:38.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- start start 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05481519a0 0x7f05481f75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 0x7f05481fa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480cbc00 con 0x7f05481fa6e0 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054809a810 con 0x7f0548151d80 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480cbd80 con 0x7f05481519a0 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33100/0 (socket says 172.21.15.82:33100) 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 -- 172.21.15.82:0/516806292 learned_addr learned my addr 172.21.15.82:0/516806292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f0547fff640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 0x7f05481fa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054cb61640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05481519a0 0x7f05481f75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 -- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05481519a0 msgr2=0x7f05481f75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05481519a0 0x7f05481f75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 -- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 msgr2=0x7f05481fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 0x7f05481fa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 -- 172.21.15.82:0/516806292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05481f7af0 con 0x7f05481fa6e0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f0547fff640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 0x7f05481fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054d362640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f0548071b90 tx=0x7f05400115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540011e10 con 0x7f05481fa6e0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054001b070 con 0x7f05481fa6e0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05481f7d90 con 0x7f05481fa6e0 2024-04-25T10:10:38.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400039a0 con 0x7f05481fa6e0 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.790+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0548159960 con 0x7f05481fa6e0 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054804efa0 con 0x7f05481fa6e0 2024-04-25T10:10:38.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0540003b60 con 0x7f05481fa6e0 2024-04-25T10:10:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f05457fa640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 0x7f0534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f0540086810 con 0x7f05481fa6e0 2024-04-25T10:10:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f054cb61640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 0x7f0534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:38.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.794+0000 7f054cb61640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 0x7f0534069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f053c0aa000 tx=0x7f053c001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:38.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.798+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05400576e0 con 0x7f05481fa6e0 2024-04-25T10:10:38.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.978+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f054806f200 con 0x7f05481fa6e0 2024-04-25T10:10:38.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.978+0000 7f05457fa640 1 -- 172.21.15.82:0/516806292 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0540056fe0 con 0x7f05481fa6e0 2024-04-25T10:10:38.982 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:10:38.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 msgr2=0x7f0534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 0x7f0534069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f053c0aa000 tx=0x7f053c001a50 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 msgr2=0x7f054815a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f0548071b90 tx=0x7f05400115a0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 shutdown_connections 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0534066f60 0x7f0534069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f05481fa6e0 0x7f054815a720 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0548151d80 0x7f05481fa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 --2- 172.21.15.82:0/516806292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05481519a0 0x7f05481f75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 >> 172.21.15.82:0/516806292 conn(0x7f05480d15b0 msgr2=0x7f0548073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 shutdown_connections 2024-04-25T10:10:38.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:38.982+0000 7f054edc5640 1 -- 172.21.15.82:0/516806292 wait complete. 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:10:39.000 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:39.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.001 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:10:39.007 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 42031sec preferred_lft 42031sec 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.008 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:10:39.009 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:10:39.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.010 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:10:39.059 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:10:39.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.061 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:10:39.110 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:10:39.111 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:10:39.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:10:39.112 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:10:39.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.185 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:10:39.201 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:10:39.198+0000 7faf3998d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:10:39.205 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:10:39.205 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:10:39.206 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:39.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.206 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:10:39.259 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:10:39.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:39.260 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:39.307 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:10:39.308 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:10:39.308 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:39.355 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:10:39.413 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:10:39.462 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:10:39.463 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:39.464 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:39.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.465 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:10:39.512 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:10:39.512 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:10:39.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:10:39.572 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:39.620 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:39.621 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:39.622 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:39.623 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:39.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:39.624 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:10:39.638 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:10:39.634+0000 7f98cde87580 -1 init, newargv = 0x55b079120eb0 newargc=15 2024-04-25T10:10:39.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[50209]: starting ceph client 2024-04-25T10:10:39.644 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[50209]: starting fuse 2024-04-25T10:10:40.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:10:40.761 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:10:40.809 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:10:40.810 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:10:40.811 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:10:40.812 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:40.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:40.813 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:10:40.859 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:10:40.861 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:10:40.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:10:40.861 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:10:40.861 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:10:40.861 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:10:40.861 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:10:40.862 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:10:40.863 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:10:40.988 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.50209.asok 2024-04-25T10:10:40.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:10:41.001 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50209.asok 2024-04-25T10:10:41.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.001 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50209.asok status 2024-04-25T10:10:41.155 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "50209", 2024-04-25T10:10:41.156 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8472, 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 8472 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3917055295 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:10:41.157 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 3917055295 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.8472 192.168.0.1:0/3917055295", 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/3917055295", 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 69, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 95, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:10:41.158 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:10:41.159 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:10:41.159 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:10:41.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.170 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:10:41.170 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:41.178 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:10:41.178 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:41.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.179 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:10:41.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.231 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T10:10:41.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 Processor -- start 2024-04-25T10:10:41.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 -- start start 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1000b0900 0x7fd10006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd100099a70 0x7fd100099f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1000d09c0 con 0x7fd10015d8a0 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd10006ee70 con 0x7fd100099a70 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd107e29640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10006f150 con 0x7fd1000b0900 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd1063c6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd100099a70 0x7fd100099f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd105bc5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd10015d8a0 0x7fd1000d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59664/0 (socket says 172.21.15.82:59664) 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd105bc5640 1 -- 172.21.15.82:0/2588922923 learned_addr learned my addr 172.21.15.82:0/2588922923 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd1063c6640 1 -- 172.21.15.82:0/2588922923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1000b0900 msgr2=0x7fd10006fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd1063c6640 1 --2- 172.21.15.82:0/2588922923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1000b0900 0x7fd10006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd1063c6640 1 -- 172.21.15.82:0/2588922923 >> v1:172.21.15.82:6789/0 conn(0x7fd10015d8a0 legacy=0x7fd1000d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd104bc3640 1 -- 172.21.15.82:0/2588922923 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397916868 0 0) 0x7fd1000d09c0 con 0x7fd10015d8a0 2024-04-25T10:10:41.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.402+0000 7fd1063c6640 1 -- 172.21.15.82:0/2588922923 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10006f430 con 0x7fd100099a70 2024-04-25T10:10:41.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd1063c6640 1 --2- 172.21.15.82:0/2588922923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd100099a70 0x7fd100099f20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b834e870523f8c29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd104bc3640 1 -- 172.21.15.82:0/2588922923 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b2070 con 0x7fd100099a70 2024-04-25T10:10:41.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd104bc3640 1 -- 172.21.15.82:0/2588922923 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80a5830 con 0x7fd100099a70 2024-04-25T10:10:41.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd104bc3640 1 -- 172.21.15.82:0/2588922923 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80a5c00 con 0x7fd100099a70 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- 172.21.15.82:0/2588922923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd100099a70 msgr2=0x7fd100099f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 --2- 172.21.15.82:0/2588922923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd100099a70 0x7fd100099f20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f801c8e0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- 172.21.15.82:0/2588922923 shutdown_connections 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 --2- 172.21.15.82:0/2588922923 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd100099a70 0x7fd100099f20 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 --2- 172.21.15.82:0/2588922923 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd1000b0900 0x7fd10006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- 172.21.15.82:0/2588922923 >> 172.21.15.82:0/2588922923 conn(0x7fd1000b3000 msgr2=0x7fd1000b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- 172.21.15.82:0/2588922923 shutdown_connections 2024-04-25T10:10:41.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- 172.21.15.82:0/2588922923 wait complete. 2024-04-25T10:10:41.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 Processor -- start 2024-04-25T10:10:41.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.406+0000 7fd107e29640 1 -- start start 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 0x7fd100156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 0x7fd100154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cc4f0 con 0x7fd1000b0900 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1000cc370 con 0x7fd10015d8a0 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10016a7b0 con 0x7fd100099a70 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33108/0 (socket says 172.21.15.82:33108) 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 -- 172.21.15.82:0/176705579 learned_addr learned my addr 172.21.15.82:0/176705579 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd105bc5640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 0x7fd100156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1063c6640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 0x7fd100154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 -- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 msgr2=0x7fd100156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 0x7fd100156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 -- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 msgr2=0x7fd100154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 0x7fd100154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 -- 172.21.15.82:0/176705579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd100155340 con 0x7fd1000b0900 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1063c6640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 0x7fd100154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd105bc5640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 0x7fd100156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd1053c4640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b3f0 tx=0x7fd0f000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 -- 172.21.15.82:0/176705579 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0015020 con 0x7fd1000b0900 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 -- 172.21.15.82:0/176705579 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0011850 con 0x7fd1000b0900 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10015abd0 con 0x7fd1000b0900 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 -- 172.21.15.82:0/176705579 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0011c40 con 0x7fd1000b0900 2024-04-25T10:10:41.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1000b4400 con 0x7fd1000b0900 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 -- 172.21.15.82:0/176705579 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0f0018020 con 0x7fd1000b0900 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 0x7fd0e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd0f67fc640 1 -- 172.21.15.82:0/176705579 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd0f0085150 con 0x7fd1000b0900 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd105bc5640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 0x7fd0e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.410+0000 7fd105bc5640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 0x7fd0e4069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc000f70 tx=0x7fd0fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 Processor -- start 2024-04-25T10:10:41.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- start start 2024-04-25T10:10:41.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7ae4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7a99d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e7980f40 con 0x55d2e7a9a290 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e78e8070 con 0x55d2e7ae3c90 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2e78701a0 con 0x55d2e7ae45b0 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33110/0 (socket says 172.21.15.82:33110) 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 -- 172.21.15.82:0/1549651856 learned_addr learned my addr 172.21.15.82:0/1549651856 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1053c4640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7a99d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd105bc5640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7ae4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 -- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 msgr2=0x55d2e7a99d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7a99d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 -- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 msgr2=0x55d2e7ae4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7ae4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 -- 172.21.15.82:0/1549651856 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e786e510 con 0x55d2e7a9a290 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1053c4640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7a99d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd105bc5640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7ae4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd1063c6640 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd0f801a510 tx=0x7fd0f8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d059402be5a676ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd0e37fe640 1 -- 172.21.15.82:0/1549651856 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80bd020 con 0x55d2e7a9a290 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd0e37fe640 1 -- 172.21.15.82:0/1549651856 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80b3040 con 0x55d2e7a9a290 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd0e37fe640 1 -- 172.21.15.82:0/1549651856 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80b2070 con 0x55d2e7a9a290 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- 172.21.15.82:0/1549651856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 msgr2=0x55d2e7a9a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fd0f801a510 tx=0x7fd0f8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- 172.21.15.82:0/1549651856 shutdown_connections 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7a99d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7a9a290 0x55d2e7a9a670 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- 172.21.15.82:0/1549651856 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7ae4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- 172.21.15.82:0/1549651856 >> 172.21.15.82:0/1549651856 conn(0x55d2e7a96240 msgr2=0x55d2e7a98630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- 172.21.15.82:0/1549651856 shutdown_connections 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- 172.21.15.82:0/1549651856 wait complete. 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 -- start start 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 Processor -- start 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.414+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7a9a290 0x55d2e7aa4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7aa58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e78a26c0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e78e4110 con 0x55d2e7a9a290 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2e799ef20 con 0x55d2e7ae45b0 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 10 client.0 _ll_register_callbacks cb 0x55d2e7a85570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33126/0 (socket says 172.21.15.82:33126) 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 -- 172.21.15.82:0/3026619016 learned_addr learned my addr 172.21.15.82:0/3026619016 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 -- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 msgr2=0x55d2e7aa58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7aa58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 -- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7a9a290 msgr2=0x55d2e7aa4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7a9a290 0x55d2e7aa4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 -- 172.21.15.82:0/3026619016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e78703b0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd1053c4640 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000bdb0 tx=0x7fd0f0059990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4f5e80204dd5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0051eb0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd0f006d1d0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0f80b3040 con 0x55d2e7ae3c90 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0f0083050 con 0x55d2e7ae3c90 2024-04-25T10:10:41.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 10 client.8487 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55d2e7969e00 con 0x55d2e7ae3c90 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80b3040 con 0x55d2e7ae3c90 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd0f0056b70 con 0x55d2e7ae3c90 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 10 client.8487 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 10 client.8487 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 20 client.8487 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aa6150 0x55d2e7af5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 4 client.8487 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2e7883d60 con 0x55d2e7aa6150 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd105bc5640 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aa6150 0x55d2e7af5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd105bc5640 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aa6150 0x55d2e7af5410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cee12b6706694a5 server_cookie=b512e76b03e56cdb in_seq=0 out_seq=0 2024-04-25T10:10:41.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 10 client.8487 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/3026619016 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2e7883d60 con 0x55d2e7aa6150 2024-04-25T10:10:41.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 10 client.8487 handle_command_reply: tid=0 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 client.8487 shutdown 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 msgr2=0x55d2e7aa5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000bdb0 tx=0x7fd0f0059990 comp rx=0 tx=0).stop 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 shutdown_connections 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aa6150 0x55d2e7af5410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7ae45b0 0x55d2e7aa58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd0e0ff9640 0 client.8487 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7aa5370 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 --2- 172.21.15.82:0/3026619016 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7a9a290 0x55d2e7aa4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 >> 172.21.15.82:0/3026619016 conn(0x55d2e7a96240 msgr2=0x55d2e7a98ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 shutdown_connections 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 1 -- 172.21.15.82:0/3026619016 wait complete. 2024-04-25T10:10:41.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.418+0000 7fd10af96000 20 client.8487 trim_cache size 0 max 16384 2024-04-25T10:10:41.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 Processor -- start 2024-04-25T10:10:41.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 -- start start 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7a99640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae45b0 0x55d2e7af5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100203280 con 0x55d2e7ae3c90 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1002023c0 con 0x55d2e7ae45b0 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10015c250 con 0x55d2e7a9a290 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33138/0 (socket says 172.21.15.82:33138) 2024-04-25T10:10:41.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 -- 172.21.15.82:0/2507874178 learned_addr learned my addr 172.21.15.82:0/2507874178 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 -- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 msgr2=0x55d2e7a99640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7a99640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 -- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae45b0 msgr2=0x55d2e7af5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae45b0 0x55d2e7af5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 -- 172.21.15.82:0/2507874178 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2e793fc50 con 0x55d2e7ae3c90 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd1053c4640 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0053d90 tx=0x7fd0f0059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330d1497f352739a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd0e2ffd640 1 -- 172.21.15.82:0/2507874178 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f0053b40 con 0x55d2e7ae3c90 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd0e2ffd640 1 -- 172.21.15.82:0/2507874178 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0056b70 con 0x55d2e7ae3c90 2024-04-25T10:10:41.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd0e2ffd640 1 -- 172.21.15.82:0/2507874178 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f006d1d0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 -- 172.21.15.82:0/2507874178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 msgr2=0x55d2e7a99b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.434+0000 7fd10af96000 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0053d90 tx=0x7fd0f0059000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2507874178 shutdown_connections 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae45b0 0x55d2e7af5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae3c90 0x55d2e7a99b80 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2507874178 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7a99640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2507874178 >> 172.21.15.82:0/2507874178 conn(0x55d2e7a96240 msgr2=0x55d2e7a9e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2507874178 shutdown_connections 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2507874178 wait complete. 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- start start 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 Processor -- start 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7b43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7b44170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2e7a9fb70 con 0x55d2e7ae45b0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd100201df0 con 0x55d2e7ae3c90 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2e7aa3200 con 0x55d2e7a9a290 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 10 client.0 _ll_register_callbacks cb 0x7fd0f80b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33152/0 (socket says 172.21.15.82:33152) 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 -- 172.21.15.82:0/2154048208 learned_addr learned my addr 172.21.15.82:0/2154048208 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 -- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 msgr2=0x55d2e7b43c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7b43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 -- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 msgr2=0x55d2e7b44170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7b44170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 -- 172.21.15.82:0/2154048208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2e78b0120 con 0x55d2e7ae45b0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd1063c6640 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80095c0 tx=0x7fd0f80a5720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51288e9d36b4ea3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:41.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f80bd070 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd0f80bade0 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd0f0056b70 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0f80b2070 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 10 client.8493 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55d2e788f170 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f0056b70 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd0f80a5a90 con 0x55d2e7ae45b0 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 10 client.8493 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 10 client.8493 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 20 client.8493 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aff8b0 0x55d2e7affcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 4 client.8493 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:10:41.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x55d2e78f55b0 con 0x55d2e7aff8b0 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd105bc5640 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aff8b0 0x55d2e7affcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 10 client.8493 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd105bc5640 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aff8b0 0x55d2e7affcc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b06fdeb23f6af04 server_cookie=dfc13f20112d56e7 in_seq=0 out_seq=0 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 1 -- 172.21.15.82:0/2154048208 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d2e78f55b0 con 0x55d2e7aff8b0 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 10 client.8493 handle_command_reply: tid=0 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 client.8493 shutdown 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 msgr2=0x55d2e7afee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80095c0 tx=0x7fd0f80a5720 comp rx=0 tx=0).stop 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 shutdown_connections 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55d2e7aff8b0 0x55d2e7affcc0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d2e7ae45b0 0x55d2e7afee90 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d2e7ae3c90 0x55d2e7b44170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd0e0ff9640 0 client.8493 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 --2- 172.21.15.82:0/2154048208 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d2e7a9a290 0x55d2e7b43c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.438+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 >> 172.21.15.82:0/2154048208 conn(0x55d2e7a96240 msgr2=0x55d2e7a9ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 shutdown_connections 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd10af96000 1 -- 172.21.15.82:0/2154048208 wait complete. 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd10af96000 20 client.8493 trim_cache size 0 max 16384 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "73692f4d-bb83-47f8-9b9f-dd0e308431e7", 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:10:41.448 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:10:41.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 msgr2=0x7fd0e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 0x7fd0e4069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd0fc000f70 tx=0x7fd0fc009040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 msgr2=0x7fd1000b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd0f000b3f0 tx=0x7fd0f000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 shutdown_connections 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd0e4066f60 0x7fd0e4069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd10015d8a0 0x7fd100154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd1000b0900 0x7fd1000b6700 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 --2- 172.21.15.82:0/176705579 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd100099a70 0x7fd100156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 >> 172.21.15.82:0/176705579 conn(0x7fd1000b3000 msgr2=0x7fd10006bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 shutdown_connections 2024-04-25T10:10:41.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.442+0000 7fd107e29640 1 -- 172.21.15.82:0/176705579 wait complete. 2024-04-25T10:10:41.450 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:10:41.442+0000 7faf35985640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55a92a57c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:10:41.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:41.460 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:10:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.570+0000 7f96ba5a3640 1 Processor -- start 2024-04-25T10:10:41.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.570+0000 7f96ba5a3640 1 -- start start 2024-04-25T10:10:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96ba5a3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f96b409a480 0x7f96b40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96ba5a3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f96b40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96ba5a3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b415bd30 con 0x7f96b415b950 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96ba5a3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b40d0510 con 0x7f96b409a480 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96ba5a3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40cc0d0 con 0x7f96b40cfb80 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f96b415b950 0x7f96b40cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59666/0 (socket says 172.21.15.82:59666) 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 -- 172.21.15.82:0/3235125735 learned_addr learned my addr 172.21.15.82:0/3235125735 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b37fe640 1 --2- 172.21.15.82:0/3235125735 >> v2:172.21.15.196:3300/0 conn(0x7f96b409a480 0x7f96b40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b8b40640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f96b40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777735667 0 0) 0x7f96b415bd30 con 0x7f96b415b950 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969c003530 con 0x7f96b415b950 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1385959282 0 0) 0x7f969c003530 con 0x7f96b415b950 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 msgr2=0x7f96b40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f96b40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 >> v2:172.21.15.196:3300/0 conn(0x7f96b409a480 msgr2=0x7f96b40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 --2- 172.21.15.82:0/3235125735 >> v2:172.21.15.196:3300/0 conn(0x7f96b409a480 0x7f96b40cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b409ab10 con 0x7f96b415b950 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f96a4001680 con 0x7f96b415b950 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 >> v1:172.21.15.82:6789/0 conn(0x7f96b415b950 legacy=0x7f96b40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f969c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c004810 0x7f969c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f969c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40d0510 con 0x7f969c003a30 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40cc0d0 con 0x7f969c004810 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00b9d0 con 0x7f96b40cfb80 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f969c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b8b40640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f969c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 -- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 msgr2=0x7f969c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40cfb80 0x7f969c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 -- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c004810 msgr2=0x7f969c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b37fe640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c004810 0x7f969c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f969c004810 0x7f969c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.574+0000 7f96b3fff640 1 -- 172.21.15.82:0/3235125735 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c003530 con 0x7f969c003a30 2024-04-25T10:10:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f969c003e50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c270 tx=0x7f96a401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3570eb1bbaf40436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4007d60 con 0x7f969c003a30 2024-04-25T10:10:41.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a40a9020 con 0x7f969c003a30 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b2ffd640 1 -- 172.21.15.82:0/3235125735 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4005ae0 con 0x7f969c003a30 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/3235125735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 msgr2=0x7f969c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/3235125735 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f969c003e50 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f96a400c270 tx=0x7f96a401ccc0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b8b40640 1 -- 172.21.15.82:0/3235125735 reap_dead start 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/3235125735 shutdown_connections 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/3235125735 >> 172.21.15.82:0/3235125735 conn(0x7f96b40b30b0 msgr2=0x7f96b40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/3235125735 shutdown_connections 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/3235125735 wait complete. 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 Processor -- start 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- start start 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b4154b80 0x7f96b40b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 0x7f96b40c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40cd390 con 0x7f969c003a30 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b406faa0 con 0x7f96b4154b80 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b40c6d00 con 0x7f96b40c7190 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33178/0 (socket says 172.21.15.82:33178) 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 -- 172.21.15.82:0/2296310107 learned_addr learned my addr 172.21.15.82:0/2296310107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b37fe640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b4154b80 0x7f96b40b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b8b40640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 0x7f96b40c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 -- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 msgr2=0x7f96b40c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 0x7f96b40c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 -- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b4154b80 msgr2=0x7f96b40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b4154b80 0x7f96b40b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 -- 172.21.15.82:0/2296310107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40b4850 con 0x7f969c003a30 2024-04-25T10:10:41.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b8b40640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 0x7f96b40c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b3fff640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ab040 tx=0x7f969c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b0ff9640 1 -- 172.21.15.82:0/2296310107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40049f0 con 0x7f969c003a30 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b0ff9640 1 -- 172.21.15.82:0/2296310107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4004b90 con 0x7f969c003a30 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b4155000 con 0x7f969c003a30 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96b0ff9640 1 -- 172.21.15.82:0/2296310107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a40070d0 con 0x7f969c003a30 2024-04-25T10:10:41.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.578+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b40b6ea0 con 0x7f969c003a30 2024-04-25T10:10:41.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b0ff9640 1 -- 172.21.15.82:0/2296310107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96a4007270 con 0x7f969c003a30 2024-04-25T10:10:41.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b0ff9640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 0x7f96a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b0ff9640 1 -- 172.21.15.82:0/2296310107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f96a411e7c0 con 0x7f969c003a30 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b37fe640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 0x7f96a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b37fe640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 0x7f96a0069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f96b4158b50 tx=0x7f96a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 Processor -- start 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 -- start start 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f41112db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f40f15ff0 con 0x558f410c90a0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f40e9e150 con 0x558f411132f0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f40f12090 con 0x558f411129d0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33192/0 (socket says 172.21.15.82:33192) 2024-04-25T10:10:41.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 -- 172.21.15.82:0/1529536893 learned_addr learned my addr 172.21.15.82:0/1529536893 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b3fff640 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f41112db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b37fe640 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 -- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 msgr2=0x558f41112db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f41112db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 -- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 msgr2=0x558f410c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 -- 172.21.15.82:0/1529536893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f410ac0d0 con 0x558f410c90a0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.582+0000 7f96b8b40640 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f96b406eaf0 tx=0x7f96ac009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d83d01cfb9a65e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9699ffb640 1 -- 172.21.15.82:0/1529536893 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac01a020 con 0x558f410c90a0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9699ffb640 1 -- 172.21.15.82:0/1529536893 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac016870 con 0x558f410c90a0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9699ffb640 1 -- 172.21.15.82:0/1529536893 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac016c60 con 0x558f410c90a0 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1529536893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 msgr2=0x558f410c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f96b406eaf0 tx=0x7f96ac009750 comp rx=0 tx=0).stop 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1529536893 shutdown_connections 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f410c9480 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- 172.21.15.82:0/1529536893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f41112db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1529536893 >> 172.21.15.82:0/1529536893 conn(0x558f410c5020 msgr2=0x558f410c7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1529536893 shutdown_connections 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1529536893 wait complete. 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- start start 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 Processor -- start 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 0x558f410f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410f8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f40eda250 con 0x558f411129d0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f40ed0690 con 0x558f411132f0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f40e9ea40 con 0x558f410c90a0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 10 client.0 _ll_register_callbacks cb 0x558f410b3e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33202/0 (socket says 172.21.15.82:33202) 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 -- 172.21.15.82:0/1192321175 learned_addr learned my addr 172.21.15.82:0/1192321175 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b3fff640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 0x558f410f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b8b40640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410f8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 -- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 msgr2=0x558f410f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 0x558f410f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 -- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 msgr2=0x558f410f8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410f8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 -- 172.21.15.82:0/1192321175 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f40fc31c0 con 0x558f411129d0 2024-04-25T10:10:41.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b3fff640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 0x558f410f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b8b40640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b37fe640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f96a8012da0 tx=0x7f96a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3cb69f9b01bd67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8026070 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f96a8022610 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96ac016870 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a8031050 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 10 client.8505 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x558f40fbf170 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac016870 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96a8034020 con 0x558f411129d0 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 10 client.8505 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 10 client.8505 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:41.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 20 client.8505 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f41122c80 0x558f41125070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 4 client.8505 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f40eb1c60 con 0x558f41122c80 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.586+0000 7f96b3fff640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f41122c80 0x558f41125070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f9692ffd640 10 client.8505 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96b3fff640 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f41122c80 0x558f41125070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2cff0b1f84a643e server_cookie=de1c3c8f272a644d in_seq=0 out_seq=0 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f9692ffd640 1 -- 172.21.15.82:0/1192321175 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558f40eb1c60 con 0x558f41122c80 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f9692ffd640 10 client.8505 handle_command_reply: tid=0 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 client.8505 shutdown 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 msgr2=0x558f410f7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f96a8012da0 tx=0x7f96a8004740 comp rx=0 tx=0).stop 2024-04-25T10:10:41.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 shutdown_connections 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f41122c80 0x558f41125070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f410f8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f410f7d30 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 --2- 172.21.15.82:0/1192321175 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f410c90a0 0x558f410f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f9692ffd640 0 client.8505 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 >> 172.21.15.82:0/1192321175 conn(0x558f410c5020 msgr2=0x558f410c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 shutdown_connections 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 1 -- 172.21.15.82:0/1192321175 wait complete. 2024-04-25T10:10:41.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.590+0000 7f96bd710000 20 client.8505 trim_cache size 0 max 16384 2024-04-25T10:10:41.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 Processor -- start 2024-04-25T10:10:41.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- start start 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 0x558f4116f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411132f0 0x558f4116fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4201ca0 con 0x558f411129d0 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4200e50 con 0x558f410c90a0 2024-04-25T10:10:41.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b415e360 con 0x558f411132f0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33206/0 (socket says 172.21.15.82:33206) 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 -- 172.21.15.82:0/37282649 learned_addr learned my addr 172.21.15.82:0/37282649 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b3fff640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 0x558f4116f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b8b40640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411132f0 0x558f4116fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 -- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411132f0 msgr2=0x558f4116fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411132f0 0x558f4116fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 -- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 msgr2=0x558f4116f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 0x558f4116f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 -- 172.21.15.82:0/37282649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f40f5cde0 con 0x558f411129d0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b3fff640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 0x558f4116f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b37fe640 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f96a8015fd0 tx=0x7f96a8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856e924226d0617d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96997fa640 1 -- 172.21.15.82:0/37282649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8026070 con 0x558f411129d0 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96997fa640 1 -- 172.21.15.82:0/37282649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a801b040 con 0x558f411129d0 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96997fa640 1 -- 172.21.15.82:0/37282649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800f040 con 0x558f411129d0 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- 172.21.15.82:0/37282649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 msgr2=0x558f4116f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f96a8015fd0 tx=0x7f96a8022000 comp rx=0 tx=0).stop 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- 172.21.15.82:0/37282649 shutdown_connections 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411132f0 0x558f4116fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f411129d0 0x558f4116f630 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- 172.21.15.82:0/37282649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f410c90a0 0x558f4116f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- 172.21.15.82:0/37282649 >> 172.21.15.82:0/37282649 conn(0x558f410c5020 msgr2=0x558f4111ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- 172.21.15.82:0/37282649 shutdown_connections 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- 172.21.15.82:0/37282649 wait complete. 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- start start 2024-04-25T10:10:41.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 Processor -- start 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f411258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f41125e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f40eb0e00 con 0x558f410c90a0 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b4200850 con 0x558f411132f0 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x558f40f15300 con 0x558f411129d0 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96bd710000 10 client.0 _ll_register_callbacks cb 0x558f41094040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33220/0 (socket says 172.21.15.82:33220) 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.606+0000 7f96b3fff640 1 -- 172.21.15.82:0/546156656 learned_addr learned my addr 172.21.15.82:0/546156656 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b37fe640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f411258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b8b40640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f41125e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 -- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 msgr2=0x558f411258c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f411258c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 -- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 msgr2=0x558f41125e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f41125e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 -- 172.21.15.82:0/546156656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558f40eb78e0 con 0x558f410c90a0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b3fff640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f96a40eb280 tx=0x7f96a40f5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad66f46aced826fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a411ce60 con 0x558f410c90a0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f96a40ed120 con 0x558f410c90a0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f96a801b040 con 0x558f410c90a0 2024-04-25T10:10:41.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f96a410b050 con 0x558f410c90a0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 10 client.8511 fetch_fsmap learned FSMap version 69 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x558f40f6f8f0 con 0x558f410c90a0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a801b040 con 0x558f410c90a0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f96a40b6070 con 0x558f410c90a0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 10 client.8511 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 10 client.8511 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 20 client.8511 populate_metadata read hostname 'smithi082' 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f411267c0 0x558f4117af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 4 client.8511 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558f40f23530 con 0x558f411267c0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b37fe640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f411267c0 0x558f4117af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 10 client.8511 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96b37fe640 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f411267c0 0x558f4117af60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a98194dd4ea4d9f1 server_cookie=249e859d34329f3e in_seq=0 out_seq=0 2024-04-25T10:10:41.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 1 -- 172.21.15.82:0/546156656 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x558f40f23530 con 0x558f411267c0 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 10 client.8511 handle_command_reply: tid=0 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 client.8511 shutdown 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 msgr2=0x558f41125380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f96a40eb280 tx=0x7f96a40f5040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 shutdown_connections 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x558f411267c0 0x558f4117af60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x558f411129d0 0x558f411258c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f9692ffd640 0 client.8511 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x558f411132f0 0x558f41125e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 --2- 172.21.15.82:0/546156656 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x558f410c90a0 0x558f41125380 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 >> 172.21.15.82:0/546156656 conn(0x558f410c5020 msgr2=0x558f411110d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 shutdown_connections 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 1 -- 172.21.15.82:0/546156656 wait complete. 2024-04-25T10:10:41.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96bd710000 20 client.8511 trim_cache size 0 max 16384 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "73692f4d-bb83-47f8-9b9f-dd0e308431e7": { 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/dir", 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "73692f4d-bb83-47f8-9b9f-dd0e308431e7", 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:10:41.620 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:10:41.621 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:10:41.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 msgr2=0x7f96a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 0x7f96a0069450 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f96b4158b50 tx=0x7f96a8008040 comp rx=0 tx=0).stop 2024-04-25T10:10:41.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 msgr2=0x7f96b40c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:41.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f96a40ab040 tx=0x7f969c005840 comp rx=0 tx=0).stop 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 shutdown_connections 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f96a0066f90 0x7f96a0069450 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f96b40c7190 0x7f96b40c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f96b4154b80 0x7f96b40b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 --2- 172.21.15.82:0/2296310107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f969c003a30 0x7f96b40c8690 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 >> 172.21.15.82:0/2296310107 conn(0x7f96b40b30b0 msgr2=0x7f96b4073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 shutdown_connections 2024-04-25T10:10:41.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:10:41.610+0000 7f96ba5a3640 1 -- 172.21.15.82:0/2296310107 wait complete. 2024-04-25T10:10:41.629 INFO:tasks.cephfs.filesystem:scrub status for tag:73692f4d-bb83-47f8-9b9f-dd0e308431e7 - {'path': '/dir', 'tag': '73692f4d-bb83-47f8-9b9f-dd0e308431e7', 'options': 'recursive,repair'} 2024-04-25T10:11:08.277 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:08.285 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:11.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:11.631 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:11:11.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 Processor -- start 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 -- start start 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f99dc0cf590 0x7f99dc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99dc072000 0x7f99dc072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99dc0cbd20 con 0x7f99dc15b6b0 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc15ba90 con 0x7f99dc072000 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e3a87640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99dc0cc000 con 0x7f99dc0cf590 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99dc072000 0x7f99dc072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f99dc072000 0x7f99dc072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33750/0 (socket says 172.21.15.82:33750) 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 -- 172.21.15.82:0/3627077284 learned_addr learned my addr 172.21.15.82:0/3627077284 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e1022640 1 --2- 172.21.15.82:0/3627077284 >> v2:172.21.15.196:3300/0 conn(0x7f99dc0cf590 0x7f99dc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 -- 172.21.15.82:0/3627077284 >> v2:172.21.15.196:3300/0 conn(0x7f99dc0cf590 msgr2=0x7f99dc0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 --2- 172.21.15.82:0/3627077284 >> v2:172.21.15.196:3300/0 conn(0x7f99dc0cf590 0x7f99dc0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 -- 172.21.15.82:0/3627077284 >> v1:172.21.15.82:6789/0 conn(0x7f99dc15b6b0 legacy=0x7f99dc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.765+0000 7f99e2024640 1 -- 172.21.15.82:0/3627077284 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc0cc2e0 con 0x7f99dc072000 2024-04-25T10:11:11.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e2024640 1 --2- 172.21.15.82:0/3627077284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc072000 0x7f99dc072430 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b4430 tx=0x7f99d4020e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607fa64eb701e96b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e0821640 1 -- 172.21.15.82:0/3627077284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40090d0 con 0x7f99dc072000 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e0821640 1 -- 172.21.15.82:0/3627077284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40a9be0 con 0x7f99dc072000 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e0821640 1 -- 172.21.15.82:0/3627077284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4009270 con 0x7f99dc072000 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 -- 172.21.15.82:0/3627077284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc072000 msgr2=0x7f99dc072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 --2- 172.21.15.82:0/3627077284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc072000 0x7f99dc072430 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f99dc0b4430 tx=0x7f99d4020e00 comp rx=0 tx=0).stop 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 -- 172.21.15.82:0/3627077284 shutdown_connections 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 --2- 172.21.15.82:0/3627077284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc072000 0x7f99dc072430 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 --2- 172.21.15.82:0/3627077284 >> v2:172.21.15.196:3300/0 conn(0x7f99dc0cf590 0x7f99dc0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 -- 172.21.15.82:0/3627077284 >> 172.21.15.82:0/3627077284 conn(0x7f99dc06ae60 msgr2=0x7f99dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 -- 172.21.15.82:0/3627077284 shutdown_connections 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.769+0000 7f99e3a87640 1 -- 172.21.15.82:0/3627077284 wait complete. 2024-04-25T10:11:11.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 Processor -- start 2024-04-25T10:11:11.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- start start 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99dc072000 0x7f99dc098730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 0x7f99dc0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0caf30 con 0x7f99dc15b6b0 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc0cadb0 con 0x7f99dc072000 2024-04-25T10:11:11.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc166290 con 0x7f99dc0cf590 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33752/0 (socket says 172.21.15.82:33752) 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 -- 172.21.15.82:0/1103524813 learned_addr learned my addr 172.21.15.82:0/1103524813 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e1823640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99dc072000 0x7f99dc098730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e1022640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 0x7f99dc0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 -- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 msgr2=0x7f99dc0ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 0x7f99dc0ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 -- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99dc072000 msgr2=0x7f99dc098730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99dc072000 0x7f99dc098730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 -- 172.21.15.82:0/1103524813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc06de30 con 0x7f99dc15b6b0 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e1022640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 0x7f99dc0ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:11.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e2024640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f99d40b3040 tx=0x7f99d401dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99ce7fc640 1 -- 172.21.15.82:0/1103524813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d4009510 con 0x7f99dc15b6b0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99ce7fc640 1 -- 172.21.15.82:0/1103524813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40c2070 con 0x7f99dc15b6b0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc098c70 con 0x7f99dc15b6b0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99ce7fc640 1 -- 172.21.15.82:0/1103524813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40bc6b0 con 0x7f99dc15b6b0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc06e0b0 con 0x7f99dc15b6b0 2024-04-25T10:11:11.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.773+0000 7f99ce7fc640 1 -- 172.21.15.82:0/1103524813 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99d40bc850 con 0x7f99dc15b6b0 2024-04-25T10:11:11.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99ce7fc640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99ce7fc640 1 -- 172.21.15.82:0/1103524813 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f99d4124b40 con 0x7f99dc15b6b0 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e1823640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e1823640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 0x7f99c0069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f99d80015b0 tx=0x7f99d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 Processor -- start 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- start start 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b59d0 0x56523d1b5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 0x56523d16bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cfb8ff0 con 0x56523d16c0a0 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cf41150 con 0x56523d1b62f0 2024-04-25T10:11:11.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523cfb5090 con 0x56523d1b59d0 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33768/0 (socket says 172.21.15.82:33768) 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 -- 172.21.15.82:0/2321736679 learned_addr learned my addr 172.21.15.82:0/2321736679 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e1022640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 0x56523d16bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e1823640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b59d0 0x56523d1b5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 -- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b59d0 msgr2=0x56523d1b5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b59d0 0x56523d1b5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 -- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 msgr2=0x56523d16bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 0x56523d16bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 -- 172.21.15.82:0/2321736679 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523d14f0d0 con 0x56523d16c0a0 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e1022640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 0x56523d16bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:11.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e2024640 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f99d40f4fc0 tx=0x7f99d40f1690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4782bebb936664a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99bf7fe640 1 -- 172.21.15.82:0/2321736679 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40f59c0 con 0x56523d16c0a0 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99bf7fe640 1 -- 172.21.15.82:0/2321736679 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40f52c0 con 0x56523d16c0a0 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99bf7fe640 1 -- 172.21.15.82:0/2321736679 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d40f9410 con 0x56523d16c0a0 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2321736679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 msgr2=0x56523d16c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f99d40f4fc0 tx=0x7f99d40f1690 comp rx=0 tx=0).stop 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2321736679 shutdown_connections 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d16c480 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b62f0 0x56523d16bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2321736679 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b59d0 0x56523d1b5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2321736679 >> 172.21.15.82:0/2321736679 conn(0x56523d168020 msgr2=0x56523d16a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2321736679 shutdown_connections 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2321736679 wait complete. 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.777+0000 7f99e6bf4000 1 -- start start 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 Processor -- start 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d19a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d19b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cf7d250 con 0x56523d1b59d0 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cf73690 con 0x56523d16c0a0 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523cf41a40 con 0x56523d1b62f0 2024-04-25T10:11:11.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 10 client.0 _ll_register_callbacks cb 0x56523d156e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33770/0 (socket says 172.21.15.82:33770) 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 -- 172.21.15.82:0/2311042225 learned_addr learned my addr 172.21.15.82:0/2311042225 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1823640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d19a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e2024640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 -- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 msgr2=0x56523d19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d19b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 -- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 msgr2=0x56523d19a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d19a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 -- 172.21.15.82:0/2311042225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523d0661c0 con 0x56523d1b59d0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1022640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f99d000df40 tx=0x7f99d000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c56dad3a3182931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0018070 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f99d0014690 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d40f52c0 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d0023050 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 10 client.8526 fetch_fsmap learned FSMap version 69 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x56523d062170 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d40f52c0 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99d0026020 con 0x56523d1b59d0 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 10 client.8526 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 10 client.8526 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 20 client.8526 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:11.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d1c5c80 0x56523d1c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 4 client.8526 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56523cf54c60 con 0x56523d1c5c80 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1823640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d1c5c80 0x56523d1c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 10 client.8526 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e1823640 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d1c5c80 0x56523d1c8070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7de031111bebecec server_cookie=8c43c6673a233c6 in_seq=0 out_seq=0 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 1 -- 172.21.15.82:0/2311042225 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56523cf54c60 con 0x56523d1c5c80 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 10 client.8526 handle_command_reply: tid=0 2024-04-25T10:11:11.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 client.8526 shutdown 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 msgr2=0x56523d19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f99d000df40 tx=0x7f99d000f670 comp rx=0 tx=0).stop 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 shutdown_connections 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d1c5c80 0x56523d1c8070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d19ad30 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/2311042225 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d19a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99bcff9640 0 client.8526 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 >> 172.21.15.82:0/2311042225 conn(0x56523d168020 msgr2=0x56523d16aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 shutdown_connections 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 1 -- 172.21.15.82:0/2311042225 wait complete. 2024-04-25T10:11:11.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.781+0000 7f99e6bf4000 20 client.8526 trim_cache size 0 max 16384 2024-04-25T10:11:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 Processor -- start 2024-04-25T10:11:11.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 -- start start 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d2120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d212b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc1ff060 con 0x56523d1b59d0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc1fe1e0 con 0x56523d16c0a0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc1fd700 con 0x56523d1b62f0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33778/0 (socket says 172.21.15.82:33778) 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 -- 172.21.15.82:0/3636487913 learned_addr learned my addr 172.21.15.82:0/3636487913 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1823640 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d2120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e2024640 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d212b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 -- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 msgr2=0x56523d212b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d212b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 -- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 msgr2=0x56523d2120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d2120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.797+0000 7f99e1022640 1 -- 172.21.15.82:0/3636487913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56523cfffde0 con 0x56523d1b59d0 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1022640 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f99d000df10 tx=0x7f99d0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef7eda7ea47dabdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99beffd640 1 -- 172.21.15.82:0/3636487913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d002d070 con 0x56523d1b59d0 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99beffd640 1 -- 172.21.15.82:0/3636487913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0000e50 con 0x56523d1b59d0 2024-04-25T10:11:11.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99beffd640 1 -- 172.21.15.82:0/3636487913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0018040 con 0x56523d1b59d0 2024-04-25T10:11:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/3636487913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 msgr2=0x56523d212630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f99d000df10 tx=0x7f99d0014000 comp rx=0 tx=0).stop 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/3636487913 shutdown_connections 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d212b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d1b59d0 0x56523d212630 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/3636487913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d16c0a0 0x56523d2120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/3636487913 >> 172.21.15.82:0/3636487913 conn(0x56523d168020 msgr2=0x56523d1c2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/3636487913 shutdown_connections 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/3636487913 wait complete. 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- start start 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 Processor -- start 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b59d0 0x56523d1c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d1c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56523cf53e00 con 0x56523d16c0a0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc1fdbe0 con 0x56523d1b59d0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56523cfb8300 con 0x56523d1b62f0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 10 client.0 _ll_register_callbacks cb 0x56523d137040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33790/0 (socket says 172.21.15.82:33790) 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 -- 172.21.15.82:0/1871471239 learned_addr learned my addr 172.21.15.82:0/1871471239 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1022640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b59d0 0x56523d1c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e2024640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d1c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 -- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 msgr2=0x56523d1c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d1c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 -- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b59d0 msgr2=0x56523d1c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b59d0 0x56523d1c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 -- 172.21.15.82:0/1871471239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56523cf5a8e0 con 0x56523d16c0a0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1823640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f99d80243f0 tx=0x7f99d8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f927fdbb8054ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003d10 con 0x56523d16c0a0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f99d8010040 con 0x56523d16c0a0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99d0000e50 con 0x56523d16c0a0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99d800e4b0 con 0x56523d16c0a0 2024-04-25T10:11:11.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 10 client.8532 fetch_fsmap learned FSMap version 69 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x56523d0128f0 con 0x56523d16c0a0 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0000e50 con 0x56523d16c0a0 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f99d8036050 con 0x56523d16c0a0 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 10 client.8532 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 10 client.8532 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.c 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 20 client.8532 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d21bf50 0x56523d21e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 4 client.8532 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56523cfc6530 con 0x56523d21bf50 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.801+0000 7f99e1022640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d21bf50 0x56523d21e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e1022640 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d21bf50 0x56523d21e340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7bbd060b807055 server_cookie=9b2d28fc77097934 in_seq=0 out_seq=0 2024-04-25T10:11:11.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99bcff9640 10 client.8532 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99bcff9640 1 -- 172.21.15.82:0/1871471239 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56523cfc6530 con 0x56523d21bf50 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99bcff9640 10 client.8532 handle_command_reply: tid=0 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 client.8532 shutdown 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 msgr2=0x56523d1c8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f99d80243f0 tx=0x7f99d8000bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 shutdown_connections 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x56523d21bf50 0x56523d21e340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56523d1b62f0 0x56523d1c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56523d1b59d0 0x56523d1c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99bcff9640 0 client.8532 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:11.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 --2- 172.21.15.82:0/1871471239 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56523d16c0a0 0x56523d1c8380 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 >> 172.21.15.82:0/1871471239 conn(0x56523d168020 msgr2=0x56523d1b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 shutdown_connections 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 1 -- 172.21.15.82:0/1871471239 wait complete. 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e6bf4000 20 client.8532 trim_cache size 0 max 16384 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:11:11.815 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:11:11.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 msgr2=0x7f99c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 0x7f99c0069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f99d80015b0 tx=0x7f99d8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 msgr2=0x7f99dc0acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f99d40b3040 tx=0x7f99d401dba0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 shutdown_connections 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99c0066f60 0x7f99c0069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f99dc0cf590 0x7f99dc0ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f99dc15b6b0 0x7f99dc0acd40 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 --2- 172.21.15.82:0/1103524813 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f99dc072000 0x7f99dc098730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 >> 172.21.15.82:0/1103524813 conn(0x7f99dc06ae60 msgr2=0x7f99dc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 shutdown_connections 2024-04-25T10:11:11.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.805+0000 7f99e3a87640 1 -- 172.21.15.82:0/1103524813 wait complete. 2024-04-25T10:11:11.824 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:11.825 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:11.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.933+0000 7f9e4d736640 1 Processor -- start 2024-04-25T10:11:11.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.933+0000 7f9e4d736640 1 -- start start 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e4d736640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e480d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e4d736640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4806be60 con 0x7f9e480d1cd0 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e4d736640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e4815de60 con 0x7f9e4815da80 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e4d736640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4806c440 con 0x7f9e480d07f0 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e477fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e480d1cd0 0x7f9e4806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51658/0 (socket says 172.21.15.82:51658) 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e477fe640 1 -- 172.21.15.82:0/3783346376 learned_addr learned my addr 172.21.15.82:0/3783346376 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e480d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849810702 0 0) 0x7f9e4806be60 con 0x7f9e480d1cd0 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e30003530 con 0x7f9e480d1cd0 2024-04-25T10:11:11.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4292818121 0 0) 0x7f9e30003530 con 0x7f9e480d1cd0 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 msgr2=0x7f9e480d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e480d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 >> v1:172.21.15.196:6789/0 conn(0x7f9e4815da80 legacy=0x7f9e480d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480d2450 con 0x7f9e480d1cd0 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9e40001a00 con 0x7f9e480d1cd0 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 >> v1:172.21.15.82:6789/0 conn(0x7f9e480d1cd0 legacy=0x7f9e4806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30003a60 0x7f9e30003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4806c440 con 0x7f9e30004810 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000bb00 con 0x7f9e30003a60 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3000bd00 con 0x7f9e480d07f0 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e46ffd640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30003a60 0x7f9e30003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e477fe640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 -- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 msgr2=0x7f9e300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480d07f0 0x7f9e300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 -- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30003a60 msgr2=0x7f9e30003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30003a60 0x7f9e30003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.937+0000 7f9e467fc640 1 -- 172.21.15.82:0/3783346376 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e30003530 con 0x7f9e30004810 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009ac0 tx=0x7f9e3801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112de4c4f3554338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380ac070 con 0x7f9e30004810 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e380059a0 con 0x7f9e30004810 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e45ffb640 1 -- 172.21.15.82:0/3783346376 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38009690 con 0x7f9e30004810 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/3783346376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e30004810 msgr2=0x7f9e30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 --2- 172.21.15.82:0/3783346376 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e30004810 0x7f9e30004c90 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009ac0 tx=0x7f9e3801ca70 comp rx=0 tx=0).stop 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e477fe640 1 -- 172.21.15.82:0/3783346376 reap_dead start 2024-04-25T10:11:11.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/3783346376 shutdown_connections 2024-04-25T10:11:11.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/3783346376 >> 172.21.15.82:0/3783346376 conn(0x7f9e4808b000 msgr2=0x7f9e480d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/3783346376 shutdown_connections 2024-04-25T10:11:11.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/3783346376 wait complete. 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 Processor -- start 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- start start 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30004810 0x7f9e4815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 0x7f9e481f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480d26f0 con 0x7f9e480b70a0 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480d2160 con 0x7f9e30004810 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e480d2870 con 0x7f9e480ccd10 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33798/0 (socket says 172.21.15.82:33798) 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 -- 172.21.15.82:0/2316730048 learned_addr learned my addr 172.21.15.82:0/2316730048 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e46ffd640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30004810 0x7f9e4815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e477fe640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 0x7f9e481f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 -- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 msgr2=0x7f9e481f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 0x7f9e481f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 -- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30004810 msgr2=0x7f9e4815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30004810 0x7f9e4815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 -- 172.21.15.82:0/2316730048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e481f4020 con 0x7f9e480b70a0 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e477fe640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 0x7f9e481f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:11.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e467fc640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009ac0 tx=0x7f9e30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380ac040 con 0x7f9e480b70a0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38004220 con 0x7f9e480b70a0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e380063c0 con 0x7f9e480b70a0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.941+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e480b62e0 con 0x7f9e480b70a0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e480b68a0 con 0x7f9e480b70a0 2024-04-25T10:11:11.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4804efa0 con 0x7f9e480b70a0 2024-04-25T10:11:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e38006560 con 0x7f9e480b70a0 2024-04-25T10:11:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e2f7fe640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 0x7f9e34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f9e3811f3f0 con 0x7f9e480b70a0 2024-04-25T10:11:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e46ffd640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 0x7f9e34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.945+0000 7f9e46ffd640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 0x7f9e34069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c10 tx=0x7f9e3c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:11.949+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e380b5030 con 0x7f9e480b70a0 2024-04-25T10:11:12.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e480d1550 con 0x7f9e480b70a0 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e2f7fe640 1 -- 172.21.15.82:0/2316730048 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9e38016780 con 0x7f9e480b70a0 2024-04-25T10:11:12.139 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:12.140 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:12.140 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 msgr2=0x7f9e34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 0x7f9e34069450 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c000c10 tx=0x7f9e3c009040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 msgr2=0x7f9e480cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009ac0 tx=0x7f9e30005840 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 shutdown_connections 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f90 0x7f9e34069450 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e480ccd10 0x7f9e481f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e480b70a0 0x7f9e480cc7d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 --2- 172.21.15.82:0/2316730048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e30004810 0x7f9e4815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 >> 172.21.15.82:0/2316730048 conn(0x7f9e4808b000 msgr2=0x7f9e4815e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 shutdown_connections 2024-04-25T10:11:12.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.137+0000 7f9e4d736640 1 -- 172.21.15.82:0/2316730048 wait complete. 2024-04-25T10:11:12.157 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T10:11:12.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.261+0000 7f66fa368640 1 Processor -- start 2024-04-25T10:11:12.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.261+0000 7f66fa368640 1 -- start start 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66fa368640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66f4159fb0 0x7f66f40cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66fa368640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 0x7f66f40cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66fa368640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40cc2b0 con 0x7f66f4159bd0 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66fa368640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f4072780 con 0x7f66f4159fb0 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66fa368640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f4071b70 con 0x7f66f40cb8c0 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66f4159fb0 0x7f66f40cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f66f4159bd0 0x7f66f40c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51664/0 (socket says 172.21.15.82:51664) 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f66f4159fb0 0x7f66f40cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33804/0 (socket says 172.21.15.82:33804) 2024-04-25T10:11:12.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f3fff640 1 -- 172.21.15.82:0/148152010 learned_addr learned my addr 172.21.15.82:0/148152010 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f8905640 1 --2- 172.21.15.82:0/148152010 >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 0x7f66f40cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 -- 172.21.15.82:0/148152010 >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 msgr2=0x7f66f40cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/148152010 >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 0x7f66f40cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 -- 172.21.15.82:0/148152010 >> v1:172.21.15.82:6789/0 conn(0x7f66f4159bd0 legacy=0x7f66f40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 -- 172.21.15.82:0/148152010 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4071e50 con 0x7f66f4159fb0 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f37fe640 1 -- 172.21.15.82:0/148152010 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1669048484 0 0) 0x7f66e4001120 con 0x7f66f4159bd0 2024-04-25T10:11:12.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f8905640 1 --2- 172.21.15.82:0/148152010 >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 0x7f66f40cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/148152010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f4159fb0 0x7f66f40cb380 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f66f4072b80 tx=0x7f66dc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd65cfd51900dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f37fe640 1 -- 172.21.15.82:0/148152010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0b1070 con 0x7f66f4159fb0 2024-04-25T10:11:12.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f37fe640 1 -- 172.21.15.82:0/148152010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc0ac630 con 0x7f66f4159fb0 2024-04-25T10:11:12.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.265+0000 7f66f37fe640 1 -- 172.21.15.82:0/148152010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0a8050 con 0x7f66f4159fb0 2024-04-25T10:11:12.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/148152010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f4159fb0 msgr2=0x7f66f40cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- 172.21.15.82:0/148152010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f4159fb0 0x7f66f40cb380 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f66f4072b80 tx=0x7f66dc01c450 comp rx=0 tx=0).stop 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/148152010 shutdown_connections 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- 172.21.15.82:0/148152010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f4159fb0 0x7f66f40cb380 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- 172.21.15.82:0/148152010 >> v2:172.21.15.196:3300/0 conn(0x7f66f40cb8c0 0x7f66f40cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/148152010 >> 172.21.15.82:0/148152010 conn(0x7f66f406ae60 msgr2=0x7f66f406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/148152010 shutdown_connections 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/148152010 wait complete. 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 Processor -- start 2024-04-25T10:11:12.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- start start 2024-04-25T10:11:12.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 0x7f66f40b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 0x7f66f40b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4163ae0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4163960 con 0x7f66f4159fb0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4163c60 con 0x7f66f4159bd0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33818/0 (socket says 172.21.15.82:33818) 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 -- 172.21.15.82:0/41286657 learned_addr learned my addr 172.21.15.82:0/41286657 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 0x7f66f40b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f8905640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 0x7f66f40b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 -- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 msgr2=0x7f66f40b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 0x7f66f40b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 -- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 msgr2=0x7f66f40b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 0x7f66f40b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 -- 172.21.15.82:0/41286657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f40b16c0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f8905640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 0x7f66f40b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 0x7f66f40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f3fff640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c7b0 tx=0x7f66e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f17fa640 1 -- 172.21.15.82:0/41286657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4018070 con 0x7f66f40cb8c0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f17fa640 1 -- 172.21.15.82:0/41286657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e40072f0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66f17fa640 1 -- 172.21.15.82:0/41286657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40047f0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f40b19a0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.269+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4151020 con 0x7f66f40cb8c0 2024-04-25T10:11:12.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f17fa640 1 -- 172.21.15.82:0/41286657 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f66e401b070 con 0x7f66f40cb8c0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f17fa640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f17fa640 1 -- 172.21.15.82:0/41286657 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f66e40847a0 con 0x7f66f40cb8c0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 0x7f66d4069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0ae000 tx=0x7f66dc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 Processor -- start 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- start start 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933ac00 0x55c1092f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c1092f0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1091d7ba0 con 0x55c10933a2e0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c10913d090 con 0x55c1092f08e0 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1090c5160 con 0x55c10933ac00 2024-04-25T10:11:12.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33822/0 (socket says 172.21.15.82:33822) 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 -- 172.21.15.82:0/2885853029 learned_addr learned my addr 172.21.15.82:0/2885853029 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 -- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933ac00 msgr2=0x55c1092f03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933ac00 0x55c1092f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 -- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 msgr2=0x55c1092f0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c1092f0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 -- 172.21.15.82:0/2885853029 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1091e6c20 con 0x55c10933a2e0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66f3fff640 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f66e405cf40 tx=0x7f66e40547f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36b2db33560d59e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66e9ffb640 1 -- 172.21.15.82:0/2885853029 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4058eb0 con 0x55c10933a2e0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66e9ffb640 1 -- 172.21.15.82:0/2885853029 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e405d470 con 0x55c10933a2e0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66e9ffb640 1 -- 172.21.15.82:0/2885853029 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e405b070 con 0x55c10933a2e0 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- 172.21.15.82:0/2885853029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 msgr2=0x55c10933a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f66e405cf40 tx=0x7f66e40547f0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- 172.21.15.82:0/2885853029 shutdown_connections 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933ac00 0x55c1092f03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c1092f0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/2885853029 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c10933a6c0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- 172.21.15.82:0/2885853029 >> 172.21.15.82:0/2885853029 conn(0x55c1092ec890 msgr2=0x55c1092eec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- 172.21.15.82:0/2885853029 shutdown_connections 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.273+0000 7f66fd4d5000 1 -- 172.21.15.82:0/2885853029 wait complete. 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- start start 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 Processor -- start 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109341640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c109341b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c109139130 con 0x55c10933ac00 2024-04-25T10:11:12.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c109101130 con 0x55c1092f08e0 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1091fb300 con 0x55c10933a2e0 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 10 client.0 _ll_register_callbacks cb 0x55c1092dbbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33830/0 (socket says 172.21.15.82:33830) 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 -- 172.21.15.82:0/1517845134 learned_addr learned my addr 172.21.15.82:0/1517845134 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f3fff640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109341640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 -- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 msgr2=0x55c109341b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c109341b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 -- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 msgr2=0x55c109341640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109341640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 -- 172.21.15.82:0/1517845134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c10911a060 con 0x55c10933ac00 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f3fff640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109341640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f8905640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00eac0 tx=0x7f66ec007f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9d6b61c974f0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00ce50 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f66ec01a480 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66e405d470 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec022050 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 10 client.8553 fetch_fsmap learned FSMap version 69 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c1090de7c0 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e405d470 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f66ec025020 con 0x55c10933ac00 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 10 client.8553 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 10 client.8553 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 20 client.8553 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c109328e00 0x55c1093291e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 4 client.8553 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1090d8cd0 con 0x55c109328e00 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f3fff640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c109328e00 0x55c1093291e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 10 client.8553 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66f3fff640 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c109328e00 0x55c1093291e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d184339939ff40 server_cookie=e5144bd61ab02af9 in_seq=0 out_seq=0 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 1 -- 172.21.15.82:0/1517845134 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c1090d8cd0 con 0x55c109328e00 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 10 client.8553 handle_command_reply: tid=0 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 client.8553 shutdown 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 msgr2=0x55c109328090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00eac0 tx=0x7f66ec007f80 comp rx=0 tx=0).stop 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 shutdown_connections 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c109328e00 0x55c1093291e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c109341b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933ac00 0x55c109328090 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66d2ffd640 0 client.8553 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:12.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1517845134 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109341640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 >> 172.21.15.82:0/1517845134 conn(0x55c1092ec890 msgr2=0x55c1092ef510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 shutdown_connections 2024-04-25T10:11:12.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1517845134 wait complete. 2024-04-25T10:11:12.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.277+0000 7f66fd4d5000 20 client.8553 trim_cache size 0 max 16384 2024-04-25T10:11:12.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 Processor -- start 2024-04-25T10:11:12.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 -- start start 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1092ec890 0x55c109396500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109396a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41fdfe0 con 0x55c10933a2e0 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41fd3c0 con 0x55c1092f08e0 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f41fd9c0 con 0x55c1092ec890 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66f8905640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.289+0000 7f66f8905640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33842/0 (socket says 172.21.15.82:33842) 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 -- 172.21.15.82:0/1589581940 learned_addr learned my addr 172.21.15.82:0/1589581940 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109396a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 -- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1092ec890 msgr2=0x55c109396500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1092ec890 0x55c109396500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 -- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 msgr2=0x55c109396a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109396a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 -- 172.21.15.82:0/1589581940 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1090f8fc0 con 0x55c10933a2e0 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f8905640 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f410 tx=0x7f66ec011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdd41b8109aa0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66e97fa640 1 -- 172.21.15.82:0/1589581940 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec0039a0 con 0x55c10933a2e0 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66e97fa640 1 -- 172.21.15.82:0/1589581940 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec02a070 con 0x55c10933a2e0 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66e97fa640 1 -- 172.21.15.82:0/1589581940 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec003d60 con 0x55c10933a2e0 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1589581940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 msgr2=0x55c109396f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00f410 tx=0x7f66ec011000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1589581940 shutdown_connections 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c10933a2e0 0x55c109396f80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092f08e0 0x55c109396a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/1589581940 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1092ec890 0x55c109396500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1589581940 >> 172.21.15.82:0/1589581940 conn(0x55c1092f5be0 msgr2=0x55c109339260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1589581940 shutdown_connections 2024-04-25T10:11:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/1589581940 wait complete. 2024-04-25T10:11:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- start start 2024-04-25T10:11:12.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 Processor -- start 2024-04-25T10:11:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 0x55c10939a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c10933e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1090d7e70 con 0x55c1092f08e0 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1090c5a50 con 0x55c1092ec890 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c10913c3a0 con 0x55c10933a2e0 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 10 client.0 _ll_register_callbacks cb 0x55c1090e9240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33846/0 (socket says 172.21.15.82:33846) 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 -- 172.21.15.82:0/3044907052 learned_addr learned my addr 172.21.15.82:0/3044907052 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f3fff640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 0x55c10939a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 -- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 msgr2=0x55c10933e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c10933e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 -- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 msgr2=0x55c10939a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 0x55c10939a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 -- 172.21.15.82:0/3044907052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c1091969e0 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f3fff640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 0x55c10939a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66eb7fe640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000c10 tx=0x7f66dc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f4e908180a9d4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0b1040 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66ec02a070 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f66dc0accd0 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66dc0ce050 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 10 client.8559 fetch_fsmap learned FSMap version 69 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c1090dc630 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec02a070 con 0x55c1092f08e0 2024-04-25T10:11:12.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f66dc0d1020 con 0x55c1092f08e0 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 10 client.8559 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 10 client.8559 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 20 client.8559 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c10933ac00 0x55c1093a3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 4 client.8559 mds_command: new command op to 8376 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 --> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55c10914a5d0 con 0x55c10933ac00 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.293+0000 7f66f3fff640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c10933ac00 0x55c1093a3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66f3fff640 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c10933ac00 0x55c1093a3380 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ce55e08196c9bc9 server_cookie=82c3850fd9a0473 in_seq=0 out_seq=0 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66d2ffd640 10 client.8559 ms_handle_connect on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66d2ffd640 1 -- 172.21.15.82:0/3044907052 <== mds.0 v2:172.21.15.82:6836/570372586 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c10914a5d0 con 0x55c10933ac00 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66d2ffd640 10 client.8559 handle_command_reply: tid=0 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 client.8559 shutdown 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 msgr2=0x55c10939aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f66dc000c10 tx=0x7f66dc0191f0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 shutdown_connections 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:6836/570372586,v1:172.21.15.82:6837/570372586] conn(0x55c10933ac00 0x55c1093a3380 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c10933a2e0 0x55c10933e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66d2ffd640 0 client.8559 ms_handle_reset on v2:172.21.15.82:6836/570372586 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1092f08e0 0x55c10939aa50 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 --2- 172.21.15.82:0/3044907052 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1092ec890 0x55c10939a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 >> 172.21.15.82:0/3044907052 conn(0x55c1092f5be0 msgr2=0x55c109339af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 shutdown_connections 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 1 -- 172.21.15.82:0/3044907052 wait complete. 2024-04-25T10:11:12.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fd4d5000 20 client.8559 trim_cache size 0 max 16384 2024-04-25T10:11:12.306 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:12.306 INFO:teuthology.orchestra.run.smithi082.stdout:[] 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 msgr2=0x7f66d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 0x7f66d4069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f66dc0ae000 tx=0x7f66dc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 msgr2=0x7f66f40c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c7b0 tx=0x7f66e400cc70 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 shutdown_connections 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f66d4066f60 0x7f66d4069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f66f4159bd0 0x7f66f40b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f66f4159fb0 0x7f66f40b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 --2- 172.21.15.82:0/41286657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f66f40cb8c0 0x7f66f40c5230 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 >> 172.21.15.82:0/41286657 conn(0x7f66f406ae60 msgr2=0x7f66f4157030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 shutdown_connections 2024-04-25T10:11:12.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.297+0000 7f66fa368640 1 -- 172.21.15.82:0/41286657 wait complete. 2024-04-25T10:11:12.315 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:11:12.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 Processor -- start 2024-04-25T10:11:12.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 -- start start 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12c8161160 0x7f12c80d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c80d0600 0x7f12c80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c80d1d40 con 0x7f12c80b2780 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c80d0a70 con 0x7f12c8161160 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12cde9e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80d2020 con 0x7f12c80d0600 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12c77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12c8161160 0x7f12c80d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12c77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f12c8161160 0x7f12c80d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33856/0 (socket says 172.21.15.82:33856) 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.413+0000 7f12c77fe640 1 -- 172.21.15.82:0/1502907017 learned_addr learned my addr 172.21.15.82:0/1502907017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c77fe640 1 -- 172.21.15.82:0/1502907017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c80d0600 msgr2=0x7f12c80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c7fff640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c80d0600 0x7f12c80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c77fe640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c80d0600 0x7f12c80cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c77fe640 1 -- 172.21.15.82:0/1502907017 >> v1:172.21.15.196:6789/0 conn(0x7f12c80b2780 legacy=0x7f12c80b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c77fe640 1 -- 172.21.15.82:0/1502907017 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80d2300 con 0x7f12c8161160 2024-04-25T10:11:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c77fe640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c8161160 0x7f12c80d00c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9cc5c22886d379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c67fc640 1 -- 172.21.15.82:0/1502907017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80ab600 con 0x7f12c8161160 2024-04-25T10:11:12.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c67fc640 1 -- 172.21.15.82:0/1502907017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80b0070 con 0x7f12c8161160 2024-04-25T10:11:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12c67fc640 1 -- 172.21.15.82:0/1502907017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80ab9c0 con 0x7f12c8161160 2024-04-25T10:11:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 -- 172.21.15.82:0/1502907017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c8161160 msgr2=0x7f12c80d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c8161160 0x7f12c80d00c0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009870 tx=0x7f12b801c460 comp rx=0 tx=0).stop 2024-04-25T10:11:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 -- 172.21.15.82:0/1502907017 shutdown_connections 2024-04-25T10:11:12.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c80d0600 0x7f12c80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 --2- 172.21.15.82:0/1502907017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c8161160 0x7f12c80d00c0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 -- 172.21.15.82:0/1502907017 >> 172.21.15.82:0/1502907017 conn(0x7f12c808b000 msgr2=0x7f12c80d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 -- 172.21.15.82:0/1502907017 shutdown_connections 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.417+0000 7f12cde9e640 1 -- 172.21.15.82:0/1502907017 wait complete. 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 Processor -- start 2024-04-25T10:11:12.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- start start 2024-04-25T10:11:12.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c80d0600 0x7f12c80b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c8161160 0x7f12c80b8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80b4510 con 0x7f12c80b2780 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80b4390 con 0x7f12c80d0600 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80b4690 con 0x7f12c8161160 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33868/0 (socket says 172.21.15.82:33868) 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 -- 172.21.15.82:0/3953638826 learned_addr learned my addr 172.21.15.82:0/3953638826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c6ffd640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c80d0600 0x7f12c80b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 -- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c8161160 msgr2=0x7f12c80b8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c8161160 0x7f12c80b8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 -- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c80d0600 msgr2=0x7f12c80b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c80d0600 0x7f12c80b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 -- 172.21.15.82:0/3953638826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80b8940 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c77fe640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ad040 tx=0x7f12b801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80be5f0 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80be7b0 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c80b8bf0 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b80beba0 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c80b91b0 con 0x7f12c80b2780 2024-04-25T10:11:12.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c804efa0 con 0x7f12c80b2780 2024-04-25T10:11:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12b80aa070 con 0x7f12c80b2780 2024-04-25T10:11:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 0x7f12b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f12b811fa60 con 0x7f12c80b2780 2024-04-25T10:11:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c6ffd640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 0x7f12b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.421+0000 7f12c6ffd640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 0x7f12b4069450 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f12c80b1f90 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.425+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12b80c4060 con 0x7f12c80b2780 2024-04-25T10:11:12.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.605+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12c804efa0 con 0x7f12c80b2780 2024-04-25T10:11:12.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.605+0000 7f12affff640 1 -- 172.21.15.82:0/3953638826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f12b8016110 con 0x7f12c80b2780 2024-04-25T10:11:12.609 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:12.610 INFO:teuthology.orchestra.run.smithi082.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 msgr2=0x7f12b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 0x7f12b4069450 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f12c80b1f90 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 msgr2=0x7f12c80b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f12b80ad040 tx=0x7f12b801aca0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 shutdown_connections 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f12b4066f90 0x7f12b4069450 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12c8161160 0x7f12c80b8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f12c80d0600 0x7f12c80b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 --2- 172.21.15.82:0/3953638826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f12c80b2780 0x7f12c80b1510 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 >> 172.21.15.82:0/3953638826 conn(0x7f12c808b000 msgr2=0x7f12c806be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 shutdown_connections 2024-04-25T10:11:12.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.609+0000 7f12cde9e640 1 -- 172.21.15.82:0/3953638826 wait complete. 2024-04-25T10:11:12.627 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T10:11:12.627 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:12.627 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:12.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:12.723 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:12.723 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:12.723 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:12.729 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 Processor -- start 2024-04-25T10:11:12.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 -- start start 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0b2090 con 0x7f8b8c15a700 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8c0b43e0 con 0x7f8b8c0b24a0 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b935db640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8c15f200 con 0x7f8b8c0b3f50 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33872/0 (socket says 172.21.15.82:33872) 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 -- 172.21.15.82:0/1468022040 learned_addr learned my addr 172.21.15.82:0/1468022040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 -- 172.21.15.82:0/1468022040 >> v1:172.21.15.196:6789/0 conn(0x7f8b8c0b24a0 legacy=0x7f8b8c06e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 -- 172.21.15.82:0/1468022040 >> v1:172.21.15.82:6789/0 conn(0x7f8b8c15a700 legacy=0x7f8b8c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.885+0000 7f8b90b76640 1 -- 172.21.15.82:0/1468022040 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0b46c0 con 0x7f8b8c0b3f50 2024-04-25T10:11:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b90b76640 1 --2- 172.21.15.82:0/1468022040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009870 tx=0x7f8b7c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13be2d660671f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b83fff640 1 -- 172.21.15.82:0/1468022040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0b1070 con 0x7f8b8c0b3f50 2024-04-25T10:11:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b83fff640 1 -- 172.21.15.82:0/1468022040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7c0ac630 con 0x7f8b8c0b3f50 2024-04-25T10:11:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b83fff640 1 -- 172.21.15.82:0/1468022040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c0a8050 con 0x7f8b8c0b3f50 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 -- 172.21.15.82:0/1468022040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c0b3f50 msgr2=0x7f8b8c15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 --2- 172.21.15.82:0/1468022040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009870 tx=0x7f8b7c01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 -- 172.21.15.82:0/1468022040 shutdown_connections 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 --2- 172.21.15.82:0/1468022040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c0b3f50 0x7f8b8c15ecc0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 -- 172.21.15.82:0/1468022040 >> 172.21.15.82:0/1468022040 conn(0x7f8b8c0b11a0 msgr2=0x7f8b8c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 -- 172.21.15.82:0/1468022040 shutdown_connections 2024-04-25T10:11:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 -- 172.21.15.82:0/1468022040 wait complete. 2024-04-25T10:11:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.889+0000 7f8b935db640 1 Processor -- start 2024-04-25T10:11:12.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- start start 2024-04-25T10:11:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b8c0b24a0 0x7f8b8c1475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 0x7f8b8c0b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0cd310 con 0x7f8b8c150910 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0ce600 con 0x7f8b8c15a700 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c0cd490 con 0x7f8b8c0b24a0 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33878/0 (socket says 172.21.15.82:33878) 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 -- 172.21.15.82:0/1601128043 learned_addr learned my addr 172.21.15.82:0/1601128043 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b90b76640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 0x7f8b8c0b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 -- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b8c0b24a0 msgr2=0x7f8b8c1475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91377640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b8c0b24a0 0x7f8b8c1475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b8c0b24a0 0x7f8b8c1475d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 -- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 msgr2=0x7f8b8c0b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 0x7f8b8c0b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 -- 172.21.15.82:0/1601128043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c0acca0 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b90b76640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 0x7f8b8c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91b78640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c7b0 tx=0x7f8b8400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b84004b70 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b84004d10 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c0acf80 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400aa90 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c0ad540 con 0x7f8b8c150910 2024-04-25T10:11:12.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c04efa0 con 0x7f8b8c150910 2024-04-25T10:11:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b8400ac30 con 0x7f8b8c150910 2024-04-25T10:11:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8b840857b0 con 0x7f8b8c150910 2024-04-25T10:11:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91377640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.893+0000 7f8b91377640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 0x7f8b70069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f70 tx=0x7f8b88009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:12.897+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b840565a0 con 0x7f8b8c150910 2024-04-25T10:11:13.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.053+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b8c06f260 con 0x7f8b8c150910 2024-04-25T10:11:13.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b81ffb640 1 -- 172.21.15.82:0/1601128043 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17347 (secure 0 0 0) 0x7f8b84055ea0 con 0x7f8b8c150910 2024-04-25T10:11:13.060 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:13.060 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:10:33.173812+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6836/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 msgr2=0x7f8b70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 0x7f8b70069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f8b88000f70 tx=0x7f8b88009040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 msgr2=0x7f8b8c150da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400c7b0 tx=0x7f8b8400cc70 comp rx=0 tx=0).stop 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 shutdown_connections 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b70066f60 0x7f8b70069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b8c150910 0x7f8b8c150da0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b8c15a700 0x7f8b8c0b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 --2- 172.21.15.82:0/1601128043 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b8c0b24a0 0x7f8b8c1475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 >> 172.21.15.82:0/1601128043 conn(0x7f8b8c0b11a0 msgr2=0x7f8b8c152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 shutdown_connections 2024-04-25T10:11:13.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.057+0000 7f8b935db640 1 -- 172.21.15.82:0/1601128043 wait complete. 2024-04-25T10:11:13.077 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:13.078 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 Processor -- start 2024-04-25T10:11:13.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 -- start start 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f11480c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11480ca4c0 con 0x7f11480cf590 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11480cd690 con 0x7f11480c9ef0 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f115063b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11480ca780 con 0x7f114815ecb0 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114e3d7640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f11480cf590 0x7f11480cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51686/0 (socket says 172.21.15.82:51686) 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114e3d7640 1 -- 172.21.15.82:0/1854101922 learned_addr learned my addr 172.21.15.82:0/1854101922 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3095154405 0 0) 0x7f11480ca4c0 con 0x7f11480cf590 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114dbd6640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f11480c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1134003530 con 0x7f11480cf590 2024-04-25T10:11:13.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2597224123 0 0) 0x7f1134003530 con 0x7f11480cf590 2024-04-25T10:11:13.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 msgr2=0x7f11480c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f11480c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 >> v1:172.21.15.196:6789/0 conn(0x7f11480c9ef0 legacy=0x7f11480ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480cd970 con 0x7f11480cf590 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1144001680 con 0x7f11480cf590 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 >> v1:172.21.15.82:6789/0 conn(0x7f11480cf590 legacy=0x7f11480cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f1134003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1134003fe0 0x7f1134004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1134004d50 0x7f11340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113400bb00 con 0x7f1134004d50 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113400bcb0 con 0x7f1134003fe0 2024-04-25T10:11:13.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113400beb0 con 0x7f114815ecb0 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1134004d50 0x7f11340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114e3d7640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f1134003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114dbd6640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1134003fe0 0x7f1134004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 -- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 msgr2=0x7f1134003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f114815ecb0 0x7f1134003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 -- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1134003fe0 msgr2=0x7f1134004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1134003fe0 0x7f1134004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.197+0000 7f114ebd8640 1 -- 172.21.15.82:0/1854101922 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134003530 con 0x7f1134004d50 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1134004d50 0x7f11340051d0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1140001e70 tx=0x7f114001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b6e85aabc6c5b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140001a90 con 0x7f1134004d50 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140003b80 con 0x7f1134004d50 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f114d3d5640 1 -- 172.21.15.82:0/1854101922 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140005080 con 0x7f1134004d50 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- 172.21.15.82:0/1854101922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1134004d50 msgr2=0x7f11340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 --2- 172.21.15.82:0/1854101922 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1134004d50 0x7f11340051d0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f1140001e70 tx=0x7f114001c780 comp rx=0 tx=0).stop 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f114ebd8640 1 -- 172.21.15.82:0/1854101922 reap_dead start 2024-04-25T10:11:13.212 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- 172.21.15.82:0/1854101922 shutdown_connections 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- 172.21.15.82:0/1854101922 >> 172.21.15.82:0/1854101922 conn(0x7f114806ae60 msgr2=0x7f114806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- 172.21.15.82:0/1854101922 shutdown_connections 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- 172.21.15.82:0/1854101922 wait complete. 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 Processor -- start 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.201+0000 7f115063b640 1 -- start start 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 0x7f1148072290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 0x7f11480c33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11480cd440 con 0x7f11480b15d0 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114815f0f0 con 0x7f11480b11f0 2024-04-25T10:11:13.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11480cdce0 con 0x7f1148158160 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33906/0 (socket says 172.21.15.82:33906) 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 -- 172.21.15.82:0/1763428826 learned_addr learned my addr 172.21.15.82:0/1763428826 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114e3d7640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 0x7f1148072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 0x7f11480c33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 -- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 msgr2=0x7f11480c33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 0x7f11480c33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 -- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 msgr2=0x7f1148072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 0x7f1148072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 -- 172.21.15.82:0/1763428826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11481585e0 con 0x7f11480b15d0 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114ebd8640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 0x7f11480c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114e3d7640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 0x7f1148072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:13.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114dbd6640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f11380115e0 tx=0x7f1138011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800f8f0 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113800fa90 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11480ad7a0 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f113800fe60 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114815d560 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114804efa0 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1138015460 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1138085b90 con 0x7f11480b15d0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114e3d7640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.215 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.205+0000 7f114e3d7640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 0x7f112c069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1144004850 tx=0x7f1144009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.209+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11480ad7a0 con 0x7f11480b15d0 2024-04-25T10:11:13.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.357+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11480b4170 con 0x7f11480b15d0 2024-04-25T10:11:13.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.357+0000 7f113effd640 1 -- 172.21.15.82:0/1763428826 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1138019030 con 0x7f11480b15d0 2024-04-25T10:11:13.361 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 msgr2=0x7f112c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 0x7f112c069450 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1144004850 tx=0x7f1144009040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 msgr2=0x7f1148157c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f11380115e0 tx=0x7f1138011aa0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 shutdown_connections 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1148158160 0x7f11480c33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f11480b15d0 0x7f1148157c20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 --2- 172.21.15.82:0/1763428826 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f11480b11f0 0x7f1148072290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 >> 172.21.15.82:0/1763428826 conn(0x7f114806ae60 msgr2=0x7f1148071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 shutdown_connections 2024-04-25T10:11:13.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.361+0000 7f115063b640 1 -- 172.21.15.82:0/1763428826 wait complete. 2024-04-25T10:11:13.381 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:13.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 Processor -- start 2024-04-25T10:11:13.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 -- start start 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25180cd350 0x7f251815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25180ce320 con 0x7f251815aac0 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25180ce600 con 0x7f25180cf0e0 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f251dfc6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25180ce0d0 con 0x7f25180cd350 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f2516ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f25180cd350 0x7f251815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f25177fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f251815aac0 0x7f25180ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51700/0 (socket says 172.21.15.82:51700) 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f25177fe640 1 -- 172.21.15.82:0/2728528028 learned_addr learned my addr 172.21.15.82:0/2728528028 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f2516ffd640 1 -- 172.21.15.82:0/2728528028 >> v1:172.21.15.196:6789/0 conn(0x7f25180cf0e0 legacy=0x7f251815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f2516ffd640 1 -- 172.21.15.82:0/2728528028 >> v1:172.21.15.82:6789/0 conn(0x7f251815aac0 legacy=0x7f25180ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f2516ffd640 1 -- 172.21.15.82:0/2728528028 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180ce8e0 con 0x7f25180cd350 2024-04-25T10:11:13.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.497+0000 7f25167fc640 1 -- 172.21.15.82:0/2728528028 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902612006 0 0) 0x7f25180ce320 con 0x7f251815aac0 2024-04-25T10:11:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 --2- 172.21.15.82:0/2728528028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cd350 0x7f251815aea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f25000097c0 tx=0x7f250001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5daef6419cc26a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f25167fc640 1 -- 172.21.15.82:0/2728528028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b0070 con 0x7f25180cd350 2024-04-25T10:11:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f25167fc640 1 -- 172.21.15.82:0/2728528028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000ab600 con 0x7f25180cd350 2024-04-25T10:11:13.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f25167fc640 1 -- 172.21.15.82:0/2728528028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000ab9d0 con 0x7f25180cd350 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- 172.21.15.82:0/2728528028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cd350 msgr2=0x7f251815aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2728528028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cd350 0x7f251815aea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f25000097c0 tx=0x7f250001c460 comp rx=0 tx=0).stop 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- 172.21.15.82:0/2728528028 shutdown_connections 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2728528028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cd350 0x7f251815aea0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- 172.21.15.82:0/2728528028 >> 172.21.15.82:0/2728528028 conn(0x7f2518099a70 msgr2=0x7f25180d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- 172.21.15.82:0/2728528028 shutdown_connections 2024-04-25T10:11:13.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- 172.21.15.82:0/2728528028 wait complete. 2024-04-25T10:11:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 Processor -- start 2024-04-25T10:11:13.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- start start 2024-04-25T10:11:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25180cd350 0x7f25180c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f251815aac0 0x7f25180ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2518168150 con 0x7f25180cf0e0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2518167fd0 con 0x7f251815aac0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f251dfc6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25181682d0 con 0x7f25180cd350 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33932/0 (socket says 172.21.15.82:33932) 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 -- 172.21.15.82:0/2585372745 learned_addr learned my addr 172.21.15.82:0/2585372745 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 -- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25180cd350 msgr2=0x7f25180c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25180cd350 0x7f25180c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 -- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f251815aac0 msgr2=0x7f25180ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2517fff640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f251815aac0 0x7f25180ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f251815aac0 0x7f25180ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.501+0000 7f2516ffd640 1 -- 172.21.15.82:0/2585372745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251815a280 con 0x7f25180cf0e0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f2516ffd640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2500019070 tx=0x7f250001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b0050 con 0x7f25180cf0e0 2024-04-25T10:11:13.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25000bc070 con 0x7f25180cf0e0 2024-04-25T10:11:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25000b5a70 con 0x7f25180cf0e0 2024-04-25T10:11:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251815bf00 con 0x7f25180cf0e0 2024-04-25T10:11:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251815c4c0 con 0x7f25180cf0e0 2024-04-25T10:11:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251804efa0 con 0x7f25180cf0e0 2024-04-25T10:11:13.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f25000ab480 con 0x7f25180cf0e0 2024-04-25T10:11:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f25177fe640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f25001206e0 con 0x7f25180cf0e0 2024-04-25T10:11:13.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.505+0000 7f25177fe640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 0x7f24f8069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2510001ca0 tx=0x7f2510007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.509+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2500016050 con 0x7f25180cf0e0 2024-04-25T10:11:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.657+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25180cdc90 con 0x7f25180cf0e0 2024-04-25T10:11:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.657+0000 7f250ffff640 1 -- 172.21.15.82:0/2585372745 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25000aa3a0 con 0x7f25180cf0e0 2024-04-25T10:11:13.661 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 msgr2=0x7f24f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 0x7f24f8069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2510001ca0 tx=0x7f2510007cf0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 msgr2=0x7f25181599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f2500019070 tx=0x7f250001aca0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 shutdown_connections 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f24f8066f90 0x7f24f8069450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f251815aac0 0x7f25180ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f25180cf0e0 0x7f25181599a0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 --2- 172.21.15.82:0/2585372745 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f25180cd350 0x7f25180c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 >> 172.21.15.82:0/2585372745 conn(0x7f2518099a70 msgr2=0x7f25180d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 shutdown_connections 2024-04-25T10:11:13.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.661+0000 7f251dfc6640 1 -- 172.21.15.82:0/2585372745 wait complete. 2024-04-25T10:11:13.678 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:13.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.797+0000 7fe3e45c9640 1 Processor -- start 2024-04-25T10:11:13.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 -- start start 2024-04-25T10:11:13.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe3dc0d1cd0 0x7fe3dc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc06c440 con 0x7fe3dc0d0620 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc06be60 con 0x7fe3dc0d1cd0 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e45c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc15de60 con 0x7fe3dc15da80 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33942/0 (socket says 172.21.15.82:33942) 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 -- 172.21.15.82:0/730488271 learned_addr learned my addr 172.21.15.82:0/730488271 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e1b64640 1 --2- 172.21.15.82:0/730488271 >> v2:172.21.15.196:3300/0 conn(0x7fe3dc0d1cd0 0x7fe3dc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2365640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 -- 172.21.15.82:0/730488271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 msgr2=0x7fe3dc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 -- 172.21.15.82:0/730488271 >> v2:172.21.15.196:3300/0 conn(0x7fe3dc0d1cd0 msgr2=0x7fe3dc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 --2- 172.21.15.82:0/730488271 >> v2:172.21.15.196:3300/0 conn(0x7fe3dc0d1cd0 0x7fe3dc0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 -- 172.21.15.82:0/730488271 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc0d2450 con 0x7fe3dc0d0620 2024-04-25T10:11:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2365640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e2b66640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06bd60 tx=0x7fe3d401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaed2f0007c981b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.801+0000 7fe3e1363640 1 -- 172.21.15.82:0/730488271 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40a5690 con 0x7fe3dc0d0620 2024-04-25T10:11:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e1363640 1 -- 172.21.15.82:0/730488271 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d40a5830 con 0x7fe3dc0d0620 2024-04-25T10:11:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e1363640 1 -- 172.21.15.82:0/730488271 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d40a5c00 con 0x7fe3dc0d0620 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- 172.21.15.82:0/730488271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 msgr2=0x7fe3dc0ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc06bd60 tx=0x7fe3d401c9f0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- 172.21.15.82:0/730488271 shutdown_connections 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc0ce3b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/730488271 >> v2:172.21.15.196:3300/0 conn(0x7fe3dc0d1cd0 0x7fe3dc0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/730488271 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- 172.21.15.82:0/730488271 >> 172.21.15.82:0/730488271 conn(0x7fe3dc08b000 msgr2=0x7fe3dc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- 172.21.15.82:0/730488271 shutdown_connections 2024-04-25T10:11:13.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- 172.21.15.82:0/730488271 wait complete. 2024-04-25T10:11:13.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 Processor -- start 2024-04-25T10:11:13.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- start start 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc0d1cd0 0x7fe3dc150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06ca30 con 0x7fe3dc0d0620 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06c8b0 con 0x7fe3dc0d1cd0 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e45c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06cbb0 con 0x7fe3dc15da80 2024-04-25T10:11:13.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33950/0 (socket says 172.21.15.82:33950) 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 -- 172.21.15.82:0/2366165737 learned_addr learned my addr 172.21.15.82:0/2366165737 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 -- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 msgr2=0x7fe3dc158580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 -- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc0d1cd0 msgr2=0x7fe3dc150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc0d1cd0 0x7fe3dc150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.805+0000 7fe3e2365640 1 -- 172.21.15.82:0/2366165737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc1520f0 con 0x7fe3dc0d0620 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e2365640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c7f0 tx=0x7fe3d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8015070 con 0x7fe3dc0d0620 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d8007820 con 0x7fe3dc0d0620 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8007c10 con 0x7fe3dc0d0620 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc151850 con 0x7fe3dc0d0620 2024-04-25T10:11:13.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc151e10 con 0x7fe3dc0d0620 2024-04-25T10:11:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc04efa0 con 0x7fe3dc0d0620 2024-04-25T10:11:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3d8018070 con 0x7fe3dc0d0620 2024-04-25T10:11:13.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 0x7fe3c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e1b64640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 0x7fe3c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe3d8085310 con 0x7fe3dc0d0620 2024-04-25T10:11:13.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.809+0000 7fe3e1b64640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 0x7fe3c0069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006ac0 tx=0x7fe3cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.813+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3d8056100 con 0x7fe3dc0d0620 2024-04-25T10:11:13.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.965+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3dc0b93d0 con 0x7fe3dc0d0620 2024-04-25T10:11:13.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.965+0000 7fe3d2ffd640 1 -- 172.21.15.82:0/2366165737 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe3d8055a00 con 0x7fe3dc0d0620 2024-04-25T10:11:13.969 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:13.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 msgr2=0x7fe3c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 0x7fe3c0069420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006ac0 tx=0x7fe3cc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 msgr2=0x7fe3dc150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800c7f0 tx=0x7fe3d800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 shutdown_connections 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe3c0066f60 0x7fe3c0069420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3dc15da80 0x7fe3dc158580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3dc0d1cd0 0x7fe3dc150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 --2- 172.21.15.82:0/2366165737 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3dc0d0620 0x7fe3dc150830 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 >> 172.21.15.82:0/2366165737 conn(0x7fe3dc08b000 msgr2=0x7fe3dc0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 shutdown_connections 2024-04-25T10:11:13.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:13.969+0000 7fe3e45c9640 1 -- 172.21.15.82:0/2366165737 wait complete. 2024-04-25T10:11:13.987 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:14.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.093+0000 7f4c0e641640 1 Processor -- start 2024-04-25T10:11:14.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.093+0000 7f4c0e641640 1 -- start start 2024-04-25T10:11:14.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0e641640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4c080ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0e641640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4c081451f0 0x7f4c08142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0e641640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080cee30 con 0x7f4c08144ca0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0e641640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c080cc2b0 con 0x7f4c081451f0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0e641640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806e290 con 0x7f4c080ce510 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c077fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4c08144ca0 0x7f4c0806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:51704/0 (socket says 172.21.15.82:51704) 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c077fe640 1 -- 172.21.15.82:0/418217327 learned_addr learned my addr 172.21.15.82:0/418217327 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/418217327 >> v2:172.21.15.196:3300/0 conn(0x7f4c081451f0 0x7f4c08142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c07fff640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4c080ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475301423 0 0) 0x7f4c080cee30 con 0x7f4c08144ca0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf0003530 con 0x7f4c08144ca0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3819422725 0 0) 0x7f4bf0003530 con 0x7f4c08144ca0 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 msgr2=0x7f4c080ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4c080ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 >> v2:172.21.15.196:3300/0 conn(0x7f4c081451f0 msgr2=0x7f4c08142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 --2- 172.21.15.82:0/418217327 >> v2:172.21.15.196:3300/0 conn(0x7f4c081451f0 0x7f4c08142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080ccc70 con 0x7f4c08144ca0 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4bfc001680 con 0x7f4c08144ca0 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 >> v1:172.21.15.82:6789/0 conn(0x7f4c08144ca0 legacy=0x7f4c0806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4bf0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf0003fb0 0x7f4bf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf0004d50 0x7f4bf00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cc2b0 con 0x7f4bf0004d50 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806e290 con 0x7f4bf0003fb0 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf000ba80 con 0x7f4c080ce510 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf0004d50 0x7f4bf00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c07fff640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4bf0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 -- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 msgr2=0x7f4bf0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c080ce510 0x7f4bf0003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 -- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf0003fb0 msgr2=0x7f4bf0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4bf0003fb0 0x7f4bf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 -- 172.21.15.82:0/418217327 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0003530 con 0x7f4bf0004d50 2024-04-25T10:11:14.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf0004d50 0x7f4bf00051d0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fc0 tx=0x7f4c0001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d06f9b375d8ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c000b2070 con 0x7f4bf0004d50 2024-04-25T10:11:14.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c000a5cd0 con 0x7f4bf0004d50 2024-04-25T10:11:14.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.097+0000 7f4c06ffd640 1 -- 172.21.15.82:0/418217327 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00009c10 con 0x7f4bf0004d50 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/418217327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf0004d50 msgr2=0x7f4bf00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 --2- 172.21.15.82:0/418217327 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4bf0004d50 0x7f4bf00051d0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fc0 tx=0x7f4c0001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0cbde640 1 -- 172.21.15.82:0/418217327 reap_dead start 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/418217327 shutdown_connections 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/418217327 >> 172.21.15.82:0/418217327 conn(0x7f4c080b11a0 msgr2=0x7f4c080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/418217327 shutdown_connections 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/418217327 wait complete. 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 Processor -- start 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- start start 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c081e4bd0 0x7f4c0812fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c08130430 0x7f4c081308e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806ee30 con 0x7f4c080c98c0 2024-04-25T10:11:14.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0806df20 con 0x7f4c081e4bd0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c080cc900 con 0x7f4c08130430 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33968/0 (socket says 172.21.15.82:33968) 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 -- 172.21.15.82:0/3396779340 learned_addr learned my addr 172.21.15.82:0/3396779340 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0cbde640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c08130430 0x7f4c081308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 -- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c08130430 msgr2=0x7f4c081308e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c08130430 0x7f4c081308e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c077fe640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c081e4bd0 0x7f4c0812fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 -- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c081e4bd0 msgr2=0x7f4c0812fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c081e4bd0 0x7f4c0812fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 -- 172.21.15.82:0/3396779340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c080cae40 con 0x7f4c080c98c0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c07fff640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8010890 tx=0x7f4bf8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf800f040 con 0x7f4c080c98c0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf800eb60 con 0x7f4c080c98c0 2024-04-25T10:11:14.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0812df00 con 0x7f4c080c98c0 2024-04-25T10:11:14.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf801c400 con 0x7f4c080c98c0 2024-04-25T10:11:14.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.101+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0812e4c0 con 0x7f4c080c98c0 2024-04-25T10:11:14.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0804efa0 con 0x7f4c080c98c0 2024-04-25T10:11:14.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bf801c5a0 con 0x7f4c080c98c0 2024-04-25T10:11:14.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c04ff9640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 0x7f4bf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c077fe640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 0x7f4bf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f4bf8085db0 con 0x7f4c080c98c0 2024-04-25T10:11:14.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.105+0000 7f4c077fe640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 0x7f4bf4069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001060 tx=0x7f4bfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.109+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bf8018030 con 0x7f4c080c98c0 2024-04-25T10:11:14.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4c080b1cc0 con 0x7f4c080c98c0 2024-04-25T10:11:14.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c04ff9640 1 -- 172.21.15.82:0/3396779340 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v95) v1 ==== 80+0+242 (secure 0 0 0) 0x7f4bf8056b70 con 0x7f4c080c98c0 2024-04-25T10:11:14.280 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/362560393 2024-04-26T10:10:33.173770+0000 2024-04-25T10:11:14.280 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/362560393 2024-04-26T10:10:33.173770+0000 2024-04-25T10:11:14.280 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:11:14.280 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6836/2865291678 2024-04-26T10:09:12.306251+0000 2024-04-25T10:11:14.281 INFO:teuthology.orchestra.run.smithi082.stderr:listed 4 entries 2024-04-25T10:11:14.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 msgr2=0x7f4bf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 0x7f4bf4069420 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc001060 tx=0x7f4bfc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:14.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 msgr2=0x7f4c081e4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8010890 tx=0x7f4bf8010d50 comp rx=0 tx=0).stop 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 shutdown_connections 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4bf4066f60 0x7f4bf4069420 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4c08130430 0x7f4c081308e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4c081e4bd0 0x7f4c0812fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 --2- 172.21.15.82:0/3396779340 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4c080c98c0 0x7f4c081e4690 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 >> 172.21.15.82:0/3396779340 conn(0x7f4c080b11a0 msgr2=0x7f4c080c64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 shutdown_connections 2024-04-25T10:11:14.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.277+0000 7f4c0e641640 1 -- 172.21.15.82:0/3396779340 wait complete. 2024-04-25T10:11:14.299 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:11:14.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:14.309 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[50209]: fuse finished with error 0 and tester_r 0 2024-04-25T10:11:14.311 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:11:14.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.311 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:14.476 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:11:14.476 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:14.483 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:11:14.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.483 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:11:14.530 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:11:14.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:14.532 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:14.532 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.538 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:14.538 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:11:14.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.539 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:11:14.587 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:11:14.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:14.588 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:11:14.588 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:14.596 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:14.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:11:14.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:14.596 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:11:14.642 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:11:14.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:11:14.644 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.769+0000 7ff09431a640 1 Processor -- start 2024-04-25T10:11:14.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.769+0000 7ff09431a640 1 -- start start 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff09431a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff08c159ea0 0x7ff08c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff09431a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c15a280 0x7ff08c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff09431a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08c15ae30 con 0x7ff08c0718c0 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff09431a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08c0b2990 con 0x7ff08c159ea0 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff09431a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c15b110 con 0x7ff08c15a280 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff08c159ea0 0x7ff08c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff08c159ea0 0x7ff08c0b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33590/0 (socket says 172.21.15.82:33590) 2024-04-25T10:11:14.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 -- 172.21.15.82:0/2135425599 learned_addr learned my addr 172.21.15.82:0/2135425599 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 -- 172.21.15.82:0/2135425599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c15a280 msgr2=0x7ff08c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0918b5640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c15a280 0x7ff08c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c15a280 0x7ff08c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 -- 172.21.15.82:0/2135425599 >> v1:172.21.15.196:6789/0 conn(0x7ff08c0718c0 legacy=0x7ff08c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 -- 172.21.15.82:0/2135425599 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c0b2620 con 0x7ff08c159ea0 2024-04-25T10:11:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.773+0000 7ff0920b6640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c159ea0 0x7ff08c0b2e20 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff088009870 tx=0x7ff08801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4753374a335a2926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0910b4640 1 -- 172.21.15.82:0/2135425599 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0880ab5f0 con 0x7ff08c159ea0 2024-04-25T10:11:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0910b4640 1 -- 172.21.15.82:0/2135425599 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0880b0070 con 0x7ff08c159ea0 2024-04-25T10:11:14.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0910b4640 1 -- 172.21.15.82:0/2135425599 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0880ab9b0 con 0x7ff08c159ea0 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2135425599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c159ea0 msgr2=0x7ff08c0b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c159ea0 0x7ff08c0b2e20 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7ff088009870 tx=0x7ff08801c520 comp rx=0 tx=0).stop 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2135425599 shutdown_connections 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c15a280 0x7ff08c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- 172.21.15.82:0/2135425599 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c159ea0 0x7ff08c0b2e20 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2135425599 >> 172.21.15.82:0/2135425599 conn(0x7ff08c0b11a0 msgr2=0x7ff08c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2135425599 shutdown_connections 2024-04-25T10:11:14.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2135425599 wait complete. 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 Processor -- start 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- start start 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08c0718c0 0x7ff08c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c159ea0 0x7ff08c161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c0711b0 con 0x7ff08c15a280 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c071030 con 0x7ff08c0718c0 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c071330 con 0x7ff08c159ea0 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33594/0 (socket says 172.21.15.82:33594) 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 -- 172.21.15.82:0/2175233847 learned_addr learned my addr 172.21.15.82:0/2175233847 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0920b6640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08c0718c0 0x7ff08c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 -- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c159ea0 msgr2=0x7ff08c161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c159ea0 0x7ff08c161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 -- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08c0718c0 msgr2=0x7ff08c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08c0718c0 0x7ff08c155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 -- 172.21.15.82:0/2175233847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c0cd600 con 0x7ff08c15a280 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff0928b7640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c7b0 tx=0x7ff08400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084004b70 con 0x7ff08c15a280 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff084004d10 con 0x7ff08c15a280 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08c0cd8e0 con 0x7ff08c15a280 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08400aa90 con 0x7ff08c15a280 2024-04-25T10:11:14.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.777+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff08c0ce3c0 con 0x7ff08c15a280 2024-04-25T10:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff08c04efa0 con 0x7ff08c15a280 2024-04-25T10:11:14.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff08400ac30 con 0x7ff08c15a280 2024-04-25T10:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff082ffd640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 0x7ff070069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7ff084085510 con 0x7ff08c15a280 2024-04-25T10:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff0920b6640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 0x7ff070069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.781+0000 7ff0920b6640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 0x7ff070069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff0880af000 tx=0x7ff0880a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.785+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff084020070 con 0x7ff08c15a280 2024-04-25T10:11:14.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.965+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff08c06e6b0 con 0x7ff08c15a280 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff082ffd640 1 -- 172.21.15.82:0/2175233847 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff084056270 con 0x7ff08c15a280 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:14.971 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:14.972 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:11:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 msgr2=0x7ff070069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 0x7ff070069450 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff0880af000 tx=0x7ff0880a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 msgr2=0x7ff08c162220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff08400c7b0 tx=0x7ff08400cc70 comp rx=0 tx=0).stop 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 shutdown_connections 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff070066f90 0x7ff070069450 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff08c159ea0 0x7ff08c161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff08c15a280 0x7ff08c162220 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 --2- 172.21.15.82:0/2175233847 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff08c0718c0 0x7ff08c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 >> 172.21.15.82:0/2175233847 conn(0x7ff08c0b11a0 msgr2=0x7ff08c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 shutdown_connections 2024-04-25T10:11:14.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:14.969+0000 7ff09431a640 1 -- 172.21.15.82:0/2175233847 wait complete. 2024-04-25T10:11:14.990 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:15.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.093+0000 7f5f04eb7640 1 Processor -- start 2024-04-25T10:11:15.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.093+0000 7f5f04eb7640 1 -- start start 2024-04-25T10:11:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5f04eb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5f04eb7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f0006d090 0x7f5f0006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5f04eb7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0006bff0 con 0x7f5f000b5650 2024-04-25T10:11:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5f04eb7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0006cc00 con 0x7f5f0006d090 2024-04-25T10:11:15.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5f04eb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0006c2d0 con 0x7f5f000b5270 2024-04-25T10:11:15.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efe59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efed9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f0006d090 0x7f5f0006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efe59c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:33008/0 (socket says 172.21.15.82:33008) 2024-04-25T10:11:15.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efe59c640 1 -- 172.21.15.82:0/3360784132 learned_addr learned my addr 172.21.15.82:0/3360784132 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efed9d640 1 -- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 msgr2=0x7f5f000b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efed9d640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efed9d640 1 -- 172.21.15.82:0/3360784132 >> v1:172.21.15.196:6789/0 conn(0x7f5f000b5650 legacy=0x7f5f000d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efed9d640 1 -- 172.21.15.82:0/3360784132 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0006c5b0 con 0x7f5f0006d090 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efd59a640 1 -- 172.21.15.82:0/3360784132 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975888639 0 0) 0x7f5f0006bff0 con 0x7f5f000b5650 2024-04-25T10:11:15.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.097+0000 7f5efe59c640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:15.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efed9d640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0006d090 0x7f5f0006d4f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009d90 tx=0x7f5ef801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6799b5681a445a93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efd59a640 1 -- 172.21.15.82:0/3360784132 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80b2070 con 0x7f5f0006d090 2024-04-25T10:11:15.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efd59a640 1 -- 172.21.15.82:0/3360784132 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef80a5940 con 0x7f5f0006d090 2024-04-25T10:11:15.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efd59a640 1 -- 172.21.15.82:0/3360784132 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef80a5d10 con 0x7f5f0006d090 2024-04-25T10:11:15.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0006d090 msgr2=0x7f5f0006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0006d090 0x7f5f0006d4f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009d90 tx=0x7f5ef801c970 comp rx=0 tx=0).stop 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3360784132 shutdown_connections 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f0006d090 0x7f5f0006d4f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3360784132 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3360784132 >> 172.21.15.82:0/3360784132 conn(0x7f5f0008b000 msgr2=0x7f5f000d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3360784132 shutdown_connections 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3360784132 wait complete. 2024-04-25T10:11:15.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 Processor -- start 2024-04-25T10:11:15.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- start start 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 0x7f5f0014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b5bd0 con 0x7f5f0006d090 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000b5a50 con 0x7f5f000b5650 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0015de20 con 0x7f5f000b5270 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33596/0 (socket says 172.21.15.82:33596) 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 -- 172.21.15.82:0/3861454614 learned_addr learned my addr 172.21.15.82:0/3861454614 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efdd9b640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efed9d640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 0x7f5f0014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 -- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 msgr2=0x7f5f000c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 -- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 msgr2=0x7f5f0014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 0x7f5f0014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 -- 172.21.15.82:0/3861454614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000ca0d0 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efed9d640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 0x7f5f0014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5efe59c640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ebe0 tx=0x7f5ef400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef400cc30 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef400cdd0 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f000c9b50 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4005bc0 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.101+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0014c720 con 0x7f5f0006d090 2024-04-25T10:11:15.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0004efa0 con 0x7f5f0006d090 2024-04-25T10:11:15.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ef4006030 con 0x7f5f0006d090 2024-04-25T10:11:15.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5eeeffd640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5ef40857d0 con 0x7f5f0006d090 2024-04-25T10:11:15.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5efdd9b640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.105+0000 7f5efdd9b640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 0x7f5ee0069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f0015d840 tx=0x7f5ef0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.109+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ef4017020 con 0x7f5f0006d090 2024-04-25T10:11:15.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.269+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f000c7070 con 0x7f5f0006d090 2024-04-25T10:11:15.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5eeeffd640 1 -- 172.21.15.82:0/3861454614 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17347 (secure 0 0 0) 0x7f5ef4056530 con 0x7f5f0006d090 2024-04-25T10:11:15.275 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:15.275 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:10:33.173812+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6836/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:15.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 msgr2=0x7f5ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 0x7f5ee0069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5f0015d840 tx=0x7f5ef0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 msgr2=0x7f5f0015d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ebe0 tx=0x7f5ef400c520 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 shutdown_connections 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ee0066f60 0x7f5ee0069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f000b5270 0x7f5f000c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f000b5650 0x7f5f0014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 --2- 172.21.15.82:0/3861454614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f0006d090 0x7f5f0015d300 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 >> 172.21.15.82:0/3861454614 conn(0x7f5f0008b000 msgr2=0x7f5f000ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 shutdown_connections 2024-04-25T10:11:15.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.273+0000 7f5f04eb7640 1 -- 172.21.15.82:0/3861454614 wait complete. 2024-04-25T10:11:15.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:47.469954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:48.247943+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:15.296 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:15.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.417+0000 7fa1a4bbe640 1 Processor -- start 2024-04-25T10:11:15.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.417+0000 7fa1a4bbe640 1 -- start start 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa1a4bbe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa1a01634c0 0x7fa1a01658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa1a4bbe640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a00ca4c0 con 0x7fa1a015ecb0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa1a4bbe640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a00cd690 con 0x7fa1a00cf590 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa1a4bbe640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a00ca780 con 0x7fa1a01634c0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa195d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa1a015ecb0 0x7fa1a00c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48318/0 (socket says 172.21.15.82:48318) 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa195d9b640 1 -- 172.21.15.82:0/929967426 learned_addr learned my addr 172.21.15.82:0/929967426 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/929967426 >> v2:172.21.15.196:3300/0 conn(0x7fa1a01634c0 0x7fa1a01658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3177480752 0 0) 0x7fa1a00ca4c0 con 0x7fa1a015ecb0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa18c003530 con 0x7fa1a015ecb0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520939681 0 0) 0x7fa18c003530 con 0x7fa1a015ecb0 2024-04-25T10:11:15.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 >> v2:172.21.15.196:3300/0 conn(0x7fa1a01634c0 msgr2=0x7fa1a01658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 --2- 172.21.15.82:0/929967426 >> v2:172.21.15.196:3300/0 conn(0x7fa1a01634c0 0x7fa1a01658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 >> v1:172.21.15.196:6789/0 conn(0x7fa1a00cf590 legacy=0x7fa1a00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a00cd970 con 0x7fa1a015ecb0 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa188001680 con 0x7fa1a015ecb0 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 >> v1:172.21.15.82:6789/0 conn(0x7fa1a015ecb0 legacy=0x7fa1a00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa18c003a30 0x7fa18c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c004810 0x7fa18c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 0x7fa18c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00c0d0 con 0x7fa18c0051d0 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00c280 con 0x7fa18c003a30 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa18c00c480 con 0x7fa18c004810 2024-04-25T10:11:15.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 0x7fa18c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19e59c640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa18c003a30 0x7fa18c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 -- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c004810 msgr2=0x7fa18c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c004810 0x7fa18c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 -- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa18c003a30 msgr2=0x7fa18c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa18c003a30 0x7fa18c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 -- 172.21.15.82:0/929967426 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c003530 con 0x7fa18c0051d0 2024-04-25T10:11:15.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19e59c640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa18c003a30 0x7fa18c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 -- 172.21.15.82:0/929967426 reap_dead start 2024-04-25T10:11:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 0x7fa18c00b9d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa198009f90 tx=0x7fa19801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393eb2ffa1b97c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198009c10 con 0x7fa18c0051d0 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1980a5c90 con 0x7fa18c0051d0 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.421+0000 7fa19dd9b640 1 -- 172.21.15.82:0/929967426 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198003be0 con 0x7fa18c0051d0 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 msgr2=0x7fa18c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 0x7fa18c00b9d0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fa198009f90 tx=0x7fa19801cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/929967426 shutdown_connections 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/929967426 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa18c0051d0 0x7fa18c00b9d0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/929967426 >> 172.21.15.82:0/929967426 conn(0x7fa1a006ae60 msgr2=0x7fa1a006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/929967426 shutdown_connections 2024-04-25T10:11:15.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/929967426 wait complete. 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 Processor -- start 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- start start 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 0x7fa1a0098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0154830 0x7fa1a0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00cd440 con 0x7fa1a0159cf0 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a015f0f0 con 0x7fa1a0154830 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a00cdce0 con 0x7fa18c0051d0 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33616/0 (socket says 172.21.15.82:33616) 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 -- 172.21.15.82:0/3813048580 learned_addr learned my addr 172.21.15.82:0/3813048580 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa19e59c640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 0x7fa1a0098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa19ed9d640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0154830 0x7fa1a0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 -- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 msgr2=0x7fa1a0098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 0x7fa1a0098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 -- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0154830 msgr2=0x7fa1a0154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0154830 0x7fa1a0154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 -- 172.21.15.82:0/3813048580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a00c8850 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa19e59c640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 0x7fa1a0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa195d9b640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c710 tx=0x7fa18800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188018070 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1880072f0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a00c8b30 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880047f0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.425+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a00c90f0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a004efa0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa18801b070 con 0x7fa1a0159cf0 2024-04-25T10:11:15.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa1977fe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 0x7fa180069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa19e59c640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 0x7fa180069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa1880848d0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.429+0000 7fa19e59c640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 0x7fa180069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1900015e0 tx=0x7fa190009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.433+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1880555a0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1a00c2730 con 0x7fa1a0159cf0 2024-04-25T10:11:15.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1977fe640 1 -- 172.21.15.82:0/3813048580 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa188054ea0 con 0x7fa1a0159cf0 2024-04-25T10:11:15.596 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 msgr2=0x7fa180069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 0x7fa180069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa1900015e0 tx=0x7fa190009040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 msgr2=0x7fa1a00ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c710 tx=0x7fa18800cbd0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 shutdown_connections 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa180066f60 0x7fa180069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa1a0154830 0x7fa1a0154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa1a0159cf0 0x7fa1a00ad500 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 --2- 172.21.15.82:0/3813048580 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa18c0051d0 0x7fa1a0098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 >> 172.21.15.82:0/3813048580 conn(0x7fa1a006ae60 msgr2=0x7fa1a00af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 shutdown_connections 2024-04-25T10:11:15.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.593+0000 7fa1a4bbe640 1 -- 172.21.15.82:0/3813048580 wait complete. 2024-04-25T10:11:15.614 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.729+0000 7fa7bbe47640 1 Processor -- start 2024-04-25T10:11:15.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.729+0000 7fa7bbe47640 1 -- start start 2024-04-25T10:11:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b415b6b0 0x7fa7b40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa7b40cf590 0x7fa7b40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b40cbd20 con 0x7fa7b415f130 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b415ba90 con 0x7fa7b40cf590 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40cff30 con 0x7fa7b415b6b0 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa7b40cf590 0x7fa7b40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa7b40cf590 0x7fa7b40cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33630/0 (socket says 172.21.15.82:33630) 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 -- 172.21.15.82:0/2835958345 learned_addr learned my addr 172.21.15.82:0/2835958345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7ba3e4640 1 --1- 172.21.15.82:0/2835958345 >> v1:172.21.15.82:6789/0 conn(0x7fa7b415f130 0x7fa7b415f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48326/0 (socket says 172.21.15.82:48326) 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 -- 172.21.15.82:0/2835958345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b415b6b0 msgr2=0x7fa7b40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b9be3640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b415b6b0 0x7fa7b40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b415b6b0 0x7fa7b40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 -- 172.21.15.82:0/2835958345 >> v1:172.21.15.82:6789/0 conn(0x7fa7b415f130 legacy=0x7fa7b415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 -- 172.21.15.82:0/2835958345 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40cc000 con 0x7fa7b40cf590 2024-04-25T10:11:15.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b8be1640 1 -- 172.21.15.82:0/2835958345 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396317573 0 0) 0x7fa7b40cbd20 con 0x7fa7b415f130 2024-04-25T10:11:15.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b93e2640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b40cf590 0x7fa7b40cf9f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a40cb9dff1a139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b8be1640 1 -- 172.21.15.82:0/2835958345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a58d0 con 0x7fa7b40cf590 2024-04-25T10:11:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b8be1640 1 -- 172.21.15.82:0/2835958345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b00a5a70 con 0x7fa7b40cf590 2024-04-25T10:11:15.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7b8be1640 1 -- 172.21.15.82:0/2835958345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a5e40 con 0x7fa7b40cf590 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 -- 172.21.15.82:0/2835958345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b40cf590 msgr2=0x7fa7b40cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b40cf590 0x7fa7b40cf9f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009e50 tx=0x7fa7b001ca00 comp rx=0 tx=0).stop 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 -- 172.21.15.82:0/2835958345 shutdown_connections 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b40cf590 0x7fa7b40cf9f0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.733+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/2835958345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b415b6b0 0x7fa7b40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/2835958345 >> 172.21.15.82:0/2835958345 conn(0x7fa7b406ae60 msgr2=0x7fa7b406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/2835958345 shutdown_connections 2024-04-25T10:11:15.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/2835958345 wait complete. 2024-04-25T10:11:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 Processor -- start 2024-04-25T10:11:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- start start 2024-04-25T10:11:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b40cf590 0x7fa7b4158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b415f130 0x7fa7b4150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40ca900 con 0x7fa7b415b6b0 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40cba30 con 0x7fa7b415f130 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40cadb0 con 0x7fa7b40cf590 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33638/0 (socket says 172.21.15.82:33638) 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 -- 172.21.15.82:0/965003941 learned_addr learned my addr 172.21.15.82:0/965003941 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:15.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 -- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b40cf590 msgr2=0x7fa7b4158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b40cf590 0x7fa7b4158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 -- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b415f130 msgr2=0x7fa7b4150e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b415f130 0x7fa7b4150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 -- 172.21.15.82:0/965003941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4158e50 con 0x7fa7b415b6b0 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7b93e2640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00af040 tx=0x7fa7b00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00bc020 con 0x7fa7b415b6b0 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b00b8470 con 0x7fa7b415b6b0 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00a58d0 con 0x7fa7b415b6b0 2024-04-25T10:11:15.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b4159130 con 0x7fa7b415b6b0 2024-04-25T10:11:15.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41596f0 con 0x7fa7b415b6b0 2024-04-25T10:11:15.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.737+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b404efa0 con 0x7fa7b415b6b0 2024-04-25T10:11:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.741+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7b00bf020 con 0x7fa7b415b6b0 2024-04-25T10:11:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.741+0000 7fa7aa7fc640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 0x7fa798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.741+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa7b0120950 con 0x7fa7b415b6b0 2024-04-25T10:11:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.741+0000 7fa7b9be3640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 0x7fa798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.741+0000 7fa7b9be3640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 0x7fa798069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00a8b0 tx=0x7fa7ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.745+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7b00c6030 con 0x7fa7b415b6b0 2024-04-25T10:11:15.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa7b406b2e0 con 0x7fa7b415b6b0 2024-04-25T10:11:15.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7aa7fc640 1 -- 172.21.15.82:0/965003941 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa7b0016620 con 0x7fa7b415b6b0 2024-04-25T10:11:15.892 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 msgr2=0x7fa798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 0x7fa798069420 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00a8b0 tx=0x7fa7ac008040 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 msgr2=0x7fa7b406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fa7b00af040 tx=0x7fa7b00197a0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 shutdown_connections 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa798066f60 0x7fa798069420 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa7b415f130 0x7fa7b4150e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa7b415b6b0 0x7fa7b406d9e0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 --2- 172.21.15.82:0/965003941 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa7b40cf590 0x7fa7b4158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.889+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 >> 172.21.15.82:0/965003941 conn(0x7fa7b406ae60 msgr2=0x7fa7b4156570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.893+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 shutdown_connections 2024-04-25T10:11:15.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:15.893+0000 7fa7bbe47640 1 -- 172.21.15.82:0/965003941 wait complete. 2024-04-25T10:11:15.909 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 Processor -- start 2024-04-25T10:11:16.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 -- start start 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2015b6b0 0x7fbd200ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fbd200cf590 0x7fbd200cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd200cbd20 con 0x7fbd2015f130 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd2015ba90 con 0x7fbd2015b6b0 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd26036640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200cc000 con 0x7fbd200cf590 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2015b6b0 0x7fbd200ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbd2015b6b0 0x7fbd200ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33652/0 (socket says 172.21.15.82:33652) 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1980816135 learned_addr learned my addr 172.21.15.82:0/1980816135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1effd640 1 --2- 172.21.15.82:0/1980816135 >> v2:172.21.15.196:3300/0 conn(0x7fbd200cf590 0x7fbd200cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1980816135 >> v2:172.21.15.196:3300/0 conn(0x7fbd200cf590 msgr2=0x7fbd200cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 --2- 172.21.15.82:0/1980816135 >> v2:172.21.15.196:3300/0 conn(0x7fbd200cf590 0x7fbd200cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1980816135 >> v1:172.21.15.196:6789/0 conn(0x7fbd2015f130 legacy=0x7fbd2015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.029+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1980816135 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200cc2e0 con 0x7fbd2015b6b0 2024-04-25T10:11:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1f7fe640 1 --2- 172.21.15.82:0/1980816135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2015b6b0 0x7fbd200ca3c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b4430 tx=0x7fbd1001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53654ab8f085a09e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1e7fc640 1 -- 172.21.15.82:0/1980816135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100b1070 con 0x7fbd2015b6b0 2024-04-25T10:11:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1e7fc640 1 -- 172.21.15.82:0/1980816135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd100ac630 con 0x7fbd2015b6b0 2024-04-25T10:11:16.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1e7fc640 1 -- 172.21.15.82:0/1980816135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100a8050 con 0x7fbd2015b6b0 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- 172.21.15.82:0/1980816135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2015b6b0 msgr2=0x7fbd200ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- 172.21.15.82:0/1980816135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2015b6b0 0x7fbd200ca3c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b4430 tx=0x7fbd1001c4e0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- 172.21.15.82:0/1980816135 shutdown_connections 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- 172.21.15.82:0/1980816135 >> v2:172.21.15.196:3300/0 conn(0x7fbd200cf590 0x7fbd200cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- 172.21.15.82:0/1980816135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd2015b6b0 0x7fbd200ca3c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- 172.21.15.82:0/1980816135 >> 172.21.15.82:0/1980816135 conn(0x7fbd2006ae60 msgr2=0x7fbd2006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- 172.21.15.82:0/1980816135 shutdown_connections 2024-04-25T10:11:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- 172.21.15.82:0/1980816135 wait complete. 2024-04-25T10:11:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 Processor -- start 2024-04-25T10:11:16.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- start start 2024-04-25T10:11:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2015b6b0 0x7fbd2015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2015f130 0x7fbd200adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200cadb0 con 0x7fbd200cf590 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200ca900 con 0x7fbd2015f130 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd26036640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200caf30 con 0x7fbd2015b6b0 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33668/0 (socket says 172.21.15.82:33668) 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.033+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1206275080 learned_addr learned my addr 172.21.15.82:0/1206275080 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1ffff640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2015f130 0x7fbd200adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1effd640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2015b6b0 0x7fbd2015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2015b6b0 msgr2=0x7fbd2015e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2015b6b0 0x7fbd2015e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2015f130 msgr2=0x7fbd200adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2015f130 0x7fbd200adab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 -- 172.21.15.82:0/1206275080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd201648c0 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1f7fe640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fbd100af040 tx=0x7fbd10019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100bc070 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd100b7430 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd20164ba0 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd100b1040 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2015de10 con 0x7fbd200cf590 2024-04-25T10:11:16.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd201650c0 con 0x7fbd200cf590 2024-04-25T10:11:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbd100bf070 con 0x7fbd200cf590 2024-04-25T10:11:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fbd1011eab0 con 0x7fbd200cf590 2024-04-25T10:11:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1effd640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.037+0000 7fbd1effd640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b0930 tx=0x7fbd14008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.041+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbd100160d0 con 0x7fbd200cf590 2024-04-25T10:11:16.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.197+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd2004efa0 con 0x7fbd200cf590 2024-04-25T10:11:16.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.197+0000 7fbd07fff640 1 -- 172.21.15.82:0/1206275080 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd100ef980 con 0x7fbd200cf590 2024-04-25T10:11:16.202 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 msgr2=0x7fbd0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fbd200b0930 tx=0x7fbd14008040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 msgr2=0x7fbd20098780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fbd100af040 tx=0x7fbd10019280 comp rx=0 tx=0).stop 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 shutdown_connections 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbd0c066f60 0x7fbd0c069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbd2015b6b0 0x7fbd2015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbd2015f130 0x7fbd200adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 --2- 172.21.15.82:0/1206275080 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbd200cf590 0x7fbd20098780 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 >> 172.21.15.82:0/1206275080 conn(0x7fbd2006ae60 msgr2=0x7fbd200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 shutdown_connections 2024-04-25T10:11:16.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.201+0000 7fbd26036640 1 -- 172.21.15.82:0/1206275080 wait complete. 2024-04-25T10:11:16.220 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.321+0000 7fe7fa5c7640 1 Processor -- start 2024-04-25T10:11:16.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.321+0000 7fe7fa5c7640 1 -- start start 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7fa5c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7fa5c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7f41451f0 0x7fe7f4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7fa5c7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7f40cee30 con 0x7fe7f4144ca0 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7fa5c7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7f40cc2b0 con 0x7fe7f41451f0 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7fa5c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406e290 con 0x7fe7f40ce510 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7f41451f0 0x7fe7f4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe7f41451f0 0x7fe7f4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33672/0 (socket says 172.21.15.82:33672) 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 -- 172.21.15.82:0/3790480409 learned_addr learned my addr 172.21.15.82:0/3790480409 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f3fff640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 -- 172.21.15.82:0/3790480409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 msgr2=0x7fe7f40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f40ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 -- 172.21.15.82:0/3790480409 >> v1:172.21.15.196:6789/0 conn(0x7fe7f4144ca0 legacy=0x7fe7f406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 -- 172.21.15.82:0/3790480409 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f40ccc70 con 0x7fe7f41451f0 2024-04-25T10:11:16.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f8b64640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f4142680 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009870 tx=0x7fe7ec01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7febe744318ab2c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f2ffd640 1 -- 172.21.15.82:0/3790480409 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0b5070 con 0x7fe7f41451f0 2024-04-25T10:11:16.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f2ffd640 1 -- 172.21.15.82:0/3790480409 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0a89d0 con 0x7fe7f41451f0 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.325+0000 7fe7f2ffd640 1 -- 172.21.15.82:0/3790480409 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0a8da0 con 0x7fe7f41451f0 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/3790480409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 msgr2=0x7fe7f4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f4142680 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009870 tx=0x7fe7ec01ed10 comp rx=0 tx=0).stop 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/3790480409 shutdown_connections 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f4142680 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/3790480409 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/3790480409 >> 172.21.15.82:0/3790480409 conn(0x7fe7f40b11a0 msgr2=0x7fe7f40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/3790480409 shutdown_connections 2024-04-25T10:11:16.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/3790480409 wait complete. 2024-04-25T10:11:16.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 Processor -- start 2024-04-25T10:11:16.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- start start 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f4144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f4144ca0 0x7fe7f4135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406ee30 con 0x7fe7f41451f0 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f406df20 con 0x7fe7f4144ca0 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7f40cc900 con 0x7fe7f40ce510 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33676/0 (socket says 172.21.15.82:33676) 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 -- 172.21.15.82:0/2968523539 learned_addr learned my addr 172.21.15.82:0/2968523539 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f3fff640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f4144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f37fe640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f4144ca0 0x7fe7f4135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 -- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 msgr2=0x7fe7f4144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f4144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 -- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f4144ca0 msgr2=0x7fe7f4135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f4144ca0 0x7fe7f4135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 -- 172.21.15.82:0/2968523539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f41e2250 con 0x7fe7f41451f0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f8b64640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009870 tx=0x7fe7ec01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0c0070 con 0x7fe7f41451f0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ec0b09f0 con 0x7fe7f41451f0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7f41e2520 con 0x7fe7f41451f0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0b5040 con 0x7fe7f41451f0 2024-04-25T10:11:16.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.329+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7f41e2ae0 con 0x7fe7f41451f0 2024-04-25T10:11:16.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7f404efa0 con 0x7fe7f41451f0 2024-04-25T10:11:16.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe7ec0c3070 con 0x7fe7f41451f0 2024-04-25T10:11:16.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7f0ff9640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 0x7fe7d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fe7ec1222e0 con 0x7fe7f41451f0 2024-04-25T10:11:16.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7f3fff640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 0x7fe7d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.333+0000 7fe7f3fff640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 0x7fe7d4069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a970 tx=0x7fe7e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.337+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7ec018870 con 0x7fe7f41451f0 2024-04-25T10:11:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.529+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7f40cd8e0 con 0x7fe7f41451f0 2024-04-25T10:11:16.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7f0ff9640 1 -- 172.21.15.82:0/2968523539 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fe7ec0f3040 con 0x7fe7f41451f0 2024-04-25T10:11:16.535 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:16.535 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:16.535 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:11:16.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 msgr2=0x7fe7d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 0x7fe7d4069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe7e400a970 tx=0x7fe7e4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 msgr2=0x7fe7f41e1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009870 tx=0x7fe7ec01bab0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 shutdown_connections 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe7d4066f60 0x7fe7d4069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe7f41451f0 0x7fe7f41e1d10 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe7f4144ca0 0x7fe7f4135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 --2- 172.21.15.82:0/2968523539 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe7f40ce510 0x7fe7f4144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 >> 172.21.15.82:0/2968523539 conn(0x7fe7f40b11a0 msgr2=0x7fe7f40c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 shutdown_connections 2024-04-25T10:11:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.533+0000 7fe7fa5c7640 1 -- 172.21.15.82:0/2968523539 wait complete. 2024-04-25T10:11:16.553 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:16.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 Processor -- start 2024-04-25T10:11:16.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 -- start start 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2dc0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2dc09a180 0x7fa2dc15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2dc0b0e10 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0d90 con 0x7fa2dc09a180 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2e1bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0180 con 0x7fa2dc0b0a30 2024-04-25T10:11:16.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2daffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa2dc06f4a0 0x7fa2dc099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48338/0 (socket says 172.21.15.82:48338) 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2daffd640 1 -- 172.21.15.82:0/2414124116 learned_addr learned my addr 172.21.15.82:0/2414124116 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2dbfff640 1 --2- 172.21.15.82:0/2414124116 >> v2:172.21.15.196:3300/0 conn(0x7fa2dc09a180 0x7fa2dc15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128561337 0 0) 0x7fa2dc0b0e10 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2c4003530 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.657+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2dc0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301114736 0 0) 0x7fa2c4003530 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 msgr2=0x7fa2dc0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2dc0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 >> v2:172.21.15.196:3300/0 conn(0x7fa2dc09a180 msgr2=0x7fa2dc15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 --2- 172.21.15.82:0/2414124116 >> v2:172.21.15.196:3300/0 conn(0x7fa2dc09a180 0x7fa2dc15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0d0460 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa2d0001680 con 0x7fa2dc06f4a0 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 >> v1:172.21.15.82:6789/0 conn(0x7fa2dc06f4a0 legacy=0x7fa2dc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c4004d50 0x7fa2c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0d90 con 0x7fa2c4003a30 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc0d0180 con 0x7fa2c4004d50 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c400ba50 con 0x7fa2dc0b0a30 2024-04-25T10:11:16.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2daffd640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 -- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 msgr2=0x7fa2c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc0b0a30 0x7fa2c4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 -- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c4004d50 msgr2=0x7fa2c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2c4004d50 0x7fa2c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 -- 172.21.15.82:0/2414124116 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c4003530 con 0x7fa2c4003a30 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2c4003e50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ba10 tx=0x7fa2cc01c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd93fd04a743d429 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc00b5e0 con 0x7fa2c4003a30 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0b0070 con 0x7fa2c4003a30 2024-04-25T10:11:16.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2da7fc640 1 -- 172.21.15.82:0/2414124116 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0ab840 con 0x7fa2c4003a30 2024-04-25T10:11:16.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/2414124116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 msgr2=0x7fa2c4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/2414124116 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2c4003e50 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00ba10 tx=0x7fa2cc01c610 comp rx=0 tx=0).stop 2024-04-25T10:11:16.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2dbfff640 1 -- 172.21.15.82:0/2414124116 reap_dead start 2024-04-25T10:11:16.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/2414124116 shutdown_connections 2024-04-25T10:11:16.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/2414124116 >> 172.21.15.82:0/2414124116 conn(0x7fa2dc0b30b0 msgr2=0x7fa2dc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/2414124116 shutdown_connections 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.661+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/2414124116 wait complete. 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 Processor -- start 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- start start 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc150ec0 0x7fa2dc0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2dc06ce60 0x7fa2dc06d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15db00 con 0x7fa2c4003a30 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15d980 con 0x7fa2dc06ce60 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2dc15dc80 con 0x7fa2dc150ec0 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33704/0 (socket says 172.21.15.82:33704) 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 -- 172.21.15.82:0/428943787 learned_addr learned my addr 172.21.15.82:0/428943787 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 -- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc150ec0 msgr2=0x7fa2dc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc150ec0 0x7fa2dc0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 -- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2dc06ce60 msgr2=0x7fa2dc06d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2dc06ce60 0x7fa2dc06d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 -- 172.21.15.82:0/428943787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2dc0c50b0 con 0x7fa2c4003a30 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2db7fe640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc01c440 tx=0x7fa2c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0ba070 con 0x7fa2c4003a30 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2cc0ab820 con 0x7fa2c4003a30 2024-04-25T10:11:16.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2dc0b66e0 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc0abbf0 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2dc0c5650 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2dc04efa0 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2cc0be400 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 0x7fa2c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.665+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fa2cc11f780 con 0x7fa2c4003a30 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.669+0000 7fa2daffd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 0x7fa2c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.669+0000 7fa2daffd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 0x7fa2c8069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0001ec0 tx=0x7fa2d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.673+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2cc016350 con 0x7fa2c4003a30 2024-04-25T10:11:16.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.829+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2dc163470 con 0x7fa2c4003a30 2024-04-25T10:11:16.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2c3fff640 1 -- 172.21.15.82:0/428943787 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17347 (secure 0 0 0) 0x7fa2cc11de20 con 0x7fa2c4003a30 2024-04-25T10:11:16.835 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:16.835 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:10:33.173812+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6836/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:16.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 msgr2=0x7fa2c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 0x7fa2c8069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0001ec0 tx=0x7fa2d0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 msgr2=0x7fa2dc14e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc01c440 tx=0x7fa2c4005840 comp rx=0 tx=0).stop 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 shutdown_connections 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2c8066f90 0x7fa2c8069450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2dc150ec0 0x7fa2dc0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2dc06ce60 0x7fa2dc06d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 --2- 172.21.15.82:0/428943787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2c4003a30 0x7fa2dc14e5d0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 >> 172.21.15.82:0/428943787 conn(0x7fa2dc0b30b0 msgr2=0x7fa2dc15c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 shutdown_connections 2024-04-25T10:11:16.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.833+0000 7fa2e1bfd640 1 -- 172.21.15.82:0/428943787 wait complete. 2024-04-25T10:11:16.853 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:11:16.854 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:16.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.961+0000 7f37d1807640 1 Processor -- start 2024-04-25T10:11:16.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.961+0000 7f37d1807640 1 -- start start 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37cc15da10 0x7f37cc071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc162310 con 0x7f37cc071e90 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc0adbd0 con 0x7f37cc0b24a0 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc0ae860 con 0x7f37cc15da10 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f37cc15da10 0x7f37cc071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37cb7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f37cc071e90 0x7f37cc161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48354/0 (socket says 172.21.15.82:48354) 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37cb7fe640 1 -- 172.21.15.82:0/2348112768 learned_addr learned my addr 172.21.15.82:0/2348112768 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 --2- 172.21.15.82:0/2348112768 >> v2:172.21.15.82:3300/0 conn(0x7f37cc15da10 0x7f37cc071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33720/0 (socket says 172.21.15.82:33720) 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 -- 172.21.15.82:0/2348112768 >> v1:172.21.15.196:6789/0 conn(0x7f37cc0b24a0 legacy=0x7f37cc1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 -- 172.21.15.82:0/2348112768 >> v1:172.21.15.82:6789/0 conn(0x7f37cc071e90 legacy=0x7f37cc161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 -- 172.21.15.82:0/2348112768 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc15e210 con 0x7f37cc15da10 2024-04-25T10:11:16.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37caffd640 1 --2- 172.21.15.82:0/2348112768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc15da10 0x7f37cc071030 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0097c0 tx=0x7f37bc01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd8bde775d4b9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37ca7fc640 1 -- 172.21.15.82:0/2348112768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b0070 con 0x7f37cc15da10 2024-04-25T10:11:16.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37ca7fc640 1 -- 172.21.15.82:0/2348112768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc0ab600 con 0x7f37cc15da10 2024-04-25T10:11:16.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37ca7fc640 1 -- 172.21.15.82:0/2348112768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0ab9d0 con 0x7f37cc15da10 2024-04-25T10:11:16.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- 172.21.15.82:0/2348112768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc15da10 msgr2=0x7f37cc071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 --2- 172.21.15.82:0/2348112768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc15da10 0x7f37cc071030 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f37bc0097c0 tx=0x7f37bc01c340 comp rx=0 tx=0).stop 2024-04-25T10:11:16.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- 172.21.15.82:0/2348112768 shutdown_connections 2024-04-25T10:11:16.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 --2- 172.21.15.82:0/2348112768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc15da10 0x7f37cc071030 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- 172.21.15.82:0/2348112768 >> 172.21.15.82:0/2348112768 conn(0x7f37cc0b11a0 msgr2=0x7f37cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- 172.21.15.82:0/2348112768 shutdown_connections 2024-04-25T10:11:16.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.965+0000 7f37d1807640 1 -- 172.21.15.82:0/2348112768 wait complete. 2024-04-25T10:11:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 Processor -- start 2024-04-25T10:11:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- start start 2024-04-25T10:11:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 0x7f37cc1534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37cc15da10 0x7f37cc1601e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc15ee70 con 0x7f37cc071e90 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc15ecf0 con 0x7f37cc0b24a0 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc16c4c0 con 0x7f37cc15da10 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33724/0 (socket says 172.21.15.82:33724) 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 -- 172.21.15.82:0/2602598693 learned_addr learned my addr 172.21.15.82:0/2602598693 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37c3fff640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 0x7f37cc1534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 -- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37cc15da10 msgr2=0x7f37cc1601e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37cc15da10 0x7f37cc1601e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 -- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 msgr2=0x7f37cc1534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 0x7f37cc1534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 -- 172.21.15.82:0/2602598693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc15c6f0 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37c3fff640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 0x7f37cc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37caffd640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f37bc018f50 tx=0x7f37bc0190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b8020 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc0b4630 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc1597e0 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0b0040 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc159da0 con 0x7f37cc071e90 2024-04-25T10:11:16.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.969+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc04efa0 con 0x7f37cc071e90 2024-04-25T10:11:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.973+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37bc0bb020 con 0x7f37cc071e90 2024-04-25T10:11:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.973+0000 7f37c8ff9640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.973+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f37bc11f0b0 con 0x7f37cc071e90 2024-04-25T10:11:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.973+0000 7f37c3fff640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.973+0000 7f37c3fff640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 0x7f37ac069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37cc169860 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:16.977+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37bc0c2030 con 0x7f37cc071e90 2024-04-25T10:11:17.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.153+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37cc0b1dc0 con 0x7f37cc071e90 2024-04-25T10:11:17.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.153+0000 7f37c8ff9640 1 -- 172.21.15.82:0/2602598693 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f37bc016050 con 0x7f37cc071e90 2024-04-25T10:11:17.157 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:17.157 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:17.158 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 msgr2=0x7f37ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 0x7f37ac069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f37cc169860 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 msgr2=0x7f37cc15c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f37bc018f50 tx=0x7f37bc0190e0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 shutdown_connections 2024-04-25T10:11:17.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f37ac066f60 0x7f37ac069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f37cc15da10 0x7f37cc1601e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f37cc0b24a0 0x7f37cc1534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 --2- 172.21.15.82:0/2602598693 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f37cc071e90 0x7f37cc15c1b0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 >> 172.21.15.82:0/2602598693 conn(0x7f37cc0b11a0 msgr2=0x7f37cc0acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 shutdown_connections 2024-04-25T10:11:17.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.157+0000 7f37d1807640 1 -- 172.21.15.82:0/2602598693 wait complete. 2024-04-25T10:11:17.175 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:17.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 Processor -- start 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 -- start start 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f24480b2e20 0x7f244806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244815ae30 con 0x7f24480b3200 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244815b110 con 0x7f244806d5d0 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f245049a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24480b2990 con 0x7f24480b2e20 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244da35640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f24480b3200 0x7f2448159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48370/0 (socket says 172.21.15.82:48370) 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244da35640 1 -- 172.21.15.82:0/328274571 learned_addr learned my addr 172.21.15.82:0/328274571 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244e236640 1 --2- 172.21.15.82:0/328274571 >> v2:172.21.15.196:3300/0 conn(0x7f24480b2e20 0x7f244806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3692821022 0 0) 0x7f244815ae30 con 0x7f24480b3200 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2434003530 con 0x7f24480b3200 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3434283619 0 0) 0x7f2434003530 con 0x7f24480b3200 2024-04-25T10:11:17.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 >> v2:172.21.15.196:3300/0 conn(0x7f24480b2e20 msgr2=0x7f244806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.281+0000 7f244d234640 1 --2- 172.21.15.82:0/328274571 >> v2:172.21.15.196:3300/0 conn(0x7f24480b2e20 0x7f244806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 >> v1:172.21.15.196:6789/0 conn(0x7f244806d5d0 legacy=0x7f244815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480b2620 con 0x7f24480b3200 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2438001680 con 0x7f24480b3200 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 >> v1:172.21.15.82:6789/0 conn(0x7f24480b3200 legacy=0x7f2448159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f2434004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24480b2990 con 0x7f24340051d0 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243400c100 con 0x7f2434003a30 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243400c300 con 0x7f2434004810 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 -- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 msgr2=0x7f2434004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f2434004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 -- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2434003a30 msgr2=0x7f2434003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 -- 172.21.15.82:0/328274571 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434003530 con 0x7f24340051d0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 -- 172.21.15.82:0/328274571 reap_dead start 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244ea37640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f2440001e70 tx=0x7f244001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305356d99d891c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440001a90 con 0x7f24340051d0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2440003bc0 con 0x7f24340051d0 2024-04-25T10:11:17.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f244d234640 1 -- 172.21.15.82:0/328274571 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440005020 con 0x7f24340051d0 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 -- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 msgr2=0x7f243400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f2440001e70 tx=0x7f244001c780 comp rx=0 tx=0).stop 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 -- 172.21.15.82:0/328274571 shutdown_connections 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 --2- 172.21.15.82:0/328274571 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 -- 172.21.15.82:0/328274571 >> 172.21.15.82:0/328274571 conn(0x7f24480b11a0 msgr2=0x7f24480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 -- 172.21.15.82:0/328274571 shutdown_connections 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.285+0000 7f245049a640 1 -- 172.21.15.82:0/328274571 wait complete. 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 Processor -- start 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- start start 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24480a8e50 0x7f24481f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2448152b10 0x7f24480ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244806ee20 con 0x7f24340051d0 2024-04-25T10:11:17.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244806eca0 con 0x7f2448152b10 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244806d090 con 0x7f24480a8e50 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33746/0 (socket says 172.21.15.82:33746) 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 -- 172.21.15.82:0/929512723 learned_addr learned my addr 172.21.15.82:0/929512723 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244da35640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24480a8e50 0x7f24481f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 -- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24480a8e50 msgr2=0x7f24481f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24480a8e50 0x7f24481f3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 -- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2448152b10 msgr2=0x7f24480ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2448152b10 0x7f24480ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 -- 172.21.15.82:0/929512723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24480a87e0 con 0x7f24340051d0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244e236640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f244400e7c0 tx=0x7f244400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444017070 con 0x7f24340051d0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244400ee70 con 0x7f24340051d0 2024-04-25T10:11:17.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244815b540 con 0x7f24340051d0 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400cb40 con 0x7f24340051d0 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244815bb00 con 0x7f24340051d0 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244804efa0 con 0x7f24340051d0 2024-04-25T10:11:17.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f244401a070 con 0x7f24340051d0 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 0x7f242c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f24440851b0 con 0x7f24340051d0 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.289+0000 7f244da35640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 0x7f242c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.293+0000 7f244da35640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 0x7f242c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f24380011f0 tx=0x7f2438009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.293+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2444055f10 con 0x7f24340051d0 2024-04-25T10:11:17.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.473+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f244806e6b0 con 0x7f24340051d0 2024-04-25T10:11:17.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.473+0000 7f243effd640 1 -- 172.21.15.82:0/929512723 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f2444055810 con 0x7f24340051d0 2024-04-25T10:11:17.478 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:17.478 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:49.240218+0000","modified":"2024-04-25T10:10:36.212278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":95,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8376},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8376":{"gid":8376,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/570372586","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":570372586},{"type":"v1","addr":"172.21.15.82:6837","nonce":570372586}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:17.478 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 69 2024-04-25T10:11:17.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 msgr2=0x7f242c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 0x7f242c069420 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f24380011f0 tx=0x7f2438009040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 msgr2=0x7f2448153240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f244400e7c0 tx=0x7f244400ec80 comp rx=0 tx=0).stop 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 shutdown_connections 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f242c066f60 0x7f242c069420 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f24480a8e50 0x7f24481f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2448152b10 0x7f24480ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 --2- 172.21.15.82:0/929512723 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f24340051d0 0x7f2448153240 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 >> 172.21.15.82:0/929512723 conn(0x7f24480b11a0 msgr2=0x7f2448071890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 shutdown_connections 2024-04-25T10:11:17.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.477+0000 7f245049a640 1 -- 172.21.15.82:0/929512723 wait complete. 2024-04-25T10:11:17.495 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.601+0000 7f788d303640 1 Processor -- start 2024-04-25T10:11:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.601+0000 7f788d303640 1 -- start start 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f788d303640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f788815ed40 0x7f78880d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f788d303640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 0x7f78880c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f788d303640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880affa0 con 0x7f78880c6e50 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f788d303640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78880c72c0 con 0x7f788815ed40 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f788d303640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815f120 con 0x7f7888160080 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f788815ed40 0x7f78880d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f78877fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78880c6e50 0x7f788806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48376/0 (socket says 172.21.15.82:48376) 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f78877fe640 1 -- 172.21.15.82:0/1335377242 learned_addr learned my addr 172.21.15.82:0/1335377242 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:17.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f78867fc640 1 --2- 172.21.15.82:0/1335377242 >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 0x7f78880c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 -- 172.21.15.82:0/1335377242 >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 msgr2=0x7f78880c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 --2- 172.21.15.82:0/1335377242 >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 0x7f78880c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 -- 172.21.15.82:0/1335377242 >> v1:172.21.15.82:6789/0 conn(0x7f78880c6e50 legacy=0x7f788806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 -- 172.21.15.82:0/1335377242 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880b0240 con 0x7f788815ed40 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7885ffb640 1 -- 172.21.15.82:0/1335377242 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2632729369 0 0) 0x7f78880affa0 con 0x7f78880c6e50 2024-04-25T10:11:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f78867fc640 1 --2- 172.21.15.82:0/1335377242 >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 0x7f78880c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7886ffd640 1 --2- 172.21.15.82:0/1335377242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788815ed40 0x7f78880d0c20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f788806bd30 tx=0x7f787801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52322b7ceec2e40b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7885ffb640 1 -- 172.21.15.82:0/1335377242 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780b0070 con 0x7f788815ed40 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7885ffb640 1 -- 172.21.15.82:0/1335377242 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78780ab600 con 0x7f788815ed40 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.605+0000 7f7885ffb640 1 -- 172.21.15.82:0/1335377242 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78780ab9d0 con 0x7f788815ed40 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/1335377242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788815ed40 msgr2=0x7f78880d0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- 172.21.15.82:0/1335377242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788815ed40 0x7f78880d0c20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f788806bd30 tx=0x7f787801c3c0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/1335377242 shutdown_connections 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- 172.21.15.82:0/1335377242 >> v2:172.21.15.196:3300/0 conn(0x7f7888160080 0x7f78880c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- 172.21.15.82:0/1335377242 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f788815ed40 0x7f78880d0c20 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/1335377242 >> 172.21.15.82:0/1335377242 conn(0x7f7888099a70 msgr2=0x7f78880d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/1335377242 shutdown_connections 2024-04-25T10:11:17.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/1335377242 wait complete. 2024-04-25T10:11:17.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 Processor -- start 2024-04-25T10:11:17.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- start start 2024-04-25T10:11:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78880c6e50 0x7f788814c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f788815ed40 0x7f78880c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880cbae0 con 0x7f7888160080 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880d1160 con 0x7f78880c6e50 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78880cbc60 con 0x7f788815ed40 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33764/0 (socket says 172.21.15.82:33764) 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 -- 172.21.15.82:0/4137146341 learned_addr learned my addr 172.21.15.82:0/4137146341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f7886ffd640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78880c6e50 0x7f788814c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78867fc640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f788815ed40 0x7f78880c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 -- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f788815ed40 msgr2=0x7f78880c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f788815ed40 0x7f78880c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 -- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78880c6e50 msgr2=0x7f788814c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78880c6e50 0x7f788814c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 -- 172.21.15.82:0/4137146341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78881f9910 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f78877fe640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f788000c700 tx=0x7f788000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7880018070 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788000ce10 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78881f9bf0 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788000a880 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.609+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78881fa1b0 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78880ce100 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f788001b070 con 0x7f7888160080 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f786f7fe640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7880084870 con 0x7f7888160080 2024-04-25T10:11:17.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f7886ffd640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.613+0000 7f7886ffd640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 0x7f7874069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78780ad000 tx=0x7f78780a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.617+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78800555d0 con 0x7f7888160080 2024-04-25T10:11:17.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.761+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f788804ef80 con 0x7f7888160080 2024-04-25T10:11:17.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.765+0000 7f786f7fe640 1 -- 172.21.15.82:0/4137146341 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+17347 (secure 0 0 0) 0x7f7880054ed0 con 0x7f7888160080 2024-04-25T10:11:17.768 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:17.768 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":95,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:10:33.173812+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:47.469954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:48.247943+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.1","mappings":[{"from":1,"to":7}]},{"pgid":"14.21","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":2}]},{"pgid":"14.33","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6836/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.765+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 msgr2=0x7f7874069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.765+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 0x7f7874069420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78780ad000 tx=0x7f78780a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 msgr2=0x7f788815f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f788000c700 tx=0x7f788000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 shutdown_connections 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7874066f60 0x7f7874069420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f788815ed40 0x7f78880c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7888160080 0x7f788815f330 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 --2- 172.21.15.82:0/4137146341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78880c6e50 0x7f788814c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 >> 172.21.15.82:0/4137146341 conn(0x7f7888099a70 msgr2=0x7f788806e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 shutdown_connections 2024-04-25T10:11:17.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.769+0000 7f788d303640 1 -- 172.21.15.82:0/4137146341 wait complete. 2024-04-25T10:11:17.786 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 Processor -- start 2024-04-25T10:11:17.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 -- start start 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7c540d4230 0x7f7c540d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5406be60 con 0x7f7c540d07f0 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c540d4650 con 0x7f7c5415da80 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c58a6f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c540d2210 con 0x7f7c540d4230 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c51d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7c540d07f0 0x7f7c5406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48388/0 (socket says 172.21.15.82:48388) 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c51d9b640 1 -- 172.21.15.82:0/4097918077 learned_addr learned my addr 172.21.15.82:0/4097918077 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.893+0000 7f7c52d9d640 1 --2- 172.21.15.82:0/4097918077 >> v2:172.21.15.82:3300/0 conn(0x7f7c540d4230 0x7f7c540d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c5159a640 1 -- 172.21.15.82:0/4097918077 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991492301 0 0) 0x7f7c5406be60 con 0x7f7c540d07f0 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c5159a640 1 -- 172.21.15.82:0/4097918077 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c3c003530 con 0x7f7c540d07f0 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c52d9d640 1 -- 172.21.15.82:0/4097918077 >> v1:172.21.15.196:6789/0 conn(0x7f7c5415da80 legacy=0x7f7c540d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c52d9d640 1 -- 172.21.15.82:0/4097918077 >> v1:172.21.15.82:6789/0 conn(0x7f7c540d07f0 legacy=0x7f7c5406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c52d9d640 1 -- 172.21.15.82:0/4097918077 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5415de60 con 0x7f7c540d4230 2024-04-25T10:11:17.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c52d9d640 1 --2- 172.21.15.82:0/4097918077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c540d4230 0x7f7c540d1cd0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0097c0 tx=0x7f7c4c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe28b39ecbb21fcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c5159a640 1 -- 172.21.15.82:0/4097918077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0b0070 con 0x7f7c540d4230 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c5159a640 1 -- 172.21.15.82:0/4097918077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c0ab600 con 0x7f7c540d4230 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c5159a640 1 -- 172.21.15.82:0/4097918077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0ab9d0 con 0x7f7c540d4230 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 -- 172.21.15.82:0/4097918077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c540d4230 msgr2=0x7f7c540d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/4097918077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c540d4230 0x7f7c540d1cd0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0097c0 tx=0x7f7c4c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 -- 172.21.15.82:0/4097918077 shutdown_connections 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/4097918077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c540d4230 0x7f7c540d1cd0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 -- 172.21.15.82:0/4097918077 >> 172.21.15.82:0/4097918077 conn(0x7f7c5408b000 msgr2=0x7f7c540d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 -- 172.21.15.82:0/4097918077 shutdown_connections 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.897+0000 7f7c58a6f640 1 -- 172.21.15.82:0/4097918077 wait complete. 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 Processor -- start 2024-04-25T10:11:17.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- start start 2024-04-25T10:11:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c540d07f0 0x7f7c5414f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 0x7f7c5415f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5406dbc0 con 0x7f7c5415da80 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5406da40 con 0x7f7c540d4230 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5416b5c0 con 0x7f7c540d07f0 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33774/0 (socket says 172.21.15.82:33774) 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 -- 172.21.15.82:0/3984856505 learned_addr learned my addr 172.21.15.82:0/3984856505 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 -- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c540d07f0 msgr2=0x7f7c5414f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c51d9b640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 0x7f7c5415f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c540d07f0 0x7f7c5414f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 -- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 msgr2=0x7f7c5415f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 0x7f7c5415f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 -- 172.21.15.82:0/3984856505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54160660 con 0x7f7c5415da80 2024-04-25T10:11:17.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c51d9b640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 0x7f7c5415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c52d9d640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0ae040 tx=0x7f7c4c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0b8020 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c0b4410 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5415fae0 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c0b0040 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c541600a0 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5404efa0 con 0x7f7c5415da80 2024-04-25T10:11:17.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7c4c0bb020 con 0x7f7c5415da80 2024-04-25T10:11:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.901+0000 7f7c42ffd640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 0x7f7c34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.905+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7c4c11ee20 con 0x7f7c5415da80 2024-04-25T10:11:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.905+0000 7f7c5259c640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 0x7f7c34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.905+0000 7f7c5259c640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 0x7f7c34069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7c44007fd0 tx=0x7f7c44007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:17.909+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7c4c0c2030 con 0x7f7c5415da80 2024-04-25T10:11:18.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.085+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7c540ca440 con 0x7f7c5415da80 2024-04-25T10:11:18.087 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:11:18.085+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:18.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.085+0000 7f7c42ffd640 1 -- 172.21.15.82:0/3984856505 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c4c016090 con 0x7f7c5415da80 2024-04-25T10:11:18.090 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:18.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 msgr2=0x7f7c34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 0x7f7c34069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7c44007fd0 tx=0x7f7c44007d60 comp rx=0 tx=0).stop 2024-04-25T10:11:18.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 msgr2=0x7f7c54153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0ae040 tx=0x7f7c4c019240 comp rx=0 tx=0).stop 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 shutdown_connections 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7c34066f60 0x7f7c34069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c5415da80 0x7f7c54153cf0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c540d4230 0x7f7c5415f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 --2- 172.21.15.82:0/3984856505 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7c540d07f0 0x7f7c5414f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 >> 172.21.15.82:0/3984856505 conn(0x7f7c5408b000 msgr2=0x7f7c540b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 shutdown_connections 2024-04-25T10:11:18.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.089+0000 7f7c58a6f640 1 -- 172.21.15.82:0/3984856505 wait complete. 2024-04-25T10:11:18.102 INFO:tasks.ceph.mds.c.smithi082.stderr: -2307> 2024-04-25T10:10:39.198+0000 7faf3998d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:18.104 INFO:tasks.ceph.mds.c.smithi082.stderr: -1884> 2024-04-25T10:10:41.442+0000 7faf35985640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55a92a57c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:11:18.110 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:11:18.140 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:11:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 Processor -- start 2024-04-25T10:11:18.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 -- start start 2024-04-25T10:11:18.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7770166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77700cbd20 con 0x7f777015b800 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77700cc000 con 0x7f777015b420 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f7774f17640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cc930 con 0x7f77700cfca0 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776dd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f777015b800 0x7f77700cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48402/0 (socket says 172.21.15.82:48402) 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776dd9b640 1 -- 172.21.15.82:0/1383947614 learned_addr learned my addr 172.21.15.82:0/1383947614 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054557922 0 0) 0x7f77700cbd20 con 0x7f777015b800 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7770166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7758003530 con 0x7f777015b800 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2275002585 0 0) 0x7f7758003530 con 0x7f777015b800 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 msgr2=0x7f7770166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7770166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 >> v1:172.21.15.196:6789/0 conn(0x7f777015b420 legacy=0x7f77700ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700cc2e0 con 0x7f777015b800 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7764001680 con 0x7f777015b800 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 >> v1:172.21.15.82:6789/0 conn(0x7f777015b800 legacy=0x7f77700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7758003a60 0x7f7758003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7758004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f7758005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cc930 con 0x7f7758004da0 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775800bbc0 con 0x7f7758003a60 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f775800bdc0 con 0x7f77700cfca0 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f7758005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776e59c640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7758003a60 0x7f7758003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776dd9b640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7758004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 -- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 msgr2=0x7f7758004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7758004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 -- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7758003a60 msgr2=0x7f7758003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7758003a60 0x7f7758003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776ed9d640 1 -- 172.21.15.82:0/1383947614 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7758003530 con 0x7f7758004da0 2024-04-25T10:11:18.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.237+0000 7f776dd9b640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700cfca0 0x7f7758004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776ed9d640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f7758005200 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f7768007330 tx=0x7f776801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2729adb325ef7a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77680b2070 con 0x7f7758004da0 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77680a5d20 con 0x7f7758004da0 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776d59a640 1 -- 172.21.15.82:0/1383947614 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7768009be0 con 0x7f7758004da0 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- 172.21.15.82:0/1383947614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 msgr2=0x7f7758005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 --2- 172.21.15.82:0/1383947614 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f7758005200 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f7768007330 tx=0x7f776801ce20 comp rx=0 tx=0).stop 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776ed9d640 1 -- 172.21.15.82:0/1383947614 reap_dead start 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- 172.21.15.82:0/1383947614 shutdown_connections 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- 172.21.15.82:0/1383947614 >> 172.21.15.82:0/1383947614 conn(0x7f777006ae60 msgr2=0x7f777006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- 172.21.15.82:0/1383947614 shutdown_connections 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- 172.21.15.82:0/1383947614 wait complete. 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 Processor -- start 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- start start 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700c6510 0x7f77700c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 0x7f77700c8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cadb0 con 0x7f7758004da0 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700cb9f0 con 0x7f77700c7f00 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f7774f17640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700caf30 con 0x7f77700c6510 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33806/0 (socket says 172.21.15.82:33806) 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.241+0000 7f776e59c640 1 -- 172.21.15.82:0/3196057454 learned_addr learned my addr 172.21.15.82:0/3196057454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776ed9d640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 0x7f77700c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776dd9b640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700c6510 0x7f77700c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 -- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700c6510 msgr2=0x7f77700c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700c6510 0x7f77700c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 -- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 msgr2=0x7f77700c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 0x7f77700c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 -- 172.21.15.82:0/3196057454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700c5300 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776ed9d640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 0x7f77700c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776e59c640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f776000e550 tx=0x7f776000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600052e0 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7760018070 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77700c55a0 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760003820 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77700c5b60 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777004efa0 con 0x7f7758004da0 2024-04-25T10:11:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f77600039c0 con 0x7f7758004da0 2024-04-25T10:11:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 0x7f7750069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f7760084c60 con 0x7f7758004da0 2024-04-25T10:11:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776dd9b640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 0x7f7750069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.245+0000 7f776dd9b640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 0x7f7750069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7764001210 tx=0x7f7764009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.249+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7760055950 con 0x7f7758004da0 2024-04-25T10:11:18.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:18.433+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f777015cca0 con 0x7f7758004da0 2024-04-25T10:11:19.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.089+0000 7f775effd640 1 -- 172.21.15.82:0/3196057454 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7760005480 con 0x7f7758004da0 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 msgr2=0x7f7750069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 0x7f7750069450 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7764001210 tx=0x7f7764009040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 msgr2=0x7f77701f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f776000e550 tx=0x7f776000ea10 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 shutdown_connections 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7750066f90 0x7f7750069450 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f77700c6510 0x7f77700c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f77700c7f00 0x7f77700c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 --2- 172.21.15.82:0/3196057454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7758004da0 0x7f77701f44f0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 >> 172.21.15.82:0/3196057454 conn(0x7f777006ae60 msgr2=0x7f77700c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 shutdown_connections 2024-04-25T10:11:19.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.093+0000 7f7774f17640 1 -- 172.21.15.82:0/3196057454 wait complete. 2024-04-25T10:11:19.119 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:11:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 Processor -- start 2024-04-25T10:11:19.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 -- start start 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 0x7f50080ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f500815a7a0 0x7f50080b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f500815e0f0 con 0x7f500815ac30 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50080b24a0 con 0x7f500815a7a0 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500eb24640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500815b0b0 con 0x7f500815a3c0 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f500815a7a0 0x7f50080b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f500815a7a0 0x7f50080b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33808/0 (socket says 172.21.15.82:33808) 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 -- 172.21.15.82:0/2502127539 learned_addr learned my addr 172.21.15.82:0/2502127539 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500d0c1640 1 --1- 172.21.15.82:0/2502127539 >> v1:172.21.15.82:6789/0 conn(0x7f500815ac30 0x7f50080cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48414/0 (socket says 172.21.15.82:48414) 2024-04-25T10:11:19.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500c8c0640 1 --2- 172.21.15.82:0/2502127539 >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 0x7f50080ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 -- 172.21.15.82:0/2502127539 >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 msgr2=0x7f50080ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 --2- 172.21.15.82:0/2502127539 >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 0x7f50080ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 -- 172.21.15.82:0/2502127539 >> v1:172.21.15.82:6789/0 conn(0x7f500815ac30 legacy=0x7f50080cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f5007fff640 1 -- 172.21.15.82:0/2502127539 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500815b390 con 0x7f500815a7a0 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f50077fe640 1 -- 172.21.15.82:0/2502127539 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887187865 0 0) 0x7f500815e0f0 con 0x7f500815ac30 2024-04-25T10:11:19.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.289+0000 7f500c8c0640 1 --2- 172.21.15.82:0/2502127539 >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 0x7f50080ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:19.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f5007fff640 1 --2- 172.21.15.82:0/2502127539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a7a0 0x7f50080b27a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f50080cbcc0 tx=0x7f4ff001ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91990dfde8696650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f50077fe640 1 -- 172.21.15.82:0/2502127539 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00b4070 con 0x7f500815a7a0 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f50077fe640 1 -- 172.21.15.82:0/2502127539 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff00a7cb0 con 0x7f500815a7a0 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f50077fe640 1 -- 172.21.15.82:0/2502127539 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff00a7e60 con 0x7f500815a7a0 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- 172.21.15.82:0/2502127539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a7a0 msgr2=0x7f50080b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- 172.21.15.82:0/2502127539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a7a0 0x7f50080b27a0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f50080cbcc0 tx=0x7f4ff001ebc0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- 172.21.15.82:0/2502127539 shutdown_connections 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- 172.21.15.82:0/2502127539 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a7a0 0x7f50080b27a0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- 172.21.15.82:0/2502127539 >> v2:172.21.15.196:3300/0 conn(0x7f500815a3c0 0x7f50080ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- 172.21.15.82:0/2502127539 >> 172.21.15.82:0/2502127539 conn(0x7f500806ae60 msgr2=0x7f500806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- 172.21.15.82:0/2502127539 shutdown_connections 2024-04-25T10:11:19.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- 172.21.15.82:0/2502127539 wait complete. 2024-04-25T10:11:19.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 Processor -- start 2024-04-25T10:11:19.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- start start 2024-04-25T10:11:19.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f500815a7a0 0x7f500815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500815ac30 0x7f50081585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500815edb0 con 0x7f500815a3c0 2024-04-25T10:11:19.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50080b2ef0 con 0x7f500815ac30 2024-04-25T10:11:19.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500eb24640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500815ef30 con 0x7f500815a7a0 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500c8c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.293+0000 7f500c8c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33812/0 (socket says 172.21.15.82:33812) 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 -- 172.21.15.82:0/3700344860 learned_addr learned my addr 172.21.15.82:0/3700344860 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500d0c1640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500815ac30 0x7f50081585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 -- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f500815a7a0 msgr2=0x7f500815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f500815a7a0 0x7f500815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 -- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500815ac30 msgr2=0x7f50081585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500815ac30 0x7f50081585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 -- 172.21.15.82:0/3700344860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500815c6c0 con 0x7f500815a3c0 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500c8c0640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800c7c0 tx=0x7f4ff800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8007a30 con 0x7f500815a3c0 2024-04-25T10:11:19.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff8007bd0 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500815c9a0 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff80113f0 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500815cf60 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5008067c30 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ff8011590 con 0x7f500815a3c0 2024-04-25T10:11:19.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 0x7f4fe8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f5007fff640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 0x7f4fe8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f4ff80854a0 con 0x7f500815a3c0 2024-04-25T10:11:19.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.297+0000 7f5007fff640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 0x7f4fe8069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00b1000 tx=0x7f4ff0004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.301+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ff8015020 con 0x7f500815a3c0 2024-04-25T10:11:19.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:19.461+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f500806edf0 con 0x7f500815a3c0 2024-04-25T10:11:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.097+0000 7f50057fa640 1 -- 172.21.15.82:0/3700344860 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4ff8056240 con 0x7f500815a3c0 2024-04-25T10:11:20.102 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:11:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 msgr2=0x7f4fe8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 0x7f4fe8069450 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f4ff00b1000 tx=0x7f4ff0004740 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 msgr2=0x7f50080c2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f4ff800c7c0 tx=0x7f4ff800cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 shutdown_connections 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4fe8066f90 0x7f4fe8069450 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f500815a7a0 0x7f500815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f500815ac30 0x7f50081585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 --2- 172.21.15.82:0/3700344860 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f500815a3c0 0x7f50080c2f30 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 >> 172.21.15.82:0/3700344860 conn(0x7f500806ae60 msgr2=0x7f5008072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 shutdown_connections 2024-04-25T10:11:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.101+0000 7f500eb24640 1 -- 172.21.15.82:0/3700344860 wait complete. 2024-04-25T10:11:20.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:11:20.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.253+0000 7f88d55d8640 1 Processor -- start 2024-04-25T10:11:20.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.253+0000 7f88d55d8640 1 -- start start 2024-04-25T10:11:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a3c0 0x7f88d00ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f88d015ac00 0x7f88d00cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d015e0f0 con 0x7f88d015a7a0 2024-04-25T10:11:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d00b2eb0 con 0x7f88d015ac00 2024-04-25T10:11:20.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b24a0 con 0x7f88d015a3c0 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88ce7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f88d015a7a0 0x7f88d00b27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48430/0 (socket says 172.21.15.82:48430) 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88ce7fc640 1 -- 172.21.15.82:0/1644010146 learned_addr learned my addr 172.21.15.82:0/1644010146 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 --2- 172.21.15.82:0/1644010146 >> v2:172.21.15.82:3300/0 conn(0x7f88d015ac00 0x7f88d00cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cdffb640 1 -- 172.21.15.82:0/1644010146 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778052933 0 0) 0x7f88d015e0f0 con 0x7f88d015a7a0 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cdffb640 1 -- 172.21.15.82:0/1644010146 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b8003530 con 0x7f88d015a7a0 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88ceffd640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a3c0 0x7f88d00ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 -- 172.21.15.82:0/1644010146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a3c0 msgr2=0x7f88d00ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a3c0 0x7f88d00ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 -- 172.21.15.82:0/1644010146 >> v1:172.21.15.82:6789/0 conn(0x7f88d015a7a0 legacy=0x7f88d00b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 -- 172.21.15.82:0/1644010146 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d015b0b0 con 0x7f88d015ac00 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cf7fe640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015ac00 0x7f88d00cf510 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097c0 tx=0x7f88c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bb026350480297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cdffb640 1 -- 172.21.15.82:0/1644010146 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c00b2070 con 0x7f88d015ac00 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cdffb640 1 -- 172.21.15.82:0/1644010146 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c00ad680 con 0x7f88d015ac00 2024-04-25T10:11:20.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88cdffb640 1 -- 172.21.15.82:0/1644010146 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c00ada50 con 0x7f88d015ac00 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- 172.21.15.82:0/1644010146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015ac00 msgr2=0x7f88d00cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015ac00 0x7f88d00cf510 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f88c00097c0 tx=0x7f88c001c470 comp rx=0 tx=0).stop 2024-04-25T10:11:20.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- 172.21.15.82:0/1644010146 shutdown_connections 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015ac00 0x7f88d00cf510 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1644010146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a3c0 0x7f88d00ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- 172.21.15.82:0/1644010146 >> 172.21.15.82:0/1644010146 conn(0x7f88d006ae60 msgr2=0x7f88d006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- 172.21.15.82:0/1644010146 shutdown_connections 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.257+0000 7f88d55d8640 1 -- 172.21.15.82:0/1644010146 wait complete. 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 Processor -- start 2024-04-25T10:11:20.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- start start 2024-04-25T10:11:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a7a0 0x7f88d0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88d015ac00 0x7f88d00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0167570 con 0x7f88d015a3c0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01673f0 con 0x7f88d015ac00 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d01676f0 con 0x7f88d015a7a0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33836/0 (socket says 172.21.15.82:33836) 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 -- 172.21.15.82:0/1655702092 learned_addr learned my addr 172.21.15.82:0/1655702092 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ce7fc640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a7a0 0x7f88d0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88cf7fe640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88d015ac00 0x7f88d00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 -- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a7a0 msgr2=0x7f88d0153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a7a0 0x7f88d0153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 -- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88d015ac00 msgr2=0x7f88d00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88d015ac00 0x7f88d00af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 -- 172.21.15.82:0/1655702092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00ace60 con 0x7f88d015a3c0 2024-04-25T10:11:20.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88ceffd640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88c800ce10 tx=0x7f88c800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8003ba0 con 0x7f88d015a3c0 2024-04-25T10:11:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c8003d40 con 0x7f88d015a3c0 2024-04-25T10:11:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d0153280 con 0x7f88d015a3c0 2024-04-25T10:11:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8004240 con 0x7f88d015a3c0 2024-04-25T10:11:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d0153840 con 0x7f88d015a3c0 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.261+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d0153070 con 0x7f88d015a3c0 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.265+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88c80043e0 con 0x7f88d015a3c0 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.265+0000 7f88b77fe640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 0x7f88bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.265+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f88c8085cf0 con 0x7f88d015a3c0 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.265+0000 7f88ce7fc640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 0x7f88bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.265+0000 7f88ce7fc640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 0x7f88bc069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007fd0 tx=0x7f88c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.269+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88c808a070 con 0x7f88d015a3c0 2024-04-25T10:11:20.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:20.429+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f88d0067c30 con 0x7f88d015a3c0 2024-04-25T10:11:21.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.101+0000 7f88b77fe640 1 -- 172.21.15.82:0/1655702092 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f88c8056bb0 con 0x7f88d015a3c0 2024-04-25T10:11:21.105 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:11:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.105+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 msgr2=0x7f88bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.105+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 0x7f88bc069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f88c4007fd0 tx=0x7f88c4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.105+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 msgr2=0x7f88d0098880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.105+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88c800ce10 tx=0x7f88c800a660 comp rx=0 tx=0).stop 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 shutdown_connections 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f88bc066f60 0x7f88bc069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f88d015a7a0 0x7f88d0153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f88d015ac00 0x7f88d00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 --2- 172.21.15.82:0/1655702092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f88d015a3c0 0x7f88d0098880 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 >> 172.21.15.82:0/1655702092 conn(0x7f88d006ae60 msgr2=0x7f88d00b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 shutdown_connections 2024-04-25T10:11:21.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.109+0000 7f88d55d8640 1 -- 172.21.15.82:0/1655702092 wait complete. 2024-04-25T10:11:21.134 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:11:21.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 Processor -- start 2024-04-25T10:11:21.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 -- start start 2024-04-25T10:11:21.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c06c3b0 0x7f028c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f028c0b2c30 0x7f028c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f028c066c30 con 0x7f028c0af280 2024-04-25T10:11:21.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f028c06daa0 con 0x7f028c0b2c30 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f0294415640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c0af660 con 0x7f028c06c3b0 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f02929b2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f028c0b2c30 0x7f028c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f02921b1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f028c0af280 0x7f028c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48434/0 (socket says 172.21.15.82:48434) 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.265+0000 7f02921b1640 1 -- 172.21.15.82:0/2403506628 learned_addr learned my addr 172.21.15.82:0/2403506628 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02919b0640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c06c3b0 0x7f028c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02929b2640 1 -- 172.21.15.82:0/2403506628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c06c3b0 msgr2=0x7f028c15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02929b2640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c06c3b0 0x7f028c15df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02929b2640 1 -- 172.21.15.82:0/2403506628 >> v1:172.21.15.82:6789/0 conn(0x7f028c0af280 legacy=0x7f028c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02929b2640 1 -- 172.21.15.82:0/2403506628 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c15e640 con 0x7f028c0b2c30 2024-04-25T10:11:21.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02929b2640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c0b2c30 0x7f028c06df30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f0284009d90 tx=0x7f028401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db89b398eccf380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02911af640 1 -- 172.21.15.82:0/2403506628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840b2070 con 0x7f028c0b2c30 2024-04-25T10:11:21.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02911af640 1 -- 172.21.15.82:0/2403506628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02840a5830 con 0x7f028c0b2c30 2024-04-25T10:11:21.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f02911af640 1 -- 172.21.15.82:0/2403506628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02840a5c00 con 0x7f028c0b2c30 2024-04-25T10:11:21.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 -- 172.21.15.82:0/2403506628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c0b2c30 msgr2=0x7f028c06df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c0b2c30 0x7f028c06df30 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f0284009d90 tx=0x7f028401c970 comp rx=0 tx=0).stop 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 -- 172.21.15.82:0/2403506628 shutdown_connections 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c0b2c30 0x7f028c06df30 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 --2- 172.21.15.82:0/2403506628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c06c3b0 0x7f028c15df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 -- 172.21.15.82:0/2403506628 >> 172.21.15.82:0/2403506628 conn(0x7f028c071810 msgr2=0x7f028c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 -- 172.21.15.82:0/2403506628 shutdown_connections 2024-04-25T10:11:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.269+0000 7f0294415640 1 -- 172.21.15.82:0/2403506628 wait complete. 2024-04-25T10:11:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 Processor -- start 2024-04-25T10:11:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- start start 2024-04-25T10:11:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c0af280 0x7f028c0ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f028c0b2c30 0x7f028c163d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c06f010 con 0x7f028c06c3b0 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028c06ee90 con 0x7f028c0b2c30 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f028c06f190 con 0x7f028c0af280 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33852/0 (socket says 172.21.15.82:33852) 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 -- 172.21.15.82:0/1677231768 learned_addr learned my addr 172.21.15.82:0/1677231768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 -- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c0af280 msgr2=0x7f028c0ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c0af280 0x7f028c0ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 -- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f028c0b2c30 msgr2=0x7f028c163d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f028c0b2c30 0x7f028c163d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 -- 172.21.15.82:0/1677231768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c163480 con 0x7f028c06c3b0 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f02921b1640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f028800c420 tx=0x7f028800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288017070 con 0x7f028c06c3b0 2024-04-25T10:11:21.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0288005cf0 con 0x7f028c06c3b0 2024-04-25T10:11:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f028c163760 con 0x7f028c06c3b0 2024-04-25T10:11:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02880072f0 con 0x7f028c06c3b0 2024-04-25T10:11:21.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028c1656e0 con 0x7f028c06c3b0 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f028c04efa0 con 0x7f028c06c3b0 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f028801a070 con 0x7f028c06c3b0 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.273+0000 7f0282ffd640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 0x7f0270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.277+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0288084340 con 0x7f028c06c3b0 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.277+0000 7f02919b0640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 0x7f0270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.277+0000 7f02919b0640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 0x7f0270069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f028c15dae0 tx=0x7f027c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.281+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0288055410 con 0x7f028c06c3b0 2024-04-25T10:11:21.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.449+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f028c1658b0 con 0x7f028c06c3b0 2024-04-25T10:11:21.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.453+0000 7f0282ffd640 1 -- 172.21.15.82:0/1677231768 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0288054d10 con 0x7f028c06c3b0 2024-04-25T10:11:21.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 msgr2=0x7f0270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 0x7f0270069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f028c15dae0 tx=0x7f027c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:21.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 msgr2=0x7f028c150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f028800c420 tx=0x7f028800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 shutdown_connections 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0270066f60 0x7f0270069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f028c0af280 0x7f028c0ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f028c0b2c30 0x7f028c163d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 --2- 172.21.15.82:0/1677231768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f028c06c3b0 0x7f028c150480 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 >> 172.21.15.82:0/1677231768 conn(0x7f028c071810 msgr2=0x7f028c161050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 shutdown_connections 2024-04-25T10:11:21.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.457+0000 7f0294415640 1 -- 172.21.15.82:0/1677231768 wait complete. 2024-04-25T10:11:21.477 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T10:11:21.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 Processor -- start 2024-04-25T10:11:21.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 -- start start 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc0f015d180 0x7fc0f015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f006b710 con 0x7fc0f00cf840 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f01617b0 con 0x7fc0f006fc50 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f6f33640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f0161560 con 0x7fc0f015d180 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc0f00cf840 0x7fc0f00cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48438/0 (socket says 172.21.15.82:48438) 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 -- 172.21.15.82:0/2609382746 learned_addr learned my addr 172.21.15.82:0/2609382746 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/2609382746 >> v2:172.21.15.196:3300/0 conn(0x7fc0f015d180 0x7fc0f015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028468372 0 0) 0x7fc0f006b710 con 0x7fc0f00cf840 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0d8003530 con 0x7fc0f00cf840 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831498971 0 0) 0x7fc0d8003530 con 0x7fc0f00cf840 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 >> v2:172.21.15.196:3300/0 conn(0x7fc0f015d180 msgr2=0x7fc0f015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 --2- 172.21.15.82:0/2609382746 >> v2:172.21.15.196:3300/0 conn(0x7fc0f015d180 0x7fc0f015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 >> v1:172.21.15.196:6789/0 conn(0x7fc0f006fc50 legacy=0x7fc0f006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f0161a90 con 0x7fc0f00cf840 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc0e4001680 con 0x7fc0f00cf840 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/2609382746 >> v2:172.21.15.196:3300/0 conn(0x7fc0f015d180 0x7fc0f015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 >> v1:172.21.15.82:6789/0 conn(0x7fc0f00cf840 legacy=0x7fc0f00cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8003a30 0x7fc0d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d80052f0 0x7fc0d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0161560 con 0x7fc0d8004810 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d800c240 con 0x7fc0d80052f0 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0d800c440 con 0x7fc0d8003a30 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f4ccf640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8003a30 0x7fc0d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d80052f0 0x7fc0d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 -- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8003a30 msgr2=0x7fc0d8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8003a30 0x7fc0d8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 -- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d80052f0 msgr2=0x7fc0d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d80052f0 0x7fc0d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0effff640 1 -- 172.21.15.82:0/2609382746 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0d8003530 con 0x7fc0d8004810 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0d80052f0 0x7fc0d800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f54d0640 1 -- 172.21.15.82:0/2609382746 reap_dead start 2024-04-25T10:11:21.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.593+0000 7fc0f4ccf640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8003a30 0x7fc0d8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0effff640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 0x7fc0d8004c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4001ec0 tx=0x7fc0e401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552260c4b37625df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4007d40 con 0x7fc0d8004810 2024-04-25T10:11:21.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40a9070 con 0x7fc0d8004810 2024-04-25T10:11:21.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0ef7fe640 1 -- 172.21.15.82:0/2609382746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40a5dd0 con 0x7fc0d8004810 2024-04-25T10:11:21.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 msgr2=0x7fc0d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 0x7fc0d8004c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4001ec0 tx=0x7fc0e401cc30 comp rx=0 tx=0).stop 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- 172.21.15.82:0/2609382746 shutdown_connections 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/2609382746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0d8004810 0x7fc0d8004c90 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- 172.21.15.82:0/2609382746 >> 172.21.15.82:0/2609382746 conn(0x7fc0f0099a70 msgr2=0x7fc0f00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- 172.21.15.82:0/2609382746 shutdown_connections 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- 172.21.15.82:0/2609382746 wait complete. 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 Processor -- start 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.597+0000 7fc0f6f33640 1 -- start start 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8004810 0x7fc0f00c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 0x7fc0f01ebce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f00701b0 con 0x7fc0f015d590 2024-04-25T10:11:21.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0070030 con 0x7fc0f01eb860 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0162290 con 0x7fc0d8004810 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33876/0 (socket says 172.21.15.82:33876) 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 -- 172.21.15.82:0/4268893884 learned_addr learned my addr 172.21.15.82:0/4268893884 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 0x7fc0f01ebce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f4ccf640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8004810 0x7fc0f00c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 -- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8004810 msgr2=0x7fc0f00c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8004810 0x7fc0f00c6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 -- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 msgr2=0x7fc0f01ebce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 0x7fc0f01ebce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 -- 172.21.15.82:0/4268893884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f01f6d60 con 0x7fc0f015d590 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f54d0640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 0x7fc0f01ebce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0effff640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc0e401c6a0 tx=0x7fc0d8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40a59f0 con 0x7fc0f015d590 2024-04-25T10:11:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e40a5b90 con 0x7fc0f015d590 2024-04-25T10:11:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f01ec710 con 0x7fc0f015d590 2024-04-25T10:11:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e40047f0 con 0x7fc0f015d590 2024-04-25T10:11:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f01eccd0 con 0x7fc0f015d590 2024-04-25T10:11:21.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f004efa0 con 0x7fc0f015d590 2024-04-25T10:11:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0e4004990 con 0x7fc0f015d590 2024-04-25T10:11:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 0x7fc0dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc0e411e760 con 0x7fc0f015d590 2024-04-25T10:11:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f4ccf640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 0x7fc0dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.601+0000 7fc0f4ccf640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 0x7fc0dc069450 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007fd0 tx=0x7fc0e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.605+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0e4016970 con 0x7fc0f015d590 2024-04-25T10:11:21.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:21.809+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fc0f00c4010 con 0x7fc0f015d590 2024-04-25T10:11:22.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.109+0000 7fc0ed7fa640 1 -- 172.21.15.82:0/4268893884 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v429) v1 ==== 167+0+0 (secure 0 0 0) 0x7fc0e40b7320 con 0x7fc0f015d590 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 msgr2=0x7fc0dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 0x7fc0dc069450 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc0e0007fd0 tx=0x7fc0e0009040 comp rx=0 tx=0).stop 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 msgr2=0x7fc0f015da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fc0e401c6a0 tx=0x7fc0d8005840 comp rx=0 tx=0).stop 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 shutdown_connections 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc0dc066f90 0x7fc0dc069450 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc0f01eb860 0x7fc0f01ebce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0f015d590 0x7fc0f015da10 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 --2- 172.21.15.82:0/4268893884 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0d8004810 0x7fc0f00c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 >> 172.21.15.82:0/4268893884 conn(0x7fc0f0099a70 msgr2=0x7fc0f00c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 shutdown_connections 2024-04-25T10:11:22.118 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.113+0000 7fc0f6f33640 1 -- 172.21.15.82:0/4268893884 wait complete. 2024-04-25T10:11:22.136 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T10:11:22.136 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:11:22.136 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:22.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.253+0000 7fd3d3a8e640 1 Processor -- start 2024-04-25T10:11:22.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 -- start start 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3cc15a280 0x7fd3cc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 0x7fd3cc071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc15ae30 con 0x7fd3cc159ea0 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b2990 con 0x7fd3cc15a280 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d3a8e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc0b3530 con 0x7fd3cc0718c0 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3cc15a280 0x7fd3cc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd3cc15a280 0x7fd3cc15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33892/0 (socket says 172.21.15.82:33892) 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d182a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fd3cc159ea0 0x7fd3cc0b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48442/0 (socket says 172.21.15.82:48442) 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 -- 172.21.15.82:0/801209345 learned_addr learned my addr 172.21.15.82:0/801209345 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d202b640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 0x7fd3cc071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 -- 172.21.15.82:0/801209345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 msgr2=0x7fd3cc071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 0x7fd3cc071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 -- 172.21.15.82:0/801209345 >> v1:172.21.15.82:6789/0 conn(0x7fd3cc159ea0 legacy=0x7fd3cc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 -- 172.21.15.82:0/801209345 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc15b110 con 0x7fd3cc15a280 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d0828640 1 -- 172.21.15.82:0/801209345 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326343151 0 0) 0x7fd3cc15ae30 con 0x7fd3cc159ea0 2024-04-25T10:11:22.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d202b640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 0x7fd3cc071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d1029640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15a280 0x7fd3cc15a700 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009870 tx=0x7fd3bc01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8ca4b304f294ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d0828640 1 -- 172.21.15.82:0/801209345 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0a7070 con 0x7fd3cc15a280 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d0828640 1 -- 172.21.15.82:0/801209345 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0b0070 con 0x7fd3cc15a280 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.257+0000 7fd3d0828640 1 -- 172.21.15.82:0/801209345 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0ab890 con 0x7fd3cc15a280 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/801209345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15a280 msgr2=0x7fd3cc15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15a280 0x7fd3cc15a700 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009870 tx=0x7fd3bc01c560 comp rx=0 tx=0).stop 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/801209345 shutdown_connections 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc0718c0 0x7fd3cc071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/801209345 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc15a280 0x7fd3cc15a700 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/801209345 >> 172.21.15.82:0/801209345 conn(0x7fd3cc0b11a0 msgr2=0x7fd3cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/801209345 shutdown_connections 2024-04-25T10:11:22.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/801209345 wait complete. 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 Processor -- start 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- start start 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc159ea0 0x7fd3cc148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc15a280 0x7fd3cc0b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc166d40 con 0x7fd3cc0718c0 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc166bc0 con 0x7fd3cc15a280 2024-04-25T10:11:22.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc166ec0 con 0x7fd3cc159ea0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33902/0 (socket says 172.21.15.82:33902) 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 -- 172.21.15.82:0/3690114167 learned_addr learned my addr 172.21.15.82:0/3690114167 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 -- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc159ea0 msgr2=0x7fd3cc148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc159ea0 0x7fd3cc148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 -- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc15a280 msgr2=0x7fd3cc0b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d202b640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc15a280 0x7fd3cc0b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc15a280 0x7fd3cc0b0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 -- 172.21.15.82:0/3690114167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc157520 con 0x7fd3cc0718c0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d182a640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80011f0 tx=0x7fd3c800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8018070 con 0x7fd3cc0718c0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c80074e0 con 0x7fd3cc0718c0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc1577c0 con 0x7fd3cc0718c0 2024-04-25T10:11:22.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c8004a10 con 0x7fd3cc0718c0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.261+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc157d80 con 0x7fd3cc0718c0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc04efa0 con 0x7fd3cc0718c0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3c801b070 con 0x7fd3cc0718c0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3c27fc640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 0x7fd3b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd3c80842b0 con 0x7fd3cc0718c0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3d1029640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 0x7fd3b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.265+0000 7fd3d1029640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 0x7fd3b0069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0af000 tx=0x7fd3bc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.269+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd3c8055380 con 0x7fd3cc0718c0 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.417+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3cc0cee10 con 0x7fd3cc0718c0 2024-04-25T10:11:22.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.417+0000 7fd3c27fc640 1 -- 172.21.15.82:0/3690114167 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+13562 (secure 0 0 0) 0x7fd3c805b080 con 0x7fd3cc0718c0 2024-04-25T10:11:22.421 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:22.421 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":98,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:21.102887+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/570372586":"2024-04-26T10:11:18.088690+0000","172.21.15.82:6836/570372586":"2024-04-26T10:11:18.088690+0000","172.21.15.82:6837/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6836/362560393":"2024-04-26T10:10:33.173770+0000","172.21.15.82:6837/2865291678":"2024-04-26T10:09:12.306251+0000","172.21.15.82:6836/2865291678":"2024-04-26T10:09:12.306251+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 msgr2=0x7fd3b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 0x7fd3b0069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc0af000 tx=0x7fd3bc004740 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 msgr2=0x7fd3cc14fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd3c80011f0 tx=0x7fd3c800c9b0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 shutdown_connections 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd3b0066f60 0x7fd3b0069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd3cc159ea0 0x7fd3cc148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd3cc15a280 0x7fd3cc0b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 --2- 172.21.15.82:0/3690114167 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd3cc0718c0 0x7fd3cc14fcc0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 >> 172.21.15.82:0/3690114167 conn(0x7fd3cc0b11a0 msgr2=0x7fd3cc15b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 shutdown_connections 2024-04-25T10:11:22.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.421+0000 7fd3d3a8e640 1 -- 172.21.15.82:0/3690114167 wait complete. 2024-04-25T10:11:22.438 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:22.439 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:22.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 Processor -- start 2024-04-25T10:11:22.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 -- start start 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb9815da80 0x7ffb980d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb980d1cd0 0x7ffb980d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9806be60 con 0x7ffb980d0620 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb9806c440 con 0x7ffb9815da80 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.549+0000 7ffba074b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9815de60 con 0x7ffb980d1cd0 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb9815da80 0x7ffb980d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb9815da80 0x7ffb980d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33918/0 (socket says 172.21.15.82:33918) 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 -- 172.21.15.82:0/2754672675 learned_addr learned my addr 172.21.15.82:0/2754672675 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9ece8640 1 --1- 172.21.15.82:0/2754672675 >> v1:172.21.15.82:6789/0 conn(0x7ffb980d0620 0x7ffb980ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48448/0 (socket says 172.21.15.82:48448) 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 -- 172.21.15.82:0/2754672675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb980d1cd0 msgr2=0x7ffb980d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/2754672675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb980d1cd0 0x7ffb980d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 -- 172.21.15.82:0/2754672675 >> v1:172.21.15.82:6789/0 conn(0x7ffb980d0620 legacy=0x7ffb980ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 -- 172.21.15.82:0/2754672675 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb980d2450 con 0x7ffb9815da80 2024-04-25T10:11:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/2754672675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb9815da80 0x7ffb980d4230 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009870 tx=0x7ffb9401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16158b94e8a7f060 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9d4e5640 1 -- 172.21.15.82:0/2754672675 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940a7070 con 0x7ffb9815da80 2024-04-25T10:11:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9d4e5640 1 -- 172.21.15.82:0/2754672675 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb940b0070 con 0x7ffb9815da80 2024-04-25T10:11:22.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffb9d4e5640 1 -- 172.21.15.82:0/2754672675 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940ab890 con 0x7ffb9815da80 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 -- 172.21.15.82:0/2754672675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb9815da80 msgr2=0x7ffb980d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 --2- 172.21.15.82:0/2754672675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb9815da80 0x7ffb980d4230 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ffb94009870 tx=0x7ffb9401c560 comp rx=0 tx=0).stop 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 -- 172.21.15.82:0/2754672675 shutdown_connections 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 --2- 172.21.15.82:0/2754672675 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb980d1cd0 0x7ffb980d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 --2- 172.21.15.82:0/2754672675 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb9815da80 0x7ffb980d4230 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 -- 172.21.15.82:0/2754672675 >> 172.21.15.82:0/2754672675 conn(0x7ffb9808b000 msgr2=0x7ffb980d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 -- 172.21.15.82:0/2754672675 shutdown_connections 2024-04-25T10:11:22.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.553+0000 7ffba074b640 1 -- 172.21.15.82:0/2754672675 wait complete. 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 Processor -- start 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- start start 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 0x7ffb98155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 0x7ffb9815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9816a730 con 0x7ffb980d1cd0 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9816a5b0 con 0x7ffb980d0620 2024-04-25T10:11:22.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9816a8b0 con 0x7ffb9815da80 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33928/0 (socket says 172.21.15.82:33928) 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 -- 172.21.15.82:0/1864902914 learned_addr learned my addr 172.21.15.82:0/1864902914 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9ece8640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 0x7ffb9815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 0x7ffb98155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 -- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 msgr2=0x7ffb9815f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 0x7ffb9815f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 -- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 msgr2=0x7ffb98155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 0x7ffb98155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 -- 172.21.15.82:0/1864902914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb98155da0 con 0x7ffb980d1cd0 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9ece8640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 0x7ffb9815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 0x7ffb98155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:22.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb9dce6640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800b4d0 tx=0x7ffb8800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88014070 con 0x7ffb980d1cd0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb88005c20 con 0x7ffb980d1cd0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb981429a0 con 0x7ffb980d1cd0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88010670 con 0x7ffb980d1cd0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb98142f60 con 0x7ffb980d1cd0 2024-04-25T10:11:22.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9804efa0 con 0x7ffb980d1cd0 2024-04-25T10:11:22.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb88010810 con 0x7ffb980d1cd0 2024-04-25T10:11:22.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.557+0000 7ffb8effd640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 0x7ffb7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.561+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ffb88084c60 con 0x7ffb980d1cd0 2024-04-25T10:11:22.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.561+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 0x7ffb7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.561+0000 7ffb9e4e7640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 0x7ffb7c069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffb940af000 tx=0x7ffb94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.565+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb98155da0 con 0x7ffb980d1cd0 2024-04-25T10:11:22.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb980b0db0 con 0x7ffb980d1cd0 2024-04-25T10:11:22.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffb8effd640 1 -- 172.21.15.82:0/1864902914 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb88055d30 con 0x7ffb980d1cd0 2024-04-25T10:11:22.724 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 msgr2=0x7ffb7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 0x7ffb7c069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ffb940af000 tx=0x7ffb94008040 comp rx=0 tx=0).stop 2024-04-25T10:11:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 msgr2=0x7ffb980b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800b4d0 tx=0x7ffb8800b990 comp rx=0 tx=0).stop 2024-04-25T10:11:22.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 shutdown_connections 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb7c066f60 0x7ffb7c069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb9815da80 0x7ffb9815f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb980d1cd0 0x7ffb980b68d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 --2- 172.21.15.82:0/1864902914 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb980d0620 0x7ffb98155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 >> 172.21.15.82:0/1864902914 conn(0x7ffb9808b000 msgr2=0x7ffb980b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 shutdown_connections 2024-04-25T10:11:22.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.721+0000 7ffba074b640 1 -- 172.21.15.82:0/1864902914 wait complete. 2024-04-25T10:11:22.742 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:11:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.845+0000 7f31a0fd6640 1 Processor -- start 2024-04-25T10:11:22.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.845+0000 7f31a0fd6640 1 -- start start 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f31a0fd6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f319c0d07f0 0x7f319c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f31a0fd6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f319c0d1cd0 0x7f319c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f31a0fd6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319c06be60 con 0x7f319c15da80 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f31a0fd6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c06c440 con 0x7f319c0d1cd0 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f31a0fd6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319c15de60 con 0x7f319c0d07f0 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f319c0d1cd0 0x7f319c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319a59c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f319c15da80 0x7f319c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48456/0 (socket says 172.21.15.82:48456) 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319a59c640 1 -- 172.21.15.82:0/718866771 learned_addr learned my addr 172.21.15.82:0/718866771 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f319c0d1cd0 0x7f319c06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33938/0 (socket says 172.21.15.82:33938) 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f3199d9b640 1 --2- 172.21.15.82:0/718866771 >> v2:172.21.15.196:3300/0 conn(0x7f319c0d07f0 0x7f319c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 -- 172.21.15.82:0/718866771 >> v2:172.21.15.196:3300/0 conn(0x7f319c0d07f0 msgr2=0x7f319c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 --2- 172.21.15.82:0/718866771 >> v2:172.21.15.196:3300/0 conn(0x7f319c0d07f0 0x7f319c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 -- 172.21.15.82:0/718866771 >> v1:172.21.15.82:6789/0 conn(0x7f319c15da80 legacy=0x7f319c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319959a640 1 -- 172.21.15.82:0/718866771 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2934396952 0 0) 0x7f319c06be60 con 0x7f319c15da80 2024-04-25T10:11:22.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.849+0000 7f319ad9d640 1 -- 172.21.15.82:0/718866771 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0d2450 con 0x7f319c0d1cd0 2024-04-25T10:11:22.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319ad9d640 1 --2- 172.21.15.82:0/718866771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d1cd0 0x7f319c06c8d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f319c08d880 tx=0x7f319401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1855ed1f3a785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319959a640 1 -- 172.21.15.82:0/718866771 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940b2070 con 0x7f319c0d1cd0 2024-04-25T10:11:22.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319959a640 1 -- 172.21.15.82:0/718866771 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31940a59e0 con 0x7f319c0d1cd0 2024-04-25T10:11:22.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319959a640 1 -- 172.21.15.82:0/718866771 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31940a5db0 con 0x7f319c0d1cd0 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- 172.21.15.82:0/718866771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d1cd0 msgr2=0x7f319c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/718866771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d1cd0 0x7f319c06c8d0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f319c08d880 tx=0x7f319401c900 comp rx=0 tx=0).stop 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- 172.21.15.82:0/718866771 shutdown_connections 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/718866771 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d1cd0 0x7f319c06c8d0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/718866771 >> v2:172.21.15.196:3300/0 conn(0x7f319c0d07f0 0x7f319c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- 172.21.15.82:0/718866771 >> 172.21.15.82:0/718866771 conn(0x7f319c08b000 msgr2=0x7f319c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- 172.21.15.82:0/718866771 shutdown_connections 2024-04-25T10:11:22.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- 172.21.15.82:0/718866771 wait complete. 2024-04-25T10:11:22.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 Processor -- start 2024-04-25T10:11:22.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- start start 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f319c0d1cd0 0x7f319c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c15da80 0x7f319c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c06d7a0 con 0x7f319c0d07f0 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319c06d620 con 0x7f319c15da80 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f31a0fd6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319c06d920 con 0x7f319c0d1cd0 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33940/0 (socket says 172.21.15.82:33940) 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 -- 172.21.15.82:0/1155955358 learned_addr learned my addr 172.21.15.82:0/1155955358 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319ad9d640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c15da80 0x7f319c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f3199d9b640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f319c0d1cd0 0x7f319c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 -- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f319c0d1cd0 msgr2=0x7f319c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f319c0d1cd0 0x7f319c0cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 -- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c15da80 msgr2=0x7f319c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c15da80 0x7f319c0cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 -- 172.21.15.82:0/1155955358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319c0c9170 con 0x7f319c0d07f0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.853+0000 7f319a59c640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c480 tx=0x7f318c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c018070 con 0x7f319c0d07f0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c007500 con 0x7f319c0d07f0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c004a30 con 0x7f319c0d07f0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c0c9450 con 0x7f319c0d07f0 2024-04-25T10:11:22.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f319c0c9a10 con 0x7f319c0d07f0 2024-04-25T10:11:22.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319c04efa0 con 0x7f319c0d07f0 2024-04-25T10:11:22.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f318c01b070 con 0x7f319c0d07f0 2024-04-25T10:11:22.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 0x7f317c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f318c0842a0 con 0x7f319c0d07f0 2024-04-25T10:11:22.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f3199d9b640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 0x7f317c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.857+0000 7f3199d9b640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 0x7f317c069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f319c0b0a90 tx=0x7f3190008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:22.861+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f318c055400 con 0x7f319c0d07f0 2024-04-25T10:11:23.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.061+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f319c0cdcb0 con 0x7f319c0d07f0 2024-04-25T10:11:23.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.113+0000 7f318affd640 1 -- 172.21.15.82:0/1155955358 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v430) v1 ==== 149+0+0 (secure 0 0 0) 0x7f318c0840c0 con 0x7f319c0d07f0 2024-04-25T10:11:23.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 msgr2=0x7f317c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 0x7f317c069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f319c0b0a90 tx=0x7f3190008040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 msgr2=0x7f319c14ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c480 tx=0x7f318c00c940 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 shutdown_connections 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f317c066f60 0x7f317c069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f319c0d1cd0 0x7f319c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f319c15da80 0x7f319c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 --2- 172.21.15.82:0/1155955358 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f319c0d07f0 0x7f319c14ab60 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 >> 172.21.15.82:0/1155955358 conn(0x7f319c08b000 msgr2=0x7f319c0b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 shutdown_connections 2024-04-25T10:11:23.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.117+0000 7f31a0fd6640 1 -- 172.21.15.82:0/1155955358 wait complete. 2024-04-25T10:11:23.135 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:11:23.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.253+0000 7f025bda3640 1 Processor -- start 2024-04-25T10:11:23.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.253+0000 7f025bda3640 1 -- start start 2024-04-25T10:11:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025bda3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f025415b6b0 0x7f02540ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025bda3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025bda3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02540cbd20 con 0x7f02540cf590 2024-04-25T10:11:23.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025bda3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025415ba90 con 0x7f025415b6b0 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025bda3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540cc000 con 0x7f0254073010 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f0259b3f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f025415b6b0 0x7f02540ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:33314/0 (socket says 172.21.15.82:33314) 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 -- 172.21.15.82:0/1483945721 learned_addr learned my addr 172.21.15.82:0/1483945721 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:23.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 -- 172.21.15.82:0/1483945721 >> v2:172.21.15.196:3300/0 conn(0x7f025415b6b0 msgr2=0x7f02540ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 --2- 172.21.15.82:0/1483945721 >> v2:172.21.15.196:3300/0 conn(0x7f025415b6b0 0x7f02540ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 -- 172.21.15.82:0/1483945721 >> v1:172.21.15.196:6789/0 conn(0x7f02540cf590 legacy=0x7f025415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.257+0000 7f025a340640 1 -- 172.21.15.82:0/1483945721 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540cc2e0 con 0x7f0254073010 2024-04-25T10:11:23.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025a340640 1 --2- 172.21.15.82:0/1483945721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f024c009e50 tx=0x7f024c01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7b80628265af8ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f0258b3d640 1 -- 172.21.15.82:0/1483945721 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0a5700 con 0x7f0254073010 2024-04-25T10:11:23.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f0258b3d640 1 -- 172.21.15.82:0/1483945721 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c0a58a0 con 0x7f0254073010 2024-04-25T10:11:23.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f0258b3d640 1 -- 172.21.15.82:0/1483945721 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c0a5c70 con 0x7f0254073010 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- 172.21.15.82:0/1483945721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 msgr2=0x7f02540cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 --2- 172.21.15.82:0/1483945721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f024c009e50 tx=0x7f024c01ca30 comp rx=0 tx=0).stop 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- 172.21.15.82:0/1483945721 shutdown_connections 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 --2- 172.21.15.82:0/1483945721 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0254073010 0x7f02540cfa20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 --2- 172.21.15.82:0/1483945721 >> v2:172.21.15.196:3300/0 conn(0x7f025415b6b0 0x7f02540ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- 172.21.15.82:0/1483945721 >> 172.21.15.82:0/1483945721 conn(0x7f025406ae60 msgr2=0x7f025406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- 172.21.15.82:0/1483945721 shutdown_connections 2024-04-25T10:11:23.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- 172.21.15.82:0/1483945721 wait complete. 2024-04-25T10:11:23.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 Processor -- start 2024-04-25T10:11:23.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.261+0000 7f025bda3640 1 -- start start 2024-04-25T10:11:23.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0254073010 0x7f02540c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f025415b6b0 0x7f025415d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540cadb0 con 0x7f02540cf590 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540ca900 con 0x7f0254073010 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540caf30 con 0x7f025415b6b0 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33944/0 (socket says 172.21.15.82:33944) 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 -- 172.21.15.82:0/2415780044 learned_addr learned my addr 172.21.15.82:0/2415780044 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f0259b3f640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0254073010 0x7f02540c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025a340640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f025415b6b0 0x7f025415d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 -- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f025415b6b0 msgr2=0x7f025415d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f025415b6b0 0x7f025415d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 -- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0254073010 msgr2=0x7f02540c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0254073010 0x7f02540c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 -- 172.21.15.82:0/2415780044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025415eae0 con 0x7f02540cf590 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025933e640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f024800c420 tx=0x7f024800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480052d0 con 0x7f02540cf590 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248005470 con 0x7f02540cf590 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0248007260 con 0x7f02540cf590 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254153340 con 0x7f02540cf590 2024-04-25T10:11:23.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254152580 con 0x7f02540cf590 2024-04-25T10:11:23.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025404efa0 con 0x7f02540cf590 2024-04-25T10:11:23.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0248007400 con 0x7f02540cf590 2024-04-25T10:11:23.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 0x7f0238069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f02480843e0 con 0x7f02540cf590 2024-04-25T10:11:23.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f0259b3f640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 0x7f0238069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.265+0000 7f0259b3f640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 0x7f0238069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f025415d050 tx=0x7f0250008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.269+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0248055540 con 0x7f02540cf590 2024-04-25T10:11:23.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.429+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0254073d90 con 0x7f02540cf590 2024-04-25T10:11:23.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f02467fc640 1 -- 172.21.15.82:0/2415780044 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0248054e40 con 0x7f02540cf590 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 msgr2=0x7f0238069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 0x7f0238069420 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f025415d050 tx=0x7f0250008040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 msgr2=0x7f0254098900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f024800c420 tx=0x7f024800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 shutdown_connections 2024-04-25T10:11:23.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0238066f60 0x7f0238069420 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f025415b6b0 0x7f025415d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f02540cf590 0x7f0254098900 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 --2- 172.21.15.82:0/2415780044 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0254073010 0x7f02540c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 >> 172.21.15.82:0/2415780044 conn(0x7f025406ae60 msgr2=0x7f02540b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 shutdown_connections 2024-04-25T10:11:23.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.437+0000 7f025bda3640 1 -- 172.21.15.82:0/2415780044 wait complete. 2024-04-25T10:11:23.457 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:23.458 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:23.493 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:23.543 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:23.544 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:23.544 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:23.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:23.549 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:11:23.550 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:23.555 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:23.556 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:23.556 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:23.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:23.595 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:11:23.595 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:23.606 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:23.606 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:11:23.606 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:23.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.761+0000 7fc870c6a640 1 Processor -- start 2024-04-25T10:11:23.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.761+0000 7fc870c6a640 1 -- start start 2024-04-25T10:11:23.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 0x7fc86c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc86c144ca0 0x7fc86c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c0cc2b0 con 0x7fc86c144ca0 2024-04-25T10:11:23.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c06e290 con 0x7fc86c0ce510 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0cee30 con 0x7fc86c06ec60 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc86c144ca0 0x7fc86c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc86c144ca0 0x7fc86c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33960/0 (socket says 172.21.15.82:33960) 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 -- 172.21.15.82:0/3385696786 learned_addr learned my addr 172.21.15.82:0/3385696786 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86a59c640 1 --2- 172.21.15.82:0/3385696786 >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 0x7fc86c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86ad9d640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 -- 172.21.15.82:0/3385696786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 msgr2=0x7fc86c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 -- 172.21.15.82:0/3385696786 >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 msgr2=0x7fc86c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 --2- 172.21.15.82:0/3385696786 >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 0x7fc86c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 -- 172.21.15.82:0/3385696786 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0ccc70 con 0x7fc86c144ca0 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86a59c640 1 --2- 172.21.15.82:0/3385696786 >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 0x7fc86c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86ad9d640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc869d9b640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c144ca0 0x7fc86c06e720 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc86c144c50 tx=0x7fc85c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6439d2ab0f86157b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86959a640 1 -- 172.21.15.82:0/3385696786 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b5070 con 0x7fc86c144ca0 2024-04-25T10:11:23.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86959a640 1 -- 172.21.15.82:0/3385696786 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0a89d0 con 0x7fc86c144ca0 2024-04-25T10:11:23.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc86959a640 1 -- 172.21.15.82:0/3385696786 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0a8da0 con 0x7fc86c144ca0 2024-04-25T10:11:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- 172.21.15.82:0/3385696786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c144ca0 msgr2=0x7fc86c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c144ca0 0x7fc86c06e720 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7fc86c144c50 tx=0x7fc85c01ed10 comp rx=0 tx=0).stop 2024-04-25T10:11:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- 172.21.15.82:0/3385696786 shutdown_connections 2024-04-25T10:11:23.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- 172.21.15.82:0/3385696786 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c144ca0 0x7fc86c06e720 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 --2- 172.21.15.82:0/3385696786 >> v2:172.21.15.196:3300/0 conn(0x7fc86c0ce510 0x7fc86c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- 172.21.15.82:0/3385696786 >> 172.21.15.82:0/3385696786 conn(0x7fc86c0b11a0 msgr2=0x7fc86c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- 172.21.15.82:0/3385696786 shutdown_connections 2024-04-25T10:11:23.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.765+0000 7fc870c6a640 1 -- 172.21.15.82:0/3385696786 wait complete. 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 Processor -- start 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- start start 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c1e5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 0x7fc86c130de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0cc900 con 0x7fc86c0ce510 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c06df20 con 0x7fc86c144ca0 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1451f0 con 0x7fc86c06ec60 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33962/0 (socket says 172.21.15.82:33962) 2024-04-25T10:11:23.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 -- 172.21.15.82:0/1611758009 learned_addr learned my addr 172.21.15.82:0/1611758009 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc86a59c640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c1e5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc86ad9d640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 0x7fc86c130de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 -- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 msgr2=0x7fc86c1e5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c1e5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 -- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 msgr2=0x7fc86c130de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 0x7fc86c130de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 -- 172.21.15.82:0/1611758009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c12edf0 con 0x7fc86c0ce510 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc86ad9d640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 0x7fc86c130de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc869d9b640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fc85c01d3c0 tx=0x7fc85c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b5070 con 0x7fc86c0ce510 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0b0c20 con 0x7fc86c0ce510 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c12f060 con 0x7fc86c0ce510 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0bb410 con 0x7fc86c0ce510 2024-04-25T10:11:23.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c1e2fd0 con 0x7fc86c0ce510 2024-04-25T10:11:23.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.769+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c04efa0 con 0x7fc86c0ce510 2024-04-25T10:11:23.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.773+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc85c0c2060 con 0x7fc86c0ce510 2024-04-25T10:11:23.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.773+0000 7fc85affd640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 0x7fc84c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.773+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc85c122250 con 0x7fc86c0ce510 2024-04-25T10:11:23.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.773+0000 7fc86a59c640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 0x7fc84c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.773+0000 7fc86a59c640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 0x7fc84c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc86000a9a0 tx=0x7fc860008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.777+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc85c018870 con 0x7fc86c0ce510 2024-04-25T10:11:23.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.973+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc86c0b1cc0 con 0x7fc86c0ce510 2024-04-25T10:11:23.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.973+0000 7fc85affd640 1 -- 172.21.15.82:0/1611758009 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3705 (secure 0 0 0) 0x7fc85c0f31f0 con 0x7fc86c0ce510 2024-04-25T10:11:23.978 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:23.978 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5474,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/2813254777","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6833","nonce":2813254777},{"type":"v1","addr":"172.21.15.196:6835","nonce":2813254777}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":6142,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6834/1851173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":1851173555},{"type":"v1","addr":"172.21.15.196:6834","nonce":1851173555}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":7791,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/1693758073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":1693758073},{"type":"v1","addr":"172.21.15.82:6835","nonce":1693758073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":53},{"gid":8682,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1589838880","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1589838880},{"type":"v1","addr":"172.21.15.82:6837","nonce":1589838880}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[]} 2024-04-25T10:11:23.978 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 71 2024-04-25T10:11:23.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 msgr2=0x7fc84c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 0x7fc84c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc86000a9a0 tx=0x7fc860008040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 msgr2=0x7fc86c1e5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fc85c01d3c0 tx=0x7fc85c01bab0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 shutdown_connections 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc84c066f60 0x7fc84c069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc86c144ca0 0x7fc86c130de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc86c0ce510 0x7fc86c1e5eb0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 --2- 172.21.15.82:0/1611758009 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc86c06ec60 0x7fc86c1e5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 >> 172.21.15.82:0/1611758009 conn(0x7fc86c0b11a0 msgr2=0x7fc86c0c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 shutdown_connections 2024-04-25T10:11:23.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:23.977+0000 7fc870c6a640 1 -- 172.21.15.82:0/1611758009 wait complete. 2024-04-25T10:11:23.996 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:11:23.996 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:11:23.997 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:23.997 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:11:23.997 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:11:23.997 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:11:23.997 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:23.997 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:11:23.998 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:11:23.998 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:23.998 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:11:23.998 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:11:23.998 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:11:23.998 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:24.007 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:11:24.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:11:24.008 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:11:24.008 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:11:24.014 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:11:24.014 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:11:24.015 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:11:24.015 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:11:24.051 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:11:24.052 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:24.052 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:11:24.053 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:24.059 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:11:24.059 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:24.060 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:11:24.060 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:24.060 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:11:24.112 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:11:24.112 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:11:24.130 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:11:24.131 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:11:24.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 Processor -- start 2024-04-25T10:11:24.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 -- start start 2024-04-25T10:11:24.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c15a280 0x7fa26c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa26c0718c0 0x7fa26c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa26c15ae30 con 0x7fa26c159ea0 2024-04-25T10:11:24.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa26c0b2990 con 0x7fa26c0718c0 2024-04-25T10:11:24.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa273948640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c0b3530 con 0x7fa26c15a280 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa2716e4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa26c159ea0 0x7fa26c0b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:48478/0 (socket says 172.21.15.82:48478) 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa2716e4640 1 -- 172.21.15.82:0/190308701 learned_addr learned my addr 172.21.15.82:0/190308701 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa270ee3640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c15a280 0x7fa26c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa271ee5640 1 --2- 172.21.15.82:0/190308701 >> v2:172.21.15.196:3300/0 conn(0x7fa26c0718c0 0x7fa26c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa263fff640 1 -- 172.21.15.82:0/190308701 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81568922 0 0) 0x7fa26c15ae30 con 0x7fa26c159ea0 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa263fff640 1 -- 172.21.15.82:0/190308701 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa258003530 con 0x7fa26c159ea0 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa271ee5640 1 -- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c15a280 msgr2=0x7fa26c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa271ee5640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c15a280 0x7fa26c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa271ee5640 1 -- 172.21.15.82:0/190308701 >> v1:172.21.15.82:6789/0 conn(0x7fa26c159ea0 legacy=0x7fa26c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.181+0000 7fa271ee5640 1 -- 172.21.15.82:0/190308701 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c15b110 con 0x7fa26c0718c0 2024-04-25T10:11:24.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa271ee5640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c0718c0 0x7fa26c071d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa264009d90 tx=0x7fa26401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaeee52da8584ac5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa263fff640 1 -- 172.21.15.82:0/190308701 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640b2070 con 0x7fa26c0718c0 2024-04-25T10:11:24.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa263fff640 1 -- 172.21.15.82:0/190308701 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2640a58c0 con 0x7fa26c0718c0 2024-04-25T10:11:24.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa263fff640 1 -- 172.21.15.82:0/190308701 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2640a5c90 con 0x7fa26c0718c0 2024-04-25T10:11:24.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c0718c0 msgr2=0x7fa26c071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c0718c0 0x7fa26c071d50 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa264009d90 tx=0x7fa26401c970 comp rx=0 tx=0).stop 2024-04-25T10:11:24.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/190308701 shutdown_connections 2024-04-25T10:11:24.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c15a280 0x7fa26c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- 172.21.15.82:0/190308701 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c0718c0 0x7fa26c071d50 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/190308701 >> 172.21.15.82:0/190308701 conn(0x7fa26c0b11a0 msgr2=0x7fa26c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/190308701 shutdown_connections 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/190308701 wait complete. 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 Processor -- start 2024-04-25T10:11:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- start start 2024-04-25T10:11:24.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c0718c0 0x7fa26c14ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 0x7fa26c0b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c06d420 con 0x7fa26c159ea0 2024-04-25T10:11:24.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c06d2a0 con 0x7fa26c15a280 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa26c06d5a0 con 0x7fa26c0718c0 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:34000/0 (socket says 172.21.15.82:34000) 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 -- 172.21.15.82:0/3393062648 learned_addr learned my addr 172.21.15.82:0/3393062648 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa2716e4640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c0718c0 0x7fa26c14ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa271ee5640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 0x7fa26c0b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 -- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c0718c0 msgr2=0x7fa26c14ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c0718c0 0x7fa26c14ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 -- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 msgr2=0x7fa26c0b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 0x7fa26c0b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 -- 172.21.15.82:0/3393062648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c1442c0 con 0x7fa26c159ea0 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa271ee5640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 0x7fa26c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa270ee3640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00e870 tx=0x7fa25c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c017070 con 0x7fa26c159ea0 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25c00c940 con 0x7fa26c159ea0 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c00ccf0 con 0x7fa26c159ea0 2024-04-25T10:11:24.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa26c14b460 con 0x7fa26c159ea0 2024-04-25T10:11:24.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.185+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c148610 con 0x7fa26c159ea0 2024-04-25T10:11:24.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa26c04efa0 con 0x7fa26c159ea0 2024-04-25T10:11:24.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa25c01a070 con 0x7fa26c159ea0 2024-04-25T10:11:24.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa261ffb640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 0x7fa250069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa25c084d50 con 0x7fa26c159ea0 2024-04-25T10:11:24.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa2716e4640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 0x7fa250069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.189+0000 7fa2716e4640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 0x7fa250069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.193+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa25c088030 con 0x7fa26c159ea0 2024-04-25T10:11:24.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:24.365+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa26c1554c0 con 0x7fa26c159ea0 2024-04-25T10:11:25.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.465+0000 7fa261ffb640 1 -- 172.21.15.82:0/3393062648 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa26c1442c0 con 0x7fa26c159ea0 2024-04-25T10:11:25.468 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:11:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 msgr2=0x7fa250069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 0x7fa250069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa268007fd0 tx=0x7fa268009040 comp rx=0 tx=0).stop 2024-04-25T10:11:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 msgr2=0x7fa26c1490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00e870 tx=0x7fa25c00ec70 comp rx=0 tx=0).stop 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 shutdown_connections 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa250066f60 0x7fa250069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa26c15a280 0x7fa26c0b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa26c159ea0 0x7fa26c1490f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 --2- 172.21.15.82:0/3393062648 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa26c0718c0 0x7fa26c14ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 >> 172.21.15.82:0/3393062648 conn(0x7fa26c0b11a0 msgr2=0x7fa26c0b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 shutdown_connections 2024-04-25T10:11:25.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.469+0000 7fa273948640 1 -- 172.21.15.82:0/3393062648 wait complete. 2024-04-25T10:11:25.491 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:11:25.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 Processor -- start 2024-04-25T10:11:25.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 -- start start 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f45400d12e0 0x7f45400d1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45400cdb10 con 0x7f45400b7010 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454015cb90 con 0x7f45400d12e0 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4545522640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454015f970 con 0x7f454015b7a0 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f4544d21640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f45400d12e0 0x7f45400d1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:59364/0 (socket says 172.21.15.82:59364) 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 -- 172.21.15.82:0/301446596 learned_addr learned my addr 172.21.15.82:0/301446596 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 -- 172.21.15.82:0/301446596 >> v2:172.21.15.196:3300/0 conn(0x7f45400d12e0 msgr2=0x7f45400d1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 --2- 172.21.15.82:0/301446596 >> v2:172.21.15.196:3300/0 conn(0x7f45400d12e0 0x7f45400d1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 -- 172.21.15.82:0/301446596 >> v1:172.21.15.196:6789/0 conn(0x7f45400b7010 legacy=0x7f45400cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.609+0000 7f453ffff640 1 -- 172.21.15.82:0/301446596 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454015fc50 con 0x7f454015b7a0 2024-04-25T10:11:25.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453ffff640 1 --2- 172.21.15.82:0/301446596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4530009870 tx=0x7f453001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8413aec0409a70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453effd640 1 -- 172.21.15.82:0/301446596 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300a7070 con 0x7f454015b7a0 2024-04-25T10:11:25.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453effd640 1 -- 172.21.15.82:0/301446596 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45300b0070 con 0x7f454015b7a0 2024-04-25T10:11:25.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453effd640 1 -- 172.21.15.82:0/301446596 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45300ab890 con 0x7f454015b7a0 2024-04-25T10:11:25.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- 172.21.15.82:0/301446596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 msgr2=0x7f454015f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- 172.21.15.82:0/301446596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4530009870 tx=0x7f453001c560 comp rx=0 tx=0).stop 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- 172.21.15.82:0/301446596 shutdown_connections 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- 172.21.15.82:0/301446596 >> v2:172.21.15.196:3300/0 conn(0x7f45400d12e0 0x7f45400d1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- 172.21.15.82:0/301446596 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f454015b7a0 0x7f454015f270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- 172.21.15.82:0/301446596 >> 172.21.15.82:0/301446596 conn(0x7f4540070a20 msgr2=0x7f4540070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- 172.21.15.82:0/301446596 shutdown_connections 2024-04-25T10:11:25.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- 172.21.15.82:0/301446596 wait complete. 2024-04-25T10:11:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 Processor -- start 2024-04-25T10:11:25.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- start start 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45400b7010 0x7f4540150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f454015b7a0 0x7f4540155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540168790 con 0x7f45400d12e0 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4540168610 con 0x7f454015b7a0 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f4545522640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4540168910 con 0x7f45400b7010 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45568/0 (socket says 172.21.15.82:45568) 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.613+0000 7f453f7fe640 1 -- 172.21.15.82:0/1386104109 learned_addr learned my addr 172.21.15.82:0/1386104109 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453ffff640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45400b7010 0x7f4540150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f4544d21640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f454015b7a0 0x7f4540155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 -- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45400b7010 msgr2=0x7f4540150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45400b7010 0x7f4540150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 -- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f454015b7a0 msgr2=0x7f4540155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f454015b7a0 0x7f4540155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 -- 172.21.15.82:0/1386104109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540151f90 con 0x7f45400d12e0 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453f7fe640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f453400c3a0 tx=0x7f453400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45340052d0 con 0x7f45400d12e0 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534005470 con 0x7f45400d12e0 2024-04-25T10:11:25.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4540152270 con 0x7f45400d12e0 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534007260 con 0x7f45400d12e0 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4540152830 con 0x7f45400d12e0 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454004ef30 con 0x7f45400d12e0 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4534007400 con 0x7f45400d12e0 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4534084270 con 0x7f45400d12e0 2024-04-25T10:11:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453ffff640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.617+0000 7f453ffff640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 0x7f452c069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f45300af000 tx=0x7f45300a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.621+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45340554d0 con 0x7f45400d12e0 2024-04-25T10:11:25.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.833+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4540071810 con 0x7f45400d12e0 2024-04-25T10:11:25.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f453cff9640 1 -- 172.21.15.82:0/1386104109 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4534054dd0 con 0x7f45400d12e0 2024-04-25T10:11:25.839 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:25.839 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:11:25.839 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.840 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.841 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:11:25.842 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.843 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:11:25.844 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.845 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:11:25.846 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.847 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.848 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.849 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.850 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.851 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.852 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.853 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:11:25.854 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:11:25.855 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:11:25.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 msgr2=0x7f452c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 0x7f452c069420 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f45300af000 tx=0x7f45300a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 msgr2=0x7f4540151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f453400c3a0 tx=0x7f453400c860 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 shutdown_connections 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f452c066f60 0x7f452c069420 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f454015b7a0 0x7f4540155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f45400d12e0 0x7f4540151620 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 --2- 172.21.15.82:0/1386104109 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f45400b7010 0x7f4540150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 >> 172.21.15.82:0/1386104109 conn(0x7f4540070a20 msgr2=0x7f454015bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 shutdown_connections 2024-04-25T10:11:25.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.837+0000 7f4545522640 1 -- 172.21.15.82:0/1386104109 wait complete. 2024-04-25T10:11:25.858 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:25.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.961+0000 7f446173e640 1 Processor -- start 2024-04-25T10:11:25.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.961+0000 7f446173e640 1 -- start start 2024-04-25T10:11:25.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f446173e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f446173e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f445c0d1cd0 0x7f445c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f446173e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445c06be60 con 0x7f445c0d0620 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f446173e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445c06c440 con 0x7f445c0d1cd0 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f446173e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c15de60 con 0x7f445c15da80 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f4453fff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f445c0d1cd0 0x7f445c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:59370/0 (socket says 172.21.15.82:59370) 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 -- 172.21.15.82:0/1825300929 learned_addr learned my addr 172.21.15.82:0/1825300929 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 -- 172.21.15.82:0/1825300929 >> v2:172.21.15.196:3300/0 conn(0x7f445c0d1cd0 msgr2=0x7f445c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 --2- 172.21.15.82:0/1825300929 >> v2:172.21.15.196:3300/0 conn(0x7f445c0d1cd0 0x7f445c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 -- 172.21.15.82:0/1825300929 >> v1:172.21.15.196:6789/0 conn(0x7f445c0d0620 legacy=0x7f445c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 -- 172.21.15.82:0/1825300929 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0d2450 con 0x7f445c15da80 2024-04-25T10:11:25.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445affd640 1 --2- 172.21.15.82:0/1825300929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fe03739b314702e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445a7fc640 1 -- 172.21.15.82:0/1825300929 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0b2070 con 0x7f445c15da80 2024-04-25T10:11:25.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445a7fc640 1 -- 172.21.15.82:0/1825300929 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0ad680 con 0x7f445c15da80 2024-04-25T10:11:25.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.965+0000 7f445a7fc640 1 -- 172.21.15.82:0/1825300929 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0ada50 con 0x7f445c15da80 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/1825300929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 msgr2=0x7f445c0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- 172.21.15.82:0/1825300929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f444c009870 tx=0x7f444c01c560 comp rx=0 tx=0).stop 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/1825300929 shutdown_connections 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- 172.21.15.82:0/1825300929 >> v2:172.21.15.196:3300/0 conn(0x7f445c0d1cd0 0x7f445c0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- 172.21.15.82:0/1825300929 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c15da80 0x7f445c0d4230 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/1825300929 >> 172.21.15.82:0/1825300929 conn(0x7f445c08b000 msgr2=0x7f445c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/1825300929 shutdown_connections 2024-04-25T10:11:25.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/1825300929 wait complete. 2024-04-25T10:11:25.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 Processor -- start 2024-04-25T10:11:25.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- start start 2024-04-25T10:11:25.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c0d0620 0x7f445c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 0x7f445c0c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c16a730 con 0x7f445c0d1cd0 2024-04-25T10:11:25.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445c16a5b0 con 0x7f445c15da80 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445c16a8b0 con 0x7f445c0d0620 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45574/0 (socket says 172.21.15.82:45574) 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 -- 172.21.15.82:0/4255161459 learned_addr learned my addr 172.21.15.82:0/4255161459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f445affd640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c0d0620 0x7f445c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f445b7fe640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 0x7f445c0c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 -- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c0d0620 msgr2=0x7f445c15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c0d0620 0x7f445c15e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 -- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 msgr2=0x7f445c0c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 0x7f445c0c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 -- 172.21.15.82:0/4255161459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445c0c7430 con 0x7f445c0d1cd0 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f445b7fe640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 0x7f445c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f4453fff640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f444400e9b0 tx=0x7f444400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444017070 con 0x7f445c0d1cd0 2024-04-25T10:11:25.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444400c9a0 con 0x7f445c0d1cd0 2024-04-25T10:11:25.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444400cd50 con 0x7f445c0d1cd0 2024-04-25T10:11:25.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445c0c7710 con 0x7f445c0d1cd0 2024-04-25T10:11:25.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.969+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445c0c7cd0 con 0x7f445c0d1cd0 2024-04-25T10:11:25.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445c04efa0 con 0x7f445c0d1cd0 2024-04-25T10:11:25.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f444401a070 con 0x7f445c0d1cd0 2024-04-25T10:11:25.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f44537fe640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 0x7f443c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f445affd640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 0x7f443c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4444085280 con 0x7f445c0d1cd0 2024-04-25T10:11:25.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.973+0000 7f445affd640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 0x7f443c069450 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af000 tx=0x7f444c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:25.977+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4444056450 con 0x7f445c0d1cd0 2024-04-25T10:11:26.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.141+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f445c156a60 con 0x7f445c0d1cd0 2024-04-25T10:11:26.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.141+0000 7f44537fe640 1 -- 172.21.15.82:0/4255161459 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13178 (secure 0 0 0) 0x7f4444055d50 con 0x7f445c0d1cd0 2024-04-25T10:11:26.144 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:26.145 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":103,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:25.466319+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:26.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 msgr2=0x7f443c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 0x7f443c069450 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af000 tx=0x7f444c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 msgr2=0x7f445c0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f444400e9b0 tx=0x7f444400ee70 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 shutdown_connections 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f443c066f90 0x7f443c069450 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f445c15da80 0x7f445c0c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f445c0d1cd0 0x7f445c0c81f0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 --2- 172.21.15.82:0/4255161459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f445c0d0620 0x7f445c15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 >> 172.21.15.82:0/4255161459 conn(0x7f445c08b000 msgr2=0x7f445c156f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 shutdown_connections 2024-04-25T10:11:26.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.145+0000 7f446173e640 1 -- 172.21.15.82:0/4255161459 wait complete. 2024-04-25T10:11:26.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:26.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.281+0000 7f1b1d82f640 1 Processor -- start 2024-04-25T10:11:26.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.281+0000 7f1b1d82f640 1 -- start start 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b1d82f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b1815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b1d82f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b180cf970 0x7f1b180c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b1d82f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b180ca4c0 con 0x7f1b180c9ef0 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b1d82f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b180ca780 con 0x7f1b180cf970 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b1d82f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180cd440 con 0x7f1b180cf590 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1b180c9ef0 0x7f1b180b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57774/0 (socket says 172.21.15.82:57774) 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 -- 172.21.15.82:0/772403725 learned_addr learned my addr 172.21.15.82:0/772403725 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:26.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958822721 0 0) 0x7f1b180ca4c0 con 0x7f1b180c9ef0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b00003530 con 0x7f1b180c9ef0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b16ffd640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b1815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b167fc640 1 --2- 172.21.15.82:0/772403725 >> v2:172.21.15.196:3300/0 conn(0x7f1b180cf970 0x7f1b180c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2831056351 0 0) 0x7f1b00003530 con 0x7f1b180c9ef0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 msgr2=0x7f1b1815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b1815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 >> v2:172.21.15.196:3300/0 conn(0x7f1b180cf970 msgr2=0x7f1b180c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 --2- 172.21.15.82:0/772403725 >> v2:172.21.15.196:3300/0 conn(0x7f1b180cf970 0x7f1b180c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180cd690 con 0x7f1b180c9ef0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1b10001a00 con 0x7f1b180c9ef0 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 >> v1:172.21.15.82:6789/0 conn(0x7f1b180c9ef0 legacy=0x7f1b180b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b00003a30 0x7f1b00003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b00004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b00004d50 0x7f1b000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180cd440 con 0x7f1b00004d50 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0000ba80 con 0x7f1b00003a30 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0000bc80 con 0x7f1b180cf590 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b00004d50 0x7f1b000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b16ffd640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b00003a30 0x7f1b00003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b167fc640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b00004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 -- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 msgr2=0x7f1b00004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180cf590 0x7f1b00004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 -- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b00003a30 msgr2=0x7f1b00003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b00003a30 0x7f1b00003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.285+0000 7f1b177fe640 1 -- 172.21.15.82:0/772403725 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b00003530 con 0x7f1b00004d50 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b177fe640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b00004d50 0x7f1b000051d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000a8f0 tx=0x7f1b1001d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875254cabf327862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000a4c0 con 0x7f1b00004d50 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b100ad650 con 0x7f1b00004d50 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b15ffb640 1 -- 172.21.15.82:0/772403725 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b100ada40 con 0x7f1b00004d50 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/772403725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b00004d50 msgr2=0x7f1b000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/772403725 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b00004d50 0x7f1b000051d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000a8f0 tx=0x7f1b1001d420 comp rx=0 tx=0).stop 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b177fe640 1 -- 172.21.15.82:0/772403725 reap_dead start 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/772403725 shutdown_connections 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/772403725 >> 172.21.15.82:0/772403725 conn(0x7f1b1806ae60 msgr2=0x7f1b1806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/772403725 shutdown_connections 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/772403725 wait complete. 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 Processor -- start 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- start start 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180c5980 0x7f1b1809ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b181f6b50 0x7f1b181f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180cfdb0 con 0x7f1b18157e10 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180cdce0 con 0x7f1b181f6b50 2024-04-25T10:11:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b181f78f0 con 0x7f1b180c5980 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45586/0 (socket says 172.21.15.82:45586) 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 -- 172.21.15.82:0/1487821657 learned_addr learned my addr 172.21.15.82:0/1487821657 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b177fe640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b181f6b50 0x7f1b181f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b167fc640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180c5980 0x7f1b1809ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 -- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180c5980 msgr2=0x7f1b1809ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180c5980 0x7f1b1809ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 -- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b181f6b50 msgr2=0x7f1b181f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b181f6b50 0x7f1b181f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 -- 172.21.15.82:0/1487821657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180c4bc0 con 0x7f1b18157e10 2024-04-25T10:11:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b16ffd640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800fb30 tx=0x7f1b0800c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08015070 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0801a460 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b180c4ea0 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0801a850 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.289+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b180c5460 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1804efa0 con 0x7f1b18157e10 2024-04-25T10:11:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b0800b050 con 0x7f1b18157e10 2024-04-25T10:11:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1afffff640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 0x7f1b04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1b08085920 con 0x7f1b18157e10 2024-04-25T10:11:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1b167fc640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 0x7f1b04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.293+0000 7f1b167fc640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 0x7f1b04069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ad9f0 tx=0x7f1b0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.297+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b0800be20 con 0x7f1b18157e10 2024-04-25T10:11:26.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.453+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b180b4170 con 0x7f1b18157e10 2024-04-25T10:11:26.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.453+0000 7f1afffff640 1 -- 172.21.15.82:0/1487821657 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b08056af0 con 0x7f1b18157e10 2024-04-25T10:11:26.457 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 msgr2=0x7f1b04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 0x7f1b04069420 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1b180ad9f0 tx=0x7f1b0c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 msgr2=0x7f1b1809c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800fb30 tx=0x7f1b0800c2e0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 shutdown_connections 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b04066f60 0x7f1b04069420 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b180c5980 0x7f1b1809ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b181f6b50 0x7f1b181f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 --2- 172.21.15.82:0/1487821657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b18157e10 0x7f1b1809c900 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 >> 172.21.15.82:0/1487821657 conn(0x7f1b1806ae60 msgr2=0x7f1b18159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 shutdown_connections 2024-04-25T10:11:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.457+0000 7f1b1d82f640 1 -- 172.21.15.82:0/1487821657 wait complete. 2024-04-25T10:11:26.477 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:11:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 Processor -- start 2024-04-25T10:11:26.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 -- start start 2024-04-25T10:11:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf00b0b00 0x7ffaf00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b1ae0 0x7ffaf0151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf0160310 con 0x7ffaf00cc650 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf015cfa0 con 0x7ffaf00b0b00 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaf5722640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf015cdf0 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf00b0b00 0x7ffaf00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffaf00b0b00 0x7ffaf00cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45598/0 (socket says 172.21.15.82:45598) 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 -- 172.21.15.82:0/773612757 learned_addr learned my addr 172.21.15.82:0/773612757 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaef7fe640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b1ae0 0x7ffaf0151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 -- 172.21.15.82:0/773612757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b1ae0 msgr2=0x7ffaf0151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b1ae0 0x7ffaf0151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 -- 172.21.15.82:0/773612757 >> v1:172.21.15.196:6789/0 conn(0x7ffaf00cc650 legacy=0x7ffaf00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.589+0000 7ffaeeffd640 1 -- 172.21.15.82:0/773612757 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf015f550 con 0x7ffaf00b0b00 2024-04-25T10:11:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaeeffd640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b0b00 0x7ffaf00cc110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009870 tx=0x7ffae001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a5539df397cfad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaedffb640 1 -- 172.21.15.82:0/773612757 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00b0070 con 0x7ffaf00b0b00 2024-04-25T10:11:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaedffb640 1 -- 172.21.15.82:0/773612757 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae00ab600 con 0x7ffaf00b0b00 2024-04-25T10:11:26.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaedffb640 1 -- 172.21.15.82:0/773612757 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae00ab9d0 con 0x7ffaf00b0b00 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- 172.21.15.82:0/773612757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b0b00 msgr2=0x7ffaf00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b0b00 0x7ffaf00cc110 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009870 tx=0x7ffae001c340 comp rx=0 tx=0).stop 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- 172.21.15.82:0/773612757 shutdown_connections 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b1ae0 0x7ffaf0151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 --2- 172.21.15.82:0/773612757 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b0b00 0x7ffaf00cc110 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- 172.21.15.82:0/773612757 >> 172.21.15.82:0/773612757 conn(0x7ffaf00b1f10 msgr2=0x7ffaf00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- 172.21.15.82:0/773612757 shutdown_connections 2024-04-25T10:11:26.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- 172.21.15.82:0/773612757 wait complete. 2024-04-25T10:11:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 Processor -- start 2024-04-25T10:11:26.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.593+0000 7ffaf5722640 1 -- start start 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b0b00 0x7ffaf0165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf00cc650 0x7ffaf0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00b1060 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00b0ee0 con 0x7ffaf00cc650 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf015bde0 con 0x7ffaf00b0b00 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45608/0 (socket says 172.21.15.82:45608) 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 -- 172.21.15.82:0/3953376831 learned_addr learned my addr 172.21.15.82:0/3953376831 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaeeffd640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b0b00 0x7ffaf0165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 -- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b0b00 msgr2=0x7ffaf0165320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaef7fe640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf00cc650 0x7ffaf0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b0b00 0x7ffaf0165320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 -- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf00cc650 msgr2=0x7ffaf0158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf00cc650 0x7ffaf0158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 -- 172.21.15.82:0/3953376831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf0165be0 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaee7fc640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b410 tx=0x7ffae400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4015030 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4004020 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf0165900 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4004450 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf01663c0 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffae4018070 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf004efa0 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffaeeffd640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.597+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ffae4084a50 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.601+0000 7ffaeeffd640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffae00ad000 tx=0x7ffae0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.601+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffae4055b90 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.749+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ffaf006fcc0 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffad77fe640 1 -- 172.21.15.82:0/3953376831 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7ffae4055490 con 0x7ffaf00b1ae0 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:26.755 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 msgr2=0x7ffadc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ffae00ad000 tx=0x7ffae0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 msgr2=0x7ffaf00cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b410 tx=0x7ffae400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 shutdown_connections 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffadc066f60 0x7ffadc069420 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffaf00cc650 0x7ffaf0158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffaf00b1ae0 0x7ffaf00cfb20 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 --2- 172.21.15.82:0/3953376831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffaf00b0b00 0x7ffaf0165320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 >> 172.21.15.82:0/3953376831 conn(0x7ffaf00b1f10 msgr2=0x7ffaf006e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 shutdown_connections 2024-04-25T10:11:26.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.753+0000 7ffaf5722640 1 -- 172.21.15.82:0/3953376831 wait complete. 2024-04-25T10:11:26.772 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:11:26.772 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:11:26.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 Processor -- start 2024-04-25T10:11:26.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 -- start start 2024-04-25T10:11:26.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3fa40adc70 0x7f3fa406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3fa406e9f0 0x7f3fa40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa40b28c0 con 0x7f3fa40cfa50 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa40cfec0 con 0x7f3fa406e9f0 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fabcd2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa415bb20 con 0x7f3fa40adc70 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3fa926d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3fa406e9f0 0x7f3fa40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.893+0000 7f3faa26f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3fa40cfa50 0x7f3fa415f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57790/0 (socket says 172.21.15.82:57790) 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3faa26f640 1 -- 172.21.15.82:0/4287675965 learned_addr learned my addr 172.21.15.82:0/4287675965 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa9a6e640 1 --2- 172.21.15.82:0/4287675965 >> v2:172.21.15.196:3300/0 conn(0x7f3fa40adc70 0x7f3fa406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa926d640 1 -- 172.21.15.82:0/4287675965 >> v2:172.21.15.196:3300/0 conn(0x7f3fa40adc70 msgr2=0x7f3fa406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa926d640 1 --2- 172.21.15.82:0/4287675965 >> v2:172.21.15.196:3300/0 conn(0x7f3fa40adc70 0x7f3fa406e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa926d640 1 -- 172.21.15.82:0/4287675965 >> v1:172.21.15.82:6789/0 conn(0x7f3fa40cfa50 legacy=0x7f3fa415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa926d640 1 -- 172.21.15.82:0/4287675965 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40b2b60 con 0x7f3fa406e9f0 2024-04-25T10:11:26.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa8a6c640 1 -- 172.21.15.82:0/4287675965 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 873259143 0 0) 0x7f3fa40b28c0 con 0x7f3fa40cfa50 2024-04-25T10:11:26.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa926d640 1 --2- 172.21.15.82:0/4287675965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa406e9f0 0x7f3fa40cf510 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3fa406d070 tx=0x7f3f9401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616be8e8e161f8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa8a6c640 1 -- 172.21.15.82:0/4287675965 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940b0070 con 0x7f3fa406e9f0 2024-04-25T10:11:26.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa8a6c640 1 -- 172.21.15.82:0/4287675965 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f940ab600 con 0x7f3fa406e9f0 2024-04-25T10:11:26.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fa8a6c640 1 -- 172.21.15.82:0/4287675965 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940ab9d0 con 0x7f3fa406e9f0 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 -- 172.21.15.82:0/4287675965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa406e9f0 msgr2=0x7f3fa40cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/4287675965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa406e9f0 0x7f3fa40cf510 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f3fa406d070 tx=0x7f3f9401c480 comp rx=0 tx=0).stop 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 -- 172.21.15.82:0/4287675965 shutdown_connections 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/4287675965 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa406e9f0 0x7f3fa40cf510 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/4287675965 >> v2:172.21.15.196:3300/0 conn(0x7f3fa40adc70 0x7f3fa406e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 -- 172.21.15.82:0/4287675965 >> 172.21.15.82:0/4287675965 conn(0x7f3fa406ae60 msgr2=0x7f3fa406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 -- 172.21.15.82:0/4287675965 shutdown_connections 2024-04-25T10:11:26.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.897+0000 7f3fabcd2640 1 -- 172.21.15.82:0/4287675965 wait complete. 2024-04-25T10:11:26.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 Processor -- start 2024-04-25T10:11:26.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- start start 2024-04-25T10:11:26.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fa406e9f0 0x7f3fa40af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40adc70 0x7f3fa415a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40b3c00 con 0x7f3fa40cfa50 2024-04-25T10:11:26.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa40b36e0 con 0x7f3fa40adc70 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa415ae50 con 0x7f3fa406e9f0 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45624/0 (socket says 172.21.15.82:45624) 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 -- 172.21.15.82:0/181178048 learned_addr learned my addr 172.21.15.82:0/181178048 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fa9a6e640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fa406e9f0 0x7f3fa40af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 -- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fa406e9f0 msgr2=0x7f3fa40af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fa406e9f0 0x7f3fa40af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 -- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40adc70 msgr2=0x7f3fa415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fa926d640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40adc70 0x7f3fa415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40adc70 0x7f3fa415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 -- 172.21.15.82:0/181178048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40afea0 con 0x7f3fa40cfa50 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3faa26f640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00caf0 tx=0x7f3f9c0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c00ac00 con 0x7f3fa40cfa50 2024-04-25T10:11:26.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c00ada0 con 0x7f3fa40cfa50 2024-04-25T10:11:26.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa40b0180 con 0x7f3fa40cfa50 2024-04-25T10:11:26.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c012640 con 0x7f3fa40cfa50 2024-04-25T10:11:26.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa40b0740 con 0x7f3fa40cfa50 2024-04-25T10:11:26.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa4067c30 con 0x7f3fa40cfa50 2024-04-25T10:11:26.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.901+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f9c0127e0 con 0x7f3fa40cfa50 2024-04-25T10:11:26.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.905+0000 7f3f9a7fc640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 0x7f3f88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.905+0000 7f3fa9a6e640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 0x7f3f88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.905+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f9c085460 con 0x7f3fa40cfa50 2024-04-25T10:11:26.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.905+0000 7f3fa9a6e640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 0x7f3f88069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fa415e3f0 tx=0x7f3fa0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:26.909+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f9c056630 con 0x7f3fa40cfa50 2024-04-25T10:11:27.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.053+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3fa40b3d80 con 0x7f3fa40cfa50 2024-04-25T10:11:27.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3f9a7fc640 1 -- 172.21.15.82:0/181178048 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f9c055f30 con 0x7f3fa40cfa50 2024-04-25T10:11:27.147 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:11:27.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 msgr2=0x7f3f88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 0x7f3f88069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3fa415e3f0 tx=0x7f3fa0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:27.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 msgr2=0x7f3fa4155200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c00caf0 tx=0x7f3f9c0021d0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 shutdown_connections 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3f88066f60 0x7f3f88069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3fa40cfa50 0x7f3fa4155200 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3fa40adc70 0x7f3fa415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.145+0000 7f3fabcd2640 1 --2- 172.21.15.82:0/181178048 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3fa406e9f0 0x7f3fa40af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.149+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 >> 172.21.15.82:0/181178048 conn(0x7f3fa406ae60 msgr2=0x7f3fa4070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.149+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 shutdown_connections 2024-04-25T10:11:27.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.149+0000 7f3fabcd2640 1 -- 172.21.15.82:0/181178048 wait complete. 2024-04-25T10:11:27.173 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 Processor -- start 2024-04-25T10:11:27.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 -- start start 2024-04-25T10:11:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2bf015da80 0x7f2bf00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bf006be60 con 0x7f2bf00d0620 2024-04-25T10:11:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bf006c440 con 0x7f2bf015da80 2024-04-25T10:11:27.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf768e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf015de60 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf542a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2bf015da80 0x7f2bf00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:59444/0 (socket says 172.21.15.82:59444) 2024-04-25T10:11:27.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 -- 172.21.15.82:0/3574991788 learned_addr learned my addr 172.21.15.82:0/3574991788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 -- 172.21.15.82:0/3574991788 >> v2:172.21.15.196:3300/0 conn(0x7f2bf015da80 msgr2=0x7f2bf00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 --2- 172.21.15.82:0/3574991788 >> v2:172.21.15.196:3300/0 conn(0x7f2bf015da80 0x7f2bf00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 -- 172.21.15.82:0/3574991788 >> v1:172.21.15.196:6789/0 conn(0x7f2bf00d0620 legacy=0x7f2bf00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.301+0000 7f2bf4c29640 1 -- 172.21.15.82:0/3574991788 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf00d2450 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf4c29640 1 --2- 172.21.15.82:0/3574991788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abdf4e42f3b7db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2be7fff640 1 -- 172.21.15.82:0/3574991788 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00ab650 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2be7fff640 1 -- 172.21.15.82:0/3574991788 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00b0070 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2be7fff640 1 -- 172.21.15.82:0/3574991788 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00aba10 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 -- 172.21.15.82:0/3574991788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 msgr2=0x7f2bf00d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 --2- 172.21.15.82:0/3574991788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2be0006fd0 tx=0x7f2be001c540 comp rx=0 tx=0).stop 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 -- 172.21.15.82:0/3574991788 shutdown_connections 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 --2- 172.21.15.82:0/3574991788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf00d00e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 --2- 172.21.15.82:0/3574991788 >> v2:172.21.15.196:3300/0 conn(0x7f2bf015da80 0x7f2bf00d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 -- 172.21.15.82:0/3574991788 >> 172.21.15.82:0/3574991788 conn(0x7f2bf008b000 msgr2=0x7f2bf00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 -- 172.21.15.82:0/3574991788 shutdown_connections 2024-04-25T10:11:27.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 -- 172.21.15.82:0/3574991788 wait complete. 2024-04-25T10:11:27.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.305+0000 7f2bf768e640 1 Processor -- start 2024-04-25T10:11:27.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- start start 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf00d0620 0x7f2bf00cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf0154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf016a730 con 0x7f2bf015da80 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf016a5b0 con 0x7f2bf00d0620 2024-04-25T10:11:27.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bf016a8b0 con 0x7f2bf00d1cd0 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45630/0 (socket says 172.21.15.82:45630) 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 -- 172.21.15.82:0/1131940066 learned_addr learned my addr 172.21.15.82:0/1131940066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf4c29640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf0154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf542a640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf00d0620 0x7f2bf00cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 -- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 msgr2=0x7f2bf0154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf0154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 -- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf00d0620 msgr2=0x7f2bf00cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf00d0620 0x7f2bf00cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 -- 172.21.15.82:0/1131940066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf01553c0 con 0x7f2bf015da80 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf5c2b640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2be800c770 tx=0x7f2be800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be8003b80 con 0x7f2bf015da80 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be8003d20 con 0x7f2bf015da80 2024-04-25T10:11:27.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bf0154e60 con 0x7f2bf015da80 2024-04-25T10:11:27.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be800aa30 con 0x7f2bf015da80 2024-04-25T10:11:27.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bf00cb710 con 0x7f2bf015da80 2024-04-25T10:11:27.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf004efa0 con 0x7f2bf015da80 2024-04-25T10:11:27.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.309+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2be800abd0 con 0x7f2bf015da80 2024-04-25T10:11:27.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.313+0000 7f2be5ffb640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 0x7f2bd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.313+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f2be8085710 con 0x7f2bf015da80 2024-04-25T10:11:27.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.313+0000 7f2bf542a640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 0x7f2bd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.313+0000 7f2bf542a640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 0x7f2bd4069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009ac0 tx=0x7f2bec009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.317+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bf01553c0 con 0x7f2bf015da80 2024-04-25T10:11:27.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:27.461+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2bf00b52d0 con 0x7f2bf015da80 2024-04-25T10:11:28.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2be5ffb640 1 -- 172.21.15.82:0/1131940066 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2be8056290 con 0x7f2bf015da80 2024-04-25T10:11:28.159 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 msgr2=0x7f2bd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 0x7f2bd4069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009ac0 tx=0x7f2bec009190 comp rx=0 tx=0).stop 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 msgr2=0x7f2bf01564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f2be800c770 tx=0x7f2be800cc30 comp rx=0 tx=0).stop 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 shutdown_connections 2024-04-25T10:11:28.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd4066f60 0x7f2bd4069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bf00d1cd0 0x7f2bf0154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bf015da80 0x7f2bf01564f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 --2- 172.21.15.82:0/1131940066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bf00d0620 0x7f2bf00cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 >> 172.21.15.82:0/1131940066 conn(0x7f2bf008b000 msgr2=0x7f2bf00b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.157+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 shutdown_connections 2024-04-25T10:11:28.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.161+0000 7f2bf768e640 1 -- 172.21.15.82:0/1131940066 wait complete. 2024-04-25T10:11:28.195 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.313+0000 7f4f88908640 1 Processor -- start 2024-04-25T10:11:28.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.313+0000 7f4f88908640 1 -- start start 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f4f8415b6b0 0x7f4f840ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f84073010 0x7f4f840cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f840cbd20 con 0x7f4f840cf590 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f8415ba90 con 0x7f4f8415b6b0 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840cc000 con 0x7f4f84073010 2024-04-25T10:11:28.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f81d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4f840cf590 0x7f4f8415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57798/0 (socket says 172.21.15.82:57798) 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f81d9b640 1 -- 172.21.15.82:0/3949705222 learned_addr learned my addr 172.21.15.82:0/3949705222 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 --2- 172.21.15.82:0/3949705222 >> v2:172.21.15.82:3300/0 conn(0x7f4f8415b6b0 0x7f4f840ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f82d9d640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f84073010 0x7f4f840cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8159a640 1 -- 172.21.15.82:0/3949705222 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984283121 0 0) 0x7f4f840cbd20 con 0x7f4f840cf590 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8159a640 1 -- 172.21.15.82:0/3949705222 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f6c003530 con 0x7f4f840cf590 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 -- 172.21.15.82:0/3949705222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f84073010 msgr2=0x7f4f840cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f84073010 0x7f4f840cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 -- 172.21.15.82:0/3949705222 >> v1:172.21.15.82:6789/0 conn(0x7f4f840cf590 legacy=0x7f4f8415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 -- 172.21.15.82:0/3949705222 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840cc2e0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8259c640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f840ca3c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097c0 tx=0x7f4f7401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e69658b596e6fbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8159a640 1 -- 172.21.15.82:0/3949705222 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b0070 con 0x7f4f8415b6b0 2024-04-25T10:11:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8159a640 1 -- 172.21.15.82:0/3949705222 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f740ab600 con 0x7f4f8415b6b0 2024-04-25T10:11:28.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f8159a640 1 -- 172.21.15.82:0/3949705222 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740ab9d0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- 172.21.15.82:0/3949705222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 msgr2=0x7f4f840ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f840ca3c0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f4f740097c0 tx=0x7f4f7401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- 172.21.15.82:0/3949705222 shutdown_connections 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f84073010 0x7f4f840cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 --2- 172.21.15.82:0/3949705222 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f840ca3c0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.317+0000 7f4f88908640 1 -- 172.21.15.82:0/3949705222 >> 172.21.15.82:0/3949705222 conn(0x7f4f8406ae60 msgr2=0x7f4f8406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- 172.21.15.82:0/3949705222 shutdown_connections 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- 172.21.15.82:0/3949705222 wait complete. 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 Processor -- start 2024-04-25T10:11:28.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- start start 2024-04-25T10:11:28.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84073010 0x7f4f84154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f840cf590 0x7f4f84098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840cadb0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840ca900 con 0x7f4f84073010 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840caf30 con 0x7f4f840cf590 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45652/0 (socket says 172.21.15.82:45652) 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 -- 172.21.15.82:0/1717713468 learned_addr learned my addr 172.21.15.82:0/1717713468 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f81d9b640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f840cf590 0x7f4f84098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 -- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f840cf590 msgr2=0x7f4f84098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f8259c640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84073010 0x7f4f84154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f840cf590 0x7f4f84098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 -- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84073010 msgr2=0x7f4f84154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84073010 0x7f4f84154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 -- 172.21.15.82:0/1717713468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f84098e80 con 0x7f4f8415b6b0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f82d9d640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c007e60 tx=0x7f4f7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c014620 con 0x7f4f8415b6b0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c019070 con 0x7f4f8415b6b0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f840ad580 con 0x7f4f8415b6b0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c014a00 con 0x7f4f8415b6b0 2024-04-25T10:11:28.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f840ac7c0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.321+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8404efa0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.325+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f7c014ba0 con 0x7f4f8415b6b0 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.325+0000 7f4f72ffd640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 0x7f4f64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.325+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f4f7c085a70 con 0x7f4f8415b6b0 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.325+0000 7f4f8259c640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 0x7f4f64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.325+0000 7f4f8259c640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 0x7f4f64069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4f740ad000 tx=0x7f4f74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.329+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f7c011260 con 0x7f4f8415b6b0 2024-04-25T10:11:28.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:28.521+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4f840b3dd0 con 0x7f4f8415b6b0 2024-04-25T10:11:29.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.193+0000 7f4f72ffd640 1 -- 172.21.15.82:0/1717713468 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4f7c056950 con 0x7f4f8415b6b0 2024-04-25T10:11:29.195 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:11:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.197+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 msgr2=0x7f4f64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 0x7f4f64069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4f740ad000 tx=0x7f4f74008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 msgr2=0x7f4f84157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c007e60 tx=0x7f4f7c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 shutdown_connections 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4f64066f60 0x7f4f64069420 secure :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4f740ad000 tx=0x7f4f74008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4f840cf590 0x7f4f84098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4f8415b6b0 0x7f4f84157c80 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 --2- 172.21.15.82:0/1717713468 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4f84073010 0x7f4f84154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 >> 172.21.15.82:0/1717713468 conn(0x7f4f8406ae60 msgr2=0x7f4f840af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 shutdown_connections 2024-04-25T10:11:29.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.201+0000 7f4f88908640 1 -- 172.21.15.82:0/1717713468 wait complete. 2024-04-25T10:11:29.232 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 Processor -- start 2024-04-25T10:11:29.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 -- start start 2024-04-25T10:11:29.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c15da10 0x7f2a5c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a5c0b24a0 0x7f2a5c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 0x7f2a5c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0ae860 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c15e210 con 0x7f2a5c071e90 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a611e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c15e4f0 con 0x7f2a5c15da10 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a5c0b24a0 0x7f2a5c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a5c0b24a0 0x7f2a5c071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45658/0 (socket says 172.21.15.82:45658) 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 -- 172.21.15.82:0/1895078141 learned_addr learned my addr 172.21.15.82:0/1895078141 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5ad9d640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c15da10 0x7f2a5c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5b59e640 1 --2- 172.21.15.82:0/1895078141 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 0x7f2a5c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 -- 172.21.15.82:0/1895078141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c15da10 msgr2=0x7f2a5c071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c15da10 0x7f2a5c071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 -- 172.21.15.82:0/1895078141 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 msgr2=0x7f2a5c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 --2- 172.21.15.82:0/1895078141 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 0x7f2a5c1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5a59c640 1 -- 172.21.15.82:0/1895078141 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0ae4c0 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.345+0000 7f2a5b59e640 1 --2- 172.21.15.82:0/1895078141 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 0x7f2a5c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a5a59c640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c0b24a0 0x7f2a5c071570 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c070fc0 tx=0x7f2a5001c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf8ad08b66eb27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a59d9b640 1 -- 172.21.15.82:0/1895078141 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500b1070 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a59d9b640 1 -- 172.21.15.82:0/1895078141 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a500ac440 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a59d9b640 1 -- 172.21.15.82:0/1895078141 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a500ac810 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- 172.21.15.82:0/1895078141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c0b24a0 msgr2=0x7f2a5c071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c0b24a0 0x7f2a5c071570 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c070fc0 tx=0x7f2a5001c140 comp rx=0 tx=0).stop 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- 172.21.15.82:0/1895078141 shutdown_connections 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 --2- 172.21.15.82:0/1895078141 >> v2:172.21.15.196:3300/0 conn(0x7f2a5c071e90 0x7f2a5c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c0b24a0 0x7f2a5c071570 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 --2- 172.21.15.82:0/1895078141 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c15da10 0x7f2a5c071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- 172.21.15.82:0/1895078141 >> 172.21.15.82:0/1895078141 conn(0x7f2a5c0b11a0 msgr2=0x7f2a5c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- 172.21.15.82:0/1895078141 shutdown_connections 2024-04-25T10:11:29.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- 172.21.15.82:0/1895078141 wait complete. 2024-04-25T10:11:29.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 Processor -- start 2024-04-25T10:11:29.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.349+0000 7f2a611e6640 1 -- start start 2024-04-25T10:11:29.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c071e90 0x7f2a5c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0b24a0 0x7f2a5c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0acf50 con 0x7f2a5c15da10 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0acdd0 con 0x7f2a5c071e90 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0ad0d0 con 0x7f2a5c0b24a0 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45662/0 (socket says 172.21.15.82:45662) 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 -- 172.21.15.82:0/3777458927 learned_addr learned my addr 172.21.15.82:0/3777458927 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5ad9d640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c071e90 0x7f2a5c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5a59c640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0b24a0 0x7f2a5c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 -- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0b24a0 msgr2=0x7f2a5c13fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0b24a0 0x7f2a5c13fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 -- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c071e90 msgr2=0x7f2a5c15cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c071e90 0x7f2a5c15cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 -- 172.21.15.82:0/3777458927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c15b3f0 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5b59e640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cef0 tx=0x7f2a5400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5401a070 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400d840 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c158880 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400dc30 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c15d040 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c04efa0 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a5401d070 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a54085940 con 0x7f2a5c15da10 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5ad9d640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.353+0000 7f2a5ad9d640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 0x7f2a48069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c150480 tx=0x7f2a4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.357+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a54056850 con 0x7f2a5c15da10 2024-04-25T10:11:29.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.521+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a5c0b1dc0 con 0x7f2a5c15da10 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a437fe640 1 -- 172.21.15.82:0/3777458927 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f2a54056150 con 0x7f2a5c15da10 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":106,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:28.527524+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 msgr2=0x7f2a48069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 0x7f2a48069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c150480 tx=0x7f2a4c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 msgr2=0x7f2a5c159360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f2a5400cef0 tx=0x7f2a5400a710 comp rx=0 tx=0).stop 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 shutdown_connections 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a48066f60 0x7f2a48069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a5c0b24a0 0x7f2a5c13fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a5c15da10 0x7f2a5c159360 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 --2- 172.21.15.82:0/3777458927 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a5c071e90 0x7f2a5c15cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.525+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 >> 172.21.15.82:0/3777458927 conn(0x7f2a5c0b11a0 msgr2=0x7f2a5c0b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.529+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 shutdown_connections 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.529+0000 7f2a611e6640 1 -- 172.21.15.82:0/3777458927 wait complete. 2024-04-25T10:11:29.548 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:29.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.653+0000 7fc25296a640 1 Processor -- start 2024-04-25T10:11:29.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.653+0000 7fc25296a640 1 -- start start 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc24c0b24d0 0x7fc24c09aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc24c06c260 0x7fc24c06c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c099cc0 con 0x7fc24c0b28b0 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c099a70 con 0x7fc24c06c260 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c099fa0 con 0x7fc24c0b24d0 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc24c06c260 0x7fc24c06c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc24c06c260 0x7fc24c06c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45676/0 (socket says 172.21.15.82:45676) 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 -- 172.21.15.82:0/106450718 learned_addr learned my addr 172.21.15.82:0/106450718 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc24bfff640 1 --2- 172.21.15.82:0/106450718 >> v2:172.21.15.196:3300/0 conn(0x7fc24c0b24d0 0x7fc24c09aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 -- 172.21.15.82:0/106450718 >> v2:172.21.15.196:3300/0 conn(0x7fc24c0b24d0 msgr2=0x7fc24c09aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 --2- 172.21.15.82:0/106450718 >> v2:172.21.15.196:3300/0 conn(0x7fc24c0b24d0 0x7fc24c09aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 -- 172.21.15.82:0/106450718 >> v1:172.21.15.196:6789/0 conn(0x7fc24c0b28b0 legacy=0x7fc24c0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 -- 172.21.15.82:0/106450718 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c09a280 con 0x7fc24c06c260 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc250f07640 1 --2- 172.21.15.82:0/106450718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c06c6d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc24c072830 tx=0x7fc24401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b6c74fccd4ff22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc24affd640 1 -- 172.21.15.82:0/106450718 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a5740 con 0x7fc24c06c260 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc24affd640 1 -- 172.21.15.82:0/106450718 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2440a58e0 con 0x7fc24c06c260 2024-04-25T10:11:29.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc24affd640 1 -- 172.21.15.82:0/106450718 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440a5cb0 con 0x7fc24c06c260 2024-04-25T10:11:29.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 -- 172.21.15.82:0/106450718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 msgr2=0x7fc24c06c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.657+0000 7fc25296a640 1 --2- 172.21.15.82:0/106450718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c06c6d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fc24c072830 tx=0x7fc24401c860 comp rx=0 tx=0).stop 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/106450718 shutdown_connections 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 --2- 172.21.15.82:0/106450718 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c06c6d0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 --2- 172.21.15.82:0/106450718 >> v2:172.21.15.196:3300/0 conn(0x7fc24c0b24d0 0x7fc24c09aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/106450718 >> 172.21.15.82:0/106450718 conn(0x7fc24c0b3020 msgr2=0x7fc24c0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/106450718 shutdown_connections 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/106450718 wait complete. 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 Processor -- start 2024-04-25T10:11:29.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- start start 2024-04-25T10:11:29.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0b24d0 0x7fc24c159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc24c0b28b0 0x7fc24c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0b0710 con 0x7fc24c06c260 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0b2d10 con 0x7fc24c0b24d0 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0b0890 con 0x7fc24c0b28b0 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45688/0 (socket says 172.21.15.82:45688) 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 -- 172.21.15.82:0/3207928504 learned_addr learned my addr 172.21.15.82:0/3207928504 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 -- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc24c0b28b0 msgr2=0x7fc24c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc24c0b28b0 0x7fc24c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 -- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0b24d0 msgr2=0x7fc24c159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0b24d0 0x7fc24c159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 -- 172.21.15.82:0/3207928504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c14fce0 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc24bfff640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d5f0 tx=0x7fc23c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0125f0 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c019070 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c14ff80 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0129d0 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c150540 con 0x7fc24c06c260 2024-04-25T10:11:29.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.661+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c04efa0 con 0x7fc24c06c260 2024-04-25T10:11:29.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.665+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc23c012b70 con 0x7fc24c06c260 2024-04-25T10:11:29.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.665+0000 7fc248ff9640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 0x7fc238069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.665+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc23c085660 con 0x7fc24c06c260 2024-04-25T10:11:29.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.665+0000 7fc24b7fe640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 0x7fc238069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.665+0000 7fc24b7fe640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 0x7fc238069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc24c160ea0 tx=0x7fc240006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.669+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc23c0564e0 con 0x7fc24c06c260 2024-04-25T10:11:29.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.817+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc24c073d70 con 0x7fc24c06c260 2024-04-25T10:11:29.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.817+0000 7fc248ff9640 1 -- 172.21.15.82:0/3207928504 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16909 (secure 0 0 0) 0x7fc23c083e20 con 0x7fc24c06c260 2024-04-25T10:11:29.822 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:29.822 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":106,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:28.527524+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 msgr2=0x7fc238069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 0x7fc238069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc24c160ea0 tx=0x7fc240006bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 msgr2=0x7fc24c15adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00d5f0 tx=0x7fc23c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 shutdown_connections 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc238066f60 0x7fc238069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc24c0b28b0 0x7fc24c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc24c0b24d0 0x7fc24c159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 --2- 172.21.15.82:0/3207928504 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc24c06c260 0x7fc24c15adf0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 >> 172.21.15.82:0/3207928504 conn(0x7fc24c0b3020 msgr2=0x7fc24c0b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 shutdown_connections 2024-04-25T10:11:29.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.821+0000 7fc25296a640 1 -- 172.21.15.82:0/3207928504 wait complete. 2024-04-25T10:11:29.839 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:29.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 Processor -- start 2024-04-25T10:11:29.960 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 -- start start 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480cf590 0x7f1b4815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 0x7f1b480c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b480c9ef0 0x7f1b480b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480ca780 con 0x7f1b480c9ef0 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b480cd440 con 0x7f1b480cf970 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ec99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480cdce0 con 0x7f1b480cf590 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b480c9ef0 0x7f1b480b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1b480c9ef0 0x7f1b480b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45690/0 (socket says 172.21.15.82:45690) 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 -- 172.21.15.82:0/1476055642 learned_addr learned my addr 172.21.15.82:0/1476055642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b47fff640 1 --2- 172.21.15.82:0/1476055642 >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 0x7f1b480c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4ca35640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480cf590 0x7f1b4815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 -- 172.21.15.82:0/1476055642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480cf590 msgr2=0x7f1b4815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480cf590 0x7f1b4815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 -- 172.21.15.82:0/1476055642 >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 msgr2=0x7f1b480c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 --2- 172.21.15.82:0/1476055642 >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 0x7f1b480c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b4d236640 1 -- 172.21.15.82:0/1476055642 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b480ca4c0 con 0x7f1b480c9ef0 2024-04-25T10:11:29.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.957+0000 7f1b47fff640 1 --2- 172.21.15.82:0/1476055642 >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 0x7f1b480c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4d236640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480c9ef0 0x7f1b480b21b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1b480d0110 tx=0x7f1b4001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39927a6418f94b42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b477fe640 1 -- 172.21.15.82:0/1476055642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400a5580 con 0x7f1b480c9ef0 2024-04-25T10:11:29.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b477fe640 1 -- 172.21.15.82:0/1476055642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b400a5720 con 0x7f1b480c9ef0 2024-04-25T10:11:29.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b477fe640 1 -- 172.21.15.82:0/1476055642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400a5af0 con 0x7f1b480c9ef0 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 -- 172.21.15.82:0/1476055642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480c9ef0 msgr2=0x7f1b480b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480c9ef0 0x7f1b480b21b0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1b480d0110 tx=0x7f1b4001c700 comp rx=0 tx=0).stop 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 -- 172.21.15.82:0/1476055642 shutdown_connections 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480c9ef0 0x7f1b480b21b0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/1476055642 >> v2:172.21.15.196:3300/0 conn(0x7f1b480cf970 0x7f1b480c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/1476055642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480cf590 0x7f1b4815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 -- 172.21.15.82:0/1476055642 >> 172.21.15.82:0/1476055642 conn(0x7f1b4806ae60 msgr2=0x7f1b4806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 -- 172.21.15.82:0/1476055642 shutdown_connections 2024-04-25T10:11:29.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 -- 172.21.15.82:0/1476055642 wait complete. 2024-04-25T10:11:29.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.961+0000 7f1b4ec99640 1 Processor -- start 2024-04-25T10:11:29.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- start start 2024-04-25T10:11:29.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 0x7f1b4815d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 0x7f1b48098520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ac2f0 con 0x7f1b480cf970 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ac170 con 0x7f1b480cf590 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b480ce150 con 0x7f1b480c9ef0 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45702/0 (socket says 172.21.15.82:45702) 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 -- 172.21.15.82:0/3092367823 learned_addr learned my addr 172.21.15.82:0/3092367823 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ca35640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 0x7f1b4815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b47fff640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 0x7f1b48098520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 -- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 msgr2=0x7f1b4815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 0x7f1b4815d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 -- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 msgr2=0x7f1b48098520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 0x7f1b48098520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 -- 172.21.15.82:0/3092367823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48152d90 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b47fff640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 0x7f1b48098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ca35640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 0x7f1b4815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4d236640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f1b400af040 tx=0x7f1b400194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400bc020 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b400b8670 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b480c7970 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b40009510 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b48153370 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4804efa0 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b400bf020 con 0x7f1b480cf970 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 0x7f1b34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b4ca35640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 0x7f1b34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.965+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b40120170 con 0x7f1b480cf970 2024-04-25T10:11:29.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.969+0000 7f1b4ca35640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 0x7f1b34069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b840 tx=0x7f1b38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:29.969+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b400c6030 con 0x7f1b480cf970 2024-04-25T10:11:30.153 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.149+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1b48073280 con 0x7f1b480cf970 2024-04-25T10:11:30.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.177+0000 7f1b457fa640 1 -- 172.21.15.82:0/3092367823 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v74) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1b480c7970 con 0x7f1b480cf970 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 msgr2=0x7f1b34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 0x7f1b34069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800b840 tx=0x7f1b38008040 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 msgr2=0x7f1b48098a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f1b400af040 tx=0x7f1b400194a0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 shutdown_connections 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b34066f60 0x7f1b34069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b480cf970 0x7f1b48098a60 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b480cf590 0x7f1b48098520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 --2- 172.21.15.82:0/3092367823 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b480c9ef0 0x7f1b4815d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 >> 172.21.15.82:0/3092367823 conn(0x7f1b4806ae60 msgr2=0x7f1b48154300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 shutdown_connections 2024-04-25T10:11:30.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.181+0000 7f1b4ec99640 1 -- 172.21.15.82:0/3092367823 wait complete. 2024-04-25T10:11:30.200 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:30.200 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:30.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 Processor -- start 2024-04-25T10:11:30.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 -- start start 2024-04-25T10:11:30.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f29dc15b6b0 0x7f29dc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc15ff50 con 0x7f29dc0cf590 2024-04-25T10:11:30.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29dc0cbd20 con 0x7f29dc073010 2024-04-25T10:11:30.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29e1af5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29dc15ba90 con 0x7f29dc15b6b0 2024-04-25T10:11:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29daffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f29dc0cf590 0x7f29dc15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57812/0 (socket says 172.21.15.82:57812) 2024-04-25T10:11:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29daffd640 1 -- 172.21.15.82:0/2546770228 learned_addr learned my addr 172.21.15.82:0/2546770228 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29db7fe640 1 --2- 172.21.15.82:0/2546770228 >> v2:172.21.15.82:3300/0 conn(0x7f29dc15b6b0 0x7f29dc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29da7fc640 1 -- 172.21.15.82:0/2546770228 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847296310 0 0) 0x7f29dc15ff50 con 0x7f29dc0cf590 2024-04-25T10:11:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29da7fc640 1 -- 172.21.15.82:0/2546770228 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29c8003530 con 0x7f29dc0cf590 2024-04-25T10:11:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29db7fe640 1 -- 172.21.15.82:0/2546770228 >> v1:172.21.15.196:6789/0 conn(0x7f29dc073010 legacy=0x7f29dc15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29db7fe640 1 -- 172.21.15.82:0/2546770228 >> v1:172.21.15.82:6789/0 conn(0x7f29dc0cf590 legacy=0x7f29dc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.325+0000 7f29db7fe640 1 -- 172.21.15.82:0/2546770228 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0cc000 con 0x7f29dc15b6b0 2024-04-25T10:11:30.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29db7fe640 1 --2- 172.21.15.82:0/2546770228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc15b6b0 0x7f29dc0ca3c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009d90 tx=0x7f29d401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0bbb049017a08c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29da7fc640 1 -- 172.21.15.82:0/2546770228 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40b2070 con 0x7f29dc15b6b0 2024-04-25T10:11:30.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29da7fc640 1 -- 172.21.15.82:0/2546770228 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d40a5a00 con 0x7f29dc15b6b0 2024-04-25T10:11:30.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29da7fc640 1 -- 172.21.15.82:0/2546770228 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d40a5dd0 con 0x7f29dc15b6b0 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- 172.21.15.82:0/2546770228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc15b6b0 msgr2=0x7f29dc0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 --2- 172.21.15.82:0/2546770228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc15b6b0 0x7f29dc0ca3c0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009d90 tx=0x7f29d401c940 comp rx=0 tx=0).stop 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- 172.21.15.82:0/2546770228 shutdown_connections 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 --2- 172.21.15.82:0/2546770228 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc15b6b0 0x7f29dc0ca3c0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- 172.21.15.82:0/2546770228 >> 172.21.15.82:0/2546770228 conn(0x7f29dc06ae60 msgr2=0x7f29dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- 172.21.15.82:0/2546770228 shutdown_connections 2024-04-25T10:11:30.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- 172.21.15.82:0/2546770228 wait complete. 2024-04-25T10:11:30.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 Processor -- start 2024-04-25T10:11:30.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- start start 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29dc073010 0x7f29dc0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29dc15b6b0 0x7f29dc152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0ca900 con 0x7f29dc0cf590 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0cba30 con 0x7f29dc15b6b0 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29e1af5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29dc0cfdb0 con 0x7f29dc073010 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45730/0 (socket says 172.21.15.82:45730) 2024-04-25T10:11:30.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 -- 172.21.15.82:0/3044793087 learned_addr learned my addr 172.21.15.82:0/3044793087 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29db7fe640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29dc073010 0x7f29dc0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29dbfff640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29dc15b6b0 0x7f29dc152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 -- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29dc073010 msgr2=0x7f29dc0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29dc073010 0x7f29dc0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 -- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29dc15b6b0 msgr2=0x7f29dc152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29dc15b6b0 0x7f29dc152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.329+0000 7f29daffd640 1 -- 172.21.15.82:0/3044793087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29dc0ad260 con 0x7f29dc0cf590 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29daffd640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c4d0 tx=0x7f29c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4017070 con 0x7f29dc0cf590 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4005cf0 con 0x7f29dc0cf590 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29dc0ad540 con 0x7f29dc0cf590 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4007270 con 0x7f29dc0cf590 2024-04-25T10:11:30.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29dc0adb00 con 0x7f29dc0cf590 2024-04-25T10:11:30.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29dc04efa0 con 0x7f29dc0cf590 2024-04-25T10:11:30.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29c401a070 con 0x7f29dc0cf590 2024-04-25T10:11:30.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:30.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29c4084430 con 0x7f29dc0cf590 2024-04-25T10:11:30.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29db7fe640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:30.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.333+0000 7f29db7fe640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 0x7f29bc069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f29d40af000 tx=0x7f29d40a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:30.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.337+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29c4054cf0 con 0x7f29dc0cf590 2024-04-25T10:11:30.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:30.533+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f29dc0afb90 con 0x7f29dc0cf590 2024-04-25T10:11:31.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.185+0000 7f29d3fff640 1 -- 172.21.15.82:0/3044793087 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v75) v1 ==== 122+0+0 (secure 0 0 0) 0x7f29c4058d90 con 0x7f29dc0cf590 2024-04-25T10:11:31.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 msgr2=0x7f29bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 0x7f29bc069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f29d40af000 tx=0x7f29d40a7070 comp rx=0 tx=0).stop 2024-04-25T10:11:31.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 msgr2=0x7f29dc0ac9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c4d0 tx=0x7f29c400c990 comp rx=0 tx=0).stop 2024-04-25T10:11:31.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 shutdown_connections 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f29bc066f60 0x7f29bc069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f29dc15b6b0 0x7f29dc152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f29dc0cf590 0x7f29dc0ac9f0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 --2- 172.21.15.82:0/3044793087 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f29dc073010 0x7f29dc0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 >> 172.21.15.82:0/3044793087 conn(0x7f29dc06ae60 msgr2=0x7f29dc0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 shutdown_connections 2024-04-25T10:11:31.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.189+0000 7f29e1af5640 1 -- 172.21.15.82:0/3044793087 wait complete. 2024-04-25T10:11:31.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:31.212 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 Processor -- start 2024-04-25T10:11:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 -- start start 2024-04-25T10:11:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 0x7f15a815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15a80b3f50 0x7f15a80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 0x7f15a80b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a815f200 con 0x7f15a80b3f50 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a80b2090 con 0x7f15a80ce430 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15ac8bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80b43e0 con 0x7f15a815a700 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15a80b3f50 0x7f15a80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f15a80b3f50 0x7f15a80cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45732/0 (socket says 172.21.15.82:45732) 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1757841024 learned_addr learned my addr 172.21.15.82:0/1757841024 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a659c640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 0x7f15a815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a6d9d640 1 --2- 172.21.15.82:0/1757841024 >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 0x7f15a80b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1757841024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 msgr2=0x7f15a815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 0x7f15a815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1757841024 >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 msgr2=0x7f15a80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1757841024 >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 0x7f15a80b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1757841024 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80b46c0 con 0x7f15a80b3f50 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a659c640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 0x7f15a815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.329+0000 7f15a6d9d640 1 --2- 172.21.15.82:0/1757841024 >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 0x7f15a80b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a80b3f50 0x7f15a80cdef0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f15a806ca20 tx=0x7f159c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b7391a553228d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a559a640 1 -- 172.21.15.82:0/1757841024 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0b0070 con 0x7f15a80b3f50 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a559a640 1 -- 172.21.15.82:0/1757841024 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c0ab600 con 0x7f15a80b3f50 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a559a640 1 -- 172.21.15.82:0/1757841024 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0ab9d0 con 0x7f15a80b3f50 2024-04-25T10:11:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1757841024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a80b3f50 msgr2=0x7f15a80cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a80b3f50 0x7f15a80cdef0 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f15a806ca20 tx=0x7f159c01c3d0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1757841024 shutdown_connections 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1757841024 >> v2:172.21.15.196:3300/0 conn(0x7f15a80ce430 0x7f15a80b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a80b3f50 0x7f15a80cdef0 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1757841024 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a815a700 0x7f15a815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1757841024 >> 172.21.15.82:0/1757841024 conn(0x7f15a80b11a0 msgr2=0x7f15a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1757841024 shutdown_connections 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1757841024 wait complete. 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 Processor -- start 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- start start 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a80ce430 0x7f15a80ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a8144e10 0x7f15a80cf410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80cd490 con 0x7f15a815a700 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a80cd310 con 0x7f15a80ce430 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a8071030 con 0x7f15a8144e10 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45736/0 (socket says 172.21.15.82:45736) 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1564855498 learned_addr learned my addr 172.21.15.82:0/1564855498 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a659c640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a80ce430 0x7f15a80ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a6d9d640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a8144e10 0x7f15a80cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a8144e10 msgr2=0x7f15a80cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a8144e10 0x7f15a80cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a80ce430 msgr2=0x7f15a80ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a80ce430 0x7f15a80ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 -- 172.21.15.82:0/1564855498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a8153ad0 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15a5d9b640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f159c000c10 tx=0x7f159c019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0b0040 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159c0b9070 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a8153d70 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.333+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c0b4810 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8154330 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a804efa0 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f159c0b49b0 con 0x7f15a815a700 2024-04-25T10:11:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f1596ffd640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 0x7f1588069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f159c11f0e0 con 0x7f15a815a700 2024-04-25T10:11:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f15a659c640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 0x7f1588069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.337+0000 7f15a659c640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 0x7f1588069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f159800a9a0 tx=0x7f1598008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.341+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f159c016110 con 0x7f15a815a700 2024-04-25T10:11:31.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.529+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15a806f260 con 0x7f15a815a700 2024-04-25T10:11:31.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.529+0000 7f1596ffd640 1 -- 172.21.15.82:0/1564855498 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f159c0eff60 con 0x7f15a815a700 2024-04-25T10:11:31.534 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:31.534 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:11:30.539889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8769":{"gid":8769,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3162787996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3162787996},{"type":"v1","addr":"172.21.15.82:6837","nonce":3162787996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 75 2024-04-25T10:11:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 msgr2=0x7f1588069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 0x7f1588069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f159800a9a0 tx=0x7f1598008040 comp rx=0 tx=0).stop 2024-04-25T10:11:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 msgr2=0x7f15a81fa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f159c000c10 tx=0x7f159c019170 comp rx=0 tx=0).stop 2024-04-25T10:11:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 shutdown_connections 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1588066f60 0x7f1588069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15a8144e10 0x7f15a80cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f15a815a700 0x7f15a81fa880 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 --2- 172.21.15.82:0/1564855498 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f15a80ce430 0x7f15a80ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 >> 172.21.15.82:0/1564855498 conn(0x7f15a80b11a0 msgr2=0x7f15a80ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 shutdown_connections 2024-04-25T10:11:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.533+0000 7f15ac8bc640 1 -- 172.21.15.82:0/1564855498 wait complete. 2024-04-25T10:11:31.551 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:31.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.665+0000 7f95ea8e0640 1 Processor -- start 2024-04-25T10:11:31.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95ea8e0640 1 -- start start 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95ea8e0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f95e4144ca0 0x7f95e406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95ea8e0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e40ccc70 con 0x7f95e406ec60 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95ea8e0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e406dc40 con 0x7f95e40ce510 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95ea8e0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e40cc2b0 con 0x7f95e4144ca0 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e8e7d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f95e406ec60 0x7f95e4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57820/0 (socket says 172.21.15.82:57820) 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e8e7d640 1 -- 172.21.15.82:0/4056150776 learned_addr learned my addr 172.21.15.82:0/4056150776 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e37fe640 1 --2- 172.21.15.82:0/4056150776 >> v2:172.21.15.82:3300/0 conn(0x7f95e4144ca0 0x7f95e406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390596490 0 0) 0x7f95e40ccc70 con 0x7f95e406ec60 2024-04-25T10:11:31.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc003530 con 0x7f95e406ec60 2024-04-25T10:11:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e37fe640 1 -- 172.21.15.82:0/4056150776 >> v1:172.21.15.196:6789/0 conn(0x7f95e40ce510 legacy=0x7f95e40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e37fe640 1 -- 172.21.15.82:0/4056150776 >> v1:172.21.15.82:6789/0 conn(0x7f95e406ec60 legacy=0x7f95e4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3065813303 0 0) 0x7f95cc003530 con 0x7f95e406ec60 2024-04-25T10:11:31.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.669+0000 7f95e37fe640 1 -- 172.21.15.82:0/4056150776 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e406df20 con 0x7f95e4144ca0 2024-04-25T10:11:31.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e37fe640 1 --2- 172.21.15.82:0/4056150776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e406e720 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097c0 tx=0x7f95d801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5643b71f96d94a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d80b5070 con 0x7f95e4144ca0 2024-04-25T10:11:31.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d80a8630 con 0x7f95e4144ca0 2024-04-25T10:11:31.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e2ffd640 1 -- 172.21.15.82:0/4056150776 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc001a00 con 0x7f95e4144ca0 2024-04-25T10:11:31.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4056150776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 msgr2=0x7f95e406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4056150776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e406e720 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f95d80097c0 tx=0x7f95d801ebb0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4056150776 shutdown_connections 2024-04-25T10:11:31.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4056150776 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e406e720 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4056150776 >> 172.21.15.82:0/4056150776 conn(0x7f95e40b11a0 msgr2=0x7f95e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4056150776 shutdown_connections 2024-04-25T10:11:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4056150776 wait complete. 2024-04-25T10:11:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 Processor -- start 2024-04-25T10:11:31.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- start start 2024-04-25T10:11:31.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95e406ec60 0x7f95e413c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95e40ce510 0x7f95e4153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4142d90 con 0x7f95e4144ca0 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e406e290 con 0x7f95e406ec60 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4142f10 con 0x7f95e40ce510 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45768/0 (socket says 172.21.15.82:45768) 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 -- 172.21.15.82:0/4136636486 learned_addr learned my addr 172.21.15.82:0/4136636486 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e3fff640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95e406ec60 0x7f95e413c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e37fe640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95e40ce510 0x7f95e4153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 -- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95e40ce510 msgr2=0x7f95e4153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95e40ce510 0x7f95e4153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 -- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95e406ec60 msgr2=0x7f95e413c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95e406ec60 0x7f95e413c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 -- 172.21.15.82:0/4136636486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40b42a0 con 0x7f95e4144ca0 2024-04-25T10:11:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e8e7d640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0018a0 tx=0x7f95dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc012400 con 0x7f95e4144ca0 2024-04-25T10:11:31.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc0125c0 con 0x7f95e4144ca0 2024-04-25T10:11:31.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e40b3ce0 con 0x7f95e4144ca0 2024-04-25T10:11:31.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0129b0 con 0x7f95e4144ca0 2024-04-25T10:11:31.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.673+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e414d840 con 0x7f95e4144ca0 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e404efa0 con 0x7f95e4144ca0 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95dc012b50 con 0x7f95e4144ca0 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95e0ff9640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95dc085a20 con 0x7f95e4144ca0 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95e3fff640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.677+0000 7f95e3fff640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 0x7f95d0069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f70 tx=0x7f95d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.681+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95dc0568a0 con 0x7f95e4144ca0 2024-04-25T10:11:31.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.841+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95e40b1790 con 0x7f95e4144ca0 2024-04-25T10:11:31.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.841+0000 7f95e0ff9640 1 -- 172.21.15.82:0/4136636486 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f95dc0561a0 con 0x7f95e4144ca0 2024-04-25T10:11:31.846 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:31.846 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":107,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:30.162220+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 msgr2=0x7f95d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 0x7f95d0069450 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f95d4000f70 tx=0x7f95d4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 msgr2=0x7f95e41537c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0018a0 tx=0x7f95dc00a430 comp rx=0 tx=0).stop 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 shutdown_connections 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f95d0066f90 0x7f95d0069450 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f95e40ce510 0x7f95e4153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f95e4144ca0 0x7f95e41537c0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 --2- 172.21.15.82:0/4136636486 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f95e406ec60 0x7f95e413c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 >> 172.21.15.82:0/4136636486 conn(0x7f95e40b11a0 msgr2=0x7f95e41443a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 shutdown_connections 2024-04-25T10:11:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.845+0000 7f95ea8e0640 1 -- 172.21.15.82:0/4136636486 wait complete. 2024-04-25T10:11:31.865 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:31.865 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:31.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.977+0000 7f9a43bfc640 1 Processor -- start 2024-04-25T10:11:31.982 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.977+0000 7f9a43bfc640 1 -- start start 2024-04-25T10:11:31.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a43bfc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3c15da10 0x7f9a3c071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a43bfc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 0x7f9a3c071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a43bfc640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a3c0adbd0 con 0x7f9a3c071e90 2024-04-25T10:11:31.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a43bfc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3c0ae860 con 0x7f9a3c15da10 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a43bfc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3c15e210 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3c15da10 0x7f9a3c071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a3c15da10 0x7f9a3c071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45770/0 (socket says 172.21.15.82:45770) 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 -- 172.21.15.82:0/1803832716 learned_addr learned my addr 172.21.15.82:0/1803832716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41197640 1 --2- 172.21.15.82:0/1803832716 >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 0x7f9a3c071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 -- 172.21.15.82:0/1803832716 >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 msgr2=0x7f9a3c071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 --2- 172.21.15.82:0/1803832716 >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 0x7f9a3c071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 -- 172.21.15.82:0/1803832716 >> v1:172.21.15.196:6789/0 conn(0x7f9a3c071e90 legacy=0x7f9a3c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 -- 172.21.15.82:0/1803832716 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c15e4f0 con 0x7f9a3c15da10 2024-04-25T10:11:31.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41197640 1 --2- 172.21.15.82:0/1803832716 >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 0x7f9a3c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a41998640 1 --2- 172.21.15.82:0/1803832716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c15da10 0x7f9a3c071030 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c04f7f0 tx=0x7f9a3801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e423cf354565c078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.985 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a40996640 1 -- 172.21.15.82:0/1803832716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380b0070 con 0x7f9a3c15da10 2024-04-25T10:11:31.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a40996640 1 -- 172.21.15.82:0/1803832716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a380ab600 con 0x7f9a3c15da10 2024-04-25T10:11:31.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.981+0000 7f9a40996640 1 -- 172.21.15.82:0/1803832716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380a4e60 con 0x7f9a3c15da10 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/1803832716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c15da10 msgr2=0x7f9a3c071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/1803832716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c15da10 0x7f9a3c071030 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c04f7f0 tx=0x7f9a3801c3a0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/1803832716 shutdown_connections 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/1803832716 >> v2:172.21.15.196:3300/0 conn(0x7f9a3c0b24a0 0x7f9a3c071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/1803832716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c15da10 0x7f9a3c071030 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/1803832716 >> 172.21.15.82:0/1803832716 conn(0x7f9a3c0b11a0 msgr2=0x7f9a3c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/1803832716 shutdown_connections 2024-04-25T10:11:31.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/1803832716 wait complete. 2024-04-25T10:11:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 Processor -- start 2024-04-25T10:11:31.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- start start 2024-04-25T10:11:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a3c071e90 0x7f9a3c168120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a3c15da10 0x7f9a3c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c16a890 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c15ef00 con 0x7f9a3c15da10 2024-04-25T10:11:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c16aa10 con 0x7f9a3c071e90 2024-04-25T10:11:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45772/0 (socket says 172.21.15.82:45772) 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 -- 172.21.15.82:0/2775205805 learned_addr learned my addr 172.21.15.82:0/2775205805 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41998640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a3c071e90 0x7f9a3c168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a42199640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a3c15da10 0x7f9a3c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 -- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a3c071e90 msgr2=0x7f9a3c168120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a3c071e90 0x7f9a3c168120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 -- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a3c15da10 msgr2=0x7f9a3c159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a3c15da10 0x7f9a3c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 -- 172.21.15.82:0/2775205805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3c0b2d10 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a41197640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c7f0 tx=0x7f9a2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c015070 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2c0077c0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c0b2ff0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2c007bb0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c0b35b0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.985+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c04efa0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.989+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a2c018070 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.989+0000 7f9a327fc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 0x7f9a20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.989+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a2c0852c0 con 0x7f9a3c0b24a0 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.989+0000 7f9a41998640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 0x7f9a20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.989+0000 7f9a41998640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 0x7f9a20069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9a380ad000 tx=0x7f9a38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:31.993+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a2c056140 con 0x7f9a3c0b24a0 2024-04-25T10:11:32.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.145+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a3c0b1dc0 con 0x7f9a20066f60 2024-04-25T10:11:32.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.165+0000 7f9a327fc640 1 -- 172.21.15.82:0/2775205805 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252418 (secure 0 0 0) 0x7f9a3c0b1dc0 con 0x7f9a20066f60 2024-04-25T10:11:32.170 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:32.171 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 msgr2=0x7f9a20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 0x7f9a20069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f9a380ad000 tx=0x7f9a38008040 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 msgr2=0x7f9a3c142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c00c7f0 tx=0x7f9a2c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 shutdown_connections 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a20066f60 0x7f9a20069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a3c15da10 0x7f9a3c159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a3c0b24a0 0x7f9a3c142cf0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 --2- 172.21.15.82:0/2775205805 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a3c071e90 0x7f9a3c168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 >> 172.21.15.82:0/2775205805 conn(0x7f9a3c0b11a0 msgr2=0x7f9a3c15f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 shutdown_connections 2024-04-25T10:11:32.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.169+0000 7f9a43bfc640 1 -- 172.21.15.82:0/2775205805 wait complete. 2024-04-25T10:11:32.188 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-25T10:11:31.442397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260956,"kb_used_data":11468,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713764,"statfs":{"total":773094113280,"available":772826894336,"internally_reserved":0,"allocated":11743232,"data_stored":6060460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293802"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.212706+0000","last_change":"2024-04-25T10:11:29.211228+0000","last_active":"2024-04-25T10:11:29.212706+0000","last_peered":"2024-04-25T10:11:29.212706+0000","last_clean":"2024-04-25T10:11:29.212706+0000","last_became_active":"2024-04-25T10:11:29.210953+0000","last_became_peered":"2024-04-25T10:11:29.210953+0000","last_unstale":"2024-04-25T10:11:29.212706+0000","last_undegraded":"2024-04-25T10:11:29.212706+0000","last_fullsized":"2024-04-25T10:11:29.212706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:22.551667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186970+0000","last_change":"2024-04-25T10:11:29.183016+0000","last_active":"2024-04-25T10:11:29.186970+0000","last_peered":"2024-04-25T10:11:29.186970+0000","last_clean":"2024-04-25T10:11:29.186970+0000","last_became_active":"2024-04-25T10:11:29.182186+0000","last_became_peered":"2024-04-25T10:11:29.182186+0000","last_unstale":"2024-04-25T10:11:29.186970+0000","last_undegraded":"2024-04-25T10:11:29.186970+0000","last_fullsized":"2024-04-25T10:11:29.186970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.234209+0000","last_change":"2024-04-25T10:11:29.233829+0000","last_active":"2024-04-25T10:11:29.234209+0000","last_peered":"2024-04-25T10:11:29.234209+0000","last_clean":"2024-04-25T10:11:29.234209+0000","last_became_active":"2024-04-25T10:11:29.233584+0000","last_became_peered":"2024-04-25T10:11:29.233584+0000","last_unstale":"2024-04-25T10:11:29.234209+0000","last_undegraded":"2024-04-25T10:11:29.234209+0000","last_fullsized":"2024-04-25T10:11:29.234209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:13.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.199812+0000","last_change":"2024-04-25T10:11:29.177949+0000","last_active":"2024-04-25T10:11:29.199812+0000","last_peered":"2024-04-25T10:11:29.199812+0000","last_clean":"2024-04-25T10:11:29.199812+0000","last_became_active":"2024-04-25T10:11:29.177211+0000","last_became_peered":"2024-04-25T10:11:29.177211+0000","last_unstale":"2024-04-25T10:11:29.199812+0000","last_undegraded":"2024-04-25T10:11:29.199812+0000","last_fullsized":"2024-04-25T10:11:29.199812+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208412+0000","last_change":"2024-04-25T10:11:29.204715+0000","last_active":"2024-04-25T10:11:29.208412+0000","last_peered":"2024-04-25T10:11:29.208412+0000","last_clean":"2024-04-25T10:11:29.208412+0000","last_became_active":"2024-04-25T10:11:29.204082+0000","last_became_peered":"2024-04-25T10:11:29.204082+0000","last_unstale":"2024-04-25T10:11:29.208412+0000","last_undegraded":"2024-04-25T10:11:29.208412+0000","last_fullsized":"2024-04-25T10:11:29.208412+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.303246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223619+0000","last_change":"2024-04-25T10:11:29.216807+0000","last_active":"2024-04-25T10:11:29.223619+0000","last_peered":"2024-04-25T10:11:29.223619+0000","last_clean":"2024-04-25T10:11:29.223619+0000","last_became_active":"2024-04-25T10:11:29.216165+0000","last_became_peered":"2024-04-25T10:11:29.216165+0000","last_unstale":"2024-04-25T10:11:29.223619+0000","last_undegraded":"2024-04-25T10:11:29.223619+0000","last_fullsized":"2024-04-25T10:11:29.223619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223309+0000","last_change":"2024-04-25T10:11:29.213933+0000","last_active":"2024-04-25T10:11:29.223309+0000","last_peered":"2024-04-25T10:11:29.223309+0000","last_clean":"2024-04-25T10:11:29.223309+0000","last_became_active":"2024-04-25T10:11:29.209929+0000","last_became_peered":"2024-04-25T10:11:29.209929+0000","last_unstale":"2024-04-25T10:11:29.223309+0000","last_undegraded":"2024-04-25T10:11:29.223309+0000","last_fullsized":"2024-04-25T10:11:29.223309+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211216+0000","last_change":"2024-04-25T10:11:29.208531+0000","last_active":"2024-04-25T10:11:29.211216+0000","last_peered":"2024-04-25T10:11:29.211216+0000","last_clean":"2024-04-25T10:11:29.211216+0000","last_became_active":"2024-04-25T10:11:29.206270+0000","last_became_peered":"2024-04-25T10:11:29.206270+0000","last_unstale":"2024-04-25T10:11:29.211216+0000","last_undegraded":"2024-04-25T10:11:29.211216+0000","last_fullsized":"2024-04-25T10:11:29.211216+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:17.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186990+0000","last_change":"2024-04-25T10:11:29.175092+0000","last_active":"2024-04-25T10:11:29.186990+0000","last_peered":"2024-04-25T10:11:29.186990+0000","last_clean":"2024-04-25T10:11:29.186990+0000","last_became_active":"2024-04-25T10:11:29.174377+0000","last_became_peered":"2024-04-25T10:11:29.174377+0000","last_unstale":"2024-04-25T10:11:29.186990+0000","last_undegraded":"2024-04-25T10:11:29.186990+0000","last_fullsized":"2024-04-25T10:11:29.186990+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:45.215104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224930+0000","last_change":"2024-04-25T10:11:29.224395+0000","last_active":"2024-04-25T10:11:29.224930+0000","last_peered":"2024-04-25T10:11:29.224930+0000","last_clean":"2024-04-25T10:11:29.224930+0000","last_became_active":"2024-04-25T10:11:29.224098+0000","last_became_peered":"2024-04-25T10:11:29.224098+0000","last_unstale":"2024-04-25T10:11:29.224930+0000","last_undegraded":"2024-04-25T10:11:29.224930+0000","last_fullsized":"2024-04-25T10:11:29.224930+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237947+0000","last_change":"2024-04-25T10:11:29.233407+0000","last_active":"2024-04-25T10:11:29.237947+0000","last_peered":"2024-04-25T10:11:29.237947+0000","last_clean":"2024-04-25T10:11:29.237947+0000","last_became_active":"2024-04-25T10:11:29.232754+0000","last_became_peered":"2024-04-25T10:11:29.232754+0000","last_unstale":"2024-04-25T10:11:29.237947+0000","last_undegraded":"2024-04-25T10:11:29.237947+0000","last_fullsized":"2024-04-25T10:11:29.237947+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.215818+0000","last_change":"2024-04-25T10:11:29.203811+0000","last_active":"2024-04-25T10:11:29.215818+0000","last_peered":"2024-04-25T10:11:29.215818+0000","last_clean":"2024-04-25T10:11:29.215818+0000","last_became_active":"2024-04-25T10:11:29.203093+0000","last_became_peered":"2024-04-25T10:11:29.203093+0000","last_unstale":"2024-04-25T10:11:29.215818+0000","last_undegraded":"2024-04-25T10:11:29.215818+0000","last_fullsized":"2024-04-25T10:11:29.215818+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.227459+0000","last_change":"2024-04-25T10:11:29.226590+0000","last_active":"2024-04-25T10:11:29.227459+0000","last_peered":"2024-04-25T10:11:29.227459+0000","last_clean":"2024-04-25T10:11:29.227459+0000","last_became_active":"2024-04-25T10:11:29.226339+0000","last_became_peered":"2024-04-25T10:11:29.226339+0000","last_unstale":"2024-04-25T10:11:29.227459+0000","last_undegraded":"2024-04-25T10:11:29.227459+0000","last_fullsized":"2024-04-25T10:11:29.227459+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.196268+0000","last_change":"2024-04-25T10:11:29.192851+0000","last_active":"2024-04-25T10:11:29.196268+0000","last_peered":"2024-04-25T10:11:29.196268+0000","last_clean":"2024-04-25T10:11:29.196268+0000","last_became_active":"2024-04-25T10:11:29.192138+0000","last_became_peered":"2024-04-25T10:11:29.192138+0000","last_unstale":"2024-04-25T10:11:29.196268+0000","last_undegraded":"2024-04-25T10:11:29.196268+0000","last_fullsized":"2024-04-25T10:11:29.196268+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.230792+0000","last_change":"2024-04-25T10:11:29.229197+0000","last_active":"2024-04-25T10:11:29.230792+0000","last_peered":"2024-04-25T10:11:29.230792+0000","last_clean":"2024-04-25T10:11:29.230792+0000","last_became_active":"2024-04-25T10:11:29.228514+0000","last_became_peered":"2024-04-25T10:11:29.228514+0000","last_unstale":"2024-04-25T10:11:29.230792+0000","last_undegraded":"2024-04-25T10:11:29.230792+0000","last_fullsized":"2024-04-25T10:11:29.230792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.205510+0000","last_change":"2024-04-25T10:11:29.198500+0000","last_active":"2024-04-25T10:11:29.205510+0000","last_peered":"2024-04-25T10:11:29.205510+0000","last_clean":"2024-04-25T10:11:29.205510+0000","last_became_active":"2024-04-25T10:11:29.197756+0000","last_became_peered":"2024-04-25T10:11:29.197756+0000","last_unstale":"2024-04-25T10:11:29.205510+0000","last_undegraded":"2024-04-25T10:11:29.205510+0000","last_fullsized":"2024-04-25T10:11:29.205510+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228567+0000","last_change":"2024-04-25T10:11:29.227891+0000","last_active":"2024-04-25T10:11:29.228567+0000","last_peered":"2024-04-25T10:11:29.228567+0000","last_clean":"2024-04-25T10:11:29.228567+0000","last_became_active":"2024-04-25T10:11:29.227557+0000","last_became_peered":"2024-04-25T10:11:29.227557+0000","last_unstale":"2024-04-25T10:11:29.228567+0000","last_undegraded":"2024-04-25T10:11:29.228567+0000","last_fullsized":"2024-04-25T10:11:29.228567+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:07.674828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223525+0000","last_change":"2024-04-25T10:11:29.216463+0000","last_active":"2024-04-25T10:11:29.223525+0000","last_peered":"2024-04-25T10:11:29.223525+0000","last_clean":"2024-04-25T10:11:29.223525+0000","last_became_active":"2024-04-25T10:11:29.214904+0000","last_became_peered":"2024-04-25T10:11:29.214904+0000","last_unstale":"2024-04-25T10:11:29.223525+0000","last_undegraded":"2024-04-25T10:11:29.223525+0000","last_fullsized":"2024-04-25T10:11:29.223525+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.221535+0000","last_change":"2024-04-25T10:11:29.216585+0000","last_active":"2024-04-25T10:11:29.221535+0000","last_peered":"2024-04-25T10:11:29.221535+0000","last_clean":"2024-04-25T10:11:29.221535+0000","last_became_active":"2024-04-25T10:11:29.216258+0000","last_became_peered":"2024-04-25T10:11:29.216258+0000","last_unstale":"2024-04-25T10:11:29.221535+0000","last_undegraded":"2024-04-25T10:11:29.221535+0000","last_fullsized":"2024-04-25T10:11:29.221535+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:15.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188724+0000","last_change":"2024-04-25T10:11:29.183504+0000","last_active":"2024-04-25T10:11:29.188724+0000","last_peered":"2024-04-25T10:11:29.188724+0000","last_clean":"2024-04-25T10:11:29.188724+0000","last_became_active":"2024-04-25T10:11:29.182421+0000","last_became_peered":"2024-04-25T10:11:29.182421+0000","last_unstale":"2024-04-25T10:11:29.188724+0000","last_undegraded":"2024-04-25T10:11:29.188724+0000","last_fullsized":"2024-04-25T10:11:29.188724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.303246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228834+0000","last_change":"2024-04-25T10:11:29.228369+0000","last_active":"2024-04-25T10:11:29.228834+0000","last_peered":"2024-04-25T10:11:29.228834+0000","last_clean":"2024-04-25T10:11:29.228834+0000","last_became_active":"2024-04-25T10:11:29.228046+0000","last_became_peered":"2024-04-25T10:11:29.228046+0000","last_unstale":"2024-04-25T10:11:29.228834+0000","last_undegraded":"2024-04-25T10:11:29.228834+0000","last_fullsized":"2024-04-25T10:11:29.228834+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:49.534581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237375+0000","last_change":"2024-04-25T10:11:29.237008+0000","last_active":"2024-04-25T10:11:29.237375+0000","last_peered":"2024-04-25T10:11:29.237375+0000","last_clean":"2024-04-25T10:11:29.237375+0000","last_became_active":"2024-04-25T10:11:29.236717+0000","last_became_peered":"2024-04-25T10:11:29.236717+0000","last_unstale":"2024-04-25T10:11:29.237375+0000","last_undegraded":"2024-04-25T10:11:29.237375+0000","last_fullsized":"2024-04-25T10:11:29.237375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.227603+0000","last_change":"2024-04-25T10:11:29.226659+0000","last_active":"2024-04-25T10:11:29.227603+0000","last_peered":"2024-04-25T10:11:29.227603+0000","last_clean":"2024-04-25T10:11:29.227603+0000","last_became_active":"2024-04-25T10:11:29.226400+0000","last_became_peered":"2024-04-25T10:11:29.226400+0000","last_unstale":"2024-04-25T10:11:29.227603+0000","last_undegraded":"2024-04-25T10:11:29.227603+0000","last_fullsized":"2024-04-25T10:11:29.227603+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:34.582899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208133+0000","last_change":"2024-04-25T10:11:29.202795+0000","last_active":"2024-04-25T10:11:29.208133+0000","last_peered":"2024-04-25T10:11:29.208133+0000","last_clean":"2024-04-25T10:11:29.208133+0000","last_became_active":"2024-04-25T10:11:29.202524+0000","last_became_peered":"2024-04-25T10:11:29.202524+0000","last_unstale":"2024-04-25T10:11:29.208133+0000","last_undegraded":"2024-04-25T10:11:29.208133+0000","last_fullsized":"2024-04-25T10:11:29.208133+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:45.215104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.234586+0000","last_change":"2024-04-25T10:11:29.227391+0000","last_active":"2024-04-25T10:11:29.234586+0000","last_peered":"2024-04-25T10:11:29.234586+0000","last_clean":"2024-04-25T10:11:29.234586+0000","last_became_active":"2024-04-25T10:11:29.226689+0000","last_became_peered":"2024-04-25T10:11:29.226689+0000","last_unstale":"2024-04-25T10:11:29.234586+0000","last_undegraded":"2024-04-25T10:11:29.234586+0000","last_fullsized":"2024-04-25T10:11:29.234586+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188425+0000","last_change":"2024-04-25T10:11:29.181851+0000","last_active":"2024-04-25T10:11:29.188425+0000","last_peered":"2024-04-25T10:11:29.188425+0000","last_clean":"2024-04-25T10:11:29.188425+0000","last_became_active":"2024-04-25T10:11:29.181116+0000","last_became_peered":"2024-04-25T10:11:29.181116+0000","last_unstale":"2024-04-25T10:11:29.188425+0000","last_undegraded":"2024-04-25T10:11:29.188425+0000","last_fullsized":"2024-04-25T10:11:29.188425+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237838+0000","last_change":"2024-04-25T10:11:29.232728+0000","last_active":"2024-04-25T10:11:29.237838+0000","last_peered":"2024-04-25T10:11:29.237838+0000","last_clean":"2024-04-25T10:11:29.237838+0000","last_became_active":"2024-04-25T10:11:29.225467+0000","last_became_peered":"2024-04-25T10:11:29.225467+0000","last_unstale":"2024-04-25T10:11:29.237838+0000","last_undegraded":"2024-04-25T10:11:29.237838+0000","last_fullsized":"2024-04-25T10:11:29.237838+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236013+0000","last_change":"2024-04-25T10:11:29.235222+0000","last_active":"2024-04-25T10:11:29.236013+0000","last_peered":"2024-04-25T10:11:29.236013+0000","last_clean":"2024-04-25T10:11:29.236013+0000","last_became_active":"2024-04-25T10:11:29.234694+0000","last_became_peered":"2024-04-25T10:11:29.234694+0000","last_unstale":"2024-04-25T10:11:29.236013+0000","last_undegraded":"2024-04-25T10:11:29.236013+0000","last_fullsized":"2024-04-25T10:11:29.236013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201541+0000","last_change":"2024-04-25T10:11:29.199218+0000","last_active":"2024-04-25T10:11:29.201541+0000","last_peered":"2024-04-25T10:11:29.201541+0000","last_clean":"2024-04-25T10:11:29.201541+0000","last_became_active":"2024-04-25T10:11:29.198875+0000","last_became_peered":"2024-04-25T10:11:29.198875+0000","last_unstale":"2024-04-25T10:11:29.201541+0000","last_undegraded":"2024-04-25T10:11:29.201541+0000","last_fullsized":"2024-04-25T10:11:29.201541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:21.102504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.226834+0000","last_change":"2024-04-25T10:11:29.219238+0000","last_active":"2024-04-25T10:11:29.226834+0000","last_peered":"2024-04-25T10:11:29.226834+0000","last_clean":"2024-04-25T10:11:29.226834+0000","last_became_active":"2024-04-25T10:11:29.218067+0000","last_became_peered":"2024-04-25T10:11:29.218067+0000","last_unstale":"2024-04-25T10:11:29.226834+0000","last_undegraded":"2024-04-25T10:11:29.226834+0000","last_fullsized":"2024-04-25T10:11:29.226834+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.221751+0000","last_change":"2024-04-25T10:11:29.201233+0000","last_active":"2024-04-25T10:11:29.221751+0000","last_peered":"2024-04-25T10:11:29.221751+0000","last_clean":"2024-04-25T10:11:29.221751+0000","last_became_active":"2024-04-25T10:11:29.199982+0000","last_became_peered":"2024-04-25T10:11:29.199982+0000","last_unstale":"2024-04-25T10:11:29.221751+0000","last_undegraded":"2024-04-25T10:11:29.221751+0000","last_fullsized":"2024-04-25T10:11:29.221751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211239+0000","last_change":"2024-04-25T10:11:29.209367+0000","last_active":"2024-04-25T10:11:29.211239+0000","last_peered":"2024-04-25T10:11:29.211239+0000","last_clean":"2024-04-25T10:11:29.211239+0000","last_became_active":"2024-04-25T10:11:29.209071+0000","last_became_peered":"2024-04-25T10:11:29.209071+0000","last_unstale":"2024-04-25T10:11:29.211239+0000","last_undegraded":"2024-04-25T10:11:29.211239+0000","last_fullsized":"2024-04-25T10:11:29.211239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:28.022053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201310+0000","last_change":"2024-04-25T10:11:29.200534+0000","last_active":"2024-04-25T10:11:29.201310+0000","last_peered":"2024-04-25T10:11:29.201310+0000","last_clean":"2024-04-25T10:11:29.201310+0000","last_became_active":"2024-04-25T10:11:29.200255+0000","last_became_peered":"2024-04-25T10:11:29.200255+0000","last_unstale":"2024-04-25T10:11:29.201310+0000","last_undegraded":"2024-04-25T10:11:29.201310+0000","last_fullsized":"2024-04-25T10:11:29.201310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:05.484998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223852+0000","last_change":"2024-04-25T10:11:28.205735+0000","last_active":"2024-04-25T10:11:29.223852+0000","last_peered":"2024-04-25T10:11:29.223852+0000","last_clean":"2024-04-25T10:11:29.223852+0000","last_became_active":"2024-04-25T10:11:28.205108+0000","last_became_peered":"2024-04-25T10:11:28.205108+0000","last_unstale":"2024-04-25T10:11:29.223852+0000","last_undegraded":"2024-04-25T10:11:29.223852+0000","last_fullsized":"2024-04-25T10:11:29.223852+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:50.630210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211323+0000","last_change":"2024-04-25T10:11:29.209380+0000","last_active":"2024-04-25T10:11:29.211323+0000","last_peered":"2024-04-25T10:11:29.211323+0000","last_clean":"2024-04-25T10:11:29.211323+0000","last_became_active":"2024-04-25T10:11:29.209110+0000","last_became_peered":"2024-04-25T10:11:29.209110+0000","last_unstale":"2024-04-25T10:11:29.211323+0000","last_undegraded":"2024-04-25T10:11:29.211323+0000","last_fullsized":"2024-04-25T10:11:29.211323+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:00.183288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200184+0000","last_change":"2024-04-25T10:11:28.197059+0000","last_active":"2024-04-25T10:11:29.200184+0000","last_peered":"2024-04-25T10:11:29.200184+0000","last_clean":"2024-04-25T10:11:29.200184+0000","last_became_active":"2024-04-25T10:11:28.196477+0000","last_became_peered":"2024-04-25T10:11:28.196477+0000","last_unstale":"2024-04-25T10:11:29.200184+0000","last_undegraded":"2024-04-25T10:11:29.200184+0000","last_fullsized":"2024-04-25T10:11:29.200184+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:52.309299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224680+0000","last_change":"2024-04-25T10:11:29.214836+0000","last_active":"2024-04-25T10:11:29.224680+0000","last_peered":"2024-04-25T10:11:29.224680+0000","last_clean":"2024-04-25T10:11:29.224680+0000","last_became_active":"2024-04-25T10:11:29.214038+0000","last_became_peered":"2024-04-25T10:11:29.214038+0000","last_unstale":"2024-04-25T10:11:29.224680+0000","last_undegraded":"2024-04-25T10:11:29.224680+0000","last_fullsized":"2024-04-25T10:11:29.224680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236769+0000","last_change":"2024-04-25T10:11:28.193383+0000","last_active":"2024-04-25T10:11:29.236769+0000","last_peered":"2024-04-25T10:11:29.236769+0000","last_clean":"2024-04-25T10:11:29.236769+0000","last_became_active":"2024-04-25T10:11:28.192595+0000","last_became_peered":"2024-04-25T10:11:28.192595+0000","last_unstale":"2024-04-25T10:11:29.236769+0000","last_undegraded":"2024-04-25T10:11:29.236769+0000","last_fullsized":"2024-04-25T10:11:29.236769+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236211+0000","last_change":"2024-04-25T10:11:29.208535+0000","last_active":"2024-04-25T10:11:29.236211+0000","last_peered":"2024-04-25T10:11:29.236211+0000","last_clean":"2024-04-25T10:11:29.236211+0000","last_became_active":"2024-04-25T10:11:29.207590+0000","last_became_peered":"2024-04-25T10:11:29.207590+0000","last_unstale":"2024-04-25T10:11:29.236211+0000","last_undegraded":"2024-04-25T10:11:29.236211+0000","last_fullsized":"2024-04-25T10:11:29.236211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225296+0000","last_change":"2024-04-25T10:11:28.184824+0000","last_active":"2024-04-25T10:11:29.225296+0000","last_peered":"2024-04-25T10:11:29.225296+0000","last_clean":"2024-04-25T10:11:29.225296+0000","last_became_active":"2024-04-25T10:11:28.184372+0000","last_became_peered":"2024-04-25T10:11:28.184372+0000","last_unstale":"2024-04-25T10:11:29.225296+0000","last_undegraded":"2024-04-25T10:11:29.225296+0000","last_fullsized":"2024-04-25T10:11:29.225296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:22.818301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.210588+0000","last_change":"2024-04-25T10:11:29.204815+0000","last_active":"2024-04-25T10:11:29.210588+0000","last_peered":"2024-04-25T10:11:29.210588+0000","last_clean":"2024-04-25T10:11:29.210588+0000","last_became_active":"2024-04-25T10:11:29.203930+0000","last_became_peered":"2024-04-25T10:11:29.203930+0000","last_unstale":"2024-04-25T10:11:29.210588+0000","last_undegraded":"2024-04-25T10:11:29.210588+0000","last_fullsized":"2024-04-25T10:11:29.210588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224059+0000","last_change":"2024-04-25T10:11:28.208539+0000","last_active":"2024-04-25T10:11:29.224059+0000","last_peered":"2024-04-25T10:11:29.224059+0000","last_clean":"2024-04-25T10:11:29.224059+0000","last_became_active":"2024-04-25T10:11:28.207800+0000","last_became_peered":"2024-04-25T10:11:28.207800+0000","last_unstale":"2024-04-25T10:11:29.224059+0000","last_undegraded":"2024-04-25T10:11:29.224059+0000","last_fullsized":"2024-04-25T10:11:29.224059+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:50.630210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225751+0000","last_change":"2024-04-25T10:11:29.225244+0000","last_active":"2024-04-25T10:11:29.225751+0000","last_peered":"2024-04-25T10:11:29.225751+0000","last_clean":"2024-04-25T10:11:29.225751+0000","last_became_active":"2024-04-25T10:11:29.224875+0000","last_became_peered":"2024-04-25T10:11:29.224875+0000","last_unstale":"2024-04-25T10:11:29.225751+0000","last_undegraded":"2024-04-25T10:11:29.225751+0000","last_fullsized":"2024-04-25T10:11:29.225751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:44.182376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236258+0000","last_change":"2024-04-25T10:11:28.200072+0000","last_active":"2024-04-25T10:11:29.236258+0000","last_peered":"2024-04-25T10:11:29.236258+0000","last_clean":"2024-04-25T10:11:29.236258+0000","last_became_active":"2024-04-25T10:11:28.199315+0000","last_became_peered":"2024-04-25T10:11:28.199315+0000","last_unstale":"2024-04-25T10:11:29.236258+0000","last_undegraded":"2024-04-25T10:11:29.236258+0000","last_fullsized":"2024-04-25T10:11:29.236258+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.391764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223873+0000","last_change":"2024-04-25T10:11:29.221541+0000","last_active":"2024-04-25T10:11:29.223873+0000","last_peered":"2024-04-25T10:11:29.223873+0000","last_clean":"2024-04-25T10:11:29.223873+0000","last_became_active":"2024-04-25T10:11:29.215809+0000","last_became_peered":"2024-04-25T10:11:29.215809+0000","last_unstale":"2024-04-25T10:11:29.223873+0000","last_undegraded":"2024-04-25T10:11:29.223873+0000","last_fullsized":"2024-04-25T10:11:29.223873+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.170839+0000","last_change":"2024-04-25T10:11:28.193710+0000","last_active":"2024-04-25T10:11:29.170839+0000","last_peered":"2024-04-25T10:11:29.170839+0000","last_clean":"2024-04-25T10:11:29.170839+0000","last_became_active":"2024-04-25T10:11:28.192936+0000","last_became_peered":"2024-04-25T10:11:28.192936+0000","last_unstale":"2024-04-25T10:11:29.170839+0000","last_undegraded":"2024-04-25T10:11:29.170839+0000","last_fullsized":"2024-04-25T10:11:29.170839+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:22.818301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.210656+0000","last_change":"2024-04-25T10:11:29.206953+0000","last_active":"2024-04-25T10:11:29.210656+0000","last_peered":"2024-04-25T10:11:29.210656+0000","last_clean":"2024-04-25T10:11:29.210656+0000","last_became_active":"2024-04-25T10:11:29.200887+0000","last_became_peered":"2024-04-25T10:11:29.200887+0000","last_unstale":"2024-04-25T10:11:29.210656+0000","last_undegraded":"2024-04-25T10:11:29.210656+0000","last_fullsized":"2024-04-25T10:11:29.210656+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.168869+0000","last_change":"2024-04-25T10:11:28.173183+0000","last_active":"2024-04-25T10:11:29.168869+0000","last_peered":"2024-04-25T10:11:29.168869+0000","last_clean":"2024-04-25T10:11:29.168869+0000","last_became_active":"2024-04-25T10:11:28.172876+0000","last_became_peered":"2024-04-25T10:11:28.172876+0000","last_unstale":"2024-04-25T10:11:29.168869+0000","last_undegraded":"2024-04-25T10:11:29.168869+0000","last_fullsized":"2024-04-25T10:11:29.168869+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:10.677015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213342+0000","last_change":"2024-04-25T10:11:29.212314+0000","last_active":"2024-04-25T10:11:29.213342+0000","last_peered":"2024-04-25T10:11:29.213342+0000","last_clean":"2024-04-25T10:11:29.213342+0000","last_became_active":"2024-04-25T10:11:29.211949+0000","last_became_peered":"2024-04-25T10:11:29.211949+0000","last_unstale":"2024-04-25T10:11:29.213342+0000","last_undegraded":"2024-04-25T10:11:29.213342+0000","last_fullsized":"2024-04-25T10:11:29.213342+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:51.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.171605+0000","last_change":"2024-04-25T10:11:28.193547+0000","last_active":"2024-04-25T10:11:29.171605+0000","last_peered":"2024-04-25T10:11:29.171605+0000","last_clean":"2024-04-25T10:11:29.171605+0000","last_became_active":"2024-04-25T10:11:28.192759+0000","last_became_peered":"2024-04-25T10:11:28.192759+0000","last_unstale":"2024-04-25T10:11:29.171605+0000","last_undegraded":"2024-04-25T10:11:29.171605+0000","last_fullsized":"2024-04-25T10:11:29.171605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:44.204407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211080+0000","last_change":"2024-04-25T10:11:29.206356+0000","last_active":"2024-04-25T10:11:29.211080+0000","last_peered":"2024-04-25T10:11:29.211080+0000","last_clean":"2024-04-25T10:11:29.211080+0000","last_became_active":"2024-04-25T10:11:29.204920+0000","last_became_peered":"2024-04-25T10:11:29.204920+0000","last_unstale":"2024-04-25T10:11:29.211080+0000","last_undegraded":"2024-04-25T10:11:29.211080+0000","last_fullsized":"2024-04-25T10:11:29.211080+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.184424+0000","last_change":"2024-04-25T10:11:28.183204+0000","last_active":"2024-04-25T10:11:29.184424+0000","last_peered":"2024-04-25T10:11:29.184424+0000","last_clean":"2024-04-25T10:11:29.184424+0000","last_became_active":"2024-04-25T10:11:28.179389+0000","last_became_peered":"2024-04-25T10:11:28.179389+0000","last_unstale":"2024-04-25T10:11:29.184424+0000","last_undegraded":"2024-04-25T10:11:29.184424+0000","last_fullsized":"2024-04-25T10:11:29.184424+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:08.793758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211190+0000","last_change":"2024-04-25T10:11:29.210116+0000","last_active":"2024-04-25T10:11:29.211190+0000","last_peered":"2024-04-25T10:11:29.211190+0000","last_clean":"2024-04-25T10:11:29.211190+0000","last_became_active":"2024-04-25T10:11:29.209843+0000","last_became_peered":"2024-04-25T10:11:29.209843+0000","last_unstale":"2024-04-25T10:11:29.211190+0000","last_undegraded":"2024-04-25T10:11:29.211190+0000","last_fullsized":"2024-04-25T10:11:29.211190+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.577147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.212964+0000","last_change":"2024-04-25T10:11:28.197173+0000","last_active":"2024-04-25T10:11:29.212964+0000","last_peered":"2024-04-25T10:11:29.212964+0000","last_clean":"2024-04-25T10:11:29.212964+0000","last_became_active":"2024-04-25T10:11:28.196478+0000","last_became_peered":"2024-04-25T10:11:28.196478+0000","last_unstale":"2024-04-25T10:11:29.212964+0000","last_undegraded":"2024-04-25T10:11:29.212964+0000","last_fullsized":"2024-04-25T10:11:29.212964+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208439+0000","last_change":"2024-04-25T10:11:29.204926+0000","last_active":"2024-04-25T10:11:29.208439+0000","last_peered":"2024-04-25T10:11:29.208439+0000","last_clean":"2024-04-25T10:11:29.208439+0000","last_became_active":"2024-04-25T10:11:29.204456+0000","last_became_peered":"2024-04-25T10:11:29.204456+0000","last_unstale":"2024-04-25T10:11:29.208439+0000","last_undegraded":"2024-04-25T10:11:29.208439+0000","last_fullsized":"2024-04-25T10:11:29.208439+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228827+0000","last_change":"2024-04-25T10:11:28.203606+0000","last_active":"2024-04-25T10:11:29.228827+0000","last_peered":"2024-04-25T10:11:29.228827+0000","last_clean":"2024-04-25T10:11:29.228827+0000","last_became_active":"2024-04-25T10:11:28.202865+0000","last_became_peered":"2024-04-25T10:11:28.202865+0000","last_unstale":"2024-04-25T10:11:29.228827+0000","last_undegraded":"2024-04-25T10:11:29.228827+0000","last_fullsized":"2024-04-25T10:11:29.228827+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:48.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237732+0000","last_change":"2024-04-25T10:11:29.219949+0000","last_active":"2024-04-25T10:11:29.237732+0000","last_peered":"2024-04-25T10:11:29.237732+0000","last_clean":"2024-04-25T10:11:29.237732+0000","last_became_active":"2024-04-25T10:11:29.217284+0000","last_became_peered":"2024-04-25T10:11:29.217284+0000","last_unstale":"2024-04-25T10:11:29.237732+0000","last_undegraded":"2024-04-25T10:11:29.237732+0000","last_fullsized":"2024-04-25T10:11:29.237732+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211416+0000","last_change":"2024-04-25T10:11:28.178250+0000","last_active":"2024-04-25T10:11:29.211416+0000","last_peered":"2024-04-25T10:11:29.211416+0000","last_clean":"2024-04-25T10:11:29.211416+0000","last_became_active":"2024-04-25T10:11:28.177870+0000","last_became_peered":"2024-04-25T10:11:28.177870+0000","last_unstale":"2024-04-25T10:11:29.211416+0000","last_undegraded":"2024-04-25T10:11:29.211416+0000","last_fullsized":"2024-04-25T10:11:29.211416+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223359+0000","last_change":"2024-04-25T10:11:29.214400+0000","last_active":"2024-04-25T10:11:29.223359+0000","last_peered":"2024-04-25T10:11:29.223359+0000","last_clean":"2024-04-25T10:11:29.223359+0000","last_became_active":"2024-04-25T10:11:29.210408+0000","last_became_peered":"2024-04-25T10:11:29.210408+0000","last_unstale":"2024-04-25T10:11:29.223359+0000","last_undegraded":"2024-04-25T10:11:29.223359+0000","last_fullsized":"2024-04-25T10:11:29.223359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.169889+0000","last_change":"2024-04-25T10:11:28.183819+0000","last_active":"2024-04-25T10:11:29.169889+0000","last_peered":"2024-04-25T10:11:29.169889+0000","last_clean":"2024-04-25T10:11:29.169889+0000","last_became_active":"2024-04-25T10:11:28.183495+0000","last_became_peered":"2024-04-25T10:11:28.183495+0000","last_unstale":"2024-04-25T10:11:29.169889+0000","last_undegraded":"2024-04-25T10:11:29.169889+0000","last_fullsized":"2024-04-25T10:11:29.169889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.391764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200257+0000","last_change":"2024-04-25T10:11:29.196681+0000","last_active":"2024-04-25T10:11:29.200257+0000","last_peered":"2024-04-25T10:11:29.200257+0000","last_clean":"2024-04-25T10:11:29.200257+0000","last_became_active":"2024-04-25T10:11:29.195959+0000","last_became_peered":"2024-04-25T10:11:29.195959+0000","last_unstale":"2024-04-25T10:11:29.200257+0000","last_undegraded":"2024-04-25T10:11:29.200257+0000","last_fullsized":"2024-04-25T10:11:29.200257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:26.550654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186660+0000","last_change":"2024-04-25T10:11:28.191807+0000","last_active":"2024-04-25T10:11:29.186660+0000","last_peered":"2024-04-25T10:11:29.186660+0000","last_clean":"2024-04-25T10:11:29.186660+0000","last_became_active":"2024-04-25T10:11:28.191530+0000","last_became_peered":"2024-04-25T10:11:28.191530+0000","last_unstale":"2024-04-25T10:11:29.186660+0000","last_undegraded":"2024-04-25T10:11:29.186660+0000","last_fullsized":"2024-04-25T10:11:29.186660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:39.965647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228546+0000","last_change":"2024-04-25T10:11:29.227894+0000","last_active":"2024-04-25T10:11:29.228546+0000","last_peered":"2024-04-25T10:11:29.228546+0000","last_clean":"2024-04-25T10:11:29.228546+0000","last_became_active":"2024-04-25T10:11:29.227607+0000","last_became_peered":"2024-04-25T10:11:29.227607+0000","last_unstale":"2024-04-25T10:11:29.228546+0000","last_undegraded":"2024-04-25T10:11:29.228546+0000","last_fullsized":"2024-04-25T10:11:29.228546+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.819178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.238149+0000","last_change":"2024-04-25T10:11:28.193856+0000","last_active":"2024-04-25T10:11:29.238149+0000","last_peered":"2024-04-25T10:11:29.238149+0000","last_clean":"2024-04-25T10:11:29.238149+0000","last_became_active":"2024-04-25T10:11:28.193056+0000","last_became_peered":"2024-04-25T10:11:28.193056+0000","last_unstale":"2024-04-25T10:11:29.238149+0000","last_undegraded":"2024-04-25T10:11:29.238149+0000","last_fullsized":"2024-04-25T10:11:29.238149+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:45.659307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200035+0000","last_change":"2024-04-25T10:11:29.195661+0000","last_active":"2024-04-25T10:11:29.200035+0000","last_peered":"2024-04-25T10:11:29.200035+0000","last_clean":"2024-04-25T10:11:29.200035+0000","last_became_active":"2024-04-25T10:11:29.195325+0000","last_became_peered":"2024-04-25T10:11:29.195325+0000","last_unstale":"2024-04-25T10:11:29.200035+0000","last_undegraded":"2024-04-25T10:11:29.200035+0000","last_fullsized":"2024-04-25T10:11:29.200035+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:24.723171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.216556+0000","last_change":"2024-04-25T10:11:28.202108+0000","last_active":"2024-04-25T10:11:29.216556+0000","last_peered":"2024-04-25T10:11:29.216556+0000","last_clean":"2024-04-25T10:11:29.216556+0000","last_became_active":"2024-04-25T10:11:28.188447+0000","last_became_peered":"2024-04-25T10:11:28.188447+0000","last_unstale":"2024-04-25T10:11:29.216556+0000","last_undegraded":"2024-04-25T10:11:29.216556+0000","last_fullsized":"2024-04-25T10:11:29.216556+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:48.768488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.209861+0000","last_change":"2024-04-25T10:11:29.200625+0000","last_active":"2024-04-25T10:11:29.209861+0000","last_peered":"2024-04-25T10:11:29.209861+0000","last_clean":"2024-04-25T10:11:29.209861+0000","last_became_active":"2024-04-25T10:11:29.199810+0000","last_became_peered":"2024-04-25T10:11:29.199810+0000","last_unstale":"2024-04-25T10:11:29.209861+0000","last_undegraded":"2024-04-25T10:11:29.209861+0000","last_fullsized":"2024-04-25T10:11:29.209861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225243+0000","last_change":"2024-04-25T10:11:28.196477+0000","last_active":"2024-04-25T10:11:29.225243+0000","last_peered":"2024-04-25T10:11:29.225243+0000","last_clean":"2024-04-25T10:11:29.225243+0000","last_became_active":"2024-04-25T10:11:28.195761+0000","last_became_peered":"2024-04-25T10:11:28.195761+0000","last_unstale":"2024-04-25T10:11:29.225243+0000","last_undegraded":"2024-04-25T10:11:29.225243+0000","last_fullsized":"2024-04-25T10:11:29.225243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:27.172729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208208+0000","last_change":"2024-04-25T10:11:29.203753+0000","last_active":"2024-04-25T10:11:29.208208+0000","last_peered":"2024-04-25T10:11:29.208208+0000","last_clean":"2024-04-25T10:11:29.208208+0000","last_became_active":"2024-04-25T10:11:29.202435+0000","last_became_peered":"2024-04-25T10:11:29.202435+0000","last_unstale":"2024-04-25T10:11:29.208208+0000","last_undegraded":"2024-04-25T10:11:29.208208+0000","last_fullsized":"2024-04-25T10:11:29.208208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.226518+0000","last_change":"2024-04-25T10:11:28.210024+0000","last_active":"2024-04-25T10:11:29.226518+0000","last_peered":"2024-04-25T10:11:29.226518+0000","last_clean":"2024-04-25T10:11:29.226518+0000","last_became_active":"2024-04-25T10:11:28.209341+0000","last_became_peered":"2024-04-25T10:11:28.209341+0000","last_unstale":"2024-04-25T10:11:29.226518+0000","last_undegraded":"2024-04-25T10:11:29.226518+0000","last_fullsized":"2024-04-25T10:11:29.226518+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:27.172729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.206257+0000","last_change":"2024-04-25T10:11:29.203454+0000","last_active":"2024-04-25T10:11:29.206257+0000","last_peered":"2024-04-25T10:11:29.206257+0000","last_clean":"2024-04-25T10:11:29.206257+0000","last_became_active":"2024-04-25T10:11:29.203148+0000","last_became_peered":"2024-04-25T10:11:29.203148+0000","last_unstale":"2024-04-25T10:11:29.206257+0000","last_undegraded":"2024-04-25T10:11:29.206257+0000","last_fullsized":"2024-04-25T10:11:29.206257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.277022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200148+0000","last_change":"2024-04-25T10:11:28.196014+0000","last_active":"2024-04-25T10:11:29.200148+0000","last_peered":"2024-04-25T10:11:29.200148+0000","last_clean":"2024-04-25T10:11:29.200148+0000","last_became_active":"2024-04-25T10:11:28.195531+0000","last_became_peered":"2024-04-25T10:11:28.195531+0000","last_unstale":"2024-04-25T10:11:29.200148+0000","last_undegraded":"2024-04-25T10:11:29.200148+0000","last_fullsized":"2024-04-25T10:11:29.200148+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:39.108320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.207417+0000","last_change":"2024-04-25T10:11:29.206432+0000","last_active":"2024-04-25T10:11:29.207417+0000","last_peered":"2024-04-25T10:11:29.207417+0000","last_clean":"2024-04-25T10:11:29.207417+0000","last_became_active":"2024-04-25T10:11:29.206133+0000","last_became_peered":"2024-04-25T10:11:29.206133+0000","last_unstale":"2024-04-25T10:11:29.207417+0000","last_undegraded":"2024-04-25T10:11:29.207417+0000","last_fullsized":"2024-04-25T10:11:29.207417+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:51.410363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.187232+0000","last_change":"2024-04-25T10:11:28.189322+0000","last_active":"2024-04-25T10:11:29.187232+0000","last_peered":"2024-04-25T10:11:29.187232+0000","last_clean":"2024-04-25T10:11:29.187232+0000","last_became_active":"2024-04-25T10:11:28.188916+0000","last_became_peered":"2024-04-25T10:11:28.188916+0000","last_unstale":"2024-04-25T10:11:29.187232+0000","last_undegraded":"2024-04-25T10:11:29.187232+0000","last_fullsized":"2024-04-25T10:11:29.187232+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:31.743669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.192300+0000","last_change":"2024-04-25T10:11:29.189753+0000","last_active":"2024-04-25T10:11:29.192300+0000","last_peered":"2024-04-25T10:11:29.192300+0000","last_clean":"2024-04-25T10:11:29.192300+0000","last_became_active":"2024-04-25T10:11:29.189358+0000","last_became_peered":"2024-04-25T10:11:29.189358+0000","last_unstale":"2024-04-25T10:11:29.192300+0000","last_undegraded":"2024-04-25T10:11:29.192300+0000","last_fullsized":"2024-04-25T10:11:29.192300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.174269+0000","last_change":"2024-04-25T10:11:28.208134+0000","last_active":"2024-04-25T10:11:29.174269+0000","last_peered":"2024-04-25T10:11:29.174269+0000","last_clean":"2024-04-25T10:11:29.174269+0000","last_became_active":"2024-04-25T10:11:28.207490+0000","last_became_peered":"2024-04-25T10:11:28.207490+0000","last_unstale":"2024-04-25T10:11:29.174269+0000","last_undegraded":"2024-04-25T10:11:29.174269+0000","last_fullsized":"2024-04-25T10:11:29.174269+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213677+0000","last_change":"2024-04-25T10:11:29.213173+0000","last_active":"2024-04-25T10:11:29.213677+0000","last_peered":"2024-04-25T10:11:29.213677+0000","last_clean":"2024-04-25T10:11:29.213677+0000","last_became_active":"2024-04-25T10:11:29.212887+0000","last_became_peered":"2024-04-25T10:11:29.212887+0000","last_unstale":"2024-04-25T10:11:29.213677+0000","last_undegraded":"2024-04-25T10:11:29.213677+0000","last_fullsized":"2024-04-25T10:11:29.213677+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:26.754293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188188+0000","last_change":"2024-04-25T10:11:28.191897+0000","last_active":"2024-04-25T10:11:29.188188+0000","last_peered":"2024-04-25T10:11:29.188188+0000","last_clean":"2024-04-25T10:11:29.188188+0000","last_became_active":"2024-04-25T10:11:28.190329+0000","last_became_peered":"2024-04-25T10:11:28.190329+0000","last_unstale":"2024-04-25T10:11:29.188188+0000","last_undegraded":"2024-04-25T10:11:29.188188+0000","last_fullsized":"2024-04-25T10:11:29.188188+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:50.564628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201078+0000","last_change":"2024-04-25T10:11:29.199211+0000","last_active":"2024-04-25T10:11:29.201078+0000","last_peered":"2024-04-25T10:11:29.201078+0000","last_clean":"2024-04-25T10:11:29.201078+0000","last_became_active":"2024-04-25T10:11:29.198914+0000","last_became_peered":"2024-04-25T10:11:29.198914+0000","last_unstale":"2024-04-25T10:11:29.201078+0000","last_undegraded":"2024-04-25T10:11:29.201078+0000","last_fullsized":"2024-04-25T10:11:29.201078+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:34.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211138+0000","last_change":"2024-04-25T10:11:28.178108+0000","last_active":"2024-04-25T10:11:29.211138+0000","last_peered":"2024-04-25T10:11:29.211138+0000","last_clean":"2024-04-25T10:11:29.211138+0000","last_became_active":"2024-04-25T10:11:28.177801+0000","last_became_peered":"2024-04-25T10:11:28.177801+0000","last_unstale":"2024-04-25T10:11:29.211138+0000","last_undegraded":"2024-04-25T10:11:29.211138+0000","last_fullsized":"2024-04-25T10:11:29.211138+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:45.659307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.196230+0000","last_change":"2024-04-25T10:11:29.194067+0000","last_active":"2024-04-25T10:11:29.196230+0000","last_peered":"2024-04-25T10:11:29.196230+0000","last_clean":"2024-04-25T10:11:29.196230+0000","last_became_active":"2024-04-25T10:11:29.192610+0000","last_became_peered":"2024-04-25T10:11:29.192610+0000","last_unstale":"2024-04-25T10:11:29.196230+0000","last_undegraded":"2024-04-25T10:11:29.196230+0000","last_fullsized":"2024-04-25T10:11:29.196230+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.184904+0000","last_change":"2024-04-25T10:11:28.194986+0000","last_active":"2024-04-25T10:11:29.184904+0000","last_peered":"2024-04-25T10:11:29.184904+0000","last_clean":"2024-04-25T10:11:29.184904+0000","last_became_active":"2024-04-25T10:11:28.194593+0000","last_became_peered":"2024-04-25T10:11:28.194593+0000","last_unstale":"2024-04-25T10:11:29.184904+0000","last_undegraded":"2024-04-25T10:11:29.184904+0000","last_fullsized":"2024-04-25T10:11:29.184904+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:03.949230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.192166+0000","last_change":"2024-04-25T10:11:29.190216+0000","last_active":"2024-04-25T10:11:29.192166+0000","last_peered":"2024-04-25T10:11:29.192166+0000","last_clean":"2024-04-25T10:11:29.192166+0000","last_became_active":"2024-04-25T10:11:29.189320+0000","last_became_peered":"2024-04-25T10:11:29.189320+0000","last_unstale":"2024-04-25T10:11:29.192166+0000","last_undegraded":"2024-04-25T10:11:29.192166+0000","last_fullsized":"2024-04-25T10:11:29.192166+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.180615+0000","last_change":"2024-04-25T10:11:28.208837+0000","last_active":"2024-04-25T10:11:29.180615+0000","last_peered":"2024-04-25T10:11:29.180615+0000","last_clean":"2024-04-25T10:11:29.180615+0000","last_became_active":"2024-04-25T10:11:28.206933+0000","last_became_peered":"2024-04-25T10:11:28.206933+0000","last_unstale":"2024-04-25T10:11:29.180615+0000","last_undegraded":"2024-04-25T10:11:29.180615+0000","last_fullsized":"2024-04-25T10:11:29.180615+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:51.292549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237665+0000","last_change":"2024-04-25T10:11:29.219389+0000","last_active":"2024-04-25T10:11:29.237665+0000","last_peered":"2024-04-25T10:11:29.237665+0000","last_clean":"2024-04-25T10:11:29.237665+0000","last_became_active":"2024-04-25T10:11:29.218517+0000","last_became_peered":"2024-04-25T10:11:29.218517+0000","last_unstale":"2024-04-25T10:11:29.237665+0000","last_undegraded":"2024-04-25T10:11:29.237665+0000","last_fullsized":"2024-04-25T10:11:29.237665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.182048+0000","last_change":"2024-04-25T10:11:28.208505+0000","last_active":"2024-04-25T10:11:29.182048+0000","last_peered":"2024-04-25T10:11:29.182048+0000","last_clean":"2024-04-25T10:11:29.182048+0000","last_became_active":"2024-04-25T10:11:28.208198+0000","last_became_peered":"2024-04-25T10:11:28.208198+0000","last_unstale":"2024-04-25T10:11:29.182048+0000","last_undegraded":"2024-04-25T10:11:29.182048+0000","last_fullsized":"2024-04-25T10:11:29.182048+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213456+0000","last_change":"2024-04-25T10:11:29.210080+0000","last_active":"2024-04-25T10:11:29.213456+0000","last_peered":"2024-04-25T10:11:29.213456+0000","last_clean":"2024-04-25T10:11:29.213456+0000","last_became_active":"2024-04-25T10:11:29.209764+0000","last_became_peered":"2024-04-25T10:11:29.209764+0000","last_unstale":"2024-04-25T10:11:29.213456+0000","last_undegraded":"2024-04-25T10:11:29.213456+0000","last_fullsized":"2024-04-25T10:11:29.213456+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:27.677352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.170987+0000","last_change":"2024-04-25T10:11:28.204006+0000","last_active":"2024-04-25T10:11:29.170987+0000","last_peered":"2024-04-25T10:11:29.170987+0000","last_clean":"2024-04-25T10:11:29.170987+0000","last_became_active":"2024-04-25T10:11:28.203010+0000","last_became_peered":"2024-04-25T10:11:28.203010+0000","last_unstale":"2024-04-25T10:11:29.170987+0000","last_undegraded":"2024-04-25T10:11:29.170987+0000","last_fullsized":"2024-04-25T10:11:29.170987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211274+0000","last_change":"2024-04-25T10:11:29.208603+0000","last_active":"2024-04-25T10:11:29.211274+0000","last_peered":"2024-04-25T10:11:29.211274+0000","last_clean":"2024-04-25T10:11:29.211274+0000","last_became_active":"2024-04-25T10:11:29.206282+0000","last_became_peered":"2024-04-25T10:11:29.206282+0000","last_unstale":"2024-04-25T10:11:29.211274+0000","last_undegraded":"2024-04-25T10:11:29.211274+0000","last_fullsized":"2024-04-25T10:11:29.211274+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.763184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.187350+0000","last_change":"2024-04-25T10:11:28.196162+0000","last_active":"2024-04-25T10:11:29.187350+0000","last_peered":"2024-04-25T10:11:29.187350+0000","last_clean":"2024-04-25T10:11:29.187350+0000","last_became_active":"2024-04-25T10:11:28.195723+0000","last_became_peered":"2024-04-25T10:11:28.195723+0000","last_unstale":"2024-04-25T10:11:29.187350+0000","last_undegraded":"2024-04-25T10:11:29.187350+0000","last_fullsized":"2024-04-25T10:11:29.187350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:51.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.230831+0000","last_change":"2024-04-25T10:11:29.229517+0000","last_active":"2024-04-25T10:11:29.230831+0000","last_peered":"2024-04-25T10:11:29.230831+0000","last_clean":"2024-04-25T10:11:29.230831+0000","last_became_active":"2024-04-25T10:11:29.228979+0000","last_became_peered":"2024-04-25T10:11:29.228979+0000","last_unstale":"2024-04-25T10:11:29.230831+0000","last_undegraded":"2024-04-25T10:11:29.230831+0000","last_fullsized":"2024-04-25T10:11:29.230831+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.174770+0000","last_change":"2024-04-25T10:11:28.203907+0000","last_active":"2024-04-25T10:11:29.174770+0000","last_peered":"2024-04-25T10:11:29.174770+0000","last_clean":"2024-04-25T10:11:29.174770+0000","last_became_active":"2024-04-25T10:11:28.203179+0000","last_became_peered":"2024-04-25T10:11:28.203179+0000","last_unstale":"2024-04-25T10:11:29.174770+0000","last_undegraded":"2024-04-25T10:11:29.174770+0000","last_fullsized":"2024-04-25T10:11:29.174770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237712+0000","last_change":"2024-04-25T10:11:29.226923+0000","last_active":"2024-04-25T10:11:29.237712+0000","last_peered":"2024-04-25T10:11:29.237712+0000","last_clean":"2024-04-25T10:11:29.237712+0000","last_became_active":"2024-04-25T10:11:29.226136+0000","last_became_peered":"2024-04-25T10:11:29.226136+0000","last_unstale":"2024-04-25T10:11:29.237712+0000","last_undegraded":"2024-04-25T10:11:29.237712+0000","last_fullsized":"2024-04-25T10:11:29.237712+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188320+0000","last_change":"2024-04-25T10:11:28.194380+0000","last_active":"2024-04-25T10:11:29.188320+0000","last_peered":"2024-04-25T10:11:29.188320+0000","last_clean":"2024-04-25T10:11:29.188320+0000","last_became_active":"2024-04-25T10:11:28.194012+0000","last_became_peered":"2024-04-25T10:11:28.194012+0000","last_unstale":"2024-04-25T10:11:29.188320+0000","last_undegraded":"2024-04-25T10:11:29.188320+0000","last_fullsized":"2024-04-25T10:11:29.188320+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.045808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236114+0000","last_change":"2024-04-25T10:11:29.208192+0000","last_active":"2024-04-25T10:11:29.236114+0000","last_peered":"2024-04-25T10:11:29.236114+0000","last_clean":"2024-04-25T10:11:29.236114+0000","last_became_active":"2024-04-25T10:11:29.207438+0000","last_became_peered":"2024-04-25T10:11:29.207438+0000","last_unstale":"2024-04-25T10:11:29.236114+0000","last_undegraded":"2024-04-25T10:11:29.236114+0000","last_fullsized":"2024-04-25T10:11:29.236114+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200892+0000","last_change":"2024-04-25T10:11:28.189384+0000","last_active":"2024-04-25T10:11:29.200892+0000","last_peered":"2024-04-25T10:11:29.200892+0000","last_clean":"2024-04-25T10:11:29.200892+0000","last_became_active":"2024-04-25T10:11:28.189004+0000","last_became_peered":"2024-04-25T10:11:28.189004+0000","last_unstale":"2024-04-25T10:11:29.200892+0000","last_undegraded":"2024-04-25T10:11:29.200892+0000","last_fullsized":"2024-04-25T10:11:29.200892+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:59.970417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.178644+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:11:29.178644+0000","last_peered":"2024-04-25T10:11:29.178644+0000","last_clean":"2024-04-25T10:11:29.178644+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:11:29.178644+0000","last_undegraded":"2024-04-25T10:11:29.178644+0000","last_fullsized":"2024-04-25T10:11:29.178644+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1201687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":613375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":603992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":611251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":611741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":611193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":603992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":1203229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:32.192 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.337+0000 7fed81edb640 1 Processor -- start 2024-04-25T10:11:32.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.337+0000 7fed81edb640 1 -- start start 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed7c0ce510 0x7fed7c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fed7c144ca0 0x7fed7c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0cee30 con 0x7fed7c06ec60 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0cc2b0 con 0x7fed7c0ce510 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c06e290 con 0x7fed7c144ca0 2024-04-25T10:11:32.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed7c0ce510 0x7fed7c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fed7c0ce510 0x7fed7c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45784/0 (socket says 172.21.15.82:45784) 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 -- 172.21.15.82:0/897298182 learned_addr learned my addr 172.21.15.82:0/897298182 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 -- 172.21.15.82:0/897298182 >> v2:172.21.15.196:3300/0 conn(0x7fed7c144ca0 msgr2=0x7fed7c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/897298182 >> v2:172.21.15.196:3300/0 conn(0x7fed7c144ca0 0x7fed7c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 -- 172.21.15.82:0/897298182 >> v1:172.21.15.196:6789/0 conn(0x7fed7c06ec60 legacy=0x7fed7c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:32.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 -- 172.21.15.82:0/897298182 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0ccc70 con 0x7fed7c0ce510 2024-04-25T10:11:32.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/897298182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c0ce8f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fed7c070fe0 tx=0x7fed6401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29d24dc2b4c93b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7affd640 1 -- 172.21.15.82:0/897298182 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640b5070 con 0x7fed7c0ce510 2024-04-25T10:11:32.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7affd640 1 -- 172.21.15.82:0/897298182 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed640a89d0 con 0x7fed7c0ce510 2024-04-25T10:11:32.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed7affd640 1 -- 172.21.15.82:0/897298182 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed640a8da0 con 0x7fed7c0ce510 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 -- 172.21.15.82:0/897298182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 msgr2=0x7fed7c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 --2- 172.21.15.82:0/897298182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c0ce8f0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fed7c070fe0 tx=0x7fed6401ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 -- 172.21.15.82:0/897298182 shutdown_connections 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.341+0000 7fed81edb640 1 --2- 172.21.15.82:0/897298182 >> v2:172.21.15.196:3300/0 conn(0x7fed7c144ca0 0x7fed7c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 --2- 172.21.15.82:0/897298182 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c0ce8f0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/897298182 >> 172.21.15.82:0/897298182 conn(0x7fed7c0b11a0 msgr2=0x7fed7c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/897298182 shutdown_connections 2024-04-25T10:11:32.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/897298182 wait complete. 2024-04-25T10:11:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 Processor -- start 2024-04-25T10:11:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- start start 2024-04-25T10:11:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 0x7fed7c13bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 0x7fed7c13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0cc900 con 0x7fed7c0ce510 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c06df20 con 0x7fed7c144ca0 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c142d90 con 0x7fed7c06ec60 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45788/0 (socket says 172.21.15.82:45788) 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 -- 172.21.15.82:0/722956933 learned_addr learned my addr 172.21.15.82:0/722956933 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 0x7fed7c13bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed7bfff640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 0x7fed7c13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 -- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 msgr2=0x7fed7c13bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 0x7fed7c13bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 -- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 msgr2=0x7fed7c13eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 0x7fed7c13eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 -- 172.21.15.82:0/722956933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c13c430 con 0x7fed7c0ce510 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 0x7fed7c13bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed7bfff640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 0x7fed7c13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed72ffd640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b3f0 tx=0x7fed6c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c015020 con 0x7fed7c0ce510 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6c011850 con 0x7fed7c0ce510 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c1f4bd0 con 0x7fed7c0ce510 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6c011c40 con 0x7fed7c0ce510 2024-04-25T10:11:32.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c0b29b0 con 0x7fed7c0ce510 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.345+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c04efa0 con 0x7fed7c0ce510 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.349+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed6c018020 con 0x7fed7c0ce510 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.349+0000 7fed78ff9640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.349+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.349+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed6c085020 con 0x7fed7c0ce510 2024-04-25T10:11:32.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.349+0000 7fed7b7fe640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 0x7fed60069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed640b2000 tx=0x7fed64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.353+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed6c01f020 con 0x7fed7c0ce510 2024-04-25T10:11:32.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.509+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fed7c0b1cc0 con 0x7fed60066f60 2024-04-25T10:11:32.521 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.517+0000 7fed78ff9640 1 -- 172.21.15.82:0/722956933 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252418 (secure 0 0 0) 0x7fed7c0b1cc0 con 0x7fed60066f60 2024-04-25T10:11:32.524 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:32.525 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 msgr2=0x7fed60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 0x7fed60069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fed640b2000 tx=0x7fed64008040 comp rx=0 tx=0).stop 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 msgr2=0x7fed7c13eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b3f0 tx=0x7fed6c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 shutdown_connections 2024-04-25T10:11:32.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fed60066f60 0x7fed60069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fed7c144ca0 0x7fed7c13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fed7c0ce510 0x7fed7c13eab0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 --2- 172.21.15.82:0/722956933 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fed7c06ec60 0x7fed7c13bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 >> 172.21.15.82:0/722956933 conn(0x7fed7c0b11a0 msgr2=0x7fed7c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 shutdown_connections 2024-04-25T10:11:32.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.521+0000 7fed81edb640 1 -- 172.21.15.82:0/722956933 wait complete. 2024-04-25T10:11:32.543 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":389,"stamp":"2024-04-25T10:11:31.442397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":67,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260956,"kb_used_data":11468,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713764,"statfs":{"total":773094113280,"available":772826894336,"internally_reserved":0,"allocated":11743232,"data_stored":6060460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.293802"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.212706+0000","last_change":"2024-04-25T10:11:29.211228+0000","last_active":"2024-04-25T10:11:29.212706+0000","last_peered":"2024-04-25T10:11:29.212706+0000","last_clean":"2024-04-25T10:11:29.212706+0000","last_became_active":"2024-04-25T10:11:29.210953+0000","last_became_peered":"2024-04-25T10:11:29.210953+0000","last_unstale":"2024-04-25T10:11:29.212706+0000","last_undegraded":"2024-04-25T10:11:29.212706+0000","last_fullsized":"2024-04-25T10:11:29.212706+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:22.551667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186970+0000","last_change":"2024-04-25T10:11:29.183016+0000","last_active":"2024-04-25T10:11:29.186970+0000","last_peered":"2024-04-25T10:11:29.186970+0000","last_clean":"2024-04-25T10:11:29.186970+0000","last_became_active":"2024-04-25T10:11:29.182186+0000","last_became_peered":"2024-04-25T10:11:29.182186+0000","last_unstale":"2024-04-25T10:11:29.186970+0000","last_undegraded":"2024-04-25T10:11:29.186970+0000","last_fullsized":"2024-04-25T10:11:29.186970+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.234209+0000","last_change":"2024-04-25T10:11:29.233829+0000","last_active":"2024-04-25T10:11:29.234209+0000","last_peered":"2024-04-25T10:11:29.234209+0000","last_clean":"2024-04-25T10:11:29.234209+0000","last_became_active":"2024-04-25T10:11:29.233584+0000","last_became_peered":"2024-04-25T10:11:29.233584+0000","last_unstale":"2024-04-25T10:11:29.234209+0000","last_undegraded":"2024-04-25T10:11:29.234209+0000","last_fullsized":"2024-04-25T10:11:29.234209+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:13.518229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.199812+0000","last_change":"2024-04-25T10:11:29.177949+0000","last_active":"2024-04-25T10:11:29.199812+0000","last_peered":"2024-04-25T10:11:29.199812+0000","last_clean":"2024-04-25T10:11:29.199812+0000","last_became_active":"2024-04-25T10:11:29.177211+0000","last_became_peered":"2024-04-25T10:11:29.177211+0000","last_unstale":"2024-04-25T10:11:29.199812+0000","last_undegraded":"2024-04-25T10:11:29.199812+0000","last_fullsized":"2024-04-25T10:11:29.199812+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208412+0000","last_change":"2024-04-25T10:11:29.204715+0000","last_active":"2024-04-25T10:11:29.208412+0000","last_peered":"2024-04-25T10:11:29.208412+0000","last_clean":"2024-04-25T10:11:29.208412+0000","last_became_active":"2024-04-25T10:11:29.204082+0000","last_became_peered":"2024-04-25T10:11:29.204082+0000","last_unstale":"2024-04-25T10:11:29.208412+0000","last_undegraded":"2024-04-25T10:11:29.208412+0000","last_fullsized":"2024-04-25T10:11:29.208412+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.303246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223619+0000","last_change":"2024-04-25T10:11:29.216807+0000","last_active":"2024-04-25T10:11:29.223619+0000","last_peered":"2024-04-25T10:11:29.223619+0000","last_clean":"2024-04-25T10:11:29.223619+0000","last_became_active":"2024-04-25T10:11:29.216165+0000","last_became_peered":"2024-04-25T10:11:29.216165+0000","last_unstale":"2024-04-25T10:11:29.223619+0000","last_undegraded":"2024-04-25T10:11:29.223619+0000","last_fullsized":"2024-04-25T10:11:29.223619+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223309+0000","last_change":"2024-04-25T10:11:29.213933+0000","last_active":"2024-04-25T10:11:29.223309+0000","last_peered":"2024-04-25T10:11:29.223309+0000","last_clean":"2024-04-25T10:11:29.223309+0000","last_became_active":"2024-04-25T10:11:29.209929+0000","last_became_peered":"2024-04-25T10:11:29.209929+0000","last_unstale":"2024-04-25T10:11:29.223309+0000","last_undegraded":"2024-04-25T10:11:29.223309+0000","last_fullsized":"2024-04-25T10:11:29.223309+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211216+0000","last_change":"2024-04-25T10:11:29.208531+0000","last_active":"2024-04-25T10:11:29.211216+0000","last_peered":"2024-04-25T10:11:29.211216+0000","last_clean":"2024-04-25T10:11:29.211216+0000","last_became_active":"2024-04-25T10:11:29.206270+0000","last_became_peered":"2024-04-25T10:11:29.206270+0000","last_unstale":"2024-04-25T10:11:29.211216+0000","last_undegraded":"2024-04-25T10:11:29.211216+0000","last_fullsized":"2024-04-25T10:11:29.211216+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:17.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186990+0000","last_change":"2024-04-25T10:11:29.175092+0000","last_active":"2024-04-25T10:11:29.186990+0000","last_peered":"2024-04-25T10:11:29.186990+0000","last_clean":"2024-04-25T10:11:29.186990+0000","last_became_active":"2024-04-25T10:11:29.174377+0000","last_became_peered":"2024-04-25T10:11:29.174377+0000","last_unstale":"2024-04-25T10:11:29.186990+0000","last_undegraded":"2024-04-25T10:11:29.186990+0000","last_fullsized":"2024-04-25T10:11:29.186990+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:45.215104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224930+0000","last_change":"2024-04-25T10:11:29.224395+0000","last_active":"2024-04-25T10:11:29.224930+0000","last_peered":"2024-04-25T10:11:29.224930+0000","last_clean":"2024-04-25T10:11:29.224930+0000","last_became_active":"2024-04-25T10:11:29.224098+0000","last_became_peered":"2024-04-25T10:11:29.224098+0000","last_unstale":"2024-04-25T10:11:29.224930+0000","last_undegraded":"2024-04-25T10:11:29.224930+0000","last_fullsized":"2024-04-25T10:11:29.224930+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237947+0000","last_change":"2024-04-25T10:11:29.233407+0000","last_active":"2024-04-25T10:11:29.237947+0000","last_peered":"2024-04-25T10:11:29.237947+0000","last_clean":"2024-04-25T10:11:29.237947+0000","last_became_active":"2024-04-25T10:11:29.232754+0000","last_became_peered":"2024-04-25T10:11:29.232754+0000","last_unstale":"2024-04-25T10:11:29.237947+0000","last_undegraded":"2024-04-25T10:11:29.237947+0000","last_fullsized":"2024-04-25T10:11:29.237947+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.215818+0000","last_change":"2024-04-25T10:11:29.203811+0000","last_active":"2024-04-25T10:11:29.215818+0000","last_peered":"2024-04-25T10:11:29.215818+0000","last_clean":"2024-04-25T10:11:29.215818+0000","last_became_active":"2024-04-25T10:11:29.203093+0000","last_became_peered":"2024-04-25T10:11:29.203093+0000","last_unstale":"2024-04-25T10:11:29.215818+0000","last_undegraded":"2024-04-25T10:11:29.215818+0000","last_fullsized":"2024-04-25T10:11:29.215818+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.227459+0000","last_change":"2024-04-25T10:11:29.226590+0000","last_active":"2024-04-25T10:11:29.227459+0000","last_peered":"2024-04-25T10:11:29.227459+0000","last_clean":"2024-04-25T10:11:29.227459+0000","last_became_active":"2024-04-25T10:11:29.226339+0000","last_became_peered":"2024-04-25T10:11:29.226339+0000","last_unstale":"2024-04-25T10:11:29.227459+0000","last_undegraded":"2024-04-25T10:11:29.227459+0000","last_fullsized":"2024-04-25T10:11:29.227459+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.196268+0000","last_change":"2024-04-25T10:11:29.192851+0000","last_active":"2024-04-25T10:11:29.196268+0000","last_peered":"2024-04-25T10:11:29.196268+0000","last_clean":"2024-04-25T10:11:29.196268+0000","last_became_active":"2024-04-25T10:11:29.192138+0000","last_became_peered":"2024-04-25T10:11:29.192138+0000","last_unstale":"2024-04-25T10:11:29.196268+0000","last_undegraded":"2024-04-25T10:11:29.196268+0000","last_fullsized":"2024-04-25T10:11:29.196268+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.230792+0000","last_change":"2024-04-25T10:11:29.229197+0000","last_active":"2024-04-25T10:11:29.230792+0000","last_peered":"2024-04-25T10:11:29.230792+0000","last_clean":"2024-04-25T10:11:29.230792+0000","last_became_active":"2024-04-25T10:11:29.228514+0000","last_became_peered":"2024-04-25T10:11:29.228514+0000","last_unstale":"2024-04-25T10:11:29.230792+0000","last_undegraded":"2024-04-25T10:11:29.230792+0000","last_fullsized":"2024-04-25T10:11:29.230792+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.205510+0000","last_change":"2024-04-25T10:11:29.198500+0000","last_active":"2024-04-25T10:11:29.205510+0000","last_peered":"2024-04-25T10:11:29.205510+0000","last_clean":"2024-04-25T10:11:29.205510+0000","last_became_active":"2024-04-25T10:11:29.197756+0000","last_became_peered":"2024-04-25T10:11:29.197756+0000","last_unstale":"2024-04-25T10:11:29.205510+0000","last_undegraded":"2024-04-25T10:11:29.205510+0000","last_fullsized":"2024-04-25T10:11:29.205510+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228567+0000","last_change":"2024-04-25T10:11:29.227891+0000","last_active":"2024-04-25T10:11:29.228567+0000","last_peered":"2024-04-25T10:11:29.228567+0000","last_clean":"2024-04-25T10:11:29.228567+0000","last_became_active":"2024-04-25T10:11:29.227557+0000","last_became_peered":"2024-04-25T10:11:29.227557+0000","last_unstale":"2024-04-25T10:11:29.228567+0000","last_undegraded":"2024-04-25T10:11:29.228567+0000","last_fullsized":"2024-04-25T10:11:29.228567+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:07.674828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223525+0000","last_change":"2024-04-25T10:11:29.216463+0000","last_active":"2024-04-25T10:11:29.223525+0000","last_peered":"2024-04-25T10:11:29.223525+0000","last_clean":"2024-04-25T10:11:29.223525+0000","last_became_active":"2024-04-25T10:11:29.214904+0000","last_became_peered":"2024-04-25T10:11:29.214904+0000","last_unstale":"2024-04-25T10:11:29.223525+0000","last_undegraded":"2024-04-25T10:11:29.223525+0000","last_fullsized":"2024-04-25T10:11:29.223525+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.221535+0000","last_change":"2024-04-25T10:11:29.216585+0000","last_active":"2024-04-25T10:11:29.221535+0000","last_peered":"2024-04-25T10:11:29.221535+0000","last_clean":"2024-04-25T10:11:29.221535+0000","last_became_active":"2024-04-25T10:11:29.216258+0000","last_became_peered":"2024-04-25T10:11:29.216258+0000","last_unstale":"2024-04-25T10:11:29.221535+0000","last_undegraded":"2024-04-25T10:11:29.221535+0000","last_fullsized":"2024-04-25T10:11:29.221535+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:15.565173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188724+0000","last_change":"2024-04-25T10:11:29.183504+0000","last_active":"2024-04-25T10:11:29.188724+0000","last_peered":"2024-04-25T10:11:29.188724+0000","last_clean":"2024-04-25T10:11:29.188724+0000","last_became_active":"2024-04-25T10:11:29.182421+0000","last_became_peered":"2024-04-25T10:11:29.182421+0000","last_unstale":"2024-04-25T10:11:29.188724+0000","last_undegraded":"2024-04-25T10:11:29.188724+0000","last_fullsized":"2024-04-25T10:11:29.188724+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:52.303246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228834+0000","last_change":"2024-04-25T10:11:29.228369+0000","last_active":"2024-04-25T10:11:29.228834+0000","last_peered":"2024-04-25T10:11:29.228834+0000","last_clean":"2024-04-25T10:11:29.228834+0000","last_became_active":"2024-04-25T10:11:29.228046+0000","last_became_peered":"2024-04-25T10:11:29.228046+0000","last_unstale":"2024-04-25T10:11:29.228834+0000","last_undegraded":"2024-04-25T10:11:29.228834+0000","last_fullsized":"2024-04-25T10:11:29.228834+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:49.534581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237375+0000","last_change":"2024-04-25T10:11:29.237008+0000","last_active":"2024-04-25T10:11:29.237375+0000","last_peered":"2024-04-25T10:11:29.237375+0000","last_clean":"2024-04-25T10:11:29.237375+0000","last_became_active":"2024-04-25T10:11:29.236717+0000","last_became_peered":"2024-04-25T10:11:29.236717+0000","last_unstale":"2024-04-25T10:11:29.237375+0000","last_undegraded":"2024-04-25T10:11:29.237375+0000","last_fullsized":"2024-04-25T10:11:29.237375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.227603+0000","last_change":"2024-04-25T10:11:29.226659+0000","last_active":"2024-04-25T10:11:29.227603+0000","last_peered":"2024-04-25T10:11:29.227603+0000","last_clean":"2024-04-25T10:11:29.227603+0000","last_became_active":"2024-04-25T10:11:29.226400+0000","last_became_peered":"2024-04-25T10:11:29.226400+0000","last_unstale":"2024-04-25T10:11:29.227603+0000","last_undegraded":"2024-04-25T10:11:29.227603+0000","last_fullsized":"2024-04-25T10:11:29.227603+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:34.582899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208133+0000","last_change":"2024-04-25T10:11:29.202795+0000","last_active":"2024-04-25T10:11:29.208133+0000","last_peered":"2024-04-25T10:11:29.208133+0000","last_clean":"2024-04-25T10:11:29.208133+0000","last_became_active":"2024-04-25T10:11:29.202524+0000","last_became_peered":"2024-04-25T10:11:29.202524+0000","last_unstale":"2024-04-25T10:11:29.208133+0000","last_undegraded":"2024-04-25T10:11:29.208133+0000","last_fullsized":"2024-04-25T10:11:29.208133+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:45.215104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.234586+0000","last_change":"2024-04-25T10:11:29.227391+0000","last_active":"2024-04-25T10:11:29.234586+0000","last_peered":"2024-04-25T10:11:29.234586+0000","last_clean":"2024-04-25T10:11:29.234586+0000","last_became_active":"2024-04-25T10:11:29.226689+0000","last_became_peered":"2024-04-25T10:11:29.226689+0000","last_unstale":"2024-04-25T10:11:29.234586+0000","last_undegraded":"2024-04-25T10:11:29.234586+0000","last_fullsized":"2024-04-25T10:11:29.234586+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188425+0000","last_change":"2024-04-25T10:11:29.181851+0000","last_active":"2024-04-25T10:11:29.188425+0000","last_peered":"2024-04-25T10:11:29.188425+0000","last_clean":"2024-04-25T10:11:29.188425+0000","last_became_active":"2024-04-25T10:11:29.181116+0000","last_became_peered":"2024-04-25T10:11:29.181116+0000","last_unstale":"2024-04-25T10:11:29.188425+0000","last_undegraded":"2024-04-25T10:11:29.188425+0000","last_fullsized":"2024-04-25T10:11:29.188425+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237838+0000","last_change":"2024-04-25T10:11:29.232728+0000","last_active":"2024-04-25T10:11:29.237838+0000","last_peered":"2024-04-25T10:11:29.237838+0000","last_clean":"2024-04-25T10:11:29.237838+0000","last_became_active":"2024-04-25T10:11:29.225467+0000","last_became_peered":"2024-04-25T10:11:29.225467+0000","last_unstale":"2024-04-25T10:11:29.237838+0000","last_undegraded":"2024-04-25T10:11:29.237838+0000","last_fullsized":"2024-04-25T10:11:29.237838+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236013+0000","last_change":"2024-04-25T10:11:29.235222+0000","last_active":"2024-04-25T10:11:29.236013+0000","last_peered":"2024-04-25T10:11:29.236013+0000","last_clean":"2024-04-25T10:11:29.236013+0000","last_became_active":"2024-04-25T10:11:29.234694+0000","last_became_peered":"2024-04-25T10:11:29.234694+0000","last_unstale":"2024-04-25T10:11:29.236013+0000","last_undegraded":"2024-04-25T10:11:29.236013+0000","last_fullsized":"2024-04-25T10:11:29.236013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201541+0000","last_change":"2024-04-25T10:11:29.199218+0000","last_active":"2024-04-25T10:11:29.201541+0000","last_peered":"2024-04-25T10:11:29.201541+0000","last_clean":"2024-04-25T10:11:29.201541+0000","last_became_active":"2024-04-25T10:11:29.198875+0000","last_became_peered":"2024-04-25T10:11:29.198875+0000","last_unstale":"2024-04-25T10:11:29.201541+0000","last_undegraded":"2024-04-25T10:11:29.201541+0000","last_fullsized":"2024-04-25T10:11:29.201541+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:21.102504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.226834+0000","last_change":"2024-04-25T10:11:29.219238+0000","last_active":"2024-04-25T10:11:29.226834+0000","last_peered":"2024-04-25T10:11:29.226834+0000","last_clean":"2024-04-25T10:11:29.226834+0000","last_became_active":"2024-04-25T10:11:29.218067+0000","last_became_peered":"2024-04-25T10:11:29.218067+0000","last_unstale":"2024-04-25T10:11:29.226834+0000","last_undegraded":"2024-04-25T10:11:29.226834+0000","last_fullsized":"2024-04-25T10:11:29.226834+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.221751+0000","last_change":"2024-04-25T10:11:29.201233+0000","last_active":"2024-04-25T10:11:29.221751+0000","last_peered":"2024-04-25T10:11:29.221751+0000","last_clean":"2024-04-25T10:11:29.221751+0000","last_became_active":"2024-04-25T10:11:29.199982+0000","last_became_peered":"2024-04-25T10:11:29.199982+0000","last_unstale":"2024-04-25T10:11:29.221751+0000","last_undegraded":"2024-04-25T10:11:29.221751+0000","last_fullsized":"2024-04-25T10:11:29.221751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211239+0000","last_change":"2024-04-25T10:11:29.209367+0000","last_active":"2024-04-25T10:11:29.211239+0000","last_peered":"2024-04-25T10:11:29.211239+0000","last_clean":"2024-04-25T10:11:29.211239+0000","last_became_active":"2024-04-25T10:11:29.209071+0000","last_became_peered":"2024-04-25T10:11:29.209071+0000","last_unstale":"2024-04-25T10:11:29.211239+0000","last_undegraded":"2024-04-25T10:11:29.211239+0000","last_fullsized":"2024-04-25T10:11:29.211239+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:28.022053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201310+0000","last_change":"2024-04-25T10:11:29.200534+0000","last_active":"2024-04-25T10:11:29.201310+0000","last_peered":"2024-04-25T10:11:29.201310+0000","last_clean":"2024-04-25T10:11:29.201310+0000","last_became_active":"2024-04-25T10:11:29.200255+0000","last_became_peered":"2024-04-25T10:11:29.200255+0000","last_unstale":"2024-04-25T10:11:29.201310+0000","last_undegraded":"2024-04-25T10:11:29.201310+0000","last_fullsized":"2024-04-25T10:11:29.201310+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:05.484998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"106'4","reported_seq":16,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223852+0000","last_change":"2024-04-25T10:11:28.205735+0000","last_active":"2024-04-25T10:11:29.223852+0000","last_peered":"2024-04-25T10:11:29.223852+0000","last_clean":"2024-04-25T10:11:29.223852+0000","last_became_active":"2024-04-25T10:11:28.205108+0000","last_became_peered":"2024-04-25T10:11:28.205108+0000","last_unstale":"2024-04-25T10:11:29.223852+0000","last_undegraded":"2024-04-25T10:11:29.223852+0000","last_fullsized":"2024-04-25T10:11:29.223852+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:50.630210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211323+0000","last_change":"2024-04-25T10:11:29.209380+0000","last_active":"2024-04-25T10:11:29.211323+0000","last_peered":"2024-04-25T10:11:29.211323+0000","last_clean":"2024-04-25T10:11:29.211323+0000","last_became_active":"2024-04-25T10:11:29.209110+0000","last_became_peered":"2024-04-25T10:11:29.209110+0000","last_unstale":"2024-04-25T10:11:29.211323+0000","last_undegraded":"2024-04-25T10:11:29.211323+0000","last_fullsized":"2024-04-25T10:11:29.211323+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:00.183288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200184+0000","last_change":"2024-04-25T10:11:28.197059+0000","last_active":"2024-04-25T10:11:29.200184+0000","last_peered":"2024-04-25T10:11:29.200184+0000","last_clean":"2024-04-25T10:11:29.200184+0000","last_became_active":"2024-04-25T10:11:28.196477+0000","last_became_peered":"2024-04-25T10:11:28.196477+0000","last_unstale":"2024-04-25T10:11:29.200184+0000","last_undegraded":"2024-04-25T10:11:29.200184+0000","last_fullsized":"2024-04-25T10:11:29.200184+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:52.309299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224680+0000","last_change":"2024-04-25T10:11:29.214836+0000","last_active":"2024-04-25T10:11:29.224680+0000","last_peered":"2024-04-25T10:11:29.224680+0000","last_clean":"2024-04-25T10:11:29.224680+0000","last_became_active":"2024-04-25T10:11:29.214038+0000","last_became_peered":"2024-04-25T10:11:29.214038+0000","last_unstale":"2024-04-25T10:11:29.224680+0000","last_undegraded":"2024-04-25T10:11:29.224680+0000","last_fullsized":"2024-04-25T10:11:29.224680+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236769+0000","last_change":"2024-04-25T10:11:28.193383+0000","last_active":"2024-04-25T10:11:29.236769+0000","last_peered":"2024-04-25T10:11:29.236769+0000","last_clean":"2024-04-25T10:11:29.236769+0000","last_became_active":"2024-04-25T10:11:28.192595+0000","last_became_peered":"2024-04-25T10:11:28.192595+0000","last_unstale":"2024-04-25T10:11:29.236769+0000","last_undegraded":"2024-04-25T10:11:29.236769+0000","last_fullsized":"2024-04-25T10:11:29.236769+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236211+0000","last_change":"2024-04-25T10:11:29.208535+0000","last_active":"2024-04-25T10:11:29.236211+0000","last_peered":"2024-04-25T10:11:29.236211+0000","last_clean":"2024-04-25T10:11:29.236211+0000","last_became_active":"2024-04-25T10:11:29.207590+0000","last_became_peered":"2024-04-25T10:11:29.207590+0000","last_unstale":"2024-04-25T10:11:29.236211+0000","last_undegraded":"2024-04-25T10:11:29.236211+0000","last_fullsized":"2024-04-25T10:11:29.236211+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225296+0000","last_change":"2024-04-25T10:11:28.184824+0000","last_active":"2024-04-25T10:11:29.225296+0000","last_peered":"2024-04-25T10:11:29.225296+0000","last_clean":"2024-04-25T10:11:29.225296+0000","last_became_active":"2024-04-25T10:11:28.184372+0000","last_became_peered":"2024-04-25T10:11:28.184372+0000","last_unstale":"2024-04-25T10:11:29.225296+0000","last_undegraded":"2024-04-25T10:11:29.225296+0000","last_fullsized":"2024-04-25T10:11:29.225296+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:22.818301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.210588+0000","last_change":"2024-04-25T10:11:29.204815+0000","last_active":"2024-04-25T10:11:29.210588+0000","last_peered":"2024-04-25T10:11:29.210588+0000","last_clean":"2024-04-25T10:11:29.210588+0000","last_became_active":"2024-04-25T10:11:29.203930+0000","last_became_peered":"2024-04-25T10:11:29.203930+0000","last_unstale":"2024-04-25T10:11:29.210588+0000","last_undegraded":"2024-04-25T10:11:29.210588+0000","last_fullsized":"2024-04-25T10:11:29.210588+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:37.892413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.224059+0000","last_change":"2024-04-25T10:11:28.208539+0000","last_active":"2024-04-25T10:11:29.224059+0000","last_peered":"2024-04-25T10:11:29.224059+0000","last_clean":"2024-04-25T10:11:29.224059+0000","last_became_active":"2024-04-25T10:11:28.207800+0000","last_became_peered":"2024-04-25T10:11:28.207800+0000","last_unstale":"2024-04-25T10:11:29.224059+0000","last_undegraded":"2024-04-25T10:11:29.224059+0000","last_fullsized":"2024-04-25T10:11:29.224059+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:50.630210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225751+0000","last_change":"2024-04-25T10:11:29.225244+0000","last_active":"2024-04-25T10:11:29.225751+0000","last_peered":"2024-04-25T10:11:29.225751+0000","last_clean":"2024-04-25T10:11:29.225751+0000","last_became_active":"2024-04-25T10:11:29.224875+0000","last_became_peered":"2024-04-25T10:11:29.224875+0000","last_unstale":"2024-04-25T10:11:29.225751+0000","last_undegraded":"2024-04-25T10:11:29.225751+0000","last_fullsized":"2024-04-25T10:11:29.225751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:35:44.182376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236258+0000","last_change":"2024-04-25T10:11:28.200072+0000","last_active":"2024-04-25T10:11:29.236258+0000","last_peered":"2024-04-25T10:11:29.236258+0000","last_clean":"2024-04-25T10:11:29.236258+0000","last_became_active":"2024-04-25T10:11:28.199315+0000","last_became_peered":"2024-04-25T10:11:28.199315+0000","last_unstale":"2024-04-25T10:11:29.236258+0000","last_undegraded":"2024-04-25T10:11:29.236258+0000","last_fullsized":"2024-04-25T10:11:29.236258+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.391764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223873+0000","last_change":"2024-04-25T10:11:29.221541+0000","last_active":"2024-04-25T10:11:29.223873+0000","last_peered":"2024-04-25T10:11:29.223873+0000","last_clean":"2024-04-25T10:11:29.223873+0000","last_became_active":"2024-04-25T10:11:29.215809+0000","last_became_peered":"2024-04-25T10:11:29.215809+0000","last_unstale":"2024-04-25T10:11:29.223873+0000","last_undegraded":"2024-04-25T10:11:29.223873+0000","last_fullsized":"2024-04-25T10:11:29.223873+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.170839+0000","last_change":"2024-04-25T10:11:28.193710+0000","last_active":"2024-04-25T10:11:29.170839+0000","last_peered":"2024-04-25T10:11:29.170839+0000","last_clean":"2024-04-25T10:11:29.170839+0000","last_became_active":"2024-04-25T10:11:28.192936+0000","last_became_peered":"2024-04-25T10:11:28.192936+0000","last_unstale":"2024-04-25T10:11:29.170839+0000","last_undegraded":"2024-04-25T10:11:29.170839+0000","last_fullsized":"2024-04-25T10:11:29.170839+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:22.818301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.210656+0000","last_change":"2024-04-25T10:11:29.206953+0000","last_active":"2024-04-25T10:11:29.210656+0000","last_peered":"2024-04-25T10:11:29.210656+0000","last_clean":"2024-04-25T10:11:29.210656+0000","last_became_active":"2024-04-25T10:11:29.200887+0000","last_became_peered":"2024-04-25T10:11:29.200887+0000","last_unstale":"2024-04-25T10:11:29.210656+0000","last_undegraded":"2024-04-25T10:11:29.210656+0000","last_fullsized":"2024-04-25T10:11:29.210656+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.797400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.168869+0000","last_change":"2024-04-25T10:11:28.173183+0000","last_active":"2024-04-25T10:11:29.168869+0000","last_peered":"2024-04-25T10:11:29.168869+0000","last_clean":"2024-04-25T10:11:29.168869+0000","last_became_active":"2024-04-25T10:11:28.172876+0000","last_became_peered":"2024-04-25T10:11:28.172876+0000","last_unstale":"2024-04-25T10:11:29.168869+0000","last_undegraded":"2024-04-25T10:11:29.168869+0000","last_fullsized":"2024-04-25T10:11:29.168869+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:10.677015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213342+0000","last_change":"2024-04-25T10:11:29.212314+0000","last_active":"2024-04-25T10:11:29.213342+0000","last_peered":"2024-04-25T10:11:29.213342+0000","last_clean":"2024-04-25T10:11:29.213342+0000","last_became_active":"2024-04-25T10:11:29.211949+0000","last_became_peered":"2024-04-25T10:11:29.211949+0000","last_unstale":"2024-04-25T10:11:29.213342+0000","last_undegraded":"2024-04-25T10:11:29.213342+0000","last_fullsized":"2024-04-25T10:11:29.213342+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:51.043662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.171605+0000","last_change":"2024-04-25T10:11:28.193547+0000","last_active":"2024-04-25T10:11:29.171605+0000","last_peered":"2024-04-25T10:11:29.171605+0000","last_clean":"2024-04-25T10:11:29.171605+0000","last_became_active":"2024-04-25T10:11:28.192759+0000","last_became_peered":"2024-04-25T10:11:28.192759+0000","last_unstale":"2024-04-25T10:11:29.171605+0000","last_undegraded":"2024-04-25T10:11:29.171605+0000","last_fullsized":"2024-04-25T10:11:29.171605+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:58:44.204407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211080+0000","last_change":"2024-04-25T10:11:29.206356+0000","last_active":"2024-04-25T10:11:29.211080+0000","last_peered":"2024-04-25T10:11:29.211080+0000","last_clean":"2024-04-25T10:11:29.211080+0000","last_became_active":"2024-04-25T10:11:29.204920+0000","last_became_peered":"2024-04-25T10:11:29.204920+0000","last_unstale":"2024-04-25T10:11:29.211080+0000","last_undegraded":"2024-04-25T10:11:29.211080+0000","last_fullsized":"2024-04-25T10:11:29.211080+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.687712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.184424+0000","last_change":"2024-04-25T10:11:28.183204+0000","last_active":"2024-04-25T10:11:29.184424+0000","last_peered":"2024-04-25T10:11:29.184424+0000","last_clean":"2024-04-25T10:11:29.184424+0000","last_became_active":"2024-04-25T10:11:28.179389+0000","last_became_peered":"2024-04-25T10:11:28.179389+0000","last_unstale":"2024-04-25T10:11:29.184424+0000","last_undegraded":"2024-04-25T10:11:29.184424+0000","last_fullsized":"2024-04-25T10:11:29.184424+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:08.793758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211190+0000","last_change":"2024-04-25T10:11:29.210116+0000","last_active":"2024-04-25T10:11:29.211190+0000","last_peered":"2024-04-25T10:11:29.211190+0000","last_clean":"2024-04-25T10:11:29.211190+0000","last_became_active":"2024-04-25T10:11:29.209843+0000","last_became_peered":"2024-04-25T10:11:29.209843+0000","last_unstale":"2024-04-25T10:11:29.211190+0000","last_undegraded":"2024-04-25T10:11:29.211190+0000","last_fullsized":"2024-04-25T10:11:29.211190+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:59.577147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"106'3","reported_seq":15,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.212964+0000","last_change":"2024-04-25T10:11:28.197173+0000","last_active":"2024-04-25T10:11:29.212964+0000","last_peered":"2024-04-25T10:11:29.212964+0000","last_clean":"2024-04-25T10:11:29.212964+0000","last_became_active":"2024-04-25T10:11:28.196478+0000","last_became_peered":"2024-04-25T10:11:28.196478+0000","last_unstale":"2024-04-25T10:11:29.212964+0000","last_undegraded":"2024-04-25T10:11:29.212964+0000","last_fullsized":"2024-04-25T10:11:29.212964+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:06.073089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208439+0000","last_change":"2024-04-25T10:11:29.204926+0000","last_active":"2024-04-25T10:11:29.208439+0000","last_peered":"2024-04-25T10:11:29.208439+0000","last_clean":"2024-04-25T10:11:29.208439+0000","last_became_active":"2024-04-25T10:11:29.204456+0000","last_became_peered":"2024-04-25T10:11:29.204456+0000","last_unstale":"2024-04-25T10:11:29.208439+0000","last_undegraded":"2024-04-25T10:11:29.208439+0000","last_fullsized":"2024-04-25T10:11:29.208439+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"106'6","reported_seq":18,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228827+0000","last_change":"2024-04-25T10:11:28.203606+0000","last_active":"2024-04-25T10:11:29.228827+0000","last_peered":"2024-04-25T10:11:29.228827+0000","last_clean":"2024-04-25T10:11:29.228827+0000","last_became_active":"2024-04-25T10:11:28.202865+0000","last_became_peered":"2024-04-25T10:11:28.202865+0000","last_unstale":"2024-04-25T10:11:29.228827+0000","last_undegraded":"2024-04-25T10:11:29.228827+0000","last_fullsized":"2024-04-25T10:11:29.228827+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:48.904419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237732+0000","last_change":"2024-04-25T10:11:29.219949+0000","last_active":"2024-04-25T10:11:29.237732+0000","last_peered":"2024-04-25T10:11:29.237732+0000","last_clean":"2024-04-25T10:11:29.237732+0000","last_became_active":"2024-04-25T10:11:29.217284+0000","last_became_peered":"2024-04-25T10:11:29.217284+0000","last_unstale":"2024-04-25T10:11:29.237732+0000","last_undegraded":"2024-04-25T10:11:29.237732+0000","last_fullsized":"2024-04-25T10:11:29.237732+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:07.083786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211416+0000","last_change":"2024-04-25T10:11:28.178250+0000","last_active":"2024-04-25T10:11:29.211416+0000","last_peered":"2024-04-25T10:11:29.211416+0000","last_clean":"2024-04-25T10:11:29.211416+0000","last_became_active":"2024-04-25T10:11:28.177870+0000","last_became_peered":"2024-04-25T10:11:28.177870+0000","last_unstale":"2024-04-25T10:11:29.211416+0000","last_undegraded":"2024-04-25T10:11:29.211416+0000","last_fullsized":"2024-04-25T10:11:29.211416+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.223359+0000","last_change":"2024-04-25T10:11:29.214400+0000","last_active":"2024-04-25T10:11:29.223359+0000","last_peered":"2024-04-25T10:11:29.223359+0000","last_clean":"2024-04-25T10:11:29.223359+0000","last_became_active":"2024-04-25T10:11:29.210408+0000","last_became_peered":"2024-04-25T10:11:29.210408+0000","last_unstale":"2024-04-25T10:11:29.223359+0000","last_undegraded":"2024-04-25T10:11:29.223359+0000","last_fullsized":"2024-04-25T10:11:29.223359+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.169889+0000","last_change":"2024-04-25T10:11:28.183819+0000","last_active":"2024-04-25T10:11:29.169889+0000","last_peered":"2024-04-25T10:11:29.169889+0000","last_clean":"2024-04-25T10:11:29.169889+0000","last_became_active":"2024-04-25T10:11:28.183495+0000","last_became_peered":"2024-04-25T10:11:28.183495+0000","last_unstale":"2024-04-25T10:11:29.169889+0000","last_undegraded":"2024-04-25T10:11:29.169889+0000","last_fullsized":"2024-04-25T10:11:29.169889+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:28.391764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200257+0000","last_change":"2024-04-25T10:11:29.196681+0000","last_active":"2024-04-25T10:11:29.200257+0000","last_peered":"2024-04-25T10:11:29.200257+0000","last_clean":"2024-04-25T10:11:29.200257+0000","last_became_active":"2024-04-25T10:11:29.195959+0000","last_became_peered":"2024-04-25T10:11:29.195959+0000","last_unstale":"2024-04-25T10:11:29.200257+0000","last_undegraded":"2024-04-25T10:11:29.200257+0000","last_fullsized":"2024-04-25T10:11:29.200257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:26.550654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.186660+0000","last_change":"2024-04-25T10:11:28.191807+0000","last_active":"2024-04-25T10:11:29.186660+0000","last_peered":"2024-04-25T10:11:29.186660+0000","last_clean":"2024-04-25T10:11:29.186660+0000","last_became_active":"2024-04-25T10:11:28.191530+0000","last_became_peered":"2024-04-25T10:11:28.191530+0000","last_unstale":"2024-04-25T10:11:29.186660+0000","last_undegraded":"2024-04-25T10:11:29.186660+0000","last_fullsized":"2024-04-25T10:11:29.186660+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:39.965647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.228546+0000","last_change":"2024-04-25T10:11:29.227894+0000","last_active":"2024-04-25T10:11:29.228546+0000","last_peered":"2024-04-25T10:11:29.228546+0000","last_clean":"2024-04-25T10:11:29.228546+0000","last_became_active":"2024-04-25T10:11:29.227607+0000","last_became_peered":"2024-04-25T10:11:29.227607+0000","last_unstale":"2024-04-25T10:11:29.228546+0000","last_undegraded":"2024-04-25T10:11:29.228546+0000","last_fullsized":"2024-04-25T10:11:29.228546+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:16.819178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"106'8","reported_seq":20,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.238149+0000","last_change":"2024-04-25T10:11:28.193856+0000","last_active":"2024-04-25T10:11:29.238149+0000","last_peered":"2024-04-25T10:11:29.238149+0000","last_clean":"2024-04-25T10:11:29.238149+0000","last_became_active":"2024-04-25T10:11:28.193056+0000","last_became_peered":"2024-04-25T10:11:28.193056+0000","last_unstale":"2024-04-25T10:11:29.238149+0000","last_undegraded":"2024-04-25T10:11:29.238149+0000","last_fullsized":"2024-04-25T10:11:29.238149+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:45.659307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200035+0000","last_change":"2024-04-25T10:11:29.195661+0000","last_active":"2024-04-25T10:11:29.200035+0000","last_peered":"2024-04-25T10:11:29.200035+0000","last_clean":"2024-04-25T10:11:29.200035+0000","last_became_active":"2024-04-25T10:11:29.195325+0000","last_became_peered":"2024-04-25T10:11:29.195325+0000","last_unstale":"2024-04-25T10:11:29.200035+0000","last_undegraded":"2024-04-25T10:11:29.200035+0000","last_fullsized":"2024-04-25T10:11:29.200035+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:24.723171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.216556+0000","last_change":"2024-04-25T10:11:28.202108+0000","last_active":"2024-04-25T10:11:29.216556+0000","last_peered":"2024-04-25T10:11:29.216556+0000","last_clean":"2024-04-25T10:11:29.216556+0000","last_became_active":"2024-04-25T10:11:28.188447+0000","last_became_peered":"2024-04-25T10:11:28.188447+0000","last_unstale":"2024-04-25T10:11:29.216556+0000","last_undegraded":"2024-04-25T10:11:29.216556+0000","last_fullsized":"2024-04-25T10:11:29.216556+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:48.768488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.209861+0000","last_change":"2024-04-25T10:11:29.200625+0000","last_active":"2024-04-25T10:11:29.209861+0000","last_peered":"2024-04-25T10:11:29.209861+0000","last_clean":"2024-04-25T10:11:29.209861+0000","last_became_active":"2024-04-25T10:11:29.199810+0000","last_became_peered":"2024-04-25T10:11:29.199810+0000","last_unstale":"2024-04-25T10:11:29.209861+0000","last_undegraded":"2024-04-25T10:11:29.209861+0000","last_fullsized":"2024-04-25T10:11:29.209861+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:29.402461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"106'1","reported_seq":13,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.225243+0000","last_change":"2024-04-25T10:11:28.196477+0000","last_active":"2024-04-25T10:11:29.225243+0000","last_peered":"2024-04-25T10:11:29.225243+0000","last_clean":"2024-04-25T10:11:29.225243+0000","last_became_active":"2024-04-25T10:11:28.195761+0000","last_became_peered":"2024-04-25T10:11:28.195761+0000","last_unstale":"2024-04-25T10:11:29.225243+0000","last_undegraded":"2024-04-25T10:11:29.225243+0000","last_fullsized":"2024-04-25T10:11:29.225243+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:27.172729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.208208+0000","last_change":"2024-04-25T10:11:29.203753+0000","last_active":"2024-04-25T10:11:29.208208+0000","last_peered":"2024-04-25T10:11:29.208208+0000","last_clean":"2024-04-25T10:11:29.208208+0000","last_became_active":"2024-04-25T10:11:29.202435+0000","last_became_peered":"2024-04-25T10:11:29.202435+0000","last_unstale":"2024-04-25T10:11:29.208208+0000","last_undegraded":"2024-04-25T10:11:29.208208+0000","last_fullsized":"2024-04-25T10:11:29.208208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.226518+0000","last_change":"2024-04-25T10:11:28.210024+0000","last_active":"2024-04-25T10:11:29.226518+0000","last_peered":"2024-04-25T10:11:29.226518+0000","last_clean":"2024-04-25T10:11:29.226518+0000","last_became_active":"2024-04-25T10:11:28.209341+0000","last_became_peered":"2024-04-25T10:11:28.209341+0000","last_unstale":"2024-04-25T10:11:29.226518+0000","last_undegraded":"2024-04-25T10:11:29.226518+0000","last_fullsized":"2024-04-25T10:11:29.226518+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:27.172729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.206257+0000","last_change":"2024-04-25T10:11:29.203454+0000","last_active":"2024-04-25T10:11:29.206257+0000","last_peered":"2024-04-25T10:11:29.206257+0000","last_clean":"2024-04-25T10:11:29.206257+0000","last_became_active":"2024-04-25T10:11:29.203148+0000","last_became_peered":"2024-04-25T10:11:29.203148+0000","last_unstale":"2024-04-25T10:11:29.206257+0000","last_undegraded":"2024-04-25T10:11:29.206257+0000","last_fullsized":"2024-04-25T10:11:29.206257+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:29.277022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200148+0000","last_change":"2024-04-25T10:11:28.196014+0000","last_active":"2024-04-25T10:11:29.200148+0000","last_peered":"2024-04-25T10:11:29.200148+0000","last_clean":"2024-04-25T10:11:29.200148+0000","last_became_active":"2024-04-25T10:11:28.195531+0000","last_became_peered":"2024-04-25T10:11:28.195531+0000","last_unstale":"2024-04-25T10:11:29.200148+0000","last_undegraded":"2024-04-25T10:11:29.200148+0000","last_fullsized":"2024-04-25T10:11:29.200148+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:39.108320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.207417+0000","last_change":"2024-04-25T10:11:29.206432+0000","last_active":"2024-04-25T10:11:29.207417+0000","last_peered":"2024-04-25T10:11:29.207417+0000","last_clean":"2024-04-25T10:11:29.207417+0000","last_became_active":"2024-04-25T10:11:29.206133+0000","last_became_peered":"2024-04-25T10:11:29.206133+0000","last_unstale":"2024-04-25T10:11:29.207417+0000","last_undegraded":"2024-04-25T10:11:29.207417+0000","last_fullsized":"2024-04-25T10:11:29.207417+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:51.410363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.187232+0000","last_change":"2024-04-25T10:11:28.189322+0000","last_active":"2024-04-25T10:11:29.187232+0000","last_peered":"2024-04-25T10:11:29.187232+0000","last_clean":"2024-04-25T10:11:29.187232+0000","last_became_active":"2024-04-25T10:11:28.188916+0000","last_became_peered":"2024-04-25T10:11:28.188916+0000","last_unstale":"2024-04-25T10:11:29.187232+0000","last_undegraded":"2024-04-25T10:11:29.187232+0000","last_fullsized":"2024-04-25T10:11:29.187232+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:31.743669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.192300+0000","last_change":"2024-04-25T10:11:29.189753+0000","last_active":"2024-04-25T10:11:29.192300+0000","last_peered":"2024-04-25T10:11:29.192300+0000","last_clean":"2024-04-25T10:11:29.192300+0000","last_became_active":"2024-04-25T10:11:29.189358+0000","last_became_peered":"2024-04-25T10:11:29.189358+0000","last_unstale":"2024-04-25T10:11:29.192300+0000","last_undegraded":"2024-04-25T10:11:29.192300+0000","last_fullsized":"2024-04-25T10:11:29.192300+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:28.183426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.174269+0000","last_change":"2024-04-25T10:11:28.208134+0000","last_active":"2024-04-25T10:11:29.174269+0000","last_peered":"2024-04-25T10:11:29.174269+0000","last_clean":"2024-04-25T10:11:29.174269+0000","last_became_active":"2024-04-25T10:11:28.207490+0000","last_became_peered":"2024-04-25T10:11:28.207490+0000","last_unstale":"2024-04-25T10:11:29.174269+0000","last_undegraded":"2024-04-25T10:11:29.174269+0000","last_fullsized":"2024-04-25T10:11:29.174269+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213677+0000","last_change":"2024-04-25T10:11:29.213173+0000","last_active":"2024-04-25T10:11:29.213677+0000","last_peered":"2024-04-25T10:11:29.213677+0000","last_clean":"2024-04-25T10:11:29.213677+0000","last_became_active":"2024-04-25T10:11:29.212887+0000","last_became_peered":"2024-04-25T10:11:29.212887+0000","last_unstale":"2024-04-25T10:11:29.213677+0000","last_undegraded":"2024-04-25T10:11:29.213677+0000","last_fullsized":"2024-04-25T10:11:29.213677+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:26.754293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188188+0000","last_change":"2024-04-25T10:11:28.191897+0000","last_active":"2024-04-25T10:11:29.188188+0000","last_peered":"2024-04-25T10:11:29.188188+0000","last_clean":"2024-04-25T10:11:29.188188+0000","last_became_active":"2024-04-25T10:11:28.190329+0000","last_became_peered":"2024-04-25T10:11:28.190329+0000","last_unstale":"2024-04-25T10:11:29.188188+0000","last_undegraded":"2024-04-25T10:11:29.188188+0000","last_fullsized":"2024-04-25T10:11:29.188188+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:50.564628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.201078+0000","last_change":"2024-04-25T10:11:29.199211+0000","last_active":"2024-04-25T10:11:29.201078+0000","last_peered":"2024-04-25T10:11:29.201078+0000","last_clean":"2024-04-25T10:11:29.201078+0000","last_became_active":"2024-04-25T10:11:29.198914+0000","last_became_peered":"2024-04-25T10:11:29.198914+0000","last_unstale":"2024-04-25T10:11:29.201078+0000","last_undegraded":"2024-04-25T10:11:29.201078+0000","last_fullsized":"2024-04-25T10:11:29.201078+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:34.282997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211138+0000","last_change":"2024-04-25T10:11:28.178108+0000","last_active":"2024-04-25T10:11:29.211138+0000","last_peered":"2024-04-25T10:11:29.211138+0000","last_clean":"2024-04-25T10:11:29.211138+0000","last_became_active":"2024-04-25T10:11:28.177801+0000","last_became_peered":"2024-04-25T10:11:28.177801+0000","last_unstale":"2024-04-25T10:11:29.211138+0000","last_undegraded":"2024-04-25T10:11:29.211138+0000","last_fullsized":"2024-04-25T10:11:29.211138+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:45.659307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.196230+0000","last_change":"2024-04-25T10:11:29.194067+0000","last_active":"2024-04-25T10:11:29.196230+0000","last_peered":"2024-04-25T10:11:29.196230+0000","last_clean":"2024-04-25T10:11:29.196230+0000","last_became_active":"2024-04-25T10:11:29.192610+0000","last_became_peered":"2024-04-25T10:11:29.192610+0000","last_unstale":"2024-04-25T10:11:29.196230+0000","last_undegraded":"2024-04-25T10:11:29.196230+0000","last_fullsized":"2024-04-25T10:11:29.196230+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.184904+0000","last_change":"2024-04-25T10:11:28.194986+0000","last_active":"2024-04-25T10:11:29.184904+0000","last_peered":"2024-04-25T10:11:29.184904+0000","last_clean":"2024-04-25T10:11:29.184904+0000","last_became_active":"2024-04-25T10:11:28.194593+0000","last_became_peered":"2024-04-25T10:11:28.194593+0000","last_unstale":"2024-04-25T10:11:29.184904+0000","last_undegraded":"2024-04-25T10:11:29.184904+0000","last_fullsized":"2024-04-25T10:11:29.184904+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:03.949230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.192166+0000","last_change":"2024-04-25T10:11:29.190216+0000","last_active":"2024-04-25T10:11:29.192166+0000","last_peered":"2024-04-25T10:11:29.192166+0000","last_clean":"2024-04-25T10:11:29.192166+0000","last_became_active":"2024-04-25T10:11:29.189320+0000","last_became_peered":"2024-04-25T10:11:29.189320+0000","last_unstale":"2024-04-25T10:11:29.192166+0000","last_undegraded":"2024-04-25T10:11:29.192166+0000","last_fullsized":"2024-04-25T10:11:29.192166+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.180615+0000","last_change":"2024-04-25T10:11:28.208837+0000","last_active":"2024-04-25T10:11:29.180615+0000","last_peered":"2024-04-25T10:11:29.180615+0000","last_clean":"2024-04-25T10:11:29.180615+0000","last_became_active":"2024-04-25T10:11:28.206933+0000","last_became_peered":"2024-04-25T10:11:28.206933+0000","last_unstale":"2024-04-25T10:11:29.180615+0000","last_undegraded":"2024-04-25T10:11:29.180615+0000","last_fullsized":"2024-04-25T10:11:29.180615+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:51.292549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237665+0000","last_change":"2024-04-25T10:11:29.219389+0000","last_active":"2024-04-25T10:11:29.237665+0000","last_peered":"2024-04-25T10:11:29.237665+0000","last_clean":"2024-04-25T10:11:29.237665+0000","last_became_active":"2024-04-25T10:11:29.218517+0000","last_became_peered":"2024-04-25T10:11:29.218517+0000","last_unstale":"2024-04-25T10:11:29.237665+0000","last_undegraded":"2024-04-25T10:11:29.237665+0000","last_fullsized":"2024-04-25T10:11:29.237665+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:51.640908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.182048+0000","last_change":"2024-04-25T10:11:28.208505+0000","last_active":"2024-04-25T10:11:29.182048+0000","last_peered":"2024-04-25T10:11:29.182048+0000","last_clean":"2024-04-25T10:11:29.182048+0000","last_became_active":"2024-04-25T10:11:28.208198+0000","last_became_peered":"2024-04-25T10:11:28.208198+0000","last_unstale":"2024-04-25T10:11:29.182048+0000","last_undegraded":"2024-04-25T10:11:29.182048+0000","last_fullsized":"2024-04-25T10:11:29.182048+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.213456+0000","last_change":"2024-04-25T10:11:29.210080+0000","last_active":"2024-04-25T10:11:29.213456+0000","last_peered":"2024-04-25T10:11:29.213456+0000","last_clean":"2024-04-25T10:11:29.213456+0000","last_became_active":"2024-04-25T10:11:29.209764+0000","last_became_peered":"2024-04-25T10:11:29.209764+0000","last_unstale":"2024-04-25T10:11:29.213456+0000","last_undegraded":"2024-04-25T10:11:29.213456+0000","last_fullsized":"2024-04-25T10:11:29.213456+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:27.677352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.170987+0000","last_change":"2024-04-25T10:11:28.204006+0000","last_active":"2024-04-25T10:11:29.170987+0000","last_peered":"2024-04-25T10:11:29.170987+0000","last_clean":"2024-04-25T10:11:29.170987+0000","last_became_active":"2024-04-25T10:11:28.203010+0000","last_became_peered":"2024-04-25T10:11:28.203010+0000","last_unstale":"2024-04-25T10:11:29.170987+0000","last_undegraded":"2024-04-25T10:11:29.170987+0000","last_fullsized":"2024-04-25T10:11:29.170987+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:51.454853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.211274+0000","last_change":"2024-04-25T10:11:29.208603+0000","last_active":"2024-04-25T10:11:29.211274+0000","last_peered":"2024-04-25T10:11:29.211274+0000","last_clean":"2024-04-25T10:11:29.211274+0000","last_became_active":"2024-04-25T10:11:29.206282+0000","last_became_peered":"2024-04-25T10:11:29.206282+0000","last_unstale":"2024-04-25T10:11:29.211274+0000","last_undegraded":"2024-04-25T10:11:29.211274+0000","last_fullsized":"2024-04-25T10:11:29.211274+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:21.763184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.187350+0000","last_change":"2024-04-25T10:11:28.196162+0000","last_active":"2024-04-25T10:11:29.187350+0000","last_peered":"2024-04-25T10:11:29.187350+0000","last_clean":"2024-04-25T10:11:29.187350+0000","last_became_active":"2024-04-25T10:11:28.195723+0000","last_became_peered":"2024-04-25T10:11:28.195723+0000","last_unstale":"2024-04-25T10:11:29.187350+0000","last_undegraded":"2024-04-25T10:11:29.187350+0000","last_fullsized":"2024-04-25T10:11:29.187350+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:51.936068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.230831+0000","last_change":"2024-04-25T10:11:29.229517+0000","last_active":"2024-04-25T10:11:29.230831+0000","last_peered":"2024-04-25T10:11:29.230831+0000","last_clean":"2024-04-25T10:11:29.230831+0000","last_became_active":"2024-04-25T10:11:29.228979+0000","last_became_peered":"2024-04-25T10:11:29.228979+0000","last_unstale":"2024-04-25T10:11:29.230831+0000","last_undegraded":"2024-04-25T10:11:29.230831+0000","last_fullsized":"2024-04-25T10:11:29.230831+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:55.386528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.174770+0000","last_change":"2024-04-25T10:11:28.203907+0000","last_active":"2024-04-25T10:11:29.174770+0000","last_peered":"2024-04-25T10:11:29.174770+0000","last_clean":"2024-04-25T10:11:29.174770+0000","last_became_active":"2024-04-25T10:11:28.203179+0000","last_became_peered":"2024-04-25T10:11:28.203179+0000","last_unstale":"2024-04-25T10:11:29.174770+0000","last_undegraded":"2024-04-25T10:11:29.174770+0000","last_fullsized":"2024-04-25T10:11:29.174770+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:44.265144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.237712+0000","last_change":"2024-04-25T10:11:29.226923+0000","last_active":"2024-04-25T10:11:29.237712+0000","last_peered":"2024-04-25T10:11:29.237712+0000","last_clean":"2024-04-25T10:11:29.237712+0000","last_became_active":"2024-04-25T10:11:29.226136+0000","last_became_peered":"2024-04-25T10:11:29.226136+0000","last_unstale":"2024-04-25T10:11:29.237712+0000","last_undegraded":"2024-04-25T10:11:29.237712+0000","last_fullsized":"2024-04-25T10:11:29.237712+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:31.451474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.188320+0000","last_change":"2024-04-25T10:11:28.194380+0000","last_active":"2024-04-25T10:11:29.188320+0000","last_peered":"2024-04-25T10:11:29.188320+0000","last_clean":"2024-04-25T10:11:29.188320+0000","last_became_active":"2024-04-25T10:11:28.194012+0000","last_became_peered":"2024-04-25T10:11:28.194012+0000","last_unstale":"2024-04-25T10:11:29.188320+0000","last_undegraded":"2024-04-25T10:11:29.188320+0000","last_fullsized":"2024-04-25T10:11:29.188320+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:40.045808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.236114+0000","last_change":"2024-04-25T10:11:29.208192+0000","last_active":"2024-04-25T10:11:29.236114+0000","last_peered":"2024-04-25T10:11:29.236114+0000","last_clean":"2024-04-25T10:11:29.236114+0000","last_became_active":"2024-04-25T10:11:29.207438+0000","last_became_peered":"2024-04-25T10:11:29.207438+0000","last_unstale":"2024-04-25T10:11:29.236114+0000","last_undegraded":"2024-04-25T10:11:29.236114+0000","last_fullsized":"2024-04-25T10:11:29.236114+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:28.148456+0000","last_clean_scrub_stamp":"2024-04-25T10:11:28.148456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:49.779185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"106'2","reported_seq":14,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.200892+0000","last_change":"2024-04-25T10:11:28.189384+0000","last_active":"2024-04-25T10:11:29.200892+0000","last_peered":"2024-04-25T10:11:29.200892+0000","last_clean":"2024-04-25T10:11:29.200892+0000","last_became_active":"2024-04-25T10:11:28.189004+0000","last_became_peered":"2024-04-25T10:11:28.189004+0000","last_unstale":"2024-04-25T10:11:29.200892+0000","last_undegraded":"2024-04-25T10:11:29.200892+0000","last_fullsized":"2024-04-25T10:11:29.200892+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:27.137758+0000","last_clean_scrub_stamp":"2024-04-25T10:11:27.137758+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:59.970417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-25T10:11:29.178644+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:11:29.178644+0000","last_peered":"2024-04-25T10:11:29.178644+0000","last_clean":"2024-04-25T10:11:29.178644+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:11:29.178644+0000","last_undegraded":"2024-04-25T10:11:29.178644+0000","last_fullsized":"2024-04-25T10:11:29.178644+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1916928,"data_stored":1201687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1335296,"data_stored":613375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":603992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32480,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339360,"statfs":{"total":96636764160,"available":96603504640,"internally_reserved":0,"allocated":1323008,"data_stored":611251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":611741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32476,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339364,"statfs":{"total":96636764160,"available":96603508736,"internally_reserved":0,"allocated":1318912,"data_stored":611193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1302528,"data_stored":603992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1925120,"data_stored":1203229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:32.546 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:32.547 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:32.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 Processor -- start 2024-04-25T10:11:32.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 -- start start 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cdc15ca90 0x7f1cdc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15f360 con 0x7f1cdc0cc130 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc15f640 con 0x7f1cdc06ac90 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce30fa640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc15ff70 con 0x7f1cdc15ca90 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1cdbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1cdc15ca90 0x7f1cdc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce1697640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1cdc0cc130 0x7f1cdc0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57830/0 (socket says 172.21.15.82:57830) 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1ce1697640 1 -- 172.21.15.82:0/2455215909 learned_addr learned my addr 172.21.15.82:0/2455215909 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1cdbfff640 1 -- 172.21.15.82:0/2455215909 >> v1:172.21.15.196:6789/0 conn(0x7f1cdc06ac90 legacy=0x7f1cdc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1cdbfff640 1 -- 172.21.15.82:0/2455215909 >> v1:172.21.15.82:6789/0 conn(0x7f1cdc0cc130 legacy=0x7f1cdc0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1cdbfff640 1 -- 172.21.15.82:0/2455215909 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc15f920 con 0x7f1cdc15ca90 2024-04-25T10:11:32.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.713+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/2455215909 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053427058 0 0) 0x7f1cdc15f360 con 0x7f1cdc0cc130 2024-04-25T10:11:32.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1cdbfff640 1 --2- 172.21.15.82:0/2455215909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0d0c10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097c0 tx=0x7f1cd001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2480a260e046fa22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/2455215909 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00b2070 con 0x7f1cdc15ca90 2024-04-25T10:11:32.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/2455215909 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd00ad680 con 0x7f1cdc15ca90 2024-04-25T10:11:32.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1cdb7fe640 1 -- 172.21.15.82:0/2455215909 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd00ada50 con 0x7f1cdc15ca90 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- 172.21.15.82:0/2455215909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 msgr2=0x7f1cdc0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/2455215909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0d0c10 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f1cd00097c0 tx=0x7f1cd001c470 comp rx=0 tx=0).stop 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- 172.21.15.82:0/2455215909 shutdown_connections 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/2455215909 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0d0c10 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- 172.21.15.82:0/2455215909 >> 172.21.15.82:0/2455215909 conn(0x7f1cdc099a70 msgr2=0x7f1cdc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- 172.21.15.82:0/2455215909 shutdown_connections 2024-04-25T10:11:32.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- 172.21.15.82:0/2455215909 wait complete. 2024-04-25T10:11:32.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 Processor -- start 2024-04-25T10:11:32.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.717+0000 7f1ce30fa640 1 -- start start 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc06ac90 0x7f1cdc148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc0cc130 0x7f1cdc0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc169a00 con 0x7f1cdc15ca90 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc169880 con 0x7f1cdc0cc130 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc169b80 con 0x7f1cdc06ac90 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45808/0 (socket says 172.21.15.82:45808) 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 -- 172.21.15.82:0/858653883 learned_addr learned my addr 172.21.15.82:0/858653883 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 -- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc06ac90 msgr2=0x7f1cdc148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce0e96640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc06ac90 0x7f1cdc148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cdbfff640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc0cc130 0x7f1cdc0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc06ac90 0x7f1cdc148e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 -- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc0cc130 msgr2=0x7f1cdc0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc0cc130 0x7f1cdc0b30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 -- 172.21.15.82:0/858653883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc1493a0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce1697640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c750 tx=0x7f1cd400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4018070 con 0x7f1cdc15ca90 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd400a640 con 0x7f1cdc15ca90 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0b39e0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd400ce60 con 0x7f1cdc15ca90 2024-04-25T10:11:32.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc150e50 con 0x7f1cdc15ca90 2024-04-25T10:11:32.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc04efa0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1cd401b070 con 0x7f1cdc15ca90 2024-04-25T10:11:32.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 0x7f1cc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce0e96640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 0x7f1cc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1cd4084760 con 0x7f1cdc15ca90 2024-04-25T10:11:32.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.721+0000 7f1ce0e96640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 0x7f1cc8069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc000f70 tx=0x7f1ccc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.725+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1cd40555e0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.945+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1cdc06b7c0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.949+0000 7f1cd97fa640 1 -- 172.21.15.82:0/858653883 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1cd4054ee0 con 0x7f1cdc15ca90 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:32.954 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 msgr2=0x7f1cc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 0x7f1cc8069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc000f70 tx=0x7f1ccc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 msgr2=0x7f1cdc0b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400c750 tx=0x7f1cd400cc10 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 shutdown_connections 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1cc8066f60 0x7f1cc8069420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1cdc15ca90 0x7f1cdc0b3fd0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1cdc0cc130 0x7f1cdc0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 --2- 172.21.15.82:0/858653883 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1cdc06ac90 0x7f1cdc148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 >> 172.21.15.82:0/858653883 conn(0x7f1cdc099a70 msgr2=0x7f1cdc15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 shutdown_connections 2024-04-25T10:11:32.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:32.953+0000 7f1ce30fa640 1 -- 172.21.15.82:0/858653883 wait complete. 2024-04-25T10:11:32.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:33.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 Processor -- start 2024-04-25T10:11:33.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 -- start start 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 0x7f7cb40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb406be60 con 0x7f7cb415da80 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb415de60 con 0x7f7cb40d1cd0 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cba888640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406c440 con 0x7f7cb40d07f0 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7cb415da80 0x7f7cb40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57840/0 (socket says 172.21.15.82:57840) 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb3fff640 1 -- 172.21.15.82:0/3434112174 learned_addr learned my addr 172.21.15.82:0/3434112174 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb37fe640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 0x7f7cb40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116877655 0 0) 0x7f7cb406be60 con 0x7f7cb415da80 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c9c003530 con 0x7f7cb415da80 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568559963 0 0) 0x7f7c9c003530 con 0x7f7cb415da80 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 msgr2=0x7f7cb40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 0x7f7cb40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 >> v1:172.21.15.196:6789/0 conn(0x7f7cb40d1cd0 legacy=0x7f7cb406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40d2450 con 0x7f7cb415da80 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7ca4001680 con 0x7f7cb415da80 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 >> v1:172.21.15.82:6789/0 conn(0x7f7cb415da80 legacy=0x7f7cb40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c003a60 0x7f7c9c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 0x7f7c9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c004d50 0x7f7c9c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb406c440 con 0x7f7c9c004d50 2024-04-25T10:11:33.094 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c00bb10 con 0x7f7c9c003a60 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c00bd10 con 0x7f7cb40d07f0 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c004d50 0x7f7c9c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb3fff640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c003a60 0x7f7c9c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 -- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 msgr2=0x7f7c9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb40d07f0 0x7f7c9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 -- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c003a60 msgr2=0x7f7c9c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7c9c003a60 0x7f7c9c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.089+0000 7f7cb8e25640 1 -- 172.21.15.82:0/3434112174 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c003530 con 0x7f7c9c004d50 2024-04-25T10:11:33.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c004d50 0x7f7c9c0051d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7cac001e40 tx=0x7f7cac01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b1cfd5aec1b0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac001a90 con 0x7f7c9c004d50 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cac003b80 con 0x7f7c9c004d50 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb2ffd640 1 -- 172.21.15.82:0/3434112174 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac005080 con 0x7f7c9c004d50 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- 172.21.15.82:0/3434112174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c004d50 msgr2=0x7f7c9c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 --2- 172.21.15.82:0/3434112174 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7c9c004d50 0x7f7c9c0051d0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7cac001e40 tx=0x7f7cac01c760 comp rx=0 tx=0).stop 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb8e25640 1 -- 172.21.15.82:0/3434112174 reap_dead start 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- 172.21.15.82:0/3434112174 shutdown_connections 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- 172.21.15.82:0/3434112174 >> 172.21.15.82:0/3434112174 conn(0x7f7cb408b000 msgr2=0x7f7cb40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- 172.21.15.82:0/3434112174 shutdown_connections 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- 172.21.15.82:0/3434112174 wait complete. 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 Processor -- start 2024-04-25T10:11:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- start start 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 0x7f7cb40a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 0x7f7cb40b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40d26f0 con 0x7f7cb40caa80 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40d2160 con 0x7f7cb415c1e0 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cba888640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40d2870 con 0x7f7cb415a3c0 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45820/0 (socket says 172.21.15.82:45820) 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 -- 172.21.15.82:0/1100095159 learned_addr learned my addr 172.21.15.82:0/1100095159 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb3fff640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 0x7f7cb40a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 0x7f7cb40b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 -- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 msgr2=0x7f7cb40a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 0x7f7cb40a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 -- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 msgr2=0x7f7cb40b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.093+0000 7f7cb37fe640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 0x7f7cb40b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb37fe640 1 -- 172.21.15.82:0/1100095159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40a12b0 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb3fff640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 0x7f7cb40a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb8e25640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 0x7f7cb40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb37fe640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e820 tx=0x7f7ca800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca8017070 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca800c940 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40cb880 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca800cd10 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb40a0fd0 con 0x7f7cb40caa80 2024-04-25T10:11:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb404efa0 con 0x7f7cb40caa80 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ca801a070 con 0x7f7cb40caa80 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 0x7f7ca0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb3fff640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 0x7f7ca0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ca80852e0 con 0x7f7cb40caa80 2024-04-25T10:11:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.097+0000 7f7cb3fff640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 0x7f7ca0069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4001ec0 tx=0x7f7ca40014d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.101+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ca80561f0 con 0x7f7cb40caa80 2024-04-25T10:11:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.289+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cb415c5c0 con 0x7f7cb40caa80 2024-04-25T10:11:33.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.289+0000 7f7cb0ff9640 1 -- 172.21.15.82:0/1100095159 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f7ca8055af0 con 0x7f7cb40caa80 2024-04-25T10:11:33.293 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:33.293 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:11:30.539889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8769":{"gid":8769,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3162787996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3162787996},{"type":"v1","addr":"172.21.15.82:6837","nonce":3162787996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:33.293 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 75 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 msgr2=0x7f7ca0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 0x7f7ca0069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7ca4001ec0 tx=0x7f7ca40014d0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 msgr2=0x7f7cb415bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7ca800e820 tx=0x7f7ca800ece0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 shutdown_connections 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ca0066f60 0x7f7ca0069420 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7cb415c1e0 0x7f7cb40b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7cb40caa80 0x7f7cb415bca0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 --2- 172.21.15.82:0/1100095159 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7cb415a3c0 0x7f7cb40a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 >> 172.21.15.82:0/1100095159 conn(0x7f7cb408b000 msgr2=0x7f7cb40b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 shutdown_connections 2024-04-25T10:11:33.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.293+0000 7f7cba888640 1 -- 172.21.15.82:0/1100095159 wait complete. 2024-04-25T10:11:33.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:28.528473+0000', 'modified': '2024-04-25T10:11:30.539889+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8769}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8769': {'gid': 8769, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/3162787996', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 3162787996}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3162787996}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:33.713 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:33.713 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:11:33.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 Processor -- start 2024-04-25T10:11:33.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 -- start start 2024-04-25T10:11:33.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa0e806fc50 0x7fa0e806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0e80cff50 0x7fa0e815d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e806b710 con 0x7fa0e806b1d0 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e8161560 con 0x7fa0e80cff50 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0ed457640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e81617b0 con 0x7fa0e806fc50 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0e80cff50 0x7fa0e815d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa0e80cff50 0x7fa0e815d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45826/0 (socket says 172.21.15.82:45826) 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e67fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa0e806b1d0 0x7fa0e80cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57856/0 (socket says 172.21.15.82:57856) 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 -- 172.21.15.82:0/4024258694 learned_addr learned my addr 172.21.15.82:0/4024258694 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/4024258694 >> v2:172.21.15.196:3300/0 conn(0x7fa0e806fc50 0x7fa0e806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 -- 172.21.15.82:0/4024258694 >> v2:172.21.15.196:3300/0 conn(0x7fa0e806fc50 msgr2=0x7fa0e806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/4024258694 >> v2:172.21.15.196:3300/0 conn(0x7fa0e806fc50 0x7fa0e806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 -- 172.21.15.82:0/4024258694 >> v1:172.21.15.82:6789/0 conn(0x7fa0e806b1d0 legacy=0x7fa0e80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.841+0000 7fa0e77fe640 1 -- 172.21.15.82:0/4024258694 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8161a90 con 0x7fa0e80cff50 2024-04-25T10:11:33.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/4024258694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815d180 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8099fa0 tx=0x7fa0d801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d29d4f3f6111c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0e5ffb640 1 -- 172.21.15.82:0/4024258694 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80ab430 con 0x7fa0e80cff50 2024-04-25T10:11:33.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0e5ffb640 1 -- 172.21.15.82:0/4024258694 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80b0070 con 0x7fa0e80cff50 2024-04-25T10:11:33.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0e5ffb640 1 -- 172.21.15.82:0/4024258694 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80ab7f0 con 0x7fa0e80cff50 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- 172.21.15.82:0/4024258694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 msgr2=0x7fa0e815d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 --2- 172.21.15.82:0/4024258694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815d180 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8099fa0 tx=0x7fa0d801c520 comp rx=0 tx=0).stop 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- 172.21.15.82:0/4024258694 shutdown_connections 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 --2- 172.21.15.82:0/4024258694 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815d180 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 --2- 172.21.15.82:0/4024258694 >> v2:172.21.15.196:3300/0 conn(0x7fa0e806fc50 0x7fa0e806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- 172.21.15.82:0/4024258694 >> 172.21.15.82:0/4024258694 conn(0x7fa0e8099a70 msgr2=0x7fa0e80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- 172.21.15.82:0/4024258694 shutdown_connections 2024-04-25T10:11:33.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- 172.21.15.82:0/4024258694 wait complete. 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 Processor -- start 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.845+0000 7fa0ed457640 1 -- start start 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e806b1d0 0x7fa0e80b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0e806fc50 0x7fa0e814b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8162410 con 0x7fa0e80cff50 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8162290 con 0x7fa0e806b1d0 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80cc6e0 con 0x7fa0e806fc50 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45836/0 (socket says 172.21.15.82:45836) 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 -- 172.21.15.82:0/1689632350 learned_addr learned my addr 172.21.15.82:0/1689632350 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0e806fc50 0x7fa0e814b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e806b1d0 0x7fa0e80b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 -- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0e806fc50 msgr2=0x7fa0e814b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0e806fc50 0x7fa0e814b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 -- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e806b1d0 msgr2=0x7fa0e80b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e806b1d0 0x7fa0e80b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 -- 172.21.15.82:0/1689632350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e814bef0 con 0x7fa0e80cff50 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801a150 tx=0x7fa0d801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 -- 172.21.15.82:0/1689632350 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80be5f0 con 0x7fa0e80cff50 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 -- 172.21.15.82:0/1689632350 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80be790 con 0x7fa0e80cff50 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 -- 172.21.15.82:0/1689632350 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80beb80 con 0x7fa0e80cff50 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e80b0d40 con 0x7fa0e80cff50 2024-04-25T10:11:33.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e80b1020 con 0x7fa0e80cff50 2024-04-25T10:11:33.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 -- 172.21.15.82:0/1689632350 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0d80b43f0 con 0x7fa0e80cff50 2024-04-25T10:11:33.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 0x7fa0c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.854 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 0x7fa0c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.849+0000 7fa0df7fe640 1 -- 172.21.15.82:0/1689632350 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0d80a8030 con 0x7fa0e80cff50 2024-04-25T10:11:33.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 0x7fa0c8069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0003fd0 tx=0x7fa0e00039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 Processor -- start 2024-04-25T10:11:33.855 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- start start 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da308d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da309240 0x5626da2be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da1a65f0 con 0x5626da2bef30 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da10dda0 con 0x5626da309240 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626da096150 con 0x5626da308920 2024-04-25T10:11:33.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45846/0 (socket says 172.21.15.82:45846) 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 -- 172.21.15.82:0/3035049311 learned_addr learned my addr 172.21.15.82:0/3035049311 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da308d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da309240 0x5626da2be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 -- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 msgr2=0x5626da308d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da308d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 -- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da309240 msgr2=0x5626da2be9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da309240 0x5626da2be9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 -- 172.21.15.82:0/3035049311 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626da0afca0 con 0x5626da2bef30 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80f8240 tx=0x7fa0d80ef9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c7e897b9d05f73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0dcff9640 1 -- 172.21.15.82:0/3035049311 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80f41b0 con 0x5626da2bef30 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0dcff9640 1 -- 172.21.15.82:0/3035049311 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80b0070 con 0x5626da2bef30 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0dcff9640 1 -- 172.21.15.82:0/3035049311 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d80f8770 con 0x5626da2bef30 2024-04-25T10:11:33.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3035049311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 msgr2=0x5626da2bf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80f8240 tx=0x7fa0d80ef9c0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3035049311 shutdown_connections 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2bf310 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da309240 0x5626da2be9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3035049311 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da308d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3035049311 >> 172.21.15.82:0/3035049311 conn(0x5626da2c52a0 msgr2=0x5626da2bd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3035049311 shutdown_connections 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3035049311 wait complete. 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 -- start start 2024-04-25T10:11:33.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.853+0000 7fa0f05c4000 1 Processor -- start 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 0x5626da2c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da0c8990 con 0x5626da2bef30 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da109e40 con 0x5626da308920 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626da099900 con 0x5626da309240 2024-04-25T10:11:33.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 10 client.0 _ll_register_callbacks cb 0x5626da1c11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45862/0 (socket says 172.21.15.82:45862) 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 -- 172.21.15.82:0/4160736885 learned_addr learned my addr 172.21.15.82:0/4160736885 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 0x5626da2c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 -- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 msgr2=0x5626da2c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 0x5626da2c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 -- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 msgr2=0x5626da3101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 -- 172.21.15.82:0/4160736885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626da0c6630 con 0x5626da2bef30 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 0x5626da2c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0016770 tx=0x7fa0e0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb1e549ddc4fbc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0014a20 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0e0014bc0 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0d80b0070 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0e0014dc0 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 10 client.8913 fetch_fsmap learned FSMap version 75 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5626da0963a0 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d80b0070 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0e0021410 con 0x5626da2bef30 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 10 client.8913 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 10 client.8913 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 20 client.8913 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da2c34b0 0x5626da2c3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 4 client.8913 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:33.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626da0a9a50 con 0x5626da2c34b0 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da2c34b0 0x5626da2c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 10 client.8913 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da2c34b0 0x5626da2c3890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7711a72a73a6d42a server_cookie=6236a6ce5b294e88 in_seq=0 out_seq=0 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/4160736885 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626da0a9a50 con 0x5626da2c34b0 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 10 client.8913 handle_command_reply: tid=0 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 client.8913 shutdown 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 msgr2=0x5626da30fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0016770 tx=0x7fa0e0004ac0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 shutdown_connections 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da2c34b0 0x5626da2c3890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da309240 0x5626da2c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0c5ffb640 0 client.8913 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:11:33.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/4160736885 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da30fc80 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 >> 172.21.15.82:0/4160736885 conn(0x5626da2c52a0 msgr2=0x5626da2bd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 shutdown_connections 2024-04-25T10:11:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 1 -- 172.21.15.82:0/4160736885 wait complete. 2024-04-25T10:11:33.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.857+0000 7fa0f05c4000 20 client.8913 trim_cache size 0 max 16384 2024-04-25T10:11:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 Processor -- start 2024-04-25T10:11:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 -- start start 2024-04-25T10:11:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da2c52a0 0x5626da2c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e8202460 con 0x5626da2bef30 2024-04-25T10:11:33.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80b04a0 con 0x5626da308920 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80b0aa0 con 0x5626da2c52a0 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45876/0 (socket says 172.21.15.82:45876) 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.873+0000 7fa0e67fc640 1 -- 172.21.15.82:0/3300172609 learned_addr learned my addr 172.21.15.82:0/3300172609 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da2c52a0 0x5626da2c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da2c52a0 msgr2=0x5626da2c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da2c52a0 0x5626da2c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 msgr2=0x5626da3127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3127b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/3300172609 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626da309070 con 0x5626da2bef30 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8161060 tx=0x7fa0d0007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b2e55d5b17ee1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c7fff640 1 -- 172.21.15.82:0/3300172609 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00075e0 con 0x5626da2bef30 2024-04-25T10:11:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c7fff640 1 -- 172.21.15.82:0/3300172609 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0021400 con 0x5626da2bef30 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c7fff640 1 -- 172.21.15.82:0/3300172609 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d00217b0 con 0x5626da2bef30 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3300172609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 msgr2=0x5626da2c2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8161060 tx=0x7fa0d0007040 comp rx=0 tx=0).stop 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3300172609 shutdown_connections 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da308920 0x5626da3127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2bef30 0x5626da2c2870 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/3300172609 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da2c52a0 0x5626da2c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3300172609 >> 172.21.15.82:0/3300172609 conn(0x5626da3103c0 msgr2=0x5626da2f4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3300172609 shutdown_connections 2024-04-25T10:11:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/3300172609 wait complete. 2024-04-25T10:11:33.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- start start 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 Processor -- start 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 0x5626da315830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da350d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da1123d0 con 0x5626da2c52a0 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626da1c9d50 con 0x5626da2bef30 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626da0a8980 con 0x5626da308920 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 10 client.0 _ll_register_callbacks cb 0x7fa0d80abf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45890/0 (socket says 172.21.15.82:45890) 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/2123939668 learned_addr learned my addr 172.21.15.82:0/2123939668 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e77fe640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da350d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 0x5626da315830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 msgr2=0x5626da350d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da350d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 msgr2=0x5626da315830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 0x5626da315830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 -- 172.21.15.82:0/2123939668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5626da18f600 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 0x5626da315830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e67fc640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0000fe0 tx=0x7fa0d0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1ef0e9743be0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0007700 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0d00047f0 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0d0021400 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0d0031050 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 10 client.8919 fetch_fsmap learned FSMap version 75 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x5626da0a29b0 con 0x5626da2c52a0 2024-04-25T10:11:33.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0021400 con 0x5626da2c52a0 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa0d0034020 con 0x5626da2c52a0 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 10 client.8919 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 10 client.8919 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 20 client.8919 populate_metadata read hostname 'smithi082' 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da309240 0x5626da351280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 4 client.8919 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5626da11b2e0 con 0x5626da309240 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.877+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da309240 0x5626da351280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0e6ffd640 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da309240 0x5626da351280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd62c144e775901c server_cookie=b1a8e89649db026e in_seq=0 out_seq=0 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0c5ffb640 10 client.8919 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0c5ffb640 1 -- 172.21.15.82:0/2123939668 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5626da11b2e0 con 0x5626da309240 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0c5ffb640 10 client.8919 handle_command_reply: tid=0 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 client.8919 shutdown 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 msgr2=0x5626da350800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0000fe0 tx=0x7fa0d0000ee0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 shutdown_connections 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x5626da309240 0x5626da351280 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5626da308920 0x5626da350d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5626da2c52a0 0x5626da350800 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0c5ffb640 0 client.8919 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 --2- 172.21.15.82:0/2123939668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5626da2bef30 0x5626da315830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 >> 172.21.15.82:0/2123939668 conn(0x5626da3103c0 msgr2=0x5626da2c1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 shutdown_connections 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 1 -- 172.21.15.82:0/2123939668 wait complete. 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0f05c4000 20 client.8919 trim_cache size 0 max 16384 2024-04-25T10:11:33.889 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8769, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.7006089539999998, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 75, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 107, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 107, 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 9.7678338730000007 2024-04-25T10:11:33.890 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 msgr2=0x7fa0c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 0x7fa0c8069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa0e0003fd0 tx=0x7fa0e00039a0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 msgr2=0x7fa0e815e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801a150 tx=0x7fa0d801ad60 comp rx=0 tx=0).stop 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 shutdown_connections 2024-04-25T10:11:33.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa0c8066f60 0x7fa0c8069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa0e806fc50 0x7fa0e814b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa0e80cff50 0x7fa0e815e060 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 --2- 172.21.15.82:0/1689632350 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa0e806b1d0 0x7fa0e80b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 >> 172.21.15.82:0/1689632350 conn(0x7fa0e8099a70 msgr2=0x7fa0e814c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 shutdown_connections 2024-04-25T10:11:33.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:33.881+0000 7fa0ed457640 1 -- 172.21.15.82:0/1689632350 wait complete. 2024-04-25T10:11:33.899 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:34.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 Processor -- start 2024-04-25T10:11:34.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 -- start start 2024-04-25T10:11:34.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f923c0ce510 0x7f923c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c0cee30 con 0x7f923c144ca0 2024-04-25T10:11:34.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c0ccc70 con 0x7f923c1451f0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f92431d7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0cc2b0 con 0x7f923c0ce510 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f923bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f923c144ca0 0x7f923c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57862/0 (socket says 172.21.15.82:57862) 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f923bfff640 1 -- 172.21.15.82:0/558370710 learned_addr learned my addr 172.21.15.82:0/558370710 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f9240f73640 1 --2- 172.21.15.82:0/558370710 >> v2:172.21.15.196:3300/0 conn(0x7f923c0ce510 0x7f923c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827532836 0 0) 0x7f923c0cee30 con 0x7f923c144ca0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.009+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9224003530 con 0x7f923c144ca0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830935949 0 0) 0x7f9224003530 con 0x7f923c144ca0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 >> v2:172.21.15.196:3300/0 conn(0x7f923c0ce510 msgr2=0x7f923c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 --2- 172.21.15.82:0/558370710 >> v2:172.21.15.196:3300/0 conn(0x7f923c0ce510 0x7f923c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 >> v1:172.21.15.196:6789/0 conn(0x7f923c1451f0 legacy=0x7f923c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c06dc40 con 0x7f923c144ca0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9230001680 con 0x7f923c144ca0 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 >> v1:172.21.15.82:6789/0 conn(0x7f923c144ca0 legacy=0x7f923c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9224003a30 0x7f9224003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f9224004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92240051d0 0x7f922400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0cc2b0 con 0x7f9224004810 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922400c0e0 con 0x7f9224003a30 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922400c2e0 con 0x7f92240051d0 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f9224004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f9240f73640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9224003a30 0x7f9224003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f9241774640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92240051d0 0x7f922400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 -- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92240051d0 msgr2=0x7f922400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92240051d0 0x7f922400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 -- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9224003a30 msgr2=0x7f9224003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9224003a30 0x7f9224003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 -- 172.21.15.82:0/558370710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9224003530 con 0x7f9224004810 2024-04-25T10:11:34.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f9241774640 1 -- 172.21.15.82:0/558370710 reap_dead start 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923bfff640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f9224004c90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f4c0 tx=0x7f923001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e278e1fd52acc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9230007d40 con 0x7f9224004810 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92300a9070 con 0x7f9224004810 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f923b7fe640 1 -- 172.21.15.82:0/558370710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92300a5e00 con 0x7f9224004810 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 -- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 msgr2=0x7f9224004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f9224004c90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f923c06f4c0 tx=0x7f923001cc60 comp rx=0 tx=0).stop 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 -- 172.21.15.82:0/558370710 shutdown_connections 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 --2- 172.21.15.82:0/558370710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f9224004c90 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 -- 172.21.15.82:0/558370710 >> 172.21.15.82:0/558370710 conn(0x7f923c0b11a0 msgr2=0x7f923c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 -- 172.21.15.82:0/558370710 shutdown_connections 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.013+0000 7f92431d7640 1 -- 172.21.15.82:0/558370710 wait complete. 2024-04-25T10:11:34.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 Processor -- start 2024-04-25T10:11:34.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- start start 2024-04-25T10:11:34.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c15b330 0x7f923c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0c68a0 0x7f923c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0cc900 con 0x7f9224004810 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c06e290 con 0x7f923c0c68a0 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ce8f0 con 0x7f923c15b330 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45904/0 (socket says 172.21.15.82:45904) 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 -- 172.21.15.82:0/4079659789 learned_addr learned my addr 172.21.15.82:0/4079659789 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9241774640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0c68a0 0x7f923c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 -- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c15b330 msgr2=0x7f923c0c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c15b330 0x7f923c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 -- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0c68a0 msgr2=0x7f923c0c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0c68a0 0x7f923c0c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 -- 172.21.15.82:0/4079659789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0ae5d0 con 0x7f9224004810 2024-04-25T10:11:34.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f9240f73640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f922c007f60 tx=0x7f922c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c01a020 con 0x7f9224004810 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922c016470 con 0x7f9224004810 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c0c5ab0 con 0x7f9224004810 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922c016860 con 0x7f9224004810 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c0ac1f0 con 0x7f9224004810 2024-04-25T10:11:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.017+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c04efa0 con 0x7f9224004810 2024-04-25T10:11:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.021+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f922c01d020 con 0x7f9224004810 2024-04-25T10:11:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.021+0000 7f92397fa640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 0x7f92280693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.021+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f922c086100 con 0x7f9224004810 2024-04-25T10:11:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.021+0000 7f923bfff640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 0x7f92280693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.021+0000 7f923bfff640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 0x7f92280693f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92300ab000 tx=0x7f9230001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.025+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f922c024030 con 0x7f9224004810 2024-04-25T10:11:34.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f923c0b1cc0 con 0x7f9224004810 2024-04-25T10:11:34.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92397fa640 1 -- 172.21.15.82:0/4079659789 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f922c057010 con 0x7f9224004810 2024-04-25T10:11:34.224 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 msgr2=0x7f92280693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 0x7f92280693f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f92300ab000 tx=0x7f9230001a50 comp rx=0 tx=0).stop 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 msgr2=0x7f923c0ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f922c007f60 tx=0x7f922c007f90 comp rx=0 tx=0).stop 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 shutdown_connections 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9228066f30 0x7f92280693f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c15b330 0x7f923c0c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c0c68a0 0x7f923c0c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 --2- 172.21.15.82:0/4079659789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9224004810 0x7f923c0ae090 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 >> 172.21.15.82:0/4079659789 conn(0x7f923c0b11a0 msgr2=0x7f923c0a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 shutdown_connections 2024-04-25T10:11:34.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:34.221+0000 7f92431d7640 1 -- 172.21.15.82:0/4079659789 wait complete. 2024-04-25T10:11:34.242 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:11:34.242 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:34.242 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:11:34.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:34.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:34.243 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:11:34.243 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:11:34.243 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:34.243 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:34.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.243 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:11:34.248 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:34.248 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.249 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41975sec preferred_lft 41975sec 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:34.250 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:34.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.252 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:34.300 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:11:34.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.301 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:11:34.354 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:11:34.354 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:34.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:34.356 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:11:34.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.433 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:34.479 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:34.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.479 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:11:34.534 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:11:34.534 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:34.534 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:34.583 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:34.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:34.583 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:34.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:34.688 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:34.737 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:34.738 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:34.739 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:34.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.740 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:34.787 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:11:34.787 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:11:34.832 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:34.847 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:34.897 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:34.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:34.898 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:34.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:11:34.909+0000 7f6b6117f580 -1 init, newargv = 0x5632126af640 newargc=15 2024-04-25T10:11:34.913 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[51868]: starting ceph client 2024-04-25T10:11:34.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[51868]: starting fuse 2024-04-25T10:11:35.140 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:11:35.137+0000 7ff91a0fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:36.017 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:36.033 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:36.081 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:36.082 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:36.083 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:36.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.085 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:36.131 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:11:36.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:11:36.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:11:36.131 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:11:36.131 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:11:36.132 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:11:36.133 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:36.256 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.51868.asok 2024-04-25T10:11:36.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:36.268 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51868.asok 2024-04-25T10:11:36.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.268 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51868.asok status 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "51868", 2024-04-25T10:11:36.424 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 8931, 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 8931 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2685041399 2024-04-25T10:11:36.425 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2685041399 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.8931 192.168.0.1:0/2685041399", 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2685041399", 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 75, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 107, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:11:36.426 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:11:36.427 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:11:36.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.437 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:11:36.438 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:11:36.444 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:36.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.444 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:36.498 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:36.498 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:36.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.547 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:36.675 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records in 2024-04-25T10:11:36.675 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records out 2024-04-25T10:11:36.676 INFO:teuthology.orchestra.run.smithi082.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0723469 s, 87.0 MB/s 2024-04-25T10:11:36.676 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:36.677 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:36.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:36.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 Processor -- start 2024-04-25T10:11:36.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 -- start start 2024-04-25T10:11:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40fc0ce510 0x7f40fc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 0x7f40fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc0cee30 con 0x7f40fc144ca0 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc0cc2b0 con 0x7f40fc0ce510 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f41032c7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06e290 con 0x7f40fc1451f0 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40fc0ce510 0x7f40fc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f40fc0ce510 0x7f40fc0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57558/0 (socket says 172.21.15.82:57558) 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 -- 172.21.15.82:0/1628515716 learned_addr learned my addr 172.21.15.82:0/1628515716 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 -- 172.21.15.82:0/1628515716 >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 msgr2=0x7f40fc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101864640 1 --2- 172.21.15.82:0/1628515716 >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 0x7f40fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 --2- 172.21.15.82:0/1628515716 >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 0x7f40fc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 -- 172.21.15.82:0/1628515716 >> v1:172.21.15.196:6789/0 conn(0x7f40fc144ca0 legacy=0x7f40fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101063640 1 -- 172.21.15.82:0/1628515716 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc0ccc70 con 0x7f40fc0ce510 2024-04-25T10:11:36.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.857+0000 7f4101864640 1 --2- 172.21.15.82:0/1628515716 >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 0x7f40fc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f4101063640 1 --2- 172.21.15.82:0/1628515716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc0ce8f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0b48f0 tx=0x7f40ec01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a266fb222449b19a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f40f3fff640 1 -- 172.21.15.82:0/1628515716 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0b5070 con 0x7f40fc0ce510 2024-04-25T10:11:36.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f40f3fff640 1 -- 172.21.15.82:0/1628515716 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec0a89d0 con 0x7f40fc0ce510 2024-04-25T10:11:36.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f40f3fff640 1 -- 172.21.15.82:0/1628515716 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0a8da0 con 0x7f40fc0ce510 2024-04-25T10:11:36.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- 172.21.15.82:0/1628515716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 msgr2=0x7f40fc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- 172.21.15.82:0/1628515716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc0ce8f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0b48f0 tx=0x7f40ec01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- 172.21.15.82:0/1628515716 shutdown_connections 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- 172.21.15.82:0/1628515716 >> v2:172.21.15.196:3300/0 conn(0x7f40fc1451f0 0x7f40fc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- 172.21.15.82:0/1628515716 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc0ce8f0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- 172.21.15.82:0/1628515716 >> 172.21.15.82:0/1628515716 conn(0x7f40fc0b11a0 msgr2=0x7f40fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- 172.21.15.82:0/1628515716 shutdown_connections 2024-04-25T10:11:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- 172.21.15.82:0/1628515716 wait complete. 2024-04-25T10:11:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 Processor -- start 2024-04-25T10:11:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- start start 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 0x7f40fc142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 0x7f40fc0c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06ee30 con 0x7f40fc0ce510 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06df20 con 0x7f40fc1451f0 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f41032c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0cc900 con 0x7f40fc144ca0 2024-04-25T10:11:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f4101063640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f4101063640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57566/0 (socket says 172.21.15.82:57566) 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.861+0000 7f4101063640 1 -- 172.21.15.82:0/223579214 learned_addr learned my addr 172.21.15.82:0/223579214 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101864640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 0x7f40fc0c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4100862640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 0x7f40fc142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 -- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 msgr2=0x7f40fc142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 0x7f40fc142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 -- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 msgr2=0x7f40fc0c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 0x7f40fc0c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 -- 172.21.15.82:0/223579214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc140370 con 0x7f40fc0ce510 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4100862640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 0x7f40fc142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101864640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 0x7f40fc0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4101063640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0b3040 tx=0x7f40ec01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0c0070 con 0x7f40fc0ce510 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ec0b0b60 con 0x7f40fc0ce510 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec0b5070 con 0x7f40fc0ce510 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc140650 con 0x7f40fc0ce510 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc140c10 con 0x7f40fc0ce510 2024-04-25T10:11:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc04efa0 con 0x7f40fc0ce510 2024-04-25T10:11:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40ec0ba3e0 con 0x7f40fc0ce510 2024-04-25T10:11:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 0x7f40e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f40ec1224c0 con 0x7f40fc0ce510 2024-04-25T10:11:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4100862640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 0x7f40e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:36.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.865+0000 7f4100862640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 0x7f40e0069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0c8bd0 tx=0x7f40f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:36.874 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:36.873+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40ec018830 con 0x7f40fc0ce510 2024-04-25T10:11:37.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.033+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40fc0b0f80 con 0x7f40fc0ce510 2024-04-25T10:11:37.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.033+0000 7f40f1ffb640 1 -- 172.21.15.82:0/223579214 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f40ec0f33d0 con 0x7f40fc0ce510 2024-04-25T10:11:37.037 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:37.037 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":107,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:30.162220+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 msgr2=0x7f40e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 0x7f40e0069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0c8bd0 tx=0x7f40f8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 msgr2=0x7f40fc13f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0b3040 tx=0x7f40ec01ba70 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 shutdown_connections 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f40e0066f60 0x7f40e0069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40fc144ca0 0x7f40fc142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40fc1451f0 0x7f40fc0c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 --2- 172.21.15.82:0/223579214 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40fc0ce510 0x7f40fc13f040 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 >> 172.21.15.82:0/223579214 conn(0x7f40fc0b11a0 msgr2=0x7f40fc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 shutdown_connections 2024-04-25T10:11:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.037+0000 7f41032c7640 1 -- 172.21.15.82:0/223579214 wait complete. 2024-04-25T10:11:37.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:27.097350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:27.504961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:37.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:37.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.177+0000 7f27b2d17640 1 Processor -- start 2024-04-25T10:11:37.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.177+0000 7f27b2d17640 1 -- start start 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b2d17640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f27ac0b1ae0 0x7f27ac151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b2d17640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac160310 con 0x7f27ac0cc820 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b2d17640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ac15f550 con 0x7f27ac0b0b00 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b2d17640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ac15cfa0 con 0x7f27ac0b1ae0 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27abfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f27ac0cc820 0x7f27ac0b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42058/0 (socket says 172.21.15.82:42058) 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27abfff640 1 -- 172.21.15.82:0/1718393312 learned_addr learned my addr 172.21.15.82:0/1718393312 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1718393312 >> v2:172.21.15.196:3300/0 conn(0x7f27ac0b1ae0 0x7f27ac151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557773318 0 0) 0x7f27ac160310 con 0x7f27ac0cc820 2024-04-25T10:11:37.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2798003530 con 0x7f27ac0cc820 2024-04-25T10:11:37.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817936100 0 0) 0x7f2798003530 con 0x7f27ac0cc820 2024-04-25T10:11:37.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 >> v2:172.21.15.196:3300/0 conn(0x7f27ac0b1ae0 msgr2=0x7f27ac151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 --2- 172.21.15.82:0/1718393312 >> v2:172.21.15.196:3300/0 conn(0x7f27ac0b1ae0 0x7f27ac151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 >> v1:172.21.15.196:6789/0 conn(0x7f27ac0b0b00 legacy=0x7f27ac0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac15f830 con 0x7f27ac0cc820 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2794001680 con 0x7f27ac0cc820 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 >> v1:172.21.15.82:6789/0 conn(0x7f27ac0cc820 legacy=0x7f27ac0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 0x7f2798003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798004810 0x7f2798004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27980051d0 0x7f279800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279800c0d0 con 0x7f2798003a30 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279800c280 con 0x7f27980051d0 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279800c480 con 0x7f2798004810 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 0x7f2798003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27abfff640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798004810 0x7f2798004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27980051d0 0x7f279800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 -- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798004810 msgr2=0x7f2798004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798004810 0x7f2798004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 -- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27980051d0 msgr2=0x7f279800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27980051d0 0x7f279800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 -- 172.21.15.82:0/1718393312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798003530 con 0x7f2798003a30 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27980051d0 0x7f279800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b12b4640 1 -- 172.21.15.82:0/1718393312 reap_dead start 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.181+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 0x7f2798003e50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f279c001e70 tx=0x7f279c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203b03a383ec5c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c001a90 con 0x7f2798003a30 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c003b80 con 0x7f2798003a30 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27ab7fe640 1 -- 172.21.15.82:0/1718393312 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c0050d0 con 0x7f2798003a30 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 msgr2=0x7f2798003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 0x7f2798003e50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f279c001e70 tx=0x7f279c01c760 comp rx=0 tx=0).stop 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- 172.21.15.82:0/1718393312 shutdown_connections 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1718393312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2798003a30 0x7f2798003e50 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- 172.21.15.82:0/1718393312 >> 172.21.15.82:0/1718393312 conn(0x7f27ac0b1f10 msgr2=0x7f27ac0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- 172.21.15.82:0/1718393312 shutdown_connections 2024-04-25T10:11:37.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- 172.21.15.82:0/1718393312 wait complete. 2024-04-25T10:11:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 Processor -- start 2024-04-25T10:11:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- start start 2024-04-25T10:11:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798003a30 0x7f27ac0c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac0b2d60 0x7f27ac0c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0b1060 con 0x7f27ac15bc50 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0b0ee0 con 0x7f27ac0b2d60 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b2d17640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ac0b17e0 con 0x7f2798003a30 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57572/0 (socket says 172.21.15.82:57572) 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 -- 172.21.15.82:0/1158067935 learned_addr learned my addr 172.21.15.82:0/1158067935 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798003a30 0x7f27ac0c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27abfff640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac0b2d60 0x7f27ac0c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 -- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798003a30 msgr2=0x7f27ac0c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798003a30 0x7f27ac0c5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 -- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac0b2d60 msgr2=0x7f27ac0c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac0b2d60 0x7f27ac0c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 -- 172.21.15.82:0/1158067935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ac0b31f0 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27b12b4640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0b1ee0 tx=0x7f27a4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.185+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a400f040 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a401b070 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ac0b34d0 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a4004430 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ac0b3d70 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ac0b36a0 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27a400e670 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27a97fa640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 0x7f278c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27a4084fc0 con 0x7f27ac15bc50 2024-04-25T10:11:37.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 0x7f278c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.189+0000 7f27b0ab3640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 0x7f278c069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f279c0ac000 tx=0x7f279c001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.193+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27a4055ed0 con 0x7f27ac15bc50 2024-04-25T10:11:37.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27ac04efa0 con 0x7f27ac15bc50 2024-04-25T10:11:37.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27a97fa640 1 -- 172.21.15.82:0/1158067935 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27a40557d0 con 0x7f27ac15bc50 2024-04-25T10:11:37.359 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:37.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 msgr2=0x7f278c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 0x7f278c069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f279c0ac000 tx=0x7f279c001520 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 msgr2=0x7f27ac15c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0b1ee0 tx=0x7f27a4010c20 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 shutdown_connections 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f278c066f60 0x7f278c069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f27ac15bc50 0x7f27ac15c0c0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f27ac0b2d60 0x7f27ac0c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 --2- 172.21.15.82:0/1158067935 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2798003a30 0x7f27ac0c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 >> 172.21.15.82:0/1158067935 conn(0x7f27ac0b1f10 msgr2=0x7f27ac0ae8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 shutdown_connections 2024-04-25T10:11:37.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.357+0000 7f27b2d17640 1 -- 172.21.15.82:0/1158067935 wait complete. 2024-04-25T10:11:37.378 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 Processor -- start 2024-04-25T10:11:37.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 -- start start 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48b40cb380 0x7f48b40cb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b4071b70 con 0x7f48b4159bd0 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b4072780 con 0x7f48b40cb380 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48ba1ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4071e50 con 0x7f48b4159fb0 2024-04-25T10:11:37.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48b40cb380 0x7f48b40cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f48b40cb380 0x7f48b40cb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57582/0 (socket says 172.21.15.82:57582) 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 -- 172.21.15.82:0/1836307600 learned_addr learned my addr 172.21.15.82:0/1836307600 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 -- 172.21.15.82:0/1836307600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 msgr2=0x7f48b40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 --2- 172.21.15.82:0/1836307600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 -- 172.21.15.82:0/1836307600 >> v1:172.21.15.196:6789/0 conn(0x7f48b4159bd0 legacy=0x7f48b40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.501+0000 7f48b3fff640 1 -- 172.21.15.82:0/1836307600 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b4072130 con 0x7f48b40cb380 2024-04-25T10:11:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b3fff640 1 --2- 172.21.15.82:0/1836307600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b40cb830 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009e80 tx=0x7f48ac01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03f5a3ffd81ea8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b27fc640 1 -- 172.21.15.82:0/1836307600 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0a5840 con 0x7f48b40cb380 2024-04-25T10:11:37.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b27fc640 1 -- 172.21.15.82:0/1836307600 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ac0a59e0 con 0x7f48b40cb380 2024-04-25T10:11:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b27fc640 1 -- 172.21.15.82:0/1836307600 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ac0a5db0 con 0x7f48b40cb380 2024-04-25T10:11:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1836307600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 msgr2=0x7f48b40cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1836307600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b40cb830 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f48ac009e80 tx=0x7f48ac01c9a0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1836307600 shutdown_connections 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1836307600 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b40cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1836307600 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b40cb830 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1836307600 >> 172.21.15.82:0/1836307600 conn(0x7f48b406ae60 msgr2=0x7f48b406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1836307600 shutdown_connections 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1836307600 wait complete. 2024-04-25T10:11:37.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 Processor -- start 2024-04-25T10:11:37.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- start start 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48b4159bd0 0x7f48b4158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b41643f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40cfc50 con 0x7f48b40cb380 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b40cfad0 con 0x7f48b4159bd0 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48ba1ca640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b4168390 con 0x7f48b4159fb0 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57592/0 (socket says 172.21.15.82:57592) 2024-04-25T10:11:37.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.505+0000 7f48b37fe640 1 -- 172.21.15.82:0/1984959966 learned_addr learned my addr 172.21.15.82:0/1984959966 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b3fff640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b41643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 -- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 msgr2=0x7f48b41643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b41643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 -- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48b4159bd0 msgr2=0x7f48b4158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48b4159bd0 0x7f48b4158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 -- 172.21.15.82:0/1984959966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b415ba00 con 0x7f48b40cb380 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b3fff640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b41643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b37fe640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c480 tx=0x7f48a400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4012830 con 0x7f48b40cb380 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4017070 con 0x7f48b40cb380 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b4155620 con 0x7f48b40cb380 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a4012c30 con 0x7f48b40cb380 2024-04-25T10:11:37.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b406e5d0 con 0x7f48b40cb380 2024-04-25T10:11:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b404efa0 con 0x7f48b40cb380 2024-04-25T10:11:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f48a401c460 con 0x7f48b40cb380 2024-04-25T10:11:37.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 0x7f48a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f48a4084f20 con 0x7f48b40cb380 2024-04-25T10:11:37.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b2ffd640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 0x7f48a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.509+0000 7f48b2ffd640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 0x7f48a0069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c2790 tx=0x7f48a8006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.513+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48b415ba00 con 0x7f48b40cb380 2024-04-25T10:11:37.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.665+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48b406dfa0 con 0x7f48b40cb380 2024-04-25T10:11:37.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.665+0000 7f489bfff640 1 -- 172.21.15.82:0/1984959966 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f48a4055e30 con 0x7f48b40cb380 2024-04-25T10:11:37.669 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:37.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 msgr2=0x7f48a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 0x7f48a0069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f48b40c2790 tx=0x7f48a8006ca0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 msgr2=0x7f48b41550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f48a400c480 tx=0x7f48a400c940 comp rx=0 tx=0).stop 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 shutdown_connections 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f48a0066f60 0x7f48a0069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f48b4159fb0 0x7f48b41643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f48b4159bd0 0x7f48b4158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 --2- 172.21.15.82:0/1984959966 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f48b40cb380 0x7f48b41550e0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 >> 172.21.15.82:0/1984959966 conn(0x7f48b406ae60 msgr2=0x7f48b406b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 shutdown_connections 2024-04-25T10:11:37.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.669+0000 7f48ba1ca640 1 -- 172.21.15.82:0/1984959966 wait complete. 2024-04-25T10:11:37.687 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:37.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 Processor -- start 2024-04-25T10:11:37.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 -- start start 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 0x7f69480cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480cbd20 con 0x7f69480cfad0 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69480cc000 con 0x7f694815b420 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694ce09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480cff20 con 0x7f694815b800 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6946d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f69480cfad0 0x7f6948072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42066/0 (socket says 172.21.15.82:42066) 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6946d9d640 1 -- 172.21.15.82:0/858829453 learned_addr learned my addr 172.21.15.82:0/858829453 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 0x7f69480cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684140082 0 0) 0x7f69480cbd20 con 0x7f69480cfad0 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6930003530 con 0x7f69480cfad0 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 170096513 0 0) 0x7f6930003530 con 0x7f69480cfad0 2024-04-25T10:11:37.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 msgr2=0x7f69480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 0x7f69480cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 >> v1:172.21.15.196:6789/0 conn(0x7f694815b420 legacy=0x7f69480ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480cc2e0 con 0x7f69480cfad0 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6940001a00 con 0x7f69480cfad0 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 >> v1:172.21.15.82:6789/0 conn(0x7f69480cfad0 legacy=0x7f6948072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6930003a60 0x7f6930003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f6930004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 0x7f69300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480cff20 con 0x7f6930004810 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693000bad0 con 0x7f6930003a60 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693000bcd0 con 0x7f694815b800 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f6930004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694659c640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6930003a60 0x7f6930003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 -- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 msgr2=0x7f69300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f694815b800 0x7f69300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 -- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6930003a60 msgr2=0x7f6930003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6930003a60 0x7f6930003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f6945d9b640 1 -- 172.21.15.82:0/858829453 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6930003530 con 0x7f6930004810 2024-04-25T10:11:37.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.797+0000 7f694659c640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6930003a60 0x7f6930003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f6945d9b640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f6930004c90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f693c009ab0 tx=0x7f693c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ab87539dd84659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0b2070 con 0x7f6930004810 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c0a55a0 con 0x7f6930004810 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694559a640 1 -- 172.21.15.82:0/858829453 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c0096b0 con 0x7f6930004810 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- 172.21.15.82:0/858829453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 msgr2=0x7f6930004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 --2- 172.21.15.82:0/858829453 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f6930004c90 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f693c009ab0 tx=0x7f693c01ca90 comp rx=0 tx=0).stop 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f6946d9d640 1 -- 172.21.15.82:0/858829453 reap_dead start 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- 172.21.15.82:0/858829453 shutdown_connections 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- 172.21.15.82:0/858829453 >> 172.21.15.82:0/858829453 conn(0x7f694806ae60 msgr2=0x7f694806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- 172.21.15.82:0/858829453 shutdown_connections 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- 172.21.15.82:0/858829453 wait complete. 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 Processor -- start 2024-04-25T10:11:37.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- start start 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f694806ea00 0x7f69480ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69480cec10 0x7f69480c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480cb9f0 con 0x7f6930004810 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69480cc930 con 0x7f694806ea00 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694ce09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69480cadb0 con 0x7f69480cec10 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.801+0000 7f694659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57612/0 (socket says 172.21.15.82:57612) 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 -- 172.21.15.82:0/1622084710 learned_addr learned my addr 172.21.15.82:0/1622084710 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6945d9b640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f694806ea00 0x7f69480ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6946d9d640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69480cec10 0x7f69480c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 -- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69480cec10 msgr2=0x7f69480c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69480cec10 0x7f69480c2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 -- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f694806ea00 msgr2=0x7f69480ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f694806ea00 0x7f69480ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 -- 172.21.15.82:0/1622084710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69480ada70 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694659c640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f693800d7d0 tx=0x7f693800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6938005270 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6938017070 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69480cd630 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693800b6b0 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69480cdbf0 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694804efa0 con 0x7f6930004810 2024-04-25T10:11:37.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f693800b850 con 0x7f6930004810 2024-04-25T10:11:37.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 0x7f6928069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:37.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6938085070 con 0x7f6930004810 2024-04-25T10:11:37.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6945d9b640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 0x7f6928069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:37.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.805+0000 7f6945d9b640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 0x7f6928069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f693c0af000 tx=0x7f693c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:37.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.809+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6938055ef0 con 0x7f6930004810 2024-04-25T10:11:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.961+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69480c2b90 con 0x7f6930004810 2024-04-25T10:11:37.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.961+0000 7f6936ffd640 1 -- 172.21.15.82:0/1622084710 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f693801ae40 con 0x7f6930004810 2024-04-25T10:11:37.965 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 msgr2=0x7f6928069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 0x7f6928069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f693c0af000 tx=0x7f693c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 msgr2=0x7f69480ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f693800d7d0 tx=0x7f693800dc90 comp rx=0 tx=0).stop 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 shutdown_connections 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6928066f90 0x7f6928069450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f69480cec10 0x7f69480c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f694806ea00 0x7f69480ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 --2- 172.21.15.82:0/1622084710 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6930004810 0x7f69480ccf30 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 >> 172.21.15.82:0/1622084710 conn(0x7f694806ae60 msgr2=0x7f69480b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 shutdown_connections 2024-04-25T10:11:37.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:37.965+0000 7f694ce09640 1 -- 172.21.15.82:0/1622084710 wait complete. 2024-04-25T10:11:37.982 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:38.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 Processor -- start 2024-04-25T10:11:38.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 -- start start 2024-04-25T10:11:38.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0ba415a700 0x7f0ba415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba40b3f50 0x7f0ba40cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ba40ce430 0x7f0ba40b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba415f200 con 0x7f0ba40ce430 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba40b2090 con 0x7f0ba415a700 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba92cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40b43e0 con 0x7f0ba40b3f50 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ba40ce430 0x7f0ba40b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0ba40ce430 0x7f0ba40b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57622/0 (socket says 172.21.15.82:57622) 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 -- 172.21.15.82:0/1449170674 learned_addr learned my addr 172.21.15.82:0/1449170674 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0b9bfff640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba40b3f50 0x7f0ba40cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba2ffd640 1 --2- 172.21.15.82:0/1449170674 >> v2:172.21.15.196:3300/0 conn(0x7f0ba415a700 0x7f0ba415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 -- 172.21.15.82:0/1449170674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba40b3f50 msgr2=0x7f0ba40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba40b3f50 0x7f0ba40cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 -- 172.21.15.82:0/1449170674 >> v2:172.21.15.196:3300/0 conn(0x7f0ba415a700 msgr2=0x7f0ba415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 --2- 172.21.15.82:0/1449170674 >> v2:172.21.15.196:3300/0 conn(0x7f0ba415a700 0x7f0ba415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.101+0000 7f0ba37fe640 1 -- 172.21.15.82:0/1449170674 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40b46c0 con 0x7f0ba40ce430 2024-04-25T10:11:38.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba37fe640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba40b24a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406bd20 tx=0x7f0b9c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55934ffc9e84b2dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba27fc640 1 -- 172.21.15.82:0/1449170674 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0a5630 con 0x7f0ba40ce430 2024-04-25T10:11:38.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba27fc640 1 -- 172.21.15.82:0/1449170674 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c0a57d0 con 0x7f0ba40ce430 2024-04-25T10:11:38.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba27fc640 1 -- 172.21.15.82:0/1449170674 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c0a5ba0 con 0x7f0ba40ce430 2024-04-25T10:11:38.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1449170674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 msgr2=0x7f0ba40b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba40b24a0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406bd20 tx=0x7f0b9c01c8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1449170674 shutdown_connections 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba40b24a0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1449170674 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba40b3f50 0x7f0ba40cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1449170674 >> v2:172.21.15.196:3300/0 conn(0x7f0ba415a700 0x7f0ba415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1449170674 >> 172.21.15.82:0/1449170674 conn(0x7f0ba40b11a0 msgr2=0x7f0ba40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1449170674 shutdown_connections 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1449170674 wait complete. 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 Processor -- start 2024-04-25T10:11:38.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- start start 2024-04-25T10:11:38.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba415a700 0x7f0ba4146ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba414e200 0x7f0ba415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40cd490 con 0x7f0ba40ce430 2024-04-25T10:11:38.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40cd310 con 0x7f0ba414e200 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba92cc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4071840 con 0x7f0ba415a700 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57638/0 (socket says 172.21.15.82:57638) 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 -- 172.21.15.82:0/1046490783 learned_addr learned my addr 172.21.15.82:0/1046490783 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 -- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba415a700 msgr2=0x7f0ba4146ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba415a700 0x7f0ba4146ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba37fe640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba414e200 0x7f0ba415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 -- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba414e200 msgr2=0x7f0ba415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba414e200 0x7f0ba415acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 -- 172.21.15.82:0/1046490783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4143730 con 0x7f0ba40ce430 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.105+0000 7f0ba2ffd640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e930 tx=0x7f0b9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94017070 con 0x7f0ba40ce430 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9400c9a0 con 0x7f0ba40ce430 2024-04-25T10:11:38.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba4147740 con 0x7f0ba40ce430 2024-04-25T10:11:38.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9400cd50 con 0x7f0ba40ce430 2024-04-25T10:11:38.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba4147d00 con 0x7f0ba40ce430 2024-04-25T10:11:38.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba404efa0 con 0x7f0ba40ce430 2024-04-25T10:11:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b9401a070 con 0x7f0ba40ce430 2024-04-25T10:11:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 0x7f0b84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b94085580 con 0x7f0ba40ce430 2024-04-25T10:11:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9bfff640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 0x7f0b84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.109+0000 7f0b9bfff640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 0x7f0b84069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ae940 tx=0x7f0b8c00f040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.113+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b94056430 con 0x7f0ba40ce430 2024-04-25T10:11:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0ba4073d90 con 0x7f0ba40ce430 2024-04-25T10:11:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0b9b7fe640 1 -- 172.21.15.82:0/1046490783 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b94055d30 con 0x7f0ba40ce430 2024-04-25T10:11:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 msgr2=0x7f0b84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 0x7f0b84069450 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f0ba40ae940 tx=0x7f0b8c00f040 comp rx=0 tx=0).stop 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 msgr2=0x7f0ba41431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400e930 tx=0x7f0b9400edf0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 shutdown_connections 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0b84066f90 0x7f0b84069450 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0ba415a700 0x7f0ba4146ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0ba414e200 0x7f0ba415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 --2- 172.21.15.82:0/1046490783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0ba40ce430 0x7f0ba41431f0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 >> 172.21.15.82:0/1046490783 conn(0x7f0ba40b11a0 msgr2=0x7f0ba40cc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 shutdown_connections 2024-04-25T10:11:38.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.281+0000 7f0ba92cc640 1 -- 172.21.15.82:0/1046490783 wait complete. 2024-04-25T10:11:38.299 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:38.300 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:38.305 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:38.331 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:38.348 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.445+0000 7f2677b48640 1 Processor -- start 2024-04-25T10:11:38.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.445+0000 7f2677b48640 1 -- start start 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f267009aa80 0x7f26700b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2670099cc0 con 0x7f26700b19f0 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2670099fa0 con 0x7f26700b24d0 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f267009aee0 con 0x7f267009aa80 2024-04-25T10:11:38.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26750e3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f267009aa80 0x7f26700b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26760e5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f26700b19f0 0x7f267006c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42070/0 (socket says 172.21.15.82:42070) 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26760e5640 1 -- 172.21.15.82:0/2626680373 learned_addr learned my addr 172.21.15.82:0/2626680373 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26748e2640 1 -- 172.21.15.82:0/2626680373 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978124164 0 0) 0x7f2670099cc0 con 0x7f26700b19f0 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26748e2640 1 -- 172.21.15.82:0/2626680373 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f265c003530 con 0x7f26700b19f0 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26750e3640 1 -- 172.21.15.82:0/2626680373 >> v1:172.21.15.196:6789/0 conn(0x7f26700b24d0 legacy=0x7f26700b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26750e3640 1 -- 172.21.15.82:0/2626680373 >> v1:172.21.15.82:6789/0 conn(0x7f26700b19f0 legacy=0x7f267006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26750e3640 1 -- 172.21.15.82:0/2626680373 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267009a280 con 0x7f267009aa80 2024-04-25T10:11:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26750e3640 1 --2- 172.21.15.82:0/2626680373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f267009aa80 0x7f26700b14b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f26600097c0 tx=0x7f266001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8a4b5e508e436a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26748e2640 1 -- 172.21.15.82:0/2626680373 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600b0070 con 0x7f267009aa80 2024-04-25T10:11:38.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26748e2640 1 -- 172.21.15.82:0/2626680373 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26600ab600 con 0x7f267009aa80 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f26748e2640 1 -- 172.21.15.82:0/2626680373 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26600ab9b0 con 0x7f267009aa80 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- 172.21.15.82:0/2626680373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f267009aa80 msgr2=0x7f26700b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 --2- 172.21.15.82:0/2626680373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f267009aa80 0x7f26700b14b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f26600097c0 tx=0x7f266001c340 comp rx=0 tx=0).stop 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- 172.21.15.82:0/2626680373 shutdown_connections 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 --2- 172.21.15.82:0/2626680373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f267009aa80 0x7f26700b14b0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- 172.21.15.82:0/2626680373 >> 172.21.15.82:0/2626680373 conn(0x7f26700b3020 msgr2=0x7f26700b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- 172.21.15.82:0/2626680373 shutdown_connections 2024-04-25T10:11:38.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.449+0000 7f2677b48640 1 -- 172.21.15.82:0/2626680373 wait complete. 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 Processor -- start 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- start start 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f267009aa80 0x7f267009f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26700b19f0 0x7f2670156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267016b900 con 0x7f26700b24d0 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267016b780 con 0x7f26700b19f0 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f267016ba80 con 0x7f267009aa80 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57650/0 (socket says 172.21.15.82:57650) 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 -- 172.21.15.82:0/547174124 learned_addr learned my addr 172.21.15.82:0/547174124 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 -- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f267009aa80 msgr2=0x7f267009f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f267009aa80 0x7f267009f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 -- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26700b19f0 msgr2=0x7f2670156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26700b19f0 0x7f2670156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 -- 172.21.15.82:0/547174124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2670161010 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26760e5640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f266800c9f0 tx=0x7f266800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668017070 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266800ab10 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2670157460 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2668012400 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2670157a20 con 0x7f26700b24d0 2024-04-25T10:11:38.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267004efa0 con 0x7f26700b24d0 2024-04-25T10:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.453+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f266801a070 con 0x7f26700b24d0 2024-04-25T10:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.457+0000 7f26667fc640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 0x7f2654069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.457+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2668084e60 con 0x7f26700b24d0 2024-04-25T10:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.457+0000 7f26758e4640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 0x7f2654069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.457+0000 7f26758e4640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 0x7f2654069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f26701fa050 tx=0x7f266c006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.461+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2668055ce0 con 0x7f26700b24d0 2024-04-25T10:11:38.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.617+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2670073d90 con 0x7f26700b24d0 2024-04-25T10:11:38.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f26667fc640 1 -- 172.21.15.82:0/547174124 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16896 (secure 0 0 0) 0x7f26680555e0 con 0x7f26700b24d0 2024-04-25T10:11:38.622 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:38.623 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":107,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:30.162220+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 msgr2=0x7f2654069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 0x7f2654069450 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f26701fa050 tx=0x7f266c006bd0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 msgr2=0x7f2670160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f266800c9f0 tx=0x7f266800ceb0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 shutdown_connections 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2654066f90 0x7f2654069450 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f26700b24d0 0x7f2670160ad0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f26700b19f0 0x7f2670156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 --2- 172.21.15.82:0/547174124 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f267009aa80 0x7f267009f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 >> 172.21.15.82:0/547174124 conn(0x7f26700b3020 msgr2=0x7f26700cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 shutdown_connections 2024-04-25T10:11:38.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.621+0000 7f2677b48640 1 -- 172.21.15.82:0/547174124 wait complete. 2024-04-25T10:11:38.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:27.097350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:27.504961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:38.641 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:38.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 Processor -- start 2024-04-25T10:11:38.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 -- start start 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5270 0x7f628c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f628c0b5650 0x7f628c0d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f628c06bff0 con 0x7f628c0d4440 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f628c06cc00 con 0x7f628c0b5650 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f629356c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c06c2d0 con 0x7f628c0b5270 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6290b07640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f628c0b5650 0x7f628c0d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6291b09640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f628c0d4440 0x7f628c06d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42076/0 (socket says 172.21.15.82:42076) 2024-04-25T10:11:38.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6291b09640 1 -- 172.21.15.82:0/3647314420 learned_addr learned my addr 172.21.15.82:0/3647314420 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6291308640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5270 0x7f628c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6290b07640 1 -- 172.21.15.82:0/3647314420 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5270 msgr2=0x7f628c0b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6290b07640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5270 0x7f628c0b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6290b07640 1 -- 172.21.15.82:0/3647314420 >> v1:172.21.15.82:6789/0 conn(0x7f628c0d4440 legacy=0x7f628c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6290b07640 1 -- 172.21.15.82:0/3647314420 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c06c5b0 con 0x7f628c0b5650 2024-04-25T10:11:38.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.753+0000 7f6283fff640 1 -- 172.21.15.82:0/3647314420 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112519684 0 0) 0x7f628c06bff0 con 0x7f628c0d4440 2024-04-25T10:11:38.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6290b07640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5650 0x7f628c0d3f00 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097a0 tx=0x7f627c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd093a115d648ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6283fff640 1 -- 172.21.15.82:0/3647314420 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0aa600 con 0x7f628c0b5650 2024-04-25T10:11:38.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6283fff640 1 -- 172.21.15.82:0/3647314420 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c0af070 con 0x7f628c0b5650 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6283fff640 1 -- 172.21.15.82:0/3647314420 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c0aa9c0 con 0x7f628c0b5650 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/3647314420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5650 msgr2=0x7f628c0d3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5650 0x7f628c0d3f00 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f627c0097a0 tx=0x7f627c01c480 comp rx=0 tx=0).stop 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/3647314420 shutdown_connections 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5650 0x7f628c0d3f00 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- 172.21.15.82:0/3647314420 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5270 0x7f628c0b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/3647314420 >> 172.21.15.82:0/3647314420 conn(0x7f628c08b000 msgr2=0x7f628c0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/3647314420 shutdown_connections 2024-04-25T10:11:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/3647314420 wait complete. 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 Processor -- start 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- start start 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5650 0x7f628c0c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c0d4440 0x7f628c0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c15dfa0 con 0x7f628c0b5270 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f628c15de20 con 0x7f628c0d4440 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f628c16a720 con 0x7f628c0b5650 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57670/0 (socket says 172.21.15.82:57670) 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 -- 172.21.15.82:0/1916019769 learned_addr learned my addr 172.21.15.82:0/1916019769 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 -- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5650 msgr2=0x7f628c0c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5650 0x7f628c0c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 -- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c0d4440 msgr2=0x7f628c0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c0d4440 0x7f628c0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 -- 172.21.15.82:0/1916019769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0c8010 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6291308640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f628800d800 tx=0x7f628800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62880059a0 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6288005b40 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f628c0c82f0 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6288005d30 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.757+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f628c0c85d0 con 0x7f628c0b5270 2024-04-25T10:11:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628c04efa0 con 0x7f628c0b5270 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f628801b020 con 0x7f628c0b5270 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f6281ffb640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6288084fe0 con 0x7f628c0b5270 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f6290b07640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:38.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.761+0000 7f6290b07640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 0x7f6270069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f627c0ae000 tx=0x7f627c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:38.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.765+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6288055e60 con 0x7f628c0b5270 2024-04-25T10:11:38.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.921+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f628c0c7070 con 0x7f628c0b5270 2024-04-25T10:11:38.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f6281ffb640 1 -- 172.21.15.82:0/1916019769 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6288055760 con 0x7f628c0b5270 2024-04-25T10:11:38.926 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:38.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 msgr2=0x7f6270069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 0x7f6270069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f627c0ae000 tx=0x7f627c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 msgr2=0x7f628c14c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f628800d800 tx=0x7f628800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 shutdown_connections 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6270066f60 0x7f6270069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f628c0b5650 0x7f628c0c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f628c0d4440 0x7f628c0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 --2- 172.21.15.82:0/1916019769 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f628c0b5270 0x7f628c14c950 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 >> 172.21.15.82:0/1916019769 conn(0x7f628c08b000 msgr2=0x7f628c15a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 shutdown_connections 2024-04-25T10:11:38.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:38.925+0000 7f629356c640 1 -- 172.21.15.82:0/1916019769 wait complete. 2024-04-25T10:11:38.943 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 Processor -- start 2024-04-25T10:11:39.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 -- start start 2024-04-25T10:11:39.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f855015d8d0 0x7f855015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855006c570 con 0x7f855006d330 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f855006c850 con 0x7f85500ce540 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f855612a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f855006cea0 con 0x7f855015d8d0 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f855015d8d0 0x7f855015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f855006d330 0x7f85500cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42086/0 (socket says 172.21.15.82:42086) 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854effd640 1 -- 172.21.15.82:0/103887562 learned_addr learned my addr 172.21.15.82:0/103887562 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854f7fe640 1 -- 172.21.15.82:0/103887562 >> v1:172.21.15.196:6789/0 conn(0x7f85500ce540 legacy=0x7f8550099df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854f7fe640 1 -- 172.21.15.82:0/103887562 >> v1:172.21.15.82:6789/0 conn(0x7f855006d330 legacy=0x7f85500cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854e7fc640 1 -- 172.21.15.82:0/103887562 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935746373 0 0) 0x7f855006c570 con 0x7f855006d330 2024-04-25T10:11:39.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.061+0000 7f854f7fe640 1 -- 172.21.15.82:0/103887562 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855006cb30 con 0x7f855015d8d0 2024-04-25T10:11:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f854f7fe640 1 --2- 172.21.15.82:0/103887562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f855015dcb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f85400097c0 tx=0x7f854001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4789e4543ddc4d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f854e7fc640 1 -- 172.21.15.82:0/103887562 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85400b0070 con 0x7f855015d8d0 2024-04-25T10:11:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f854e7fc640 1 -- 172.21.15.82:0/103887562 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540004230 con 0x7f855015d8d0 2024-04-25T10:11:39.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f854e7fc640 1 -- 172.21.15.82:0/103887562 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8540003620 con 0x7f855015d8d0 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- 172.21.15.82:0/103887562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 msgr2=0x7f855015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 --2- 172.21.15.82:0/103887562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f855015dcb0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f85400097c0 tx=0x7f854001c440 comp rx=0 tx=0).stop 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- 172.21.15.82:0/103887562 shutdown_connections 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 --2- 172.21.15.82:0/103887562 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f855015dcb0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- 172.21.15.82:0/103887562 >> 172.21.15.82:0/103887562 conn(0x7f85500d15b0 msgr2=0x7f85500d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- 172.21.15.82:0/103887562 shutdown_connections 2024-04-25T10:11:39.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- 172.21.15.82:0/103887562 wait complete. 2024-04-25T10:11:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 Processor -- start 2024-04-25T10:11:39.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.065+0000 7f855612a640 1 -- start start 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f855006d330 0x7f85500b4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85500ce540 0x7f85500b4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cc7e0 con 0x7f855015d8d0 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cc660 con 0x7f855006d330 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500cc960 con 0x7f85500ce540 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57690/0 (socket says 172.21.15.82:57690) 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 -- 172.21.15.82:0/2102472146 learned_addr learned my addr 172.21.15.82:0/2102472146 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:39.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854f7fe640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f855006d330 0x7f85500b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854effd640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85500ce540 0x7f85500b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 -- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85500ce540 msgr2=0x7f85500b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85500ce540 0x7f85500b4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 -- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f855006d330 msgr2=0x7f85500b4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f855006d330 0x7f85500b4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 -- 172.21.15.82:0/2102472146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f855015fb30 con 0x7f855015d8d0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f854ffff640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f854800c730 tx=0x7f854800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548003b80 con 0x7f855015d8d0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8548003d20 con 0x7f855015d8d0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855015ed70 con 0x7f855015d8d0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548005310 con 0x7f855015d8d0 2024-04-25T10:11:39.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855015f330 con 0x7f855015d8d0 2024-04-25T10:11:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855004efa0 con 0x7f855015d8d0 2024-04-25T10:11:39.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85480056c0 con 0x7f855015d8d0 2024-04-25T10:11:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.069+0000 7f8537fff640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 0x7f853c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.073+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8548085b80 con 0x7f855015d8d0 2024-04-25T10:11:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.073+0000 7f854f7fe640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 0x7f853c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.073+0000 7f854f7fe640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 0x7f853c069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f85400ad000 tx=0x7f85400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.077 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.073+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8548056b70 con 0x7f855015d8d0 2024-04-25T10:11:39.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8550070a90 con 0x7f855015d8d0 2024-04-25T10:11:39.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f8537fff640 1 -- 172.21.15.82:0/2102472146 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8548056470 con 0x7f855015d8d0 2024-04-25T10:11:39.236 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:39.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 msgr2=0x7f853c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 0x7f853c069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f85400ad000 tx=0x7f85400a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 msgr2=0x7f85500b64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.233+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f854800c730 tx=0x7f854800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 shutdown_connections 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f853c066f60 0x7f853c069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85500ce540 0x7f85500b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f855015d8d0 0x7f85500b64f0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 --2- 172.21.15.82:0/2102472146 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f855006d330 0x7f85500b4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 >> 172.21.15.82:0/2102472146 conn(0x7f85500d15b0 msgr2=0x7f855006f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 shutdown_connections 2024-04-25T10:11:39.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.237+0000 7f855612a640 1 -- 172.21.15.82:0/2102472146 wait complete. 2024-04-25T10:11:39.254 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:39.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 Processor -- start 2024-04-25T10:11:39.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 -- start start 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc3c0cf590 0x7fdc3c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c0cbd20 con 0x7fdc3c15b6b0 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c0cc000 con 0x7fdc3c072000 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc43a4e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c15ba90 con 0x7fdc3c0cf590 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc40fe9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc3c0cf590 0x7fdc3c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc417ea640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdc3c15b6b0 0x7fdc3c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42094/0 (socket says 172.21.15.82:42094) 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc417ea640 1 -- 172.21.15.82:0/3823069791 learned_addr learned my addr 172.21.15.82:0/3823069791 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:39.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc40fe9640 1 -- 172.21.15.82:0/3823069791 >> v1:172.21.15.196:6789/0 conn(0x7fdc3c072000 legacy=0x7fdc3c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc40fe9640 1 -- 172.21.15.82:0/3823069791 >> v1:172.21.15.82:6789/0 conn(0x7fdc3c15b6b0 legacy=0x7fdc3c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.377+0000 7fdc40fe9640 1 -- 172.21.15.82:0/3823069791 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0cc2e0 con 0x7fdc3c0cf590 2024-04-25T10:11:39.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc40fe9640 1 --2- 172.21.15.82:0/3823069791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0cfa20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0097c0 tx=0x7fdc2c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b359db34d5b527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc33fff640 1 -- 172.21.15.82:0/3823069791 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c0b0070 con 0x7fdc3c0cf590 2024-04-25T10:11:39.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc33fff640 1 -- 172.21.15.82:0/3823069791 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c004230 con 0x7fdc3c0cf590 2024-04-25T10:11:39.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc33fff640 1 -- 172.21.15.82:0/3823069791 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c003620 con 0x7fdc3c0cf590 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- 172.21.15.82:0/3823069791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c0cf590 msgr2=0x7fdc3c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/3823069791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0cfa20 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0097c0 tx=0x7fdc2c01c460 comp rx=0 tx=0).stop 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- 172.21.15.82:0/3823069791 shutdown_connections 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/3823069791 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0cfa20 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- 172.21.15.82:0/3823069791 >> 172.21.15.82:0/3823069791 conn(0x7fdc3c06ae60 msgr2=0x7fdc3c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- 172.21.15.82:0/3823069791 shutdown_connections 2024-04-25T10:11:39.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- 172.21.15.82:0/3823069791 wait complete. 2024-04-25T10:11:39.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 Processor -- start 2024-04-25T10:11:39.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- start start 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc3c15b6b0 0x7fdc3c0ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0caf30 con 0x7fdc3c072000 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0cadb0 con 0x7fdc3c0cf590 2024-04-25T10:11:39.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc43a4e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1692b0 con 0x7fdc3c15b6b0 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.381+0000 7fdc417ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57716/0 (socket says 172.21.15.82:57716) 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 -- 172.21.15.82:0/4012731063 learned_addr learned my addr 172.21.15.82:0/4012731063 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc41feb640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc3c15b6b0 0x7fdc3c0ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc40fe9640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 -- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc3c15b6b0 msgr2=0x7fdc3c0ad940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc3c15b6b0 0x7fdc3c0ad940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 -- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c0cf590 msgr2=0x7fdc3c0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0ad400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 -- 172.21.15.82:0/4012731063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0988a0 con 0x7fdc3c072000 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc417ea640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c370 tx=0x7fdc3800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38017070 con 0x7fdc3c072000 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc38005cf0 con 0x7fdc3c072000 2024-04-25T10:11:39.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c098b40 con 0x7fdc3c072000 2024-04-25T10:11:39.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380072f0 con 0x7fdc3c072000 2024-04-25T10:11:39.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c152500 con 0x7fdc3c072000 2024-04-25T10:11:39.389 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c04efa0 con 0x7fdc3c072000 2024-04-25T10:11:39.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc3801a070 con 0x7fdc3c072000 2024-04-25T10:11:39.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 0x7fdc20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdc38084460 con 0x7fdc3c072000 2024-04-25T10:11:39.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc40fe9640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 0x7fdc20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.385+0000 7fdc40fe9640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 0x7fdc20069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0ad000 tx=0x7fdc2c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.389+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc38054c90 con 0x7fdc3c072000 2024-04-25T10:11:39.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.549+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc3c0c60f0 con 0x7fdc3c072000 2024-04-25T10:11:39.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.549+0000 7fdc31ffb640 1 -- 172.21.15.82:0/4012731063 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc38058d30 con 0x7fdc3c072000 2024-04-25T10:11:39.553 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 msgr2=0x7fdc20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 0x7fdc20069450 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0ad000 tx=0x7fdc2c0a6000 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 msgr2=0x7fdc3c06ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fdc3800c370 tx=0x7fdc3800c830 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 shutdown_connections 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc20066f90 0x7fdc20069450 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc3c15b6b0 0x7fdc3c0ad940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc3c0cf590 0x7fdc3c0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 --2- 172.21.15.82:0/4012731063 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc3c072000 0x7fdc3c06ce50 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.555 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 >> 172.21.15.82:0/4012731063 conn(0x7fdc3c06ae60 msgr2=0x7fdc3c14fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 shutdown_connections 2024-04-25T10:11:39.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.553+0000 7fdc43a4e640 1 -- 172.21.15.82:0/4012731063 wait complete. 2024-04-25T10:11:39.572 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 Processor -- start 2024-04-25T10:11:39.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 -- start start 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf940d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9406be60 con 0x7fcf9415da80 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9415de60 con 0x7fcf940d1cd0 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf98ba1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9406c440 con 0x7fcf940d07f0 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcf9415da80 0x7fcf940d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:42104/0 (socket says 172.21.15.82:42104) 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9259c640 1 -- 172.21.15.82:0/3553235893 learned_addr learned my addr 172.21.15.82:0/3553235893 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237269479 0 0) 0x7fcf9406be60 con 0x7fcf9415da80 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf7c003530 con 0x7fcf9415da80 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf91d9b640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf940d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358773965 0 0) 0x7fcf7c003530 con 0x7fcf9415da80 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 msgr2=0x7fcf940d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf940d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 >> v1:172.21.15.196:6789/0 conn(0x7fcf940d1cd0 legacy=0x7fcf9406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940d2450 con 0x7fcf9415da80 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcf84001680 con 0x7fcf9415da80 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 >> v1:172.21.15.82:6789/0 conn(0x7fcf9415da80 legacy=0x7fcf940d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c003a60 0x7fcf7c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c004da0 0x7fcf7c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9406c440 con 0x7fcf7c004da0 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00bbc0 con 0x7fcf7c003a60 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00bdc0 con 0x7fcf940d07f0 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c004da0 0x7fcf7c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf9259c640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c003a60 0x7fcf7c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.685+0000 7fcf91d9b640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 -- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 msgr2=0x7fcf7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf940d07f0 0x7fcf7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 -- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c003a60 msgr2=0x7fcf7c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf7c003a60 0x7fcf7c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 -- 172.21.15.82:0/3553235893 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c003530 con 0x7fcf7c004da0 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c004da0 0x7fcf7c005220 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001e70 tx=0x7fcf8c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc2fcac4dabbaae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c001a90 con 0x7fcf7c004da0 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c003b80 con 0x7fcf7c004da0 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf9159a640 1 -- 172.21.15.82:0/3553235893 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c005080 con 0x7fcf7c004da0 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 -- 172.21.15.82:0/3553235893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c004da0 msgr2=0x7fcf7c005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/3553235893 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf7c004da0 0x7fcf7c005220 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c001e70 tx=0x7fcf8c01c780 comp rx=0 tx=0).stop 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf92d9d640 1 -- 172.21.15.82:0/3553235893 reap_dead start 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 -- 172.21.15.82:0/3553235893 shutdown_connections 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 -- 172.21.15.82:0/3553235893 >> 172.21.15.82:0/3553235893 conn(0x7fcf9408b000 msgr2=0x7fcf940d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 -- 172.21.15.82:0/3553235893 shutdown_connections 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.689+0000 7fcf98ba1640 1 -- 172.21.15.82:0/3553235893 wait complete. 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 Processor -- start 2024-04-25T10:11:39.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- start start 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7c004da0 0x7fcf9414fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf940b6c50 0x7fcf940b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d26f0 con 0x7fcf940b51c0 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d2160 con 0x7fcf940b6c50 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf940d2870 con 0x7fcf7c004da0 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:57732/0 (socket says 172.21.15.82:57732) 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 -- 172.21.15.82:0/1022168626 learned_addr learned my addr 172.21.15.82:0/1022168626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf91d9b640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf940b6c50 0x7fcf940b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf9259c640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7c004da0 0x7fcf9414fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 -- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7c004da0 msgr2=0x7fcf9414fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7c004da0 0x7fcf9414fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 -- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf940b6c50 msgr2=0x7fcf940b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf940b6c50 0x7fcf940b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 -- 172.21.15.82:0/1022168626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf940b7710 con 0x7fcf940b51c0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf92d9d640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c01c5b0 tx=0x7fcf7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c001a90 con 0x7fcf940b51c0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c003980 con 0x7fcf940b51c0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c005840 con 0x7fcf940b51c0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf940b79f0 con 0x7fcf940b51c0 2024-04-25T10:11:39.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf940b7fb0 con 0x7fcf940b51c0 2024-04-25T10:11:39.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9404efa0 con 0x7fcf940b51c0 2024-04-25T10:11:39.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf8c0bb020 con 0x7fcf940b51c0 2024-04-25T10:11:39.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.693+0000 7fcf82ffd640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 0x7fcf74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:39.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.697+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcf8c11ec00 con 0x7fcf940b51c0 2024-04-25T10:11:39.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.697+0000 7fcf9259c640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 0x7fcf74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:39.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.697+0000 7fcf9259c640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 0x7fcf74069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001ec0 tx=0x7fcf84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:39.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.701+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf8c125080 con 0x7fcf940b51c0 2024-04-25T10:11:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.865+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcf940737e0 con 0x7fcf940b51c0 2024-04-25T10:11:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.865+0000 7fcf82ffd640 1 -- 172.21.15.82:0/1022168626 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcf8c0c2020 con 0x7fcf940b51c0 2024-04-25T10:11:39.868 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 msgr2=0x7fcf74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 0x7fcf74069420 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001ec0 tx=0x7fcf84009040 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 msgr2=0x7fcf940b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c01c5b0 tx=0x7fcf7c005840 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 shutdown_connections 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcf74066f60 0x7fcf74069420 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcf940b51c0 0x7fcf940b5640 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcf940b6c50 0x7fcf940b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 --2- 172.21.15.82:0/1022168626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcf7c004da0 0x7fcf9414fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 >> 172.21.15.82:0/1022168626 conn(0x7fcf9408b000 msgr2=0x7fcf940b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 shutdown_connections 2024-04-25T10:11:39.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:39.869+0000 7fcf98ba1640 1 -- 172.21.15.82:0/1022168626 wait complete. 2024-04-25T10:11:39.886 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:11:39.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:39.887 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:39.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[51868]: fuse finished with error 0 and tester_r 0 2024-04-25T10:11:39.897 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:45.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:11:45.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:45.999 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:46.007 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.177+0000 7fe12d40f640 1 Processor -- start 2024-04-25T10:11:46.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 -- start start 2024-04-25T10:11:46.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 0x7fe1280b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe12815ab30 con 0x7fe128066bb0 2024-04-25T10:11:46.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe128067100 con 0x7fe12806c3b0 2024-04-25T10:11:46.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe12d40f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280cf560 con 0x7fe12815a750 2024-04-25T10:11:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe1277fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 0x7fe1280b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:47378/0 (socket says 172.21.15.82:47378) 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 -- 172.21.15.82:0/567106553 learned_addr learned my addr 172.21.15.82:0/567106553 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 -- 172.21.15.82:0/567106553 >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 msgr2=0x7fe1280b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 --2- 172.21.15.82:0/567106553 >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 0x7fe1280b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 -- 172.21.15.82:0/567106553 >> v1:172.21.15.196:6789/0 conn(0x7fe128066bb0 legacy=0x7fe12806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe125ffb640 1 -- 172.21.15.82:0/567106553 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749280346 0 0) 0x7fe12815ab30 con 0x7fe128066bb0 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe126ffd640 1 -- 172.21.15.82:0/567106553 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1280cf840 con 0x7fe12815a750 2024-04-25T10:11:46.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.181+0000 7fe1277fe640 1 --2- 172.21.15.82:0/567106553 >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 0x7fe1280b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:46.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe126ffd640 1 --2- 172.21.15.82:0/567106553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009870 tx=0x7fe11c01c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88ad07180ca9fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe125ffb640 1 -- 172.21.15.82:0/567106553 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0b1070 con 0x7fe12815a750 2024-04-25T10:11:46.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe125ffb640 1 -- 172.21.15.82:0/567106553 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe11c0a4e20 con 0x7fe12815a750 2024-04-25T10:11:46.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe125ffb640 1 -- 172.21.15.82:0/567106553 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe11c0ac660 con 0x7fe12815a750 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- 172.21.15.82:0/567106553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 msgr2=0x7fe12806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- 172.21.15.82:0/567106553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe11c009870 tx=0x7fe11c01c440 comp rx=0 tx=0).stop 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- 172.21.15.82:0/567106553 shutdown_connections 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- 172.21.15.82:0/567106553 >> v2:172.21.15.196:3300/0 conn(0x7fe12806c3b0 0x7fe1280b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- 172.21.15.82:0/567106553 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe12815a750 0x7fe12806ee60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- 172.21.15.82:0/567106553 >> 172.21.15.82:0/567106553 conn(0x7fe128071810 msgr2=0x7fe128071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- 172.21.15.82:0/567106553 shutdown_connections 2024-04-25T10:11:46.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- 172.21.15.82:0/567106553 wait complete. 2024-04-25T10:11:46.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 Processor -- start 2024-04-25T10:11:46.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- start start 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe128066bb0 0x7fe1280b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe12815a750 0x7fe1281428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128165920 con 0x7fe12806c3b0 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1281657a0 con 0x7fe12815a750 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.185+0000 7fe12d40f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe128165aa0 con 0x7fe128066bb0 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55296/0 (socket says 172.21.15.82:55296) 2024-04-25T10:11:46.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 -- 172.21.15.82:0/4172290939 learned_addr learned my addr 172.21.15.82:0/4172290939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe126ffd640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe128066bb0 0x7fe1280b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1277fe640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe12815a750 0x7fe1281428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 -- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe128066bb0 msgr2=0x7fe1280b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe128066bb0 0x7fe1280b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 -- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe12815a750 msgr2=0x7fe1281428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe12815a750 0x7fe1281428f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 -- 172.21.15.82:0/4172290939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128154950 con 0x7fe12806c3b0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe1267fc640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c520 tx=0x7fe11800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe118017070 con 0x7fe12806c3b0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe118003600 con 0x7fe12806c3b0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1281f77d0 con 0x7fe12806c3b0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1180039f0 con 0x7fe12806c3b0 2024-04-25T10:11:46.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1281544c0 con 0x7fe12806c3b0 2024-04-25T10:11:46.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12804efa0 con 0x7fe12806c3b0 2024-04-25T10:11:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe11801a070 con 0x7fe12806c3b0 2024-04-25T10:11:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 0x7fe114069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1180845f0 con 0x7fe12806c3b0 2024-04-25T10:11:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe126ffd640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 0x7fe114069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.189+0000 7fe126ffd640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 0x7fe114069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0ae000 tx=0x7fe11c0a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.193+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe118055460 con 0x7fe12806c3b0 2024-04-25T10:11:46.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.389+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1280727c0 con 0x7fe12806c3b0 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.389+0000 7fe10f7fe640 1 -- 172.21.15.82:0/4172290939 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fe118054d60 con 0x7fe12806c3b0 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:46.392 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:11:30.539889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8769":{"gid":8769,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3162787996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3162787996},{"type":"v1","addr":"172.21.15.82:6837","nonce":3162787996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:46.393 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 75 2024-04-25T10:11:46.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 msgr2=0x7fe114069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 0x7fe114069420 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe11c0ae000 tx=0x7fe11c0a6070 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 msgr2=0x7fe1280b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fe11800c520 tx=0x7fe11800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 shutdown_connections 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe114066f60 0x7fe114069420 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe12815a750 0x7fe1281428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe12806c3b0 0x7fe1280b0ca0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 --2- 172.21.15.82:0/4172290939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe128066bb0 0x7fe1280b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 >> 172.21.15.82:0/4172290939 conn(0x7fe128071810 msgr2=0x7fe1280afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 shutdown_connections 2024-04-25T10:11:46.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.393+0000 7fe12d40f640 1 -- 172.21.15.82:0/4172290939 wait complete. 2024-04-25T10:11:46.412 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:11:46.556 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:11:46.556 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:11:46.556 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.661+0000 7f3d73031640 1 Processor -- start 2024-04-25T10:11:46.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 -- start start 2024-04-25T10:11:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d6c0af390 0x7f3d6c066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3d6c15dd50 0x7f3d6c06bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c067210 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b2f10 con 0x7f3d6c0af390 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d73031640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c06d850 con 0x7f3d6c15dd50 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3d6c0af390 0x7f3d6c066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d715ce640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3d6c06c1e0 0x7f3d6c1613a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53598/0 (socket says 172.21.15.82:53598) 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d715ce640 1 -- 172.21.15.82:0/3754406647 learned_addr learned my addr 172.21.15.82:0/3754406647 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/3754406647 >> v2:172.21.15.82:3300/0 conn(0x7f3d6c0af390 0x7f3d6c066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55300/0 (socket says 172.21.15.82:55300) 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 -- 172.21.15.82:0/3754406647 >> v2:172.21.15.196:3300/0 conn(0x7f3d6c15dd50 msgr2=0x7f3d6c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/3754406647 >> v2:172.21.15.196:3300/0 conn(0x7f3d6c15dd50 0x7f3d6c06bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 -- 172.21.15.82:0/3754406647 >> v1:172.21.15.82:6789/0 conn(0x7f3d6c06c1e0 legacy=0x7f3d6c1613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 -- 172.21.15.82:0/3754406647 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c06c5f0 con 0x7f3d6c0af390 2024-04-25T10:11:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.665+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/3754406647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c0af390 0x7f3d6c066cd0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0746e0 tx=0x7f3d6801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29f1e275e6ce866f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d637fe640 1 -- 172.21.15.82:0/3754406647 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b0070 con 0x7f3d6c0af390 2024-04-25T10:11:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d637fe640 1 -- 172.21.15.82:0/3754406647 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d680ab600 con 0x7f3d6c0af390 2024-04-25T10:11:46.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d637fe640 1 -- 172.21.15.82:0/3754406647 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680ab9d0 con 0x7f3d6c0af390 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- 172.21.15.82:0/3754406647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c0af390 msgr2=0x7f3d6c066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- 172.21.15.82:0/3754406647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c0af390 0x7f3d6c066cd0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c0746e0 tx=0x7f3d6801c330 comp rx=0 tx=0).stop 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- 172.21.15.82:0/3754406647 shutdown_connections 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- 172.21.15.82:0/3754406647 >> v2:172.21.15.196:3300/0 conn(0x7f3d6c15dd50 0x7f3d6c06bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- 172.21.15.82:0/3754406647 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c0af390 0x7f3d6c066cd0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- 172.21.15.82:0/3754406647 >> 172.21.15.82:0/3754406647 conn(0x7f3d6c0717f0 msgr2=0x7f3d6c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- 172.21.15.82:0/3754406647 shutdown_connections 2024-04-25T10:11:46.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- 172.21.15.82:0/3754406647 wait complete. 2024-04-25T10:11:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 Processor -- start 2024-04-25T10:11:46.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- start start 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d6c0af390 0x7f3d6c14ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d6c15dd50 0x7f3d6c0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c16a740 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c15eb80 con 0x7f3d6c15dd50 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d73031640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c16a8c0 con 0x7f3d6c0af390 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55308/0 (socket says 172.21.15.82:55308) 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 -- 172.21.15.82:0/2293349817 learned_addr learned my addr 172.21.15.82:0/2293349817 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d63fff640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d6c0af390 0x7f3d6c14ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d715ce640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d6c15dd50 0x7f3d6c0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 -- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d6c0af390 msgr2=0x7f3d6c14ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d6c0af390 0x7f3d6c14ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 -- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d6c15dd50 msgr2=0x7f3d6c0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d6c15dd50 0x7f3d6c0b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.669+0000 7f3d70dcd640 1 -- 172.21.15.82:0/2293349817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c14c720 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d70dcd640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3d680ae040 tx=0x7f3d680190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b8070 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d680b4610 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0aeba0 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d680b0040 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c14b680 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d6c04efa0 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d680bb070 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 0x7f3d58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d6811efb0 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d63fff640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 0x7f3d58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.673+0000 7f3d63fff640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 0x7f3d58069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c14d050 tx=0x7f3d5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.677+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d6c14c720 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.853+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d6c0cf680 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.853+0000 7f3d617fa640 1 -- 172.21.15.82:0/2293349817 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3d6c06c5f0 con 0x7f3d6c06c1e0 2024-04-25T10:11:46.857 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:46.858 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:11:30.539889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8769":{"gid":8769,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3162787996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3162787996},{"type":"v1","addr":"172.21.15.82:6837","nonce":3162787996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:11:46.858 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 75 2024-04-25T10:11:46.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 msgr2=0x7f3d58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 0x7f3d58069450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c14d050 tx=0x7f3d5c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 msgr2=0x7f3d6c0ae660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f3d680ae040 tx=0x7f3d680190d0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 shutdown_connections 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3d58066f90 0x7f3d58069450 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3d6c0af390 0x7f3d6c14ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3d6c15dd50 0x7f3d6c0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 --2- 172.21.15.82:0/2293349817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3d6c06c1e0 0x7f3d6c0ae660 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 >> 172.21.15.82:0/2293349817 conn(0x7f3d6c0717f0 msgr2=0x7f3d6c15eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 shutdown_connections 2024-04-25T10:11:46.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.857+0000 7f3d73031640 1 -- 172.21.15.82:0/2293349817 wait complete. 2024-04-25T10:11:46.875 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T10:11:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 Processor -- start 2024-04-25T10:11:46.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 -- start start 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79b3a210 0x55bd79b37070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a5f0 0x55bd79b3ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd79b3b970 con 0x55bd79aae350 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd79a0b9f0 con 0x55bd79b3a210 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2aba155080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd79b3baf0 con 0x55bd79b3a5f0 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55330/0 (socket says 172.21.15.82:55330) 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 -- 172.21.15.82:0/2788505022 learned_addr learned my addr 172.21.15.82:0/2788505022 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:46.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 -- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a5f0 msgr2=0x55bd79b3ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a5f0 0x55bd79b3ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 -- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79b3a210 msgr2=0x55bd79b37070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79b3a210 0x55bd79b37070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.901+0000 7f2ab7c73640 1 -- 172.21.15.82:0/2788505022 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd79b397b0 con 0x55bd79aae350 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ad040 tx=0x7f2aa8003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535e5c1762680168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab6470640 1 -- 172.21.15.82:0/2788505022 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80af040 con 0x55bd79aae350 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab6470640 1 -- 172.21.15.82:0/2788505022 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa80b9070 con 0x55bd79aae350 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab6470640 1 -- 172.21.15.82:0/2788505022 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80aae10 con 0x55bd79aae350 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/2788505022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 msgr2=0x55bd79b3d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ad040 tx=0x7f2aa8003030 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/2788505022 shutdown_connections 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a5f0 0x55bd79b3ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79b3a210 0x55bd79b37070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- 172.21.15.82:0/2788505022 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79aae350 0x55bd79b3d5a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/2788505022 >> 172.21.15.82:0/2788505022 conn(0x55bd79a8fab0 msgr2=0x55bd79a68240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/2788505022 shutdown_connections 2024-04-25T10:11:46.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/2788505022 wait complete. 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 Processor -- start 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- start start 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79aae350 0x55bd79b43cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a210 0x55bd79bd2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 0x55bd79a66840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd79b39a50 con 0x55bd79b3a5f0 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd79b39550 con 0x55bd79aae350 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bd79b39cf0 con 0x55bd79b3a210 2024-04-25T10:11:46.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 0x55bd79a66840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 0x55bd79a66840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55334/0 (socket says 172.21.15.82:55334) 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 -- 172.21.15.82:0/3342825365 learned_addr learned my addr 172.21.15.82:0/3342825365 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79aae350 0x55bd79b43cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 -- 172.21.15.82:0/3342825365 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a210 msgr2=0x55bd79bd2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bd79b3a210 0x55bd79bd2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 -- 172.21.15.82:0/3342825365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79aae350 msgr2=0x55bd79b43cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bd79aae350 0x55bd79b43cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 -- 172.21.15.82:0/3342825365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd79b36c10 con 0x55bd79b3a5f0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab8474640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 0x55bd79a66840 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000a190 tx=0x7f2ab0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab546e640 1 -- 172.21.15.82:0/3342825365 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab000ae10 con 0x55bd79b3a5f0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab546e640 1 -- 172.21.15.82:0/3342825365 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab00115f0 con 0x55bd79b3a5f0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bd79b36430 con 0x55bd79b3a5f0 2024-04-25T10:11:46.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2ab546e640 1 -- 172.21.15.82:0/3342825365 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab00117e0 con 0x55bd79b3a5f0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.905+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bd79b36970 con 0x55bd79b3a5f0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab546e640 1 -- 172.21.15.82:0/3342825365 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ab001a020 con 0x55bd79b3a5f0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab546e640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aa4065970 0x7f2aa4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab546e640 1 -- 172.21.15.82:0/3342825365 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2ab0083be0 con 0x55bd79b3a5f0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aa4065970 0x7f2aa4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55bd79b33330 0x55bd79b33710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e107) v9 -- 0x55bd79b320d0 con 0x55bd79b33330 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab7472640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55bd79b33330 0x55bd79b33710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab7c73640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aa4065970 0x7f2aa4067e30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ad040 tx=0x7f2aa80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab7472640 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55bd79b33330 0x55bd79b33710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:46.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab7472640 1 -- 172.21.15.82:0/3342825365 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f2aac00b3d0 con 0x55bd79b33330 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55bd79b33330 msgr2=0x55bd79b33710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55bd79b33330 0x55bd79b33710 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aa4065970 msgr2=0x7f2aa4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2aa4065970 0x7f2aa4067e30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80ad040 tx=0x7f2aa80a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 msgr2=0x55bd79a66840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 --2- 172.21.15.82:0/3342825365 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bd79b3a5f0 0x55bd79a66840 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2ab000a190 tx=0x7f2ab0005f90 comp rx=0 tx=0).stop 2024-04-25T10:11:46.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2ab8474640 1 -- 172.21.15.82:0/3342825365 reap_dead start 2024-04-25T10:11:46.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 shutdown_connections 2024-04-25T10:11:46.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 >> 172.21.15.82:0/3342825365 conn(0x55bd79a8fab0 msgr2=0x55bd79b37310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:46.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 shutdown_connections 2024-04-25T10:11:46.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:46.909+0000 7f2aba155080 1 -- 172.21.15.82:0/3342825365 wait complete. 2024-04-25T10:11:46.917 INFO:teuthology.orchestra.run.smithi082.stdout:����� 2024-04-25T10:11:46.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:47.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 Processor -- start 2024-04-25T10:11:47.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 -- start start 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc159ea0 0x7f44bc0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44bc15a280 0x7f44bc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc15ae30 con 0x7f44bc0718c0 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc0b2990 con 0x7f44bc15a280 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c3375640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc15b110 con 0x7f44bc159ea0 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44bc15a280 0x7f44bc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f44bc15a280 0x7f44bc15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55346/0 (socket says 172.21.15.82:55346) 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 -- 172.21.15.82:0/2781949282 learned_addr learned my addr 172.21.15.82:0/2781949282 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c1912640 1 --1- 172.21.15.82:0/2781949282 >> v1:172.21.15.82:6789/0 conn(0x7f44bc0718c0 0x7f44bc06d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53614/0 (socket says 172.21.15.82:53614) 2024-04-25T10:11:47.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c1111640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc159ea0 0x7f44bc0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 -- 172.21.15.82:0/2781949282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc159ea0 msgr2=0x7f44bc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc159ea0 0x7f44bc0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 -- 172.21.15.82:0/2781949282 >> v1:172.21.15.82:6789/0 conn(0x7f44bc0718c0 legacy=0x7f44bc06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44c0910640 1 -- 172.21.15.82:0/2781949282 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0b2620 con 0x7f44bc15a280 2024-04-25T10:11:47.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.069+0000 7f44b3fff640 1 -- 172.21.15.82:0/2781949282 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453076884 0 0) 0x7f44bc15ae30 con 0x7f44bc0718c0 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc15a280 0x7f44bc15a700 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab75782fac77c94b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b3fff640 1 -- 172.21.15.82:0/2781949282 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0a7070 con 0x7f44bc15a280 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b3fff640 1 -- 172.21.15.82:0/2781949282 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac0b0070 con 0x7f44bc15a280 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b3fff640 1 -- 172.21.15.82:0/2781949282 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0ab890 con 0x7f44bc15a280 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2781949282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc15a280 msgr2=0x7f44bc15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc15a280 0x7f44bc15a700 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac01c560 comp rx=0 tx=0).stop 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2781949282 shutdown_connections 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc15a280 0x7f44bc15a700 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- 172.21.15.82:0/2781949282 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc159ea0 0x7f44bc0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2781949282 >> 172.21.15.82:0/2781949282 conn(0x7f44bc0b11a0 msgr2=0x7f44bc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2781949282 shutdown_connections 2024-04-25T10:11:47.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2781949282 wait complete. 2024-04-25T10:11:47.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 Processor -- start 2024-04-25T10:11:47.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- start start 2024-04-25T10:11:47.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 0x7f44bc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 0x7f44bc140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc167340 con 0x7f44bc159ea0 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc1671c0 con 0x7f44bc15a280 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc1674c0 con 0x7f44bc0718c0 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55354/0 (socket says 172.21.15.82:55354) 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 -- 172.21.15.82:0/2195199493 learned_addr learned my addr 172.21.15.82:0/2195199493 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c1111640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 0x7f44bc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c1912640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 0x7f44bc140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 -- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 msgr2=0x7f44bc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 0x7f44bc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 -- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 msgr2=0x7f44bc140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 0x7f44bc140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 -- 172.21.15.82:0/2195199493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0a8720 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c1111640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 0x7f44bc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c1912640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 0x7f44bc140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c0910640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0ad040 tx=0x7f44ac01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0be5f0 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac0be790 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc0a8a00 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0beb80 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.073+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc0cdf10 con 0x7f44bc159ea0 2024-04-25T10:11:47.080 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc0ce150 con 0x7f44bc159ea0 2024-04-25T10:11:47.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f44ac0b43f0 con 0x7f44bc159ea0 2024-04-25T10:11:47.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44b1ffb640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 0x7f44a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:47.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f44ac11fa80 con 0x7f44bc159ea0 2024-04-25T10:11:47.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44c1111640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 0x7f44a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:47.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.077+0000 7f44c1111640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 0x7f44a0069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009840 tx=0x7f44b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:47.083 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.081+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f44bc0ce150 con 0x7f44bc159ea0 2024-04-25T10:11:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.269+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f44bc04efa0 con 0x7f44bc159ea0 2024-04-25T10:11:47.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.269+0000 7f44b1ffb640 1 -- 172.21.15.82:0/2195199493 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v75) v1 ==== 53+0+73 (secure 0 0 0) 0x7f44ac0c4060 con 0x7f44bc159ea0 2024-04-25T10:11:47.274 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 msgr2=0x7f44a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 0x7f44a0069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009840 tx=0x7f44b8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 msgr2=0x7f44bc1f8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0ad040 tx=0x7f44ac01ada0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 shutdown_connections 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f44a0066f60 0x7f44a0069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f44bc15a280 0x7f44bc140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f44bc159ea0 0x7f44bc1f8070 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 --2- 172.21.15.82:0/2195199493 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f44bc0718c0 0x7f44bc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 >> 172.21.15.82:0/2195199493 conn(0x7f44bc0b11a0 msgr2=0x7f44bc153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 shutdown_connections 2024-04-25T10:11:47.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:47.273+0000 7f44c3375640 1 -- 172.21.15.82:0/2195199493 wait complete. 2024-04-25T10:11:47.291 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:11:47.291 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:47.291 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:11:47.291 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:47.292 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:47.292 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:11:47.292 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:11:47.292 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:47.292 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:47.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.292 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:11:47.299 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41962sec preferred_lft 41962sec 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:47.300 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:47.301 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:11:47.302 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:47.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.302 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:47.354 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:11:47.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.355 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:11:47.405 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:47.406 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:47.406 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:47.406 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:11:47.406 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:47.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.406 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:11:47.406 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:11:47.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:47.406 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:47.407 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:47.407 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:11:47.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.476 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:11:47.527 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:47.527 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:47.527 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:47.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.527 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:11:47.580 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:11:47.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:47.580 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:47.626 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:47.626 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:47.626 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:47.675 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:47.736 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:47.783 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:47.783 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:47.784 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:47.785 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:47.786 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:47.786 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:47.786 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:47.786 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:47.786 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:47.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.787 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:47.834 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:11:47.835 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:11:47.880 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:47.894 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:47.942 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:47.943 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:47.944 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:47.945 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:47.945 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:47.945 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:47.945 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:47.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.946 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:47.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:11:47.957+0000 7f884266f580 -1 init, newargv = 0x56515df60400 newargc=15 2024-04-25T10:11:47.960 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[52328]: starting ceph client 2024-04-25T10:11:47.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[52328]: starting fuse 2024-04-25T10:11:49.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:11:49.083 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:11:49.132 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:11:49.133 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:49.134 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:49.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.135 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:11:49.187 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:11:49.187 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:11:49.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:11:49.187 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:11:49.187 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:11:49.187 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:49.187 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:11:49.188 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:11:49.189 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:49.309 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.52328.asok 2024-04-25T10:11:49.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:49.320 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52328.asok 2024-04-25T10:11:49.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52328.asok status 2024-04-25T10:11:49.469 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:11:49.469 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:11:49.469 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "52328", 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9042, 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:11:49.470 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 9042 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2180048763 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2180048763 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:11:49.471 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.9042 192.168.0.1:0/2180048763", 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2180048763", 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 75, 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 107, 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:11:49.472 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:11:49.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.483 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:11:49.483 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:49.490 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:11:49.490 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:49.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:49.546 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:49.546 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:49.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.596 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records in 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records out 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi082.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0554676 s, 113 MB/s 2024-04-25T10:11:49.707 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:49.707 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:49.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.714 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:11:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records in 2024-04-25T10:11:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:6+0 records out 2024-04-25T10:11:49.832 INFO:teuthology.orchestra.run.smithi082.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0665945 s, 94.5 MB/s 2024-04-25T10:11:49.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.834 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:11:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:./ 2024-04-25T10:11:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:./file2_sixmegs 2024-04-25T10:11:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:./file3_sixmegs 2024-04-25T10:11:49.845 INFO:teuthology.orchestra.run.smithi082.stdout:./file1_sixmegs 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:11:49.845 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:49.950 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:11:49.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T10:11:49.960 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:50.062 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:11:50.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:50.071 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:11:50.071 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:50.071 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:11:50.072 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:50.072 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T10:11:50.072 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:50.178 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627778 2024-04-25T10:11:50.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T10:11:50.188 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:11:50.292 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:11:50.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:11:50.302 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:50.302 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:50.351 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:50.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.517+0000 7ff221b6f640 1 Processor -- start 2024-04-25T10:11:50.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.517+0000 7ff221b6f640 1 -- start start 2024-04-25T10:11:50.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff21c06fc50 0x7ff21c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06b1d0 0x7ff21c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff21c0cfd80 0x7ff21c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c161560 con 0x7ff21c06fc50 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c06b710 con 0x7ff21c0cfd80 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff221b6f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c1617b0 con 0x7ff21c06b1d0 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff21c06fc50 0x7ff21c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff21c06fc50 0x7ff21c06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55364/0 (socket says 172.21.15.82:55364) 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 -- 172.21.15.82:0/1398539746 learned_addr learned my addr 172.21.15.82:0/1398539746 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 -- 172.21.15.82:0/1398539746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06b1d0 msgr2=0x7ff21c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/1398539746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06b1d0 0x7ff21c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 -- 172.21.15.82:0/1398539746 >> v2:172.21.15.196:3300/0 conn(0x7ff21c0cfd80 msgr2=0x7ff21c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/1398539746 >> v2:172.21.15.196:3300/0 conn(0x7ff21c0cfd80 0x7ff21c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 -- 172.21.15.82:0/1398539746 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c161a90 con 0x7ff21c06fc50 2024-04-25T10:11:50.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.521+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/1398539746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c06fc50 0x7ff21c06ac90 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0b6430 tx=0x7ff21401d420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a56bc493f8b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff21a7fc640 1 -- 172.21.15.82:0/1398539746 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a5860 con 0x7ff21c06fc50 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff21a7fc640 1 -- 172.21.15.82:0/1398539746 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2140a5a00 con 0x7ff21c06fc50 2024-04-25T10:11:50.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff21a7fc640 1 -- 172.21.15.82:0/1398539746 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2140a5dd0 con 0x7ff21c06fc50 2024-04-25T10:11:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- 172.21.15.82:0/1398539746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c06fc50 msgr2=0x7ff21c06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- 172.21.15.82:0/1398539746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c06fc50 0x7ff21c06ac90 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0b6430 tx=0x7ff21401d420 comp rx=0 tx=0).stop 2024-04-25T10:11:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- 172.21.15.82:0/1398539746 shutdown_connections 2024-04-25T10:11:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- 172.21.15.82:0/1398539746 >> v2:172.21.15.196:3300/0 conn(0x7ff21c0cfd80 0x7ff21c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- 172.21.15.82:0/1398539746 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06b1d0 0x7ff21c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- 172.21.15.82:0/1398539746 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c06fc50 0x7ff21c06ac90 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- 172.21.15.82:0/1398539746 >> 172.21.15.82:0/1398539746 conn(0x7ff21c099a70 msgr2=0x7ff21c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- 172.21.15.82:0/1398539746 shutdown_connections 2024-04-25T10:11:50.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- 172.21.15.82:0/1398539746 wait complete. 2024-04-25T10:11:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 Processor -- start 2024-04-25T10:11:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- start start 2024-04-25T10:11:50.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff21c06b1d0 0x7ff21c0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06fc50 0x7ff21c14b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0701b0 con 0x7ff21c0cfd80 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c070030 con 0x7ff21c06b1d0 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.525+0000 7ff221b6f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c162290 con 0x7ff21c06fc50 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55366/0 (socket says 172.21.15.82:55366) 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 -- 172.21.15.82:0/96228968 learned_addr learned my addr 172.21.15.82:0/96228968 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21affd640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06fc50 0x7ff21c14b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff21c06b1d0 0x7ff21c0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 -- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06fc50 msgr2=0x7ff21c14b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06fc50 0x7ff21c14b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 -- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff21c06b1d0 msgr2=0x7ff21c0b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff21c06b1d0 0x7ff21c0b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 -- 172.21.15.82:0/96228968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c14bf10 con 0x7ff21c0cfd80 2024-04-25T10:11:50.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.529+0000 7ff21bfff640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b390 tx=0x7ff20400b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204015020 con 0x7ff21c0cfd80 2024-04-25T10:11:50.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204011830 con 0x7ff21c0cfd80 2024-04-25T10:11:50.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204011c20 con 0x7ff21c0cfd80 2024-04-25T10:11:50.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c0b10a0 con 0x7ff21c0cfd80 2024-04-25T10:11:50.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c15f990 con 0x7ff21c0cfd80 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c04efa0 con 0x7ff21c0cfd80 2024-04-25T10:11:50.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.533+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff204018020 con 0x7ff21c0cfd80 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.537+0000 7ff213fff640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 0x7ff1fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.537+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff2040850a0 con 0x7ff21c0cfd80 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.537+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 0x7ff1fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.537+0000 7ff21b7fe640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 0x7ff1fc069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff2140ae000 tx=0x7ff214004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.541+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff20401f020 con 0x7ff21c0cfd80 2024-04-25T10:11:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.697+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff21c0b6480 con 0x7ff21c0cfd80 2024-04-25T10:11:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff213fff640 1 -- 172.21.15.82:0/96228968 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) v1 ==== 74+0+16946 (secure 0 0 0) 0x7ff2040172c0 con 0x7ff21c0cfd80 2024-04-25T10:11:50.703 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:50.703 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":108,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:50.522688+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 msgr2=0x7ff1fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 0x7ff1fc069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff2140ae000 tx=0x7ff214004ac0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 msgr2=0x7ff21c0b0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ff20400b390 tx=0x7ff20400b850 comp rx=0 tx=0).stop 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 shutdown_connections 2024-04-25T10:11:50.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff1fc066f90 0x7ff1fc069450 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff21c06fc50 0x7ff21c14b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff21c0cfd80 0x7ff21c0b0620 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 --2- 172.21.15.82:0/96228968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff21c06b1d0 0x7ff21c0b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 >> 172.21.15.82:0/96228968 conn(0x7ff21c099a70 msgr2=0x7ff21c0cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 shutdown_connections 2024-04-25T10:11:50.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.701+0000 7ff221b6f640 1 -- 172.21.15.82:0/96228968 wait complete. 2024-04-25T10:11:50.722 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:27.097350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:27.504961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:50.723 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:50.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 Processor -- start 2024-04-25T10:11:50.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 -- start start 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 0x7fab440ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab440cf590 0x7fab440cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 0x7fab4415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4415ba90 con 0x7fab440cf590 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440cff30 con 0x7fab4415f130 2024-04-25T10:11:50.844 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab48c37640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cbd20 con 0x7fab4415b6b0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab440cf590 0x7fab440cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fab440cf590 0x7fab440cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55382/0 (socket says 172.21.15.82:55382) 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 -- 172.21.15.82:0/3098766565 learned_addr learned my addr 172.21.15.82:0/3098766565 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab4259c640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 0x7fab440ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab42d9d640 1 --2- 172.21.15.82:0/3098766565 >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 0x7fab4415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 -- 172.21.15.82:0/3098766565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 msgr2=0x7fab440ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 0x7fab440ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 -- 172.21.15.82:0/3098766565 >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 msgr2=0x7fab4415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 --2- 172.21.15.82:0/3098766565 >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 0x7fab4415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab41d9b640 1 -- 172.21.15.82:0/3098766565 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440cc000 con 0x7fab440cf590 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab4259c640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 0x7fab440ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.845 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.841+0000 7fab42d9d640 1 --2- 172.21.15.82:0/3098766565 >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 0x7fab4415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab41d9b640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab440cf590 0x7fab440cf9f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fab440c2de0 tx=0x7fab3801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53357a026b201e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab4159a640 1 -- 172.21.15.82:0/3098766565 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380aa610 con 0x7fab440cf590 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab4159a640 1 -- 172.21.15.82:0/3098766565 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380af070 con 0x7fab440cf590 2024-04-25T10:11:50.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab4159a640 1 -- 172.21.15.82:0/3098766565 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380aa9d0 con 0x7fab440cf590 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- 172.21.15.82:0/3098766565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab440cf590 msgr2=0x7fab440cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab440cf590 0x7fab440cf9f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fab440c2de0 tx=0x7fab3801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- 172.21.15.82:0/3098766565 shutdown_connections 2024-04-25T10:11:50.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- 172.21.15.82:0/3098766565 >> v2:172.21.15.196:3300/0 conn(0x7fab4415f130 0x7fab4415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab440cf590 0x7fab440cf9f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- 172.21.15.82:0/3098766565 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab4415b6b0 0x7fab440ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- 172.21.15.82:0/3098766565 >> 172.21.15.82:0/3098766565 conn(0x7fab4406ae60 msgr2=0x7fab4406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- 172.21.15.82:0/3098766565 shutdown_connections 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- 172.21.15.82:0/3098766565 wait complete. 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 Processor -- start 2024-04-25T10:11:50.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- start start 2024-04-25T10:11:50.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 0x7fab441f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab4415b6b0 0x7fab441f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4415fef0 con 0x7fab4415f130 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cba30 con 0x7fab4415b6b0 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab48c37640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440ca900 con 0x7fab440cf590 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55390/0 (socket says 172.21.15.82:55390) 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 -- 172.21.15.82:0/2572617021 learned_addr learned my addr 172.21.15.82:0/2572617021 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab4259c640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 0x7fab441f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab41d9b640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab4415b6b0 0x7fab441f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 -- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 msgr2=0x7fab441f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 0x7fab441f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 -- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab4415b6b0 msgr2=0x7fab441f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab4415b6b0 0x7fab441f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 -- 172.21.15.82:0/2572617021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab441653e0 con 0x7fab4415f130 2024-04-25T10:11:50.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab4259c640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 0x7fab441f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab42d9d640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00cd50 tx=0x7fab3c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.845+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00f680 con 0x7fab4415f130 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00f840 con 0x7fab4415f130 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab44165970 con 0x7fab4415f130 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00fc30 con 0x7fab4415f130 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab440cd2b0 con 0x7fab4415f130 2024-04-25T10:11:50.852 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4404efa0 con 0x7fab4415f130 2024-04-25T10:11:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab3c0213f0 con 0x7fab4415f130 2024-04-25T10:11:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab32ffd640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 0x7fab24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fab3c085f00 con 0x7fab4415f130 2024-04-25T10:11:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab4259c640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 0x7fab24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:50.853 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.849+0000 7fab4259c640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 0x7fab24069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fab34008930 tx=0x7fab34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:50.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:50.853+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab3c014030 con 0x7fab4415f130 2024-04-25T10:11:51.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.009+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab440b30a0 con 0x7fab4415f130 2024-04-25T10:11:51.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.009+0000 7fab32ffd640 1 -- 172.21.15.82:0/2572617021 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab3c056d60 con 0x7fab4415f130 2024-04-25T10:11:51.013 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 msgr2=0x7fab24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 0x7fab24069420 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fab34008930 tx=0x7fab34008040 comp rx=0 tx=0).stop 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 msgr2=0x7fab441569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00cd50 tx=0x7fab3c00a650 comp rx=0 tx=0).stop 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 shutdown_connections 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fab24066f60 0x7fab24069420 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fab4415f130 0x7fab441569d0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fab4415b6b0 0x7fab441f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 --2- 172.21.15.82:0/2572617021 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fab440cf590 0x7fab441f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 >> 172.21.15.82:0/2572617021 conn(0x7fab4406ae60 msgr2=0x7fab440b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 shutdown_connections 2024-04-25T10:11:51.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.013+0000 7fab48c37640 1 -- 172.21.15.82:0/2572617021 wait complete. 2024-04-25T10:11:51.030 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:51.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.137+0000 7f859aebb640 1 Processor -- start 2024-04-25T10:11:51.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 -- start start 2024-04-25T10:11:51.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85940cf590 0x7f85940cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8594072000 0x7f8594072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85940cbd20 con 0x7f859415b6b0 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859415ba90 con 0x7f85940cf590 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f859aebb640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85940cc000 con 0x7f8594072000 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85940cf590 0x7f85940cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f85940cf590 0x7f85940cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55398/0 (socket says 172.21.15.82:55398) 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 -- 172.21.15.82:0/1974140908 learned_addr learned my addr 172.21.15.82:0/1974140908 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8599458640 1 --2- 172.21.15.82:0/1974140908 >> v2:172.21.15.196:3300/0 conn(0x7f8594072000 0x7f8594072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 -- 172.21.15.82:0/1974140908 >> v2:172.21.15.196:3300/0 conn(0x7f8594072000 msgr2=0x7f8594072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 --2- 172.21.15.82:0/1974140908 >> v2:172.21.15.196:3300/0 conn(0x7f8594072000 0x7f8594072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 -- 172.21.15.82:0/1974140908 >> v1:172.21.15.196:6789/0 conn(0x7f859415b6b0 legacy=0x7f85940ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 -- 172.21.15.82:0/1974140908 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85940cc2e0 con 0x7f85940cf590 2024-04-25T10:11:51.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f8593fff640 1 --2- 172.21.15.82:0/1974140908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85940cf590 0x7f85940cfa20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f859406f000 tx=0x7f857c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69afb37875a23efc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f85937fe640 1 -- 172.21.15.82:0/1974140908 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0ab610 con 0x7f85940cf590 2024-04-25T10:11:51.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f85937fe640 1 -- 172.21.15.82:0/1974140908 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f857c0b0070 con 0x7f85940cf590 2024-04-25T10:11:51.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.141+0000 7f85937fe640 1 -- 172.21.15.82:0/1974140908 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f857c0ab9d0 con 0x7f85940cf590 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/1974140908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85940cf590 msgr2=0x7f85940cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- 172.21.15.82:0/1974140908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85940cf590 0x7f85940cfa20 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f859406f000 tx=0x7f857c01c520 comp rx=0 tx=0).stop 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/1974140908 shutdown_connections 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- 172.21.15.82:0/1974140908 >> v2:172.21.15.196:3300/0 conn(0x7f8594072000 0x7f8594072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- 172.21.15.82:0/1974140908 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f85940cf590 0x7f85940cfa20 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/1974140908 >> 172.21.15.82:0/1974140908 conn(0x7f859406ae60 msgr2=0x7f859406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/1974140908 shutdown_connections 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/1974140908 wait complete. 2024-04-25T10:11:51.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 Processor -- start 2024-04-25T10:11:51.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- start start 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f859415b6b0 0x7f85940cdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85940afe50 0x7f85940ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940caf30 con 0x7f8594072000 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cadb0 con 0x7f859415b6b0 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85941682a0 con 0x7f85940afe50 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55408/0 (socket says 172.21.15.82:55408) 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 -- 172.21.15.82:0/3298700040 learned_addr learned my addr 172.21.15.82:0/3298700040 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8593fff640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f859415b6b0 0x7f85940cdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8599458640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85940afe50 0x7f85940ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 -- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85940afe50 msgr2=0x7f85940ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85940afe50 0x7f85940ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 -- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f859415b6b0 msgr2=0x7f85940cdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f859415b6b0 0x7f85940cdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 -- 172.21.15.82:0/3298700040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85940ce220 con 0x7f8594072000 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f8598c57640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c730 tx=0x7f858c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c016020 con 0x7f8594072000 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c006d60 con 0x7f8594072000 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85940ce4c0 con 0x7f8594072000 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c003bc0 con 0x7f8594072000 2024-04-25T10:11:51.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.145+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85941533c0 con 0x7f8594072000 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859404efa0 con 0x7f8594072000 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f858c00a850 con 0x7f8594072000 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f85917fa640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f858c084ea0 con 0x7f8594072000 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f8593fff640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.149+0000 7f8593fff640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 0x7f8574069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f857c0af000 tx=0x7f857c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.153+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f858c020020 con 0x7f8594072000 2024-04-25T10:11:51.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85940aeb80 con 0x7f8594072000 2024-04-25T10:11:51.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f85917fa640 1 -- 172.21.15.82:0/3298700040 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) v1 ==== 104+0+11 (secure 0 0 0) 0x7f858c0079f0 con 0x7f8594072000 2024-04-25T10:11:51.303 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 msgr2=0x7f8574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 0x7f8574069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f857c0af000 tx=0x7f857c0a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 msgr2=0x7f8594098680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f858c00c730 tx=0x7f858c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 shutdown_connections 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8574066f60 0x7f8574069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f85940afe50 0x7f85940ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f859415b6b0 0x7f85940cdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 --2- 172.21.15.82:0/3298700040 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8594072000 0x7f8594098680 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 >> 172.21.15.82:0/3298700040 conn(0x7f859406ae60 msgr2=0x7f85940b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 shutdown_connections 2024-04-25T10:11:51.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.301+0000 7f859aebb640 1 -- 172.21.15.82:0/3298700040 wait complete. 2024-04-25T10:11:51.320 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:51.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 Processor -- start 2024-04-25T10:11:51.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 -- start start 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a940b3cf0 0x7f9a940cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a940cfad0 0x7f9a9415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a940cbf10 con 0x7f9a9415e4f0 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940cff40 con 0x7f9a940b3cf0 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a9a0fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940cc1f0 con 0x7f9a940cfad0 2024-04-25T10:11:51.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a940b3cf0 0x7f9a940cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a92ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a940cfad0 0x7f9a9415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a93fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f9a9415e4f0 0x7f9a94161370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:56018/0 (socket says 172.21.15.82:56018) 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a93fff640 1 -- 172.21.15.82:0/660921394 learned_addr learned my addr 172.21.15.82:0/660921394 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a940b3cf0 0x7f9a940cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:37500/0 (socket says 172.21.15.82:37500) 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 -- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a940cfad0 msgr2=0x7f9a9415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 --2- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a940cfad0 0x7f9a9415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 -- 172.21.15.82:0/660921394 >> v1:172.21.15.196:6789/0 conn(0x7f9a9415e4f0 legacy=0x7f9a94161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.441+0000 7f9a937fe640 1 -- 172.21.15.82:0/660921394 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940cc4d0 con 0x7f9a940b3cf0 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a937fe640 1 --2- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940b3cf0 0x7f9a940cf590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009d90 tx=0x7f9a8c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9df1bb76dad082f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a927fc640 1 -- 172.21.15.82:0/660921394 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0b2070 con 0x7f9a940b3cf0 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a927fc640 1 -- 172.21.15.82:0/660921394 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0a5750 con 0x7f9a940b3cf0 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a927fc640 1 -- 172.21.15.82:0/660921394 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0a5b20 con 0x7f9a940b3cf0 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940b3cf0 msgr2=0x7f9a940cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940b3cf0 0x7f9a940cf590 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c009d90 tx=0x7f9a8c01c970 comp rx=0 tx=0).stop 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/660921394 shutdown_connections 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a940cfad0 0x7f9a9415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/660921394 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940b3cf0 0x7f9a940cf590 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/660921394 >> 172.21.15.82:0/660921394 conn(0x7f9a9406ae60 msgr2=0x7f9a9406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/660921394 shutdown_connections 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/660921394 wait complete. 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 Processor -- start 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.445+0000 7f9a9a0fd640 1 -- start start 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 0x7f9a941538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a9415e4f0 0x7f9a9415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b4250 con 0x7f9a940b3cf0 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940b40d0 con 0x7f9a940cfad0 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9415e900 con 0x7f9a9415e4f0 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55418/0 (socket says 172.21.15.82:55418) 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 -- 172.21.15.82:0/2936770280 learned_addr learned my addr 172.21.15.82:0/2936770280 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a93fff640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a9415e4f0 0x7f9a9415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a92ffd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 0x7f9a941538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 -- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a9415e4f0 msgr2=0x7f9a9415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a9415e4f0 0x7f9a9415c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 -- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 msgr2=0x7f9a941538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 0x7f9a941538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 -- 172.21.15.82:0/2936770280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94152e40 con 0x7f9a940b3cf0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a92ffd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 0x7f9a941538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a937fe640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c01a5a0 tx=0x7f9a8c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0b2050 con 0x7f9a940b3cf0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0be070 con 0x7f9a940b3cf0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a940c7190 con 0x7f9a940b3cf0 2024-04-25T10:11:51.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0ad870 con 0x7f9a940b3cf0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a94098e60 con 0x7f9a940b3cf0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9404efa0 con 0x7f9a940b3cf0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a8c0ada10 con 0x7f9a940b3cf0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 0x7f9a80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f9a8c1201d0 con 0x7f9a940b3cf0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a92ffd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 0x7f9a80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.449+0000 7f9a92ffd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 0x7f9a80069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9a941548e0 tx=0x7f9a88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.453+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a8c016560 con 0x7f9a940b3cf0 2024-04-25T10:11:51.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a940b3060 con 0x7f9a940b3cf0 2024-04-25T10:11:51.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a7bfff640 1 -- 172.21.15.82:0/2936770280 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a8c0f1040 con 0x7f9a940b3cf0 2024-04-25T10:11:51.619 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 msgr2=0x7f9a80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 0x7f9a80069420 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9a941548e0 tx=0x7f9a88008040 comp rx=0 tx=0).stop 2024-04-25T10:11:51.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 msgr2=0x7f9a94098920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c01a5a0 tx=0x7f9a8c019710 comp rx=0 tx=0).stop 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 shutdown_connections 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9a80066f60 0x7f9a80069420 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a9415e4f0 0x7f9a9415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a940cfad0 0x7f9a941538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 --2- 172.21.15.82:0/2936770280 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a940b3cf0 0x7f9a94098920 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 >> 172.21.15.82:0/2936770280 conn(0x7f9a9406ae60 msgr2=0x7f9a940cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 shutdown_connections 2024-04-25T10:11:51.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.617+0000 7f9a9a0fd640 1 -- 172.21.15.82:0/2936770280 wait complete. 2024-04-25T10:11:51.638 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:51.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 Processor -- start 2024-04-25T10:11:51.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 -- start start 2024-04-25T10:11:51.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46a815da10 0x7f46a8071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f46a80b24a0 0x7f46a8071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a80adbd0 con 0x7f46a8071e90 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a80ae860 con 0x7f46a815da10 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46af2e3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a815e210 con 0x7f46a80b24a0 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46a815da10 0x7f46a8071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f46a815da10 0x7f46a8071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55426/0 (socket says 172.21.15.82:55426) 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 -- 172.21.15.82:0/2931367509 learned_addr learned my addr 172.21.15.82:0/2931367509 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ac87e640 1 --2- 172.21.15.82:0/2931367509 >> v2:172.21.15.196:3300/0 conn(0x7f46a80b24a0 0x7f46a8071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 -- 172.21.15.82:0/2931367509 >> v2:172.21.15.196:3300/0 conn(0x7f46a80b24a0 msgr2=0x7f46a8071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 --2- 172.21.15.82:0/2931367509 >> v2:172.21.15.196:3300/0 conn(0x7f46a80b24a0 0x7f46a8071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 -- 172.21.15.82:0/2931367509 >> v1:172.21.15.196:6789/0 conn(0x7f46a8071e90 legacy=0x7f46a81614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:51.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.753+0000 7f46ad07f640 1 -- 172.21.15.82:0/2931367509 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a815e4f0 con 0x7f46a815da10 2024-04-25T10:11:51.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46ad07f640 1 --2- 172.21.15.82:0/2931367509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a8071030 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f46a804f7f0 tx=0x7f469801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdc111b3461d9f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f469ffff640 1 -- 172.21.15.82:0/2931367509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980b0070 con 0x7f46a815da10 2024-04-25T10:11:51.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f469ffff640 1 -- 172.21.15.82:0/2931367509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46980ab600 con 0x7f46a815da10 2024-04-25T10:11:51.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f469ffff640 1 -- 172.21.15.82:0/2931367509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46980a4e60 con 0x7f46a815da10 2024-04-25T10:11:51.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 -- 172.21.15.82:0/2931367509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 msgr2=0x7f46a8071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 --2- 172.21.15.82:0/2931367509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a8071030 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f46a804f7f0 tx=0x7f469801c3a0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 -- 172.21.15.82:0/2931367509 shutdown_connections 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 --2- 172.21.15.82:0/2931367509 >> v2:172.21.15.196:3300/0 conn(0x7f46a80b24a0 0x7f46a8071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 --2- 172.21.15.82:0/2931367509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a8071030 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 -- 172.21.15.82:0/2931367509 >> 172.21.15.82:0/2931367509 conn(0x7f46a80b11a0 msgr2=0x7f46a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 -- 172.21.15.82:0/2931367509 shutdown_connections 2024-04-25T10:11:51.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.757+0000 7f46af2e3640 1 -- 172.21.15.82:0/2931367509 wait complete. 2024-04-25T10:11:51.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 Processor -- start 2024-04-25T10:11:51.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- start start 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46a8071e90 0x7f46a8146020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46a80b24a0 0x7f46a8159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a816a890 con 0x7f46a815da10 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a815ef00 con 0x7f46a80b24a0 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a816aa10 con 0x7f46a8071e90 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.766 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55442/0 (socket says 172.21.15.82:55442) 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 -- 172.21.15.82:0/1189585135 learned_addr learned my addr 172.21.15.82:0/1189585135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad07f640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46a8071e90 0x7f46a8146020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ac87e640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46a80b24a0 0x7f46a8159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 -- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46a8071e90 msgr2=0x7f46a8146020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46a8071e90 0x7f46a8146020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 -- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46a80b24a0 msgr2=0x7f46a8159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46a80b24a0 0x7f46a8159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 -- 172.21.15.82:0/1189585135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a8166960 con 0x7f46a815da10 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46ad880640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b970 tx=0x7f46a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0019070 con 0x7f46a815da10 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a000cb40 con 0x7f46a815da10 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a0014440 con 0x7f46a815da10 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a8166c40 con 0x7f46a815da10 2024-04-25T10:11:51.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a81636f0 con 0x7f46a815da10 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.761+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a804efa0 con 0x7f46a815da10 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.765+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46a001c070 con 0x7f46a815da10 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.765+0000 7f469dffb640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 0x7f468c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.765+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f46a0084e00 con 0x7f46a815da10 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.765+0000 7f46ad07f640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 0x7f468c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:51.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.765+0000 7f46ad07f640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 0x7f468c069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f46980ad000 tx=0x7f4698008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:51.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.769+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f46a0055cf0 con 0x7f46a815da10 2024-04-25T10:11:51.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.925+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f46a80b1dc0 con 0x7f46a815da10 2024-04-25T10:11:51.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.925+0000 7f469dffb640 1 -- 172.21.15.82:0/1189585135 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7f46a00555f0 con 0x7f46a815da10 2024-04-25T10:11:51.930 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 msgr2=0x7f468c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 0x7f468c069420 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f46980ad000 tx=0x7f4698008040 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 msgr2=0x7f46a815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f46a000b970 tx=0x7f46a000be30 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 shutdown_connections 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f468c066f60 0x7f468c069420 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f46a815da10 0x7f46a815a0f0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f46a80b24a0 0x7f46a8159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 --2- 172.21.15.82:0/1189585135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f46a8071e90 0x7f46a8146020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 >> 172.21.15.82:0/1189585135 conn(0x7f46a80b11a0 msgr2=0x7f46a815f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 shutdown_connections 2024-04-25T10:11:51.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:51.929+0000 7f46af2e3640 1 -- 172.21.15.82:0/1189585135 wait complete. 2024-04-25T10:11:51.947 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:11:51.947 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:51.953 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:52.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.097+0000 7fc068438640 1 Processor -- start 2024-04-25T10:11:52.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.097+0000 7fc068438640 1 -- start start 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc068438640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d07f0 0x7fc0600d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc068438640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0600d1cd0 0x7fc06006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc068438640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06006be60 con 0x7fc06015da80 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc068438640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06006c440 con 0x7fc0600d1cd0 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc068438640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06015de60 con 0x7fc0600d07f0 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0600d1cd0 0x7fc06006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc0600d1cd0 0x7fc06006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55448/0 (socket says 172.21.15.82:55448) 2024-04-25T10:11:52.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 -- 172.21.15.82:0/4072524313 learned_addr learned my addr 172.21.15.82:0/4072524313 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0659d3640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d07f0 0x7fc0600d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 -- 172.21.15.82:0/4072524313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d07f0 msgr2=0x7fc0600d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d07f0 0x7fc0600d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 -- 172.21.15.82:0/4072524313 >> v1:172.21.15.196:6789/0 conn(0x7fc06015da80 legacy=0x7fc0600d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 -- 172.21.15.82:0/4072524313 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0600d2450 con 0x7fc0600d1cd0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.101+0000 7fc0669d5640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d1cd0 0x7fc06006c8d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc058009e50 tx=0x7fc05801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9338e78aacf13017 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0651d2640 1 -- 172.21.15.82:0/4072524313 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580a5820 con 0x7fc0600d1cd0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0651d2640 1 -- 172.21.15.82:0/4072524313 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0580a59c0 con 0x7fc0600d1cd0 2024-04-25T10:11:52.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0651d2640 1 -- 172.21.15.82:0/4072524313 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580a5d90 con 0x7fc0600d1cd0 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/4072524313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d1cd0 msgr2=0x7fc06006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d1cd0 0x7fc06006c8d0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fc058009e50 tx=0x7fc05801c9c0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/4072524313 shutdown_connections 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d07f0 0x7fc0600d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- 172.21.15.82:0/4072524313 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d1cd0 0x7fc06006c8d0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/4072524313 >> 172.21.15.82:0/4072524313 conn(0x7fc06008b000 msgr2=0x7fc0600d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/4072524313 shutdown_connections 2024-04-25T10:11:52.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/4072524313 wait complete. 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 Processor -- start 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- start start 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d1cd0 0x7fc060155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06015da80 0x7fc060155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d7a0 con 0x7fc0600d07f0 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d620 con 0x7fc06015da80 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06006d920 con 0x7fc0600d1cd0 2024-04-25T10:11:52.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55460/0 (socket says 172.21.15.82:55460) 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 -- 172.21.15.82:0/2872026418 learned_addr learned my addr 172.21.15.82:0/2872026418 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0659d3640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d1cd0 0x7fc060155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0669d5640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06015da80 0x7fc060155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 -- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d1cd0 msgr2=0x7fc060155700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d1cd0 0x7fc060155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 -- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06015da80 msgr2=0x7fc060155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06015da80 0x7fc060155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 -- 172.21.15.82:0/2872026418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060160080 con 0x7fc0600d07f0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc0661d4640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b410 tx=0x7fc05c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c015030 con 0x7fc0600d07f0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05c004020 con 0x7fc0600d07f0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc060160360 con 0x7fc0600d07f0 2024-04-25T10:11:52.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05c004450 con 0x7fc0600d07f0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.105+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc060160920 con 0x7fc0600d07f0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06004efa0 con 0x7fc0600d07f0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc05c018070 con 0x7fc0600d07f0 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc056ffd640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 0x7fc044069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc05c084d80 con 0x7fc0600d07f0 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc0659d3640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 0x7fc044069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.109+0000 7fc0659d3640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 0x7fc044069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc0600b2bf0 tx=0x7fc050008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.113+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc05c055be0 con 0x7fc0600d07f0 2024-04-25T10:11:52.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.273+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0600cfe60 con 0x7fc0600d07f0 2024-04-25T10:11:52.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.277+0000 7fc056ffd640 1 -- 172.21.15.82:0/2872026418 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16946 (secure 0 0 0) 0x7fc05c0554e0 con 0x7fc0600d07f0 2024-04-25T10:11:52.280 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:11:52.280 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":109,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:11:51.533226+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.277+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 msgr2=0x7fc044069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.277+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 0x7fc044069450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc0600b2bf0 tx=0x7fc050008040 comp rx=0 tx=0).stop 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.277+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 msgr2=0x7fc0600b6570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.277+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fc05c00b410 tx=0x7fc05c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 shutdown_connections 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc044066f90 0x7fc044069450 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc0600d1cd0 0x7fc060155700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc06015da80 0x7fc060155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 --2- 172.21.15.82:0/2872026418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc0600d07f0 0x7fc0600b6570 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 >> 172.21.15.82:0/2872026418 conn(0x7fc06008b000 msgr2=0x7fc0600cf120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 shutdown_connections 2024-04-25T10:11:52.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.281+0000 7fc068438640 1 -- 172.21.15.82:0/2872026418 wait complete. 2024-04-25T10:11:52.298 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:27.097350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:27.504961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:52.299 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:52.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 Processor -- start 2024-04-25T10:11:52.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 -- start start 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 0x7f2a440ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a440ca4c0 con 0x7f2a440cf970 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a440ca780 con 0x7f2a440ce170 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a4a916640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440cd440 con 0x7f2a440cf590 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a48eb3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 0x7f2a440ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:47528/0 (socket says 172.21.15.82:47528) 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 -- 172.21.15.82:0/3636097739 learned_addr learned my addr 172.21.15.82:0/3636097739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:52.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a3bfff640 1 --1- 172.21.15.82:0/3636097739 >> v1:172.21.15.196:6789/0 conn(0x7f2a440cf970 0x7f2a440c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:56050/0 (socket says 172.21.15.82:56050) 2024-04-25T10:11:52.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 -- 172.21.15.82:0/3636097739 >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 msgr2=0x7f2a440ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 --2- 172.21.15.82:0/3636097739 >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 0x7f2a440ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 -- 172.21.15.82:0/3636097739 >> v1:172.21.15.196:6789/0 conn(0x7f2a440cf970 legacy=0x7f2a440c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a43fff640 1 -- 172.21.15.82:0/3636097739 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a440cd690 con 0x7f2a440cf590 2024-04-25T10:11:52.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.421+0000 7f2a48eb3640 1 --2- 172.21.15.82:0/3636097739 >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 0x7f2a440ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:11:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a43fff640 1 --2- 172.21.15.82:0/3636097739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009870 tx=0x7f2a3401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=528dff6a86c0af90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a437fe640 1 -- 172.21.15.82:0/3636097739 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340a7070 con 0x7f2a440cf590 2024-04-25T10:11:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a437fe640 1 -- 172.21.15.82:0/3636097739 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a340b0070 con 0x7f2a440cf590 2024-04-25T10:11:52.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a437fe640 1 -- 172.21.15.82:0/3636097739 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a340ab890 con 0x7f2a440cf590 2024-04-25T10:11:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- 172.21.15.82:0/3636097739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 msgr2=0x7f2a4415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 --2- 172.21.15.82:0/3636097739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009870 tx=0x7f2a3401c560 comp rx=0 tx=0).stop 2024-04-25T10:11:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- 172.21.15.82:0/3636097739 shutdown_connections 2024-04-25T10:11:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 --2- 172.21.15.82:0/3636097739 >> v2:172.21.15.196:3300/0 conn(0x7f2a440ce170 0x7f2a440ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 --2- 172.21.15.82:0/3636097739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440cf590 0x7f2a4415ecb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- 172.21.15.82:0/3636097739 >> 172.21.15.82:0/3636097739 conn(0x7f2a4406ae60 msgr2=0x7f2a4406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- 172.21.15.82:0/3636097739 shutdown_connections 2024-04-25T10:11:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- 172.21.15.82:0/3636097739 wait complete. 2024-04-25T10:11:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 Processor -- start 2024-04-25T10:11:52.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.425+0000 7f2a4a916640 1 -- start start 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440ce170 0x7f2a44097e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a440cf590 0x7f2a4415c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440cfed0 con 0x7f2a440cf970 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440cfd50 con 0x7f2a440cf590 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a440ac170 con 0x7f2a440ce170 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55470/0 (socket says 172.21.15.82:55470) 2024-04-25T10:11:52.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 -- 172.21.15.82:0/2739431077 learned_addr learned my addr 172.21.15.82:0/2739431077 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a43fff640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440ce170 0x7f2a44097e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a3bfff640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a440cf590 0x7f2a4415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 -- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440ce170 msgr2=0x7f2a44097e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440ce170 0x7f2a44097e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 -- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a440cf590 msgr2=0x7f2a4415c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a440cf590 0x7f2a4415c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 -- 172.21.15.82:0/2739431077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4415b780 con 0x7f2a440cf970 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a48eb3640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e60 tx=0x7f2a3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c018030 con 0x7f2a440cf970 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c00cdf0 con 0x7f2a440cf970 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4415ba60 con 0x7f2a440cf970 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c003bc0 con 0x7f2a440cf970 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4415c020 con 0x7f2a440cf970 2024-04-25T10:11:52.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4404efa0 con 0x7f2a440cf970 2024-04-25T10:11:52.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a3c01b070 con 0x7f2a440cf970 2024-04-25T10:11:52.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2a3c085ba0 con 0x7f2a440cf970 2024-04-25T10:11:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.429+0000 7f2a43fff640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.433+0000 7f2a43fff640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 0x7f2a24069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a340af000 tx=0x7f2a340a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.438 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.437+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a3c010040 con 0x7f2a440cf970 2024-04-25T10:11:52.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.581+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a4415c1f0 con 0x7f2a440cf970 2024-04-25T10:11:52.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.581+0000 7f2a417fa640 1 -- 172.21.15.82:0/2739431077 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a3c056a00 con 0x7f2a440cf970 2024-04-25T10:11:52.586 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 msgr2=0x7f2a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 0x7f2a24069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2a340af000 tx=0x7f2a340a7000 comp rx=0 tx=0).stop 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 msgr2=0x7f2a440cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c009e60 tx=0x7f2a3c00c490 comp rx=0 tx=0).stop 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 shutdown_connections 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a24066f60 0x7f2a24069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a440cf970 0x7f2a440cbde0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a440cf590 0x7f2a4415c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 --2- 172.21.15.82:0/2739431077 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a440ce170 0x7f2a44097e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 >> 172.21.15.82:0/2739431077 conn(0x7f2a4406ae60 msgr2=0x7f2a440726b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 shutdown_connections 2024-04-25T10:11:52.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.585+0000 7f2a4a916640 1 -- 172.21.15.82:0/2739431077 wait complete. 2024-04-25T10:11:52.603 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 Processor -- start 2024-04-25T10:11:52.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 -- start start 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb2e40cf590 0x7fb2e415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb2e40ce170 0x7fb2e40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e40ca4c0 con 0x7fb2e40cf970 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40ca780 con 0x7fb2e40cf590 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2ea677640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40cd440 con 0x7fb2e40ce170 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2db7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb2e40cf970 0x7fb2e40c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53626/0 (socket says 172.21.15.82:53626) 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2db7fe640 1 -- 172.21.15.82:0/3221617988 learned_addr learned my addr 172.21.15.82:0/3221617988 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:52.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/3221617988 >> v2:172.21.15.82:3300/0 conn(0x7fb2e40cf590 0x7fb2e415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e8c14640 1 --2- 172.21.15.82:0/3221617988 >> v2:172.21.15.196:3300/0 conn(0x7fb2e40ce170 0x7fb2e40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e37fe640 1 -- 172.21.15.82:0/3221617988 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1234442905 0 0) 0x7fb2e40ca4c0 con 0x7fb2e40cf970 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e37fe640 1 -- 172.21.15.82:0/3221617988 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2d0003530 con 0x7fb2e40cf970 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e3fff640 1 -- 172.21.15.82:0/3221617988 >> v2:172.21.15.196:3300/0 conn(0x7fb2e40ce170 msgr2=0x7fb2e40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/3221617988 >> v2:172.21.15.196:3300/0 conn(0x7fb2e40ce170 0x7fb2e40ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e3fff640 1 -- 172.21.15.82:0/3221617988 >> v1:172.21.15.82:6789/0 conn(0x7fb2e40cf970 legacy=0x7fb2e40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.721+0000 7fb2e3fff640 1 -- 172.21.15.82:0/3221617988 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e40cd690 con 0x7fb2e40cf590 2024-04-25T10:11:52.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/3221617988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e415ecb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40c9940 tx=0x7fb2d401e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=555116f42830dca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e37fe640 1 -- 172.21.15.82:0/3221617988 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40b4070 con 0x7fb2e40cf590 2024-04-25T10:11:52.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e37fe640 1 -- 172.21.15.82:0/3221617988 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d40a7790 con 0x7fb2e40cf590 2024-04-25T10:11:52.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e37fe640 1 -- 172.21.15.82:0/3221617988 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d40a7b60 con 0x7fb2e40cf590 2024-04-25T10:11:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/3221617988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 msgr2=0x7fb2e415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- 172.21.15.82:0/3221617988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e415ecb0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb2e40c9940 tx=0x7fb2d401e8b0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/3221617988 shutdown_connections 2024-04-25T10:11:52.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- 172.21.15.82:0/3221617988 >> v2:172.21.15.196:3300/0 conn(0x7fb2e40ce170 0x7fb2e40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- 172.21.15.82:0/3221617988 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e415ecb0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/3221617988 >> 172.21.15.82:0/3221617988 conn(0x7fb2e406ae60 msgr2=0x7fb2e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/3221617988 shutdown_connections 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/3221617988 wait complete. 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 Processor -- start 2024-04-25T10:11:52.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- start start 2024-04-25T10:11:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2e40ce170 0x7fb2e40c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2e40cf970 0x7fb2e41636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40cfed0 con 0x7fb2e40cf590 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40cfd50 con 0x7fb2e40cf970 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e40ac170 con 0x7fb2e40ce170 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55496/0 (socket says 172.21.15.82:55496) 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 -- 172.21.15.82:0/1695724820 learned_addr learned my addr 172.21.15.82:0/1695724820 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2e40ce170 0x7fb2e40c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e8c14640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2e40cf970 0x7fb2e41636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 -- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2e40ce170 msgr2=0x7fb2e40c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2e40ce170 0x7fb2e40c3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 -- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2e40cf970 msgr2=0x7fb2e41636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2e40cf970 0x7fb2e41636a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 -- 172.21.15.82:0/1695724820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4163fa0 con 0x7fb2e40cf590 2024-04-25T10:11:52.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2db7fe640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c450 tx=0x7fb2cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc017070 con 0x7fb2e40cf590 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc005cf0 con 0x7fb2e40cf590 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4164280 con 0x7fb2e40cf590 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc007290 con 0x7fb2e40cf590 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.725+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e4152b30 con 0x7fb2e40cf590 2024-04-25T10:11:52.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e404efa0 con 0x7fb2e40cf590 2024-04-25T10:11:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2cc01a070 con 0x7fb2e40cf590 2024-04-25T10:11:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2e17fa640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 0x7fb2c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb2cc084820 con 0x7fb2e40cf590 2024-04-25T10:11:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 0x7fb2c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:52.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.729+0000 7fb2e3fff640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 0x7fb2c4069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b1000 tx=0x7fb2d40a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:52.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.733+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2cc021080 con 0x7fb2e40cf590 2024-04-25T10:11:52.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.881+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2e40ca4c0 con 0x7fb2e40cf590 2024-04-25T10:11:52.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2e17fa640 1 -- 172.21.15.82:0/1695724820 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v109) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb2cc055760 con 0x7fb2e40cf590 2024-04-25T10:11:52.886 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:11:52.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 msgr2=0x7fb2c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 0x7fb2c4069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40b1000 tx=0x7fb2d40a9070 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 msgr2=0x7fb2e41525f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c450 tx=0x7fb2cc00c910 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 shutdown_connections 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb2c4066f60 0x7fb2c4069420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb2e40cf970 0x7fb2e41636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb2e40cf590 0x7fb2e41525f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 --2- 172.21.15.82:0/1695724820 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb2e40ce170 0x7fb2e40c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 >> 172.21.15.82:0/1695724820 conn(0x7fb2e406ae60 msgr2=0x7fb2e40b23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 shutdown_connections 2024-04-25T10:11:52.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:52.885+0000 7fb2ea677640 1 -- 172.21.15.82:0/1695724820 wait complete. 2024-04-25T10:11:52.904 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:53.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.013+0000 7f2e29d20640 1 Processor -- start 2024-04-25T10:11:53.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.013+0000 7f2e29d20640 1 -- start start 2024-04-25T10:11:53.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e29d20640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2e2415da80 0x7f2e240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e29d20640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2406be60 con 0x7f2e240d1cd0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e29d20640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2415de60 con 0x7f2e240d07f0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e29d20640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2406c440 con 0x7f2e2415da80 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e22ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2e240d1cd0 0x7f2e240d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53636/0 (socket says 172.21.15.82:53636) 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e22ffd640 1 -- 172.21.15.82:0/1988818934 learned_addr learned my addr 172.21.15.82:0/1988818934 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e237fe640 1 --2- 172.21.15.82:0/1988818934 >> v2:172.21.15.196:3300/0 conn(0x7f2e2415da80 0x7f2e240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3004436438 0 0) 0x7f2e2406be60 con 0x7f2e240d1cd0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003530 con 0x7f2e240d1cd0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918421373 0 0) 0x7f2e0c003530 con 0x7f2e240d1cd0 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 >> v2:172.21.15.196:3300/0 conn(0x7f2e2415da80 msgr2=0x7f2e240d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 --2- 172.21.15.82:0/1988818934 >> v2:172.21.15.196:3300/0 conn(0x7f2e2415da80 0x7f2e240d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 >> v1:172.21.15.196:6789/0 conn(0x7f2e240d07f0 legacy=0x7f2e2406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240d2450 con 0x7f2e240d1cd0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2e18001800 con 0x7f2e240d1cd0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e237fe640 1 --2- 172.21.15.82:0/1988818934 >> v2:172.21.15.196:3300/0 conn(0x7f2e2415da80 0x7f2e240d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 >> v1:172.21.15.82:6789/0 conn(0x7f2e240d1cd0 legacy=0x7f2e240d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c003a30 0x7f2e0c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e0c004810 0x7f2e0c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 0x7f2e0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2406c440 con 0x7f2e0c0052f0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c00c1c0 con 0x7f2e0c003a30 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c00c3c0 con 0x7f2e0c004810 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 0x7f2e0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e237fe640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c003a30 0x7f2e0c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e22ffd640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e0c004810 0x7f2e0c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 -- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e0c004810 msgr2=0x7f2e0c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e0c004810 0x7f2e0c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 -- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c003a30 msgr2=0x7f2e0c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c003a30 0x7f2e0c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.017+0000 7f2e23fff640 1 -- 172.21.15.82:0/1988818934 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c003530 con 0x7f2e0c0052f0 2024-04-25T10:11:53.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e23fff640 1 -- 172.21.15.82:0/1988818934 reap_dead start 2024-04-25T10:11:53.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e23fff640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 0x7f2e0c00b9d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c001750 tx=0x7f2e1c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe40b1169969084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0013a0 con 0x7f2e0c0052f0 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c0052d0 con 0x7f2e0c0052f0 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e227fc640 1 -- 172.21.15.82:0/1988818934 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c003840 con 0x7f2e0c0052f0 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 msgr2=0x7f2e0c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 0x7f2e0c00b9d0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c001750 tx=0x7f2e1c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/1988818934 shutdown_connections 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 --2- 172.21.15.82:0/1988818934 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e0c0052f0 0x7f2e0c00b9d0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/1988818934 >> 172.21.15.82:0/1988818934 conn(0x7f2e2408b000 msgr2=0x7f2e240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/1988818934 shutdown_connections 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/1988818934 wait complete. 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 Processor -- start 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- start start 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c0052f0 0x7f2e240b66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e240b8180 0x7f2e240b8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240d22a0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240d2120 con 0x7f2e0c0052f0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240d26f0 con 0x7f2e240b8180 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55514/0 (socket says 172.21.15.82:55514) 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 -- 172.21.15.82:0/3987016642 learned_addr learned my addr 172.21.15.82:0/3987016642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e237fe640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c0052f0 0x7f2e240b66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e23fff640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e240b8180 0x7f2e240b8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 -- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e240b8180 msgr2=0x7f2e240b8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e240b8180 0x7f2e240b8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 -- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c0052f0 msgr2=0x7f2e240b66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c0052f0 0x7f2e240b66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 -- 172.21.15.82:0/3987016642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e24145a50 con 0x7f2e240ccbf0 2024-04-25T10:11:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e22ffd640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e800 tx=0x7f2e1800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e180070d0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e18018070 con 0x7f2e240ccbf0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240b8f90 con 0x7f2e240ccbf0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e180074d0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.021+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e24146000 con 0x7f2e240ccbf0 2024-04-25T10:11:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2404efa0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e180057b0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e0bfff640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 0x7f2e10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e237fe640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 0x7f2e10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2e18084ee0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.025+0000 7f2e237fe640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 0x7f2e10069450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009880 tx=0x7f2e1400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.029+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e18055dd0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.181+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e240cdcb0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.181+0000 7f2e0bfff640 1 -- 172.21.15.82:0/3987016642 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v109) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2e180556d0 con 0x7f2e240ccbf0 2024-04-25T10:11:53.185 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 msgr2=0x7f2e10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 0x7f2e10069450 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009880 tx=0x7f2e1400b040 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 msgr2=0x7f2e240cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e800 tx=0x7f2e1800ecc0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 shutdown_connections 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2e10066f90 0x7f2e10069450 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2e240b8180 0x7f2e240b8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2e240ccbf0 0x7f2e240cd050 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 --2- 172.21.15.82:0/3987016642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2e0c0052f0 0x7f2e240b66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 >> 172.21.15.82:0/3987016642 conn(0x7f2e2408b000 msgr2=0x7f2e240cf1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 shutdown_connections 2024-04-25T10:11:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.185+0000 7f2e29d20640 1 -- 172.21.15.82:0/3987016642 wait complete. 2024-04-25T10:11:53.203 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:53.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 Processor -- start 2024-04-25T10:11:53.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 -- start start 2024-04-25T10:11:53.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc5815aac0 0x7fdc580ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc580ce320 con 0x7fdc580cf0e0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580cd5f0 con 0x7fdc5815aac0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ecb3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580ce0d0 con 0x7fdc580cd180 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ca4f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fdc5815aac0 0x7fdc580ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc57fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fdc580cf0e0 0x7fdc5815aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:53644/0 (socket says 172.21.15.82:53644) 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc57fff640 1 -- 172.21.15.82:0/3690820430 learned_addr learned my addr 172.21.15.82:0/3690820430 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5d250640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc577fe640 1 -- 172.21.15.82:0/3690820430 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4292694147 0 0) 0x7fdc580ce320 con 0x7fdc580cf0e0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc577fe640 1 -- 172.21.15.82:0/3690820430 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc40003530 con 0x7fdc580cf0e0 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ca4f640 1 -- 172.21.15.82:0/3690820430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 msgr2=0x7fdc5815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5815b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ca4f640 1 -- 172.21.15.82:0/3690820430 >> v1:172.21.15.82:6789/0 conn(0x7fdc580cf0e0 legacy=0x7fdc5815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.321+0000 7fdc5ca4f640 1 -- 172.21.15.82:0/3690820430 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580ce600 con 0x7fdc5815aac0 2024-04-25T10:11:53.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5815aac0 0x7fdc580ccc40 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097c0 tx=0x7fdc4801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca79c35d562eb1b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc577fe640 1 -- 172.21.15.82:0/3690820430 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480b2070 con 0x7fdc5815aac0 2024-04-25T10:11:53.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc577fe640 1 -- 172.21.15.82:0/3690820430 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480ad640 con 0x7fdc5815aac0 2024-04-25T10:11:53.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc577fe640 1 -- 172.21.15.82:0/3690820430 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480ada10 con 0x7fdc5815aac0 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/3690820430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5815aac0 msgr2=0x7fdc580ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5815aac0 0x7fdc580ccc40 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fdc480097c0 tx=0x7fdc4801c4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/3690820430 shutdown_connections 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/3690820430 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc5815aac0 0x7fdc580ccc40 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/3690820430 >> 172.21.15.82:0/3690820430 conn(0x7fdc58099a70 msgr2=0x7fdc580d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/3690820430 shutdown_connections 2024-04-25T10:11:53.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/3690820430 wait complete. 2024-04-25T10:11:53.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 Processor -- start 2024-04-25T10:11:53.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.325+0000 7fdc5ecb3640 1 -- start start 2024-04-25T10:11:53.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc5815aac0 0x7fdc580c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580cb9a0 con 0x7fdc580cf0e0 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580cb820 con 0x7fdc5815aac0 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580cbb20 con 0x7fdc580cd180 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55526/0 (socket says 172.21.15.82:55526) 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 -- 172.21.15.82:0/1948508807 learned_addr learned my addr 172.21.15.82:0/1948508807 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5d250640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc5815aac0 0x7fdc580c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 -- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 msgr2=0x7fdc5814eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5814eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 -- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc5815aac0 msgr2=0x7fdc580c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc5815aac0 0x7fdc580c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 -- 172.21.15.82:0/1948508807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5815c260 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc57fff640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000cb70 tx=0x7fdc5000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50017070 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5000ad40 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc580c8ba0 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50012620 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5815c500 con 0x7fdc580cf0e0 2024-04-25T10:11:53.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5804efa0 con 0x7fdc580cf0e0 2024-04-25T10:11:53.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdc5001a070 con 0x7fdc580cf0e0 2024-04-25T10:11:53.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:53.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:53.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fdc50084f90 con 0x7fdc580cf0e0 2024-04-25T10:11:53.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.329+0000 7fdc5ca4f640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 0x7fdc44069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdc480af000 tx=0x7fdc48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:53.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.333+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdc50055e80 con 0x7fdc580cf0e0 2024-04-25T10:11:53.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc580ce3e0 con 0x7fdc580cf0e0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc557fa640 1 -- 172.21.15.82:0/1948508807 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v109) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc50055780 con 0x7fdc580cf0e0 2024-04-25T10:11:53.508 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 msgr2=0x7fdc44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 0x7fdc44069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fdc480af000 tx=0x7fdc48008040 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 msgr2=0x7fdc580c7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000cb70 tx=0x7fdc5000a430 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 shutdown_connections 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdc44066f60 0x7fdc44069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdc5815aac0 0x7fdc580c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdc580cf0e0 0x7fdc580c7fe0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 --2- 172.21.15.82:0/1948508807 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdc580cd180 0x7fdc5814eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:53.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 >> 172.21.15.82:0/1948508807 conn(0x7fdc58099a70 msgr2=0x7fdc58157640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:53.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 shutdown_connections 2024-04-25T10:11:53.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:53.505+0000 7fdc5ecb3640 1 -- 172.21.15.82:0/1948508807 wait complete. 2024-04-25T10:11:53.526 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:53.535 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[52328]: fuse finished with error 0 and tester_r 0 2024-04-25T10:11:53.537 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:59.639 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:11:59.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:59.639 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:59.647 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.816+0000 7f8d8b169640 1 Processor -- start 2024-04-25T10:11:59.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.816+0000 7f8d8b169640 1 -- start start 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.816+0000 7f8d8b169640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8d840b24a0 0x7f8d840ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.816+0000 7f8d8b169640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840ae860 con 0x7f8d8415da10 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d8b169640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8415e210 con 0x7f8d84071740 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d8b169640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8415e4f0 con 0x7f8d840b24a0 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d88f05640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8d8415da10 0x7f8d84071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56178/0 (socket says 172.21.15.82:56178) 2024-04-25T10:11:59.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d88f05640 1 -- 172.21.15.82:0/454122333 learned_addr learned my addr 172.21.15.82:0/454122333 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 --2- 172.21.15.82:0/454122333 >> v2:172.21.15.196:3300/0 conn(0x7f8d840b24a0 0x7f8d840ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741260299 0 0) 0x7f8d840ae860 con 0x7f8d8415da10 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d6c003530 con 0x7f8d8415da10 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 592237237 0 0) 0x7f8d6c003530 con 0x7f8d8415da10 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 >> v2:172.21.15.196:3300/0 conn(0x7f8d840b24a0 msgr2=0x7f8d840ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 --2- 172.21.15.82:0/454122333 >> v2:172.21.15.196:3300/0 conn(0x7f8d840b24a0 0x7f8d840ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 >> v1:172.21.15.196:6789/0 conn(0x7f8d84071740 legacy=0x7f8d840acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840ae4c0 con 0x7f8d8415da10 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8d74001680 con 0x7f8d8415da10 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 >> v1:172.21.15.82:6789/0 conn(0x7f8d8415da10 legacy=0x7f8d84071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c003a30 0x7f8d6c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d6c004810 0x7f8d6c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d6c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8415e4f0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c210 con 0x7f8d6c003a30 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c00c410 con 0x7f8d6c004810 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d6c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d88f05640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c003a30 0x7f8d6c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d83fff640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d6c004810 0x7f8d6c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 -- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d6c004810 msgr2=0x7f8d6c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d6c004810 0x7f8d6c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 -- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c003a30 msgr2=0x7f8d6c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c003a30 0x7f8d6c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 -- 172.21.15.82:0/454122333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c003530 con 0x7f8d6c0052d0 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d88f05640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d6c003a30 0x7f8d6c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:59.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 -- 172.21.15.82:0/454122333 reap_dead start 2024-04-25T10:11:59.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d89706640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d6c00b9d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009fc0 tx=0x7f8d7c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16854dbeb9882e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c009c10 con 0x7f8d6c0052d0 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c003be0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.820+0000 7f8d837fe640 1 -- 172.21.15.82:0/454122333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c0a58b0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 msgr2=0x7f8d6c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d6c00b9d0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009fc0 tx=0x7f8d7c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/454122333 shutdown_connections 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 --2- 172.21.15.82:0/454122333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d6c00b9d0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/454122333 >> 172.21.15.82:0/454122333 conn(0x7f8d840b11a0 msgr2=0x7f8d840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/454122333 shutdown_connections 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/454122333 wait complete. 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 Processor -- start 2024-04-25T10:11:59.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- start start 2024-04-25T10:11:59.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d84157690 0x7f8d84151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d84152120 0x7f8d8415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840713c0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84071240 con 0x7f8d84152120 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84071540 con 0x7f8d84157690 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d89706640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d84152120 0x7f8d8415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32920/0 (socket says 172.21.15.82:32920) 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d83fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d84157690 0x7f8d84151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 -- 172.21.15.82:0/448380072 learned_addr learned my addr 172.21.15.82:0/448380072 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 -- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d84157690 msgr2=0x7f8d84151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d84157690 0x7f8d84151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 -- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d84152120 msgr2=0x7f8d8415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d84152120 0x7f8d8415b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 -- 172.21.15.82:0/448380072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d84158f80 con 0x7f8d6c0052d0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d88f05640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001210 tx=0x7f8d7400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400c660 con 0x7f8d6c0052d0 2024-04-25T10:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74018070 con 0x7f8d6c0052d0 2024-04-25T10:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8406d500 con 0x7f8d6c0052d0 2024-04-25T10:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7400ca00 con 0x7f8d6c0052d0 2024-04-25T10:11:59.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.824+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8406dac0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d7400cba0 con 0x7f8d6c0052d0 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d84152530 con 0x7f8d6c0052d0 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d817fa640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 0x7f8d70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d83fff640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 0x7f8d70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f8d74084c30 con 0x7f8d6c0052d0 2024-04-25T10:11:59.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.828+0000 7f8d83fff640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 0x7f8d70069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c10 tx=0x7f8d78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:59.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:11:59.832+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d84152530 con 0x7f8d6c0052d0 2024-04-25T10:12:00.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.016+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d84159150 con 0x7f8d6c0052d0 2024-04-25T10:12:00.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d817fa640 1 -- 172.21.15.82:0/448380072 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8d740554d0 con 0x7f8d6c0052d0 2024-04-25T10:12:00.023 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:00.023 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":75,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:11:30.539889+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8769},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8769":{"gid":8769,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/3162787996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":3162787996},{"type":"v1","addr":"172.21.15.82:6837","nonce":3162787996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:00.023 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 75 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 msgr2=0x7f8d70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 0x7f8d70069450 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8d78000c10 tx=0x7f8d78009040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 msgr2=0x7f8d841472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f8d74001210 tx=0x7f8d7400ec10 comp rx=0 tx=0).stop 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 shutdown_connections 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8d70066f90 0x7f8d70069450 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8d84157690 0x7f8d84151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8d84152120 0x7f8d8415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 --2- 172.21.15.82:0/448380072 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8d6c0052d0 0x7f8d841472c0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 >> 172.21.15.82:0/448380072 conn(0x7f8d840b11a0 msgr2=0x7f8d84160120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 shutdown_connections 2024-04-25T10:12:00.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.020+0000 7f8d8b169640 1 -- 172.21.15.82:0/448380072 wait complete. 2024-04-25T10:12:00.041 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:12:00.182 INFO:teuthology.orchestra.run.smithi082.stdout:{"message":"","return_code":0} 2024-04-25T10:12:00.182 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:12:00.182 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:12:00.182 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:00.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:00.188 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi082.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:00.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:12:00.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:00.189 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:00.234 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:12:00.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:12:00.235 DEBUG:teuthology.orchestra.run.smithi082:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:12:00.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:00.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:12:00.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 Processor -- start 2024-04-25T10:12:00.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 -- start start 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcbf00cdee0 0x7fcbf006b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf015e3a0 con 0x7fcbf015de40 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf015e680 con 0x7fcbf0157ec0 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf85a7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf015e960 con 0x7fcbf00cdee0 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6b44640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcbf015de40 0x7fcbf006b590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:56194/0 (socket says 172.21.15.82:56194) 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6b44640 1 -- 172.21.15.82:0/3727845615 learned_addr learned my addr 172.21.15.82:0/3727845615 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/3727845615 >> v2:172.21.15.196:3300/0 conn(0x7fcbf00cdee0 0x7fcbf006b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547012937 0 0) 0x7fcbf015e3a0 con 0x7fcbf015de40 2024-04-25T10:12:00.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbdc003530 con 0x7fcbf015de40 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258750794 0 0) 0x7fcbdc003530 con 0x7fcbf015de40 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 >> v2:172.21.15.196:3300/0 conn(0x7fcbf00cdee0 msgr2=0x7fcbf006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 --2- 172.21.15.82:0/3727845615 >> v2:172.21.15.196:3300/0 conn(0x7fcbf00cdee0 0x7fcbf006b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 >> v1:172.21.15.196:6789/0 conn(0x7fcbf0157ec0 legacy=0x7fcbf00cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf015ec40 con 0x7fcbf015de40 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcbec001680 con 0x7fcbf015de40 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 >> v1:172.21.15.82:6789/0 conn(0x7fcbf015de40 legacy=0x7fcbf006b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbdc0039c0 0x7fcbdc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc0051d0 0x7fcbdc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015e960 con 0x7fcbdc004810 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00c070 con 0x7fcbdc0039c0 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbdc00c270 con 0x7fcbdc0051d0 2024-04-25T10:12:00.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6343640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbdc0039c0 0x7fcbdc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 -- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc0051d0 msgr2=0x7fcbdc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc0051d0 0x7fcbdc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc0051d0 0x7fcbdc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 -- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbdc0039c0 msgr2=0x7fcbdc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbdc0039c0 0x7fcbdc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf5b42640 1 -- 172.21.15.82:0/3727845615 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc003530 con 0x7fcbdc004810 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6343640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbdc0039c0 0x7fcbdc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc0051d0 0x7fcbdc00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.400+0000 7fcbf6b44640 1 -- 172.21.15.82:0/3727845615 reap_dead start 2024-04-25T10:12:00.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c90 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009ab0 tx=0x7fcbe001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834f6e739a84050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00b2070 con 0x7fcbdc004810 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00a5640 con 0x7fcbdc004810 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf5341640 1 -- 172.21.15.82:0/3727845615 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00096b0 con 0x7fcbdc004810 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 -- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 msgr2=0x7fcbdc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c90 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009ab0 tx=0x7fcbe001ca50 comp rx=0 tx=0).stop 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 -- 172.21.15.82:0/3727845615 shutdown_connections 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/3727845615 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbdc004810 0x7fcbdc004c90 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 -- 172.21.15.82:0/3727845615 >> 172.21.15.82:0/3727845615 conn(0x7fcbf00cc680 msgr2=0x7fcbf00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 -- 172.21.15.82:0/3727845615 shutdown_connections 2024-04-25T10:12:00.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.404+0000 7fcbf85a7640 1 -- 172.21.15.82:0/3727845615 wait complete. 2024-04-25T10:12:00.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 Processor -- start 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- start start 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc004810 0x7fcbf01f7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 0x7fcbf0154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0066f60 con 0x7fcbf01f3960 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0066de0 con 0x7fcbf00b4b40 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0161490 con 0x7fcbdc004810 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32930/0 (socket says 172.21.15.82:32930) 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 -- 172.21.15.82:0/157299567 learned_addr learned my addr 172.21.15.82:0/157299567 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6343640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc004810 0x7fcbf01f7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 0x7fcbf0154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 -- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc004810 msgr2=0x7fcbf01f7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc004810 0x7fcbf01f7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 -- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 msgr2=0x7fcbf0154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 0x7fcbf0154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 -- 172.21.15.82:0/157299567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0154850 con 0x7fcbf01f3960 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 0x7fcbf0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0071430 tx=0x7fcbec010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbe6ffd640 1 -- 172.21.15.82:0/157299567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec01a070 con 0x7fcbf01f3960 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbe6ffd640 1 -- 172.21.15.82:0/157299567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec011040 con 0x7fcbf01f3960 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbe6ffd640 1 -- 172.21.15.82:0/157299567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00ecc0 con 0x7fcbf01f3960 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf01f43b0 con 0x7fcbf01f3960 2024-04-25T10:12:00.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.408+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf01f7180 con 0x7fcbf01f3960 2024-04-25T10:12:00.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbe6ffd640 1 -- 172.21.15.82:0/157299567 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbec01d070 con 0x7fcbf01f3960 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbe6ffd640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbe6ffd640 1 -- 172.21.15.82:0/157299567 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcbec0877f0 con 0x7fcbf01f3960 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 Processor -- start 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 0x7fcbd4069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8007fd0 tx=0x7fcbe8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- start start 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a181775d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a181776290 0x55a181776670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18165cf40 con 0x55a1817bfc90 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1815c4070 con 0x55a1817c05b0 2024-04-25T10:12:00.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a18154c1a0 con 0x55a181776290 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32944/0 (socket says 172.21.15.82:32944) 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 -- 172.21.15.82:0/346010775 learned_addr learned my addr 172.21.15.82:0/346010775 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 -- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a181776290 msgr2=0x55a181776670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a181776290 0x55a181776670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 -- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 msgr2=0x55a181775d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a181775d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 -- 172.21.15.82:0/346010775 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a18154a510 con 0x55a1817bfc90 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbf6343640 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80016d0 tx=0x7fcbe8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ab854972c9ceb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbd3fff640 1 -- 172.21.15.82:0/346010775 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8026070 con 0x55a1817bfc90 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbd3fff640 1 -- 172.21.15.82:0/346010775 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8005650 con 0x55a1817bfc90 2024-04-25T10:12:00.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbd3fff640 1 -- 172.21.15.82:0/346010775 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8005800 con 0x55a1817bfc90 2024-04-25T10:12:00.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- 172.21.15.82:0/346010775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 msgr2=0x55a1817c0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fcbe80016d0 tx=0x7fcbe8012000 comp rx=0 tx=0).stop 2024-04-25T10:12:00.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- 172.21.15.82:0/346010775 shutdown_connections 2024-04-25T10:12:00.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a181776290 0x55a181776670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a181775d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 --2- 172.21.15.82:0/346010775 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817c0070 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- 172.21.15.82:0/346010775 >> 172.21.15.82:0/346010775 conn(0x55a181772240 msgr2=0x55a181774630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- 172.21.15.82:0/346010775 shutdown_connections 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.412+0000 7fcbfb714000 1 -- 172.21.15.82:0/346010775 wait complete. 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- start start 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 Processor -- start 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a181780e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181781370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18157e6c0 con 0x55a1817c05b0 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1815c0110 con 0x55a181776290 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a181631c10 con 0x55a1817bfc90 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 10 client.0 _ll_register_callbacks cb 0x55a181761570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32952/0 (socket says 172.21.15.82:32952) 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 -- 172.21.15.82:0/1322028536 learned_addr learned my addr 172.21.15.82:0/1322028536 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181781370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6343640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a181780e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 -- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 msgr2=0x55a181781370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181781370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 -- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 msgr2=0x55a181780e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a181780e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 -- 172.21.15.82:0/1322028536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a18154c3b0 con 0x55a1817c05b0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0601c0 tx=0x7fcbec0587a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f477a3a7e6aab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0606f0 con 0x55a1817c05b0 2024-04-25T10:12:00.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcbec060260 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe8005650 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbec076050 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 10 client.9120 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55a181645e00 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8005650 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbec057430 con 0x55a1817c05b0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 10 client.9120 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 10 client.9120 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 20 client.9120 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a181782180 0x55a1817d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 4 client.9120 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a18155fd60 con 0x55a181782180 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6343640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a181782180 0x55a1817d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.426 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 10 client.9120 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbf6343640 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a181782180 0x55a1817d1410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6bab8a3be9df7a server_cookie=5a4edd9876327ad0 in_seq=0 out_seq=0 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 1 -- 172.21.15.82:0/1322028536 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a18155fd60 con 0x55a181782180 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 10 client.9120 handle_command_reply: tid=0 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 client.9120 shutdown 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 msgr2=0x55a1817818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0601c0 tx=0x7fcbec0587a0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 shutdown_connections 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a181782180 0x55a1817d1410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181781370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817c05b0 0x55a1817818b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbd17fa640 0 client.9120 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 --2- 172.21.15.82:0/1322028536 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a181780e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 >> 172.21.15.82:0/1322028536 conn(0x55a181772240 msgr2=0x55a181774ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 shutdown_connections 2024-04-25T10:12:00.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 1 -- 172.21.15.82:0/1322028536 wait complete. 2024-04-25T10:12:00.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.416+0000 7fcbfb714000 20 client.9120 trim_cache size 0 max 16384 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 Processor -- start 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 -- start start 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 0x7ff508099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff508099fb0 0x7ff5080b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5080cf120 con 0x7ff50815ed60 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50809a410 con 0x7ff508099fb0 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50f4da640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cf3e0 con 0x7ff5080d01a0 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50ca75640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 0x7ff508099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50da77640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff508099fb0 0x7ff5080b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50ca75640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 0x7ff508099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:47176/0 (socket says 172.21.15.82:47176) 2024-04-25T10:12:00.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50ca75640 1 -- 172.21.15.82:0/2426357903 learned_addr learned my addr 172.21.15.82:0/2426357903 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50da77640 1 -- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 msgr2=0x7ff508099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50da77640 1 --2- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 0x7ff508099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50da77640 1 -- 172.21.15.82:0/2426357903 >> v1:172.21.15.196:6789/0 conn(0x7ff50815ed60 legacy=0x7ff5080ccd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.420+0000 7ff50da77640 1 -- 172.21.15.82:0/2426357903 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5080cf6c0 con 0x7ff508099fb0 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50da77640 1 --2- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff5080b0cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d90 tx=0x7ff50001c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fab1c03f6cdf05b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff4fffff640 1 -- 172.21.15.82:0/2426357903 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000b2070 con 0x7ff508099fb0 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff4fffff640 1 -- 172.21.15.82:0/2426357903 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5000a5920 con 0x7ff508099fb0 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff4fffff640 1 -- 172.21.15.82:0/2426357903 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a5cf0 con 0x7ff508099fb0 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 -- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 msgr2=0x7ff5080b0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 --2- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff5080b0cd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff500009d90 tx=0x7ff50001c970 comp rx=0 tx=0).stop 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 -- 172.21.15.82:0/2426357903 shutdown_connections 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 --2- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff5080b0cd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 --2- 172.21.15.82:0/2426357903 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff5080d01a0 0x7ff508099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 -- 172.21.15.82:0/2426357903 >> 172.21.15.82:0/2426357903 conn(0x7ff5080b3040 msgr2=0x7ff5080b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 -- 172.21.15.82:0/2426357903 shutdown_connections 2024-04-25T10:12:00.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.424+0000 7ff50f4da640 1 -- 172.21.15.82:0/2426357903 wait complete. 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 Processor -- start 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- start start 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff50815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080d01a0 0x7ff5080c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff50815ed60 0x7ff5080c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5080cd410 con 0x7ff5080d01a0 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50809aa40 con 0x7ff508099fb0 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50815bbb0 con 0x7ff50815ed60 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080d01a0 0x7ff5080c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080d01a0 0x7ff5080c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32968/0 (socket says 172.21.15.82:32968) 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 -- 172.21.15.82:0/3222786613 learned_addr learned my addr 172.21.15.82:0/3222786613 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50d276640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff50815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50da77640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff50815ed60 0x7ff5080c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 -- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff50815ed60 msgr2=0x7ff5080c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff50815ed60 0x7ff5080c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 -- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 msgr2=0x7ff50815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff50815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 -- 172.21.15.82:0/3222786613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50815acc0 con 0x7ff5080d01a0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50d276640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff508099fb0 0x7ff50815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50da77640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff50815ed60 0x7ff5080c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50ca75640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff5080d01a0 0x7ff5080c7680 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff4f800e870 tx=0x7ff4f800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f8017070 con 0x7ff5080d01a0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f800c8e0 con 0x7ff5080d01a0 2024-04-25T10:12:00.436 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- 172.21.15.82:0/3222786613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50815afa0 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f800ccf0 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- 172.21.15.82:0/3222786613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50815b560 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff50f4da640 1 -- 172.21.15.82:0/3222786613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff508068340 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.428+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4f801a070 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff4fdffb640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f60 0x7ff4ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff4f8085890 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff50d276640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f60 0x7ff4ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7ff4f80862d0 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff50f4da640 1 -- 172.21.15.82:0/3222786613 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7ff5080b0d20 con 0x7ff5080d01a0 2024-04-25T10:12:00.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.432+0000 7ff50d276640 1 --2- 172.21.15.82:0/3222786613 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff4ec066f60 0x7ff4ec069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff5080cf050 tx=0x7ff504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 Processor -- start 2024-04-25T10:12:00.439 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- start start 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181775b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a1817d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf0200770 con 0x55a181776290 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01ff910 con 0x55a1817c05b0 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00b3520 con 0x55a1817bfc90 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32976/0 (socket says 172.21.15.82:32976) 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 -- 172.21.15.82:0/957848961 learned_addr learned my addr 172.21.15.82:0/957848961 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 -- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 msgr2=0x55a181775b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181775b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 -- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 msgr2=0x55a1817d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a1817d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 -- 172.21.15.82:0/957848961 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a18161bc50 con 0x55a181776290 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbf6343640 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800f660 tx=0x7fcbe8005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358c573e4b372f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbd37fe640 1 -- 172.21.15.82:0/957848961 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8037070 con 0x55a181776290 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbd37fe640 1 -- 172.21.15.82:0/957848961 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8027040 con 0x55a181776290 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbd37fe640 1 -- 172.21.15.82:0/957848961 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8026050 con 0x55a181776290 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- 172.21.15.82:0/957848961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 msgr2=0x55a181775640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800f660 tx=0x7fcbe8005000 comp rx=0 tx=0).stop 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- 172.21.15.82:0/957848961 shutdown_connections 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817bfc90 0x55a181775b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a1817c05b0 0x55a1817d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 --2- 172.21.15.82:0/957848961 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a181776290 0x55a181775640 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- 172.21.15.82:0/957848961 >> 172.21.15.82:0/957848961 conn(0x55a181772240 msgr2=0x55a18177ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.436+0000 7fcbfb714000 1 -- 172.21.15.82:0/957848961 shutdown_connections 2024-04-25T10:12:00.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- 172.21.15.82:0/957848961 wait complete. 2024-04-25T10:12:00.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- start start 2024-04-25T10:12:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 Processor -- start 2024-04-25T10:12:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a1817d3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817c05b0 0x55a1817dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a18177bb70 con 0x55a1817bfc90 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf01ff370 con 0x55a181776290 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a181566bd0 con 0x55a1817c05b0 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 10 client.0 _ll_register_callbacks cb 0x7fcbe80017a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32984/0 (socket says 172.21.15.82:32984) 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 -- 172.21.15.82:0/2747430828 learned_addr learned my addr 172.21.15.82:0/2747430828 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf6343640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a1817d3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 -- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817c05b0 msgr2=0x55a1817dc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf6b44640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817c05b0 0x55a1817dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817c05b0 0x55a1817dc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 -- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 msgr2=0x55a1817d3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a1817d3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 -- 172.21.15.82:0/2747430828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a18158c120 con 0x55a1817bfc90 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf6343640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a1817d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf5b42640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0000c10 tx=0x7fcbe00197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0ea9f62937b824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00b2050 con 0x55a1817bfc90 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbe8027040 con 0x55a1817bfc90 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fcbe00096b0 con 0x55a1817bfc90 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbe00c4050 con 0x55a1817bfc90 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 10 client.9129 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:00.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55a18156b170 con 0x55a1817bfc90 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8027040 con 0x55a1817bfc90 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcbe00c7020 con 0x55a1817bfc90 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 10 client.9129 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 10 client.9129 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 20 client.9129 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a1817dc960 0x55a181820180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 4 client.9129 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55a1815d15b0 con 0x55a1817dc960 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf6343640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a1817dc960 0x55a181820180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbf6343640 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a1817dc960 0x55a181820180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635cb0d7274ad150 server_cookie=676be2db9a479847 in_seq=0 out_seq=0 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.440+0000 7fcbd17fa640 10 client.9129 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbd17fa640 1 -- 172.21.15.82:0/2747430828 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a1815d15b0 con 0x55a1817dc960 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbd17fa640 10 client.9129 handle_command_reply: tid=0 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 client.9129 shutdown 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 msgr2=0x55a1817dbee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0000c10 tx=0x7fcbe00197f0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 shutdown_connections 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55a1817dc960 0x55a181820180 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55a1817c05b0 0x55a1817dc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a1817bfc90 0x55a1817dbee0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 --2- 172.21.15.82:0/2747430828 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a181776290 0x55a1817d3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbd17fa640 0 client.9129 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 >> 172.21.15.82:0/2747430828 conn(0x55a181772240 msgr2=0x55a181774630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 shutdown_connections 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 1 -- 172.21.15.82:0/2747430828 wait complete. 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbfb714000 20 client.9129 trim_cache size 0 max 16384 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "272c08c5-0492-4e91-8c66-3bd0145fb8a8", 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:12:00.454 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 msgr2=0x7fcbd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 0x7fcbd4069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8007fd0 tx=0x7fcbe8009040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 msgr2=0x7fcbf0160600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0071430 tx=0x7fcbec010c80 comp rx=0 tx=0).stop 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 shutdown_connections 2024-04-25T10:12:00.455 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcbf01f3960 0x7fcbf0160600 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcbf00b4b40 0x7fcbf0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 --2- 172.21.15.82:0/157299567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcbdc004810 0x7fcbf01f7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 >> 172.21.15.82:0/157299567 conn(0x7fcbf00cc680 msgr2=0x7fcbf00b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 shutdown_connections 2024-04-25T10:12:00.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.444+0000 7fcbf85a7640 1 -- 172.21.15.82:0/157299567 wait complete. 2024-04-25T10:12:00.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:00.462 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:12:00.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.564+0000 7f5f5fa6d640 1 Processor -- start 2024-04-25T10:12:00.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.564+0000 7f5f5fa6d640 1 -- start start 2024-04-25T10:12:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5f580af280 0x7f5f5815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f580b2c30 0x7f5f5806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f5806c1e0 0x7f5f58161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f5806daa0 con 0x7f5f580b2c30 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f58066c30 con 0x7f5f580af280 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5fa6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f580af660 con 0x7f5f5806c1e0 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f580b2c30 0x7f5f5806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5f580b2c30 0x7f5f5806bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32994/0 (socket says 172.21.15.82:32994) 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 -- 172.21.15.82:0/651628113 learned_addr learned my addr 172.21.15.82:0/651628113 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f5806c1e0 0x7f5f58161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d809640 1 --2- 172.21.15.82:0/651628113 >> v2:172.21.15.196:3300/0 conn(0x7f5f580af280 0x7f5f5815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 -- 172.21.15.82:0/651628113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f5806c1e0 msgr2=0x7f5f58161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f5806c1e0 0x7f5f58161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 -- 172.21.15.82:0/651628113 >> v2:172.21.15.196:3300/0 conn(0x7f5f580af280 msgr2=0x7f5f5815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 --2- 172.21.15.82:0/651628113 >> v2:172.21.15.196:3300/0 conn(0x7f5f580af280 0x7f5f5815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.572 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 -- 172.21.15.82:0/651628113 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5815e640 con 0x7f5f580b2c30 2024-04-25T10:12:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5d008640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f580b2c30 0x7f5f5806bca0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5f580cfbf0 tx=0x7f5f4801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e8acb9fba76fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.568+0000 7f5f5c807640 1 -- 172.21.15.82:0/651628113 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480b0070 con 0x7f5f580b2c30 2024-04-25T10:12:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5c807640 1 -- 172.21.15.82:0/651628113 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480ab600 con 0x7f5f580b2c30 2024-04-25T10:12:00.574 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5c807640 1 -- 172.21.15.82:0/651628113 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480ab9d0 con 0x7f5f580b2c30 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/651628113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f580b2c30 msgr2=0x7f5f5806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f580b2c30 0x7f5f5806bca0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5f580cfbf0 tx=0x7f5f4801c260 comp rx=0 tx=0).stop 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/651628113 shutdown_connections 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f5806c1e0 0x7f5f58161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/651628113 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f580b2c30 0x7f5f5806bca0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/651628113 >> v2:172.21.15.196:3300/0 conn(0x7f5f580af280 0x7f5f5815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/651628113 >> 172.21.15.82:0/651628113 conn(0x7f5f58071810 msgr2=0x7f5f58071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/651628113 shutdown_connections 2024-04-25T10:12:00.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/651628113 wait complete. 2024-04-25T10:12:00.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 Processor -- start 2024-04-25T10:12:00.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- start start 2024-04-25T10:12:00.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 0x7f5f5815eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.577 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f580b2c30 0x7f5f5815f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5806e2a0 con 0x7f5f5806c1e0 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5806e120 con 0x7f5f580af280 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5806e420 con 0x7f5f580b2c30 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33006/0 (socket says 172.21.15.82:33006) 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 -- 172.21.15.82:0/1543296207 learned_addr learned my addr 172.21.15.82:0/1543296207 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 0x7f5f5815eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 -- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f580b2c30 msgr2=0x7f5f5815f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f580b2c30 0x7f5f5815f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 -- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 msgr2=0x7f5f5815eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 0x7f5f5815eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 -- 172.21.15.82:0/1543296207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58153ec0 con 0x7f5f5806c1e0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 0x7f5f5815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400d800 tx=0x7f5f5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f4e7fc640 1 -- 172.21.15.82:0/1543296207 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400b660 con 0x7f5f5806c1e0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f4e7fc640 1 -- 172.21.15.82:0/1543296207 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f54019070 con 0x7f5f5806c1e0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f581541a0 con 0x7f5f5806c1e0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f4e7fc640 1 -- 172.21.15.82:0/1543296207 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5400ba00 con 0x7f5f5806c1e0 2024-04-25T10:12:00.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.572+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f58154480 con 0x7f5f5806c1e0 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f4e7fc640 1 -- 172.21.15.82:0/1543296207 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5f5400bba0 con 0x7f5f5806c1e0 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f4e7fc640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 0x7f5f3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f4e7fc640 1 -- 172.21.15.82:0/1543296207 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5f54085640 con 0x7f5f5806c1e0 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 0x7f5f3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 0x7f5f3c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5f480ad000 tx=0x7f5f48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 Processor -- start 2024-04-25T10:12:00.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 -- start start 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc96eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc96a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc94f0ff0 con 0x55adc96ee2f0 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc9479150 con 0x55adc96ed9d0 2024-04-25T10:12:00.583 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc94ed090 con 0x55adc96a40a0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33014/0 (socket says 172.21.15.82:33014) 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 -- 172.21.15.82:0/1164499788 learned_addr learned my addr 172.21.15.82:0/1164499788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc96a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc96eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 -- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 msgr2=0x55adc96a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc96a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 -- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 msgr2=0x55adc96eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc96eddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 -- 172.21.15.82:0/1164499788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adc96870d0 con 0x55adc96ee2f0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc96a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c10 tx=0x7f5f480a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15abb439a0dbdd00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f3b7fe640 1 -- 172.21.15.82:0/1164499788 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480c6070 con 0x55adc96ee2f0 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f3b7fe640 1 -- 172.21.15.82:0/1164499788 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480b5460 con 0x55adc96ee2f0 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.576+0000 7f5f3b7fe640 1 -- 172.21.15.82:0/1164499788 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480b0040 con 0x55adc96ee2f0 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/1164499788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 msgr2=0x55adc96a3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f5f48000c10 tx=0x7f5f480a6000 comp rx=0 tx=0).stop 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/1164499788 shutdown_connections 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc96a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96a3b60 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1164499788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc96eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/1164499788 >> 172.21.15.82:0/1164499788 conn(0x55adc96a0020 msgr2=0x55adc96a2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/1164499788 shutdown_connections 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/1164499788 wait complete. 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- start start 2024-04-25T10:12:00.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 Processor -- start 2024-04-25T10:12:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 0x55adc96d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ed9d0 0x55adc96d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc94b5250 con 0x55adc96ee2f0 2024-04-25T10:12:00.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc94ab690 con 0x55adc96a40a0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc9479a40 con 0x55adc96ed9d0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 10 client.0 _ll_register_callbacks cb 0x55adc968ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33022/0 (socket says 172.21.15.82:33022) 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 -- 172.21.15.82:0/2746627607 learned_addr learned my addr 172.21.15.82:0/2746627607 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5d008640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ed9d0 0x55adc96d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5d809640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 0x55adc96d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 -- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ed9d0 msgr2=0x55adc96d2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ed9d0 0x55adc96d2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 -- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 msgr2=0x55adc96d27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 0x55adc96d27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 -- 172.21.15.82:0/2746627607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adc959e1c0 con 0x55adc96ee2f0 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5d809640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 0x55adc96d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5e00a640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5f500079c0 tx=0x7f5f50007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a576d3c9a2bc0ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f50018020 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5f5000fc60 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f480b5460 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f50022050 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 10 client.9141 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55adc959a170 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f480b5460 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f50025020 con 0x55adc96ee2f0 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 10 client.9141 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 10 client.9141 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:00.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 20 client.9141 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc96fdc80 0x55adc9700070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 4 client.9141 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55adc948cc60 con 0x55adc96fdc80 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5d809640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc96fdc80 0x55adc9700070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 10 client.9141 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f5d809640 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc96fdc80 0x55adc9700070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f358c94bc6afdfb8 server_cookie=a1b7c836ca967653 in_seq=0 out_seq=0 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 1 -- 172.21.15.82:0/2746627607 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55adc948cc60 con 0x55adc96fdc80 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.580+0000 7f5f38ff9640 10 client.9141 handle_command_reply: tid=0 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 client.9141 shutdown 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 msgr2=0x55adc96d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f5f500079c0 tx=0x7f5f50007e80 comp rx=0 tx=0).stop 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 shutdown_connections 2024-04-25T10:12:00.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc96fdc80 0x55adc9700070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ed9d0 0x55adc96d2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ee2f0 0x55adc96d3270 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f38ff9640 0 client.9141 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 --2- 172.21.15.82:0/2746627607 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96a40a0 0x55adc96d27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 >> 172.21.15.82:0/2746627607 conn(0x55adc96a0020 msgr2=0x55adc96a2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 shutdown_connections 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 1 -- 172.21.15.82:0/2746627607 wait complete. 2024-04-25T10:12:00.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.584+0000 7f5f62bda000 20 client.9141 trim_cache size 0 max 16384 2024-04-25T10:12:00.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 Processor -- start 2024-04-25T10:12:00.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 -- start start 2024-04-25T10:12:00.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc9749ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ee2f0 0x55adc974a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58203270 con 0x55adc96ed9d0 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f58202470 con 0x55adc96ee2f0 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5814b3e0 con 0x55adc96a40a0 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33036/0 (socket says 172.21.15.82:33036) 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 -- 172.21.15.82:0/3490411606 learned_addr learned my addr 172.21.15.82:0/3490411606 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d809640 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc9749ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 -- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 msgr2=0x55adc9749ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc9749ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 -- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ee2f0 msgr2=0x55adc974a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ee2f0 0x55adc974a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 -- 172.21.15.82:0/3490411606 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55adc9537de0 con 0x55adc96ed9d0 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f5d008640 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5f48006db0 tx=0x7f5f48004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb8e9cbe31acd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f3affd640 1 -- 172.21.15.82:0/3490411606 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480cf070 con 0x55adc96ed9d0 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f3affd640 1 -- 172.21.15.82:0/3490411606 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f48004240 con 0x55adc96ed9d0 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.596+0000 7f5f3affd640 1 -- 172.21.15.82:0/3490411606 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f480c6040 con 0x55adc96ed9d0 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/3490411606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 msgr2=0x55adc974a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5f48006db0 tx=0x7f5f48004740 comp rx=0 tx=0).stop 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/3490411606 shutdown_connections 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ee2f0 0x55adc974a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96ed9d0 0x55adc974a420 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/3490411606 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96a40a0 0x55adc9749ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/3490411606 >> 172.21.15.82:0/3490411606 conn(0x55adc96a0020 msgr2=0x55adc96ec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/3490411606 shutdown_connections 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/3490411606 wait complete. 2024-04-25T10:12:00.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- start start 2024-04-25T10:12:00.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 Processor -- start 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc9701110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ee2f0 0x55adc9701650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55adc948be00 con 0x55adc96a40a0 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5815d110 con 0x55adc96ed9d0 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55adc94f0300 con 0x55adc96ee2f0 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 10 client.0 _ll_register_callbacks cb 0x55adc966f040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:33038/0 (socket says 172.21.15.82:33038) 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 -- 172.21.15.82:0/1216418193 learned_addr learned my addr 172.21.15.82:0/1216418193 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 -- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ee2f0 msgr2=0x55adc9701650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.609 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ee2f0 0x55adc9701650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 -- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 msgr2=0x55adc9701110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc9701110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 -- 172.21.15.82:0/1216418193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55adc94928e0 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d809640 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b4b0 tx=0x7f5f5400d750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93043099a416fb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5405c0b0 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5f54052410 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f48004240 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f5405e020 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 10 client.9147 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x55adc954a8f0 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f48004240 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5f54013070 con 0x55adc96a40a0 2024-04-25T10:12:00.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 10 client.9147 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 10 client.9147 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 20 client.9147 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc9753f50 0x55adc9756340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 4 client.9147 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55adc94fe530 con 0x55adc9753f50 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc9753f50 0x55adc9756340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 10 client.9147 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f5d008640 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc9753f50 0x55adc9756340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd5a3d26899e7e server_cookie=32d071c6da7a6ff2 in_seq=0 out_seq=0 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 1 -- 172.21.15.82:0/1216418193 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55adc94fe530 con 0x55adc9753f50 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 10 client.9147 handle_command_reply: tid=0 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 client.9147 shutdown 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 msgr2=0x55adc9700bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400b4b0 tx=0x7f5f5400d750 comp rx=0 tx=0).stop 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 shutdown_connections 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x55adc9753f50 0x55adc9756340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55adc96ee2f0 0x55adc9701650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f38ff9640 0 client.9147 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55adc96ed9d0 0x55adc9701110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 --2- 172.21.15.82:0/1216418193 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55adc96a40a0 0x55adc9700bd0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 >> 172.21.15.82:0/1216418193 conn(0x55adc96a0020 msgr2=0x55adc96ec960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 shutdown_connections 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 1 -- 172.21.15.82:0/1216418193 wait complete. 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.600+0000 7f5f62bda000 20 client.9147 trim_cache size 0 max 16384 2024-04-25T10:12:00.612 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "272c08c5-0492-4e91-8c66-3bd0145fb8a8": { 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "272c08c5-0492-4e91-8c66-3bd0145fb8a8", 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:12:00.613 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 msgr2=0x7f5f3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 0x7f5f3c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5f480ad000 tx=0x7f5f48008040 comp rx=0 tx=0).stop 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 msgr2=0x7f5f5813c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400d800 tx=0x7f5f5400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 shutdown_connections 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5f3c066f60 0x7f5f3c069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5f580b2c30 0x7f5f5815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5f580af280 0x7f5f5815eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 --2- 172.21.15.82:0/1543296207 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5f5806c1e0 0x7f5f5813c4d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 >> 172.21.15.82:0/1543296207 conn(0x7f5f58071810 msgr2=0x7f5f580b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 shutdown_connections 2024-04-25T10:12:00.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:00.604+0000 7f5f5fa6d640 1 -- 172.21.15.82:0/1543296207 wait complete. 2024-04-25T10:12:00.622 INFO:tasks.cephfs.filesystem:scrub status for tag:272c08c5-0492-4e91-8c66-3bd0145fb8a8 - {'path': '/', 'tag': '272c08c5-0492-4e91-8c66-3bd0145fb8a8', 'options': 'recursive,repair'} 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:01.612+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T10:12:00.447436+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7ff4f8083d20 con 0x7ff5080d01a0 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stdout: cluster: 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stdout: id: 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stdout: health: HEALTH_OK 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi082.stdout: services: 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: mgr: y(active, since 10m), standbys: x 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: data: 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: volumes: 1/1 healthy 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: pgs: 97 active+clean 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:01.618 INFO:teuthology.orchestra.run.smithi082.stdout: io: 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout: client: 803 B/s wr, 0 op/s rd, 0 op/s wr 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:00.447436+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:00.447440+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:00.447444+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:12:01.619 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:00.449131+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:12:04.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:04.620+0000 7ff4fdffb640 1 -- 172.21.15.82:0/3222786613 <== mon.0 v2:172.21.15.82:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T10:12:04.119859+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7ff4f8086000 con 0x7ff5080d01a0 2024-04-25T10:12:04.624 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:04.119859+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:12:08.444 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:08.447 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:30.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:30.623 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:12:30.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 Processor -- start 2024-04-25T10:12:30.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 -- start start 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f39bc0cf590 0x7f39bc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf970 0x7f39bc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 0x7f39bc0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0ca780 con 0x7f39bc0cf590 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc0cd440 con 0x7f39bc0c9ef0 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c4712640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0cdce0 con 0x7f39bc0cf970 2024-04-25T10:12:30.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.756+0000 7f39c24ae640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f39bc0cf590 0x7f39bc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f39bc0cf590 0x7f39bc15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56236/0 (socket says 172.21.15.82:56236) 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 -- 172.21.15.82:0/925415055 learned_addr learned my addr 172.21.15.82:0/925415055 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c2caf640 1 --2- 172.21.15.82:0/925415055 >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 0x7f39bc0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c1cad640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf970 0x7f39bc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 -- 172.21.15.82:0/925415055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf970 msgr2=0x7f39bc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf970 0x7f39bc0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 -- 172.21.15.82:0/925415055 >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 msgr2=0x7f39bc0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 --2- 172.21.15.82:0/925415055 >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 0x7f39bc0b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 -- 172.21.15.82:0/925415055 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0ca4c0 con 0x7f39bc0cf590 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c2caf640 1 --2- 172.21.15.82:0/925415055 >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 0x7f39bc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c24ae640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf590 0x7f39bc15ecb0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0cf310 tx=0x7f39b801c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee17ba067710dfbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c14ac640 1 -- 172.21.15.82:0/925415055 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80a55a0 con 0x7f39bc0cf590 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c14ac640 1 -- 172.21.15.82:0/925415055 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80a5740 con 0x7f39bc0cf590 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c14ac640 1 -- 172.21.15.82:0/925415055 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80a5b10 con 0x7f39bc0cf590 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 -- 172.21.15.82:0/925415055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf590 msgr2=0x7f39bc15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf590 0x7f39bc15ecb0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0cf310 tx=0x7f39b801c700 comp rx=0 tx=0).stop 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 -- 172.21.15.82:0/925415055 shutdown_connections 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 --2- 172.21.15.82:0/925415055 >> v2:172.21.15.196:3300/0 conn(0x7f39bc0c9ef0 0x7f39bc0b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf970 0x7f39bc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 --2- 172.21.15.82:0/925415055 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf590 0x7f39bc15ecb0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 -- 172.21.15.82:0/925415055 >> 172.21.15.82:0/925415055 conn(0x7f39bc06ae60 msgr2=0x7f39bc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 -- 172.21.15.82:0/925415055 shutdown_connections 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.760+0000 7f39c4712640 1 -- 172.21.15.82:0/925415055 wait complete. 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 Processor -- start 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- start start 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0c9ef0 0x7f39bc0c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf590 0x7f39bc145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0ac2f0 con 0x7f39bc0cf970 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0ac170 con 0x7f39bc0c9ef0 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc0ce170 con 0x7f39bc0cf590 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56252/0 (socket says 172.21.15.82:56252) 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 -- 172.21.15.82:0/2237257219 learned_addr learned my addr 172.21.15.82:0/2237257219 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c1cad640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf590 0x7f39bc145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0c9ef0 0x7f39bc0c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 -- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf590 msgr2=0x7f39bc145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf590 0x7f39bc145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 -- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0c9ef0 msgr2=0x7f39bc0c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0c9ef0 0x7f39bc0c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 -- 172.21.15.82:0/2237257219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc0c4080 con 0x7f39bc0cf970 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c2caf640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f39b400e5e0 tx=0x7f39b400eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39b2ffd640 1 -- 172.21.15.82:0/2237257219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b400f040 con 0x7f39bc0cf970 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39b2ffd640 1 -- 172.21.15.82:0/2237257219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4004230 con 0x7f39bc0cf970 2024-04-25T10:12:30.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc0c4360 con 0x7f39bc0cf970 2024-04-25T10:12:30.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39b2ffd640 1 -- 172.21.15.82:0/2237257219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4003620 con 0x7f39bc0cf970 2024-04-25T10:12:30.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.764+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc158280 con 0x7f39bc0cf970 2024-04-25T10:12:30.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39b2ffd640 1 -- 172.21.15.82:0/2237257219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39b40037c0 con 0x7f39bc0cf970 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39b2ffd640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 0x7f39a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39b2ffd640 1 -- 172.21.15.82:0/2237257219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f39b4084e80 con 0x7f39bc0cf970 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 0x7f39a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 0x7f39a0069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ae000 tx=0x7f39b80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 Processor -- start 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- start start 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 0x561ff03f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03a90a0 0x561ff03a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff01f5ff0 con 0x561ff03f32f0 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff017e150 con 0x561ff03a90a0 2024-04-25T10:12:30.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ff01f2090 con 0x561ff03f29d0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56268/0 (socket says 172.21.15.82:56268) 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 -- 172.21.15.82:0/1628828567 learned_addr learned my addr 172.21.15.82:0/1628828567 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c24ae640 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 0x561ff03f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 -- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 msgr2=0x561ff03f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 0x561ff03f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 -- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03a90a0 msgr2=0x561ff03a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03a90a0 0x561ff03a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 -- 172.21.15.82:0/1628828567 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff038c0d0 con 0x561ff03f32f0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c24ae640 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 0x561ff03f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c1cad640 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b820 tx=0x7f39ac00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277d7c98a108b963 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f399ffff640 1 -- 172.21.15.82:0/1628828567 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac018020 con 0x561ff03f32f0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f399ffff640 1 -- 172.21.15.82:0/1628828567 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00f9c0 con 0x561ff03f32f0 2024-04-25T10:12:30.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f399ffff640 1 -- 172.21.15.82:0/1628828567 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00fdb0 con 0x561ff03f32f0 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- 172.21.15.82:0/1628828567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 msgr2=0x561ff03a8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b820 tx=0x7f39ac00bce0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- 172.21.15.82:0/1628828567 shutdown_connections 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03a90a0 0x561ff03a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff03a8b60 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 --2- 172.21.15.82:0/1628828567 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f29d0 0x561ff03f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- 172.21.15.82:0/1628828567 >> 172.21.15.82:0/1628828567 conn(0x561ff03a5020 msgr2=0x561ff03a7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- 172.21.15.82:0/1628828567 shutdown_connections 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.768+0000 7f39c787f000 1 -- 172.21.15.82:0/1628828567 wait complete. 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- start start 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 Processor -- start 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff03d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff01ba250 con 0x561ff03a90a0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff01b0690 con 0x561ff03f29d0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ff017ea40 con 0x561ff03f32f0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 10 client.0 _ll_register_callbacks cb 0x561ff0393e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56274/0 (socket says 172.21.15.82:56274) 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 -- 172.21.15.82:0/2889740071 learned_addr learned my addr 172.21.15.82:0/2889740071 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c2caf640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff03d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c1cad640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 -- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 msgr2=0x561ff03d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff03d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 -- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 msgr2=0x561ff03d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.783 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 -- 172.21.15.82:0/2889740071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff02a31c0 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c1cad640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c24ae640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f39b80072a0 tx=0x7f39b801af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d761821d27c82272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80c7020 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f39b80c4cd0 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39ac00f9c0 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b80c4ed0 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 10 client.9159 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x561ff029f170 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00f9c0 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39b80c3470 con 0x561ff03a90a0 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 10 client.9159 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:30.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 10 client.9159 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 20 client.9159 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0402c80 0x561ff0405070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 4 client.9159 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561ff0191c60 con 0x561ff0402c80 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c1cad640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0402c80 0x561ff0405070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 10 client.9159 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c1cad640 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0402c80 0x561ff0405070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88a70847f9879487 server_cookie=2a48eaed894c1fd3 in_seq=0 out_seq=0 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 1 -- 172.21.15.82:0/2889740071 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561ff0191c60 con 0x561ff0402c80 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 10 client.9159 handle_command_reply: tid=0 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 client.9159 shutdown 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 msgr2=0x561ff03d77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f39b80072a0 tx=0x7f39b801af40 comp rx=0 tx=0).stop 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 shutdown_connections 2024-04-25T10:12:30.785 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0402c80 0x561ff0405070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff03d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f399d7fa640 0 client.9159 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 --2- 172.21.15.82:0/2889740071 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03d77f0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 >> 172.21.15.82:0/2889740071 conn(0x561ff03a5020 msgr2=0x561ff03a7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 shutdown_connections 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.772+0000 7f39c787f000 1 -- 172.21.15.82:0/2889740071 wait complete. 2024-04-25T10:12:30.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.776+0000 7f39c787f000 20 client.9159 trim_cache size 0 max 16384 2024-04-25T10:12:30.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 Processor -- start 2024-04-25T10:12:30.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 -- start start 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff0400380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc201120 con 0x561ff03a90a0 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc200320 con 0x561ff03f29d0 2024-04-25T10:12:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.788+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc15b480 con 0x561ff03f32f0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56286/0 (socket says 172.21.15.82:56286) 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 -- 172.21.15.82:0/1829226521 learned_addr learned my addr 172.21.15.82:0/1829226521 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 -- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 msgr2=0x561ff0400380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff0400380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 -- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 msgr2=0x561ff03ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 -- 172.21.15.82:0/1829226521 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ff023cde0 con 0x561ff03a90a0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f39b801af80 tx=0x7f39b80c0000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef6e91e2cab1444 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399f7fe640 1 -- 172.21.15.82:0/1829226521 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80d9070 con 0x561ff03a90a0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399f7fe640 1 -- 172.21.15.82:0/1829226521 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80c3470 con 0x561ff03a90a0 2024-04-25T10:12:30.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399f7fe640 1 -- 172.21.15.82:0/1829226521 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b80c7030 con 0x561ff03a90a0 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/1829226521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 msgr2=0x561ff03a8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f39b801af80 tx=0x7f39b80c0000 comp rx=0 tx=0).stop 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/1829226521 shutdown_connections 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03f32f0 0x561ff0400380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff03ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- 172.21.15.82:0/1829226521 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03a90a0 0x561ff03a8a50 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/1829226521 >> 172.21.15.82:0/1829226521 conn(0x561ff03a5020 msgr2=0x561ff03ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/1829226521 shutdown_connections 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/1829226521 wait complete. 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- start start 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 Processor -- start 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03a90a0 0x561ff045ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff045b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff0190e00 con 0x561ff03f32f0 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc1ffe20 con 0x561ff03f29d0 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ff01f5300 con 0x561ff03a90a0 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 10 client.0 _ll_register_callbacks cb 0x561ff0374040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56296/0 (socket says 172.21.15.82:56296) 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 -- 172.21.15.82:0/979781189 learned_addr learned my addr 172.21.15.82:0/979781189 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:30.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 -- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03a90a0 msgr2=0x561ff045ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c24ae640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03a90a0 0x561ff045ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c1cad640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff045b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03a90a0 0x561ff045ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 -- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 msgr2=0x561ff045b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff045b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 -- 172.21.15.82:0/979781189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff01978e0 con 0x561ff03f32f0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c2caf640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f39b4018000 tx=0x7f39b4055e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3c819d53d2c959 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b406d070 con 0x561ff03f32f0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f39b4054c30 con 0x561ff03f32f0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39b80c3470 con 0x561ff03f32f0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39b4020070 con 0x561ff03f32f0 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 10 client.9165 fetch_fsmap learned FSMap version 75 2024-04-25T10:12:30.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=75,monmap=2+}) v3 -- 0x561ff024f8f0 con 0x561ff03f32f0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.792+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b80c3470 con 0x561ff03f32f0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 75) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f39b4012050 con 0x561ff03f32f0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 10 client.9165 fetch_fsmap finished waiting for FSMap version 75 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 10 client.9165 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 20 client.9165 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0405880 0x561ff0405c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 4 client.9165 mds_command: new command op to 8769 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 --> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561ff0203530 con 0x561ff0405880 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c24ae640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0405880 0x561ff0405c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f399d7fa640 10 client.9165 ms_handle_connect on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c24ae640 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0405880 0x561ff0405c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5779a9428b038b4 server_cookie=96183fb24b9d269a in_seq=0 out_seq=0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f399d7fa640 1 -- 172.21.15.82:0/979781189 <== mds.0 v2:172.21.15.82:6836/3162787996 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561ff0203530 con 0x561ff0405880 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f399d7fa640 10 client.9165 handle_command_reply: tid=0 2024-04-25T10:12:30.803 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 client.9165 shutdown 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 msgr2=0x561ff0404a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f39b4018000 tx=0x7f39b4055e20 comp rx=0 tx=0).stop 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 shutdown_connections 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:6836/3162787996,v1:172.21.15.82:6837/3162787996] conn(0x561ff0405880 0x561ff0405c90 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ff03f32f0 0x561ff0404a20 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ff03f29d0 0x561ff045b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f399d7fa640 0 client.9165 ms_handle_reset on v2:172.21.15.82:6836/3162787996 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 --2- 172.21.15.82:0/979781189 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ff03a90a0 0x561ff045ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 >> 172.21.15.82:0/979781189 conn(0x561ff03a5020 msgr2=0x561ff03fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 shutdown_connections 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 1 -- 172.21.15.82:0/979781189 wait complete. 2024-04-25T10:12:30.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c787f000 20 client.9165 trim_cache size 0 max 16384 2024-04-25T10:12:30.805 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:30.805 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:12:30.805 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:12:30.805 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 msgr2=0x7f39a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 0x7f39a0069420 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f39b80ae000 tx=0x7f39b80a7070 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 msgr2=0x7f39bc15c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f39b400e5e0 tx=0x7f39b400eaa0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 shutdown_connections 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f39a0066f60 0x7f39a0069420 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f39bc0cf590 0x7f39bc145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39bc0cf970 0x7f39bc15c9b0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 --2- 172.21.15.82:0/2237257219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39bc0c9ef0 0x7f39bc0c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 >> 172.21.15.82:0/2237257219 conn(0x7f39bc06ae60 msgr2=0x7f39bc071e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 shutdown_connections 2024-04-25T10:12:30.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:30.796+0000 7f39c4712640 1 -- 172.21.15.82:0/2237257219 wait complete. 2024-04-25T10:12:30.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:30.916 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:31.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 Processor -- start 2024-04-25T10:12:31.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 -- start start 2024-04-25T10:12:31.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2aa80afb90 0x7f2aa80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa809a950 con 0x7f2aa815b920 2024-04-25T10:12:31.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa809ac30 con 0x7f2aa80afb90 2024-04-25T10:12:31.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aad7b2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80ae380 con 0x7f2aa809a180 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:60152/0 (socket says 172.21.15.82:60152) 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 -- 172.21.15.82:0/681742242 learned_addr learned my addr 172.21.15.82:0/681742242 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa77fe640 1 --2- 172.21.15.82:0/681742242 >> v2:172.21.15.196:3300/0 conn(0x7f2aa80afb90 0x7f2aa80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 -- 172.21.15.82:0/681742242 >> v2:172.21.15.196:3300/0 conn(0x7f2aa80afb90 msgr2=0x7f2aa80cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 --2- 172.21.15.82:0/681742242 >> v2:172.21.15.196:3300/0 conn(0x7f2aa80afb90 0x7f2aa80cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 -- 172.21.15.82:0/681742242 >> v1:172.21.15.196:6789/0 conn(0x7f2aa815b920 legacy=0x7f2aa8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.040+0000 7f2aa67fc640 1 -- 172.21.15.82:0/681742242 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80ae660 con 0x7f2aa809a180 2024-04-25T10:12:31.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aa67fc640 1 --2- 172.21.15.82:0/681742242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009870 tx=0x7f2a9c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=718d5a72282637ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aa5ffb640 1 -- 172.21.15.82:0/681742242 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0b1070 con 0x7f2aa809a180 2024-04-25T10:12:31.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aa5ffb640 1 -- 172.21.15.82:0/681742242 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0a8050 con 0x7f2aa809a180 2024-04-25T10:12:31.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aa5ffb640 1 -- 172.21.15.82:0/681742242 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0ac860 con 0x7f2aa809a180 2024-04-25T10:12:31.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 -- 172.21.15.82:0/681742242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 msgr2=0x7f2aa80cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/681742242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009870 tx=0x7f2a9c01c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 -- 172.21.15.82:0/681742242 shutdown_connections 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/681742242 >> v2:172.21.15.196:3300/0 conn(0x7f2aa80afb90 0x7f2aa80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/681742242 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80cc130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 -- 172.21.15.82:0/681742242 >> 172.21.15.82:0/681742242 conn(0x7f2aa80b3010 msgr2=0x7f2aa80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 -- 172.21.15.82:0/681742242 shutdown_connections 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.044+0000 7f2aad7b2640 1 -- 172.21.15.82:0/681742242 wait complete. 2024-04-25T10:12:31.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 Processor -- start 2024-04-25T10:12:31.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- start start 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 0x7f2aa80b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa81684c0 con 0x7f2aa80afb90 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa80cd610 con 0x7f2aa815b920 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa8168640 con 0x7f2aa809a180 2024-04-25T10:12:31.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56298/0 (socket says 172.21.15.82:56298) 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 -- 172.21.15.82:0/1807966073 learned_addr learned my addr 172.21.15.82:0/1807966073 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa6ffd640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa77fe640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 0x7f2aa80b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 -- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 msgr2=0x7f2aa80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 -- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 msgr2=0x7f2aa80b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 0x7f2aa80b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 -- 172.21.15.82:0/1807966073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa80b09d0 con 0x7f2aa80afb90 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa77fe640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 0x7f2aa80b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa6ffd640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aa67fc640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c000c10 tx=0x7f2a9c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0bc070 con 0x7f2aa80afb90 2024-04-25T10:12:31.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0b7630 con 0x7f2aa80afb90 2024-04-25T10:12:31.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa8070c80 con 0x7f2aa80afb90 2024-04-25T10:12:31.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0b1040 con 0x7f2aa80afb90 2024-04-25T10:12:31.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa80b4f60 con 0x7f2aa80afb90 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa804efa0 con 0x7f2aa80afb90 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.048+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a9c0bf070 con 0x7f2aa80afb90 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.052+0000 7f2a8f7fe640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 0x7f2a94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.052+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2a9c11ebc0 con 0x7f2aa80afb90 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.052+0000 7f2aa6ffd640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 0x7f2a94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.052+0000 7f2aa6ffd640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 0x7f2a94069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809f450 tx=0x7f2a98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.056+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a9c0ace20 con 0x7f2aa80afb90 2024-04-25T10:12:31.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.248+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2aa8099f40 con 0x7f2aa80afb90 2024-04-25T10:12:31.252 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:12:31.248+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:31.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2a8f7fe640 1 -- 172.21.15.82:0/1807966073 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2a9c016150 con 0x7f2aa80afb90 2024-04-25T10:12:31.254 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 msgr2=0x7f2a94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 0x7f2a94069420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2aa809f450 tx=0x7f2a98009040 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 msgr2=0x7f2aa80c96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c000c10 tx=0x7f2a9c01ad10 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 shutdown_connections 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a94066f60 0x7f2a94069420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2aa815b920 0x7f2aa80b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2aa80afb90 0x7f2aa80c96a0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 --2- 172.21.15.82:0/1807966073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2aa809a180 0x7f2aa80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 >> 172.21.15.82:0/1807966073 conn(0x7f2aa80b3010 msgr2=0x7f2aa80cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 shutdown_connections 2024-04-25T10:12:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.252+0000 7f2aad7b2640 1 -- 172.21.15.82:0/1807966073 wait complete. 2024-04-25T10:12:31.270 INFO:tasks.ceph.mds.c.smithi082.stderr: -6121> 2024-04-25T10:11:35.137+0000 7ff91a0fb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:31.277 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T10:12:31.321 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:12:31.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- start start 2024-04-25T10:12:31.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 Processor -- start 2024-04-25T10:12:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 0x56267d6f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.329 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e30e800 0x56267d6f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e308000 con 0x56267d53f800 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e308180 con 0x56267d53fc00 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56267e308300 con 0x56267e30e800 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56324/0 (socket says 172.21.15.82:56324) 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 -- 172.21.15.82:0/1241517761 learned_addr learned my addr 172.21.15.82:0/1241517761 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 0x56267d6f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dd322640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e30e800 0x56267d6f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 -- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e30e800 msgr2=0x56267d6f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e30e800 0x56267d6f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 -- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 msgr2=0x56267d6f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 0x56267d6f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 -- 172.21.15.82:0/1241517761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267d5e0b40 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 0x56267d6f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8dc320640 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x56267e3a0bd0 tx=0x56267d6320c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5292f872939fc3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8da31c640 1 -- 172.21.15.82:0/1241517761 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d574b60 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8da31c640 1 -- 172.21.15.82:0/1241517761 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56267d588c40 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8da31c640 1 -- 172.21.15.82:0/1241517761 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d5741a0 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56267d5e10e0 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56267d5e12c0 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 4 waiting for MDS map... 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8da31c640 1 -- 172.21.15.82:0/1241517761 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x56267d631a00 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8da31c640 1 -- 172.21.15.82:0/1241517761 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 76) v1 ==== 2723+0+0 (secure 0 0 0) 0x56267d595180 con 0x56267d53f800 2024-04-25T10:12:31.331 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 4 Got MDS map 76 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 10 main: JournalTool::main 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 Processor -- start 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.320+0000 7ff8de362400 1 -- start start 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e308480 con 0x56267e4ba400 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e308600 con 0x56267e4ba800 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56267e308780 con 0x56267e4ba000 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56338/0 (socket says 172.21.15.82:56338) 2024-04-25T10:12:31.332 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 -- 172.21.15.82:0/2743688092 learned_addr learned my addr 172.21.15.82:0/2743688092 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 -- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba000 msgr2=0x56267d6f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 -- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba800 msgr2=0x56267d6fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 -- 172.21.15.82:0/2743688092 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267d5e0960 con 0x56267e4ba400 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x56267e4746f0 tx=0x56267e472c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efbb846ec0fc9b7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d8318640 1 -- 172.21.15.82:0/2743688092 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d574b60 con 0x56267e4ba400 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d8318640 1 -- 172.21.15.82:0/2743688092 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56267d5881c0 con 0x56267e4ba400 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d8318640 1 -- 172.21.15.82:0/2743688092 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d5741a0 con 0x56267e4ba400 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/2743688092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 msgr2=0x56267d6fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x56267e4746f0 tx=0x56267e472c00 comp rx=0 tx=0).stop 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/2743688092 shutdown_connections 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- 172.21.15.82:0/2743688092 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/2743688092 >> 172.21.15.82:0/2743688092 conn(0x56267e30e000 msgr2=0x56267d56b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/2743688092 shutdown_connections 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/2743688092 wait complete. 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 Processor -- start 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- start start 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e309e00 con 0x56267e4ba000 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56267e309c80 con 0x56267e4ba400 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56267e309b00 con 0x56267e4ba800 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 0x56267d6f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56348/0 (socket says 172.21.15.82:56348) 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 -- 172.21.15.82:0/959232466 learned_addr learned my addr 172.21.15.82:0/959232466 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dc320640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba800 msgr2=0x56267d6fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e4ba800 0x56267d6fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba400 msgr2=0x56267d6fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267e4ba400 0x56267d6fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56267d5e1e00 con 0x56267e4ba000 2024-04-25T10:12:31.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8dd322640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 0x56267d6f8580 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x56267e4753e0 tx=0x56267e472d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d6b15640 1 -- 172.21.15.82:0/959232466 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d574b60 con 0x56267e4ba000 2024-04-25T10:12:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d6b15640 1 -- 172.21.15.82:0/959232466 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56267d589340 con 0x56267e4ba000 2024-04-25T10:12:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56267d87a000 con 0x56267e4ba000 2024-04-25T10:12:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8d6b15640 1 -- 172.21.15.82:0/959232466 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56267d5741a0 con 0x56267e4ba000 2024-04-25T10:12:31.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56267d87a3c0 con 0x56267e4ba000 2024-04-25T10:12:31.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.324+0000 7ff8de362400 4 main: JournalTool: resolving pool 15 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8d6b15640 1 -- 172.21.15.82:0/959232466 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x56267e4b4c00 con 0x56267e4ba000 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8d6b15640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56267d878800 0x56267d6fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8d6b15640 1 -- 172.21.15.82:0/959232466 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x56267d876a00 con 0x56267e4ba000 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 4 main: Executing for rank 0 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 4 Filtering by inode '1099511627777' 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56267d878800 0x56267d6fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56267e4bac00 0x56267d6fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x56267e4bb000 con 0x56267e4bac00 2024-04-25T10:12:31.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dcb21640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56267d878800 0x56267d6fb700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x56267e4746f0 tx=0x56267d632180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dc320640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56267e4bac00 0x56267d6fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dc320640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56267e4bac00 0x56267d6fac00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dc320640 1 -- 172.21.15.82:0/959232466 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x56267d6ba000 con 0x56267e4bac00 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 4 Pointer 400.00000000 is readable 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x56267d878c00 con 0x56267e4bac00 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8dc320640 1 -- 172.21.15.82:0/959232466 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x56267d6ba000 con 0x56267e4bac00 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 10 Starting journal scan from offset 0x405f0e 2024-04-25T10:12:31.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.328+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x56267d879000 con 0x56267e4bac00 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8dc320640 1 -- 172.21.15.82:0/959232466 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x56267d6ba000 con 0x56267e4bac00 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 10 Parsing data, 0x37c bytes available 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 10 Attempting decode at 0x405f0e 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 10 Consumed 0x37c bytes 2024-04-25T10:12:31.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 10 Valid entry at 0x405f0e 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56267d879400 0x56267d6fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x56267d879800 con 0x56267d879400 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8dd322640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56267d879400 0x56267d6fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.332+0000 7ff8dd322640 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56267d879400 0x56267d6fb180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8dd322640 1 -- 172.21.15.82:0/959232466 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56267d6ba500 con 0x56267d879400 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stdout:Events by type: 2024-04-25T10:12:31.343 INFO:teuthology.orchestra.run.smithi082.stdout:Errors: 0 2024-04-25T10:12:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 Reached end of journal objects 2024-04-25T10:12:31.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 Events scanned, 0 gaps 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 Found 1 valid events 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 Selected 0 events events for processing 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 4 0 events 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56267e4bac00 msgr2=0x56267d6fac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x56267e4bac00 0x56267d6fac00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56267d879400 msgr2=0x56267d6fb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x56267d879400 0x56267d6fb180 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56267d878800 msgr2=0x56267d6fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x56267d878800 0x56267d6fb700 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x56267e4746f0 tx=0x56267d632180 comp rx=0 tx=0).stop 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 msgr2=0x56267d6f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/959232466 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267e4ba000 0x56267d6f8580 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x56267e4753e0 tx=0x56267e472d20 comp rx=0 tx=0).stop 2024-04-25T10:12:31.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8dcb21640 1 -- 172.21.15.82:0/959232466 reap_dead start 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8dcb21640 1 -- 172.21.15.82:0/959232466 reap_dead start 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 shutdown_connections 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 >> 172.21.15.82:0/959232466 conn(0x56267e30e000 msgr2=0x56267d56b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 shutdown_connections 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/959232466 wait complete. 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 msgr2=0x56267d6f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x56267e3a0bd0 tx=0x56267d6320c0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 shutdown_connections 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56267e30e800 0x56267d6f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56267d53f800 0x56267d6f9b80 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 --2- 172.21.15.82:0/1241517761 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56267d53fc00 0x56267d6f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 >> 172.21.15.82:0/1241517761 conn(0x56267d53ec00 msgr2=0x56267d56c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 shutdown_connections 2024-04-25T10:12:31.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.336+0000 7ff8de362400 1 -- 172.21.15.82:0/1241517761 wait complete. 2024-04-25T10:12:31.348 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.070319, result: Events by type: Errors: 0 2024-04-25T10:12:31.348 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- start start 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 Processor -- start 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 0x5609a8862580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 0x5609a8862b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4180 con 0x5609a86a9000 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4300 con 0x5609a86a9400 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4480 con 0x5609a86a9800 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56366/0 (socket says 172.21.15.82:56366) 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 -- 172.21.15.82:0/3603882288 learned_addr learned my addr 172.21.15.82:0/3603882288 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 0x5609a8862b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562997640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 0x5609a8862580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 -- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 msgr2=0x5609a8862580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 0x5609a8862580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 -- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 msgr2=0x5609a8862b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 0x5609a8862b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 -- 172.21.15.82:0/3603882288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a9456960 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 0x5609a8862b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562997640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 0x5609a8862580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3563198640 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x5609a9509350 tx=0x5609a876b020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3c19e73ea4ca56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3560192640 1 -- 172.21.15.82:0/3603882288 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86de4e0 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3560192640 1 -- 172.21.15.82:0/3603882288 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609a86f2e00 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3560192640 1 -- 172.21.15.82:0/3603882288 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86deb60 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a9456d20 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5609a9456f00 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 4 waiting for MDS map... 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3560192640 1 -- 172.21.15.82:0/3603882288 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x5609a95b6600 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3560192640 1 -- 172.21.15.82:0/3603882288 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 76) v1 ==== 2723+0+0 (secure 0 0 0) 0x5609a86ff180 con 0x5609a86a9000 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 4 Got MDS map 76 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 10 main: JournalTool::main 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 Processor -- start 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- start start 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bcc00 0x5609a8863600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd000 0x5609a8863b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4600 con 0x5609a95bd400 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4780 con 0x5609a95bd000 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a4900 con 0x5609a95bcc00 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56376/0 (socket says 172.21.15.82:56376) 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 -- 172.21.15.82:0/188927610 learned_addr learned my addr 172.21.15.82:0/188927610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562997640 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd000 0x5609a8863b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 -- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bcc00 msgr2=0x5609a8863600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bcc00 0x5609a8863600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 -- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd000 msgr2=0x5609a8863b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd000 0x5609a8863b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 -- 172.21.15.82:0/188927610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a9456d20 con 0x5609a95bd400 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f3562196640 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x5609a95ca6c0 tx=0x5609a95cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4625b98414d77ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f355e18e640 1 -- 172.21.15.82:0/188927610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86de4e0 con 0x5609a95bd400 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f355e18e640 1 -- 172.21.15.82:0/188927610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609a86f2540 con 0x5609a95bd400 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.424+0000 7f355e18e640 1 -- 172.21.15.82:0/188927610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86deb60 con 0x5609a95bd400 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/188927610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 msgr2=0x5609a8864100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x5609a95ca6c0 tx=0x5609a95cecc0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/188927610 shutdown_connections 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bd400 0x5609a8864100 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd000 0x5609a8863b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- 172.21.15.82:0/188927610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bcc00 0x5609a8863600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/188927610 >> 172.21.15.82:0/188927610 conn(0x5609a86a8c00 msgr2=0x5609a86d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/188927610 shutdown_connections 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/188927610 wait complete. 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 Processor -- start 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- start start 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 0x5609a8865180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bd000 0x5609a8864c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd400 0x5609a8864680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a5c80 con 0x5609a95bcc00 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a5b00 con 0x5609a95bd400 2024-04-25T10:12:31.440 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5609a95a5980 con 0x5609a95bd000 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 0x5609a8865180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 0x5609a8865180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56382/0 (socket says 172.21.15.82:56382) 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 learned_addr learned my addr 172.21.15.82:0/682505691 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562997640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bd000 0x5609a8864c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bd000 msgr2=0x5609a8864c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a95bd000 0x5609a8864c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd400 msgr2=0x5609a8864680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a95bd400 0x5609a8864680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609a9457c20 con 0x5609a95bcc00 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f3562196640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 0x5609a8865180 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x5609a95cb380 tx=0x5609a95cec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 -- 172.21.15.82:0/682505691 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86de000 con 0x5609a95bcc00 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 -- 172.21.15.82:0/682505691 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5609a86f2c40 con 0x5609a95bcc00 2024-04-25T10:12:31.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5609a9457e00 con 0x5609a95bcc00 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 -- 172.21.15.82:0/682505691 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5609a86de4e0 con 0x5609a95bcc00 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609a962a1e0 con 0x5609a95bcc00 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f35641d8400 4 main: JournalTool: resolving pool 15 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 -- 172.21.15.82:0/682505691 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5609a9606c00 con 0x5609a95bcc00 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5609a962cc00 0x5609a8865700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.428+0000 7f355c98b640 1 -- 172.21.15.82:0/682505691 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x5609a9624400 con 0x5609a95bcc00 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562997640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5609a962cc00 0x5609a8865700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 main: Executing for rank 0 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 Filtering by inode '1099511627778' 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5609a95bd800 0x5609a8864100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x5609a95bdc00 con 0x5609a95bd800 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562997640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5609a962cc00 0x5609a8865700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5609a86f97d0 tx=0x5609a946e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562196640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5609a95bd800 0x5609a8864100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562196640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5609a95bd800 0x5609a8864100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5609a8824000 con 0x5609a95bd800 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 Pointer 400.00000000 is readable 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x5609a962d000 con 0x5609a95bd800 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5609a8824000 con 0x5609a95bd800 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Starting journal scan from offset 0x405f0e 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x5609a962d400 con 0x5609a95bd800 2024-04-25T10:12:31.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3562196640 1 -- 172.21.15.82:0/682505691 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5609a8824000 con 0x5609a95bd800 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Parsing data, 0x37c bytes available 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Attempting decode at 0x405f0e 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Consumed 0x37c bytes 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 10 Valid entry at 0x405f0e 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5609a962d800 0x5609a8863b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x5609a962dc00 con 0x5609a962d800 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3563198640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5609a962d800 0x5609a8863b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.432+0000 7f3563198640 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5609a962d800 0x5609a8863b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f3563198640 1 -- 172.21.15.82:0/682505691 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5609a8824500 con 0x5609a962d800 2024-04-25T10:12:31.445 INFO:teuthology.orchestra.run.smithi082.stdout:Events by type: 2024-04-25T10:12:31.445 INFO:teuthology.orchestra.run.smithi082.stdout:Errors: 0 2024-04-25T10:12:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 Reached end of journal objects 2024-04-25T10:12:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T10:12:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 Events scanned, 0 gaps 2024-04-25T10:12:31.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 Found 1 valid events 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 Selected 0 events events for processing 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 4 0 events 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5609a95bd800 msgr2=0x5609a8864100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5609a95bd800 0x5609a8864100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5609a962d800 msgr2=0x5609a8863b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5609a962d800 0x5609a8863b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5609a962cc00 msgr2=0x5609a8865700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5609a962cc00 0x5609a8865700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5609a86f97d0 tx=0x5609a946e000 comp rx=0 tx=0).stop 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 msgr2=0x5609a8865180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/682505691 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a95bcc00 0x5609a8865180 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x5609a95cb380 tx=0x5609a95cec00 comp rx=0 tx=0).stop 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f3562997640 1 -- 172.21.15.82:0/682505691 reap_dead start 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f3562997640 1 -- 172.21.15.82:0/682505691 reap_dead start 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 shutdown_connections 2024-04-25T10:12:31.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 >> 172.21.15.82:0/682505691 conn(0x5609a86a8c00 msgr2=0x5609a86d7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 shutdown_connections 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/682505691 wait complete. 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 msgr2=0x5609a8863080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x5609a9509350 tx=0x5609a876b020 comp rx=0 tx=0).stop 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 shutdown_connections 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5609a86a9000 0x5609a8863080 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5609a86a9400 0x5609a8862b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 --2- 172.21.15.82:0/3603882288 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5609a86a9800 0x5609a8862580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 >> 172.21.15.82:0/3603882288 conn(0x5609a86a9c00 msgr2=0x5609a86d7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 shutdown_connections 2024-04-25T10:12:31.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.436+0000 7f35641d8400 1 -- 172.21.15.82:0/3603882288 wait complete. 2024-04-25T10:12:31.450 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.101892, result: Events by type: Errors: 0 2024-04-25T10:12:31.450 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T10:12:31.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 Processor -- start 2024-04-25T10:12:31.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- start start 2024-04-25T10:12:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e69b2f0 0x55e94e70b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e710dc0 0x55e94e727830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e680d40 con 0x55e94e69b6d0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e5f89f0 con 0x55e94e69b2f0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e94e680fe0 con 0x55e94e710dc0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56404/0 (socket says 172.21.15.82:56404) 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/3473704968 learned_addr learned my addr 172.21.15.82:0/3473704968 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d4320c640 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e69b2f0 0x55e94e70b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d43a0d640 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e710dc0 0x55e94e727830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e710dc0 msgr2=0x55e94e727830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e710dc0 0x55e94e727830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e69b2f0 msgr2=0x55e94e70b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e69b2f0 0x55e94e70b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/3473704968 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94e7275a0 con 0x55e94e69b6d0 2024-04-25T10:12:31.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800a750 tx=0x7f1d3800ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e60a49b7c2beabb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d41a09640 1 -- 172.21.15.82:0/3473704968 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ae60 con 0x55e94e69b6d0 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d41a09640 1 -- 172.21.15.82:0/3473704968 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d38008c00 con 0x55e94e69b6d0 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d41a09640 1 -- 172.21.15.82:0/3473704968 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380133f0 con 0x55e94e69b6d0 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- 172.21.15.82:0/3473704968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 msgr2=0x55e94e692630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800a750 tx=0x7f1d3800ac10 comp rx=0 tx=0).stop 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- 172.21.15.82:0/3473704968 shutdown_connections 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e710dc0 0x55e94e727830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e692630 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- 172.21.15.82:0/3473704968 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e69b2f0 0x55e94e70b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- 172.21.15.82:0/3473704968 >> 172.21.15.82:0/3473704968 conn(0x55e94e67c230 msgr2=0x55e94e654cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- 172.21.15.82:0/3473704968 shutdown_connections 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- 172.21.15.82:0/3473704968 wait complete. 2024-04-25T10:12:31.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 Processor -- start 2024-04-25T10:12:31.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- start start 2024-04-25T10:12:31.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e69b2f0 0x55e94e698670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e727910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e710dc0 0x55e94e727e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e70fec0 con 0x55e94e69b6d0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e94e70fc20 con 0x55e94e710dc0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d456ee080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e94e710160 con 0x55e94e69b2f0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e727910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e727910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56414/0 (socket says 172.21.15.82:56414) 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/2195792704 learned_addr learned my addr 172.21.15.82:0/2195792704 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d4320c640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e69b2f0 0x55e94e698670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d43a0d640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e710dc0 0x55e94e727e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e69b2f0 msgr2=0x55e94e698670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e94e69b2f0 0x55e94e698670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e710dc0 msgr2=0x55e94e727e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e710dc0 0x55e94e727e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.524+0000 7f1d42a0b640 1 -- 172.21.15.82:0/2195792704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e94e70a240 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d43a0d640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e94e710dc0 0x55e94e727e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d42a0b640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e727910 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1d38013ec0 tx=0x7f1d38003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 -- 172.21.15.82:0/2195792704 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38022020 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 -- 172.21.15.82:0/2195792704 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3801c630 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e94e70a4e0 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 -- 172.21.15.82:0/2195792704 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ae60 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e94e696560 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 -- 172.21.15.82:0/2195792704 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d38025020 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d30065940 0x7f1d30067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d40a07640 1 -- 172.21.15.82:0/2195792704 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f1d38010020 con 0x55e94e69b6d0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d4320c640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d30065940 0x7f1d30067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d456ee080 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e94e6973c0 0x55e94e6977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v9 -- 0x55e94e7087d0 con 0x55e94e6973c0 2024-04-25T10:12:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d43a0d640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e94e6973c0 0x55e94e6977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d4320c640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d30065940 0x7f1d30067e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0ab000 tx=0x7f1d3c0a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.528+0000 7f1d43a0d640 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e94e6973c0 0x55e94e6977a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d43a0d640 1 -- 172.21.15.82:0/2195792704 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f1d3400f3b0 con 0x55e94e6973c0 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e94e6973c0 msgr2=0x55e94e6977a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e94e6973c0 0x55e94e6977a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d30065940 msgr2=0x7f1d30067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d30065940 0x7f1d30067e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0ab000 tx=0x7f1d3c0a5000 comp rx=0 tx=0).stop 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 msgr2=0x55e94e727910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 --2- 172.21.15.82:0/2195792704 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e94e69b6d0 0x55e94e727910 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1d38013ec0 tx=0x7f1d38003030 comp rx=0 tx=0).stop 2024-04-25T10:12:31.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d43a0d640 1 -- 172.21.15.82:0/2195792704 reap_dead start 2024-04-25T10:12:31.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 shutdown_connections 2024-04-25T10:12:31.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 >> 172.21.15.82:0/2195792704 conn(0x55e94e67c230 msgr2=0x55e94e654cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 shutdown_connections 2024-04-25T10:12:31.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.532+0000 7f1d456ee080 1 -- 172.21.15.82:0/2195792704 wait complete. 2024-04-25T10:12:31.539 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:12:31.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 Processor -- start 2024-04-25T10:12:31.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 -- start start 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 0x7f6fbc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc15b800 0x7f6fbc0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6fbc0cfad0 0x7f6fbc15f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cff20 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc930 con 0x7f6fbc15b420 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc425f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cbd20 con 0x7f6fbc15b800 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6fbc0cfad0 0x7f6fbc15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6fbc0cfad0 0x7f6fbc15f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56430/0 (socket says 172.21.15.82:56430) 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 -- 172.21.15.82:0/4027389095 learned_addr learned my addr 172.21.15.82:0/4027389095 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc17fa640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc15b800 0x7f6fbc0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc1ffb640 1 --2- 172.21.15.82:0/4027389095 >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 0x7f6fbc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 -- 172.21.15.82:0/4027389095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc15b800 msgr2=0x7f6fbc0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc15b800 0x7f6fbc0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 -- 172.21.15.82:0/4027389095 >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 msgr2=0x7f6fbc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/4027389095 >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 0x7f6fbc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc27fc640 1 -- 172.21.15.82:0/4027389095 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0cc000 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.705 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.700+0000 7f6fc1ffb640 1 --2- 172.21.15.82:0/4027389095 >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 0x7f6fbc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc0cfad0 0x7f6fbc15f130 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0d0180 tx=0x7f6fb401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fc8a1eebb03ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc0ff9640 1 -- 172.21.15.82:0/4027389095 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40a5580 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc0ff9640 1 -- 172.21.15.82:0/4027389095 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40a5720 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc0ff9640 1 -- 172.21.15.82:0/4027389095 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40a5af0 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/4027389095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc0cfad0 msgr2=0x7f6fbc15f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc0cfad0 0x7f6fbc15f130 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0d0180 tx=0x7f6fb401c700 comp rx=0 tx=0).stop 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/4027389095 shutdown_connections 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc0cfad0 0x7f6fbc15f130 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- 172.21.15.82:0/4027389095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc15b800 0x7f6fbc0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- 172.21.15.82:0/4027389095 >> v2:172.21.15.196:3300/0 conn(0x7f6fbc15b420 0x7f6fbc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/4027389095 >> 172.21.15.82:0/4027389095 conn(0x7f6fbc06ae60 msgr2=0x7f6fbc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/4027389095 shutdown_connections 2024-04-25T10:12:31.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/4027389095 wait complete. 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 Processor -- start 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- start start 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc0cfad0 0x7f6fbc098540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 0x7f6fbc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0caa80 con 0x7f6fbc15b800 2024-04-25T10:12:31.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0ca900 con 0x7f6fbc15b420 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cadb0 con 0x7f6fbc0cfad0 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56432/0 (socket says 172.21.15.82:56432) 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 -- 172.21.15.82:0/291065163 learned_addr learned my addr 172.21.15.82:0/291065163 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc1ffb640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc0cfad0 0x7f6fbc098540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc17fa640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 0x7f6fbc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 -- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc0cfad0 msgr2=0x7f6fbc098540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc0cfad0 0x7f6fbc098540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 -- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 msgr2=0x7f6fbc159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 0x7f6fbc159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 -- 172.21.15.82:0/291065163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc098a80 con 0x7f6fbc15b800 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc17fa640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 0x7f6fbc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc27fc640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40af040 tx=0x7f6fb40194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4009510 con 0x7f6fbc15b800 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb40be070 con 0x7f6fbc15b800 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc165800 con 0x7f6fbc15b800 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.704+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb40b88b0 con 0x7f6fbc15b800 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc0ccfd0 con 0x7f6fbc15b800 2024-04-25T10:12:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc04efa0 con 0x7f6fbc15b800 2024-04-25T10:12:31.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fb40b8a50 con 0x7f6fbc15b800 2024-04-25T10:12:31.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fb27fc640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 0x7f6fa0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f6fb4120310 con 0x7f6fbc15b800 2024-04-25T10:12:31.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fc1ffb640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 0x7f6fa0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.708+0000 7f6fc1ffb640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 0x7f6fa0069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800a9a0 tx=0x7f6fb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.712+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fb40165e0 con 0x7f6fbc15b800 2024-04-25T10:12:31.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:31.908+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6fbc0cf700 con 0x7f6fbc15b800 2024-04-25T10:12:32.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.260+0000 7f6fb27fc640 1 -- 172.21.15.82:0/291065163 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v77) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6fb40f1300 con 0x7f6fbc15b800 2024-04-25T10:12:32.263 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:12:32.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.260+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 msgr2=0x7f6fa0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.260+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 0x7f6fa0069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800a9a0 tx=0x7f6fb8008040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 msgr2=0x7f6fbc15a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f6fb40af040 tx=0x7f6fb40194a0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 shutdown_connections 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6fa0066f60 0x7f6fa0069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6fbc15b800 0x7f6fbc15a050 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6fbc15b420 0x7f6fbc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 --2- 172.21.15.82:0/291065163 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6fbc0cfad0 0x7f6fbc098540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 >> 172.21.15.82:0/291065163 conn(0x7f6fbc06ae60 msgr2=0x7f6fbc0afd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 shutdown_connections 2024-04-25T10:12:32.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.264+0000 7f6fc425f640 1 -- 172.21.15.82:0/291065163 wait complete. 2024-04-25T10:12:32.291 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:32.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 Processor -- start 2024-04-25T10:12:32.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 -- start start 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 0x7f5bbc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 0x7f5bbc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5bbc06ec60 0x7f5bbc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cc2b0 con 0x7f5bbc06ec60 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc06e290 con 0x7f5bbc144ca0 2024-04-25T10:12:32.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bc224e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cee30 con 0x7f5bbc0ce510 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5bbc06ec60 0x7f5bbc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f5bbc06ec60 0x7f5bbc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56438/0 (socket says 172.21.15.82:56438) 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 -- 172.21.15.82:0/913806713 learned_addr learned my addr 172.21.15.82:0/913806713 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbb7fe640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 0x7f5bbc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbaffd640 1 --2- 172.21.15.82:0/913806713 >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 0x7f5bbc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 -- 172.21.15.82:0/913806713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 msgr2=0x7f5bbc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 0x7f5bbc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 -- 172.21.15.82:0/913806713 >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 msgr2=0x7f5bbc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/913806713 >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 0x7f5bbc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbbfff640 1 -- 172.21.15.82:0/913806713 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0ccc70 con 0x7f5bbc06ec60 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbaffd640 1 --2- 172.21.15.82:0/913806713 >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 0x7f5bbc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.408+0000 7f5bbb7fe640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 0x7f5bbc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc06ec60 0x7f5bbc142680 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc06cfe0 tx=0x7f5bb401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaf907395ad39ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bba7fc640 1 -- 172.21.15.82:0/913806713 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40b4070 con 0x7f5bbc06ec60 2024-04-25T10:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bba7fc640 1 -- 172.21.15.82:0/913806713 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb40af400 con 0x7f5bbc06ec60 2024-04-25T10:12:32.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bba7fc640 1 -- 172.21.15.82:0/913806713 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40af7d0 con 0x7f5bbc06ec60 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- 172.21.15.82:0/913806713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc06ec60 msgr2=0x7f5bbc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc06ec60 0x7f5bbc142680 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc06cfe0 tx=0x7f5bb401f1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- 172.21.15.82:0/913806713 shutdown_connections 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc06ec60 0x7f5bbc142680 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- 172.21.15.82:0/913806713 >> v2:172.21.15.196:3300/0 conn(0x7f5bbc144ca0 0x7f5bbc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- 172.21.15.82:0/913806713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc0ce510 0x7f5bbc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- 172.21.15.82:0/913806713 >> 172.21.15.82:0/913806713 conn(0x7f5bbc0b11a0 msgr2=0x7f5bbc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- 172.21.15.82:0/913806713 shutdown_connections 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- 172.21.15.82:0/913806713 wait complete. 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 Processor -- start 2024-04-25T10:12:32.416 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- start start 2024-04-25T10:12:32.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc06ec60 0x7f5bbc1e5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 0x7f5bbc0b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0cc900 con 0x7f5bbc144ca0 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc06df20 con 0x7f5bbc0ce510 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bc224e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1451f0 con 0x7f5bbc06ec60 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56442/0 (socket says 172.21.15.82:56442) 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.412+0000 7f5bbbfff640 1 -- 172.21.15.82:0/572771107 learned_addr learned my addr 172.21.15.82:0/572771107 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbb7fe640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc06ec60 0x7f5bbc1e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbaffd640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 0x7f5bbc0b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 -- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc06ec60 msgr2=0x7f5bbc1e5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc06ec60 0x7f5bbc1e5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 -- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 msgr2=0x7f5bbc0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 0x7f5bbc0b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 -- 172.21.15.82:0/572771107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc141270 con 0x7f5bbc144ca0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbaffd640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 0x7f5bbc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbbfff640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40b2040 tx=0x7f5bb401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40b4040 con 0x7f5bbc144ca0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb40c0070 con 0x7f5bbc144ca0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc141510 con 0x7f5bbc144ca0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4009510 con 0x7f5bbc144ca0 2024-04-25T10:12:32.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc141ad0 con 0x7f5bbc144ca0 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc04efa0 con 0x7f5bbc144ca0 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5bb40c5460 con 0x7f5bbc144ca0 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f5bb4123ff0 con 0x7f5bbc144ca0 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbb7fe640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.416+0000 7f5bbb7fe640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 0x7f5ba8069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc1e5c90 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.420+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5bb4018d40 con 0x7f5bbc144ca0 2024-04-25T10:12:32.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.612+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bbc0cb6b0 con 0x7f5bbc144ca0 2024-04-25T10:12:32.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.612+0000 7f5ba3fff640 1 -- 172.21.15.82:0/572771107 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f5bb40f4fd0 con 0x7f5bbc144ca0 2024-04-25T10:12:32.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:32.617 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":78,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:12:32.268482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"c","rank":0,"incarnation":78,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/603032759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":603032759},{"type":"v1","addr":"172.21.15.82:6837","nonce":603032759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:32.617 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 78 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 msgr2=0x7f5ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 0x7f5ba8069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc1e5c90 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 msgr2=0x7f5bbc15ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40b2040 tx=0x7f5bb401bf80 comp rx=0 tx=0).stop 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 shutdown_connections 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5ba8066f60 0x7f5ba8069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5bbc144ca0 0x7f5bbc15ab50 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5bbc0ce510 0x7f5bbc0b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 --2- 172.21.15.82:0/572771107 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f5bbc06ec60 0x7f5bbc1e5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 >> 172.21.15.82:0/572771107 conn(0x7f5bbc0b11a0 msgr2=0x7f5bbc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 shutdown_connections 2024-04-25T10:12:32.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:32.616+0000 7f5bc224e640 1 -- 172.21.15.82:0/572771107 wait complete. 2024-04-25T10:12:33.039 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:28.528473+0000', 'modified': '2024-04-25T10:12:32.268482+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9177': {'gid': 9177, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/603032759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 603032759}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 603032759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:33.039 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9177:up:replay 2024-04-25T10:12:34.039 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:34.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 Processor -- start 2024-04-25T10:12:34.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 -- start start 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc45c0ce510 0x7fc45c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc45c1451f0 0x7fc45c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45c0cee30 con 0x7fc45c144ca0 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c0cc2b0 con 0x7fc45c1451f0 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc463c04640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45c06e290 con 0x7fc45c0ce510 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc4621a1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc45c1451f0 0x7fc45c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc46119f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc45c144ca0 0x7fc45c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:52290/0 (socket says 172.21.15.82:52290) 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.168+0000 7fc46119f640 1 -- 172.21.15.82:0/67750904 learned_addr learned my addr 172.21.15.82:0/67750904 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:12:34.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4619a0640 1 --2- 172.21.15.82:0/67750904 >> v2:172.21.15.196:3300/0 conn(0x7fc45c0ce510 0x7fc45c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4621a1640 1 -- 172.21.15.82:0/67750904 >> v2:172.21.15.196:3300/0 conn(0x7fc45c0ce510 msgr2=0x7fc45c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4621a1640 1 --2- 172.21.15.82:0/67750904 >> v2:172.21.15.196:3300/0 conn(0x7fc45c0ce510 0x7fc45c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4621a1640 1 -- 172.21.15.82:0/67750904 >> v1:172.21.15.82:6789/0 conn(0x7fc45c144ca0 legacy=0x7fc45c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4621a1640 1 -- 172.21.15.82:0/67750904 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0ccc70 con 0x7fc45c1451f0 2024-04-25T10:12:34.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc46099e640 1 -- 172.21.15.82:0/67750904 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193425830 0 0) 0x7fc45c0cee30 con 0x7fc45c144ca0 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc4621a1640 1 --2- 172.21.15.82:0/67750904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c1451f0 0x7fc45c142680 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fc45c144c50 tx=0x7fc45401f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13b5b0b1ac2bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc46099e640 1 -- 172.21.15.82:0/67750904 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540b4070 con 0x7fc45c1451f0 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc46099e640 1 -- 172.21.15.82:0/67750904 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540af600 con 0x7fc45c1451f0 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc46099e640 1 -- 172.21.15.82:0/67750904 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4540af9d0 con 0x7fc45c1451f0 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 -- 172.21.15.82:0/67750904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c1451f0 msgr2=0x7fc45c142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 --2- 172.21.15.82:0/67750904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c1451f0 0x7fc45c142680 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fc45c144c50 tx=0x7fc45401f110 comp rx=0 tx=0).stop 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 -- 172.21.15.82:0/67750904 shutdown_connections 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 --2- 172.21.15.82:0/67750904 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c1451f0 0x7fc45c142680 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 --2- 172.21.15.82:0/67750904 >> v2:172.21.15.196:3300/0 conn(0x7fc45c0ce510 0x7fc45c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 -- 172.21.15.82:0/67750904 >> 172.21.15.82:0/67750904 conn(0x7fc45c0b11a0 msgr2=0x7fc45c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 -- 172.21.15.82:0/67750904 shutdown_connections 2024-04-25T10:12:34.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.172+0000 7fc463c04640 1 -- 172.21.15.82:0/67750904 wait complete. 2024-04-25T10:12:34.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 Processor -- start 2024-04-25T10:12:34.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- start start 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc45c144ca0 0x7fc45c0aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc45c1451f0 0x7fc45c0aa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c06ee30 con 0x7fc45c0ce510 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c06df20 con 0x7fc45c144ca0 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45c0cc900 con 0x7fc45c1451f0 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56460/0 (socket says 172.21.15.82:56460) 2024-04-25T10:12:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 -- 172.21.15.82:0/4059430095 learned_addr learned my addr 172.21.15.82:0/4059430095 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 -- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc45c1451f0 msgr2=0x7fc45c0aa570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc45c1451f0 0x7fc45c0aa570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 -- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc45c144ca0 msgr2=0x7fc45c0aa030 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc45c144ca0 0x7fc45c0aa030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 -- 172.21.15.82:0/4059430095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c0a8720 con 0x7fc45c0ce510 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4619a0640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d800 tx=0x7fc45800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800b6c0 con 0x7fc45c0ce510 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc458019070 con 0x7fc45c0ce510 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45c0a8a00 con 0x7fc45c0ce510 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45800ba60 con 0x7fc45c0ce510 2024-04-25T10:12:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45c0a8fc0 con 0x7fc45c0ce510 2024-04-25T10:12:34.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45c04efa0 con 0x7fc45c0ce510 2024-04-25T10:12:34.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc45800bc00 con 0x7fc45c0ce510 2024-04-25T10:12:34.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.176+0000 7fc4527fc640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 0x7fc440069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.180+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fc4580856c0 con 0x7fc45c0ce510 2024-04-25T10:12:34.182 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.180+0000 7fc46119f640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 0x7fc440069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.180+0000 7fc46119f640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 0x7fc440069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc44c007fd0 tx=0x7fc44c0078c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.184+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4580564d0 con 0x7fc45c0ce510 2024-04-25T10:12:34.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.376+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc45c0b1cc0 con 0x7fc45c0ce510 2024-04-25T10:12:34.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.376+0000 7fc4527fc640 1 -- 172.21.15.82:0/4059430095 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fc458055dd0 con 0x7fc45c0ce510 2024-04-25T10:12:34.380 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:34.380 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":80,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:12:33.277052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"c","rank":0,"incarnation":78,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/603032759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":603032759},{"type":"v1","addr":"172.21.15.82:6837","nonce":603032759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:34.380 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 80 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 msgr2=0x7fc440069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 0x7fc440069450 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fc44c007fd0 tx=0x7fc44c0078c0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 msgr2=0x7fc45c1e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fc45800d800 tx=0x7fc45800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 shutdown_connections 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc440066f90 0x7fc440069450 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc45c1451f0 0x7fc45c0aa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc45c144ca0 0x7fc45c0aa030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 --2- 172.21.15.82:0/4059430095 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc45c0ce510 0x7fc45c1e6790 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 >> 172.21.15.82:0/4059430095 conn(0x7fc45c0b11a0 msgr2=0x7fc45c0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 shutdown_connections 2024-04-25T10:12:34.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:34.380+0000 7fc463c04640 1 -- 172.21.15.82:0/4059430095 wait complete. 2024-04-25T10:12:34.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:28.528473+0000', 'modified': '2024-04-25T10:12:33.277052+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9177': {'gid': 9177, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/603032759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 603032759}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 603032759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:34.802 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9177:up:rejoin 2024-04-25T10:12:35.804 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 Processor -- start 2024-04-25T10:12:35.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 -- start start 2024-04-25T10:12:35.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f30600b24a0 0x7f30600ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600ae860 con 0x7f3060071740 2024-04-25T10:12:35.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306015e210 con 0x7f306015da10 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f30660a5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306015e4f0 con 0x7f30600b24a0 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3060071740 0x7f30600acbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58002/0 (socket says 172.21.15.82:58002) 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 -- 172.21.15.82:0/4089899264 learned_addr learned my addr 172.21.15.82:0/4089899264 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305ffff640 1 --2- 172.21.15.82:0/4089899264 >> v2:172.21.15.196:3300/0 conn(0x7f30600b24a0 0x7f30600ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361101075 0 0) 0x7f30600ae860 con 0x7f3060071740 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3048003530 con 0x7f3060071740 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1348443247 0 0) 0x7f3048003530 con 0x7f3060071740 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 >> v2:172.21.15.196:3300/0 conn(0x7f30600b24a0 msgr2=0x7f30600ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 --2- 172.21.15.82:0/4089899264 >> v2:172.21.15.196:3300/0 conn(0x7f30600b24a0 0x7f30600ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 >> v1:172.21.15.196:6789/0 conn(0x7f306015da10 legacy=0x7f3060071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30600ae4c0 con 0x7f3060071740 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3054001a00 con 0x7f3060071740 2024-04-25T10:12:35.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 >> v1:172.21.15.82:6789/0 conn(0x7f3060071740 legacy=0x7f30600acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3048003a30 0x7f3048003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f3048004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30480052d0 0x7f304800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015e4f0 con 0x7f3048004810 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304800c210 con 0x7f30480052d0 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304800c410 con 0x7f3048003a30 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f3048004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305ffff640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30480052d0 0x7f304800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305f7fe640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3048003a30 0x7f3048003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 -- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3048003a30 msgr2=0x7f3048003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3048003a30 0x7f3048003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 -- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30480052d0 msgr2=0x7f304800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30480052d0 0x7f304800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305effd640 1 -- 172.21.15.82:0/4089899264 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048003530 con 0x7f3048004810 2024-04-25T10:12:35.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305ffff640 1 -- 172.21.15.82:0/4089899264 reap_dead start 2024-04-25T10:12:35.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.932+0000 7f305f7fe640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3048003a30 0x7f3048003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f305effd640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f3048004c90 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f3060070e10 tx=0x7f305401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b8a94093fda36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305400a4a0 con 0x7f3048004810 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30540ad650 con 0x7f3048004810 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f305e7fc640 1 -- 172.21.15.82:0/4089899264 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30540ada80 con 0x7f3048004810 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 msgr2=0x7f3048004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f3048004c90 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f3060070e10 tx=0x7f305401d320 comp rx=0 tx=0).stop 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- 172.21.15.82:0/4089899264 shutdown_connections 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 --2- 172.21.15.82:0/4089899264 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f3048004c90 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- 172.21.15.82:0/4089899264 >> 172.21.15.82:0/4089899264 conn(0x7f30600b11a0 msgr2=0x7f30600b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- 172.21.15.82:0/4089899264 shutdown_connections 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- 172.21.15.82:0/4089899264 wait complete. 2024-04-25T10:12:35.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 Processor -- start 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.936+0000 7f30660a5640 1 -- start start 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30601567a0 0x7f30600cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3060158a10 0x7f3060158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060161d00 con 0x7f3048004810 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015ef00 con 0x7f30601567a0 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3060161e80 con 0x7f3060158a10 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55936/0 (socket says 172.21.15.82:55936) 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 -- 172.21.15.82:0/3835125406 learned_addr learned my addr 172.21.15.82:0/3835125406 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305effd640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30601567a0 0x7f30600cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305ffff640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3060158a10 0x7f3060158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 -- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3060158a10 msgr2=0x7f3060158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3060158a10 0x7f3060158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 -- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30601567a0 msgr2=0x7f30600cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30601567a0 0x7f30600cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 -- 172.21.15.82:0/3835125406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30601549d0 con 0x7f3048004810 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305f7fe640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f305800d4a0 tx=0x7f305800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058005280 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3058005440 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058007260 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3060154cb0 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3060155270 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306004efa0 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3058019020 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 0x7f304c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f3058084860 con 0x7f3048004810 2024-04-25T10:12:35.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305effd640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 0x7f304c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.940+0000 7f305effd640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 0x7f304c069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f30540ad000 tx=0x7f30540a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:35.944+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3058055700 con 0x7f3048004810 2024-04-25T10:12:36.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.132+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30600b1dc0 con 0x7f3048004810 2024-04-25T10:12:36.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.132+0000 7f3047fff640 1 -- 172.21.15.82:0/3835125406 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f3058055000 con 0x7f3048004810 2024-04-25T10:12:36.136 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.136 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:12:34.285349+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9177},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9177":{"gid":9177,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/603032759","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":603032759},{"type":"v1","addr":"172.21.15.82:6837","nonce":603032759}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:36.136 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 81 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 msgr2=0x7f304c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 0x7f304c069450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f30540ad000 tx=0x7f30540a7000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 msgr2=0x7f30601543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f305800d4a0 tx=0x7f305800d960 comp rx=0 tx=0).stop 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 shutdown_connections 2024-04-25T10:12:36.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f304c066f90 0x7f304c069450 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3060158a10 0x7f3060158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f30601567a0 0x7f30600cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 --2- 172.21.15.82:0/3835125406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3048004810 0x7f30601543c0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 >> 172.21.15.82:0/3835125406 conn(0x7f30600b11a0 msgr2=0x7f30600acb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 shutdown_connections 2024-04-25T10:12:36.139 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.136+0000 7f30660a5640 1 -- 172.21.15.82:0/3835125406 wait complete. 2024-04-25T10:12:36.556 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:28.528473+0000', 'modified': '2024-04-25T10:12:34.285349+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 110, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9177}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9177': {'gid': 9177, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/603032759', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6836', 'nonce': 603032759}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 603032759}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:36.557 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:36.557 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:12:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 Processor -- start 2024-04-25T10:12:36.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 -- start start 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a800ce510 0x7f2a800ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a80144ca0 0x7f2a8006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a800cee30 con 0x7f2a8006ec60 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a800cc2b0 con 0x7f2a80144ca0 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a8796a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8006e290 con 0x7f2a800ce510 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a80144ca0 0x7f2a8006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a80144ca0 0x7f2a8006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55938/0 (socket says 172.21.15.82:55938) 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a85f07640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2a8006ec60 0x7f2a80142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58016/0 (socket says 172.21.15.82:58016) 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 -- 172.21.15.82:0/3687895112 learned_addr learned my addr 172.21.15.82:0/3687895112 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a85706640 1 --2- 172.21.15.82:0/3687895112 >> v2:172.21.15.196:3300/0 conn(0x7f2a800ce510 0x7f2a800ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 -- 172.21.15.82:0/3687895112 >> v2:172.21.15.196:3300/0 conn(0x7f2a800ce510 msgr2=0x7f2a800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 --2- 172.21.15.82:0/3687895112 >> v2:172.21.15.196:3300/0 conn(0x7f2a800ce510 0x7f2a800ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 -- 172.21.15.82:0/3687895112 >> v1:172.21.15.82:6789/0 conn(0x7f2a8006ec60 legacy=0x7f2a80142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a77fff640 1 -- 172.21.15.82:0/3687895112 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536557894 0 0) 0x7f2a800cee30 con 0x7f2a8006ec60 2024-04-25T10:12:36.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.688+0000 7f2a84f05640 1 -- 172.21.15.82:0/3687895112 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800ccc70 con 0x7f2a80144ca0 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a84f05640 1 --2- 172.21.15.82:0/3687895112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a80144ca0 0x7f2a8006e720 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f2a80144c50 tx=0x7f2a7001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5295e41e9337a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a77fff640 1 -- 172.21.15.82:0/3687895112 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b5070 con 0x7f2a80144ca0 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a77fff640 1 -- 172.21.15.82:0/3687895112 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a700a8a10 con 0x7f2a80144ca0 2024-04-25T10:12:36.695 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a77fff640 1 -- 172.21.15.82:0/3687895112 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700a8de0 con 0x7f2a80144ca0 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- 172.21.15.82:0/3687895112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a80144ca0 msgr2=0x7f2a8006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 --2- 172.21.15.82:0/3687895112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a80144ca0 0x7f2a8006e720 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f2a80144c50 tx=0x7f2a7001ed10 comp rx=0 tx=0).stop 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- 172.21.15.82:0/3687895112 shutdown_connections 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 --2- 172.21.15.82:0/3687895112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a80144ca0 0x7f2a8006e720 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 --2- 172.21.15.82:0/3687895112 >> v2:172.21.15.196:3300/0 conn(0x7f2a800ce510 0x7f2a800ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- 172.21.15.82:0/3687895112 >> 172.21.15.82:0/3687895112 conn(0x7f2a800b11a0 msgr2=0x7f2a800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- 172.21.15.82:0/3687895112 shutdown_connections 2024-04-25T10:12:36.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- 172.21.15.82:0/3687895112 wait complete. 2024-04-25T10:12:36.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 Processor -- start 2024-04-25T10:12:36.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.692+0000 7f2a8796a640 1 -- start start 2024-04-25T10:12:36.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800ce510 0x7f2a800ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144ca0 0x7f2a80140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a800cc900 con 0x7f2a8006ec60 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8006df20 con 0x7f2a80144ca0 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a80142d90 con 0x7f2a800ce510 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55940/0 (socket says 172.21.15.82:55940) 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 -- 172.21.15.82:0/600018897 learned_addr learned my addr 172.21.15.82:0/600018897 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 -- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800ce510 msgr2=0x7f2a800ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800ce510 0x7f2a800ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 -- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144ca0 msgr2=0x7f2a80140c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144ca0 0x7f2a80140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 -- 172.21.15.82:0/600018897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a800ad2c0 con 0x7f2a8006ec60 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a85706640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d800 tx=0x7f2a7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 -- 172.21.15.82:0/600018897 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00b6c0 con 0x7f2a8006ec60 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 -- 172.21.15.82:0/600018897 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c019070 con 0x7f2a8006ec60 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a800ad5a0 con 0x7f2a8006ec60 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 -- 172.21.15.82:0/600018897 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00ba60 con 0x7f2a8006ec60 2024-04-25T10:12:36.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a800adb60 con 0x7f2a8006ec60 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 -- 172.21.15.82:0/600018897 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a7c00bc00 con 0x7f2a8006ec60 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a84f05640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a75ffb640 1 -- 172.21.15.82:0/600018897 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f2a7c085790 con 0x7f2a8006ec60 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a84f05640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2a700b2000 tx=0x7f2a70004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.696+0000 7f2a8aad7000 1 Processor -- start 2024-04-25T10:12:36.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- start start 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff6240 0x55b7d8fab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8fac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8e935f0 con 0x55b7d8ff5920 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8dfada0 con 0x55b7d8ff6240 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8d83150 con 0x55b7d8fabf30 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55942/0 (socket says 172.21.15.82:55942) 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 -- 172.21.15.82:0/4019111676 learned_addr learned my addr 172.21.15.82:0/4019111676 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85f07640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8fac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 -- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 msgr2=0x55b7d8fac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a84f05640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff6240 0x55b7d8fab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8fac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 -- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff6240 msgr2=0x55b7d8fab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff6240 0x55b7d8fab9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 -- 172.21.15.82:0/4019111676 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7d8d9cca0 con 0x55b7d8ff5920 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85f07640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8fac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85706640 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c05dfe0 tx=0x7f2a7c055a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6ac2bdb686fa95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a6affd640 1 -- 172.21.15.82:0/4019111676 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c05e510 con 0x55b7d8ff5920 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a6affd640 1 -- 172.21.15.82:0/4019111676 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c053ea0 con 0x55b7d8ff5920 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a6affd640 1 -- 172.21.15.82:0/4019111676 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c05c0c0 con 0x55b7d8ff5920 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- 172.21.15.82:0/4019111676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 msgr2=0x55b7d8ff5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c05dfe0 tx=0x7f2a7c055a70 comp rx=0 tx=0).stop 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- 172.21.15.82:0/4019111676 shutdown_connections 2024-04-25T10:12:36.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8fac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff6240 0x55b7d8fab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/4019111676 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d8ff5d00 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- 172.21.15.82:0/4019111676 >> 172.21.15.82:0/4019111676 conn(0x55b7d8fb22a0 msgr2=0x55b7d8faa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- 172.21.15.82:0/4019111676 shutdown_connections 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- 172.21.15.82:0/4019111676 wait complete. 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- start start 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 Processor -- start 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8ffcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8ffd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8db5990 con 0x55b7d8ff6240 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8df6e40 con 0x55b7d8ff5920 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8d86900 con 0x55b7d8fabf30 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a8aad7000 10 client.0 _ll_register_callbacks cb 0x55b7d8eae1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85f07640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85f07640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55952/0 (socket says 172.21.15.82:55952) 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.700+0000 7f2a85f07640 1 -- 172.21.15.82:0/3464049457 learned_addr learned my addr 172.21.15.82:0/3464049457 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85706640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8ffcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a84f05640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8ffd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 -- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 msgr2=0x55b7d8ffcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8ffcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 -- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 msgr2=0x55b7d8ffd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8ffd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 -- 172.21.15.82:0/3464049457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7d8db3630 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d430 tx=0x7f2a7800d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c2d8a3ab2f0227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a780070d0 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f2a78013c80 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a7c053ea0 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a78007270 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 10 client.9252 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b7d8d833a0 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c053ea0 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a78007410 con 0x55b7d8ff6240 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 10 client.9252 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:36.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 10 client.9252 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 20 client.9252 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d8fb1160 0x55b7d9006ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 4 client.9252 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7d8d96a50 con 0x55b7d8fb1160 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85706640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d8fb1160 0x55b7d9006ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 10 client.9252 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a85706640 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d8fb1160 0x55b7d9006ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e8fc5ae67b9de21 server_cookie=6146b6a34de7e024 in_seq=0 out_seq=0 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 1 -- 172.21.15.82:0/3464049457 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b7d8d96a50 con 0x55b7d8fb1160 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 10 client.9252 handle_command_reply: tid=0 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 client.9252 shutdown 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 msgr2=0x55b7d8fafaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800d430 tx=0x7f2a7800d8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 shutdown_connections 2024-04-25T10:12:36.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d8fb1160 0x55b7d9006ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fafaa0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8ffd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a5ffff640 0 client.9252 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3464049457 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8ffcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 >> 172.21.15.82:0/3464049457 conn(0x55b7d8fb22a0 msgr2=0x55b7d8faa9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 shutdown_connections 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3464049457 wait complete. 2024-04-25T10:12:36.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.704+0000 7f2a8aad7000 20 client.9252 trim_cache size 0 max 16384 2024-04-25T10:12:36.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 Processor -- start 2024-04-25T10:12:36.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- start start 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8faf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8faf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801ff500 con 0x55b7d8ff6240 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801526c0 con 0x55b7d8ff5920 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a801fef00 con 0x55b7d8fabf30 2024-04-25T10:12:36.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55962/0 (socket says 172.21.15.82:55962) 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 -- 172.21.15.82:0/3269292535 learned_addr learned my addr 172.21.15.82:0/3269292535 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a84f05640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8faf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85706640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8faf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 -- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 msgr2=0x55b7d8faf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8faf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 -- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 msgr2=0x55b7d8faf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8faf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 -- 172.21.15.82:0/3269292535 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7d8ff6070 con 0x55b7d8ff6240 2024-04-25T10:12:36.732 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a84f05640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8faf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85706640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8faf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a85f07640 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800f010 tx=0x7f2a78006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e5ba7f4ad8eca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a6a7fc640 1 -- 172.21.15.82:0/3269292535 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78022040 con 0x55b7d8ff6240 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a6a7fc640 1 -- 172.21.15.82:0/3269292535 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7802b070 con 0x55b7d8ff6240 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a6a7fc640 1 -- 172.21.15.82:0/3269292535 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78007410 con 0x55b7d8ff6240 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3269292535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 msgr2=0x55b7d8fff8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800f010 tx=0x7f2a78006000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3269292535 shutdown_connections 2024-04-25T10:12:36.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff6240 0x55b7d8fff8b0 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8ff5920 0x55b7d8faf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/3269292535 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8fabf30 0x55b7d8faf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3269292535 >> 172.21.15.82:0/3269292535 conn(0x55b7d8fb22a0 msgr2=0x55b7d8fe2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3269292535 shutdown_connections 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- 172.21.15.82:0/3269292535 wait complete. 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 -- start start 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.720+0000 7f2a8aad7000 1 Processor -- start 2024-04-25T10:12:36.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8fabf30 0x55b7d90398d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8ff6240 0x55b7d903a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8dff3d0 con 0x55b7d8ff5920 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8eb6d50 con 0x55b7d8fabf30 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7d8d95980 con 0x55b7d8ff6240 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 10 client.0 _ll_register_callbacks cb 0x7f2a7c0520a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55966/0 (socket says 172.21.15.82:55966) 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 -- 172.21.15.82:0/2558142831 learned_addr learned my addr 172.21.15.82:0/2558142831 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a85706640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8fabf30 0x55b7d90398d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a85f07640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8ff6240 0x55b7d903a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 -- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8ff6240 msgr2=0x55b7d903a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8ff6240 0x55b7d903a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 -- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8fabf30 msgr2=0x55b7d90398d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8fabf30 0x55b7d90398d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 -- 172.21.15.82:0/2558142831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7d8e7c600 con 0x55b7d8ff5920 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a84f05640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001ed50 tx=0x7f2a7001d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3663ba26c1167f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:36.736 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b5070 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f2a700bb8d0 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a7802b070 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a700bbad0 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 10 client.9258 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b7d8d8f9b0 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7802b070 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2a700bbc70 con 0x55b7d8ff5920 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 10 client.9258 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 10 client.9258 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:12:36.737 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 20 client.9258 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d903acb0 0x55b7d90025e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 4 client.9258 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b7d8e082e0 con 0x55b7d903acb0 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a85706640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d903acb0 0x55b7d90025e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 10 client.9258 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a85706640 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d903acb0 0x55b7d90025e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4abf87d09dde470a server_cookie=e17723b423b92770 in_seq=0 out_seq=0 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 1 -- 172.21.15.82:0/2558142831 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55b7d8e082e0 con 0x55b7d903acb0 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 10 client.9258 handle_command_reply: tid=0 2024-04-25T10:12:36.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 client.9258 shutdown 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 msgr2=0x55b7d903a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001ed50 tx=0x7f2a7001d550 comp rx=0 tx=0).stop 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 shutdown_connections 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55b7d903acb0 0x55b7d90025e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55b7d8ff6240 0x55b7d903a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55b7d8ff5920 0x55b7d903a230 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 --2- 172.21.15.82:0/2558142831 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55b7d8fabf30 0x55b7d90398d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a5ffff640 0 client.9258 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 >> 172.21.15.82:0/2558142831 conn(0x55b7d8fb22a0 msgr2=0x55b7d8fae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 shutdown_connections 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 1 -- 172.21.15.82:0/2558142831 wait complete. 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8aad7000 20 client.9258 trim_cache size 0 max 16384 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:12:36.740 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9177, 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4554264300000002, 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 81, 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 110, 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 110, 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 5.4057822040000003 2024-04-25T10:12:36.741 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 msgr2=0x7f2a64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 0x7f2a64069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f2a700b2000 tx=0x7f2a70004740 comp rx=0 tx=0).stop 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 msgr2=0x7f2a801f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00d800 tx=0x7f2a7c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 shutdown_connections 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a64066f60 0x7f2a64069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a800ce510 0x7f2a800ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a80144ca0 0x7f2a80140c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 --2- 172.21.15.82:0/600018897 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a8006ec60 0x7f2a801f2e30 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 >> 172.21.15.82:0/600018897 conn(0x7f2a800b11a0 msgr2=0x7f2a800af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 shutdown_connections 2024-04-25T10:12:36.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.724+0000 7f2a8796a640 1 -- 172.21.15.82:0/600018897 wait complete. 2024-04-25T10:12:36.745 DEBUG:teuthology.orchestra.run.smithi082:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T10:12:36.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:36.747 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T10:12:36.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 Processor -- start 2024-04-25T10:12:36.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 -- start start 2024-04-25T10:12:36.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 0x7fdd580cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5809a800 con 0x7fdd5815dca0 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5806c290 con 0x7fdd5809aab0 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd5ca11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5806bfb0 con 0x7fdd5806cdc0 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd56d9d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 0x7fdd580cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:57116/0 (socket says 172.21.15.82:57116) 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 -- 172.21.15.82:0/355164558 learned_addr learned my addr 172.21.15.82:0/355164558 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 -- 172.21.15.82:0/355164558 >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 msgr2=0x7fdd580cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 --2- 172.21.15.82:0/355164558 >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 0x7fdd580cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 -- 172.21.15.82:0/355164558 >> v1:172.21.15.196:6789/0 conn(0x7fdd5815dca0 legacy=0x7fdd5806c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd55d9b640 1 -- 172.21.15.82:0/355164558 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5809a520 con 0x7fdd5806cdc0 2024-04-25T10:12:36.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.856+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/355164558 >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 0x7fdd580cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:36.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd55d9b640 1 --2- 172.21.15.82:0/355164558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009870 tx=0x7fdd4801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2c5908f2618b115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5559a640 1 -- 172.21.15.82:0/355164558 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480b1070 con 0x7fdd5806cdc0 2024-04-25T10:12:36.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5559a640 1 -- 172.21.15.82:0/355164558 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480ac620 con 0x7fdd5806cdc0 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5559a640 1 -- 172.21.15.82:0/355164558 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480ac9f0 con 0x7fdd5806cdc0 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- 172.21.15.82:0/355164558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 msgr2=0x7fdd580cf700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/355164558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009870 tx=0x7fdd4801c4d0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- 172.21.15.82:0/355164558 shutdown_connections 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/355164558 >> v2:172.21.15.196:3300/0 conn(0x7fdd5809aab0 0x7fdd580cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/355164558 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd580cf700 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- 172.21.15.82:0/355164558 >> 172.21.15.82:0/355164558 conn(0x7fdd580b2fb0 msgr2=0x7fdd580b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- 172.21.15.82:0/355164558 shutdown_connections 2024-04-25T10:12:36.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- 172.21.15.82:0/355164558 wait complete. 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 Processor -- start 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.860+0000 7fdd5ca11640 1 -- start start 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd58070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd5809aab0 0x7fdd5814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5816a9f0 con 0x7fdd5815dca0 2024-04-25T10:12:36.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5816a870 con 0x7fdd5809aab0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5816ab70 con 0x7fdd5806cdc0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55972/0 (socket says 172.21.15.82:55972) 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3653668673 learned_addr learned my addr 172.21.15.82:0/3653668673 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd58070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd55d9b640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd5809aab0 0x7fdd5814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 msgr2=0x7fdd58070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd58070640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd5809aab0 msgr2=0x7fdd5814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd5809aab0 0x7fdd5814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3653668673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5814cfa0 con 0x7fdd5815dca0 2024-04-25T10:12:36.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd58070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000efc0 tx=0x7fdd5000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 -- 172.21.15.82:0/3653668673 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50018030 con 0x7fdd5815dca0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 -- 172.21.15.82:0/3653668673 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5000cdf0 con 0x7fdd5815dca0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd580cabf0 con 0x7fdd5815dca0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 -- 172.21.15.82:0/3653668673 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50003bc0 con 0x7fdd5815dca0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd580cb1b0 con 0x7fdd5815dca0 2024-04-25T10:12:36.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 -- 172.21.15.82:0/3653668673 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdd5001b070 con 0x7fdd5815dca0 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 0x7fdd38069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd46ffd640 1 -- 172.21.15.82:0/3653668673 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fdd50085bf0 con 0x7fdd5815dca0 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 0x7fdd38069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.864+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 0x7fdd38069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdd581587e0 tx=0x7fdd4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 Processor -- start 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- start start 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb6a5b0 0x55de1cb1fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb20290 0x55de1cb20670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1ca06f40 con 0x55de1cb69c90 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1c96e070 con 0x55de1cb6a5b0 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de1c8f61a0 con 0x55de1cb20290 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55988/0 (socket says 172.21.15.82:55988) 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 -- 172.21.15.82:0/2916280202 learned_addr learned my addr 172.21.15.82:0/2916280202 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd55d9b640 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb6a5b0 0x55de1cb1fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb20290 0x55de1cb20670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 -- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb20290 msgr2=0x55de1cb20670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb20290 0x55de1cb20670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 -- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb6a5b0 msgr2=0x55de1cb1fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb6a5b0 0x55de1cb1fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 -- 172.21.15.82:0/2916280202 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1c8f4510 con 0x55de1cb69c90 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c001e70 tx=0x7fdd4c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d43b5191b40d2bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd37fff640 1 -- 172.21.15.82:0/2916280202 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003600 con 0x55de1cb69c90 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd37fff640 1 -- 172.21.15.82:0/2916280202 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0037a0 con 0x55de1cb69c90 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd37fff640 1 -- 172.21.15.82:0/2916280202 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003b50 con 0x55de1cb69c90 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2916280202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 msgr2=0x55de1cb6a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c001e70 tx=0x7fdd4c011000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2916280202 shutdown_connections 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb20290 0x55de1cb20670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb6a5b0 0x55de1cb1fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2916280202 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb69c90 0x55de1cb6a070 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2916280202 >> 172.21.15.82:0/2916280202 conn(0x55de1cb1c240 msgr2=0x55de1cb1e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2916280202 shutdown_connections 2024-04-25T10:12:36.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2916280202 wait complete. 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 -- start start 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.868+0000 7fdd5fb7e000 1 Processor -- start 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cb2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cb2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1c9286c0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1c96a110 con 0x55de1cb20290 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de1cb1c0c0 con 0x55de1cb69c90 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 10 client.0 _ll_register_callbacks cb 0x55de1cb0b570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:55994/0 (socket says 172.21.15.82:55994) 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 -- 172.21.15.82:0/1887430252 learned_addr learned my addr 172.21.15.82:0/1887430252 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd55d9b640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cb2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5659c640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cb2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 -- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 msgr2=0x55de1cb2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cb2b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 -- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 msgr2=0x55de1cb2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cb2ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 -- 172.21.15.82:0/1887430252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de1c8f63b0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c990 tx=0x7fdd50056ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed95497391c11bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd500556d0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fdd50055870 con 0x55de1cb6a5b0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd4c0037a0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd5005c360 con 0x55de1cb6a5b0 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 10 client.9267 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:36.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55de1c9efe00 con 0x55de1cb6a5b0 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0037a0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd5005c040 con 0x55de1cb6a5b0 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 10 client.9267 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 10 client.9267 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 20 client.9267 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb2c180 0x55de1cb7b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 4 client.9267 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de1c909d60 con 0x55de1cb2c180 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5659c640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb2c180 0x55de1cb7b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5659c640 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb2c180 0x55de1cb7b410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa3748f57e890bd server_cookie=6df54dd1cfbcadbd in_seq=0 out_seq=0 2024-04-25T10:12:36.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 10 client.9267 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 Processor -- start 2024-04-25T10:12:36.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 -- start start 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 1 -- 172.21.15.82:0/1887430252 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55de1c909d60 con 0x55de1cb2c180 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 10 client.9267 handle_command_reply: tid=0 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 client.9267 shutdown 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 msgr2=0x55de1cb2b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000c990 tx=0x7fdd50056ac0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 shutdown_connections 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb2c180 0x55de1cb7b410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cb2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb2b8b0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd357fa640 0 client.9267 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/1887430252 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cb2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 >> 172.21.15.82:0/1887430252 conn(0x55de1cb1c240 msgr2=0x55de1cb1eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 shutdown_connections 2024-04-25T10:12:36.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/1887430252 wait complete. 2024-04-25T10:12:36.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7fdd5fb7e000 20 client.9267 trim_cache size 0 max 16384 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8c040cb9f0 0x7f8c040cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040cc020 con 0x7f8c040cf140 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040ce120 con 0x7f8c0415a700 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c08f0e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040cc300 con 0x7f8c040cb9f0 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c02d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8c040cf140 0x7f8c040ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:58030/0 (socket says 172.21.15.82:58030) 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.872+0000 7f8c02d9d640 1 -- 172.21.15.82:0/3659920447 learned_addr learned my addr 172.21.15.82:0/3659920447 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229682047 0 0) 0x7f8c040cc020 con 0x7f8c040cf140 2024-04-25T10:12:36.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8bfbfff640 1 --2- 172.21.15.82:0/3659920447 >> v2:172.21.15.196:3300/0 conn(0x7f8c040cb9f0 0x7f8c040cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bf0005740 con 0x7f8c040cf140 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055245398 0 0) 0x7f8bf0005740 con 0x7f8c040cf140 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 >> v2:172.21.15.196:3300/0 conn(0x7f8c040cb9f0 msgr2=0x7f8c040cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 --2- 172.21.15.82:0/3659920447 >> v2:172.21.15.196:3300/0 conn(0x7f8c040cb9f0 0x7f8c040cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 >> v1:172.21.15.196:6789/0 conn(0x7f8c0415a700 legacy=0x7f8c040cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040ce400 con 0x7f8c040cf140 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8bfc001a00 con 0x7f8c040cf140 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 >> v1:172.21.15.82:6789/0 conn(0x7f8c040cf140 legacy=0x7f8c040ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 0x7f8bf0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bf0006a20 0x7f8bf0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0007500 0x7f8bf000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cc300 con 0x7f8bf0005bd0 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf000e3d0 con 0x7f8bf0007500 2024-04-25T10:12:36.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf000e5d0 con 0x7f8bf0006a20 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 0x7f8bf0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c02d9d640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0007500 0x7f8bf000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8bfbfff640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bf0006a20 0x7f8bf0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 -- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bf0006a20 msgr2=0x7f8bf0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8bf0006a20 0x7f8bf0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 -- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0007500 msgr2=0x7f8bf000dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0007500 0x7f8bf000dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c02d9d640 1 -- 172.21.15.82:0/3659920447 reap_dead start 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 -- 172.21.15.82:0/3659920447 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf0005740 con 0x7f8bf0005bd0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c0259c640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 0x7f8bf0005ff0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4001750 tx=0x7f8bf401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24073e8a1f4d273 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40013a0 con 0x7f8bf0005bd0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40052f0 con 0x7f8bf0005bd0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c01d9b640 1 -- 172.21.15.82:0/3659920447 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4003840 con 0x7f8bf0005bd0 2024-04-25T10:12:36.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 -- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 msgr2=0x7f8bf0005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 0x7f8bf0005ff0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4001750 tx=0x7f8bf401c390 comp rx=0 tx=0).stop 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 -- 172.21.15.82:0/3659920447 shutdown_connections 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 --2- 172.21.15.82:0/3659920447 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8bf0005bd0 0x7f8bf0005ff0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 -- 172.21.15.82:0/3659920447 >> 172.21.15.82:0/3659920447 conn(0x7f8c040d0420 msgr2=0x7f8c040d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.876+0000 7f8c08f0e640 1 -- 172.21.15.82:0/3659920447 shutdown_connections 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 -- 172.21.15.82:0/3659920447 wait complete. 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 Processor -- start 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 -- start start 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0005bd0 0x7f8c041f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c040c6840 0x7f8c040b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c04157210 0x7f8c04157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cdc10 con 0x7f8c040c6840 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cbdd0 con 0x7f8bf0005bd0 2024-04-25T10:12:36.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c08f0e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040cdd90 con 0x7f8c04157210 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c040c6840 0x7f8c040b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c040c6840 0x7f8c040b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56022/0 (socket says 172.21.15.82:56022) 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 -- 172.21.15.82:0/637392623 learned_addr learned my addr 172.21.15.82:0/637392623 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c02d9d640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c04157210 0x7f8c04157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8c0259c640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0005bd0 0x7f8c041f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 -- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c04157210 msgr2=0x7f8c04157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8c04157210 0x7f8c04157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 -- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0005bd0 msgr2=0x7f8c041f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8bf0005bd0 0x7f8c041f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 -- 172.21.15.82:0/637392623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040b35a0 con 0x7f8c040c6840 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.880+0000 7f8bfbfff640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8c040c6840 0x7f8c040b4640 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f8bec00f630 tx=0x7f8bec00faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec019020 con 0x7f8c040c6840 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec014870 con 0x7f8c040c6840 2024-04-25T10:12:36.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec014c60 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c08f0e640 1 -- 172.21.15.82:0/637392623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c040b3880 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c08f0e640 1 -- 172.21.15.82:0/637392623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c040b3e40 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c08f0e640 1 -- 172.21.15.82:0/637392623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8c04068340 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bec01c020 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8be4066f90 0x7f8be4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f8bec086040 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c0259c640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8be4066f90 0x7f8be4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+516 (secure 0 0 0) 0x7f8bec023020 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c08f0e640 1 -- 172.21.15.82:0/637392623 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f8c040b63a0 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 7 ==== log(2 entries from seq 1257 at 2024-04-25T10:12:35.281979+0000) v1 ==== 389+0+0 (secure 0 0 0) 0x7f8bec084480 con 0x7f8c040c6840 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.884+0000 7f8c0259c640 1 --2- 172.21.15.82:0/637392623 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8be4066f90 0x7f8be4069450 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00069a0 tx=0x7f8bf40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.896 INFO:teuthology.orchestra.run.smithi082.stdout: cluster: 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: id: 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: health: HEALTH_OK 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: services: 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: mgr: y(active, since 10m), standbys: x 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: data: 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: volumes: 1/1 healthy 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: pools: 3 pools, 97 pgs 2024-04-25T10:12:36.897 INFO:teuthology.orchestra.run.smithi082.stdout: objects: 30 objects, 19 MiB 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: pgs: 97 active+clean 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: io: 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:35.281979+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T10:12:36.898 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:35.282020+0000 mon.a [INF] Cluster is now healthy 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 Processor -- start 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- start start 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb69c90 0x55de1cb7d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb6a5b0 0x55de1cbcb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58203b10 con 0x55de1cb20290 2024-04-25T10:12:36.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58153d50 con 0x55de1cb69c90 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58159d50 con 0x55de1cb6a5b0 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56038/0 (socket says 172.21.15.82:56038) 2024-04-25T10:12:36.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 -- 172.21.15.82:0/2516640634 learned_addr learned my addr 172.21.15.82:0/2516640634 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 -- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb6a5b0 msgr2=0x55de1cbcb2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb6a5b0 0x55de1cbcb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 -- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb69c90 msgr2=0x55de1cb7d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb69c90 0x55de1cb7d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 -- 172.21.15.82:0/2516640634 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de1c9c5c50 con 0x55de1cb20290 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5659c640 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c014000 tx=0x7fdd4c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fa81cf76d917d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd377fe640 1 -- 172.21.15.82:0/2516640634 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c003a50 con 0x55de1cb20290 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd377fe640 1 -- 172.21.15.82:0/2516640634 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c003bf0 con 0x55de1cb20290 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd377fe640 1 -- 172.21.15.82:0/2516640634 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c007030 con 0x55de1cb20290 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2516640634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 msgr2=0x55de1cb7d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c014000 tx=0x7fdd4c005000 comp rx=0 tx=0).stop 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2516640634 shutdown_connections 2024-04-25T10:12:36.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb6a5b0 0x55de1cbcb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb69c90 0x55de1cb7d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/2516640634 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb20290 0x55de1cb7d420 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2516640634 >> 172.21.15.82:0/2516640634 conn(0x55de1cb1c240 msgr2=0x55de1cb24d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2516640634 shutdown_connections 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.892+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/2516640634 wait complete. 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- start start 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 Processor -- start 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cbca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cbca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de1c910bd0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd58153750 con 0x55de1cb20290 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55de1ca24f20 con 0x55de1cb69c90 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 10 client.0 _ll_register_callbacks cb 0x7fdd4c007ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:36.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56054/0 (socket says 172.21.15.82:56054) 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3846195078 learned_addr learned my addr 172.21.15.82:0/3846195078 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 msgr2=0x55de1cbca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cbca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 msgr2=0x55de1cbca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cbca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 -- 172.21.15.82:0/3846195078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de1c936120 con 0x55de1cb6a5b0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd56d9d640 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fdd500526e0 tx=0x7fdd50004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660dd1ac924b924a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50055be0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fdd500521e0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd4c003bf0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd50065050 con 0x55de1cb6a5b0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 10 client.9279 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55de1c915170 con 0x55de1cb6a5b0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c003bf0 con 0x55de1cb6a5b0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdd5005c040 con 0x55de1cb6a5b0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 10 client.9279 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 10 client.9279 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 20 client.9279 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb85970 0x55de1cb85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 4 client.9279 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55de1c97b5b0 con 0x55de1cb85970 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb85970 0x55de1cb85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 10 client.9279 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5659c640 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb85970 0x55de1cb85d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c37092e519ddeb9 server_cookie=c375f4485c1ccdcb in_seq=0 out_seq=0 2024-04-25T10:12:36.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 1 -- 172.21.15.82:0/3846195078 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55de1c97b5b0 con 0x55de1cb85970 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 10 client.9279 handle_command_reply: tid=0 2024-04-25T10:12:36.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 client.9279 shutdown 2024-04-25T10:12:36.910 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:12:36.896+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:12:36.910 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:12:36.896+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 msgr2=0x55de1cb84e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fdd500526e0 tx=0x7fdd50004a50 comp rx=0 tx=0).stop 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 shutdown_connections 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x55de1cb85970 0x55de1cb85d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55de1cb69c90 0x55de1cbca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55de1cb6a5b0 0x55de1cb84e90 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd357fa640 0 client.9279 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 --2- 172.21.15.82:0/3846195078 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55de1cb20290 0x55de1cbca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 >> 172.21.15.82:0/3846195078 conn(0x55de1cb1c240 msgr2=0x55de1cb25430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 shutdown_connections 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 1 -- 172.21.15.82:0/3846195078 wait complete. 2024-04-25T10:12:36.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.896+0000 7fdd5fb7e000 20 client.9279 trim_cache size 0 max 16384 2024-04-25T10:12:36.912 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:36.912 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0, 2024-04-25T10:12:36.912 INFO:teuthology.orchestra.run.smithi082.stdout: "scrub_tag": "e2be0496-12dc-41d7-bcff-93df11b72fe2", 2024-04-25T10:12:36.912 INFO:teuthology.orchestra.run.smithi082.stdout: "mode": "asynchronous" 2024-04-25T10:12:36.912 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:36.912 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:12:36.900+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:12:36.912 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:12:36.900+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 msgr2=0x7fdd38069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 0x7fdd38069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdd581587e0 tx=0x7fdd4c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 msgr2=0x7fdd580c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000efc0 tx=0x7fdd5000c490 comp rx=0 tx=0).stop 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 shutdown_connections 2024-04-25T10:12:36.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fdd38066f90 0x7fdd38069450 secure :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdd581587e0 tx=0x7fdd4c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fdd5815dca0 0x7fdd580c8590 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fdd5809aab0 0x7fdd5814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 --2- 172.21.15.82:0/3653668673 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fdd5806cdc0 0x7fdd58070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 >> 172.21.15.82:0/3653668673 conn(0x7fdd580b2fb0 msgr2=0x7fdd5815c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 shutdown_connections 2024-04-25T10:12:36.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:36.900+0000 7fdd5ca11640 1 -- 172.21.15.82:0/3653668673 wait complete. 2024-04-25T10:12:36.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:36.917 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.024+0000 7fe10600c640 1 Processor -- start 2024-04-25T10:12:37.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.024+0000 7fe10600c640 1 -- start start 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe10600c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 0x7fe1000c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe10600c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe10600c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000c9ef0 con 0x7fe100159fb0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe10600c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe100072780 con 0x7fe100159bd0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe10600c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe100071b70 con 0x7fe1000b28c0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:57180/0 (socket says 172.21.15.82:57180) 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 -- 172.21.15.82:0/4152657971 learned_addr learned my addr 172.21.15.82:0/4152657971 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/4152657971 >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 0x7fe1000c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 -- 172.21.15.82:0/4152657971 >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 msgr2=0x7fe1000c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 --2- 172.21.15.82:0/4152657971 >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 0x7fe1000c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 -- 172.21.15.82:0/4152657971 >> v1:172.21.15.196:6789/0 conn(0x7fe100159fb0 legacy=0x7fe1000b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0fffff640 1 -- 172.21.15.82:0/4152657971 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe100071e50 con 0x7fe1000b28c0 2024-04-25T10:12:37.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.028+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/4152657971 >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 0x7fe1000c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:37.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fffff640 1 --2- 172.21.15.82:0/4152657971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d150 tx=0x7fe0f801ec00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af02fe011efd0cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fe7fc640 1 -- 172.21.15.82:0/4152657971 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800b2d0 con 0x7fe1000b28c0 2024-04-25T10:12:37.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fe7fc640 1 -- 172.21.15.82:0/4152657971 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f800b470 con 0x7fe1000b28c0 2024-04-25T10:12:37.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fe7fc640 1 -- 172.21.15.82:0/4152657971 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80a7c70 con 0x7fe1000b28c0 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- 172.21.15.82:0/4152657971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 msgr2=0x7fe100163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- 172.21.15.82:0/4152657971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe0f800d150 tx=0x7fe0f801ec00 comp rx=0 tx=0).stop 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- 172.21.15.82:0/4152657971 shutdown_connections 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- 172.21.15.82:0/4152657971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100163960 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- 172.21.15.82:0/4152657971 >> v2:172.21.15.196:3300/0 conn(0x7fe100159bd0 0x7fe1000c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- 172.21.15.82:0/4152657971 >> 172.21.15.82:0/4152657971 conn(0x7fe10006ae60 msgr2=0x7fe10006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- 172.21.15.82:0/4152657971 shutdown_connections 2024-04-25T10:12:37.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- 172.21.15.82:0/4152657971 wait complete. 2024-04-25T10:12:37.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 Processor -- start 2024-04-25T10:12:37.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- start start 2024-04-25T10:12:37.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 0x7fe100158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100166030 con 0x7fe100159fb0 2024-04-25T10:12:37.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe100165eb0 con 0x7fe100159bd0 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe10600c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001661b0 con 0x7fe1000b28c0 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56060/0 (socket says 172.21.15.82:56060) 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0fffff640 1 -- 172.21.15.82:0/2358764631 learned_addr learned my addr 172.21.15.82:0/2358764631 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.032+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0feffd640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 0x7fe100158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 -- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 msgr2=0x7fe100098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 -- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 msgr2=0x7fe100158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 0x7fe100158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 -- 172.21.15.82:0/2358764631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1001f5d40 con 0x7fe100159fb0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0feffd640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 0x7fe100158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0fffff640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801c830 tx=0x7fe0f801b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 -- 172.21.15.82:0/2358764631 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80be020 con 0x7fe100159fb0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 -- 172.21.15.82:0/2358764631 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f8009800 con 0x7fe100159fb0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1001f5fe0 con 0x7fe100159fb0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 -- 172.21.15.82:0/2358764631 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8009bb0 con 0x7fe100159fb0 2024-04-25T10:12:37.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1001f65a0 con 0x7fe100159fb0 2024-04-25T10:12:37.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 -- 172.21.15.82:0/2358764631 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0f80c1020 con 0x7fe100159fb0 2024-04-25T10:12:37.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 0x7fe0ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0e7fff640 1 -- 172.21.15.82:0/2358764631 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fe0f8122550 con 0x7fe100159fb0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 0x7fe0ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 0x7fe0ec069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe10015d190 tx=0x7fe0f0009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe109179000 1 Processor -- start 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.036+0000 7fe109179000 1 -- start start 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f00db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796eb6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564796d03ff0 con 0x564796eb70a0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x564796c8c150 con 0x564796f009d0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564796d00090 con 0x564796f012f0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56070/0 (socket says 172.21.15.82:56070) 2024-04-25T10:12:37.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/1771931810 learned_addr learned my addr 172.21.15.82:0/1771931810 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 msgr2=0x564796eb6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796eb6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 msgr2=0x564796f00db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f00db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/1771931810 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564796e9a0d0 con 0x564796eb70a0 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ef270 tx=0x7fe0f80ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5fb24789e4602c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0e57fa640 1 -- 172.21.15.82:0/1771931810 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80f6f60 con 0x564796eb70a0 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0e57fa640 1 -- 172.21.15.82:0/1771931810 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80fb520 con 0x564796eb70a0 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0e57fa640 1 -- 172.21.15.82:0/1771931810 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f8009d50 con 0x564796eb70a0 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/1771931810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 msgr2=0x564796eb7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80ef270 tx=0x7fe0f80ba000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/1771931810 shutdown_connections 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796eb6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796eb7480 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- 172.21.15.82:0/1771931810 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f00db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/1771931810 >> 172.21.15.82:0/1771931810 conn(0x564796eb3020 msgr2=0x564796eb5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/1771931810 shutdown_connections 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/1771931810 wait complete. 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- start start 2024-04-25T10:12:37.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 Processor -- start 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796ee57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796ee5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564796cc8250 con 0x564796f012f0 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x564796cbe690 con 0x564796f009d0 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564796c8ca40 con 0x564796eb70a0 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 10 client.0 _ll_register_callbacks cb 0x564796ea1e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56082/0 (socket says 172.21.15.82:56082) 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/4109083582 learned_addr learned my addr 172.21.15.82:0/4109083582 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796ee57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0feffd640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796ee5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 msgr2=0x564796ee57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796ee57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 msgr2=0x564796ee5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796ee5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 -- 172.21.15.82:0/4109083582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564796db11c0 con 0x564796f012f0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0feffd640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796ee5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0fffff640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80251f0 tx=0x7fe0f800ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4329860aa9996a06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f800b2e0 con 0x564796f012f0 2024-04-25T10:12:37.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fe0f80f9070 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f80fb520 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f800b480 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 10 client.9288 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x564796dad170 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.040+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80fb520 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0f80f3070 con 0x564796f012f0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 10 client.9288 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 10 client.9288 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 20 client.9288 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f10c80 0x564796f13070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 4 client.9288 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564796c9fc60 con 0x564796f10c80 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f10c80 0x564796f13070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0de7fc640 10 client.9288 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f10c80 0x564796f13070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3e14f21caa20b88 server_cookie=85dbc310a65a291e in_seq=0 out_seq=0 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0de7fc640 1 -- 172.21.15.82:0/4109083582 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564796c9fc60 con 0x564796f10c80 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0de7fc640 10 client.9288 handle_command_reply: tid=0 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 client.9288 shutdown 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 msgr2=0x564796ee6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fe0f80251f0 tx=0x7fe0f800ba20 comp rx=0 tx=0).stop 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 shutdown_connections 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f10c80 0x564796f13070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796ee6270 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe0de7fc640 0 client.9288 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796ee5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 --2- 172.21.15.82:0/4109083582 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796ee57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 >> 172.21.15.82:0/4109083582 conn(0x564796eb3020 msgr2=0x564796eb5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 shutdown_connections 2024-04-25T10:12:37.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 1 -- 172.21.15.82:0/4109083582 wait complete. 2024-04-25T10:12:37.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.044+0000 7fe109179000 20 client.9288 trim_cache size 0 max 16384 2024-04-25T10:12:37.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.056+0000 7fe109179000 1 Processor -- start 2024-04-25T10:12:37.061 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.056+0000 7fe109179000 1 -- start start 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796f61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f61a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001fefa0 con 0x564796f012f0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001fe1a0 con 0x564796f009d0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000c71b0 con 0x564796eb70a0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56088/0 (socket says 172.21.15.82:56088) 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 -- 172.21.15.82:0/3894990060 learned_addr learned my addr 172.21.15.82:0/3894990060 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 -- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 msgr2=0x564796f61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796f61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 -- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 msgr2=0x564796f61a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f61a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 -- 172.21.15.82:0/3894990060 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564796d4ade0 con 0x564796f012f0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0fffff640 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801d2b0 tx=0x7fe0f811d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5970ccda93a998a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0e4ff9640 1 -- 172.21.15.82:0/3894990060 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80f2950 con 0x564796f012f0 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0e4ff9640 1 -- 172.21.15.82:0/3894990060 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80fb520 con 0x564796f012f0 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0e4ff9640 1 -- 172.21.15.82:0/3894990060 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f80fb180 con 0x564796f012f0 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3894990060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 msgr2=0x564796f61fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801d2b0 tx=0x7fe0f811d000 comp rx=0 tx=0).stop 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3894990060 shutdown_connections 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796f012f0 0x564796f61fd0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f61a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- 172.21.15.82:0/3894990060 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796eb70a0 0x564796f61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3894990060 >> 172.21.15.82:0/3894990060 conn(0x564796eb3020 msgr2=0x564796f0dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3894990060 shutdown_connections 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3894990060 wait complete. 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- start start 2024-04-25T10:12:37.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 Processor -- start 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f658c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796f20e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564796c9ee00 con 0x564796eb70a0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10015b0e0 con 0x564796f009d0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564796d03300 con 0x564796f012f0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 10 client.0 _ll_register_callbacks cb 0x564796e82040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:56102/0 (socket says 172.21.15.82:56102) 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 -- 172.21.15.82:0/3480754001 learned_addr learned my addr 172.21.15.82:0/3480754001 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0feffd640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f658c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 -- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 msgr2=0x564796f20e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796f20e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 -- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 msgr2=0x564796f658c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f658c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 -- 172.21.15.82:0/3480754001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564796ca58e0 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0ff7fe640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00042d0 tx=0x7fe0f0004300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb31d04b1a0028e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0003aa0 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0f80fb520 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fe0f0003c40 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0f0036050 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 10 client.9294 fetch_fsmap learned FSMap version 81 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x564796d5d8f0 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f80fb520 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0f0039020 con 0x564796eb70a0 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 10 client.9294 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 10 client.9294 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 20 client.9294 populate_metadata read hostname 'smithi082' 2024-04-25T10:12:37.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f21a90 0x564796f21ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 4 client.9294 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.060+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564796d11530 con 0x564796f21a90 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0feffd640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f21a90 0x564796f21ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0feffd640 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f21a90 0x564796f21ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=216107dd4d13026b server_cookie=f6cab32f904ce983 in_seq=0 out_seq=0 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0de7fc640 10 client.9294 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0de7fc640 1 -- 172.21.15.82:0/3480754001 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x564796d11530 con 0x564796f21a90 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0de7fc640 10 client.9294 handle_command_reply: tid=0 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 client.9294 shutdown 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 msgr2=0x564796f65380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00042d0 tx=0x7fe0f0004300 comp rx=0 tx=0).stop 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 shutdown_connections 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x564796f21a90 0x564796f21ea0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564796f012f0 0x564796f20e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564796f009d0 0x564796f658c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 --2- 172.21.15.82:0/3480754001 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564796eb70a0 0x564796f65380 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe0de7fc640 0 client.9294 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 >> 172.21.15.82:0/3480754001 conn(0x564796eb3020 msgr2=0x564796eff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 shutdown_connections 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 1 -- 172.21.15.82:0/3480754001 wait complete. 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe109179000 20 client.9294 trim_cache size 0 max 16384 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:37.073 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": { 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: "e2be0496-12dc-41d7-bcff-93df11b72fe2": { 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: "path": "/", 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: "tag": "e2be0496-12dc-41d7-bcff-93df11b72fe2", 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: "options": "recursive,repair" 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:12:37.074 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 msgr2=0x7fe0ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 0x7fe0ec069450 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe10015d190 tx=0x7fe0f0009190 comp rx=0 tx=0).stop 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 msgr2=0x7fe1001591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fe0f801c830 tx=0x7fe0f801b9a0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 shutdown_connections 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe0ec066f90 0x7fe0ec069450 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe100159fb0 0x7fe1001591d0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe100159bd0 0x7fe100158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 --2- 172.21.15.82:0/2358764631 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1000b28c0 0x7fe100098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 >> 172.21.15.82:0/2358764631 conn(0x7fe10006ae60 msgr2=0x7fe1000af4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 shutdown_connections 2024-04-25T10:12:37.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:37.064+0000 7fe10600c640 1 -- 172.21.15.82:0/2358764631 wait complete. 2024-04-25T10:12:37.081 INFO:tasks.cephfs.filesystem:scrub status for tag:e2be0496-12dc-41d7-bcff-93df11b72fe2 - {'path': '/', 'tag': 'e2be0496-12dc-41d7-bcff-93df11b72fe2', 'options': 'recursive,repair'} 2024-04-25T10:12:37.275 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:12:37.272+0000 7f1040753640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:38.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:38.300+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 8 ==== log(8 entries from seq 1 at 2024-04-25T10:12:36.902636+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f8bec0547f0 con 0x7f8c040c6840 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.902636+0000 mds.c [INF] scrub queued for path: / 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.902639+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.902644+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.903177+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.903184+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T10:12:38.306 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.903890+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:12:38.307 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.903898+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T10:12:38.307 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:36.904738+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T10:12:38.673 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:38.676 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:42.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:12:42.520+0000 7f8bfaffd640 1 -- 172.21.15.82:0/637392623 <== mon.0 v2:172.21.15.82:3300/0 9 ==== log(1 entries from seq 9 at 2024-04-25T10:12:41.327347+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8bec014e00 con 0x7f8c040c6840 2024-04-25T10:12:42.526 INFO:teuthology.orchestra.run.smithi082.stdout:2024-04-25T10:12:41.327347+0000 mds.c [INF] scrub summary: idle 2024-04-25T10:13:07.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:07.083 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:07.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.211+0000 7f94f930e640 1 Processor -- start 2024-04-25T10:13:07.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.211+0000 7f94f930e640 1 -- start start 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 0x7f94f415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f94f40cf970 0x7f94f40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f40c9ef0 0x7f94f40b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f40ca780 con 0x7f94f40c9ef0 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f40cd440 con 0x7f94f40cf970 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f930e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40cdce0 con 0x7f94f40cf590 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f40c9ef0 0x7f94f40b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f94f40c9ef0 0x7f94f40b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45946/0 (socket says 172.21.15.82:45946) 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 -- 172.21.15.82:0/2980734649 learned_addr learned my addr 172.21.15.82:0/2980734649 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 0x7f94f415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/2980734649 >> v2:172.21.15.196:3300/0 conn(0x7f94f40cf970 0x7f94f40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.219 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 -- 172.21.15.82:0/2980734649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 msgr2=0x7f94f415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 0x7f94f415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 -- 172.21.15.82:0/2980734649 >> v2:172.21.15.196:3300/0 conn(0x7f94f40cf970 msgr2=0x7f94f40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 --2- 172.21.15.82:0/2980734649 >> v2:172.21.15.196:3300/0 conn(0x7f94f40cf970 0x7f94f40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f37fe640 1 -- 172.21.15.82:0/2980734649 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f40ca4c0 con 0x7f94f40c9ef0 2024-04-25T10:13:07.220 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.215+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 0x7f94f415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f37fe640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f40b21b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f94f40d0110 tx=0x7f94ec01c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3965c687ad6413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f27fc640 1 -- 172.21.15.82:0/2980734649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0a5580 con 0x7f94f40c9ef0 2024-04-25T10:13:07.221 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f27fc640 1 -- 172.21.15.82:0/2980734649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec0a5720 con 0x7f94f40c9ef0 2024-04-25T10:13:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f27fc640 1 -- 172.21.15.82:0/2980734649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec0a5af0 con 0x7f94f40c9ef0 2024-04-25T10:13:07.222 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- 172.21.15.82:0/2980734649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 msgr2=0x7f94f40b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f40b21b0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f94f40d0110 tx=0x7f94ec01c700 comp rx=0 tx=0).stop 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- 172.21.15.82:0/2980734649 shutdown_connections 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f40b21b0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- 172.21.15.82:0/2980734649 >> v2:172.21.15.196:3300/0 conn(0x7f94f40cf970 0x7f94f40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- 172.21.15.82:0/2980734649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf590 0x7f94f415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- 172.21.15.82:0/2980734649 >> 172.21.15.82:0/2980734649 conn(0x7f94f406ae60 msgr2=0x7f94f406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- 172.21.15.82:0/2980734649 shutdown_connections 2024-04-25T10:13:07.223 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- 172.21.15.82:0/2980734649 wait complete. 2024-04-25T10:13:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 Processor -- start 2024-04-25T10:13:07.224 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- start start 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f40cf590 0x7f94f4098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf970 0x7f94f415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ac2f0 con 0x7f94f40c9ef0 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ac170 con 0x7f94f40cf590 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f930e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40ce150 con 0x7f94f40cf970 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45952/0 (socket says 172.21.15.82:45952) 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 -- 172.21.15.82:0/891105495 learned_addr learned my addr 172.21.15.82:0/891105495 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f37fe640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf970 0x7f94f415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f40cf590 0x7f94f4098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 -- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf970 msgr2=0x7f94f415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf970 0x7f94f415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 -- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f40cf590 msgr2=0x7f94f4098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f40cf590 0x7f94f4098940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 -- 172.21.15.82:0/891105495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f415a850 con 0x7f94f40c9ef0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f94e400c7f0 tx=0x7f94e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.219+0000 7f94ebfff640 1 -- 172.21.15.82:0/891105495 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4015070 con 0x7f94f40c9ef0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ebfff640 1 -- 172.21.15.82:0/891105495 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e4007a30 con 0x7f94f40c9ef0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f4159550 con 0x7f94f40c9ef0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ebfff640 1 -- 172.21.15.82:0/891105495 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4007e00 con 0x7f94f40c9ef0 2024-04-25T10:13:07.227 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f4159b10 con 0x7f94f40c9ef0 2024-04-25T10:13:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ebfff640 1 -- 172.21.15.82:0/891105495 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94e4018070 con 0x7f94f40c9ef0 2024-04-25T10:13:07.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ebfff640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 0x7f94d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ebfff640 1 -- 172.21.15.82:0/891105495 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f94e40850d0 con 0x7f94f40c9ef0 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 0x7f94d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 0x7f94d4069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f94f4071170 tx=0x7f94dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 Processor -- start 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 -- start start 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9e1be30 0x5643e9e1c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9dd24b0 0x5643e9dd2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643e9c1aff0 con 0x5643e9e1c750 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643e9ba3150 con 0x5643e9e1be30 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643e9c17090 con 0x5643e9dd24b0 2024-04-25T10:13:07.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45960/0 (socket says 172.21.15.82:45960) 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1316204722 learned_addr learned my addr 172.21.15.82:0/1316204722 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9dd24b0 msgr2=0x5643e9dd2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9dd24b0 0x5643e9dd2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9e1be30 msgr2=0x5643e9e1c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.223+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9e1be30 0x5643e9e1c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1316204722 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643e9db10d0 con 0x5643e9e1c750 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d9a0 tx=0x7f94dc010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebed7b8ab92928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94e8ff9640 1 -- 172.21.15.82:0/1316204722 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc023070 con 0x5643e9e1c750 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94e8ff9640 1 -- 172.21.15.82:0/1316204722 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc01a040 con 0x5643e9e1c750 2024-04-25T10:13:07.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94e8ff9640 1 -- 172.21.15.82:0/1316204722 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc019050 con 0x5643e9e1c750 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/1316204722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 msgr2=0x5643e9dd1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d9a0 tx=0x7f94dc010000 comp rx=0 tx=0).stop 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/1316204722 shutdown_connections 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9dd24b0 0x5643e9dd2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1c750 0x5643e9dd1f70 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1316204722 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9e1be30 0x5643e9e1c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/1316204722 >> 172.21.15.82:0/1316204722 conn(0x5643e9dca020 msgr2=0x5643e9dcc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/1316204722 shutdown_connections 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/1316204722 wait complete. 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- start start 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 Processor -- start 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e1b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dd88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643e9c1f620 con 0x5643e9e1be30 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643e9bd5690 con 0x5643e9dd24b0 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643e9ba3a40 con 0x5643e9e1c750 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 10 client.0 _ll_register_callbacks cb 0x5643e9db8e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45966/0 (socket says 172.21.15.82:45966) 2024-04-25T10:13:07.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 -- 172.21.15.82:0/994198236 learned_addr learned my addr 172.21.15.82:0/994198236 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94f37fe640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dd88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 -- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 msgr2=0x5643e9dd88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dd88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 -- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 msgr2=0x5643e9e1b7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e1b7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 -- 172.21.15.82:0/994198236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643e9cc81c0 con 0x5643e9e1be30 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94f37fe640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dd88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f94dc021040 tx=0x7f94dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b9895baeef95c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc02d020 con 0x5643e9e1be30 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f94dc003620 con 0x5643e9e1be30 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94dc01a040 con 0x5643e9e1be30 2024-04-25T10:13:07.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94dc019020 con 0x5643e9e1be30 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 10 client.9306 fetch_fsmap learned FSMap version 81 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x5643e9cc4170 con 0x5643e9e1be30 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc01a040 con 0x5643e9e1be30 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94dc0038a0 con 0x5643e9e1be30 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 10 client.9306 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 10 client.9306 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 20 client.9306 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9dd9610 0x5643e9e2e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 4 client.9306 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.227+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643e9bb6c60 con 0x5643e9dd9610 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9dd9610 0x5643e9e2e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9dd9610 0x5643e9e2e0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79d2b90d8b9d2b9b server_cookie=1796ad646c48e666 in_seq=0 out_seq=0 2024-04-25T10:13:07.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94d1ffb640 10 client.9306 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94d1ffb640 1 -- 172.21.15.82:0/994198236 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5643e9bb6c60 con 0x5643e9dd9610 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94d1ffb640 10 client.9306 handle_command_reply: tid=0 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 client.9306 shutdown 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 msgr2=0x5643e9dd8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f94dc021040 tx=0x7f94dc004740 comp rx=0 tx=0).stop 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 shutdown_connections 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9dd9610 0x5643e9e2e0f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dd88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9dd8360 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 --2- 172.21.15.82:0/994198236 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e1b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94d1ffb640 0 client.9306 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 >> 172.21.15.82:0/994198236 conn(0x5643e9dca020 msgr2=0x5643e9dd3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 shutdown_connections 2024-04-25T10:13:07.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 1 -- 172.21.15.82:0/994198236 wait complete. 2024-04-25T10:13:07.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.231+0000 7f94fc47b000 20 client.9306 trim_cache size 0 max 16384 2024-04-25T10:13:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 Processor -- start 2024-04-25T10:13:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- start start 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e30110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f41fef20 con 0x5643e9e1be30 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c4640 con 0x5643e9dd24b0 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40cc6f0 con 0x5643e9e1c750 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45974/0 (socket says 172.21.15.82:45974) 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1943218774 learned_addr learned my addr 172.21.15.82:0/1943218774 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e30110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 msgr2=0x5643e9dfa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 msgr2=0x5643e9e30110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e30110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 -- 172.21.15.82:0/1943218774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643e9c61de0 con 0x5643e9e1be30 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d770 tx=0x7f94dc028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e144cb8f5210bd8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94d3fff640 1 -- 172.21.15.82:0/1943218774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc02d040 con 0x5643e9e1be30 2024-04-25T10:13:07.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94d3fff640 1 -- 172.21.15.82:0/1943218774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc024040 con 0x5643e9e1be30 2024-04-25T10:13:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94d3fff640 1 -- 172.21.15.82:0/1943218774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc023070 con 0x5643e9e1be30 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- 172.21.15.82:0/1943218774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 msgr2=0x5643e9e30650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00d770 tx=0x7f94dc028000 comp rx=0 tx=0).stop 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- 172.21.15.82:0/1943218774 shutdown_connections 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9e30650 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 --2- 172.21.15.82:0/1943218774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9e30110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- 172.21.15.82:0/1943218774 >> 172.21.15.82:0/1943218774 conn(0x5643e9dca020 msgr2=0x5643e9e28b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- 172.21.15.82:0/1943218774 shutdown_connections 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- 172.21.15.82:0/1943218774 wait complete. 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 -- start start 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.247+0000 7f94fc47b000 1 Processor -- start 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9df98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643e9bb5e00 con 0x5643e9e1be30 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40c4020 con 0x5643e9dd24b0 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5643e9c1a300 con 0x5643e9e1c750 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 10 client.0 _ll_register_callbacks cb 0x5643e9d99040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:07.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:45990/0 (socket says 172.21.15.82:45990) 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 -- 172.21.15.82:0/3198950977 learned_addr learned my addr 172.21.15.82:0/3198950977 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9df98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f37fe640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 -- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 msgr2=0x5643e9dfa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 -- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 msgr2=0x5643e9df98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9df98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 -- 172.21.15.82:0/3198950977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5643e9bbc8e0 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94ea7fc640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f94dc034010 tx=0x7f94dc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a102aa3ce7fa263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041050 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f94dc047cf0 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94dc024040 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94dc04e050 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 10 client.9312 fetch_fsmap learned FSMap version 81 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x5643e9c748f0 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc024040 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94dc051020 con 0x5643e9e1be30 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 10 client.9312 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 10 client.9312 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T10:13:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 20 client.9312 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9e81fc0 0x5643e9e843b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 4 client.9312 mds_command: new command op to 9177 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 --> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643e9c28530 con 0x5643e9e81fc0 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9e81fc0 0x5643e9e843b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 10 client.9312 ms_handle_connect on v2:172.21.15.82:6836/603032759 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f2ffd640 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9e81fc0 0x5643e9e843b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe72791b2113eb96 server_cookie=a03eb1ea672cdd33 in_seq=0 out_seq=0 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 1 -- 172.21.15.82:0/3198950977 <== mds.0 v2:172.21.15.82:6836/603032759 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5643e9c28530 con 0x5643e9e81fc0 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 10 client.9312 handle_command_reply: tid=0 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 client.9312 shutdown 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 msgr2=0x5643e9df9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f94dc034010 tx=0x7f94dc013000 comp rx=0 tx=0).stop 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 shutdown_connections 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:6836/603032759,v1:172.21.15.82:6837/603032759] conn(0x5643e9e81fc0 0x5643e9e843b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5643e9e1c750 0x5643e9dfa360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5643e9e1be30 0x5643e9df9e20 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94d1ffb640 0 client.9312 ms_handle_reset on v2:172.21.15.82:6836/603032759 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 --2- 172.21.15.82:0/3198950977 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5643e9dd24b0 0x5643e9df98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 >> 172.21.15.82:0/3198950977 conn(0x5643e9dca020 msgr2=0x5643e9e29420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 shutdown_connections 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 1 -- 172.21.15.82:0/3198950977 wait complete. 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94fc47b000 20 client.9312 trim_cache size 0 max 16384 2024-04-25T10:13:07.262 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:07.263 INFO:teuthology.orchestra.run.smithi082.stdout: "status": "no active scrubs running", 2024-04-25T10:13:07.263 INFO:teuthology.orchestra.run.smithi082.stdout: "scrubs": {} 2024-04-25T10:13:07.263 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 msgr2=0x7f94d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 0x7f94d4069450 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f94f4071170 tx=0x7f94dc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 msgr2=0x7f94f415a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f94e400c7f0 tx=0x7f94e400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 shutdown_connections 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f94d4066f90 0x7f94d4069450 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f94f40cf970 0x7f94f415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f94f40cf590 0x7f94f4098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 --2- 172.21.15.82:0/891105495 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f94f40c9ef0 0x7f94f415a310 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 >> 172.21.15.82:0/891105495 conn(0x7f94f406ae60 msgr2=0x7f94f4154230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 shutdown_connections 2024-04-25T10:13:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.251+0000 7f94f930e640 1 -- 172.21.15.82:0/891105495 wait complete. 2024-04-25T10:13:07.272 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:13:07.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:07.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 Processor -- start 2024-04-25T10:13:07.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 -- start start 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f41540ce510 0x7f41540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f41541451f0 0x7f4154142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41540cee30 con 0x7f4154144ca0 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41540cc2b0 con 0x7f41541451f0 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f415aae9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415406e290 con 0x7f41540ce510 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4159086640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f41541451f0 0x7f4154142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4153fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4154144ca0 0x7f415406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59882/0 (socket says 172.21.15.82:59882) 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4153fff640 1 -- 172.21.15.82:0/2328722795 learned_addr learned my addr 172.21.15.82:0/2328722795 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4158885640 1 --2- 172.21.15.82:0/2328722795 >> v2:172.21.15.196:3300/0 conn(0x7f41540ce510 0x7f41540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4159086640 1 -- 172.21.15.82:0/2328722795 >> v2:172.21.15.196:3300/0 conn(0x7f41540ce510 msgr2=0x7f41540ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4159086640 1 --2- 172.21.15.82:0/2328722795 >> v2:172.21.15.196:3300/0 conn(0x7f41540ce510 0x7f41540ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4159086640 1 -- 172.21.15.82:0/2328722795 >> v1:172.21.15.82:6789/0 conn(0x7f4154144ca0 legacy=0x7f415406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f4159086640 1 -- 172.21.15.82:0/2328722795 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540ccc70 con 0x7f41541451f0 2024-04-25T10:13:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.499+0000 7f41537fe640 1 -- 172.21.15.82:0/2328722795 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149100166 0 0) 0x7f41540cee30 con 0x7f4154144ca0 2024-04-25T10:13:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f4159086640 1 --2- 172.21.15.82:0/2328722795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41541451f0 0x7f4154142680 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4154144c50 tx=0x7f414c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d92ce112be2e58c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f41537fe640 1 -- 172.21.15.82:0/2328722795 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0b4070 con 0x7f41541451f0 2024-04-25T10:13:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f41537fe640 1 -- 172.21.15.82:0/2328722795 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c0af600 con 0x7f41541451f0 2024-04-25T10:13:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f41537fe640 1 -- 172.21.15.82:0/2328722795 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c0af9d0 con 0x7f41541451f0 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- 172.21.15.82:0/2328722795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41541451f0 msgr2=0x7f4154142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 --2- 172.21.15.82:0/2328722795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41541451f0 0x7f4154142680 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f4154144c50 tx=0x7f414c01f110 comp rx=0 tx=0).stop 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- 172.21.15.82:0/2328722795 shutdown_connections 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 --2- 172.21.15.82:0/2328722795 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f41541451f0 0x7f4154142680 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 --2- 172.21.15.82:0/2328722795 >> v2:172.21.15.196:3300/0 conn(0x7f41540ce510 0x7f41540ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.508 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- 172.21.15.82:0/2328722795 >> 172.21.15.82:0/2328722795 conn(0x7f41540b11a0 msgr2=0x7f41540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- 172.21.15.82:0/2328722795 shutdown_connections 2024-04-25T10:13:07.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- 172.21.15.82:0/2328722795 wait complete. 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 Processor -- start 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.503+0000 7f415aae9640 1 -- start start 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41540ce510 0x7f41541f1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41541451f0 0x7f41541f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415406ee30 con 0x7f4154144ca0 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415406df20 con 0x7f41540ce510 2024-04-25T10:13:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41540cc900 con 0x7f41541451f0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46016/0 (socket says 172.21.15.82:46016) 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 -- 172.21.15.82:0/1710410654 learned_addr learned my addr 172.21.15.82:0/1710410654 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4159086640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41541451f0 0x7f41541f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 -- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41541451f0 msgr2=0x7f41541f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41541451f0 0x7f41541f1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 -- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41540ce510 msgr2=0x7f41541f1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4158885640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41540ce510 0x7f41541f1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41540ce510 0x7f41541f1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 -- 172.21.15.82:0/1710410654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41541e5110 con 0x7f4154144ca0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4153fff640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f414800c4b0 tx=0x7f414800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148018070 con 0x7f4154144ca0 2024-04-25T10:13:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4148007500 con 0x7f4154144ca0 2024-04-25T10:13:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41541e2d00 con 0x7f4154144ca0 2024-04-25T10:13:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148004a30 con 0x7f4154144ca0 2024-04-25T10:13:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41541e32c0 con 0x7f4154144ca0 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41541e34c0 con 0x7f4154144ca0 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f414801b070 con 0x7f4154144ca0 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 0x7f4140069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f4148084490 con 0x7f4154144ca0 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4158885640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 0x7f4140069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.507+0000 7f4158885640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 0x7f4140069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4144009790 tx=0x7f4144008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.511+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41541e34c0 con 0x7f4154144ca0 2024-04-25T10:13:07.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.703+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f415404efa0 con 0x7f4154144ca0 2024-04-25T10:13:07.710 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:13:07.707+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:07.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.707+0000 7f41517fa640 1 -- 172.21.15.82:0/1710410654 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f41480553f0 con 0x7f4154144ca0 2024-04-25T10:13:07.713 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:07.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 msgr2=0x7f4140069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 0x7f4140069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f4144009790 tx=0x7f4144008040 comp rx=0 tx=0).stop 2024-04-25T10:13:07.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 msgr2=0x7f41541f19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.716 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f414800c4b0 tx=0x7f414800c970 comp rx=0 tx=0).stop 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 shutdown_connections 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4140066f90 0x7f4140069450 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f41541451f0 0x7f41541f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4154144ca0 0x7f41541f19a0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 --2- 172.21.15.82:0/1710410654 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f41540ce510 0x7f41541f1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 >> 172.21.15.82:0/1710410654 conn(0x7f41540b11a0 msgr2=0x7f41540718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 shutdown_connections 2024-04-25T10:13:07.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.711+0000 7f415aae9640 1 -- 172.21.15.82:0/1710410654 wait complete. 2024-04-25T10:13:07.719 INFO:tasks.ceph.mds.c.smithi082.stderr: -1158> 2024-04-25T10:12:36.896+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T10:13:07.719 INFO:tasks.ceph.mds.c.smithi082.stderr: -1155> 2024-04-25T10:12:36.896+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T10:13:07.719 INFO:tasks.ceph.mds.c.smithi082.stderr: -1106> 2024-04-25T10:12:36.900+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T10:13:07.719 INFO:tasks.ceph.mds.c.smithi082.stderr: -1103> 2024-04-25T10:12:36.900+0000 7f103c74b640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T10:13:07.719 INFO:tasks.ceph.mds.c.smithi082.stderr: -995> 2024-04-25T10:12:37.272+0000 7f1040753640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:07.735 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T10:13:07.746 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:13:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 -- start start 2024-04-25T10:13:07.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 Processor -- start 2024-04-25T10:13:07.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 0x5640d9e9d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 0x5640d9e9e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640daac6480 con 0x5640d9ce3c00 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640daac6600 con 0x5640daabdc00 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640daac6780 con 0x5640daabd400 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a3199640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a3199640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46066/0 (socket says 172.21.15.82:46066) 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.763+0000 7f85a3199640 1 -- 172.21.15.82:0/3135872626 learned_addr learned my addr 172.21.15.82:0/3135872626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a2998640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 0x5640d9e9d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 0x5640d9e9e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 -- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 msgr2=0x5640d9e9e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 0x5640d9e9e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 -- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 msgr2=0x5640d9e9d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 0x5640d9e9d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 -- 172.21.15.82:0/3135872626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640daa9ab40 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a2998640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 0x5640d9e9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 0x5640d9e9e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x5640dab45410 tx=0x5640daaa8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97ded804ef2d83a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a0994640 1 -- 172.21.15.82:0/3135872626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d181a0 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a0994640 1 -- 172.21.15.82:0/3135872626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640d9d2ce00 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a0994640 1 -- 172.21.15.82:0/3135872626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d184e0 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640daa9af00 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5640daa9b0e0 con 0x5640d9ce3c00 2024-04-25T10:13:07.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 4 waiting for MDS map... 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a0994640 1 -- 172.21.15.82:0/3135872626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x5640daab8600 con 0x5640d9ce3c00 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a0994640 1 -- 172.21.15.82:0/3135872626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 82) v1 ==== 2723+0+0 (secure 0 0 0) 0x5640d9d39180 con 0x5640d9ce3c00 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 4 Got MDS map 82 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 10 main: main 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 4 main: connecting to RADOS... 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 Processor -- start 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- start start 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac00800 0x5640d9e9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640d9d42180 con 0x5640daabc400 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640daac6f00 con 0x5640dac00800 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640daac7080 con 0x5640dac00c00 2024-04-25T10:13:07.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46078/0 (socket says 172.21.15.82:46078) 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 -- 172.21.15.82:0/2905202681 learned_addr learned my addr 172.21.15.82:0/2905202681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a3199640 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac00800 0x5640d9e9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a2998640 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 -- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 msgr2=0x5640d9e9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 -- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac00800 msgr2=0x5640d9e9ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac00800 0x5640d9e9ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 -- 172.21.15.82:0/2905202681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640daa9af00 con 0x5640daabc400 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a399a640 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x5640d9d33980 tx=0x5640daaa82a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edef68c749d6265e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f859e990640 1 -- 172.21.15.82:0/2905202681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d181a0 con 0x5640daabc400 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f859e990640 1 -- 172.21.15.82:0/2905202681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640d9d2c1c0 con 0x5640daabc400 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f859e990640 1 -- 172.21.15.82:0/2905202681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d184e0 con 0x5640daabc400 2024-04-25T10:13:07.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/2905202681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 msgr2=0x5640d9e9e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x5640d9d33980 tx=0x5640daaa82a0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/2905202681 shutdown_connections 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac00800 0x5640d9e9ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 --2- 172.21.15.82:0/2905202681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640daabc400 0x5640d9e9e680 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/2905202681 >> 172.21.15.82:0/2905202681 conn(0x5640daabd800 msgr2=0x5640dac5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/2905202681 shutdown_connections 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.767+0000 7f85a49da400 1 -- 172.21.15.82:0/2905202681 wait complete. 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 Processor -- start 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- start start 2024-04-25T10:13:07.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac01400 0x5640d9e9e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 0x5640d9e9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640daac7380 con 0x5640dac01000 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640daac7200 con 0x5640dac01400 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640daac7500 con 0x5640dac00c00 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 0x5640d9e9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 0x5640d9e9ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46090/0 (socket says 172.21.15.82:46090) 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 -- 172.21.15.82:0/3333030714 learned_addr learned my addr 172.21.15.82:0/3333030714 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a2998640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a399a640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac01400 0x5640d9e9e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 -- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 msgr2=0x5640d9e9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640dac00c00 0x5640d9e9f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 -- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac01400 msgr2=0x5640d9e9e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac01400 0x5640d9e9e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 -- 172.21.15.82:0/3333030714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640daa9be00 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a399a640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640dac01400 0x5640d9e9e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a3199640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 0x5640d9e9ec00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x5640dac0f1d0 tx=0x5640daaa8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 -- 172.21.15.82:0/3333030714 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d18b60 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 -- 172.21.15.82:0/3333030714 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5640d9d2cc40 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 -- 172.21.15.82:0/3333030714 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5640d9d181a0 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5640daa9ad20 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640daa9a960 con 0x5640dac01000 2024-04-25T10:13:07.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 4 main: resolving pool 15 2024-04-25T10:13:07.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 -- 172.21.15.82:0/3333030714 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5640daac8c00 con 0x5640dac01000 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5640da019000 0x5640da0fe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f859d18d640 1 -- 172.21.15.82:0/3333030714 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x5640daaca600 con 0x5640dac01000 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 4 main: creating IoCtx.. 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a399a640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5640da019000 0x5640da0fe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5640dac00800 0x5640d9e9f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.771+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e111) v9 -- 0x5640daabc400 con 0x5640dac00800 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a2998640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5640dac00800 0x5640d9e9f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a399a640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5640da019000 0x5640da0fe000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5640d9d33980 tx=0x5640daaa84e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a2998640 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5640dac00800 0x5640d9e9f700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:07.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a2998640 1 -- 172.21.15.82:0/3333030714 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x5640d9dee000 con 0x5640dac00800 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "0": { 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "data": { 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "version": 5, 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "inotable": { 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "projected_free": [ 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "start": 1099511627779, 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "len": 1099511627773 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: ], 2024-04-25T10:13:07.790 INFO:teuthology.orchestra.run.smithi082.stdout: "free": [ 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: "start": 1099511627779, 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: "len": 1099511627773 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: "result": 0 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:07.791 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:07.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5640dac00800 msgr2=0x5640d9e9f700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5640dac00800 0x5640d9e9f700 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5640da019000 msgr2=0x5640da0fe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5640da019000 0x5640da0fe000 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x5640d9d33980 tx=0x5640daaa84e0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 msgr2=0x5640d9e9ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3333030714 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640dac01000 0x5640d9e9ec00 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x5640dac0f1d0 tx=0x5640daaa8060 comp rx=0 tx=0).stop 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a2998640 1 -- 172.21.15.82:0/3333030714 reap_dead start 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 shutdown_connections 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 >> 172.21.15.82:0/3333030714 conn(0x5640dac01800 msgr2=0x5640dac5a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 shutdown_connections 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3333030714 wait complete. 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 msgr2=0x5640d9e9d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x5640dab45410 tx=0x5640daaa8000 comp rx=0 tx=0).stop 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 shutdown_connections 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5640daabd400 0x5640d9e9e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5640daabdc00 0x5640d9e9d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 --2- 172.21.15.82:0/3135872626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5640d9ce3c00 0x5640d9e9d600 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:07.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 >> 172.21.15.82:0/3135872626 conn(0x5640d9ce2c00 msgr2=0x5640d9d11800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:07.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 shutdown_connections 2024-04-25T10:13:07.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:07.775+0000 7f85a49da400 1 -- 172.21.15.82:0/3135872626 wait complete. 2024-04-25T10:13:07.795 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.060150, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T10:13:07.795 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:07.795 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:07.883 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:07.954 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:07.954 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:07.954 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:07.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:07.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:13:07.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:07.961 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:08.010 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:13:08.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:08.011 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:13:08.011 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:08.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:08.018 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:13:08.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:08.018 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:13:08.064 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:13:08.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:08.065 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:08.066 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:08.072 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:08.072 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:13:08.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:08.072 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:13:08.117 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:13:08.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:08.119 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:13:08.119 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:08.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:08.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:13:08.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:08.125 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:13:08.173 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:13:08.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:08.174 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 Processor -- start 2024-04-25T10:13:08.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 -- start start 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e9406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9406b5b0 con 0x7f6e940cc130 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9415f360 con 0x7f6e9415ca90 2024-04-25T10:13:08.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e9acf6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415ff70 con 0x7f6e9406ac90 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e99293640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6e940cc130 0x7f6e940cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59908/0 (socket says 172.21.15.82:59908) 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e99293640 1 -- 172.21.15.82:0/3655417096 learned_addr learned my addr 172.21.15.82:0/3655417096 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e98a92640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e9406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975133732 0 0) 0x7f6e9406b5b0 con 0x7f6e940cc130 2024-04-25T10:13:08.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c003530 con 0x7f6e940cc130 2024-04-25T10:13:08.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1505370812 0 0) 0x7f6e7c003530 con 0x7f6e940cc130 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 msgr2=0x7f6e9406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e9406b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 >> v1:172.21.15.196:6789/0 conn(0x7f6e9415ca90 legacy=0x7f6e940d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9415f640 con 0x7f6e940cc130 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6e8c001a00 con 0x7f6e940cc130 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 >> v1:172.21.15.82:6789/0 conn(0x7f6e940cc130 legacy=0x7f6e940cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e7c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e7c003fe0 0x7f6e7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e7c004d50 0x7f6e7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00bb00 con 0x7f6e7c003fe0 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00bcb0 con 0x7f6e7c004d50 2024-04-25T10:13:08.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00beb0 con 0x7f6e9406ac90 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e7c003fe0 0x7f6e7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e98a92640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e7c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e99293640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e7c004d50 0x7f6e7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 -- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 msgr2=0x7f6e7c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e9406ac90 0x7f6e7c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 -- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e7c004d50 msgr2=0x7f6e7c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e7c004d50 0x7f6e7c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e93fff640 1 -- 172.21.15.82:0/3655417096 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c003530 con 0x7f6e7c003fe0 2024-04-25T10:13:08.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.303+0000 7f6e99293640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e7c004d50 0x7f6e7c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:08.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e93fff640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e7c003fe0 0x7f6e7c004810 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f6e88001780 tx=0x7f6e8801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fc8a31a3baadaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880013a0 con 0x7f6e7c003fe0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880052f0 con 0x7f6e7c003fe0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e937fe640 1 -- 172.21.15.82:0/3655417096 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88003840 con 0x7f6e7c003fe0 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- 172.21.15.82:0/3655417096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e7c003fe0 msgr2=0x7f6e7c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/3655417096 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e7c003fe0 0x7f6e7c004810 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f6e88001780 tx=0x7f6e8801c360 comp rx=0 tx=0).stop 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e99293640 1 -- 172.21.15.82:0/3655417096 reap_dead start 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- 172.21.15.82:0/3655417096 shutdown_connections 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- 172.21.15.82:0/3655417096 >> 172.21.15.82:0/3655417096 conn(0x7f6e94099a70 msgr2=0x7f6e940d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- 172.21.15.82:0/3655417096 shutdown_connections 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- 172.21.15.82:0/3655417096 wait complete. 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 Processor -- start 2024-04-25T10:13:08.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.307+0000 7f6e9acf6640 1 -- start start 2024-04-25T10:13:08.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7c003fe0 0x7f6e940cf540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e941f4a00 0x7f6e941f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940d1320 con 0x7f6e940ce010 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415fc00 con 0x7f6e941f4a00 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940cf0b0 con 0x7f6e7c003fe0 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46102/0 (socket says 172.21.15.82:46102) 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 -- 172.21.15.82:0/2772563421 learned_addr learned my addr 172.21.15.82:0/2772563421 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e98a92640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7c003fe0 0x7f6e940cf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e99293640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e941f4a00 0x7f6e941f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 -- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7c003fe0 msgr2=0x7f6e940cf540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7c003fe0 0x7f6e940cf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 -- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e941f4a00 msgr2=0x7f6e941f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e941f4a00 0x7f6e941f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 -- 172.21.15.82:0/2772563421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940b4550 con 0x7f6e940ce010 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e93fff640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005040 tx=0x7f6e7c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.316 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e88007a40 con 0x7f6e940ce010 2024-04-25T10:13:08.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880bb070 con 0x7f6e940ce010 2024-04-25T10:13:08.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940b33d0 con 0x7f6e940ce010 2024-04-25T10:13:08.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880069e0 con 0x7f6e940ce010 2024-04-25T10:13:08.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e940ce780 con 0x7f6e940ce010 2024-04-25T10:13:08.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9404efa0 con 0x7f6e940ce010 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e880b4040 con 0x7f6e940ce010 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f6e8811f640 con 0x7f6e940ce010 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.311+0000 7f6e98a92640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.315+0000 7f6e98a92640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009970 tx=0x7f6e84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.319+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e940b4550 con 0x7f6e940ce010 2024-04-25T10:13:08.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e9406b7c0 con 0x7f6e940ce010 2024-04-25T10:13:08.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e917fa640 1 -- 172.21.15.82:0/2772563421 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4149 (secure 0 0 0) 0x7f6e880a4e20 con 0x7f6e940ce010 2024-04-25T10:13:08.514 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:08.515 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":82,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:13:07.712073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:08.515 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 82 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 msgr2=0x7f6e80069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009970 tx=0x7f6e84008040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 msgr2=0x7f6e940b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005040 tx=0x7f6e7c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 shutdown_connections 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6e80066f90 0x7f6e80069450 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6e941f4a00 0x7f6e941f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6e940ce010 0x7f6e940b4010 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 --2- 172.21.15.82:0/2772563421 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6e7c003fe0 0x7f6e940cf540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 >> 172.21.15.82:0/2772563421 conn(0x7f6e94099a70 msgr2=0x7f6e940c8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 shutdown_connections 2024-04-25T10:13:08.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.511+0000 7f6e9acf6640 1 -- 172.21.15.82:0/2772563421 wait complete. 2024-04-25T10:13:08.534 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:08.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 Processor -- start 2024-04-25T10:13:08.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 -- start start 2024-04-25T10:13:08.743 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f391006fc50 0x7f391006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f39100cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391006b710 con 0x7f39100cfd80 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3910161560 con 0x7f391006fc50 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3917384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39101617b0 con 0x7f391006b1d0 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915921640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f39100cfd80 0x7f391015d180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59920/0 (socket says 172.21.15.82:59920) 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915921640 1 -- 172.21.15.82:0/1209048736 learned_addr learned my addr 172.21.15.82:0/1209048736 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 --2- 172.21.15.82:0/1209048736 >> v2:172.21.15.196:3300/0 conn(0x7f391006fc50 0x7f391006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f391491f640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f39100cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147216356 0 0) 0x7f391006b710 con 0x7f39100cfd80 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f8003530 con 0x7f39100cfd80 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1117763082 0 0) 0x7f38f8003530 con 0x7f39100cfd80 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 msgr2=0x7f39100cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f39100cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 >> v2:172.21.15.196:3300/0 conn(0x7f391006fc50 msgr2=0x7f391006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 --2- 172.21.15.82:0/1209048736 >> v2:172.21.15.196:3300/0 conn(0x7f391006fc50 0x7f391006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910161a90 con 0x7f39100cfd80 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f3900001680 con 0x7f39100cfd80 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 >> v1:172.21.15.82:6789/0 conn(0x7f39100cfd80 legacy=0x7f391015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f8003a30 0x7f38f8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f38f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38f8004d50 0x7f38f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3910161560 con 0x7f38f8003a30 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39101617b0 con 0x7f38f8004d50 2024-04-25T10:13:08.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f800b9d0 con 0x7f391006b1d0 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f8003a30 0x7f38f8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915921640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38f8004d50 0x7f38f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f391491f640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f38f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 -- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 msgr2=0x7f38f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f391006b1d0 0x7f38f8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 -- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38f8004d50 msgr2=0x7f38f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f38f8004d50 0x7f38f80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.647+0000 7f3915120640 1 -- 172.21.15.82:0/1209048736 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f8003530 con 0x7f38f8003a30 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3915120640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f8003a30 0x7f38f8003e50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f3908009ae0 tx=0x7f390801ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a079eea0c8762a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080096b0 con 0x7f38f8003a30 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39080a6460 con 0x7f38f8003a30 2024-04-25T10:13:08.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f390ffff640 1 -- 172.21.15.82:0/1209048736 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080a6850 con 0x7f38f8003a30 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- 172.21.15.82:0/1209048736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f8003a30 msgr2=0x7f38f8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 --2- 172.21.15.82:0/1209048736 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f38f8003a30 0x7f38f8003e50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f3908009ae0 tx=0x7f390801ca80 comp rx=0 tx=0).stop 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3915921640 1 -- 172.21.15.82:0/1209048736 reap_dead start 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- 172.21.15.82:0/1209048736 shutdown_connections 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- 172.21.15.82:0/1209048736 >> 172.21.15.82:0/1209048736 conn(0x7f3910099a70 msgr2=0x7f39100d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- 172.21.15.82:0/1209048736 shutdown_connections 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- 172.21.15.82:0/1209048736 wait complete. 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 Processor -- start 2024-04-25T10:13:08.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.651+0000 7f3917384640 1 -- start start 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38f8003a30 0x7f39100b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39100b1440 0x7f39100c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100701b0 con 0x7f39100c7bf0 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3910070030 con 0x7f39100b1440 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3910162290 con 0x7f38f8003a30 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46110/0 (socket says 172.21.15.82:46110) 2024-04-25T10:13:08.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 -- 172.21.15.82:0/913841094 learned_addr learned my addr 172.21.15.82:0/913841094 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915120640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38f8003a30 0x7f39100b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 -- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38f8003a30 msgr2=0x7f39100b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38f8003a30 0x7f39100b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f391491f640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39100b1440 0x7f39100c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 -- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39100b1440 msgr2=0x7f39100c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39100b1440 0x7f39100c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 -- 172.21.15.82:0/913841094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100cbdf0 con 0x7f39100c7bf0 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915921640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f391006bd70 tx=0x7f390000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3900007440 con 0x7f39100c7bf0 2024-04-25T10:13:08.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39000075e0 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39100b1c30 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39000049f0 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100b21f0 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39100b28b0 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f390001b020 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 0x7f38fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f3900084db0 con 0x7f39100c7bf0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915120640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 0x7f38fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.655+0000 7f3915120640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 0x7f38fc069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f39080ad000 tx=0x7f39080b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:08.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.659+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39100b28b0 con 0x7f39100c7bf0 2024-04-25T10:13:08.758 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:08.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.819+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f391004efa0 con 0x7f39100c7bf0 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f390dffb640 1 -- 172.21.15.82:0/913841094 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17203 (secure 0 0 0) 0x7f3900089070 con 0x7f39100c7bf0 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":111,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:07.712099+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6836/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6837/3162787996":"2024-04-26T10:12:31.253838+0000","172.21.15.82:6836/3162787996":"2024-04-26T10:12:31.253838+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:08.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 msgr2=0x7f38fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 0x7f38fc069450 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f39080ad000 tx=0x7f39080b2040 comp rx=0 tx=0).stop 2024-04-25T10:13:08.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 msgr2=0x7f39100c8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:08.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f391006bd70 tx=0x7f390000d9d0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 shutdown_connections 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f38fc066f90 0x7f38fc069450 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f39100c7bf0 0x7f39100c8070 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f39100b1440 0x7f39100c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 --2- 172.21.15.82:0/913841094 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f38f8003a30 0x7f39100b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 >> 172.21.15.82:0/913841094 conn(0x7f3910099a70 msgr2=0x7f39100ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 shutdown_connections 2024-04-25T10:13:08.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:08.823+0000 7f3917384640 1 -- 172.21.15.82:0/913841094 wait complete. 2024-04-25T10:13:08.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:27.097350+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:27.504961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:08.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:09.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.003+0000 7ffb3b5c7640 1 Processor -- start 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.003+0000 7ffb3b5c7640 1 -- start start 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb3b5c7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffb3415b6b0 0x7ffb340ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb3b5c7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb34073010 0x7ffb340cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb3b5c7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb340cbd20 con 0x7ffb340cf590 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb3b5c7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3415ba90 con 0x7ffb34073010 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb3b5c7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb340cc000 con 0x7ffb3415b6b0 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb34073010 0x7ffb340cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb34073010 0x7ffb340cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46112/0 (socket says 172.21.15.82:46112) 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 -- 172.21.15.82:0/819617881 learned_addr learned my addr 172.21.15.82:0/819617881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 -- 172.21.15.82:0/819617881 >> v2:172.21.15.196:3300/0 conn(0x7ffb3415b6b0 msgr2=0x7ffb340ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39363640 1 --2- 172.21.15.82:0/819617881 >> v2:172.21.15.196:3300/0 conn(0x7ffb3415b6b0 0x7ffb340ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 --2- 172.21.15.82:0/819617881 >> v2:172.21.15.196:3300/0 conn(0x7ffb3415b6b0 0x7ffb340ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 -- 172.21.15.82:0/819617881 >> v1:172.21.15.196:6789/0 conn(0x7ffb340cf590 legacy=0x7ffb3415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 -- 172.21.15.82:0/819617881 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb340cc2e0 con 0x7ffb34073010 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb39b64640 1 --2- 172.21.15.82:0/819617881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340cfa20 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ffb340d0180 tx=0x7ffb2c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c7d2818a7f623 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb27fff640 1 -- 172.21.15.82:0/819617881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0a5680 con 0x7ffb34073010 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb27fff640 1 -- 172.21.15.82:0/819617881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c0a5820 con 0x7ffb34073010 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.007+0000 7ffb27fff640 1 -- 172.21.15.82:0/819617881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0a5bf0 con 0x7ffb34073010 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/819617881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 msgr2=0x7ffb340cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/819617881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340cfa20 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7ffb340d0180 tx=0x7ffb2c01c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/819617881 shutdown_connections 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/819617881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340cfa20 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/819617881 >> v2:172.21.15.196:3300/0 conn(0x7ffb3415b6b0 0x7ffb340ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/819617881 >> 172.21.15.82:0/819617881 conn(0x7ffb3406ae60 msgr2=0x7ffb3406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/819617881 shutdown_connections 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/819617881 wait complete. 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 Processor -- start 2024-04-25T10:13:09.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- start start 2024-04-25T10:13:09.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb340cf590 0x7ffb340980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 0x7ffb34098610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340cadb0 con 0x7ffb34073010 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340ca900 con 0x7ffb3415b6b0 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb3b5c7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb340caf30 con 0x7ffb340cf590 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46114/0 (socket says 172.21.15.82:46114) 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 -- 172.21.15.82:0/3207829859 learned_addr learned my addr 172.21.15.82:0/3207829859 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb38b62640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb340cf590 0x7ffb340980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 -- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb340cf590 msgr2=0x7ffb340980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb340cf590 0x7ffb340980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39b64640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 0x7ffb34098610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 -- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 msgr2=0x7ffb34098610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 0x7ffb34098610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 -- 172.21.15.82:0/3207829859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34152a20 con 0x7ffb34073010 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39b64640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 0x7ffb34098610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb39363640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800c7f0 tx=0x7ffb2800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28015070 con 0x7ffb34073010 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb28007820 con 0x7ffb34073010 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.011+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb28007c10 con 0x7ffb34073010 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb34152cc0 con 0x7ffb34073010 2024-04-25T10:13:09.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3415aea0 con 0x7ffb34073010 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb34153440 con 0x7ffb34073010 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb28018070 con 0x7ffb34073010 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb25ffb640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 0x7ffb18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7ffb28085300 con 0x7ffb34073010 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb38b62640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 0x7ffb18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.015+0000 7ffb38b62640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 0x7ffb18069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffb3414e9a0 tx=0x7ffb30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.019+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb34153440 con 0x7ffb34073010 2024-04-25T10:13:09.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.167+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb3404efa0 con 0x7ffb34073010 2024-04-25T10:13:09.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb25ffb640 1 -- 172.21.15.82:0/3207829859 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v111) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb280560b0 con 0x7ffb34073010 2024-04-25T10:13:09.174 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 msgr2=0x7ffb18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 0x7ffb18069420 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffb3414e9a0 tx=0x7ffb30009040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 msgr2=0x7ffb340ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800c7f0 tx=0x7ffb2800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 shutdown_connections 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb18066f60 0x7ffb18069420 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb340cf590 0x7ffb340980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb3415b6b0 0x7ffb34098610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 --2- 172.21.15.82:0/3207829859 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb34073010 0x7ffb340ce1e0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 >> 172.21.15.82:0/3207829859 conn(0x7ffb3406ae60 msgr2=0x7ffb34157a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 shutdown_connections 2024-04-25T10:13:09.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.171+0000 7ffb3b5c7640 1 -- 172.21.15.82:0/3207829859 wait complete. 2024-04-25T10:13:09.191 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:09.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.311+0000 7f60eb9a5640 1 Processor -- start 2024-04-25T10:13:09.317 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.311+0000 7f60eb9a5640 1 -- start start 2024-04-25T10:13:09.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60eb9a5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60e406fc50 0x7f60e406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60eb9a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e406b1d0 0x7f60e40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60eb9a5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e406b710 con 0x7f60e40cfd80 2024-04-25T10:13:09.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60eb9a5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e4161560 con 0x7f60e406fc50 2024-04-25T10:13:09.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60eb9a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e41617b0 con 0x7f60e406b1d0 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60e406fc50 0x7f60e406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f60e406fc50 0x7f60e406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46116/0 (socket says 172.21.15.82:46116) 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 -- 172.21.15.82:0/1963663829 learned_addr learned my addr 172.21.15.82:0/1963663829 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 -- 172.21.15.82:0/1963663829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e406b1d0 msgr2=0x7f60e40cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 --2- 172.21.15.82:0/1963663829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e406b1d0 0x7f60e40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 -- 172.21.15.82:0/1963663829 >> v1:172.21.15.196:6789/0 conn(0x7f60e40cfd80 legacy=0x7f60e415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 -- 172.21.15.82:0/1963663829 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4161a90 con 0x7f60e406fc50 2024-04-25T10:13:09.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60e9741640 1 --2- 172.21.15.82:0/1963663829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406fc50 0x7f60e406ac90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009870 tx=0x7f60d401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cf625c8b4881cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60dbfff640 1 -- 172.21.15.82:0/1963663829 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b1070 con 0x7f60e406fc50 2024-04-25T10:13:09.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60dbfff640 1 -- 172.21.15.82:0/1963663829 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d40ac620 con 0x7f60e406fc50 2024-04-25T10:13:09.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.315+0000 7f60dbfff640 1 -- 172.21.15.82:0/1963663829 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40ac9f0 con 0x7f60e406fc50 2024-04-25T10:13:09.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/1963663829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406fc50 msgr2=0x7f60e406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/1963663829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406fc50 0x7f60e406ac90 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f60d4009870 tx=0x7f60d401c4f0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/1963663829 shutdown_connections 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/1963663829 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e406b1d0 0x7f60e40cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/1963663829 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406fc50 0x7f60e406ac90 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/1963663829 >> 172.21.15.82:0/1963663829 conn(0x7f60e4099a70 msgr2=0x7f60e40d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/1963663829 shutdown_connections 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/1963663829 wait complete. 2024-04-25T10:13:09.322 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 Processor -- start 2024-04-25T10:13:09.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- start start 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 0x7f60e40c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 0x7f60e4151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e40701b0 con 0x7f60e406b1d0 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4070030 con 0x7f60e406fc50 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4162290 con 0x7f60e40cfd80 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46118/0 (socket says 172.21.15.82:46118) 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 -- 172.21.15.82:0/893892266 learned_addr learned my addr 172.21.15.82:0/893892266 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9f42640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 0x7f60e4151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e8f40640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 0x7f60e40c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 -- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 msgr2=0x7f60e4151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 0x7f60e4151cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 -- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 msgr2=0x7f60e40c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 0x7f60e40c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 -- 172.21.15.82:0/893892266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e40c9480 con 0x7f60e406b1d0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9f42640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 0x7f60e4151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e8f40640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 0x7f60e40c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60e9741640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f60d401aba0 tx=0x7f60d401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b1040 con 0x7f60e406b1d0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d40bd070 con 0x7f60e406b1d0 2024-04-25T10:13:09.326 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e41477c0 con 0x7f60e406b1d0 2024-04-25T10:13:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40b7880 con 0x7f60e406b1d0 2024-04-25T10:13:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e4158ef0 con 0x7f60e406b1d0 2024-04-25T10:13:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.319+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e40b0360 con 0x7f60e406b1d0 2024-04-25T10:13:09.327 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.323+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60d40b7a20 con 0x7f60e406b1d0 2024-04-25T10:13:09.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.323+0000 7f60d9ffb640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 0x7f60c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.323+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f60d411f820 con 0x7f60e406b1d0 2024-04-25T10:13:09.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.323+0000 7f60e8f40640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 0x7f60c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.328 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.323+0000 7f60e8f40640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 0x7f60c8069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f60e406ad40 tx=0x7f60dc0073c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.330 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.327+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60e40b0360 con 0x7f60e406b1d0 2024-04-25T10:13:09.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f60e40b6480 con 0x7f60e406b1d0 2024-04-25T10:13:09.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60d9ffb640 1 -- 172.21.15.82:0/893892266 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v111) v1 ==== 104+0+11 (secure 0 0 0) 0x7f60e41477c0 con 0x7f60e406b1d0 2024-04-25T10:13:09.479 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:09.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 msgr2=0x7f60c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 0x7f60c8069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f60e406ad40 tx=0x7f60dc0073c0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 msgr2=0x7f60e4147280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f60d401aba0 tx=0x7f60d401ad30 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 shutdown_connections 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f60c8066f90 0x7f60c8069450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f60e40cfd80 0x7f60e4151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f60e406fc50 0x7f60e40c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 --2- 172.21.15.82:0/893892266 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f60e406b1d0 0x7f60e4147280 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 >> 172.21.15.82:0/893892266 conn(0x7f60e4099a70 msgr2=0x7f60e4073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 shutdown_connections 2024-04-25T10:13:09.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.475+0000 7f60eb9a5640 1 -- 172.21.15.82:0/893892266 wait complete. 2024-04-25T10:13:09.497 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:09.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.607+0000 7fba7ddea640 1 Processor -- start 2024-04-25T10:13:09.613 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 -- start start 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba7815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba780cf970 0x7fba780c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba780ca4c0 con 0x7fba780c9ef0 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba780ca780 con 0x7fba780cf970 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba7ddea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba780cd440 con 0x7fba780cf590 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba77fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fba780c9ef0 0x7fba780b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59926/0 (socket says 172.21.15.82:59926) 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba77fff640 1 -- 172.21.15.82:0/3294331304 learned_addr learned my addr 172.21.15.82:0/3294331304 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:09.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba76ffd640 1 --2- 172.21.15.82:0/3294331304 >> v2:172.21.15.82:3300/0 conn(0x7fba780cf970 0x7fba780c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba777fe640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba7815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba767fc640 1 -- 172.21.15.82:0/3294331304 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121130830 0 0) 0x7fba780ca4c0 con 0x7fba780c9ef0 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba767fc640 1 -- 172.21.15.82:0/3294331304 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba60003530 con 0x7fba780c9ef0 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba76ffd640 1 -- 172.21.15.82:0/3294331304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 msgr2=0x7fba7815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba76ffd640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba7815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba76ffd640 1 -- 172.21.15.82:0/3294331304 >> v1:172.21.15.82:6789/0 conn(0x7fba780c9ef0 legacy=0x7fba780b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.616 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.611+0000 7fba76ffd640 1 -- 172.21.15.82:0/3294331304 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780cd690 con 0x7fba780cf970 2024-04-25T10:13:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba76ffd640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c99b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097c0 tx=0x7fba6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf9b020054fa566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba767fc640 1 -- 172.21.15.82:0/3294331304 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0b0070 con 0x7fba780cf970 2024-04-25T10:13:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba767fc640 1 -- 172.21.15.82:0/3294331304 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c0ab600 con 0x7fba780cf970 2024-04-25T10:13:09.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba767fc640 1 -- 172.21.15.82:0/3294331304 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c0ab9d0 con 0x7fba780cf970 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/3294331304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 msgr2=0x7fba780c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c99b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0097c0 tx=0x7fba6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/3294331304 shutdown_connections 2024-04-25T10:13:09.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c99b0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- 172.21.15.82:0/3294331304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba7815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/3294331304 >> 172.21.15.82:0/3294331304 conn(0x7fba7806ae60 msgr2=0x7fba7806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/3294331304 shutdown_connections 2024-04-25T10:13:09.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/3294331304 wait complete. 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 Processor -- start 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- start start 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 0x7fba78154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba780c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba78165650 con 0x7fba780cf970 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba781654d0 con 0x7fba780c9ef0 2024-04-25T10:13:09.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba781657d0 con 0x7fba780cf590 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46126/0 (socket says 172.21.15.82:46126) 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 -- 172.21.15.82:0/2063732772 learned_addr learned my addr 172.21.15.82:0/2063732772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba76ffd640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba780c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba777fe640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 0x7fba78154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 -- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 msgr2=0x7fba780c6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba780c6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 -- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 msgr2=0x7fba78154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 0x7fba78154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 -- 172.21.15.82:0/2063732772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78155cc0 con 0x7fba780cf970 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba777fe640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 0x7fba78154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba77fff640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fba7000cb70 tx=0x7fba7000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70017070 con 0x7fba780cf970 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7000ad40 con 0x7fba780cf970 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba70012620 con 0x7fba780cf970 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7806b780 con 0x7fba780cf970 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.615+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba780c70f0 con 0x7fba780cf970 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7804efa0 con 0x7fba780cf970 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba7001a070 con 0x7fba780cf970 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba5ffff640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 0x7fba64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fba70085130 con 0x7fba780cf970 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba777fe640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 0x7fba64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.619+0000 7fba777fe640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 0x7fba64069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.623+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba70055e50 con 0x7fba780cf970 2024-04-25T10:13:09.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba780b4170 con 0x7fba780cf970 2024-04-25T10:13:09.786 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba5ffff640 1 -- 172.21.15.82:0/2063732772 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v111) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba70055750 con 0x7fba780cf970 2024-04-25T10:13:09.786 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:09.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 msgr2=0x7fba64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 0x7fba64069450 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fba68009ae0 tx=0x7fba68008040 comp rx=0 tx=0).stop 2024-04-25T10:13:09.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 msgr2=0x7fba780c7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.788 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fba7000cb70 tx=0x7fba7000a430 comp rx=0 tx=0).stop 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 shutdown_connections 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f90 0x7fba64069450 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba780cf590 0x7fba780c6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba780cf970 0x7fba780c7eb0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 --2- 172.21.15.82:0/2063732772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba780c9ef0 0x7fba78154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 >> 172.21.15.82:0/2063732772 conn(0x7fba7806ae60 msgr2=0x7fba7806e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 shutdown_connections 2024-04-25T10:13:09.789 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.783+0000 7fba7ddea640 1 -- 172.21.15.82:0/2063732772 wait complete. 2024-04-25T10:13:09.805 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:09.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.911+0000 7f35d1df2640 1 Processor -- start 2024-04-25T10:13:09.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.911+0000 7f35d1df2640 1 -- start start 2024-04-25T10:13:09.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35cc0b3f50 0x7f35cc15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 0x7f35cc0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35cc0b2090 con 0x7f35cc15a700 2024-04-25T10:13:09.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc15f200 con 0x7f35cc0b3f50 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35cc0b43e0 con 0x7f35cc0b24a0 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35cc0b3f50 0x7f35cc15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f35cc0b3f50 0x7f35cc15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46140/0 (socket says 172.21.15.82:46140) 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 -- 172.21.15.82:0/1400600258 learned_addr learned my addr 172.21.15.82:0/1400600258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35cbfff640 1 --2- 172.21.15.82:0/1400600258 >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 0x7f35cc0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 -- 172.21.15.82:0/1400600258 >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 msgr2=0x7f35cc0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 --2- 172.21.15.82:0/1400600258 >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 0x7f35cc0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 -- 172.21.15.82:0/1400600258 >> v1:172.21.15.196:6789/0 conn(0x7f35cc15a700 legacy=0x7f35cc0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 -- 172.21.15.82:0/1400600258 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc0b46c0 con 0x7f35cc0b3f50 2024-04-25T10:13:09.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35cbfff640 1 --2- 172.21.15.82:0/1400600258 >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 0x7f35cc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35caffd640 1 --2- 172.21.15.82:0/1400600258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc0b3f50 0x7f35cc15ecc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0b3e20 tx=0x7f35b401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149d13db569166bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35ca7fc640 1 -- 172.21.15.82:0/1400600258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40b0070 con 0x7f35cc0b3f50 2024-04-25T10:13:09.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35ca7fc640 1 -- 172.21.15.82:0/1400600258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40ab600 con 0x7f35cc0b3f50 2024-04-25T10:13:09.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35ca7fc640 1 -- 172.21.15.82:0/1400600258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40ab9d0 con 0x7f35cc0b3f50 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- 172.21.15.82:0/1400600258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc0b3f50 msgr2=0x7f35cc15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 --2- 172.21.15.82:0/1400600258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc0b3f50 0x7f35cc15ecc0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f35cc0b3e20 tx=0x7f35b401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- 172.21.15.82:0/1400600258 shutdown_connections 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 --2- 172.21.15.82:0/1400600258 >> v2:172.21.15.196:3300/0 conn(0x7f35cc0b24a0 0x7f35cc0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 --2- 172.21.15.82:0/1400600258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc0b3f50 0x7f35cc15ecc0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- 172.21.15.82:0/1400600258 >> 172.21.15.82:0/1400600258 conn(0x7f35cc0b11a0 msgr2=0x7f35cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:09.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- 172.21.15.82:0/1400600258 shutdown_connections 2024-04-25T10:13:09.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.915+0000 7f35d1df2640 1 -- 172.21.15.82:0/1400600258 wait complete. 2024-04-25T10:13:09.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 Processor -- start 2024-04-25T10:13:09.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- start start 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35cc0b24a0 0x7f35cc13f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 0x7f35cc0a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0cd310 con 0x7f35cc15a700 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0ce600 con 0x7f35cc0b24a0 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35cc0cd490 con 0x7f35cc1592b0 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46144/0 (socket says 172.21.15.82:46144) 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 -- 172.21.15.82:0/4178403384 learned_addr learned my addr 172.21.15.82:0/4178403384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35cb7fe640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35cc0b24a0 0x7f35cc13f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35cbfff640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 0x7f35cc0a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 -- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 msgr2=0x7f35cc0a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 0x7f35cc0a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 -- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35cc0b24a0 msgr2=0x7f35cc13f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35cc0b24a0 0x7f35cc13f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 -- 172.21.15.82:0/4178403384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc154000 con 0x7f35cc15a700 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35cbfff640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 0x7f35cc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35caffd640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f35b401aa50 tx=0x7f35b401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40ba020 con 0x7f35cc15a700 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b40b6620 con 0x7f35cc15a700 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35cc0a92b0 con 0x7f35cc15a700 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b40b0040 con 0x7f35cc15a700 2024-04-25T10:13:09.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.919+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35cc140420 con 0x7f35cc15a700 2024-04-25T10:13:09.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc1596c0 con 0x7f35cc15a700 2024-04-25T10:13:09.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f35b40bd020 con 0x7f35cc15a700 2024-04-25T10:13:09.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35c3fff640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 0x7f35ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:09.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f35b411f4d0 con 0x7f35cc15a700 2024-04-25T10:13:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35cb7fe640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 0x7f35ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:09.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.923+0000 7f35cb7fe640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 0x7f35ac069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35bc004e30 tx=0x7f35bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:09.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:09.927+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f35cc1596c0 con 0x7f35cc15a700 2024-04-25T10:13:10.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.103+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35cc04efa0 con 0x7f35cc15a700 2024-04-25T10:13:10.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.103+0000 7f35c3fff640 1 -- 172.21.15.82:0/4178403384 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f35b40c4030 con 0x7f35cc15a700 2024-04-25T10:13:10.109 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:10.109 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1833713793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1833713793},{"type":"v1","addr":"172.21.15.82:6837","nonce":1833713793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:13:07.712073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:10.109 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 83 2024-04-25T10:13:10.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 msgr2=0x7f35ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 0x7f35ac069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f35bc004e30 tx=0x7f35bc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 msgr2=0x7f35cc158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f35b401aa50 tx=0x7f35b401abe0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 shutdown_connections 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f35ac066f60 0x7f35ac069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f35cc1592b0 0x7f35cc0a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f35cc15a700 0x7f35cc158d70 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 --2- 172.21.15.82:0/4178403384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f35cc0b24a0 0x7f35cc13f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 >> 172.21.15.82:0/4178403384 conn(0x7f35cc0b11a0 msgr2=0x7f35cc0cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 shutdown_connections 2024-04-25T10:13:10.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.107+0000 7f35d1df2640 1 -- 172.21.15.82:0/4178403384 wait complete. 2024-04-25T10:13:10.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:10.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 Processor -- start 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 -- start start 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fba500ce510 0x7fba500ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba50144ca0 0x7fba5006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba500cee30 con 0x7fba5006ec60 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba500cc2b0 con 0x7fba50144ca0 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba584e8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba5006e290 con 0x7fba500ce510 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba56a85640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fba5006ec60 0x7fba50142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59938/0 (socket says 172.21.15.82:59938) 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba56a85640 1 -- 172.21.15.82:0/3976003543 learned_addr learned my addr 172.21.15.82:0/3976003543 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fba50144ca0 0x7fba5006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba56284640 1 --2- 172.21.15.82:0/3976003543 >> v2:172.21.15.196:3300/0 conn(0x7fba500ce510 0x7fba500ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55282640 1 -- 172.21.15.82:0/3976003543 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413913753 0 0) 0x7fba500cee30 con 0x7fba5006ec60 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55282640 1 -- 172.21.15.82:0/3976003543 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba3c003530 con 0x7fba5006ec60 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 -- 172.21.15.82:0/3976003543 >> v2:172.21.15.196:3300/0 conn(0x7fba500ce510 msgr2=0x7fba500ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 --2- 172.21.15.82:0/3976003543 >> v2:172.21.15.196:3300/0 conn(0x7fba500ce510 0x7fba500ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 -- 172.21.15.82:0/3976003543 >> v1:172.21.15.82:6789/0 conn(0x7fba5006ec60 legacy=0x7fba50142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 -- 172.21.15.82:0/3976003543 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba500ccc70 con 0x7fba50144ca0 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.231+0000 7fba55a83640 1 --2- 172.21.15.82:0/3976003543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba50144ca0 0x7fba5006e720 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fba50144c50 tx=0x7fba4001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a77f2ada7db9d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55282640 1 -- 172.21.15.82:0/3976003543 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400b5070 con 0x7fba50144ca0 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55282640 1 -- 172.21.15.82:0/3976003543 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba400a8970 con 0x7fba50144ca0 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55282640 1 -- 172.21.15.82:0/3976003543 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400a8d40 con 0x7fba50144ca0 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/3976003543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba50144ca0 msgr2=0x7fba5006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- 172.21.15.82:0/3976003543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba50144ca0 0x7fba5006e720 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fba50144c50 tx=0x7fba4001ec70 comp rx=0 tx=0).stop 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/3976003543 shutdown_connections 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- 172.21.15.82:0/3976003543 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba50144ca0 0x7fba5006e720 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- 172.21.15.82:0/3976003543 >> v2:172.21.15.196:3300/0 conn(0x7fba500ce510 0x7fba500ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/3976003543 >> 172.21.15.82:0/3976003543 conn(0x7fba500b11a0 msgr2=0x7fba500b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/3976003543 shutdown_connections 2024-04-25T10:13:10.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/3976003543 wait complete. 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 Processor -- start 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- start start 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5006ec60 0x7fba501e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba50144ca0 0x7fba501e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba500cc900 con 0x7fba500ce510 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5006df20 con 0x7fba5006ec60 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba50142d90 con 0x7fba50144ca0 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46174/0 (socket says 172.21.15.82:46174) 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 -- 172.21.15.82:0/4059372638 learned_addr learned my addr 172.21.15.82:0/4059372638 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:10.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 -- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba50144ca0 msgr2=0x7fba501e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba50144ca0 0x7fba501e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 -- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5006ec60 msgr2=0x7fba501e3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5006ec60 0x7fba501e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 -- 172.21.15.82:0/4059372638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba501e1cc0 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba55a83640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fba400b3040 tx=0x7fba4001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400bf070 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba400b0660 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba501e1fa0 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba400b5040 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba501e2560 con 0x7fba500ce510 2024-04-25T10:13:10.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.235+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba5004efa0 con 0x7fba500ce510 2024-04-25T10:13:10.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.239+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba400c2070 con 0x7fba500ce510 2024-04-25T10:13:10.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.239+0000 7fba46ffd640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 0x7fba34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.239+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fba40121920 con 0x7fba500ce510 2024-04-25T10:13:10.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.239+0000 7fba56284640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 0x7fba34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.239+0000 7fba56284640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 0x7fba34069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006aa0 tx=0x7fba4c006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.243+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba400187d0 con 0x7fba500ce510 2024-04-25T10:13:10.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.391+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba500b1cc0 con 0x7fba500ce510 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.391+0000 7fba46ffd640 1 -- 172.21.15.82:0/4059372638 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17203 (secure 0 0 0) 0x7fba400f26e0 con 0x7fba500ce510 2024-04-25T10:13:10.396 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:10.397 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":111,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:07.712099+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6836/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6837/3162787996":"2024-04-26T10:12:31.253838+0000","172.21.15.82:6836/3162787996":"2024-04-26T10:12:31.253838+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 msgr2=0x7fba34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 0x7fba34069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fba4c006aa0 tx=0x7fba4c006a30 comp rx=0 tx=0).stop 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 msgr2=0x7fba501328d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fba400b3040 tx=0x7fba4001ba10 comp rx=0 tx=0).stop 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 shutdown_connections 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba34066f60 0x7fba34069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba50144ca0 0x7fba501e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba500ce510 0x7fba501328d0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 --2- 172.21.15.82:0/4059372638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba5006ec60 0x7fba501e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 >> 172.21.15.82:0/4059372638 conn(0x7fba500b11a0 msgr2=0x7fba500af5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 shutdown_connections 2024-04-25T10:13:10.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.395+0000 7fba584e8640 1 -- 172.21.15.82:0/4059372638 wait complete. 2024-04-25T10:13:10.415 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:10.416 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:10.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 Processor -- start 2024-04-25T10:13:10.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 -- start start 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92e00ce510 0x7f92e00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92e0144ca0 0x7f92e006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00cee30 con 0x7f92e006ec60 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e00cc2b0 con 0x7f92e00ce510 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e80dd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e006e290 con 0x7f92e0144ca0 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92e00ce510 0x7f92e00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92e00ce510 0x7f92e00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46184/0 (socket says 172.21.15.82:46184) 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 -- 172.21.15.82:0/1791567381 learned_addr learned my addr 172.21.15.82:0/1791567381 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5678640 1 --2- 172.21.15.82:0/1791567381 >> v2:172.21.15.196:3300/0 conn(0x7f92e0144ca0 0x7f92e006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 -- 172.21.15.82:0/1791567381 >> v2:172.21.15.196:3300/0 conn(0x7f92e0144ca0 msgr2=0x7f92e006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 --2- 172.21.15.82:0/1791567381 >> v2:172.21.15.196:3300/0 conn(0x7f92e0144ca0 0x7f92e006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 -- 172.21.15.82:0/1791567381 >> v1:172.21.15.196:6789/0 conn(0x7f92e006ec60 legacy=0x7f92e0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.519+0000 7f92e5e79640 1 -- 172.21.15.82:0/1791567381 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00ccc70 con 0x7f92e00ce510 2024-04-25T10:13:10.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e5e79640 1 --2- 172.21.15.82:0/1791567381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e00ce510 0x7f92e00ce8f0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92e0070fe0 tx=0x7f92dc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad3268e8dc4d054 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e4e77640 1 -- 172.21.15.82:0/1791567381 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0b5070 con 0x7f92e00ce510 2024-04-25T10:13:10.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e4e77640 1 -- 172.21.15.82:0/1791567381 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0a89d0 con 0x7f92e00ce510 2024-04-25T10:13:10.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e4e77640 1 -- 172.21.15.82:0/1791567381 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc0a8da0 con 0x7f92e00ce510 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- 172.21.15.82:0/1791567381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e00ce510 msgr2=0x7f92e00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1791567381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e00ce510 0x7f92e00ce8f0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f92e0070fe0 tx=0x7f92dc01ecd0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- 172.21.15.82:0/1791567381 shutdown_connections 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1791567381 >> v2:172.21.15.196:3300/0 conn(0x7f92e0144ca0 0x7f92e006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1791567381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e00ce510 0x7f92e00ce8f0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- 172.21.15.82:0/1791567381 >> 172.21.15.82:0/1791567381 conn(0x7f92e00b11a0 msgr2=0x7f92e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- 172.21.15.82:0/1791567381 shutdown_connections 2024-04-25T10:13:10.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- 172.21.15.82:0/1791567381 wait complete. 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 Processor -- start 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- start start 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92e006ec60 0x7f92e015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e00ce510 0x7f92e0071030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e00cc900 con 0x7f92e0144ca0 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e006df20 con 0x7f92e00ce510 2024-04-25T10:13:10.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.523+0000 7f92e80dd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0142d90 con 0x7f92e006ec60 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46194/0 (socket says 172.21.15.82:46194) 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 -- 172.21.15.82:0/1237256262 learned_addr learned my addr 172.21.15.82:0/1237256262 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e5e79640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92e006ec60 0x7f92e015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e5678640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e00ce510 0x7f92e0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 -- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92e006ec60 msgr2=0x7f92e015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92e006ec60 0x7f92e015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 -- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e00ce510 msgr2=0x7f92e0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e00ce510 0x7f92e0071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 -- 172.21.15.82:0/1237256262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00c95f0 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e667a640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c780 tx=0x7f92d800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8004b70 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8004d10 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e00cad80 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800aa90 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e00cb340 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e00b1cc0 con 0x7f92e0144ca0 2024-04-25T10:13:10.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92d800ac30 con 0x7f92e0144ca0 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 0x7f92c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f92d8085590 con 0x7f92e0144ca0 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e5e79640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 0x7f92c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.527+0000 7f92e5e79640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 0x7f92c4069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0b2000 tx=0x7f92dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.531+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92d8020070 con 0x7f92e0144ca0 2024-04-25T10:13:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.707+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92e00afac0 con 0x7f92e0144ca0 2024-04-25T10:13:10.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.707+0000 7f92d67fc640 1 -- 172.21.15.82:0/1237256262 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f92d80562b0 con 0x7f92e0144ca0 2024-04-25T10:13:10.713 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:10.713 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1833713793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1833713793},{"type":"v1","addr":"172.21.15.82:6837","nonce":1833713793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:13:07.712073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:10.713 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 83 2024-04-25T10:13:10.714 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 msgr2=0x7f92c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 0x7f92c4069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f92dc0b2000 tx=0x7f92dc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 msgr2=0x7f92e0071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f92d800c780 tx=0x7f92d800cc40 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 shutdown_connections 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f92c4066f60 0x7f92c4069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92e0144ca0 0x7f92e0071570 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92e00ce510 0x7f92e0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 --2- 172.21.15.82:0/1237256262 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92e006ec60 0x7f92e015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 >> 172.21.15.82:0/1237256262 conn(0x7f92e00b11a0 msgr2=0x7f92e00af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 shutdown_connections 2024-04-25T10:13:10.715 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.711+0000 7f92e80dd640 1 -- 172.21.15.82:0/1237256262 wait complete. 2024-04-25T10:13:10.730 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:10.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 Processor -- start 2024-04-25T10:13:10.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 -- start start 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f0594144ca0 0x7f059406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f059406ec60 0x7f0594142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05940ccc70 con 0x7f05940ce510 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05940cc2b0 con 0x7f059406ec60 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f059a0e8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059406e290 con 0x7f0594144ca0 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f0593fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f059406ec60 0x7f0594142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f05937fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f05940ce510 0x7f05940ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59948/0 (socket says 172.21.15.82:59948) 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.831+0000 7f05937fe640 1 -- 172.21.15.82:0/3412264548 learned_addr learned my addr 172.21.15.82:0/3412264548 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f0593fff640 1 -- 172.21.15.82:0/3412264548 >> v2:172.21.15.196:3300/0 conn(0x7f0594144ca0 msgr2=0x7f059406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f0593fff640 1 --2- 172.21.15.82:0/3412264548 >> v2:172.21.15.196:3300/0 conn(0x7f0594144ca0 0x7f059406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f0593fff640 1 -- 172.21.15.82:0/3412264548 >> v1:172.21.15.82:6789/0 conn(0x7f05940ce510 legacy=0x7f05940ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f05927fc640 1 -- 172.21.15.82:0/3412264548 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231885867 0 0) 0x7f05940ccc70 con 0x7f05940ce510 2024-04-25T10:13:10.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f0593fff640 1 -- 172.21.15.82:0/3412264548 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059406dc40 con 0x7f059406ec60 2024-04-25T10:13:10.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f0593fff640 1 --2- 172.21.15.82:0/3412264548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f0594142680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0594070fe0 tx=0x7f058c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0b558cb40ff941 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f05927fc640 1 -- 172.21.15.82:0/3412264548 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0b4070 con 0x7f059406ec60 2024-04-25T10:13:10.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f05927fc640 1 -- 172.21.15.82:0/3412264548 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c0af600 con 0x7f059406ec60 2024-04-25T10:13:10.840 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f05927fc640 1 -- 172.21.15.82:0/3412264548 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c0af9d0 con 0x7f059406ec60 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 -- 172.21.15.82:0/3412264548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 msgr2=0x7f0594142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 --2- 172.21.15.82:0/3412264548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f0594142680 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f0594070fe0 tx=0x7f058c01f110 comp rx=0 tx=0).stop 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 -- 172.21.15.82:0/3412264548 shutdown_connections 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 --2- 172.21.15.82:0/3412264548 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f0594142680 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 --2- 172.21.15.82:0/3412264548 >> v2:172.21.15.196:3300/0 conn(0x7f0594144ca0 0x7f059406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 -- 172.21.15.82:0/3412264548 >> 172.21.15.82:0/3412264548 conn(0x7f05940b11a0 msgr2=0x7f05940b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:10.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 -- 172.21.15.82:0/3412264548 shutdown_connections 2024-04-25T10:13:10.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.835+0000 7f059a0e8640 1 -- 172.21.15.82:0/3412264548 wait complete. 2024-04-25T10:13:10.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 Processor -- start 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- start start 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05940ce510 0x7f05940ac540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0594144ca0 0x7f05940aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05941451f0 con 0x7f059406ec60 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05940cc900 con 0x7f05940ce510 2024-04-25T10:13:10.846 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0594145370 con 0x7f0594144ca0 2024-04-25T10:13:10.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46218/0 (socket says 172.21.15.82:46218) 2024-04-25T10:13:10.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 -- 172.21.15.82:0/2397760563 learned_addr learned my addr 172.21.15.82:0/2397760563 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 -- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0594144ca0 msgr2=0x7f05940aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0594144ca0 0x7f05940aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 -- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05940ce510 msgr2=0x7f05940ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05940ce510 0x7f05940ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 -- 172.21.15.82:0/2397760563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059415b630 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f05937fe640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f05840011f0 tx=0x7f058400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584018070 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05840074e0 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584004a10 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0594136a40 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0594137000 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059404efa0 con 0x7f059406ec60 2024-04-25T10:13:10.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.839+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0584004bb0 con 0x7f059406ec60 2024-04-25T10:13:10.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.843+0000 7f057bfff640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 0x7f0580069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:10.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.843+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f0584084560 con 0x7f059406ec60 2024-04-25T10:13:10.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.843+0000 7f0592ffd640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 0x7f0580069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:10.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.843+0000 7f0592ffd640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 0x7f0580069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f05941f6e10 tx=0x7f0588006c00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:10.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:10.847+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0584055320 con 0x7f059406ec60 2024-04-25T10:13:11.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.027+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05940b1cc0 con 0x7f059406ec60 2024-04-25T10:13:11.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.027+0000 7f057bfff640 1 -- 172.21.15.82:0/2397760563 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f0584054c20 con 0x7f059406ec60 2024-04-25T10:13:11.032 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:11.032 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":83,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1833713793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1833713793},{"type":"v1","addr":"172.21.15.82:6837","nonce":1833713793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":82,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:28.528473+0000","modified":"2024-04-25T10:13:07.712073+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":111,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:11.032 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 83 2024-04-25T10:13:11.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 msgr2=0x7f0580069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 0x7f0580069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f05941f6e10 tx=0x7f0588006c00 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 msgr2=0x7f05940ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f05840011f0 tx=0x7f058400c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 shutdown_connections 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0580066f60 0x7f0580069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0594144ca0 0x7f05940aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05940ce510 0x7f05940ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 --2- 172.21.15.82:0/2397760563 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059406ec60 0x7f05940ade20 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 >> 172.21.15.82:0/2397760563 conn(0x7f05940b11a0 msgr2=0x7f05940cb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 shutdown_connections 2024-04-25T10:13:11.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.031+0000 7f059a0e8640 1 -- 172.21.15.82:0/2397760563 wait complete. 2024-04-25T10:13:11.050 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:11.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.151+0000 7f984263f640 1 Processor -- start 2024-04-25T10:13:11.157 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.151+0000 7f984263f640 1 -- start start 2024-04-25T10:13:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f984263f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 0x7f983c166550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.159 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f984263f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c0affa0 con 0x7f983c160080 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f984263f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983c15f120 con 0x7f983c15ed40 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f984263f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0240 con 0x7f983c0cbae0 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f98337fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f983c160080 0x7f983c0c6910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59952/0 (socket says 172.21.15.82:59952) 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f98337fe640 1 -- 172.21.15.82:0/2726753037 learned_addr learned my addr 172.21.15.82:0/2726753037 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819837189 0 0) 0x7f983c0affa0 con 0x7f983c160080 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9828003530 con 0x7f983c160080 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 664178341 0 0) 0x7f9828003530 con 0x7f983c160080 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 msgr2=0x7f983c166550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 0x7f983c166550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 >> v1:172.21.15.196:6789/0 conn(0x7f983c15ed40 legacy=0x7f983c0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0b0520 con 0x7f983c160080 2024-04-25T10:13:11.160 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f98240016a0 con 0x7f983c160080 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 >> v1:172.21.15.82:6789/0 conn(0x7f983c160080 legacy=0x7f983c0c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 0x7f9828003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828004150 0x7f9828004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f98280052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0240 con 0x7f9828004e20 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f982800bd30 con 0x7f9828004150 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982800bf30 con 0x7f983c0cbae0 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f98280052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.161 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 -- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 msgr2=0x7f9828003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0cbae0 0x7f9828003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 -- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828004150 msgr2=0x7f9828004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9828004150 0x7f9828004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.162 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.155+0000 7f9840bdc640 1 -- 172.21.15.82:0/2726753037 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828003530 con 0x7f9828004e20 2024-04-25T10:13:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f9840bdc640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f98280052a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f983c0724d0 tx=0x7f983401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26c5a2f257b7681 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98340b2070 con 0x7f9828004e20 2024-04-25T10:13:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340a5c40 con 0x7f9828004e20 2024-04-25T10:13:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983b7fe640 1 -- 172.21.15.82:0/2726753037 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834006e60 con 0x7f9828004e20 2024-04-25T10:13:11.163 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/2726753037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 msgr2=0x7f98280052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 --2- 172.21.15.82:0/2726753037 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f98280052a0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f983c0724d0 tx=0x7f983401cc10 comp rx=0 tx=0).stop 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f9840bdc640 1 -- 172.21.15.82:0/2726753037 reap_dead start 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/2726753037 shutdown_connections 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/2726753037 >> 172.21.15.82:0/2726753037 conn(0x7f983c099a70 msgr2=0x7f983c0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/2726753037 shutdown_connections 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/2726753037 wait complete. 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 Processor -- start 2024-04-25T10:13:11.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- start start 2024-04-25T10:13:11.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f983c0c7550 0x7f983c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 0x7f983c15c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0c7020 con 0x7f9828004e20 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0b0ae0 con 0x7f983c0c7550 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f983c0c71a0 con 0x7f983c0c7f00 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46230/0 (socket says 172.21.15.82:46230) 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 -- 172.21.15.82:0/1197135626 learned_addr learned my addr 172.21.15.82:0/1197135626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f9840bdc640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 0x7f983c15c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 -- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 msgr2=0x7f983c15c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 0x7f983c15c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 -- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f983c0c7550 msgr2=0x7f983c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f983c0c7550 0x7f983c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 -- 172.21.15.82:0/1197135626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c0c7a10 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f9840bdc640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 0x7f983c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f983bfff640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f982c00c410 tx=0x7f982c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c017070 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c003600 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f983c0ce1d0 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c003a30 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.159+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f983c0ce790 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983c04efa0 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f982c01a070 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f98397fa640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 0x7f981c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f98337fe640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 0x7f981c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f982c0845c0 con 0x7f9828004e20 2024-04-25T10:13:11.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.163+0000 7f98337fe640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 0x7f981c069450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9824004880 tx=0x7f9824001510 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.170 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.167+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f982c055410 con 0x7f9828004e20 2024-04-25T10:13:11.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.315+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f983c0ada30 con 0x7f9828004e20 2024-04-25T10:13:11.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f98397fa640 1 -- 172.21.15.82:0/1197135626 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17203 (secure 0 0 0) 0x7f982c054d10 con 0x7f9828004e20 2024-04-25T10:13:11.321 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:11.322 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":111,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:07.712099+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:27.097350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:27.504961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6836/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6837/3162787996":"2024-04-26T10:12:31.253838+0000","172.21.15.82:6836/3162787996":"2024-04-26T10:12:31.253838+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 msgr2=0x7f981c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 0x7f981c069450 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f9824004880 tx=0x7f9824001510 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 msgr2=0x7f983c0cecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f982c00c410 tx=0x7f982c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 shutdown_connections 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f981c066f90 0x7f981c069450 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f983c0c7f00 0x7f983c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f983c0c7550 0x7f983c06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 --2- 172.21.15.82:0/1197135626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9828004e20 0x7f983c0cecb0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 >> 172.21.15.82:0/1197135626 conn(0x7f983c099a70 msgr2=0x7f983c15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 shutdown_connections 2024-04-25T10:13:11.325 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.319+0000 7f984263f640 1 -- 172.21.15.82:0/1197135626 wait complete. 2024-04-25T10:13:11.339 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:11.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 Processor -- start 2024-04-25T10:13:11.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 -- start start 2024-04-25T10:13:11.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2b80cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b8162a40 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b80d2530 con 0x7fe2b80d0510 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bee82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80d2810 con 0x7fe2b80d0130 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b7fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe2b80d1fe0 0x7fe2b8161c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59968/0 (socket says 172.21.15.82:59968) 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b7fff640 1 -- 172.21.15.82:0/1241857986 learned_addr learned my addr 172.21.15.82:0/1241857986 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bcc1e640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2b80cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167743860 0 0) 0x7fe2b8162a40 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003530 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 588808752 0 0) 0x7fe2a8003530 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 msgr2=0x7fe2b80cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2b80cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 >> v1:172.21.15.196:6789/0 conn(0x7fe2b80d0510 legacy=0x7fe2b8162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80b3160 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe2a4001680 con 0x7fe2b80d1fe0 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 >> v1:172.21.15.82:6789/0 conn(0x7fe2b80d1fe0 legacy=0x7fe2b8161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2a8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a8003fe0 0x7fe2a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80d2810 con 0x7fe2a8004d50 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800bb30 con 0x7fe2a8003fe0 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800bd30 con 0x7fe2b80d0130 2024-04-25T10:13:11.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bcc1e640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2a8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2b7fff640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a8003fe0 0x7fe2a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 -- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 msgr2=0x7fe2a8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b80d0130 0x7fe2a8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 -- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a8003fe0 msgr2=0x7fe2a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2a8003fe0 0x7fe2a8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.451+0000 7fe2bd41f640 1 -- 172.21.15.82:0/1241857986 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8003530 con 0x7fe2a8004d50 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bd41f640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0001e70 tx=0x7fe2b001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145769591c2d553e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0001a90 con 0x7fe2a8004d50 2024-04-25T10:13:11.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b0003b80 con 0x7fe2a8004d50 2024-04-25T10:13:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b77fe640 1 -- 172.21.15.82:0/1241857986 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0005080 con 0x7fe2a8004d50 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/1241857986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 msgr2=0x7fe2a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 --2- 172.21.15.82:0/1241857986 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0001e70 tx=0x7fe2b001c780 comp rx=0 tx=0).stop 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bd41f640 1 -- 172.21.15.82:0/1241857986 reap_dead start 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/1241857986 shutdown_connections 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/1241857986 >> 172.21.15.82:0/1241857986 conn(0x7fe2b808b000 msgr2=0x7fe2b80d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/1241857986 shutdown_connections 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/1241857986 wait complete. 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 Processor -- start 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- start start 2024-04-25T10:13:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b8152ac0 0x7fe2b815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b815b6d0 0x7fe2b80b8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b4c50 con 0x7fe2a8004d50 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b3720 con 0x7fe2b8152ac0 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80b4dd0 con 0x7fe2b815b6d0 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46240/0 (socket says 172.21.15.82:46240) 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 -- 172.21.15.82:0/211254690 learned_addr learned my addr 172.21.15.82:0/211254690 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 -- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b815b6d0 msgr2=0x7fe2b80b8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b815b6d0 0x7fe2b80b8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 -- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b8152ac0 msgr2=0x7fe2b815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b8152ac0 0x7fe2b815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 -- 172.21.15.82:0/211254690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80b6520 con 0x7fe2a8004d50 2024-04-25T10:13:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bcc1e640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac011610 tx=0x7fe2ac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac015870 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac015a10 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80ca230 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac015dc0 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.455+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81f4000 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b804efa0 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe2ac00f460 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2b57fa640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2b7fff640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7fe2ac085b10 con 0x7fe2a8004d50 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.459+0000 7fe2b7fff640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 0x7fe29c069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40018f0 tx=0x7fe2a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.463+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe2ac019030 con 0x7fe2a8004d50 2024-04-25T10:13:11.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.651+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe2b80b47e0 con 0x7fe2a8004d50 2024-04-25T10:13:11.735 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.731+0000 7fe2b57fa640 1 -- 172.21.15.82:0/211254690 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe2ac056960 con 0x7fe2a8004d50 2024-04-25T10:13:11.736 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 msgr2=0x7fe29c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 0x7fe29c069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40018f0 tx=0x7fe2a4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 msgr2=0x7fe2b80c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac011610 tx=0x7fe2ac011ad0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 shutdown_connections 2024-04-25T10:13:11.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe29c066f60 0x7fe29c069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe2b815b6d0 0x7fe2b80b8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe2b8152ac0 0x7fe2b815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 --2- 172.21.15.82:0/211254690 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe2a8004d50 0x7fe2b80c9cf0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 >> 172.21.15.82:0/211254690 conn(0x7fe2b808b000 msgr2=0x7fe2b80b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 shutdown_connections 2024-04-25T10:13:11.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.735+0000 7fe2bee82640 1 -- 172.21.15.82:0/211254690 wait complete. 2024-04-25T10:13:11.756 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.875+0000 7fafa36ea640 1 Processor -- start 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.875+0000 7fafa36ea640 1 -- start start 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa36ea640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faf9c15b6b0 0x7faf9c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa36ea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c15ff50 con 0x7faf9c0cf590 2024-04-25T10:13:11.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa36ea640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9c0cbd20 con 0x7faf9c073010 2024-04-25T10:13:11.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa36ea640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9c15ba90 con 0x7faf9c15b6b0 2024-04-25T10:13:11.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa0c85640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faf9c0cf590 0x7faf9c15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59978/0 (socket says 172.21.15.82:59978) 2024-04-25T10:13:11.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa0c85640 1 -- 172.21.15.82:0/3258882440 learned_addr learned my addr 172.21.15.82:0/3258882440 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 --2- 172.21.15.82:0/3258882440 >> v2:172.21.15.196:3300/0 conn(0x7faf9c15b6b0 0x7faf9c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570102797 0 0) 0x7faf9c15ff50 con 0x7faf9c0cf590 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf88003530 con 0x7faf9c0cf590 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795349048 0 0) 0x7faf88003530 con 0x7faf9c0cf590 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 >> v2:172.21.15.196:3300/0 conn(0x7faf9c15b6b0 msgr2=0x7faf9c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 --2- 172.21.15.82:0/3258882440 >> v2:172.21.15.196:3300/0 conn(0x7faf9c15b6b0 0x7faf9c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 >> v1:172.21.15.196:6789/0 conn(0x7faf9c073010 legacy=0x7faf9c15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0cc000 con 0x7faf9c0cf590 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7faf98001680 con 0x7faf9c0cf590 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 >> v1:172.21.15.82:6789/0 conn(0x7faf9c0cf590 legacy=0x7faf9c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 0x7faf88003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88004810 0x7faf88004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf88005240 0x7faf8800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c15ba90 con 0x7faf88003a30 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c1a0 con 0x7faf88004810 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8800c3a0 con 0x7faf88005240 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 0x7faf88003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa0c85640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88004810 0x7faf88004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1c87640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf88005240 0x7faf8800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 -- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf88005240 msgr2=0x7faf8800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf88005240 0x7faf8800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 -- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88004810 msgr2=0x7faf88004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:11.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88004810 0x7faf88004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 -- 172.21.15.82:0/3258882440 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf88003530 con 0x7faf88003a30 2024-04-25T10:13:11.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa0c85640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88004810 0x7faf88004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1c87640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf88005240 0x7faf8800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:11.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1c87640 1 -- 172.21.15.82:0/3258882440 reap_dead start 2024-04-25T10:13:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7fafa1486640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 0x7faf88003e50 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00ba30 tx=0x7faf8c01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1d6571d76a6e75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c00b600 con 0x7faf88003a30 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8c0b0070 con 0x7faf88003a30 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.879+0000 7faf93fff640 1 -- 172.21.15.82:0/3258882440 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8c0ab8a0 con 0x7faf88003a30 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 msgr2=0x7faf88003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 0x7faf88003e50 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00ba30 tx=0x7faf8c01c670 comp rx=0 tx=0).stop 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3258882440 shutdown_connections 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3258882440 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf88003a30 0x7faf88003e50 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3258882440 >> 172.21.15.82:0/3258882440 conn(0x7faf9c06ae60 msgr2=0x7faf9c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3258882440 shutdown_connections 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3258882440 wait complete. 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 Processor -- start 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- start start 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88003a30 0x7faf9c15e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf9c0c2e70 0x7faf9c0b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0cfdd0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0cba30 con 0x7faf88003a30 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c0cadb0 con 0x7faf9c0c2e70 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46262/0 (socket says 172.21.15.82:46262) 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 -- 172.21.15.82:0/3351632275 learned_addr learned my addr 172.21.15.82:0/3351632275 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa1486640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88003a30 0x7faf9c15e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa1c87640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf9c0c2e70 0x7faf9c0b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 -- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf9c0c2e70 msgr2=0x7faf9c0b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf9c0c2e70 0x7faf9c0b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 -- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88003a30 msgr2=0x7faf9c15e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88003a30 0x7faf9c15e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:11.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 -- 172.21.15.82:0/3351632275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9c0c4e00 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa0c85640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7faf98001000 tx=0x7faf9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98019070 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf980074e0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c0c50e0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf980049f0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.883+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c0cd0b0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c04efa0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf9801c070 con 0x7faf9c0c48b0 2024-04-25T10:13:11.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7faf91ffb640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 0x7faf80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:11.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7faf98084cb0 con 0x7faf9c0c48b0 2024-04-25T10:13:11.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7fafa1486640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 0x7faf80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:11.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.887+0000 7fafa1486640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 0x7faf80069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0af000 tx=0x7faf8c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:11.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:11.891+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf98055b00 con 0x7faf9c0c48b0 2024-04-25T10:13:12.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.071+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faf9c071270 con 0x7faf9c0c48b0 2024-04-25T10:13:12.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7faf91ffb640 1 -- 172.21.15.82:0/3351632275 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7faf98055400 con 0x7faf9c0c48b0 2024-04-25T10:13:12.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 msgr2=0x7faf80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 0x7faf80069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7faf8c0af000 tx=0x7faf8c0046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 msgr2=0x7faf9c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7faf98001000 tx=0x7faf9800dd40 comp rx=0 tx=0).stop 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 shutdown_connections 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faf80066f60 0x7faf80069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faf9c0c2e70 0x7faf9c0b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faf9c0c48b0 0x7faf9c0ce8f0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 --2- 172.21.15.82:0/3351632275 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faf88003a30 0x7faf9c15e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 >> 172.21.15.82:0/3351632275 conn(0x7faf9c06ae60 msgr2=0x7faf9c06e640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 shutdown_connections 2024-04-25T10:13:12.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.735+0000 7fafa36ea640 1 -- 172.21.15.82:0/3351632275 wait complete. 2024-04-25T10:13:12.763 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:13:12.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 Processor -- start 2024-04-25T10:13:12.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 -- start start 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72580ae6c0 0x7f72580b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725815e0f0 con 0x7f725815a3c0 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72580b24a0 con 0x7f72580cf510 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f7260315640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f725815b0b0 con 0x7f72580ae6c0 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725d8b0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f72580ae6c0 0x7f72580b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725e0b1640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f725815a3c0 0x7f725815a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59994/0 (socket says 172.21.15.82:59994) 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725e0b1640 1 -- 172.21.15.82:0/2303966284 learned_addr learned my addr 172.21.15.82:0/2303966284 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725d8b0640 1 -- 172.21.15.82:0/2303966284 >> v1:172.21.15.196:6789/0 conn(0x7f72580cf510 legacy=0x7f725815e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725d8b0640 1 -- 172.21.15.82:0/2303966284 >> v1:172.21.15.82:6789/0 conn(0x7f725815a3c0 legacy=0x7f725815a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.887+0000 7f725d8b0640 1 -- 172.21.15.82:0/2303966284 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725815b390 con 0x7f72580ae6c0 2024-04-25T10:13:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f725d8b0640 1 --2- 172.21.15.82:0/2303966284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580b27a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f724c00ba60 tx=0x7f724c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4879f6245c36e358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f725d0af640 1 -- 172.21.15.82:0/2303966284 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0b4070 con 0x7f72580ae6c0 2024-04-25T10:13:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f725d0af640 1 -- 172.21.15.82:0/2303966284 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c0a7a10 con 0x7f72580ae6c0 2024-04-25T10:13:12.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f725d0af640 1 -- 172.21.15.82:0/2303966284 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0a7de0 con 0x7f72580ae6c0 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 -- 172.21.15.82:0/2303966284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 msgr2=0x7f72580b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 --2- 172.21.15.82:0/2303966284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580b27a0 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f724c00ba60 tx=0x7f724c01eb30 comp rx=0 tx=0).stop 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 -- 172.21.15.82:0/2303966284 shutdown_connections 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 --2- 172.21.15.82:0/2303966284 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580b27a0 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 -- 172.21.15.82:0/2303966284 >> 172.21.15.82:0/2303966284 conn(0x7f725806ae60 msgr2=0x7f725806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:12.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 -- 172.21.15.82:0/2303966284 shutdown_connections 2024-04-25T10:13:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.891+0000 7f7260315640 1 -- 172.21.15.82:0/2303966284 wait complete. 2024-04-25T10:13:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 Processor -- start 2024-04-25T10:13:12.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- start start 2024-04-25T10:13:12.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf510 0x7f72580ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815a3c0 0x7f72580adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580b2e60 con 0x7f72580ae6c0 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580b2ce0 con 0x7f725815a3c0 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cc570 con 0x7f72580cf510 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46284/0 (socket says 172.21.15.82:46284) 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 -- 172.21.15.82:0/3896998845 learned_addr learned my addr 172.21.15.82:0/3896998845 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725d8b0640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf510 0x7f72580ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 -- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf510 msgr2=0x7f72580ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf510 0x7f72580ad5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 -- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815a3c0 msgr2=0x7f72580adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:12.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e8b2640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815a3c0 0x7f72580adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815a3c0 0x7f72580adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 -- 172.21.15.82:0/3896998845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580ac9c0 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f725e0b1640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f725400c370 tx=0x7f725400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7254017070 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7254005cf0 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72580acca0 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72540072f0 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72580ad260 con 0x7f72580ae6c0 2024-04-25T10:13:12.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7258067c30 con 0x7f72580ae6c0 2024-04-25T10:13:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.895+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f725401a070 con 0x7f72580ae6c0 2024-04-25T10:13:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.899+0000 7f724affd640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.899+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6302+0+0 (secure 0 0 0) 0x7f7254084620 con 0x7f72580ae6c0 2024-04-25T10:13:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.899+0000 7f725d8b0640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:12.903 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.899+0000 7f725d8b0640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f724c0b1000 tx=0x7f724c0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:12.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:12.903+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72540553e0 con 0x7f72580ae6c0 2024-04-25T10:13:13.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.047+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f725806edf0 con 0x7f72580ae6c0 2024-04-25T10:13:13.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.747+0000 7f724affd640 1 -- 172.21.15.82:0/3896998845 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7254054ce0 con 0x7f72580ae6c0 2024-04-25T10:13:13.753 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 msgr2=0x7f723c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f724c0b1000 tx=0x7f724c0a9070 comp rx=0 tx=0).stop 2024-04-25T10:13:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 msgr2=0x7f72580ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f725400c370 tx=0x7f725400c830 comp rx=0 tx=0).stop 2024-04-25T10:13:13.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 shutdown_connections 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f723c066f60 0x7f723c069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f72580cf510 0x7f72580ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f725815a3c0 0x7f72580adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 --2- 172.21.15.82:0/3896998845 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f72580ae6c0 0x7f72580ac170 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 >> 172.21.15.82:0/3896998845 conn(0x7f725806ae60 msgr2=0x7f72580b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 shutdown_connections 2024-04-25T10:13:13.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.755+0000 7f7260315640 1 -- 172.21.15.82:0/3896998845 wait complete. 2024-04-25T10:13:13.781 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:13.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 Processor -- start 2024-04-25T10:13:13.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 -- start start 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20780adc70 0x7f207806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f207806e9f0 0x7f20780cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20780b28c0 con 0x7f20780cfa50 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20780cfec0 con 0x7f20780adc70 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207face640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207815bb20 con 0x7f207806e9f0 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20780adc70 0x7f207806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20780adc70 0x7f207806e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46298/0 (socket says 172.21.15.82:46298) 2024-04-25T10:13:13.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 -- 172.21.15.82:0/405640220 learned_addr learned my addr 172.21.15.82:0/405640220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:13.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 -- 172.21.15.82:0/405640220 >> v2:172.21.15.196:3300/0 conn(0x7f207806e9f0 msgr2=0x7f20780cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:13.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 --2- 172.21.15.82:0/405640220 >> v2:172.21.15.196:3300/0 conn(0x7f207806e9f0 0x7f20780cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 -- 172.21.15.82:0/405640220 >> v1:172.21.15.82:6789/0 conn(0x7f20780cfa50 legacy=0x7f207815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.903+0000 7f207d86a640 1 -- 172.21.15.82:0/405640220 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20780b2b60 con 0x7f20780adc70 2024-04-25T10:13:13.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207d86a640 1 --2- 172.21.15.82:0/405640220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f207806e4b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f207806f460 tx=0x7f207001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77141e7db7a42f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207c868640 1 -- 172.21.15.82:0/405640220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700a55d0 con 0x7f20780adc70 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207c868640 1 -- 172.21.15.82:0/405640220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20700a5770 con 0x7f20780adc70 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207c868640 1 -- 172.21.15.82:0/405640220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20700a5b40 con 0x7f20780adc70 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- 172.21.15.82:0/405640220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 msgr2=0x7f207806e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 --2- 172.21.15.82:0/405640220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f207806e4b0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f207806f460 tx=0x7f207001c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- 172.21.15.82:0/405640220 shutdown_connections 2024-04-25T10:13:13.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 --2- 172.21.15.82:0/405640220 >> v2:172.21.15.196:3300/0 conn(0x7f207806e9f0 0x7f20780cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 --2- 172.21.15.82:0/405640220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f207806e4b0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- 172.21.15.82:0/405640220 >> 172.21.15.82:0/405640220 conn(0x7f207806ae60 msgr2=0x7f207806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- 172.21.15.82:0/405640220 shutdown_connections 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- 172.21.15.82:0/405640220 wait complete. 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 Processor -- start 2024-04-25T10:13:13.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 -- start start 2024-04-25T10:13:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.907+0000 7f207face640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f207806e9f0 0x7f20780c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 0x7f20780be680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b3c00 con 0x7f20780adc70 2024-04-25T10:13:13.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20780b36e0 con 0x7f207806e9f0 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2078159f80 con 0x7f20780cfa50 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46314/0 (socket says 172.21.15.82:46314) 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 -- 172.21.15.82:0/3424661385 learned_addr learned my addr 172.21.15.82:0/3424661385 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207e06b640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 0x7f20780be680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 -- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 msgr2=0x7f20780be680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 0x7f20780be680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 -- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f207806e9f0 msgr2=0x7f20780c4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:13.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d86a640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f207806e9f0 0x7f20780c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f207806e9f0 0x7f20780c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 -- 172.21.15.82:0/3424661385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20781596b0 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207e06b640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 0x7f20780be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d069640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f206800b550 tx=0x7f206800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068014070 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2068010890 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2078098ed0 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2068010c80 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20780729f0 con 0x7f20780adc70 2024-04-25T10:13:13.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2078067c30 con 0x7f20780adc70 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2068017070 con 0x7f20780adc70 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 0x7f205c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5864+0+0 (secure 0 0 0) 0x7f2068084e20 con 0x7f20780adc70 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d86a640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 0x7f205c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.911+0000 7f207d86a640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 0x7f205c069450 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f20700ae000 tx=0x7f20700a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:13.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:13.915+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20781596b0 con 0x7f20780adc70 2024-04-25T10:13:14.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.079+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f207806d760 con 0x7f20780adc70 2024-04-25T10:13:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.759+0000 7f206e7fc640 1 -- 172.21.15.82:0/3424661385 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2068010e20 con 0x7f20780adc70 2024-04-25T10:13:14.762 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.759+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 msgr2=0x7f205c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.759+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 0x7f205c069450 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f20700ae000 tx=0x7f20700a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.759+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 msgr2=0x7f2078098990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.759+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f206800b550 tx=0x7f206800ba10 comp rx=0 tx=0).stop 2024-04-25T10:13:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 shutdown_connections 2024-04-25T10:13:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f205c066f90 0x7f205c069450 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20780cfa50 0x7f20780be680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20780adc70 0x7f2078098990 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 --2- 172.21.15.82:0/3424661385 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f207806e9f0 0x7f20780c4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 >> 172.21.15.82:0/3424661385 conn(0x7f207806ae60 msgr2=0x7f207806dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 shutdown_connections 2024-04-25T10:13:14.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.763+0000 7f207face640 1 -- 172.21.15.82:0/3424661385 wait complete. 2024-04-25T10:13:14.791 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:13:14.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 Processor -- start 2024-04-25T10:13:14.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 -- start start 2024-04-25T10:13:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f56800cf590 0x7f56800cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f5680072000 0x7f5680072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56800cbd20 con 0x7f568015b6b0 2024-04-25T10:13:14.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568015ba90 con 0x7f56800cf590 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5686fba640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56800cc000 con 0x7f5680072000 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f56800cf590 0x7f56800cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f56800cf590 0x7f56800cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36532/0 (socket says 172.21.15.82:36532) 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 -- 172.21.15.82:0/2943434272 learned_addr learned my addr 172.21.15.82:0/2943434272 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f5685557640 1 --2- 172.21.15.82:0/2943434272 >> v2:172.21.15.196:3300/0 conn(0x7f5680072000 0x7f5680072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 -- 172.21.15.82:0/2943434272 >> v2:172.21.15.196:3300/0 conn(0x7f5680072000 msgr2=0x7f5680072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 --2- 172.21.15.82:0/2943434272 >> v2:172.21.15.196:3300/0 conn(0x7f5680072000 0x7f5680072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 -- 172.21.15.82:0/2943434272 >> v1:172.21.15.196:6789/0 conn(0x7f568015b6b0 legacy=0x7f56800ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.911+0000 7f567ffff640 1 -- 172.21.15.82:0/2943434272 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56800cc2e0 con 0x7f56800cf590 2024-04-25T10:13:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f567ffff640 1 --2- 172.21.15.82:0/2943434272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56800cf590 0x7f56800cfa20 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f568006f000 tx=0x7f567001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab933c53f0adae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f567f7fe640 1 -- 172.21.15.82:0/2943434272 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab610 con 0x7f56800cf590 2024-04-25T10:13:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f567f7fe640 1 -- 172.21.15.82:0/2943434272 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56700b0070 con 0x7f56800cf590 2024-04-25T10:13:14.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f567f7fe640 1 -- 172.21.15.82:0/2943434272 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56700ab9d0 con 0x7f56800cf590 2024-04-25T10:13:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 -- 172.21.15.82:0/2943434272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56800cf590 msgr2=0x7f56800cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 --2- 172.21.15.82:0/2943434272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56800cf590 0x7f56800cfa20 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f568006f000 tx=0x7f567001c520 comp rx=0 tx=0).stop 2024-04-25T10:13:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 -- 172.21.15.82:0/2943434272 shutdown_connections 2024-04-25T10:13:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 --2- 172.21.15.82:0/2943434272 >> v2:172.21.15.196:3300/0 conn(0x7f5680072000 0x7f5680072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 --2- 172.21.15.82:0/2943434272 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f56800cf590 0x7f56800cfa20 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 -- 172.21.15.82:0/2943434272 >> 172.21.15.82:0/2943434272 conn(0x7f568006ae60 msgr2=0x7f568006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 -- 172.21.15.82:0/2943434272 shutdown_connections 2024-04-25T10:13:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.915+0000 7f5686fba640 1 -- 172.21.15.82:0/2943434272 wait complete. 2024-04-25T10:13:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 Processor -- start 2024-04-25T10:13:14.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- start start 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5680072000 0x7f5680098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56800cf590 0x7f5680156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800caf30 con 0x7f568015b6b0 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56800cadb0 con 0x7f5680072000 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56801682a0 con 0x7f56800cf590 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36546/0 (socket says 172.21.15.82:36546) 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 -- 172.21.15.82:0/1977469817 learned_addr learned my addr 172.21.15.82:0/1977469817 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:14.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5684d56640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5680072000 0x7f5680098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f567ffff640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56800cf590 0x7f5680156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 -- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56800cf590 msgr2=0x7f5680156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56800cf590 0x7f5680156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 -- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5680072000 msgr2=0x7f5680098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5680072000 0x7f5680098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 -- 172.21.15.82:0/1977469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5680157760 con 0x7f568015b6b0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5685557640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f5678007e40 tx=0x7f567800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678014620 con 0x7f568015b6b0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5678019070 con 0x7f568015b6b0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5680157a40 con 0x7f568015b6b0 2024-04-25T10:13:14.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5678014a20 con 0x7f568015b6b0 2024-04-25T10:13:14.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5680157430 con 0x7f568015b6b0 2024-04-25T10:13:14.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.919+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568004efa0 con 0x7f568015b6b0 2024-04-25T10:13:14.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.923+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5678014bc0 con 0x7f568015b6b0 2024-04-25T10:13:14.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.923+0000 7f567d7fa640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 0x7f566c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:14.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.923+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f5678086660 con 0x7f568015b6b0 2024-04-25T10:13:14.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.923+0000 7f5684d56640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 0x7f566c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:14.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.923+0000 7f5684d56640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 0x7f566c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f56740015b0 tx=0x7f5674009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:14.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:14.927+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5678057800 con 0x7f568015b6b0 2024-04-25T10:13:15.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.083+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f56800ae190 con 0x7f568015b6b0 2024-04-25T10:13:15.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f567d7fa640 1 -- 172.21.15.82:0/1977469817 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5678057100 con 0x7f568015b6b0 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 msgr2=0x7f566c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 0x7f566c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f56740015b0 tx=0x7f5674009040 comp rx=0 tx=0).stop 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 msgr2=0x7f5680158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f5678007e40 tx=0x7f567800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 shutdown_connections 2024-04-25T10:13:15.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f566c066f60 0x7f566c069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f56800cf590 0x7f5680156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f568015b6b0 0x7f5680158520 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 --2- 172.21.15.82:0/1977469817 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5680072000 0x7f5680098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 >> 172.21.15.82:0/1977469817 conn(0x7f568006ae60 msgr2=0x7f5680159ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 shutdown_connections 2024-04-25T10:13:15.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.091+0000 7f5686fba640 1 -- 172.21.15.82:0/1977469817 wait complete. 2024-04-25T10:13:15.113 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T10:13:15.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 Processor -- start 2024-04-25T10:13:15.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 -- start start 2024-04-25T10:13:15.234 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58d815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f58d80b3f50 0x7f58d80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58d80b2090 con 0x7f58d80ce430 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58d815f200 con 0x7f58d80b3f50 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58dc9b8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80b43e0 con 0x7f58d815a700 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58d6d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f58d80ce430 0x7f58d80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55268/0 (socket says 172.21.15.82:55268) 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58d6d9d640 1 -- 172.21.15.82:0/632416649 learned_addr learned my addr 172.21.15.82:0/632416649 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58d659c640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58d815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.227+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/632416649 >> v2:172.21.15.196:3300/0 conn(0x7f58d80b3f50 0x7f58d80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252257481 0 0) 0x7f58d80b2090 con 0x7f58d80ce430 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58c0003530 con 0x7f58d80ce430 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 970173956 0 0) 0x7f58c0003530 con 0x7f58d80ce430 2024-04-25T10:13:15.235 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 msgr2=0x7f58d815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58d815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 >> v2:172.21.15.196:3300/0 conn(0x7f58d80b3f50 msgr2=0x7f58d80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 --2- 172.21.15.82:0/632416649 >> v2:172.21.15.196:3300/0 conn(0x7f58d80b3f50 0x7f58d80cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80b46c0 con 0x7f58d80ce430 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f58d0001a00 con 0x7f58d80ce430 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 >> v1:172.21.15.82:6789/0 conn(0x7f58d80ce430 legacy=0x7f58d80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58c0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58c0003fb0 0x7f58c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0004d50 0x7f58c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d815f200 con 0x7f58c0003fb0 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80b43e0 con 0x7f58c0004d50 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c000dd40 con 0x7f58d815a700 2024-04-25T10:13:15.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58c0003fb0 0x7f58c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d659c640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58c0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d6d9d640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0004d50 0x7f58c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 -- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 msgr2=0x7f58c0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58c0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 -- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0004d50 msgr2=0x7f58c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58c0004d50 0x7f58c00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 -- 172.21.15.82:0/632416649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c0003530 con 0x7f58c0003fb0 2024-04-25T10:13:15.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d659c640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58d815a700 0x7f58c0003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:15.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58c0003fb0 0x7f58c0004810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009ae0 tx=0x7f58cc01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dbc1746f86df0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0b2070 con 0x7f58c0003fb0 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc0a5560 con 0x7f58c0003fb0 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d559a640 1 -- 172.21.15.82:0/632416649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc0096b0 con 0x7f58c0003fb0 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58dc9b8640 1 -- 172.21.15.82:0/632416649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58c0003fb0 msgr2=0x7f58c0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/632416649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58c0003fb0 0x7f58c0004810 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f58cc009ae0 tx=0x7f58cc01ca80 comp rx=0 tx=0).stop 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58d6d9d640 1 -- 172.21.15.82:0/632416649 reap_dead start 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58dc9b8640 1 -- 172.21.15.82:0/632416649 shutdown_connections 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.231+0000 7f58dc9b8640 1 -- 172.21.15.82:0/632416649 >> 172.21.15.82:0/632416649 conn(0x7f58d80b11a0 msgr2=0x7f58d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- 172.21.15.82:0/632416649 shutdown_connections 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- 172.21.15.82:0/632416649 wait complete. 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 Processor -- start 2024-04-25T10:13:15.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- start start 2024-04-25T10:13:15.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c0003fb0 0x7f58d81fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58d814ead0 0x7f58d8155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cd490 con 0x7f58d815dff0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80cd310 con 0x7f58d814ead0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d80b2bb0 con 0x7f58c0003fb0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36554/0 (socket says 172.21.15.82:36554) 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 -- 172.21.15.82:0/2160967809 learned_addr learned my addr 172.21.15.82:0/2160967809 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d5d9b640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58d814ead0 0x7f58d8155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d659c640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c0003fb0 0x7f58d81fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 -- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c0003fb0 msgr2=0x7f58d81fa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c0003fb0 0x7f58d81fa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 -- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58d814ead0 msgr2=0x7f58d8155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58d814ead0 0x7f58d8155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 -- 172.21.15.82:0/2160967809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d815cc70 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58d6d9d640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000f90 tx=0x7f58d000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d000eb90 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d000ed30 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58d815cf50 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00146a0 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.235+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58d814ef30 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58d804efa0 con 0x7f58d815dff0 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58d0014840 con 0x7f58d815dff0 2024-04-25T10:13:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58c6ffd640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 0x7f58b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f58d0084dc0 con 0x7f58d815dff0 2024-04-25T10:13:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58d659c640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 0x7f58b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.239+0000 7f58d659c640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 0x7f58b8069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58c80099d0 tx=0x7f58c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.243+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58d0055f80 con 0x7f58d815dff0 2024-04-25T10:13:15.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.479+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f58d806f260 con 0x7f58d815dff0 2024-04-25T10:13:15.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58c6ffd640 1 -- 172.21.15.82:0/2160967809 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v505) v1 ==== 146+0+0 (secure 0 0 0) 0x7f58d0055880 con 0x7f58d815dff0 2024-04-25T10:13:15.767 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 msgr2=0x7f58b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 0x7f58b8069420 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58c80099d0 tx=0x7f58c8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 msgr2=0x7f58d815e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f58d0000f90 tx=0x7f58d000e530 comp rx=0 tx=0).stop 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 shutdown_connections 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f58b8066f60 0x7f58b8069420 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58d815dff0 0x7f58d815e470 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58d814ead0 0x7f58d8155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 --2- 172.21.15.82:0/2160967809 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58c0003fb0 0x7f58d81fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 >> 172.21.15.82:0/2160967809 conn(0x7f58d80b11a0 msgr2=0x7f58d80afb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 shutdown_connections 2024-04-25T10:13:15.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.763+0000 7f58dc9b8640 1 -- 172.21.15.82:0/2160967809 wait complete. 2024-04-25T10:13:15.789 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:13:15.789 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T10:13:15.790 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 Processor -- start 2024-04-25T10:13:15.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 -- start start 2024-04-25T10:13:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c40ce510 0x7f21c40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c4144ca0 0x7f21c406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c40cee30 con 0x7f21c406ec60 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c40cc2b0 con 0x7f21c4144ca0 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21ca127640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406e290 con 0x7f21c40ce510 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c4144ca0 0x7f21c406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f21c4144ca0 0x7f21c406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36556/0 (socket says 172.21.15.82:36556) 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 -- 172.21.15.82:0/2902894352 learned_addr learned my addr 172.21.15.82:0/2902894352 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:15.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c37fe640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c40ce510 0x7f21c40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 -- 172.21.15.82:0/2902894352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c40ce510 msgr2=0x7f21c40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c40ce510 0x7f21c40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 -- 172.21.15.82:0/2902894352 >> v1:172.21.15.196:6789/0 conn(0x7f21c406ec60 legacy=0x7f21c4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.911+0000 7f21c2ffd640 1 -- 172.21.15.82:0/2902894352 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40ccc70 con 0x7f21c4144ca0 2024-04-25T10:13:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c4144ca0 0x7f21c406e720 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b798ac63b25bca6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c27fc640 1 -- 172.21.15.82:0/2902894352 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80b5070 con 0x7f21c4144ca0 2024-04-25T10:13:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c27fc640 1 -- 172.21.15.82:0/2902894352 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b80a8a50 con 0x7f21c4144ca0 2024-04-25T10:13:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c27fc640 1 -- 172.21.15.82:0/2902894352 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80a8e20 con 0x7f21c4144ca0 2024-04-25T10:13:15.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/2902894352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c4144ca0 msgr2=0x7f21c406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c4144ca0 0x7f21c406e720 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f21b8009870 tx=0x7f21b801ed50 comp rx=0 tx=0).stop 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/2902894352 shutdown_connections 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c4144ca0 0x7f21c406e720 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- 172.21.15.82:0/2902894352 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c40ce510 0x7f21c40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/2902894352 >> 172.21.15.82:0/2902894352 conn(0x7f21c40b11a0 msgr2=0x7f21c40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/2902894352 shutdown_connections 2024-04-25T10:13:15.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/2902894352 wait complete. 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 Processor -- start 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- start start 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21c40ce510 0x7f21c41f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c4144ca0 0x7f21c41e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40cc900 con 0x7f21c406ec60 2024-04-25T10:13:15.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c406df20 con 0x7f21c40ce510 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4142d90 con 0x7f21c4144ca0 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36568/0 (socket says 172.21.15.82:36568) 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 -- 172.21.15.82:0/1733067341 learned_addr learned my addr 172.21.15.82:0/1733067341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c3fff640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c4144ca0 0x7f21c41e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21c40ce510 0x7f21c41f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 -- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c4144ca0 msgr2=0x7f21c41e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c4144ca0 0x7f21c41e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 -- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21c40ce510 msgr2=0x7f21c41f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21c40ce510 0x7f21c41f13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 -- 172.21.15.82:0/1733067341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40c9540 con 0x7f21c406ec60 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21c37fe640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d800 tx=0x7f21b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400b660 con 0x7f21c406ec60 2024-04-25T10:13:15.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b4019070 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c40c9820 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400ba00 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.915+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c40ca0c0 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c404efa0 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21b400bba0 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21abfff640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 0x7f21b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f21b40852e0 con 0x7f21c406ec60 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 0x7f21b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:15.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.919+0000 7f21c2ffd640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 0x7f21b0069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b2000 tx=0x7f21b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:15.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:15.923+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f21b40564a0 con 0x7f21c406ec60 2024-04-25T10:13:16.085 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.079+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21c40b1cc0 con 0x7f21c406ec60 2024-04-25T10:13:16.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21abfff640 1 -- 172.21.15.82:0/1733067341 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13443 (secure 0 0 0) 0x7f21b4055da0 con 0x7f21c406ec60 2024-04-25T10:13:16.087 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:16.087 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":113,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:14.755161+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6836/603032759":"2024-04-26T10:13:07.712051+0000","172.21.15.82:6837/3162787996":"2024-04-26T10:12:31.253838+0000","172.21.15.82:6836/3162787996":"2024-04-26T10:12:31.253838+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 msgr2=0x7f21b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 0x7f21b0069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b2000 tx=0x7f21b8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 msgr2=0x7f21c415adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f21b400d800 tx=0x7f21b400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 shutdown_connections 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f21b0066f60 0x7f21b0069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f21c4144ca0 0x7f21c41e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f21c40ce510 0x7f21c41f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 --2- 172.21.15.82:0/1733067341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f21c406ec60 0x7f21c415adb0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 >> 172.21.15.82:0/1733067341 conn(0x7f21c40b11a0 msgr2=0x7f21c4143ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 shutdown_connections 2024-04-25T10:13:16.090 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.083+0000 7f21ca127640 1 -- 172.21.15.82:0/1733067341 wait complete. 2024-04-25T10:13:16.107 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:16.107 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:16.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 Processor -- start 2024-04-25T10:13:16.226 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 -- start start 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f977815b6b0 0x7f97780ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.229 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f97780cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97780cbd20 con 0x7f977815f130 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977815ba90 con 0x7f977815b6b0 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f977dbda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cff30 con 0x7f97780cf590 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f977815f130 0x7f977815f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55282/0 (socket says 172.21.15.82:55282) 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 -- 172.21.15.82:0/2394454101 learned_addr learned my addr 172.21.15.82:0/2394454101 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9776ffd640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f97780cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97777fe640 1 --2- 172.21.15.82:0/2394454101 >> v2:172.21.15.196:3300/0 conn(0x7f977815b6b0 0x7f97780ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153458338 0 0) 0x7f97780cbd20 con 0x7f977815f130 2024-04-25T10:13:16.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9760003530 con 0x7f977815f130 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220387858 0 0) 0x7f9760003530 con 0x7f977815f130 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 msgr2=0x7f97780cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f97780cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 >> v2:172.21.15.196:3300/0 conn(0x7f977815b6b0 msgr2=0x7f97780ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 --2- 172.21.15.82:0/2394454101 >> v2:172.21.15.196:3300/0 conn(0x7f977815b6b0 0x7f97780ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780cc000 con 0x7f977815f130 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9770001a00 con 0x7f977815f130 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 >> v1:172.21.15.82:6789/0 conn(0x7f977815f130 legacy=0x7f977815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760003a30 0x7f9760003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.231 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f9760004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9760004d50 0x7f97600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977815ba90 con 0x7f9760004d50 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976000b9d0 con 0x7f9760003a30 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976000bb50 con 0x7f97780cf590 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9760004d50 0x7f97600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9776ffd640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f9760004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f97777fe640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760003a30 0x7f9760003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 -- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 msgr2=0x7f9760004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f97780cf590 0x7f9760004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 -- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760003a30 msgr2=0x7f9760003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9760003a30 0x7f9760003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.223+0000 7f9777fff640 1 -- 172.21.15.82:0/2394454101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9760003530 con 0x7f9760004d50 2024-04-25T10:13:16.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f9777fff640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9760004d50 0x7f97600051d0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f977000a8a0 tx=0x7f977001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9685a9d5ea56b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000a4c0 con 0x7f9760004d50 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97700a9020 con 0x7f9760004d50 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f97767fc640 1 -- 172.21.15.82:0/2394454101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97700072d0 con 0x7f9760004d50 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- 172.21.15.82:0/2394454101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9760004d50 msgr2=0x7f97600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 --2- 172.21.15.82:0/2394454101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9760004d50 0x7f97600051d0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f977000a8a0 tx=0x7f977001d3c0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f9777fff640 1 -- 172.21.15.82:0/2394454101 reap_dead start 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- 172.21.15.82:0/2394454101 shutdown_connections 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- 172.21.15.82:0/2394454101 >> 172.21.15.82:0/2394454101 conn(0x7f977806ae60 msgr2=0x7f977806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- 172.21.15.82:0/2394454101 shutdown_connections 2024-04-25T10:13:16.237 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- 172.21.15.82:0/2394454101 wait complete. 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 Processor -- start 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.227+0000 7f977dbda640 1 -- start start 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9778098780 0x7f977815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97780c4bc0 0x7f97780c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cc8a0 con 0x7f97781572a0 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cc5c0 con 0x7f97780c4bc0 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cba30 con 0x7f9778098780 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.238 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36592/0 (socket says 172.21.15.82:36592) 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 -- 172.21.15.82:0/3054754806 learned_addr learned my addr 172.21.15.82:0/3054754806 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f97777fe640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9778098780 0x7f977815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9777fff640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97780c4bc0 0x7f97780c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 -- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9778098780 msgr2=0x7f977815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9778098780 0x7f977815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 -- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97780c4bc0 msgr2=0x7f97780c5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97780c4bc0 0x7f97780c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 -- 172.21.15.82:0/3054754806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780c3880 con 0x7f97781572a0 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f9776ffd640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f976c011640 tx=0x7f976c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.239 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00f8c0 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976c00fa60 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976c00fe30 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97780c3b20 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780c40e0 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977804efa0 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f976c015460 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 0x7f9764069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f976c0859b0 con 0x7f97781572a0 2024-04-25T10:13:16.240 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f97777fe640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 0x7f9764069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.231+0000 7f97777fe640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 0x7f9764069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9768009ab0 tx=0x7f9768008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.241 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.235+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f976c019030 con 0x7f97781572a0 2024-04-25T10:13:16.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.391+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9778072000 con 0x7f97781572a0 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.391+0000 7f975ffff640 1 -- 172.21.15.82:0/3054754806 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f976c0569b0 con 0x7f97781572a0 2024-04-25T10:13:16.395 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:16.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 msgr2=0x7f9764069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 0x7f9764069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9768009ab0 tx=0x7f9768008040 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 msgr2=0x7f9778157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f976c011640 tx=0x7f976c011b00 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 shutdown_connections 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9764066f60 0x7f9764069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f97780c4bc0 0x7f97780c5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f97781572a0 0x7f9778157700 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 --2- 172.21.15.82:0/3054754806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9778098780 0x7f977815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 >> 172.21.15.82:0/3054754806 conn(0x7f977806ae60 msgr2=0x7f97780b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 shutdown_connections 2024-04-25T10:13:16.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.395+0000 7f977dbda640 1 -- 172.21.15.82:0/3054754806 wait complete. 2024-04-25T10:13:16.414 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:13:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 Processor -- start 2024-04-25T10:13:16.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 -- start start 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f477c09a180 0x7f477c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0afb90 0x7f477c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c09a950 con 0x7f477c15b920 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c09ac30 con 0x7f477c09a180 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4782653640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0ae380 con 0x7f477c0afb90 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f477c09a180 0x7f477c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f477c09a180 0x7f477c0cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36596/0 (socket says 172.21.15.82:36596) 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 -- 172.21.15.82:0/1659186646 learned_addr learned my addr 172.21.15.82:0/1659186646 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f4780bf0640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0afb90 0x7f477c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 -- 172.21.15.82:0/1659186646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0afb90 msgr2=0x7f477c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0afb90 0x7f477c0cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 -- 172.21.15.82:0/1659186646 >> v1:172.21.15.196:6789/0 conn(0x7f477c15b920 legacy=0x7f477c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.525 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.523+0000 7f477b7fe640 1 -- 172.21.15.82:0/1659186646 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0ae660 con 0x7f477c09a180 2024-04-25T10:13:16.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477b7fe640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c0cc130 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4220069d00a58acf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477affd640 1 -- 172.21.15.82:0/1659186646 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640b2070 con 0x7f477c09a180 2024-04-25T10:13:16.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477affd640 1 -- 172.21.15.82:0/1659186646 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47640a9070 con 0x7f477c09a180 2024-04-25T10:13:16.526 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477affd640 1 -- 172.21.15.82:0/1659186646 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640ad880 con 0x7f477c09a180 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/1659186646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 msgr2=0x7f477c0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c0cc130 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f476401c490 comp rx=0 tx=0).stop 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/1659186646 shutdown_connections 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c0afb90 0x7f477c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- 172.21.15.82:0/1659186646 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c0cc130 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/1659186646 >> 172.21.15.82:0/1659186646 conn(0x7f477c0b3010 msgr2=0x7f477c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/1659186646 shutdown_connections 2024-04-25T10:13:16.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/1659186646 wait complete. 2024-04-25T10:13:16.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 Processor -- start 2024-04-25T10:13:16.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- start start 2024-04-25T10:13:16.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c0afb90 0x7f477c1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c15b920 0x7f477c153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c1684c0 con 0x7f477c09a180 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0cd5d0 con 0x7f477c0afb90 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c168640 con 0x7f477c15b920 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36604/0 (socket says 172.21.15.82:36604) 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 -- 172.21.15.82:0/2025698312 learned_addr learned my addr 172.21.15.82:0/2025698312 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 -- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c15b920 msgr2=0x7f477c153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c15b920 0x7f477c153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 -- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c0afb90 msgr2=0x7f477c1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c0afb90 0x7f477c1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 -- 172.21.15.82:0/2025698312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0b6fa0 con 0x7f477c09a180 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477bfff640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f477400ba50 tx=0x7f477400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.531 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774019070 con 0x7f477c09a180 2024-04-25T10:13:16.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400cc10 con 0x7f477c09a180 2024-04-25T10:13:16.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c158cd0 con 0x7f477c09a180 2024-04-25T10:13:16.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774014440 con 0x7f477c09a180 2024-04-25T10:13:16.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c159290 con 0x7f477c09a180 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c04efa0 con 0x7f477c09a180 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f477401c070 con 0x7f477c09a180 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 0x7f475c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477b7fe640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 0x7f475c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4774084d10 con 0x7f477c09a180 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.527+0000 7f477b7fe640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 0x7f475c069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f47640af000 tx=0x7f4764004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.535+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f477408a020 con 0x7f477c09a180 2024-04-25T10:13:16.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.735+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f477c0cc3b0 con 0x7f477c09a180 2024-04-25T10:13:16.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4778ff9640 1 -- 172.21.15.82:0/2025698312 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v506) v1 ==== 147+0+0 (secure 0 0 0) 0x7f4774055e30 con 0x7f477c09a180 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 msgr2=0x7f475c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 0x7f475c069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f47640af000 tx=0x7f4764004740 comp rx=0 tx=0).stop 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 msgr2=0x7f477c156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f477400ba50 tx=0x7f477400bf10 comp rx=0 tx=0).stop 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 shutdown_connections 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f475c066f60 0x7f475c069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f477c15b920 0x7f477c153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f477c0afb90 0x7f477c1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 --2- 172.21.15.82:0/2025698312 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f477c09a180 0x7f477c156920 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 >> 172.21.15.82:0/2025698312 conn(0x7f477c0b3010 msgr2=0x7f477c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 shutdown_connections 2024-04-25T10:13:16.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.763+0000 7f4782653640 1 -- 172.21.15.82:0/2025698312 wait complete. 2024-04-25T10:13:16.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:13:16.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 Processor -- start 2024-04-25T10:13:16.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 -- start start 2024-04-25T10:13:16.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 0x7f1f940cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940cbd20 con 0x7f1f94072000 2024-04-25T10:13:16.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f940cc000 con 0x7f1f9415b6b0 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9b4e2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9415ba90 con 0x7f1f940cf590 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f99a7f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1f94072000 0x7f1f94072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55284/0 (socket says 172.21.15.82:55284) 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f99a7f640 1 -- 172.21.15.82:0/4171599294 learned_addr learned my addr 172.21.15.82:0/4171599294 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f98a7d640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 0x7f1f940cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355099510 0 0) 0x7f1f940cbd20 con 0x7f1f94072000 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f80003530 con 0x7f1f94072000 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1723238329 0 0) 0x7f1f80003530 con 0x7f1f94072000 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 msgr2=0x7f1f940cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 0x7f1f940cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 >> v1:172.21.15.196:6789/0 conn(0x7f1f9415b6b0 legacy=0x7f1f940ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f940cc2e0 con 0x7f1f94072000 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f1f8c001a00 con 0x7f1f94072000 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 >> v1:172.21.15.82:6789/0 conn(0x7f1f94072000 legacy=0x7f1f94072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f80003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f80004810 0x7f1f80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 0x7f1f800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9415ba90 con 0x7f1f80003a60 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8000bad0 con 0x7f1f80004810 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8000bcd0 con 0x7f1f940cf590 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f80003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f98a7d640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f80004810 0x7f1f80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 -- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 msgr2=0x7f1f800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f940cf590 0x7f1f800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 -- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f80004810 msgr2=0x7f1f80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f80004810 0x7f1f80004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.899+0000 7f1f9927e640 1 -- 172.21.15.82:0/4171599294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f80003530 con 0x7f1f80003a60 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9927e640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f80003e80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001780 tx=0x7f1f8401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9859222da66e232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840013a0 con 0x7f1f80003a60 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f840052f0 con 0x7f1f80003a60 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f8bfff640 1 -- 172.21.15.82:0/4171599294 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84003840 con 0x7f1f80003a60 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/4171599294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 msgr2=0x7f1f80003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/4171599294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f80003e80 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001780 tx=0x7f1f8401c3f0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f99a7f640 1 -- 172.21.15.82:0/4171599294 reap_dead start 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/4171599294 shutdown_connections 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/4171599294 >> 172.21.15.82:0/4171599294 conn(0x7f1f9406ae60 msgr2=0x7f1f9406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:16.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/4171599294 shutdown_connections 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/4171599294 wait complete. 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 Processor -- start 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- start start 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94155910 0x7f1f941596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f94159c00 0x7f1f941e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f940cff00 con 0x7f1f80003a60 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f940cfd80 con 0x7f1f94155910 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9b4e2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f940cadb0 con 0x7f1f94159c00 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9927e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9927e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36622/0 (socket says 172.21.15.82:36622) 2024-04-25T10:13:16.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f9927e640 1 -- 172.21.15.82:0/691931876 learned_addr learned my addr 172.21.15.82:0/691931876 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.903+0000 7f1f99a7f640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f94159c00 0x7f1f941e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f98a7d640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94155910 0x7f1f941596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 -- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f94159c00 msgr2=0x7f1f941e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f94159c00 0x7f1f941e9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 -- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94155910 msgr2=0x7f1f941596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94155910 0x7f1f941596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 -- 172.21.15.82:0/691931876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f940aee00 con 0x7f1f80003a60 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9927e640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001780 tx=0x7f1f80005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84007a40 con 0x7f1f80003a60 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f840bb070 con 0x7f1f80003a60 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f940af0e0 con 0x7f1f80003a60 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f840013a0 con 0x7f1f80003a60 2024-04-25T10:13:16.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f940af6a0 con 0x7f1f80003a60 2024-04-25T10:13:16.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9404efa0 con 0x7f1f80003a60 2024-04-25T10:13:16.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f840b4040 con 0x7f1f80003a60 2024-04-25T10:13:16.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 0x7f1f78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:16.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1f8411ed80 con 0x7f1f80003a60 2024-04-25T10:13:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f98a7d640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 0x7f1f78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:16.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.907+0000 7f1f98a7d640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 0x7f1f78069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009850 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:16.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:16.911+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f84016130 con 0x7f1f80003a60 2024-04-25T10:13:17.072 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.067+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f94071190 con 0x7f1f80003a60 2024-04-25T10:13:17.078 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f89ffb640 1 -- 172.21.15.82:0/691931876 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f840efeb0 con 0x7f1f80003a60 2024-04-25T10:13:17.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 msgr2=0x7f1f78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 0x7f1f78069420 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009850 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2024-04-25T10:13:17.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 msgr2=0x7f1f9415bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f1f84001780 tx=0x7f1f80005840 comp rx=0 tx=0).stop 2024-04-25T10:13:17.081 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 shutdown_connections 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1f78066f60 0x7f1f78069420 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1f94159c00 0x7f1f941e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1f94155910 0x7f1f941596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 --2- 172.21.15.82:0/691931876 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1f80003a60 0x7f1f9415bdf0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.075+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 >> 172.21.15.82:0/691931876 conn(0x7f1f9406ae60 msgr2=0x7f1f9406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.079+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 shutdown_connections 2024-04-25T10:13:17.082 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.079+0000 7f1f9b4e2640 1 -- 172.21.15.82:0/691931876 wait complete. 2024-04-25T10:13:17.097 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:17.097 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:17.139 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:17.198 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:17.198 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:17.199 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.204 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:17.204 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:13:17.205 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:17.211 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:17.211 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:17.253 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:13:17.254 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:17.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:17.261 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:13:17.261 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:17.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.415+0000 7f1d9a29f640 1 Processor -- start 2024-04-25T10:13:17.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.415+0000 7f1d9a29f640 1 -- start start 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1d940ce510 0x7f1d940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 0x7f1d9406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1d9406ec60 0x7f1d94142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d940cc2b0 con 0x7f1d940ce510 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d9406e290 con 0x7f1d9406ec60 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9a29f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940cee30 con 0x7f1d94144ca0 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1d940ce510 0x7f1d940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1d940ce510 0x7f1d940ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36638/0 (socket says 172.21.15.82:36638) 2024-04-25T10:13:17.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 -- 172.21.15.82:0/950657552 learned_addr learned my addr 172.21.15.82:0/950657552 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d937fe640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 0x7f1d9406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d9883c640 1 --2- 172.21.15.82:0/950657552 >> v2:172.21.15.196:3300/0 conn(0x7f1d9406ec60 0x7f1d94142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 -- 172.21.15.82:0/950657552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 msgr2=0x7f1d9406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 0x7f1d9406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 -- 172.21.15.82:0/950657552 >> v2:172.21.15.196:3300/0 conn(0x7f1d9406ec60 msgr2=0x7f1d94142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 --2- 172.21.15.82:0/950657552 >> v2:172.21.15.196:3300/0 conn(0x7f1d9406ec60 0x7f1d94142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d93fff640 1 -- 172.21.15.82:0/950657552 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d940ccc70 con 0x7f1d940ce510 2024-04-25T10:13:17.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.419+0000 7f1d937fe640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 0x7f1d9406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d93fff640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d940ce510 0x7f1d940ce8f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1d94145840 tx=0x7f1d8c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751ad3f6943dc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d92ffd640 1 -- 172.21.15.82:0/950657552 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0b4070 con 0x7f1d940ce510 2024-04-25T10:13:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d92ffd640 1 -- 172.21.15.82:0/950657552 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d8c0af400 con 0x7f1d940ce510 2024-04-25T10:13:17.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d92ffd640 1 -- 172.21.15.82:0/950657552 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c0af7d0 con 0x7f1d940ce510 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 -- 172.21.15.82:0/950657552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d940ce510 msgr2=0x7f1d940ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d940ce510 0x7f1d940ce8f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1d94145840 tx=0x7f1d8c01f1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 -- 172.21.15.82:0/950657552 shutdown_connections 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/950657552 >> v2:172.21.15.196:3300/0 conn(0x7f1d9406ec60 0x7f1d94142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d94144ca0 0x7f1d9406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/950657552 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d940ce510 0x7f1d940ce8f0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 -- 172.21.15.82:0/950657552 >> 172.21.15.82:0/950657552 conn(0x7f1d940b11a0 msgr2=0x7f1d940b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 -- 172.21.15.82:0/950657552 shutdown_connections 2024-04-25T10:13:17.424 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 -- 172.21.15.82:0/950657552 wait complete. 2024-04-25T10:13:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.423+0000 7f1d9a29f640 1 Processor -- start 2024-04-25T10:13:17.425 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- start start 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d9406ec60 0x7f1d9415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d940ce510 0x7f1d941e6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d940cc900 con 0x7f1d94144ca0 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9406df20 con 0x7f1d9406ec60 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d941451f0 con 0x7f1d940ce510 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36650/0 (socket says 172.21.15.82:36650) 2024-04-25T10:13:17.427 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 -- 172.21.15.82:0/735447912 learned_addr learned my addr 172.21.15.82:0/735447912 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d93fff640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d9406ec60 0x7f1d9415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d937fe640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d940ce510 0x7f1d941e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 -- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d940ce510 msgr2=0x7f1d941e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d940ce510 0x7f1d941e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 -- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d9406ec60 msgr2=0x7f1d9415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d9406ec60 0x7f1d9415ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 -- 172.21.15.82:0/735447912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d941e50e0 con 0x7f1d94144ca0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9883c640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d5c0 tx=0x7f1d8400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840125f0 con 0x7f1d94144ca0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d84019070 con 0x7f1d94144ca0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d941e53c0 con 0x7f1d94144ca0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d840129d0 con 0x7f1d94144ca0 2024-04-25T10:13:17.428 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d941e5980 con 0x7f1d94144ca0 2024-04-25T10:13:17.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9404efa0 con 0x7f1d94144ca0 2024-04-25T10:13:17.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d84012b70 con 0x7f1d94144ca0 2024-04-25T10:13:17.429 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 0x7f1d7406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1d84085370 con 0x7f1d94144ca0 2024-04-25T10:13:17.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.427+0000 7f1d93fff640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 0x7f1d7406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.430 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.431+0000 7f1d93fff640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 0x7f1d7406d830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0b1000 tx=0x7f1d8c0aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.433 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.435+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d84056410 con 0x7f1d94144ca0 2024-04-25T10:13:17.610 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.611+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d9406c030 con 0x7f1d94144ca0 2024-04-25T10:13:17.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.611+0000 7f1d90ff9640 1 -- 172.21.15.82:0/735447912 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f1d84055d10 con 0x7f1d94144ca0 2024-04-25T10:13:17.611 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:17.611 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":85,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5717,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3196048761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3196048761},{"type":"v1","addr":"172.21.15.196:6833","nonce":3196048761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":5720,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/3189478950","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":3189478950},{"type":"v1","addr":"172.21.15.196:6835","nonce":3189478950}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":8766,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6835/4149410850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":4149410850},{"type":"v1","addr":"172.21.15.82:6835","nonce":4149410850}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":72},{"gid":9327,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/1833713793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6836","nonce":1833713793},{"type":"v1","addr":"172.21.15.82:6837","nonce":1833713793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[]} 2024-04-25T10:13:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 85 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 msgr2=0x7f1d7406d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 0x7f1d7406d830 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c0b1000 tx=0x7f1d8c0aa000 comp rx=0 tx=0).stop 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 msgr2=0x7f1d941e6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400d5c0 tx=0x7f1d8400da80 comp rx=0 tx=0).stop 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 shutdown_connections 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1d7406b370 0x7f1d7406d830 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1d940ce510 0x7f1d941e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1d94144ca0 0x7f1d941e6b30 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 --2- 172.21.15.82:0/735447912 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1d9406ec60 0x7f1d9415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 >> 172.21.15.82:0/735447912 conn(0x7f1d940b11a0 msgr2=0x7f1d940b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 shutdown_connections 2024-04-25T10:13:17.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.615+0000 7f1d9a29f640 1 -- 172.21.15.82:0/735447912 wait complete. 2024-04-25T10:13:17.630 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:13:17.630 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:13:17.630 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:17.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:17.630 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:13:17.630 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:13:17.631 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:17.631 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:17.631 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:13:17.631 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:13:17.631 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:17.632 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:17.632 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:13:17.632 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:13:17.632 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:17.632 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:17.639 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:17.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:13:17.640 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:17.640 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:13:17.644 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:17.644 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:13:17.647 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:17.647 DEBUG:teuthology.orchestra.run.smithi196:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:13:17.683 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:13:17.683 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:17.683 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:13:17.684 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:17.693 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:13:17.693 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:17.694 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:13:17.694 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:17.695 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:13:17.739 INFO:tasks.ceph.mds.a.smithi082.stdout:starting mds.a at 2024-04-25T10:13:17.739 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:13:17.762 INFO:tasks.ceph.mds.d.smithi196.stdout:starting mds.d at 2024-04-25T10:13:17.773 INFO:tasks.ceph.mds.b.smithi196.stdout:starting mds.b at 2024-04-25T10:13:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 Processor -- start 2024-04-25T10:13:17.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 -- start start 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffb4c0cf7e0 0x7ffb4c0b3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb4c06d9b0 con 0x7ffb4c0cfbc0 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c0b3960 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb5207d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb4c06dc50 con 0x7ffb4c0cf7e0 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffb4c0cfbc0 0x7ffb4c071eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55304/0 (socket says 172.21.15.82:55304) 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36672/0 (socket says 172.21.15.82:36672) 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4affd640 1 -- 172.21.15.82:0/1411148903 learned_addr learned my addr 172.21.15.82:0/1411148903 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4b7fe640 1 --2- 172.21.15.82:0/1411148903 >> v2:172.21.15.196:3300/0 conn(0x7ffb4c0cf7e0 0x7ffb4c0b3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 -- 172.21.15.82:0/1411148903 >> v2:172.21.15.196:3300/0 conn(0x7ffb4c0cf7e0 msgr2=0x7ffb4c0b3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:17.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 --2- 172.21.15.82:0/1411148903 >> v2:172.21.15.196:3300/0 conn(0x7ffb4c0cf7e0 0x7ffb4c0b3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4a7fc640 1 -- 172.21.15.82:0/1411148903 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345147067 0 0) 0x7ffb4c06d9b0 con 0x7ffb4c0cfbc0 2024-04-25T10:13:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 -- 172.21.15.82:0/1411148903 >> v1:172.21.15.82:6789/0 conn(0x7ffb4c0cfbc0 legacy=0x7ffb4c071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 -- 172.21.15.82:0/1411148903 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c06df30 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4bfff640 1 --2- 172.21.15.82:0/1411148903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c072890 tx=0x7ffb4401c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6715982d591eb5fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4a7fc640 1 -- 172.21.15.82:0/1411148903 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb440b2070 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4a7fc640 1 -- 172.21.15.82:0/1411148903 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb440a5710 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.811+0000 7ffb4a7fc640 1 -- 172.21.15.82:0/1411148903 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb440a5ae0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/1411148903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 msgr2=0x7ffb4c06e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- 172.21.15.82:0/1411148903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c072890 tx=0x7ffb4401c7e0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/1411148903 shutdown_connections 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- 172.21.15.82:0/1411148903 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c06e730 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- 172.21.15.82:0/1411148903 >> v2:172.21.15.196:3300/0 conn(0x7ffb4c0cf7e0 0x7ffb4c0b3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/1411148903 >> 172.21.15.82:0/1411148903 conn(0x7ffb4c06ac90 msgr2=0x7ffb4c06b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/1411148903 shutdown_connections 2024-04-25T10:13:17.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/1411148903 wait complete. 2024-04-25T10:13:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 Processor -- start 2024-04-25T10:13:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- start start 2024-04-25T10:13:17.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 0x7ffb4c159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 0x7ffb4c151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c16a8f0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c16a770 con 0x7ffb4c0cf7e0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c16aa70 con 0x7ffb4c0cfbc0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36678/0 (socket says 172.21.15.82:36678) 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 -- 172.21.15.82:0/11321635 learned_addr learned my addr 172.21.15.82:0/11321635 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4affd640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 0x7ffb4c159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4bfff640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 0x7ffb4c151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 -- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 msgr2=0x7ffb4c151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 0x7ffb4c151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 -- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 msgr2=0x7ffb4c159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 0x7ffb4c159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 -- 172.21.15.82:0/11321635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c159aa0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4bfff640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 0x7ffb4c151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4affd640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 0x7ffb4c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb4b7fe640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e8a0 tx=0x7ffb3c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c017070 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3c00c8e0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4c159d80 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3c00ccb0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c15a340 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.815+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb4c04efa0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.819+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb3c01a070 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.819+0000 7ffb33fff640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 0x7ffb38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.819+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ffb3c0851a0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.819+0000 7ffb4affd640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 0x7ffb38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:17.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.819+0000 7ffb4affd640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 0x7ffb38069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c168730 tx=0x7ffb40007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:17.822 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.823+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb3c0562d0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:17.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:13:17.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:13:17.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:17.995+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffb4c0d0700 con 0x7ffb4c0ac9f0 2024-04-25T10:13:18.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.099+0000 7ffb33fff640 1 -- 172.21.15.82:0/11321635 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v117) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffb3c055bd0 con 0x7ffb4c0ac9f0 2024-04-25T10:13:18.100 INFO:teuthology.orchestra.run.smithi082.stderr: removed all blocklist entries 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 msgr2=0x7ffb38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 0x7ffb38069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c168730 tx=0x7ffb40007cb0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 msgr2=0x7ffb4c13d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c00e8a0 tx=0x7ffb3c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 shutdown_connections 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb38066f60 0x7ffb38069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb4c0cfbc0 0x7ffb4c151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb4c0cf7e0 0x7ffb4c159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 --2- 172.21.15.82:0/11321635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb4c0ac9f0 0x7ffb4c13d2d0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 >> 172.21.15.82:0/11321635 conn(0x7ffb4c06ac90 msgr2=0x7ffb4c15ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 shutdown_connections 2024-04-25T10:13:18.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.103+0000 7ffb5207d640 1 -- 172.21.15.82:0/11321635 wait complete. 2024-04-25T10:13:18.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:13:18.289 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 Processor -- start 2024-04-25T10:13:18.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 -- start start 2024-04-25T10:13:18.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f40a415da10 0x7f40a4071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a4162310 con 0x7f40a4071e90 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a40adbd0 con 0x7f40a40b24a0 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40ac5f3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40ae860 con 0x7f40a415da10 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aab90640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f40a4071e90 0x7f40a4161c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55312/0 (socket says 172.21.15.82:55312) 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aab90640 1 -- 172.21.15.82:0/912913665 learned_addr learned my addr 172.21.15.82:0/912913665 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aa38f640 1 --2- 172.21.15.82:0/912913665 >> v2:172.21.15.196:3300/0 conn(0x7f40a415da10 0x7f40a4071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136459347 0 0) 0x7f40a4162310 con 0x7f40a4071e90 2024-04-25T10:13:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4090003530 con 0x7f40a4071e90 2024-04-25T10:13:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3364716580 0 0) 0x7f4090003530 con 0x7f40a4071e90 2024-04-25T10:13:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 >> v2:172.21.15.196:3300/0 conn(0x7f40a415da10 msgr2=0x7f40a4071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 --2- 172.21.15.82:0/912913665 >> v2:172.21.15.196:3300/0 conn(0x7f40a415da10 0x7f40a4071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 >> v1:172.21.15.196:6789/0 conn(0x7f40a40b24a0 legacy=0x7f40a41614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a415e210 con 0x7f40a4071e90 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f409c001a00 con 0x7f40a4071e90 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 >> v1:172.21.15.82:6789/0 conn(0x7f40a4071e90 legacy=0x7f40a4161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4090003a30 0x7f4090003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 0x7f4090004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40900052d0 0x7f409000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40ae860 con 0x7f4090004810 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409000c210 con 0x7f40900052d0 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409000c410 con 0x7f4090003a30 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 0x7f4090004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aa38f640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4090003a30 0x7f4090003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aab90640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40900052d0 0x7f409000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 -- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4090003a30 msgr2=0x7f4090003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4090003a30 0x7f4090003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 -- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40900052d0 msgr2=0x7f409000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40900052d0 0x7f409000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40a9b8e640 1 -- 172.21.15.82:0/912913665 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4090003530 con 0x7f4090004810 2024-04-25T10:13:18.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aab90640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f40900052d0 0x7f409000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:18.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.291+0000 7f40aab90640 1 -- 172.21.15.82:0/912913665 reap_dead start 2024-04-25T10:13:18.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 0x7f4090004c90 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f4094001780 tx=0x7f409401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e453570ea5c0f6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940013a0 con 0x7f4090004810 2024-04-25T10:13:18.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40940052f0 con 0x7f4090004810 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40a938d640 1 -- 172.21.15.82:0/912913665 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4094003840 con 0x7f4090004810 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 msgr2=0x7f4090004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 0x7f4090004c90 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f4094001780 tx=0x7f409401c400 comp rx=0 tx=0).stop 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- 172.21.15.82:0/912913665 shutdown_connections 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/912913665 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4090004810 0x7f4090004c90 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- 172.21.15.82:0/912913665 >> 172.21.15.82:0/912913665 conn(0x7f40a40b11a0 msgr2=0x7f40a40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- 172.21.15.82:0/912913665 shutdown_connections 2024-04-25T10:13:18.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- 172.21.15.82:0/912913665 wait complete. 2024-04-25T10:13:18.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 Processor -- start 2024-04-25T10:13:18.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- start start 2024-04-25T10:13:18.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 0x7f40a40ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 0x7f40a40b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415ee70 con 0x7f40a4146f90 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a415ecf0 con 0x7f4090004810 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.295+0000 7f40ac5f3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41614f0 con 0x7f40a40b33b0 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36700/0 (socket says 172.21.15.82:36700) 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 -- 172.21.15.82:0/1080105649 learned_addr learned my addr 172.21.15.82:0/1080105649 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aa38f640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 0x7f40a40ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 0x7f40a40b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 -- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 msgr2=0x7f40a40b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 0x7f40a40b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 -- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 msgr2=0x7f40a40ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 0x7f40a40ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 -- 172.21.15.82:0/1080105649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4144e50 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40a9b8e640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 0x7f40a40b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aa38f640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 0x7f40a40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aab90640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f40a4070fc0 tx=0x7f409c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c00eb40 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c018070 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a41450c0 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c013420 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a4145680 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a4145880 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f409c0135c0 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 0x7f4088069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f409c0851b0 con 0x7f40a4146f90 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aa38f640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 0x7f4088069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.299+0000 7f40aa38f640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 0x7f4088069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009af0 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.303+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f40a4145880 con 0x7f40a4146f90 2024-04-25T10:13:18.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.511+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f40a404efa0 con 0x7f40a4146f90 2024-04-25T10:13:18.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.511+0000 7f409affd640 1 -- 172.21.15.82:0/1080105649 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f409c056380 con 0x7f40a4146f90 2024-04-25T10:13:18.513 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:18.513 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:18.513 INFO:teuthology.orchestra.run.smithi082.stdout: "auth_dump": [ 2024-04-25T10:13:18.513 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.a", 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmvaPyDRAAJ/E6OzAFa7xHaw9Hp/fHHQ==", 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:13:18.514 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:13:18.515 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.515 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.515 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.515 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.515 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.b", 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmYl7SHBAAaUgGBrT1wgX10r401lAlFw==", 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:13:18.516 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.517 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.517 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.517 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.517 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.c", 2024-04-25T10:13:18.518 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmBq6CFhAAt+5djISkq1T8Nld8SGl2JA==", 2024-04-25T10:13:18.518 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.518 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.518 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:13:18.518 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mds.d", 2024-04-25T10:13:18.519 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmi0y6IhAAB5NvXvi/riYm3/CyrrZqPw==", 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.520 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.0", 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDnKSpmJA75NxAAhIlyxDXybDJnOuhrl3hhXw==", 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.522 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.523 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.1", 2024-04-25T10:13:18.523 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDrKSpmtDIFARAAm1eufEtIOcZ/QgkZoBpgXA==", 2024-04-25T10:13:18.523 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.523 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.523 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.2", 2024-04-25T10:13:18.524 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDuKSpmsRTUBhAAtJ4yshFjLLyfcFdSFpha6A==", 2024-04-25T10:13:18.525 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.525 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.525 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.525 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.525 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.3", 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDxKSpmvgp6DhAAUUcfxT6MWS8wbRHKBu2a1g==", 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.526 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.527 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.527 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.527 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.527 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.527 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.4", 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD3KSpmQxEWEBAAEoiFzJnNGJqT9SguM7lRVg==", 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.528 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.5", 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD6KSpmIihNFhAAw4nEXxvRmSSXMw+K8gf6pA==", 2024-04-25T10:13:18.529 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.530 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.530 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.530 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.530 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.6", 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQD9KSpm0Kh6HBAA3iBnPpgrN/c7K+ovSeyNbg==", 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.531 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.532 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.532 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.532 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.532 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "osd.7", 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQAAKipmTlB3IRAAy0fSKvn3i2t191s7GKH9uA==", 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile osd", 2024-04-25T10:13:18.533 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.0", 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA==", 2024-04-25T10:13:18.534 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.535 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.535 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow r", 2024-04-25T10:13:18.535 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:18.535 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.535 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.1", 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmwPecMBAAuwvwowFDsjOm4o21TondLw==", 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.536 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:13:18.537 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:13:18.537 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:13:18.537 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.537 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.537 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.2", 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmb1WuLRAA12jQhtJv8cVakRc0ZzXYWg==", 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:13:18.538 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.3", 2024-04-25T10:13:18.539 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmPmjVNRAAyn8KrdWQ5p09VpUHkzeOGQ==", 2024-04-25T10:13:18.540 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.540 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow", 2024-04-25T10:13:18.540 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow r", 2024-04-25T10:13:18.540 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow rw", 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow rwx" 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "client.admin", 2024-04-25T10:13:18.541 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDjKSpmHUqoMRAAcQGXV2/JdiCOOZgEFSYMCw==", 2024-04-25T10:13:18.542 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.542 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:13:18.542 INFO:teuthology.orchestra.run.smithi082.stdout: "mgr": "allow *", 2024-04-25T10:13:18.542 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow *", 2024-04-25T10:13:18.542 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.x", 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpmFtNCChAAalQk5QZvNVUN9Jz7J0a2IQ==", 2024-04-25T10:13:18.543 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:18.544 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "entity": "mgr.y", 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "key": "AQDkKSpm2lN+BxAAmFDJ3OfqVWiLDeBIj5UuRA==", 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "caps": { 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "mds": "allow *", 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:18.545 INFO:teuthology.orchestra.run.smithi082.stdout: "osd": "allow *" 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi082.stdout: ] 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:18.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 msgr2=0x7f4088069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 0x7f4088069420 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f40a0009af0 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:18.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 msgr2=0x7f40a4147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f40a4070fc0 tx=0x7f409c007bc0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 shutdown_connections 2024-04-25T10:13:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4088066f60 0x7f4088069420 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f40a40b33b0 0x7f40a40b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f40a4146f90 0x7f40a4147410 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 --2- 172.21.15.82:0/1080105649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4090004810 0x7f40a40ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 >> 172.21.15.82:0/1080105649 conn(0x7f40a40b11a0 msgr2=0x7f40a415ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 shutdown_connections 2024-04-25T10:13:18.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.515+0000 7f40ac5f3640 1 -- 172.21.15.82:0/1080105649 wait complete. 2024-04-25T10:13:18.552 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:18.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 Processor -- start 2024-04-25T10:13:18.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 -- start start 2024-04-25T10:13:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4300cc1a0 0x7fb43006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb43006b1d0 0x7fb4300cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb43006b5e0 con 0x7fb43015ba20 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4300d1430 con 0x7fb4300cc1a0 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb435ed5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb43015c260 con 0x7fb43006b1d0 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4300cc1a0 0x7fb43006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fb4300cc1a0 0x7fb43006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36714/0 (socket says 172.21.15.82:36714) 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 -- 172.21.15.82:0/3009550639 learned_addr learned my addr 172.21.15.82:0/3009550639 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42ffff640 1 --2- 172.21.15.82:0/3009550639 >> v2:172.21.15.196:3300/0 conn(0x7fb43006b1d0 0x7fb4300cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 -- 172.21.15.82:0/3009550639 >> v2:172.21.15.196:3300/0 conn(0x7fb43006b1d0 msgr2=0x7fb4300cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 --2- 172.21.15.82:0/3009550639 >> v2:172.21.15.196:3300/0 conn(0x7fb43006b1d0 0x7fb4300cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 -- 172.21.15.82:0/3009550639 >> v1:172.21.15.196:6789/0 conn(0x7fb43015ba20 legacy=0x7fb43015fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.663+0000 7fb42effd640 1 -- 172.21.15.82:0/3009550639 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43015c540 con 0x7fb4300cc1a0 2024-04-25T10:13:18.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42effd640 1 --2- 172.21.15.82:0/3009550639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4300cc1a0 0x7fb43006ac90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb430072410 tx=0x7fb42401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be6f6b6a10408b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42e7fc640 1 -- 172.21.15.82:0/3009550639 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240b0070 con 0x7fb4300cc1a0 2024-04-25T10:13:18.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42e7fc640 1 -- 172.21.15.82:0/3009550639 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4240ab600 con 0x7fb4300cc1a0 2024-04-25T10:13:18.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42e7fc640 1 -- 172.21.15.82:0/3009550639 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4240ab9d0 con 0x7fb4300cc1a0 2024-04-25T10:13:18.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- 172.21.15.82:0/3009550639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4300cc1a0 msgr2=0x7fb43006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- 172.21.15.82:0/3009550639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4300cc1a0 0x7fb43006ac90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fb430072410 tx=0x7fb42401c300 comp rx=0 tx=0).stop 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- 172.21.15.82:0/3009550639 shutdown_connections 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- 172.21.15.82:0/3009550639 >> v2:172.21.15.196:3300/0 conn(0x7fb43006b1d0 0x7fb4300cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- 172.21.15.82:0/3009550639 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb4300cc1a0 0x7fb43006ac90 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- 172.21.15.82:0/3009550639 >> 172.21.15.82:0/3009550639 conn(0x7fb430099a70 msgr2=0x7fb4300d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- 172.21.15.82:0/3009550639 shutdown_connections 2024-04-25T10:13:18.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- 172.21.15.82:0/3009550639 wait complete. 2024-04-25T10:13:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 Processor -- start 2024-04-25T10:13:18.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- start start 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43006b1d0 0x7fb4300b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4300cc1a0 0x7fb4300afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43015bf80 con 0x7fb43015ba20 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb43015be00 con 0x7fb43006b1d0 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb435ed5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb430168720 con 0x7fb4300cc1a0 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36724/0 (socket says 172.21.15.82:36724) 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 -- 172.21.15.82:0/2098192815 learned_addr learned my addr 172.21.15.82:0/2098192815 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42effd640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4300cc1a0 0x7fb4300afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43006b1d0 0x7fb4300b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 -- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4300cc1a0 msgr2=0x7fb4300afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4300cc1a0 0x7fb4300afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 -- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43006b1d0 msgr2=0x7fb4300b56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43006b1d0 0x7fb4300b56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 -- 172.21.15.82:0/2098192815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300b0c10 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb42ffff640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb428007e40 tx=0x7fb42800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.667+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428014620 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb428019070 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4300adec0 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb428014a20 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4300b4bf0 con 0x7fb43015ba20 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb43004efa0 con 0x7fb43015ba20 2024-04-25T10:13:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb428014bc0 con 0x7fb43015ba20 2024-04-25T10:13:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb417fff640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 0x7fb41c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 0x7fb41c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb428085730 con 0x7fb43015ba20 2024-04-25T10:13:18.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.671+0000 7fb42f7fe640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 0x7fb41c069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb4300ca6c0 tx=0x7fb420008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.675+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb428056900 con 0x7fb43015ba20 2024-04-25T10:13:18.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.823+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb43006b7c0 con 0x7fb43015ba20 2024-04-25T10:13:18.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb417fff640 1 -- 172.21.15.82:0/2098192815 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+13186 (secure 0 0 0) 0x7fb428056200 con 0x7fb43015ba20 2024-04-25T10:13:18.826 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:18.826 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":117,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:18.098800+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:18.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 msgr2=0x7fb41c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 0x7fb41c069420 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb4300ca6c0 tx=0x7fb420008040 comp rx=0 tx=0).stop 2024-04-25T10:13:18.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 msgr2=0x7fb4300b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fb428007e40 tx=0x7fb42800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 shutdown_connections 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb41c066f60 0x7fb41c069420 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4300cc1a0 0x7fb4300afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb43015ba20 0x7fb4300b0380 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 --2- 172.21.15.82:0/2098192815 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb43006b1d0 0x7fb4300b56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 >> 172.21.15.82:0/2098192815 conn(0x7fb430099a70 msgr2=0x7fb43015e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 shutdown_connections 2024-04-25T10:13:18.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.827+0000 7fb435ed5640 1 -- 172.21.15.82:0/2098192815 wait complete. 2024-04-25T10:13:18.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:18.844 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 Processor -- start 2024-04-25T10:13:18.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 -- start start 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f984415b6b0 0x7f98440ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9844073010 0x7f98440cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98440cbd20 con 0x7f98440cf590 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984415ba90 con 0x7f9844073010 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9849c56640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98440cc000 con 0x7f984415b6b0 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9844073010 0x7f98440cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9844073010 0x7f98440cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36740/0 (socket says 172.21.15.82:36740) 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 -- 172.21.15.82:0/4164654117 learned_addr learned my addr 172.21.15.82:0/4164654117 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:18.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f98437fe640 1 --2- 172.21.15.82:0/4164654117 >> v2:172.21.15.196:3300/0 conn(0x7f984415b6b0 0x7f98440ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 -- 172.21.15.82:0/4164654117 >> v2:172.21.15.196:3300/0 conn(0x7f984415b6b0 msgr2=0x7f98440ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 --2- 172.21.15.82:0/4164654117 >> v2:172.21.15.196:3300/0 conn(0x7f984415b6b0 0x7f98440ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 -- 172.21.15.82:0/4164654117 >> v1:172.21.15.196:6789/0 conn(0x7f98440cf590 legacy=0x7f984415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.967+0000 7f9843fff640 1 -- 172.21.15.82:0/4164654117 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98440cc2e0 con 0x7f9844073010 2024-04-25T10:13:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9843fff640 1 --2- 172.21.15.82:0/4164654117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440cfa20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f98440d0180 tx=0x7f983c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993ff8e888df5572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f98427fc640 1 -- 172.21.15.82:0/4164654117 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0a56c0 con 0x7f9844073010 2024-04-25T10:13:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f98427fc640 1 -- 172.21.15.82:0/4164654117 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c0a5860 con 0x7f9844073010 2024-04-25T10:13:18.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f98427fc640 1 -- 172.21.15.82:0/4164654117 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0a5c30 con 0x7f9844073010 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- 172.21.15.82:0/4164654117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 msgr2=0x7f98440cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 --2- 172.21.15.82:0/4164654117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440cfa20 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f98440d0180 tx=0x7f983c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- 172.21.15.82:0/4164654117 shutdown_connections 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 --2- 172.21.15.82:0/4164654117 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440cfa20 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 --2- 172.21.15.82:0/4164654117 >> v2:172.21.15.196:3300/0 conn(0x7f984415b6b0 0x7f98440ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- 172.21.15.82:0/4164654117 >> 172.21.15.82:0/4164654117 conn(0x7f984406ae60 msgr2=0x7f984406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- 172.21.15.82:0/4164654117 shutdown_connections 2024-04-25T10:13:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- 172.21.15.82:0/4164654117 wait complete. 2024-04-25T10:13:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 Processor -- start 2024-04-25T10:13:18.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.971+0000 7f9849c56640 1 -- start start 2024-04-25T10:13:18.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98440cf590 0x7f98440c1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 0x7f98440af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98440cadb0 con 0x7f9844073010 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98440ca900 con 0x7f984415b6b0 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98440caf30 con 0x7f98440cf590 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36748/0 (socket says 172.21.15.82:36748) 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 -- 172.21.15.82:0/582037416 learned_addr learned my addr 172.21.15.82:0/582037416 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9843fff640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 0x7f98440af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 -- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98440cf590 msgr2=0x7f98440c1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9842ffd640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98440cf590 0x7f98440c1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98440cf590 0x7f98440c1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 -- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 msgr2=0x7f98440af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:18.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 0x7f98440af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 -- 172.21.15.82:0/582037416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98440b1a60 con 0x7f9844073010 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9843fff640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 0x7f98440af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f98437fe640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f983400e870 tx=0x7f983400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834017070 con 0x7f9844073010 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400c940 con 0x7f9844073010 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98440b1fb0 con 0x7f9844073010 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400ccf0 con 0x7f9844073010 2024-04-25T10:13:18.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984415a920 con 0x7f9844073010 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984404ef80 con 0x7f9844073010 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f983401a070 con 0x7f9844073010 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9834085120 con 0x7f9844073010 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9842ffd640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:18.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.975+0000 7f9842ffd640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 0x7f9830069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f98440aef90 tx=0x7f9838009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:18.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:18.979+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9834056260 con 0x7f9844073010 2024-04-25T10:13:19.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.127+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9844165e80 con 0x7f9844073010 2024-04-25T10:13:19.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f982bfff640 1 -- 172.21.15.82:0/582037416 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v117) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9834055b60 con 0x7f9844073010 2024-04-25T10:13:19.130 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 msgr2=0x7f9830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 0x7f9830069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f98440aef90 tx=0x7f9838009040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 msgr2=0x7f98440b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f983400e870 tx=0x7f983400ec70 comp rx=0 tx=0).stop 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 shutdown_connections 2024-04-25T10:13:19.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9830066f60 0x7f9830069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98440cf590 0x7f98440c1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f984415b6b0 0x7f98440af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 --2- 172.21.15.82:0/582037416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9844073010 0x7f98440b3d30 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 >> 172.21.15.82:0/582037416 conn(0x7f984406ae60 msgr2=0x7f9844156470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 shutdown_connections 2024-04-25T10:13:19.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.131+0000 7f9849c56640 1 -- 172.21.15.82:0/582037416 wait complete. 2024-04-25T10:13:19.148 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:13:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 Processor -- start 2024-04-25T10:13:19.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 -- start start 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8dc071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa8dc071e90 0x7fa8dc071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc0adbd0 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc0ae860 con 0x7fa8dc071e90 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8e187e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15e210 con 0x7fa8dc15da10 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8da7fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa8dc0b24a0 0x7fa8dc1614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55326/0 (socket says 172.21.15.82:55326) 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8da7fc640 1 -- 172.21.15.82:0/2454990008 learned_addr learned my addr 172.21.15.82:0/2454990008 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:19.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/2454990008 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc071e90 0x7fa8dc071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8daffd640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8dc071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162954925 0 0) 0x7fa8dc0adbd0 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c4003530 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.267+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224156565 0 0) 0x7fa8c4003530 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 msgr2=0x7fa8dc071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8dc071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc071e90 msgr2=0x7fa8dc071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 --2- 172.21.15.82:0/2454990008 >> v2:172.21.15.196:3300/0 conn(0x7fa8dc071e90 0x7fa8dc071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc15e4f0 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa8d0001680 con 0x7fa8dc0b24a0 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 >> v1:172.21.15.82:6789/0 conn(0x7fa8dc0b24a0 legacy=0x7fa8dc1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8c4003a30 0x7fa8c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8c4004810 0x7fa8c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc0ae860 con 0x7fa8c4004810 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc15e210 con 0x7fa8c4003a30 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c400bad0 con 0x7fa8dc15da10 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8c4004810 0x7fa8c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8daffd640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8c4003a30 0x7fa8c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 -- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 msgr2=0x7fa8c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8dc15da10 0x7fa8c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 -- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8c4003a30 msgr2=0x7fa8c4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8c4003a30 0x7fa8c4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 -- 172.21.15.82:0/2454990008 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4003530 con 0x7fa8c4004810 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8da7fc640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8c4004810 0x7fa8c4004c90 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0001ec0 tx=0x7fa8d001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e932d912edcb39d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0007d80 con 0x7fa8c4004810 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d00a9020 con 0x7fa8c4004810 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8d9ffb640 1 -- 172.21.15.82:0/2454990008 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0005b00 con 0x7fa8c4004810 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 -- 172.21.15.82:0/2454990008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8c4004810 msgr2=0x7fa8c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 --2- 172.21.15.82:0/2454990008 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8c4004810 0x7fa8c4004c90 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0001ec0 tx=0x7fa8d001cc20 comp rx=0 tx=0).stop 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8db7fe640 1 -- 172.21.15.82:0/2454990008 reap_dead start 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 -- 172.21.15.82:0/2454990008 shutdown_connections 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 -- 172.21.15.82:0/2454990008 >> 172.21.15.82:0/2454990008 conn(0x7fa8dc0b11a0 msgr2=0x7fa8dc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 -- 172.21.15.82:0/2454990008 shutdown_connections 2024-04-25T10:13:19.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.271+0000 7fa8e187e640 1 -- 172.21.15.82:0/2454990008 wait complete. 2024-04-25T10:13:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 Processor -- start 2024-04-25T10:13:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- start start 2024-04-25T10:13:19.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8c4004810 0x7fa8dc1607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc15ef00 0x7fa8dc0b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc168a00 con 0x7fa8dc167210 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc168880 con 0x7fa8dc15ef00 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc168b80 con 0x7fa8c4004810 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36766/0 (socket says 172.21.15.82:36766) 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 -- 172.21.15.82:0/3010670480 learned_addr learned my addr 172.21.15.82:0/3010670480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8da7fc640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc15ef00 0x7fa8dc0b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8daffd640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8c4004810 0x7fa8dc1607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 -- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8c4004810 msgr2=0x7fa8dc1607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8c4004810 0x7fa8dc1607e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 -- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc15ef00 msgr2=0x7fa8dc0b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc15ef00 0x7fa8dc0b02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 -- 172.21.15.82:0/3010670480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc15f390 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8db7fe640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b43e0 tx=0x7fa8cc011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc003620 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc01b070 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc15f670 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc003a40 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc15fc30 con 0x7fa8dc167210 2024-04-25T10:13:19.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.275+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8dc15fe30 con 0x7fa8dc167210 2024-04-25T10:13:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.279+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8cc003be0 con 0x7fa8dc167210 2024-04-25T10:13:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.279+0000 7fa8c3fff640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 0x7fa8c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.279+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa8cc085050 con 0x7fa8dc167210 2024-04-25T10:13:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.279+0000 7fa8daffd640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 0x7fa8c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.279+0000 7fa8daffd640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 0x7fa8c8069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.283+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8dc15fe30 con 0x7fa8dc167210 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.443+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa8dc04efa0 con 0x7fa8dc167210 2024-04-25T10:13:19.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.443+0000 7fa8c3fff640 1 -- 172.21.15.82:0/3010670480 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v117) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa8cc025020 con 0x7fa8dc167210 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout:[ 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout: { 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout: "poolnum": 1, 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout: "poolname": ".mgr" 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:19.444 INFO:teuthology.orchestra.run.smithi082.stdout:] 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 msgr2=0x7fa8c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 0x7fa8c8069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa8d4009fd0 tx=0x7fa8d400b040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 msgr2=0x7fa8dc0b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0b43e0 tx=0x7fa8cc011920 comp rx=0 tx=0).stop 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 shutdown_connections 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa8c8066f60 0x7fa8c8069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa8dc167210 0x7fa8dc0b0800 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa8dc15ef00 0x7fa8dc0b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 --2- 172.21.15.82:0/3010670480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa8c4004810 0x7fa8dc1607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 >> 172.21.15.82:0/3010670480 conn(0x7fa8dc0b11a0 msgr2=0x7fa8dc0b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 shutdown_connections 2024-04-25T10:13:19.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.447+0000 7fa8e187e640 1 -- 172.21.15.82:0/3010670480 wait complete. 2024-04-25T10:13:19.463 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:13:19.463 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:13:19.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 Processor -- start 2024-04-25T10:13:19.582 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 -- start start 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 0x7f2bec0d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec15dd60 con 0x7f2bec0d21f0 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec0d2650 con 0x7f2bec0d12c0 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf4198640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec06c810 con 0x7f2bec15d980 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36774/0 (socket says 172.21.15.82:36774) 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 -- 172.21.15.82:0/1092025997 learned_addr learned my addr 172.21.15.82:0/1092025997 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:19.584 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf2735640 1 --1- 172.21.15.82:0/1092025997 >> v1:172.21.15.82:6789/0 conn(0x7f2bec0d21f0 0x7f2bec0ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55330/0 (socket says 172.21.15.82:55330) 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1f34640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 0x7f2bec0d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 -- 172.21.15.82:0/1092025997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 msgr2=0x7f2bec0d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 0x7f2bec0d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 -- 172.21.15.82:0/1092025997 >> v1:172.21.15.82:6789/0 conn(0x7f2bec0d21f0 legacy=0x7f2bec0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf0f32640 1 -- 172.21.15.82:0/1092025997 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158403635 0 0) 0x7f2bec15dd60 con 0x7f2bec0d21f0 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1733640 1 -- 172.21.15.82:0/1092025997 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0d4340 con 0x7f2bec0d12c0 2024-04-25T10:13:19.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.583+0000 7f2bf1f34640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 0x7f2bec0d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.586 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f2bec06c990 tx=0x7f2be001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cab278afee9cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf0f32640 1 -- 172.21.15.82:0/1092025997 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00b1070 con 0x7f2bec0d12c0 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf0f32640 1 -- 172.21.15.82:0/1092025997 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00ac620 con 0x7f2bec0d12c0 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf0f32640 1 -- 172.21.15.82:0/1092025997 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00ac9f0 con 0x7f2bec0d12c0 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- 172.21.15.82:0/1092025997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d12c0 msgr2=0x7f2bec0d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f2bec06c990 tx=0x7f2be001c4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- 172.21.15.82:0/1092025997 shutdown_connections 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d12c0 0x7f2bec0d1cb0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- 172.21.15.82:0/1092025997 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec15d980 0x7f2bec0d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- 172.21.15.82:0/1092025997 >> 172.21.15.82:0/1092025997 conn(0x7f2bec08b000 msgr2=0x7f2bec0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- 172.21.15.82:0/1092025997 shutdown_connections 2024-04-25T10:13:19.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- 172.21.15.82:0/1092025997 wait complete. 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 Processor -- start 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- start start 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec0d12c0 0x7f2bec1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec15d980 0x7f2bec15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec06c4a0 con 0x7f2bec0d21f0 2024-04-25T10:13:19.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec06c1c0 con 0x7f2bec15d980 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf4198640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec06dcb0 con 0x7f2bec0d12c0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36776/0 (socket says 172.21.15.82:36776) 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 -- 172.21.15.82:0/4288264670 learned_addr learned my addr 172.21.15.82:0/4288264670 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1f34640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec0d12c0 0x7f2bec1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf2735640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec15d980 0x7f2bec15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 -- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec0d12c0 msgr2=0x7f2bec1552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.587+0000 7f2bf1733640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec0d12c0 0x7f2bec1552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1733640 1 -- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec15d980 msgr2=0x7f2bec15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1733640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec15d980 0x7f2bec15c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1733640 1 -- 172.21.15.82:0/4288264670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec160c10 con 0x7f2bec0d21f0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1733640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2be00af040 tx=0x7f2be0019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00bc070 con 0x7f2bec0d21f0 2024-04-25T10:13:19.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be00b7650 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec160370 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be00b1040 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec160930 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec067c30 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2be00ac4e0 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 0x7f2bd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2be011e840 con 0x7f2bec0d21f0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1f34640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 0x7f2bd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.591+0000 7f2bf1f34640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 0x7f2bd0069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2bec140370 tx=0x7f2be8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.595+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2be0016130 con 0x7f2bec0d21f0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:19.747+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2bec0d08f0 con 0x7f2bec0d21f0 2024-04-25T10:13:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.103+0000 7f2bde7fc640 1 -- 172.21.15.82:0/4288264670 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v118) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2be00efa20 con 0x7f2bec0d21f0 2024-04-25T10:13:20.103 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' created 2024-04-25T10:13:20.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 msgr2=0x7f2bd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 0x7f2bd0069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f2bec140370 tx=0x7f2be8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 msgr2=0x7f2bec15c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f2be00af040 tx=0x7f2be0019250 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 shutdown_connections 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2bd0066f60 0x7f2bd0069420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2bec15d980 0x7f2bec15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2bec0d21f0 0x7f2bec15c000 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 --2- 172.21.15.82:0/4288264670 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2bec0d12c0 0x7f2bec1552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 >> 172.21.15.82:0/4288264670 conn(0x7f2bec08b000 msgr2=0x7f2bec0b83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 shutdown_connections 2024-04-25T10:13:20.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.107+0000 7f2bf4198640 1 -- 172.21.15.82:0/4288264670 wait complete. 2024-04-25T10:13:20.131 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:13:20.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 Processor -- start 2024-04-25T10:13:20.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 -- start start 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd51c0cc820 0x7fd51c0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b1910 0x7fd51c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd51c160310 con 0x7fd51c0b0b00 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd51c15cfa0 con 0x7fd51c0cc820 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd520d7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c15cdf0 con 0x7fd51c0b1910 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd51c0cc820 0x7fd51c0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd51c0cc820 0x7fd51c0b13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36778/0 (socket says 172.21.15.82:36778) 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 -- 172.21.15.82:0/1323607732 learned_addr learned my addr 172.21.15.82:0/1323607732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd51ad9d640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b1910 0x7fd51c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 -- 172.21.15.82:0/1323607732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b1910 msgr2=0x7fd51c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b1910 0x7fd51c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 -- 172.21.15.82:0/1323607732 >> v1:172.21.15.196:6789/0 conn(0x7fd51c0b0b00 legacy=0x7fd51c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.263+0000 7fd519d9b640 1 -- 172.21.15.82:0/1323607732 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c15f550 con 0x7fd51c0cc820 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd519d9b640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c0cc820 0x7fd51c0b13d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fd510009870 tx=0x7fd51001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18024e6d9ecd0e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd51959a640 1 -- 172.21.15.82:0/1323607732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100b0070 con 0x7fd51c0cc820 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd51959a640 1 -- 172.21.15.82:0/1323607732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5100ab600 con 0x7fd51c0cc820 2024-04-25T10:13:20.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd51959a640 1 -- 172.21.15.82:0/1323607732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5100ab9d0 con 0x7fd51c0cc820 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 -- 172.21.15.82:0/1323607732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c0cc820 msgr2=0x7fd51c0b13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c0cc820 0x7fd51c0b13d0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fd510009870 tx=0x7fd51001c380 comp rx=0 tx=0).stop 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 -- 172.21.15.82:0/1323607732 shutdown_connections 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b1910 0x7fd51c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 --2- 172.21.15.82:0/1323607732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c0cc820 0x7fd51c0b13d0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 -- 172.21.15.82:0/1323607732 >> 172.21.15.82:0/1323607732 conn(0x7fd51c0b1f10 msgr2=0x7fd51c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 -- 172.21.15.82:0/1323607732 shutdown_connections 2024-04-25T10:13:20.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.267+0000 7fd520d7b640 1 -- 172.21.15.82:0/1323607732 wait complete. 2024-04-25T10:13:20.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 Processor -- start 2024-04-25T10:13:20.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- start start 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b0b00 0x7fd51c0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c0b1910 0x7fd51c0af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0b1060 con 0x7fd51c14fc40 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c0b0ee0 con 0x7fd51c0b1910 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c15bde0 con 0x7fd51c0b0b00 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36790/0 (socket says 172.21.15.82:36790) 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 -- 172.21.15.82:0/2666410135 learned_addr learned my addr 172.21.15.82:0/2666410135 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51a59c640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b0b00 0x7fd51c0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd519d9b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c0b1910 0x7fd51c0af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 -- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b0b00 msgr2=0x7fd51c0c9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b0b00 0x7fd51c0c9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 -- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c0b1910 msgr2=0x7fd51c0af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c0b1910 0x7fd51c0af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 -- 172.21.15.82:0/2666410135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51c0c78b0 con 0x7fd51c14fc40 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51ad9d640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fd514007e40 tx=0x7fd51400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514014620 con 0x7fd51c14fc40 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd514019070 con 0x7fd51c14fc40 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd514014a20 con 0x7fd51c14fc40 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c0c7b90 con 0x7fd51c14fc40 2024-04-25T10:13:20.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c0c8150 con 0x7fd51c14fc40 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c04efa0 con 0x7fd51c14fc40 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd514014bc0 con 0x7fd51c14fc40 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fd514085870 con 0x7fd51c14fc40 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.271+0000 7fd51a59c640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.275+0000 7fd51a59c640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd50c001650 tx=0x7fd50c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.279+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd5140568d0 con 0x7fd51c14fc40 2024-04-25T10:13:20.423 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:20.423+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd51c0b6380 con 0x7fd51c14fc40 2024-04-25T10:13:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.107+0000 7fd50affd640 1 -- 172.21.15.82:0/2666410135 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v119) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd5140561d0 con 0x7fd51c14fc40 2024-04-25T10:13:21.109 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' created 2024-04-25T10:13:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 msgr2=0x7fd4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 0x7fd4fc069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd50c001650 tx=0x7fd50c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 msgr2=0x7fd51c0af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fd514007e40 tx=0x7fd51400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 shutdown_connections 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd4fc066f60 0x7fd4fc069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd51c14fc40 0x7fd51c0af900 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd51c0b1910 0x7fd51c0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 --2- 172.21.15.82:0/2666410135 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd51c0b0b00 0x7fd51c0c9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 >> 172.21.15.82:0/2666410135 conn(0x7fd51c0b1f10 msgr2=0x7fd51c15d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 shutdown_connections 2024-04-25T10:13:21.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.115+0000 7fd520d7b640 1 -- 172.21.15.82:0/2666410135 wait complete. 2024-04-25T10:13:21.150 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:13:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 Processor -- start 2024-04-25T10:13:21.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 -- start start 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a7815b420 0x7f6a780ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a780cbd20 con 0x7f6a7815b800 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a780cc000 con 0x7f6a780cfca0 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a7dde2640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a780cc930 con 0x7f6a7815b420 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a777fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6a7815b420 0x7f6a780ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a76ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6a7815b800 0x7f6a780cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55338/0 (socket says 172.21.15.82:55338) 2024-04-25T10:13:21.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a76ffd640 1 -- 172.21.15.82:0/2917748014 learned_addr learned my addr 172.21.15.82:0/2917748014 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a777fe640 1 -- 172.21.15.82:0/2917748014 >> v1:172.21.15.196:6789/0 conn(0x7f6a780cfca0 legacy=0x7f6a78072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a777fe640 1 -- 172.21.15.82:0/2917748014 >> v1:172.21.15.82:6789/0 conn(0x7f6a7815b800 legacy=0x7f6a780cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a777fe640 1 -- 172.21.15.82:0/2917748014 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780cc2e0 con 0x7f6a7815b420 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.275+0000 7f6a767fc640 1 -- 172.21.15.82:0/2917748014 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 862003122 0 0) 0x7f6a780cbd20 con 0x7f6a7815b800 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a777fe640 1 --2- 172.21.15.82:0/2917748014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780ca3c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f6a780ccab0 tx=0x7f6a6801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=390304328605cbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a767fc640 1 -- 172.21.15.82:0/2917748014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a680b1070 con 0x7f6a7815b420 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a767fc640 1 -- 172.21.15.82:0/2917748014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a680ac630 con 0x7f6a7815b420 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a767fc640 1 -- 172.21.15.82:0/2917748014 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a680a8050 con 0x7f6a7815b420 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- 172.21.15.82:0/2917748014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 msgr2=0x7f6a780ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/2917748014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780ca3c0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f6a780ccab0 tx=0x7f6a6801c430 comp rx=0 tx=0).stop 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- 172.21.15.82:0/2917748014 shutdown_connections 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/2917748014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780ca3c0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- 172.21.15.82:0/2917748014 >> 172.21.15.82:0/2917748014 conn(0x7f6a7806ae60 msgr2=0x7f6a7806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- 172.21.15.82:0/2917748014 shutdown_connections 2024-04-25T10:13:21.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- 172.21.15.82:0/2917748014 wait complete. 2024-04-25T10:13:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 Processor -- start 2024-04-25T10:13:21.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- start start 2024-04-25T10:13:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a780cfca0 0x7f6a7815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7815b800 0x7f6a780c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780caf30 con 0x7f6a7815b420 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a780cadb0 con 0x7f6a7815b800 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a7dde2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a78072f20 con 0x7f6a780cfca0 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36802/0 (socket says 172.21.15.82:36802) 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 -- 172.21.15.82:0/4108460628 learned_addr learned my addr 172.21.15.82:0/4108460628 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a777fe640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a780cfca0 0x7f6a7815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a77fff640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7815b800 0x7f6a780c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 -- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a780cfca0 msgr2=0x7f6a7815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a780cfca0 0x7f6a7815e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 -- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7815b800 msgr2=0x7f6a780c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7815b800 0x7f6a780c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 -- 172.21.15.82:0/4108460628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a780c4c50 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a76ffd640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c4b0 tx=0x7f6a6c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c018070 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c007500 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.279+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c004a30 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a780c4f30 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a780c54f0 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7804efa0 con 0x7f6a7815b420 2024-04-25T10:13:21.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a6c004bd0 con 0x7f6a7815b420 2024-04-25T10:13:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a5ffff640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 0x7f6a64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f6a6c084390 con 0x7f6a7815b420 2024-04-25T10:13:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a777fe640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 0x7f6a64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.283+0000 7f6a777fe640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 0x7f6a64069450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6a680ae000 tx=0x7f6a680a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.287+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a6c055270 con 0x7f6a7815b420 2024-04-25T10:13:21.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:21.483+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6a780b1c00 con 0x7f6a7815b420 2024-04-25T10:13:22.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.111+0000 7f6a5ffff640 1 -- 172.21.15.82:0/4108460628 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v88) v1 ==== 174+0+0 (secure 0 0 0) 0x7f6a6c054b70 con 0x7f6a7815b420 2024-04-25T10:13:22.112 INFO:teuthology.orchestra.run.smithi082.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 msgr2=0x7f6a64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 0x7f6a64069450 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6a680ae000 tx=0x7f6a680a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 msgr2=0x7f6a780c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c4b0 tx=0x7f6a6c00c970 comp rx=0 tx=0).stop 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 shutdown_connections 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6a64066f90 0x7f6a64069450 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6a7815b800 0x7f6a780c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6a7815b420 0x7f6a780c4100 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 --2- 172.21.15.82:0/4108460628 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6a780cfca0 0x7f6a7815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 >> 172.21.15.82:0/4108460628 conn(0x7f6a7806ae60 msgr2=0x7f6a780b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.115+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 shutdown_connections 2024-04-25T10:13:22.117 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.119+0000 7f6a7dde2640 1 -- 172.21.15.82:0/4108460628 wait complete. 2024-04-25T10:13:22.154 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:22.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.271+0000 7f1547b20640 1 Processor -- start 2024-04-25T10:13:22.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.271+0000 7f1547b20640 1 -- start start 2024-04-25T10:13:22.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f1547b20640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f15400ce510 0x7f15400ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f1547b20640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15400cee30 con 0x7f1540144ca0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f1547b20640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15400ccc70 con 0x7f15401451f0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f1547b20640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15400cc2b0 con 0x7f15400ce510 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15450bb640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f1540144ca0 0x7f154006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55346/0 (socket says 172.21.15.82:55346) 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15450bb640 1 -- 172.21.15.82:0/525067529 learned_addr learned my addr 172.21.15.82:0/525067529 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15458bc640 1 --2- 172.21.15.82:0/525067529 >> v2:172.21.15.196:3300/0 conn(0x7f15400ce510 0x7f15400ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129649920 0 0) 0x7f15400cee30 con 0x7f1540144ca0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f152c003530 con 0x7f1540144ca0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3830632689 0 0) 0x7f152c003530 con 0x7f1540144ca0 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 >> v2:172.21.15.196:3300/0 conn(0x7f15400ce510 msgr2=0x7f15400ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 --2- 172.21.15.82:0/525067529 >> v2:172.21.15.196:3300/0 conn(0x7f15400ce510 0x7f15400ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 >> v1:172.21.15.196:6789/0 conn(0x7f15401451f0 legacy=0x7f1540142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154006dc40 con 0x7f1540144ca0 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f153c001680 con 0x7f1540144ca0 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 >> v1:172.21.15.82:6789/0 conn(0x7f1540144ca0 legacy=0x7f154006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152c003a80 0x7f152c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f152c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15400cc2b0 con 0x7f152c005220 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00c130 con 0x7f152c004810 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c00c330 con 0x7f152c003a80 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f152c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15458bc640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152c003a80 0x7f152c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15450bb640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 -- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152c003a80 msgr2=0x7f152c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f152c003a80 0x7f152c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 -- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152c004810 msgr2=0x7f152c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 -- 172.21.15.82:0/525067529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c003530 con 0x7f152c005220 2024-04-25T10:13:22.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15450bb640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f152c004810 0x7f152c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 -- 172.21.15.82:0/525067529 reap_dead start 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.275+0000 7f15460bd640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f152c00b9d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e40 tx=0x7f153801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ad7a3aabb4ad00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538001a90 con 0x7f152c005220 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1538003b80 con 0x7f152c005220 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15448ba640 1 -- 172.21.15.82:0/525067529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538005020 con 0x7f152c005220 2024-04-25T10:13:22.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 msgr2=0x7f152c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f152c00b9d0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e40 tx=0x7f153801c760 comp rx=0 tx=0).stop 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/525067529 shutdown_connections 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 --2- 172.21.15.82:0/525067529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f152c00b9d0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/525067529 >> 172.21.15.82:0/525067529 conn(0x7f15400b11a0 msgr2=0x7f15400b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/525067529 shutdown_connections 2024-04-25T10:13:22.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/525067529 wait complete. 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 Processor -- start 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- start start 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f154015ab70 0x7f1540142570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15400abd70 0x7f15400ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15400cc900 con 0x7f152c005220 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154006e290 con 0x7f154015ab70 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15400cca80 con 0x7f15400abd70 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36818/0 (socket says 172.21.15.82:36818) 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 -- 172.21.15.82:0/3963654473 learned_addr learned my addr 172.21.15.82:0/3963654473 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15450bb640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f154015ab70 0x7f1540142570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15460bd640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15400abd70 0x7f15400ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 -- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15400abd70 msgr2=0x7f15400ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15400abd70 0x7f15400ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 -- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f154015ab70 msgr2=0x7f1540142570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f154015ab70 0x7f1540142570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 -- 172.21.15.82:0/3963654473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540142ab0 con 0x7f152c005220 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15458bc640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f1530011590 tx=0x7f1530011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530015870 con 0x7f152c005220 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530015a10 con 0x7f152c005220 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f154013f8d0 con 0x7f152c005220 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530015dc0 con 0x7f152c005220 2024-04-25T10:13:22.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.279+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154013fe90 con 0x7f152c005220 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f154004efa0 con 0x7f152c005220 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1530027430 con 0x7f152c005220 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f15367fc640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 0x7f1524069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1530085980 con 0x7f152c005220 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f15450bb640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 0x7f1524069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.283+0000 7f15450bb640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 0x7f1524069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f153c000c10 tx=0x7f153c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.286 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.287+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1540142ab0 con 0x7f152c005220 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.443+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15400b1cc0 con 0x7f152c005220 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f15367fc640 1 -- 172.21.15.82:0/3963654473 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16977 (secure 0 0 0) 0x7f1530019030 con 0x7f152c005220 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":120,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:21.484715+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 msgr2=0x7f1524069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 0x7f1524069450 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f153c000c10 tx=0x7f153c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 msgr2=0x7f15400ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f1530011590 tx=0x7f1530011a50 comp rx=0 tx=0).stop 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 shutdown_connections 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1524066f90 0x7f1524069450 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f15400abd70 0x7f15400ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f154015ab70 0x7f1540142570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 --2- 172.21.15.82:0/3963654473 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f152c005220 0x7f15400ae1d0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 >> 172.21.15.82:0/3963654473 conn(0x7f15400b11a0 msgr2=0x7f15400b2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 shutdown_connections 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.447+0000 7f1547b20640 1 -- 172.21.15.82:0/3963654473 wait complete. 2024-04-25T10:13:22.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:22.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.583+0000 7ffb2edde640 1 Processor -- start 2024-04-25T10:13:22.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.583+0000 7ffb2edde640 1 -- start start 2024-04-25T10:13:22.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2edde640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffb280cf840 0x7ffb280cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2edde640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb2815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2edde640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2806b710 con 0x7ffb2806fc50 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2edde640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb28161560 con 0x7ffb280cf840 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2edde640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281617b0 con 0x7ffb2815d180 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffb2806fc50 0x7ffb2806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55362/0 (socket says 172.21.15.82:55362) 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 -- 172.21.15.82:0/534198863 learned_addr learned my addr 172.21.15.82:0/534198863 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb27fff640 1 --2- 172.21.15.82:0/534198863 >> v2:172.21.15.196:3300/0 conn(0x7ffb280cf840 0x7ffb280cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2d37b640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb2815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66733134 0 0) 0x7ffb2806b710 con 0x7ffb2806fc50 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb10003530 con 0x7ffb2806fc50 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850697312 0 0) 0x7ffb10003530 con 0x7ffb2806fc50 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 msgr2=0x7ffb2815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb2815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 >> v2:172.21.15.196:3300/0 conn(0x7ffb280cf840 msgr2=0x7ffb280cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 --2- 172.21.15.82:0/534198863 >> v2:172.21.15.196:3300/0 conn(0x7ffb280cf840 0x7ffb280cfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28161a90 con 0x7ffb2806fc50 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ffb18001680 con 0x7ffb2806fc50 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 >> v1:172.21.15.82:6789/0 conn(0x7ffb2806fc50 legacy=0x7ffb2806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10003a30 0x7ffb10003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10004810 0x7ffb10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28161560 con 0x7ffb10003a30 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb281617b0 con 0x7ffb10004810 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1000ba50 con 0x7ffb2815d180 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10003a30 0x7ffb10003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2d37b640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 -- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 msgr2=0x7ffb100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 -- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10004810 msgr2=0x7ffb10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10004810 0x7ffb10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2cb7a640 1 -- 172.21.15.82:0/534198863 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10003530 con 0x7ffb10003a30 2024-04-25T10:13:22.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.587+0000 7ffb2d37b640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb2815d180 0x7ffb100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10003a30 0x7ffb10003e50 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c240 tx=0x7ffb1801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dbe9c91a35303d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18007d60 con 0x7ffb10003a30 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb180a9020 con 0x7ffb10003a30 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb277fe640 1 -- 172.21.15.82:0/534198863 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18005ae0 con 0x7ffb10003a30 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- 172.21.15.82:0/534198863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10003a30 msgr2=0x7ffb10003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 --2- 172.21.15.82:0/534198863 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb10003a30 0x7ffb10003e50 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c240 tx=0x7ffb1801c9d0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2d37b640 1 -- 172.21.15.82:0/534198863 reap_dead start 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- 172.21.15.82:0/534198863 shutdown_connections 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- 172.21.15.82:0/534198863 >> 172.21.15.82:0/534198863 conn(0x7ffb28099a70 msgr2=0x7ffb280d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- 172.21.15.82:0/534198863 shutdown_connections 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- 172.21.15.82:0/534198863 wait complete. 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 Processor -- start 2024-04-25T10:13:22.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- start start 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 0x7ffb280b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb281430e0 0x7ffb28143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb28070030 con 0x7ffb281571f0 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280d0e30 con 0x7ffb10003a30 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2edde640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb280701b0 con 0x7ffb281430e0 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36836/0 (socket says 172.21.15.82:36836) 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 -- 172.21.15.82:0/890907012 learned_addr learned my addr 172.21.15.82:0/890907012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 0x7ffb280b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2d37b640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb281430e0 0x7ffb28143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 -- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb281430e0 msgr2=0x7ffb28143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb281430e0 0x7ffb28143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 -- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 msgr2=0x7ffb280b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 0x7ffb280b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 -- 172.21.15.82:0/890907012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2815f0b0 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 0x7ffb280b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb27fff640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00e8a0 tx=0x7ffb1c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c017070 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.591+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c00c8e0 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00ccd0 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2815f390 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2815f950 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2804efa0 con 0x7ffb281571f0 2024-04-25T10:13:22.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb1c01a070 con 0x7ffb281571f0 2024-04-25T10:13:22.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb257fa640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb1c085320 con 0x7ffb281571f0 2024-04-25T10:13:22.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.595+0000 7ffb2cb7a640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 0x7ffb14069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c240 tx=0x7ffb18001ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.599+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb1c056230 con 0x7ffb281571f0 2024-04-25T10:13:22.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.763+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb2815fd30 con 0x7ffb281571f0 2024-04-25T10:13:22.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.763+0000 7ffb257fa640 1 -- 172.21.15.82:0/890907012 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16977 (secure 0 0 0) 0x7ffb1c055b30 con 0x7ffb281571f0 2024-04-25T10:13:22.765 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:22.765 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":120,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:21.484715+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 msgr2=0x7ffb14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 0x7ffb14069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7ffb1800c240 tx=0x7ffb18001ac0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 msgr2=0x7ffb280b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00e8a0 tx=0x7ffb1c00ed60 comp rx=0 tx=0).stop 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 shutdown_connections 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffb14066f60 0x7ffb14069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffb281430e0 0x7ffb28143560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffb281571f0 0x7ffb280b2880 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 --2- 172.21.15.82:0/890907012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffb10003a30 0x7ffb280b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 >> 172.21.15.82:0/890907012 conn(0x7ffb28099a70 msgr2=0x7ffb280adb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 shutdown_connections 2024-04-25T10:13:22.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.767+0000 7ffb2edde640 1 -- 172.21.15.82:0/890907012 wait complete. 2024-04-25T10:13:22.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:13:22.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.903+0000 7fece1faa640 1 Processor -- start 2024-04-25T10:13:22.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.903+0000 7fece1faa640 1 -- start start 2024-04-25T10:13:22.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fece1faa640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.905 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fece1faa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fece1faa640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc0cbd20 con 0x7fecdc15f130 2024-04-25T10:13:22.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fece1faa640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc15ba90 con 0x7fecdc15b6b0 2024-04-25T10:13:22.906 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fece1faa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cff30 con 0x7fecdc0cf590 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdaffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdaffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:34150/0 (socket says 172.21.15.82:34150) 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:41504/0 (socket says 172.21.15.82:41504) 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdaffd640 1 -- 172.21.15.82:0/2891972304 learned_addr learned my addr 172.21.15.82:0/2891972304 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 -- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 msgr2=0x7fecdc0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 -- 172.21.15.82:0/2891972304 >> v1:172.21.15.196:6789/0 conn(0x7fecdc15f130 legacy=0x7fecdc15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.907 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdb7fe640 1 -- 172.21.15.82:0/2891972304 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0cc000 con 0x7fecdc15b6b0 2024-04-25T10:13:22.908 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.907+0000 7fecdaffd640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:22.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097c0 tx=0x7feccc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ada0ca0d77e20fb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecda7fc640 1 -- 172.21.15.82:0/2891972304 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc0a7070 con 0x7fecdc15b6b0 2024-04-25T10:13:22.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecda7fc640 1 -- 172.21.15.82:0/2891972304 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc0b0070 con 0x7fecdc15b6b0 2024-04-25T10:13:22.909 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecda7fc640 1 -- 172.21.15.82:0/2891972304 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc0ab890 con 0x7fecdc15b6b0 2024-04-25T10:13:22.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15b6b0 msgr2=0x7fecdc0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.910 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097c0 tx=0x7feccc01c470 comp rx=0 tx=0).stop 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- 172.21.15.82:0/2891972304 shutdown_connections 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc0cf590 0x7fecdc0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- 172.21.15.82:0/2891972304 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15b6b0 0x7fecdc0ca3c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- 172.21.15.82:0/2891972304 >> 172.21.15.82:0/2891972304 conn(0x7fecdc06ae60 msgr2=0x7fecdc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- 172.21.15.82:0/2891972304 shutdown_connections 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- 172.21.15.82:0/2891972304 wait complete. 2024-04-25T10:13:22.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 Processor -- start 2024-04-25T10:13:22.912 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- start start 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc15b6b0 0x7fecdc0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15f130 0x7fecdc0cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0ca900 con 0x7fecdc0cf590 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cba30 con 0x7fecdc15f130 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fece1faa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0cadb0 con 0x7fecdc15b6b0 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36842/0 (socket says 172.21.15.82:36842) 2024-04-25T10:13:22.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 -- 172.21.15.82:0/4218856333 learned_addr learned my addr 172.21.15.82:0/4218856333 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdaffd640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc15b6b0 0x7fecdc0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdbfff640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15f130 0x7fecdc0cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 -- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc15b6b0 msgr2=0x7fecdc0cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc15b6b0 0x7fecdc0cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 -- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15f130 msgr2=0x7fecdc0cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15f130 0x7fecdc0cde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.911+0000 7fecdb7fe640 1 -- 172.21.15.82:0/4218856333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc153550 con 0x7fecdc0cf590 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecdb7fe640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ad040 tx=0x7feccc01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc006e20 con 0x7fecdc0cf590 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc0bf610 con 0x7fecdc0cf590 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc153830 con 0x7fecdc0cf590 2024-04-25T10:13:22.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc0bf9e0 con 0x7fecdc0cf590 2024-04-25T10:13:22.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc153df0 con 0x7fecdc0cf590 2024-04-25T10:13:22.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc04efa0 con 0x7fecdc0cf590 2024-04-25T10:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feccc0bd020 con 0x7fecdc0cf590 2024-04-25T10:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 0x7fecc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecdaffd640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 0x7fecc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feccc11f520 con 0x7fecdc0cf590 2024-04-25T10:13:22.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.915+0000 7fecdaffd640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 0x7fecc8069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009630 tx=0x7fecd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.919 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:22.919+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feccc0c6030 con 0x7fecdc0cf590 2024-04-25T10:13:23.105 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:13:23.107+0000 7f711e4c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:23.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:23.115+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fecdc06b450 con 0x7fecdc0cf590 2024-04-25T10:13:24.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.115+0000 7fecc3fff640 1 -- 172.21.15.82:0/4218856333 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) v1 ==== 118+0+0 (secure 0 0 0) 0x7feccc016090 con 0x7fecdc0cf590 2024-04-25T10:13:24.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 msgr2=0x7fecc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 0x7fecc8069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecd0009630 tx=0x7fecd0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 msgr2=0x7fecdc15a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7feccc0ad040 tx=0x7feccc01acb0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 shutdown_connections 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fecc8066f60 0x7fecc8069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fecdc15b6b0 0x7fecdc0cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fecdc15f130 0x7fecdc0cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 --2- 172.21.15.82:0/4218856333 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fecdc0cf590 0x7fecdc15a210 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 >> 172.21.15.82:0/4218856333 conn(0x7fecdc06ae60 msgr2=0x7fecdc0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 shutdown_connections 2024-04-25T10:13:24.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.119+0000 7fece1faa640 1 -- 172.21.15.82:0/4218856333 wait complete. 2024-04-25T10:13:24.140 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:13:24.141 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:13:24.269 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 Processor -- start 2024-04-25T10:13:24.270 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 -- start start 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa2ec1634c0 0x7fa2ec1658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec0ca4c0 con 0x7fa2ec0cf590 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ec0cd690 con 0x7fa2ec15ecb0 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2f1b2c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ec0ca780 con 0x7fa2ec1634c0 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2eb7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa2ec0cf590 0x7fa2ec0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:55378/0 (socket says 172.21.15.82:55378) 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2eb7fe640 1 -- 172.21.15.82:0/3448235259 learned_addr learned my addr 172.21.15.82:0/3448235259 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ebfff640 1 --2- 172.21.15.82:0/3448235259 >> v2:172.21.15.82:3300/0 conn(0x7fa2ec1634c0 0x7fa2ec1658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ea7fc640 1 -- 172.21.15.82:0/3448235259 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233153302 0 0) 0x7fa2ec0ca4c0 con 0x7fa2ec0cf590 2024-04-25T10:13:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ea7fc640 1 -- 172.21.15.82:0/3448235259 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2d4003530 con 0x7fa2ec0cf590 2024-04-25T10:13:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ebfff640 1 -- 172.21.15.82:0/3448235259 >> v1:172.21.15.196:6789/0 conn(0x7fa2ec15ecb0 legacy=0x7fa2ec0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ebfff640 1 -- 172.21.15.82:0/3448235259 >> v1:172.21.15.82:6789/0 conn(0x7fa2ec0cf590 legacy=0x7fa2ec0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.271+0000 7fa2ebfff640 1 -- 172.21.15.82:0/3448235259 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec0cd970 con 0x7fa2ec1634c0 2024-04-25T10:13:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 --2- 172.21.15.82:0/3448235259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec1658b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009d90 tx=0x7fa2e401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74296c7ec680191c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ea7fc640 1 -- 172.21.15.82:0/3448235259 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40b2070 con 0x7fa2ec1634c0 2024-04-25T10:13:24.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ea7fc640 1 -- 172.21.15.82:0/3448235259 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e40a57f0 con 0x7fa2ec1634c0 2024-04-25T10:13:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ea7fc640 1 -- 172.21.15.82:0/3448235259 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40a5bc0 con 0x7fa2ec1634c0 2024-04-25T10:13:24.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/3448235259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 msgr2=0x7fa2ec1658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/3448235259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec1658b0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fa2e4009d90 tx=0x7fa2e401c930 comp rx=0 tx=0).stop 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/3448235259 shutdown_connections 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/3448235259 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec1658b0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/3448235259 >> 172.21.15.82:0/3448235259 conn(0x7fa2ec06ae60 msgr2=0x7fa2ec06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/3448235259 shutdown_connections 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/3448235259 wait complete. 2024-04-25T10:13:24.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 Processor -- start 2024-04-25T10:13:24.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- start start 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 0x7fa2ec0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 0x7fa2ec0c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0cdce0 con 0x7fa2ec1634c0 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec0cd440 con 0x7fa2ec15ecb0 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2f1b2c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ec168820 con 0x7fa2ec0cf590 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36846/0 (socket says 172.21.15.82:36846) 2024-04-25T10:13:24.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 -- 172.21.15.82:0/1747418148 learned_addr learned my addr 172.21.15.82:0/1747418148 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2eaffd640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 0x7fa2ec0c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 -- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 msgr2=0x7fa2ec0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2eb7fe640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 0x7fa2ec0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 0x7fa2ec0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 -- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 msgr2=0x7fa2ec0c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 0x7fa2ec0c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2ebfff640 1 -- 172.21.15.82:0/1747418148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ec06e220 con 0x7fa2ec1634c0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2eb7fe640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 0x7fa2ec0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.275+0000 7fa2eaffd640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 0x7fa2ec0c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2ebfff640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40b0040 tx=0x7fa2e40196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40b2050 con 0x7fa2ec1634c0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e40be070 con 0x7fa2ec1634c0 2024-04-25T10:13:24.280 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2ec06e500 con 0x7fa2ec1634c0 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e40ad890 con 0x7fa2ec1634c0 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ec0b0930 con 0x7fa2ec1634c0 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ec04efa0 con 0x7fa2ec1634c0 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2e40ada30 con 0x7fa2ec1634c0 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 0x7fa2d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2eb7fe640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 0x7fa2d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2e4120230 con 0x7fa2ec1634c0 2024-04-25T10:13:24.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.279+0000 7fa2eb7fe640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 0x7fa2d8069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005890 tx=0x7fa2dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.283+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2e4016520 con 0x7fa2ec1634c0 2024-04-25T10:13:24.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:24.483+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2ec0b4170 con 0x7fa2ec1634c0 2024-04-25T10:13:24.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:13:24.561 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:13:25.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.147+0000 7fa2d3fff640 1 -- 172.21.15.82:0/1747418148 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa2e40f10c0 con 0x7fa2ec1634c0 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 msgr2=0x7fa2d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 0x7fa2d8069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc005890 tx=0x7fa2dc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 msgr2=0x7fa2ec0c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40b0040 tx=0x7fa2e40196d0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 shutdown_connections 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2d8066f60 0x7fa2d8069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2ec1634c0 0x7fa2ec0c6430 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2ec15ecb0 0x7fa2ec0c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 --2- 172.21.15.82:0/1747418148 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2ec0cf590 0x7fa2ec0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 >> 172.21.15.82:0/1747418148 conn(0x7fa2ec06ae60 msgr2=0x7fa2ec0cbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 shutdown_connections 2024-04-25T10:13:25.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.151+0000 7fa2f1b2c640 1 -- 172.21.15.82:0/1747418148 wait complete. 2024-04-25T10:13:25.171 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:13:25.172 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:25.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 Processor -- start 2024-04-25T10:13:25.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 -- start start 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc67c144ca0 0x7fc67c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67c0ccc70 con 0x7fc67c06ec60 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67c06dc40 con 0x7fc67c0ce510 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc683c8c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67c0cc2b0 con 0x7fc67c144ca0 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc67c144ca0 0x7fc67c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc67c144ca0 0x7fc67c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43658/0 (socket says 172.21.15.82:43658) 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc682229640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc67c06ec60 0x7fc67c142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36794/0 (socket says 172.21.15.82:36794) 2024-04-25T10:13:25.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 -- 172.21.15.82:0/2616007816 learned_addr learned my addr 172.21.15.82:0/2616007816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 -- 172.21.15.82:0/2616007816 >> v1:172.21.15.196:6789/0 conn(0x7fc67c0ce510 legacy=0x7fc67c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 -- 172.21.15.82:0/2616007816 >> v1:172.21.15.82:6789/0 conn(0x7fc67c06ec60 legacy=0x7fc67c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.295+0000 7fc681227640 1 -- 172.21.15.82:0/2616007816 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c06df20 con 0x7fc67c144ca0 2024-04-25T10:13:25.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681227640 1 --2- 172.21.15.82:0/2616007816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c144ca0 0x7fc67c06e720 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcbafb74bcd09fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc680a26640 1 -- 172.21.15.82:0/2616007816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0ab620 con 0x7fc67c144ca0 2024-04-25T10:13:25.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc680a26640 1 -- 172.21.15.82:0/2616007816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66c0b0070 con 0x7fc67c144ca0 2024-04-25T10:13:25.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc680a26640 1 -- 172.21.15.82:0/2616007816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66c0ab9e0 con 0x7fc67c144ca0 2024-04-25T10:13:25.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2616007816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c144ca0 msgr2=0x7fc67c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2616007816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c144ca0 0x7fc67c06e720 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fc66c009870 tx=0x7fc66c01c440 comp rx=0 tx=0).stop 2024-04-25T10:13:25.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2616007816 shutdown_connections 2024-04-25T10:13:25.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2616007816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c144ca0 0x7fc67c06e720 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2616007816 >> 172.21.15.82:0/2616007816 conn(0x7fc67c0b11a0 msgr2=0x7fc67c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2616007816 shutdown_connections 2024-04-25T10:13:25.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2616007816 wait complete. 2024-04-25T10:13:25.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 Processor -- start 2024-04-25T10:13:25.299 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- start start 2024-04-25T10:13:25.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc67c0ce510 0x7fc67c1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 0x7fc67c0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c142d90 con 0x7fc67c06ec60 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c06e290 con 0x7fc67c0afb30 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67c142f10 con 0x7fc67c0ce510 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43674/0 (socket says 172.21.15.82:43674) 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 -- 172.21.15.82:0/2495793806 learned_addr learned my addr 172.21.15.82:0/2495793806 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681227640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc67c0ce510 0x7fc67c1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc682229640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 0x7fc67c0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 -- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc67c0ce510 msgr2=0x7fc67c1f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc67c0ce510 0x7fc67c1f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 -- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 msgr2=0x7fc67c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 0x7fc67c0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 -- 172.21.15.82:0/2495793806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67c1f73c0 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc682229640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 0x7fc67c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc681a28640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b440 tx=0x7fc67800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678015020 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc678011890 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc67c1f7690 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc678011c80 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.299+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67c1f7c50 con 0x7fc67c06ec60 2024-04-25T10:13:25.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc67c04efa0 con 0x7fc67c06ec60 2024-04-25T10:13:25.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc678018020 con 0x7fc67c06ec60 2024-04-25T10:13:25.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc6727fc640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc678084fb0 con 0x7fc67c06ec60 2024-04-25T10:13:25.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc681227640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.303+0000 7fc681227640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 0x7fc660069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0af000 tx=0x7fc66c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.307+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc67801f030 con 0x7fc67c06ec60 2024-04-25T10:13:25.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.499+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc67c0b1cc0 con 0x7fc67c06ec60 2024-04-25T10:13:25.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc6727fc640 1 -- 172.21.15.82:0/2495793806 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fc67800d040 con 0x7fc67c06ec60 2024-04-25T10:13:25.502 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:25.502 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:24.487528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/781412207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":781412207},{"type":"v1","addr":"172.21.15.82:6837","nonce":781412207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:25.502 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 91 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 msgr2=0x7fc660069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 0x7fc660069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0af000 tx=0x7fc66c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 msgr2=0x7fc67c0c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fc67800b440 tx=0x7fc67800b900 comp rx=0 tx=0).stop 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 shutdown_connections 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc660066f60 0x7fc660069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc67c0ce510 0x7fc67c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc67c0afb30 0x7fc67c0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 --2- 172.21.15.82:0/2495793806 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc67c06ec60 0x7fc67c0c4680 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 >> 172.21.15.82:0/2495793806 conn(0x7fc67c0b11a0 msgr2=0x7fc67c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 shutdown_connections 2024-04-25T10:13:25.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.503+0000 7fc683c8c640 1 -- 172.21.15.82:0/2495793806 wait complete. 2024-04-25T10:13:25.521 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:25.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 Processor -- start 2024-04-25T10:13:25.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 -- start start 2024-04-25T10:13:25.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1288160080 0x7f12880c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12880c7300 con 0x7f12880c6e50 2024-04-25T10:13:25.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12880affa0 con 0x7f1288160080 2024-04-25T10:13:25.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128e781640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128815f120 con 0x7f128815ed40 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f1287fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f12877fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1288160080 0x7f12880c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128cd1e640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f12880c6e50 0x7f128806ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:41862/0 (socket says 172.21.15.82:41862) 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f1287fff640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:60962/0 (socket says 172.21.15.82:60962) 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f128cd1e640 1 -- 172.21.15.82:0/2682400012 learned_addr learned my addr 172.21.15.82:0/2682400012 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:25.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f12877fe640 1 -- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 msgr2=0x7f12880d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f12877fe640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f12877fe640 1 -- 172.21.15.82:0/2682400012 >> v1:172.21.15.196:6789/0 conn(0x7f12880c6e50 legacy=0x7f128806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f12877fe640 1 -- 172.21.15.82:0/2682400012 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12880b0240 con 0x7f1288160080 2024-04-25T10:13:25.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.639+0000 7f1287fff640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:25.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f12877fe640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f12880c6910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097c0 tx=0x7f127c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8355815ab7fc77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f1286ffd640 1 -- 172.21.15.82:0/2682400012 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0b0070 con 0x7f1288160080 2024-04-25T10:13:25.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f1286ffd640 1 -- 172.21.15.82:0/2682400012 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c0ab600 con 0x7f1288160080 2024-04-25T10:13:25.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f1286ffd640 1 -- 172.21.15.82:0/2682400012 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0ab9d0 con 0x7f1288160080 2024-04-25T10:13:25.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 msgr2=0x7f12880c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f12880c6910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f127c0097c0 tx=0x7f127c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- 172.21.15.82:0/2682400012 shutdown_connections 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f12880c6910 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 --2- 172.21.15.82:0/2682400012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f128815ed40 0x7f12880d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- 172.21.15.82:0/2682400012 >> 172.21.15.82:0/2682400012 conn(0x7f1288099a70 msgr2=0x7f12880d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- 172.21.15.82:0/2682400012 shutdown_connections 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- 172.21.15.82:0/2682400012 wait complete. 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 Processor -- start 2024-04-25T10:13:25.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.643+0000 7f128e781640 1 -- start start 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12880c6e50 0x7f128815f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f128815cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12880cbae0 con 0x7f128815ed40 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12880d1160 con 0x7f1288160080 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12880cbc60 con 0x7f12880c6e50 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43688/0 (socket says 172.21.15.82:43688) 2024-04-25T10:13:25.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 -- 172.21.15.82:0/645962459 learned_addr learned my addr 172.21.15.82:0/645962459 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128cd1e640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f128815cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1287fff640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12880c6e50 0x7f128815f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 -- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12880c6e50 msgr2=0x7f128815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12880c6e50 0x7f128815f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 -- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 msgr2=0x7f128815cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f128815cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 -- 172.21.15.82:0/645962459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128815cab0 con 0x7f128815ed40 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f12877fe640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f127c0ae040 tx=0x7f127c01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0b0050 con 0x7f128815ed40 2024-04-25T10:13:25.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c0bc070 con 0x7f128815ed40 2024-04-25T10:13:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1288156930 con 0x7f128815ed40 2024-04-25T10:13:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0b5870 con 0x7f128815ed40 2024-04-25T10:13:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128815fa90 con 0x7f128815ed40 2024-04-25T10:13:25.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128804efa0 con 0x7f128815ed40 2024-04-25T10:13:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f127c0b5a10 con 0x7f128815ed40 2024-04-25T10:13:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 0x7f1274069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.647+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f127c11f890 con 0x7f128815ed40 2024-04-25T10:13:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.651+0000 7f1287fff640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 0x7f1274069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.651+0000 7f1287fff640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 0x7f1274069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12880af850 tx=0x7f1278008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.655+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f127c016090 con 0x7f128815ed40 2024-04-25T10:13:25.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.807+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12880d0dc0 con 0x7f128815ed40 2024-04-25T10:13:25.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.807+0000 7f1284ff9640 1 -- 172.21.15.82:0/645962459 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f127c0f07a0 con 0x7f128815ed40 2024-04-25T10:13:25.809 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:25.809 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":121,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:23.113792+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:25.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 msgr2=0x7f1274069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 0x7f1274069420 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f12880af850 tx=0x7f1278008040 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 msgr2=0x7f1288157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f127c0ae040 tx=0x7f127c01ac20 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 shutdown_connections 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1274066f60 0x7f1274069420 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1288160080 0x7f128815cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f128815ed40 0x7f1288157410 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 --2- 172.21.15.82:0/645962459 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f12880c6e50 0x7f128815f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 >> 172.21.15.82:0/645962459 conn(0x7f1288099a70 msgr2=0x7f12880b2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 shutdown_connections 2024-04-25T10:13:25.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.811+0000 7f128e781640 1 -- 172.21.15.82:0/645962459 wait complete. 2024-04-25T10:13:25.827 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:13:25.827 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 Processor -- start 2024-04-25T10:13:25.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 -- start start 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9d8071030 0x7fd9d8161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071410 0x7fd9d806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d8162680 con 0x7fd9d8071e90 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d8162960 con 0x7fd9d8071030 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9de396640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d8162c40 con 0x7fd9d8071410 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9d8071030 0x7fd9d8161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd9d8071030 0x7fd9d8161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43694/0 (socket says 172.21.15.82:43694) 2024-04-25T10:13:25.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 -- 172.21.15.82:0/3124030005 learned_addr learned my addr 172.21.15.82:0/3124030005 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d77fe640 1 --1- 172.21.15.82:0/3124030005 >> v1:172.21.15.82:6789/0 conn(0x7fd9d8071e90 0x7fd9d80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36806/0 (socket says 172.21.15.82:36806) 2024-04-25T10:13:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 -- 172.21.15.82:0/3124030005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071410 msgr2=0x7fd9d806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/3124030005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071410 0x7fd9d806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 -- 172.21.15.82:0/3124030005 >> v1:172.21.15.82:6789/0 conn(0x7fd9d8071e90 legacy=0x7fd9d80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 -- 172.21.15.82:0/3124030005 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d8162f20 con 0x7fd9d8071030 2024-04-25T10:13:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/3124030005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071030 0x7fd9d8161fb0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009e50 tx=0x7fd9d001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee2eefc6de7af31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d6ffd640 1 -- 172.21.15.82:0/3124030005 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00a5770 con 0x7fd9d8071030 2024-04-25T10:13:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d6ffd640 1 -- 172.21.15.82:0/3124030005 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9d00a5910 con 0x7fd9d8071030 2024-04-25T10:13:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.935+0000 7fd9d6ffd640 1 -- 172.21.15.82:0/3124030005 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9d00a5ce0 con 0x7fd9d8071030 2024-04-25T10:13:25.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/3124030005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071030 msgr2=0x7fd9d8161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- 172.21.15.82:0/3124030005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071030 0x7fd9d8161fb0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009e50 tx=0x7fd9d001c9a0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/3124030005 shutdown_connections 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- 172.21.15.82:0/3124030005 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071410 0x7fd9d806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- 172.21.15.82:0/3124030005 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071030 0x7fd9d8161fb0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/3124030005 >> 172.21.15.82:0/3124030005 conn(0x7fd9d80b11a0 msgr2=0x7fd9d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/3124030005 shutdown_connections 2024-04-25T10:13:25.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/3124030005 wait complete. 2024-04-25T10:13:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 Processor -- start 2024-04-25T10:13:25.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- start start 2024-04-25T10:13:25.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 0x7fd9d814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 0x7fd9d8155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d806e170 con 0x7fd9d8071410 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d806dff0 con 0x7fd9d8071e90 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d806e2f0 con 0x7fd9d8071030 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43700/0 (socket says 172.21.15.82:43700) 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 -- 172.21.15.82:0/4072706509 learned_addr learned my addr 172.21.15.82:0/4072706509 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 0x7fd9d814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9dc933640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 0x7fd9d8155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 -- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 msgr2=0x7fd9d814f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 0x7fd9d814f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 -- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 msgr2=0x7fd9d8155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 0x7fd9d8155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 -- 172.21.15.82:0/4072706509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d814ff80 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 0x7fd9d814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9dc933640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 0x7fd9d8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d77fe640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800c340 tx=0x7fd9c800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80074e0 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c8007680 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d8150260 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.939+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8004a30 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d814fa30 con 0x7fd9d8071410 2024-04-25T10:13:25.943 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d804efa0 con 0x7fd9d8071410 2024-04-25T10:13:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9c801a020 con 0x7fd9d8071410 2024-04-25T10:13:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9d4ff9640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd9c8084980 con 0x7fd9d8071410 2024-04-25T10:13:25.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.943+0000 7fd9d7fff640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 0x7fd9b8069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ae000 tx=0x7fd9d00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:25.947+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9c8055800 con 0x7fd9d8071410 2024-04-25T10:13:26.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.095+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9d814fd30 con 0x7fd9b8066f60 2024-04-25T10:13:26.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.115+0000 7fd9d4ff9640 1 -- 172.21.15.82:0/4072706509 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252495 (secure 0 0 0) 0x7fd9d814fd30 con 0x7fd9b8066f60 2024-04-25T10:13:26.117 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:26.119 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 msgr2=0x7fd9b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 0x7fd9b8069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd9d00ae000 tx=0x7fd9d00a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 msgr2=0x7fd9d81523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800c340 tx=0x7fd9c800c800 comp rx=0 tx=0).stop 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 shutdown_connections 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd9b8066f60 0x7fd9b8069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.120 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd9d8071e90 0x7fd9d8155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd9d8071410 0x7fd9d81523b0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 --2- 172.21.15.82:0/4072706509 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd9d8071030 0x7fd9d814f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 >> 172.21.15.82:0/4072706509 conn(0x7fd9d80b11a0 msgr2=0x7fd9d80b0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 shutdown_connections 2024-04-25T10:13:26.121 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.119+0000 7fd9de396640 1 -- 172.21.15.82:0/4072706509 wait complete. 2024-04-25T10:13:26.134 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-25T10:13:25.479358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262412,"kb_used_data":12940,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712308,"statfs":{"total":773094113280,"available":772825403392,"internally_reserved":0,"allocated":13250560,"data_stored":6795256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.372712"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122655+0000","last_change":"2024-04-25T10:13:22.140920+0000","last_active":"2024-04-25T10:13:23.122655+0000","last_peered":"2024-04-25T10:13:23.122655+0000","last_clean":"2024-04-25T10:13:23.122655+0000","last_became_active":"2024-04-25T10:13:22.140198+0000","last_became_peered":"2024-04-25T10:13:22.140198+0000","last_unstale":"2024-04-25T10:13:23.122655+0000","last_undegraded":"2024-04-25T10:13:23.122655+0000","last_fullsized":"2024-04-25T10:13:23.122655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:54.475106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831056+0000","last_change":"2024-04-25T10:13:22.170359+0000","last_active":"2024-04-25T10:13:24.831056+0000","last_peered":"2024-04-25T10:13:24.831056+0000","last_clean":"2024-04-25T10:13:24.831056+0000","last_became_active":"2024-04-25T10:13:22.170005+0000","last_became_peered":"2024-04-25T10:13:22.170005+0000","last_unstale":"2024-04-25T10:13:24.831056+0000","last_undegraded":"2024-04-25T10:13:24.831056+0000","last_fullsized":"2024-04-25T10:13:24.831056+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:30.954353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831557+0000","last_change":"2024-04-25T10:13:22.168641+0000","last_active":"2024-04-25T10:13:24.831557+0000","last_peered":"2024-04-25T10:13:24.831557+0000","last_clean":"2024-04-25T10:13:24.831557+0000","last_became_active":"2024-04-25T10:13:22.168416+0000","last_became_peered":"2024-04-25T10:13:22.168416+0000","last_unstale":"2024-04-25T10:13:24.831557+0000","last_undegraded":"2024-04-25T10:13:24.831557+0000","last_fullsized":"2024-04-25T10:13:24.831557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:28.671762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698420+0000","last_change":"2024-04-25T10:13:22.178046+0000","last_active":"2024-04-25T10:13:24.698420+0000","last_peered":"2024-04-25T10:13:24.698420+0000","last_clean":"2024-04-25T10:13:24.698420+0000","last_became_active":"2024-04-25T10:13:22.174113+0000","last_became_peered":"2024-04-25T10:13:22.174113+0000","last_unstale":"2024-04-25T10:13:24.698420+0000","last_undegraded":"2024-04-25T10:13:24.698420+0000","last_fullsized":"2024-04-25T10:13:24.698420+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121280+0000","last_change":"2024-04-25T10:13:22.164811+0000","last_active":"2024-04-25T10:13:23.121280+0000","last_peered":"2024-04-25T10:13:23.121280+0000","last_clean":"2024-04-25T10:13:23.121280+0000","last_became_active":"2024-04-25T10:13:22.164191+0000","last_became_peered":"2024-04-25T10:13:22.164191+0000","last_unstale":"2024-04-25T10:13:23.121280+0000","last_undegraded":"2024-04-25T10:13:23.121280+0000","last_fullsized":"2024-04-25T10:13:23.121280+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:10.076284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119281+0000","last_change":"2024-04-25T10:13:22.141772+0000","last_active":"2024-04-25T10:13:23.119281+0000","last_peered":"2024-04-25T10:13:23.119281+0000","last_clean":"2024-04-25T10:13:23.119281+0000","last_became_active":"2024-04-25T10:13:22.141514+0000","last_became_peered":"2024-04-25T10:13:22.141514+0000","last_unstale":"2024-04-25T10:13:23.119281+0000","last_undegraded":"2024-04-25T10:13:23.119281+0000","last_fullsized":"2024-04-25T10:13:23.119281+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:04.935575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119695+0000","last_change":"2024-04-25T10:13:22.175230+0000","last_active":"2024-04-25T10:13:23.119695+0000","last_peered":"2024-04-25T10:13:23.119695+0000","last_clean":"2024-04-25T10:13:23.119695+0000","last_became_active":"2024-04-25T10:13:22.174928+0000","last_became_peered":"2024-04-25T10:13:22.174928+0000","last_unstale":"2024-04-25T10:13:23.119695+0000","last_undegraded":"2024-04-25T10:13:23.119695+0000","last_fullsized":"2024-04-25T10:13:23.119695+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:03.284671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830407+0000","last_change":"2024-04-25T10:13:22.150837+0000","last_active":"2024-04-25T10:13:24.830407+0000","last_peered":"2024-04-25T10:13:24.830407+0000","last_clean":"2024-04-25T10:13:24.830407+0000","last_became_active":"2024-04-25T10:13:22.150515+0000","last_became_peered":"2024-04-25T10:13:22.150515+0000","last_unstale":"2024-04-25T10:13:24.830407+0000","last_undegraded":"2024-04-25T10:13:24.830407+0000","last_fullsized":"2024-04-25T10:13:24.830407+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698819+0000","last_change":"2024-04-25T10:13:22.170815+0000","last_active":"2024-04-25T10:13:24.698819+0000","last_peered":"2024-04-25T10:13:24.698819+0000","last_clean":"2024-04-25T10:13:24.698819+0000","last_became_active":"2024-04-25T10:13:22.170175+0000","last_became_peered":"2024-04-25T10:13:22.170175+0000","last_unstale":"2024-04-25T10:13:24.698819+0000","last_undegraded":"2024-04-25T10:13:24.698819+0000","last_fullsized":"2024-04-25T10:13:24.698819+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469594+0000","last_change":"2024-04-25T10:13:22.134687+0000","last_active":"2024-04-25T10:13:24.469594+0000","last_peered":"2024-04-25T10:13:24.469594+0000","last_clean":"2024-04-25T10:13:24.469594+0000","last_became_active":"2024-04-25T10:13:22.134484+0000","last_became_peered":"2024-04-25T10:13:22.134484+0000","last_unstale":"2024-04-25T10:13:24.469594+0000","last_undegraded":"2024-04-25T10:13:24.469594+0000","last_fullsized":"2024-04-25T10:13:24.469594+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:32.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122130+0000","last_change":"2024-04-25T10:13:22.161945+0000","last_active":"2024-04-25T10:13:23.122130+0000","last_peered":"2024-04-25T10:13:23.122130+0000","last_clean":"2024-04-25T10:13:23.122130+0000","last_became_active":"2024-04-25T10:13:22.161131+0000","last_became_peered":"2024-04-25T10:13:22.161131+0000","last_unstale":"2024-04-25T10:13:23.122130+0000","last_undegraded":"2024-04-25T10:13:23.122130+0000","last_fullsized":"2024-04-25T10:13:23.122130+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.387388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120039+0000","last_change":"2024-04-25T10:13:22.164368+0000","last_active":"2024-04-25T10:13:23.120039+0000","last_peered":"2024-04-25T10:13:23.120039+0000","last_clean":"2024-04-25T10:13:23.120039+0000","last_became_active":"2024-04-25T10:13:22.164133+0000","last_became_peered":"2024-04-25T10:13:22.164133+0000","last_unstale":"2024-04-25T10:13:23.120039+0000","last_undegraded":"2024-04-25T10:13:23.120039+0000","last_fullsized":"2024-04-25T10:13:23.120039+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:06.448170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469242+0000","last_change":"2024-04-25T10:13:22.160394+0000","last_active":"2024-04-25T10:13:24.469242+0000","last_peered":"2024-04-25T10:13:24.469242+0000","last_clean":"2024-04-25T10:13:24.469242+0000","last_became_active":"2024-04-25T10:13:22.160035+0000","last_became_peered":"2024-04-25T10:13:22.160035+0000","last_unstale":"2024-04-25T10:13:24.469242+0000","last_undegraded":"2024-04-25T10:13:24.469242+0000","last_fullsized":"2024-04-25T10:13:24.469242+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.105353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831383+0000","last_change":"2024-04-25T10:13:22.176353+0000","last_active":"2024-04-25T10:13:24.831383+0000","last_peered":"2024-04-25T10:13:24.831383+0000","last_clean":"2024-04-25T10:13:24.831383+0000","last_became_active":"2024-04-25T10:13:22.176093+0000","last_became_peered":"2024-04-25T10:13:22.176093+0000","last_unstale":"2024-04-25T10:13:24.831383+0000","last_undegraded":"2024-04-25T10:13:24.831383+0000","last_fullsized":"2024-04-25T10:13:24.831383+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120781+0000","last_change":"2024-04-25T10:13:22.161972+0000","last_active":"2024-04-25T10:13:23.120781+0000","last_peered":"2024-04-25T10:13:23.120781+0000","last_clean":"2024-04-25T10:13:23.120781+0000","last_became_active":"2024-04-25T10:13:22.160381+0000","last_became_peered":"2024-04-25T10:13:22.160381+0000","last_unstale":"2024-04-25T10:13:23.120781+0000","last_undegraded":"2024-04-25T10:13:23.120781+0000","last_fullsized":"2024-04-25T10:13:23.120781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:43.716686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119148+0000","last_change":"2024-04-25T10:13:22.138350+0000","last_active":"2024-04-25T10:13:23.119148+0000","last_peered":"2024-04-25T10:13:23.119148+0000","last_clean":"2024-04-25T10:13:23.119148+0000","last_became_active":"2024-04-25T10:13:22.138032+0000","last_became_peered":"2024-04-25T10:13:22.138032+0000","last_unstale":"2024-04-25T10:13:23.119148+0000","last_undegraded":"2024-04-25T10:13:23.119148+0000","last_fullsized":"2024-04-25T10:13:23.119148+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469797+0000","last_change":"2024-04-25T10:13:22.155563+0000","last_active":"2024-04-25T10:13:24.469797+0000","last_peered":"2024-04-25T10:13:24.469797+0000","last_clean":"2024-04-25T10:13:24.469797+0000","last_became_active":"2024-04-25T10:13:22.155239+0000","last_became_peered":"2024-04-25T10:13:22.155239+0000","last_unstale":"2024-04-25T10:13:24.469797+0000","last_undegraded":"2024-04-25T10:13:24.469797+0000","last_fullsized":"2024-04-25T10:13:24.469797+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831458+0000","last_change":"2024-04-25T10:13:22.156706+0000","last_active":"2024-04-25T10:13:24.831458+0000","last_peered":"2024-04-25T10:13:24.831458+0000","last_clean":"2024-04-25T10:13:24.831458+0000","last_became_active":"2024-04-25T10:13:22.156432+0000","last_became_peered":"2024-04-25T10:13:22.156432+0000","last_unstale":"2024-04-25T10:13:24.831458+0000","last_undegraded":"2024-04-25T10:13:24.831458+0000","last_fullsized":"2024-04-25T10:13:24.831458+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:36.752369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121428+0000","last_change":"2024-04-25T10:13:22.178118+0000","last_active":"2024-04-25T10:13:23.121428+0000","last_peered":"2024-04-25T10:13:23.121428+0000","last_clean":"2024-04-25T10:13:23.121428+0000","last_became_active":"2024-04-25T10:13:22.177680+0000","last_became_peered":"2024-04-25T10:13:22.177680+0000","last_unstale":"2024-04-25T10:13:23.121428+0000","last_undegraded":"2024-04-25T10:13:23.121428+0000","last_fullsized":"2024-04-25T10:13:23.121428+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832113+0000","last_change":"2024-04-25T10:13:22.138209+0000","last_active":"2024-04-25T10:13:24.832113+0000","last_peered":"2024-04-25T10:13:24.832113+0000","last_clean":"2024-04-25T10:13:24.832113+0000","last_became_active":"2024-04-25T10:13:22.137892+0000","last_became_peered":"2024-04-25T10:13:22.137892+0000","last_unstale":"2024-04-25T10:13:24.832113+0000","last_undegraded":"2024-04-25T10:13:24.832113+0000","last_fullsized":"2024-04-25T10:13:24.832113+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:08.214176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698263+0000","last_change":"2024-04-25T10:13:22.172056+0000","last_active":"2024-04-25T10:13:24.698263+0000","last_peered":"2024-04-25T10:13:24.698263+0000","last_clean":"2024-04-25T10:13:24.698263+0000","last_became_active":"2024-04-25T10:13:22.171405+0000","last_became_peered":"2024-04-25T10:13:22.171405+0000","last_unstale":"2024-04-25T10:13:24.698263+0000","last_undegraded":"2024-04-25T10:13:24.698263+0000","last_fullsized":"2024-04-25T10:13:24.698263+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121512+0000","last_change":"2024-04-25T10:13:22.145457+0000","last_active":"2024-04-25T10:13:23.121512+0000","last_peered":"2024-04-25T10:13:23.121512+0000","last_clean":"2024-04-25T10:13:23.121512+0000","last_became_active":"2024-04-25T10:13:22.144567+0000","last_became_peered":"2024-04-25T10:13:22.144567+0000","last_unstale":"2024-04-25T10:13:23.121512+0000","last_undegraded":"2024-04-25T10:13:23.121512+0000","last_fullsized":"2024-04-25T10:13:23.121512+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.105353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698009+0000","last_change":"2024-04-25T10:13:22.174646+0000","last_active":"2024-04-25T10:13:24.698009+0000","last_peered":"2024-04-25T10:13:24.698009+0000","last_clean":"2024-04-25T10:13:24.698009+0000","last_became_active":"2024-04-25T10:13:22.173972+0000","last_became_peered":"2024-04-25T10:13:22.173972+0000","last_unstale":"2024-04-25T10:13:24.698009+0000","last_undegraded":"2024-04-25T10:13:24.698009+0000","last_fullsized":"2024-04-25T10:13:24.698009+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:59.496070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829777+0000","last_change":"2024-04-25T10:13:22.179273+0000","last_active":"2024-04-25T10:13:24.829777+0000","last_peered":"2024-04-25T10:13:24.829777+0000","last_clean":"2024-04-25T10:13:24.829777+0000","last_became_active":"2024-04-25T10:13:22.177205+0000","last_became_peered":"2024-04-25T10:13:22.177205+0000","last_unstale":"2024-04-25T10:13:24.829777+0000","last_undegraded":"2024-04-25T10:13:24.829777+0000","last_fullsized":"2024-04-25T10:13:24.829777+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:42.499470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830628+0000","last_change":"2024-04-25T10:13:22.165684+0000","last_active":"2024-04-25T10:13:24.830628+0000","last_peered":"2024-04-25T10:13:24.830628+0000","last_clean":"2024-04-25T10:13:24.830628+0000","last_became_active":"2024-04-25T10:13:22.164972+0000","last_became_peered":"2024-04-25T10:13:22.164972+0000","last_unstale":"2024-04-25T10:13:24.830628+0000","last_undegraded":"2024-04-25T10:13:24.830628+0000","last_fullsized":"2024-04-25T10:13:24.830628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:59.496070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831640+0000","last_change":"2024-04-25T10:13:22.130217+0000","last_active":"2024-04-25T10:13:24.831640+0000","last_peered":"2024-04-25T10:13:24.831640+0000","last_clean":"2024-04-25T10:13:24.831640+0000","last_became_active":"2024-04-25T10:13:22.129934+0000","last_became_peered":"2024-04-25T10:13:22.129934+0000","last_unstale":"2024-04-25T10:13:24.831640+0000","last_undegraded":"2024-04-25T10:13:24.831640+0000","last_fullsized":"2024-04-25T10:13:24.831640+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.872588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121854+0000","last_change":"2024-04-25T10:13:22.171590+0000","last_active":"2024-04-25T10:13:23.121854+0000","last_peered":"2024-04-25T10:13:23.121854+0000","last_clean":"2024-04-25T10:13:23.121854+0000","last_became_active":"2024-04-25T10:13:22.170946+0000","last_became_peered":"2024-04-25T10:13:22.170946+0000","last_unstale":"2024-04-25T10:13:23.121854+0000","last_undegraded":"2024-04-25T10:13:23.121854+0000","last_fullsized":"2024-04-25T10:13:23.121854+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:42.499470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119944+0000","last_change":"2024-04-25T10:13:22.153736+0000","last_active":"2024-04-25T10:13:23.119944+0000","last_peered":"2024-04-25T10:13:23.119944+0000","last_clean":"2024-04-25T10:13:23.119944+0000","last_became_active":"2024-04-25T10:13:22.153476+0000","last_became_peered":"2024-04-25T10:13:22.153476+0000","last_unstale":"2024-04-25T10:13:23.119944+0000","last_undegraded":"2024-04-25T10:13:23.119944+0000","last_fullsized":"2024-04-25T10:13:23.119944+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:10.382084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832422+0000","last_change":"2024-04-25T10:13:22.133755+0000","last_active":"2024-04-25T10:13:24.832422+0000","last_peered":"2024-04-25T10:13:24.832422+0000","last_clean":"2024-04-25T10:13:24.832422+0000","last_became_active":"2024-04-25T10:13:22.133433+0000","last_became_peered":"2024-04-25T10:13:22.133433+0000","last_unstale":"2024-04-25T10:13:24.832422+0000","last_undegraded":"2024-04-25T10:13:24.832422+0000","last_fullsized":"2024-04-25T10:13:24.832422+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:22.231820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831429+0000","last_change":"2024-04-25T10:13:22.163180+0000","last_active":"2024-04-25T10:13:24.831429+0000","last_peered":"2024-04-25T10:13:24.831429+0000","last_clean":"2024-04-25T10:13:24.831429+0000","last_became_active":"2024-04-25T10:13:22.162882+0000","last_became_peered":"2024-04-25T10:13:22.162882+0000","last_unstale":"2024-04-25T10:13:24.831429+0000","last_undegraded":"2024-04-25T10:13:24.831429+0000","last_fullsized":"2024-04-25T10:13:24.831429+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:08.933078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829947+0000","last_change":"2024-04-25T10:13:22.151875+0000","last_active":"2024-04-25T10:13:24.829947+0000","last_peered":"2024-04-25T10:13:24.829947+0000","last_clean":"2024-04-25T10:13:24.829947+0000","last_became_active":"2024-04-25T10:13:22.151124+0000","last_became_peered":"2024-04-25T10:13:22.151124+0000","last_unstale":"2024-04-25T10:13:24.829947+0000","last_undegraded":"2024-04-25T10:13:24.829947+0000","last_fullsized":"2024-04-25T10:13:24.829947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:50.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829756+0000","last_change":"2024-04-25T10:13:22.178679+0000","last_active":"2024-04-25T10:13:24.829756+0000","last_peered":"2024-04-25T10:13:24.829756+0000","last_clean":"2024-04-25T10:13:24.829756+0000","last_became_active":"2024-04-25T10:13:22.176484+0000","last_became_peered":"2024-04-25T10:13:22.176484+0000","last_unstale":"2024-04-25T10:13:24.829756+0000","last_undegraded":"2024-04-25T10:13:24.829756+0000","last_fullsized":"2024-04-25T10:13:24.829756+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469190+0000","last_change":"2024-04-25T10:13:22.134103+0000","last_active":"2024-04-25T10:13:24.469190+0000","last_peered":"2024-04-25T10:13:24.469190+0000","last_clean":"2024-04-25T10:13:24.469190+0000","last_became_active":"2024-04-25T10:13:22.133899+0000","last_became_peered":"2024-04-25T10:13:22.133899+0000","last_unstale":"2024-04-25T10:13:24.469190+0000","last_undegraded":"2024-04-25T10:13:24.469190+0000","last_fullsized":"2024-04-25T10:13:24.469190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:10.076284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121263+0000","last_change":"2024-04-25T10:13:21.157367+0000","last_active":"2024-04-25T10:13:23.121263+0000","last_peered":"2024-04-25T10:13:23.121263+0000","last_clean":"2024-04-25T10:13:23.121263+0000","last_became_active":"2024-04-25T10:13:21.156537+0000","last_became_peered":"2024-04-25T10:13:21.156537+0000","last_unstale":"2024-04-25T10:13:23.121263+0000","last_undegraded":"2024-04-25T10:13:23.121263+0000","last_fullsized":"2024-04-25T10:13:23.121263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119708+0000","last_change":"2024-04-25T10:13:22.140984+0000","last_active":"2024-04-25T10:13:23.119708+0000","last_peered":"2024-04-25T10:13:23.119708+0000","last_clean":"2024-04-25T10:13:23.119708+0000","last_became_active":"2024-04-25T10:13:22.140710+0000","last_became_peered":"2024-04-25T10:13:22.140710+0000","last_unstale":"2024-04-25T10:13:23.119708+0000","last_undegraded":"2024-04-25T10:13:23.119708+0000","last_fullsized":"2024-04-25T10:13:23.119708+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:56.599361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469587+0000","last_change":"2024-04-25T10:13:21.133173+0000","last_active":"2024-04-25T10:13:24.469587+0000","last_peered":"2024-04-25T10:13:24.469587+0000","last_clean":"2024-04-25T10:13:24.469587+0000","last_became_active":"2024-04-25T10:13:21.132811+0000","last_became_peered":"2024-04-25T10:13:21.132811+0000","last_unstale":"2024-04-25T10:13:24.469587+0000","last_undegraded":"2024-04-25T10:13:24.469587+0000","last_fullsized":"2024-04-25T10:13:24.469587+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:48.803866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830297+0000","last_change":"2024-04-25T10:13:22.155705+0000","last_active":"2024-04-25T10:13:24.830297+0000","last_peered":"2024-04-25T10:13:24.830297+0000","last_clean":"2024-04-25T10:13:24.830297+0000","last_became_active":"2024-04-25T10:13:22.154950+0000","last_became_peered":"2024-04-25T10:13:22.154950+0000","last_unstale":"2024-04-25T10:13:24.830297+0000","last_undegraded":"2024-04-25T10:13:24.830297+0000","last_fullsized":"2024-04-25T10:13:24.830297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:55.710602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698616+0000","last_change":"2024-04-25T10:13:21.157108+0000","last_active":"2024-04-25T10:13:24.698616+0000","last_peered":"2024-04-25T10:13:24.698616+0000","last_clean":"2024-04-25T10:13:24.698616+0000","last_became_active":"2024-04-25T10:13:21.156170+0000","last_became_peered":"2024-04-25T10:13:21.156170+0000","last_unstale":"2024-04-25T10:13:24.698616+0000","last_undegraded":"2024-04-25T10:13:24.698616+0000","last_fullsized":"2024-04-25T10:13:24.698616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698988+0000","last_change":"2024-04-25T10:13:22.178710+0000","last_active":"2024-04-25T10:13:24.698988+0000","last_peered":"2024-04-25T10:13:24.698988+0000","last_clean":"2024-04-25T10:13:24.698988+0000","last_became_active":"2024-04-25T10:13:22.178102+0000","last_became_peered":"2024-04-25T10:13:22.178102+0000","last_unstale":"2024-04-25T10:13:24.698988+0000","last_undegraded":"2024-04-25T10:13:24.698988+0000","last_fullsized":"2024-04-25T10:13:24.698988+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:08.933078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469305+0000","last_change":"2024-04-25T10:13:21.135597+0000","last_active":"2024-04-25T10:13:24.469305+0000","last_peered":"2024-04-25T10:13:24.469305+0000","last_clean":"2024-04-25T10:13:24.469305+0000","last_became_active":"2024-04-25T10:13:21.135265+0000","last_became_peered":"2024-04-25T10:13:21.135265+0000","last_unstale":"2024-04-25T10:13:24.469305+0000","last_undegraded":"2024-04-25T10:13:24.469305+0000","last_fullsized":"2024-04-25T10:13:24.469305+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:39.638106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469485+0000","last_change":"2024-04-25T10:13:22.130931+0000","last_active":"2024-04-25T10:13:24.469485+0000","last_peered":"2024-04-25T10:13:24.469485+0000","last_clean":"2024-04-25T10:13:24.469485+0000","last_became_active":"2024-04-25T10:13:22.130660+0000","last_became_peered":"2024-04-25T10:13:22.130660+0000","last_unstale":"2024-04-25T10:13:24.469485+0000","last_undegraded":"2024-04-25T10:13:24.469485+0000","last_fullsized":"2024-04-25T10:13:24.469485+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.602657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121236+0000","last_change":"2024-04-25T10:13:21.134313+0000","last_active":"2024-04-25T10:13:23.121236+0000","last_peered":"2024-04-25T10:13:23.121236+0000","last_clean":"2024-04-25T10:13:23.121236+0000","last_became_active":"2024-04-25T10:13:21.133430+0000","last_became_peered":"2024-04-25T10:13:21.133430+0000","last_unstale":"2024-04-25T10:13:23.121236+0000","last_undegraded":"2024-04-25T10:13:23.121236+0000","last_fullsized":"2024-04-25T10:13:23.121236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:00.827119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120723+0000","last_change":"2024-04-25T10:13:22.162338+0000","last_active":"2024-04-25T10:13:23.120723+0000","last_peered":"2024-04-25T10:13:23.120723+0000","last_clean":"2024-04-25T10:13:23.120723+0000","last_became_active":"2024-04-25T10:13:22.161285+0000","last_became_peered":"2024-04-25T10:13:22.161285+0000","last_unstale":"2024-04-25T10:13:23.120723+0000","last_undegraded":"2024-04-25T10:13:23.120723+0000","last_fullsized":"2024-04-25T10:13:23.120723+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:02.670910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119155+0000","last_change":"2024-04-25T10:13:21.141367+0000","last_active":"2024-04-25T10:13:23.119155+0000","last_peered":"2024-04-25T10:13:23.119155+0000","last_clean":"2024-04-25T10:13:23.119155+0000","last_became_active":"2024-04-25T10:13:21.141007+0000","last_became_peered":"2024-04-25T10:13:21.141007+0000","last_unstale":"2024-04-25T10:13:23.119155+0000","last_undegraded":"2024-04-25T10:13:23.119155+0000","last_fullsized":"2024-04-25T10:13:23.119155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:39.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831818+0000","last_change":"2024-04-25T10:13:22.135859+0000","last_active":"2024-04-25T10:13:24.831818+0000","last_peered":"2024-04-25T10:13:24.831818+0000","last_clean":"2024-04-25T10:13:24.831818+0000","last_became_active":"2024-04-25T10:13:22.135500+0000","last_became_peered":"2024-04-25T10:13:22.135500+0000","last_unstale":"2024-04-25T10:13:24.831818+0000","last_undegraded":"2024-04-25T10:13:24.831818+0000","last_fullsized":"2024-04-25T10:13:24.831818+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:26.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829946+0000","last_change":"2024-04-25T10:13:21.140271+0000","last_active":"2024-04-25T10:13:24.829946+0000","last_peered":"2024-04-25T10:13:24.829946+0000","last_clean":"2024-04-25T10:13:24.829946+0000","last_became_active":"2024-04-25T10:13:21.139166+0000","last_became_peered":"2024-04-25T10:13:21.139166+0000","last_unstale":"2024-04-25T10:13:24.829946+0000","last_undegraded":"2024-04-25T10:13:24.829946+0000","last_fullsized":"2024-04-25T10:13:24.829946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469518+0000","last_change":"2024-04-25T10:13:22.134349+0000","last_active":"2024-04-25T10:13:24.469518+0000","last_peered":"2024-04-25T10:13:24.469518+0000","last_clean":"2024-04-25T10:13:24.469518+0000","last_became_active":"2024-04-25T10:13:22.133511+0000","last_became_peered":"2024-04-25T10:13:22.133511+0000","last_unstale":"2024-04-25T10:13:24.469518+0000","last_undegraded":"2024-04-25T10:13:24.469518+0000","last_fullsized":"2024-04-25T10:13:24.469518+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:54.475106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119407+0000","last_change":"2024-04-25T10:13:21.135976+0000","last_active":"2024-04-25T10:13:23.119407+0000","last_peered":"2024-04-25T10:13:23.119407+0000","last_clean":"2024-04-25T10:13:23.119407+0000","last_became_active":"2024-04-25T10:13:21.135448+0000","last_became_peered":"2024-04-25T10:13:21.135448+0000","last_unstale":"2024-04-25T10:13:23.119407+0000","last_undegraded":"2024-04-25T10:13:23.119407+0000","last_fullsized":"2024-04-25T10:13:23.119407+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:45.481869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120592+0000","last_change":"2024-04-25T10:13:22.172294+0000","last_active":"2024-04-25T10:13:23.120592+0000","last_peered":"2024-04-25T10:13:23.120592+0000","last_clean":"2024-04-25T10:13:23.120592+0000","last_became_active":"2024-04-25T10:13:22.170734+0000","last_became_peered":"2024-04-25T10:13:22.170734+0000","last_unstale":"2024-04-25T10:13:23.120592+0000","last_undegraded":"2024-04-25T10:13:23.120592+0000","last_fullsized":"2024-04-25T10:13:23.120592+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:32.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122161+0000","last_change":"2024-04-25T10:13:21.136753+0000","last_active":"2024-04-25T10:13:23.122161+0000","last_peered":"2024-04-25T10:13:23.122161+0000","last_clean":"2024-04-25T10:13:23.122161+0000","last_became_active":"2024-04-25T10:13:21.135942+0000","last_became_peered":"2024-04-25T10:13:21.135942+0000","last_unstale":"2024-04-25T10:13:23.122161+0000","last_undegraded":"2024-04-25T10:13:23.122161+0000","last_fullsized":"2024-04-25T10:13:23.122161+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830431+0000","last_change":"2024-04-25T10:13:22.151962+0000","last_active":"2024-04-25T10:13:24.830431+0000","last_peered":"2024-04-25T10:13:24.830431+0000","last_clean":"2024-04-25T10:13:24.830431+0000","last_became_active":"2024-04-25T10:13:22.151288+0000","last_became_peered":"2024-04-25T10:13:22.151288+0000","last_unstale":"2024-04-25T10:13:24.830431+0000","last_undegraded":"2024-04-25T10:13:24.830431+0000","last_fullsized":"2024-04-25T10:13:24.830431+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:05.796548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830324+0000","last_change":"2024-04-25T10:13:21.142538+0000","last_active":"2024-04-25T10:13:24.830324+0000","last_peered":"2024-04-25T10:13:24.830324+0000","last_clean":"2024-04-25T10:13:24.830324+0000","last_became_active":"2024-04-25T10:13:21.141723+0000","last_became_peered":"2024-04-25T10:13:21.141723+0000","last_unstale":"2024-04-25T10:13:24.830324+0000","last_undegraded":"2024-04-25T10:13:24.830324+0000","last_fullsized":"2024-04-25T10:13:24.830324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:55.291423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832138+0000","last_change":"2024-04-25T10:13:22.154863+0000","last_active":"2024-04-25T10:13:24.832138+0000","last_peered":"2024-04-25T10:13:24.832138+0000","last_clean":"2024-04-25T10:13:24.832138+0000","last_became_active":"2024-04-25T10:13:22.154548+0000","last_became_peered":"2024-04-25T10:13:22.154548+0000","last_unstale":"2024-04-25T10:13:24.832138+0000","last_undegraded":"2024-04-25T10:13:24.832138+0000","last_fullsized":"2024-04-25T10:13:24.832138+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.684867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830465+0000","last_change":"2024-04-25T10:13:21.149283+0000","last_active":"2024-04-25T10:13:24.830465+0000","last_peered":"2024-04-25T10:13:24.830465+0000","last_clean":"2024-04-25T10:13:24.830465+0000","last_became_active":"2024-04-25T10:13:21.148399+0000","last_became_peered":"2024-04-25T10:13:21.148399+0000","last_unstale":"2024-04-25T10:13:24.830465+0000","last_undegraded":"2024-04-25T10:13:24.830465+0000","last_fullsized":"2024-04-25T10:13:24.830465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.767138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830725+0000","last_change":"2024-04-25T10:13:22.153448+0000","last_active":"2024-04-25T10:13:24.830725+0000","last_peered":"2024-04-25T10:13:24.830725+0000","last_clean":"2024-04-25T10:13:24.830725+0000","last_became_active":"2024-04-25T10:13:22.144919+0000","last_became_peered":"2024-04-25T10:13:22.144919+0000","last_unstale":"2024-04-25T10:13:24.830725+0000","last_undegraded":"2024-04-25T10:13:24.830725+0000","last_fullsized":"2024-04-25T10:13:24.830725+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832597+0000","last_change":"2024-04-25T10:13:21.136638+0000","last_active":"2024-04-25T10:13:24.832597+0000","last_peered":"2024-04-25T10:13:24.832597+0000","last_clean":"2024-04-25T10:13:24.832597+0000","last_became_active":"2024-04-25T10:13:21.136274+0000","last_became_peered":"2024-04-25T10:13:21.136274+0000","last_unstale":"2024-04-25T10:13:24.832597+0000","last_undegraded":"2024-04-25T10:13:24.832597+0000","last_fullsized":"2024-04-25T10:13:24.832597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120911+0000","last_change":"2024-04-25T10:13:22.169587+0000","last_active":"2024-04-25T10:13:23.120911+0000","last_peered":"2024-04-25T10:13:23.120911+0000","last_clean":"2024-04-25T10:13:23.120911+0000","last_became_active":"2024-04-25T10:13:22.168905+0000","last_became_peered":"2024-04-25T10:13:22.168905+0000","last_unstale":"2024-04-25T10:13:23.120911+0000","last_undegraded":"2024-04-25T10:13:23.120911+0000","last_fullsized":"2024-04-25T10:13:23.120911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:49.588817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831430+0000","last_change":"2024-04-25T10:13:21.156529+0000","last_active":"2024-04-25T10:13:24.831430+0000","last_peered":"2024-04-25T10:13:24.831430+0000","last_clean":"2024-04-25T10:13:24.831430+0000","last_became_active":"2024-04-25T10:13:21.156141+0000","last_became_peered":"2024-04-25T10:13:21.156141+0000","last_unstale":"2024-04-25T10:13:24.831430+0000","last_undegraded":"2024-04-25T10:13:24.831430+0000","last_fullsized":"2024-04-25T10:13:24.831430+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469670+0000","last_change":"2024-04-25T10:13:22.178090+0000","last_active":"2024-04-25T10:13:24.469670+0000","last_peered":"2024-04-25T10:13:24.469670+0000","last_clean":"2024-04-25T10:13:24.469670+0000","last_became_active":"2024-04-25T10:13:22.177870+0000","last_became_peered":"2024-04-25T10:13:22.177870+0000","last_unstale":"2024-04-25T10:13:24.469670+0000","last_undegraded":"2024-04-25T10:13:24.469670+0000","last_fullsized":"2024-04-25T10:13:24.469670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:49.588817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832307+0000","last_change":"2024-04-25T10:13:21.150192+0000","last_active":"2024-04-25T10:13:24.832307+0000","last_peered":"2024-04-25T10:13:24.832307+0000","last_clean":"2024-04-25T10:13:24.832307+0000","last_became_active":"2024-04-25T10:13:21.149784+0000","last_became_peered":"2024-04-25T10:13:21.149784+0000","last_unstale":"2024-04-25T10:13:24.832307+0000","last_undegraded":"2024-04-25T10:13:24.832307+0000","last_fullsized":"2024-04-25T10:13:24.832307+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.378823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832613+0000","last_change":"2024-04-25T10:13:22.146064+0000","last_active":"2024-04-25T10:13:24.832613+0000","last_peered":"2024-04-25T10:13:24.832613+0000","last_clean":"2024-04-25T10:13:24.832613+0000","last_became_active":"2024-04-25T10:13:22.145779+0000","last_became_peered":"2024-04-25T10:13:22.145779+0000","last_unstale":"2024-04-25T10:13:24.832613+0000","last_undegraded":"2024-04-25T10:13:24.832613+0000","last_fullsized":"2024-04-25T10:13:24.832613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.128817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830811+0000","last_change":"2024-04-25T10:13:21.136420+0000","last_active":"2024-04-25T10:13:24.830811+0000","last_peered":"2024-04-25T10:13:24.830811+0000","last_clean":"2024-04-25T10:13:24.830811+0000","last_became_active":"2024-04-25T10:13:21.136121+0000","last_became_peered":"2024-04-25T10:13:21.136121+0000","last_unstale":"2024-04-25T10:13:24.830811+0000","last_undegraded":"2024-04-25T10:13:24.830811+0000","last_fullsized":"2024-04-25T10:13:24.830811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830645+0000","last_change":"2024-04-25T10:13:22.149933+0000","last_active":"2024-04-25T10:13:24.830645+0000","last_peered":"2024-04-25T10:13:24.830645+0000","last_clean":"2024-04-25T10:13:24.830645+0000","last_became_active":"2024-04-25T10:13:22.149205+0000","last_became_peered":"2024-04-25T10:13:22.149205+0000","last_unstale":"2024-04-25T10:13:24.830645+0000","last_undegraded":"2024-04-25T10:13:24.830645+0000","last_fullsized":"2024-04-25T10:13:24.830645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.470033+0000","last_change":"2024-04-25T10:13:21.139798+0000","last_active":"2024-04-25T10:13:24.470033+0000","last_peered":"2024-04-25T10:13:24.470033+0000","last_clean":"2024-04-25T10:13:24.470033+0000","last_became_active":"2024-04-25T10:13:21.139442+0000","last_became_peered":"2024-04-25T10:13:21.139442+0000","last_unstale":"2024-04-25T10:13:24.470033+0000","last_undegraded":"2024-04-25T10:13:24.470033+0000","last_fullsized":"2024-04-25T10:13:24.470033+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:38.282002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122480+0000","last_change":"2024-04-25T10:13:22.145387+0000","last_active":"2024-04-25T10:13:23.122480+0000","last_peered":"2024-04-25T10:13:23.122480+0000","last_clean":"2024-04-25T10:13:23.122480+0000","last_became_active":"2024-04-25T10:13:22.144610+0000","last_became_peered":"2024-04-25T10:13:22.144610+0000","last_unstale":"2024-04-25T10:13:23.122480+0000","last_undegraded":"2024-04-25T10:13:23.122480+0000","last_fullsized":"2024-04-25T10:13:23.122480+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.602657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831986+0000","last_change":"2024-04-25T10:13:21.132512+0000","last_active":"2024-04-25T10:13:24.831986+0000","last_peered":"2024-04-25T10:13:24.831986+0000","last_clean":"2024-04-25T10:13:24.831986+0000","last_became_active":"2024-04-25T10:13:21.132057+0000","last_became_peered":"2024-04-25T10:13:21.132057+0000","last_unstale":"2024-04-25T10:13:24.831986+0000","last_undegraded":"2024-04-25T10:13:24.831986+0000","last_fullsized":"2024-04-25T10:13:24.831986+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119770+0000","last_change":"2024-04-25T10:13:22.155284+0000","last_active":"2024-04-25T10:13:23.119770+0000","last_peered":"2024-04-25T10:13:23.119770+0000","last_clean":"2024-04-25T10:13:23.119770+0000","last_became_active":"2024-04-25T10:13:22.154998+0000","last_became_peered":"2024-04-25T10:13:22.154998+0000","last_unstale":"2024-04-25T10:13:23.119770+0000","last_undegraded":"2024-04-25T10:13:23.119770+0000","last_fullsized":"2024-04-25T10:13:23.119770+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:17.402902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698772+0000","last_change":"2024-04-25T10:13:21.158040+0000","last_active":"2024-04-25T10:13:24.698772+0000","last_peered":"2024-04-25T10:13:24.698772+0000","last_clean":"2024-04-25T10:13:24.698772+0000","last_became_active":"2024-04-25T10:13:21.157331+0000","last_became_peered":"2024-04-25T10:13:21.157331+0000","last_unstale":"2024-04-25T10:13:24.698772+0000","last_undegraded":"2024-04-25T10:13:24.698772+0000","last_fullsized":"2024-04-25T10:13:24.698772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.378823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830838+0000","last_change":"2024-04-25T10:13:22.177488+0000","last_active":"2024-04-25T10:13:24.830838+0000","last_peered":"2024-04-25T10:13:24.830838+0000","last_clean":"2024-04-25T10:13:24.830838+0000","last_became_active":"2024-04-25T10:13:22.176824+0000","last_became_peered":"2024-04-25T10:13:22.176824+0000","last_unstale":"2024-04-25T10:13:24.830838+0000","last_undegraded":"2024-04-25T10:13:24.830838+0000","last_fullsized":"2024-04-25T10:13:24.830838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698461+0000","last_change":"2024-04-25T10:13:21.157947+0000","last_active":"2024-04-25T10:13:24.698461+0000","last_peered":"2024-04-25T10:13:24.698461+0000","last_clean":"2024-04-25T10:13:24.698461+0000","last_became_active":"2024-04-25T10:13:21.157130+0000","last_became_peered":"2024-04-25T10:13:21.157130+0000","last_unstale":"2024-04-25T10:13:24.698461+0000","last_undegraded":"2024-04-25T10:13:24.698461+0000","last_fullsized":"2024-04-25T10:13:24.698461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119135+0000","last_change":"2024-04-25T10:13:22.171686+0000","last_active":"2024-04-25T10:13:23.119135+0000","last_peered":"2024-04-25T10:13:23.119135+0000","last_clean":"2024-04-25T10:13:23.119135+0000","last_became_active":"2024-04-25T10:13:22.171245+0000","last_became_peered":"2024-04-25T10:13:22.171245+0000","last_unstale":"2024-04-25T10:13:23.119135+0000","last_undegraded":"2024-04-25T10:13:23.119135+0000","last_fullsized":"2024-04-25T10:13:23.119135+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:24.894164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119316+0000","last_change":"2024-04-25T10:13:21.155110+0000","last_active":"2024-04-25T10:13:23.119316+0000","last_peered":"2024-04-25T10:13:23.119316+0000","last_clean":"2024-04-25T10:13:23.119316+0000","last_became_active":"2024-04-25T10:13:21.154755+0000","last_became_peered":"2024-04-25T10:13:21.154755+0000","last_unstale":"2024-04-25T10:13:23.119316+0000","last_undegraded":"2024-04-25T10:13:23.119316+0000","last_fullsized":"2024-04-25T10:13:23.119316+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:06.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830536+0000","last_change":"2024-04-25T10:13:22.158950+0000","last_active":"2024-04-25T10:13:24.830536+0000","last_peered":"2024-04-25T10:13:24.830536+0000","last_clean":"2024-04-25T10:13:24.830536+0000","last_became_active":"2024-04-25T10:13:22.158728+0000","last_became_peered":"2024-04-25T10:13:22.158728+0000","last_unstale":"2024-04-25T10:13:24.830536+0000","last_undegraded":"2024-04-25T10:13:24.830536+0000","last_fullsized":"2024-04-25T10:13:24.830536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697922+0000","last_change":"2024-04-25T10:13:21.156709+0000","last_active":"2024-04-25T10:13:24.697922+0000","last_peered":"2024-04-25T10:13:24.697922+0000","last_clean":"2024-04-25T10:13:24.697922+0000","last_became_active":"2024-04-25T10:13:21.155827+0000","last_became_peered":"2024-04-25T10:13:21.155827+0000","last_unstale":"2024-04-25T10:13:24.697922+0000","last_undegraded":"2024-04-25T10:13:24.697922+0000","last_fullsized":"2024-04-25T10:13:24.697922+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832381+0000","last_change":"2024-04-25T10:13:22.155227+0000","last_active":"2024-04-25T10:13:24.832381+0000","last_peered":"2024-04-25T10:13:24.832381+0000","last_clean":"2024-04-25T10:13:24.832381+0000","last_became_active":"2024-04-25T10:13:22.154516+0000","last_became_peered":"2024-04-25T10:13:22.154516+0000","last_unstale":"2024-04-25T10:13:24.832381+0000","last_undegraded":"2024-04-25T10:13:24.832381+0000","last_fullsized":"2024-04-25T10:13:24.832381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:07.205431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830992+0000","last_change":"2024-04-25T10:13:21.132768+0000","last_active":"2024-04-25T10:13:24.830992+0000","last_peered":"2024-04-25T10:13:24.830992+0000","last_clean":"2024-04-25T10:13:24.830992+0000","last_became_active":"2024-04-25T10:13:21.132275+0000","last_became_peered":"2024-04-25T10:13:21.132275+0000","last_unstale":"2024-04-25T10:13:24.830992+0000","last_undegraded":"2024-04-25T10:13:24.830992+0000","last_fullsized":"2024-04-25T10:13:24.830992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830114+0000","last_change":"2024-04-25T10:13:22.151164+0000","last_active":"2024-04-25T10:13:24.830114+0000","last_peered":"2024-04-25T10:13:24.830114+0000","last_clean":"2024-04-25T10:13:24.830114+0000","last_became_active":"2024-04-25T10:13:22.149322+0000","last_became_peered":"2024-04-25T10:13:22.149322+0000","last_unstale":"2024-04-25T10:13:24.830114+0000","last_undegraded":"2024-04-25T10:13:24.830114+0000","last_fullsized":"2024-04-25T10:13:24.830114+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:17.897835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697439+0000","last_change":"2024-04-25T10:13:21.156461+0000","last_active":"2024-04-25T10:13:24.697439+0000","last_peered":"2024-04-25T10:13:24.697439+0000","last_clean":"2024-04-25T10:13:24.697439+0000","last_became_active":"2024-04-25T10:13:21.155691+0000","last_became_peered":"2024-04-25T10:13:21.155691+0000","last_unstale":"2024-04-25T10:13:24.697439+0000","last_undegraded":"2024-04-25T10:13:24.697439+0000","last_fullsized":"2024-04-25T10:13:24.697439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120153+0000","last_change":"2024-04-25T10:13:22.139584+0000","last_active":"2024-04-25T10:13:23.120153+0000","last_peered":"2024-04-25T10:13:23.120153+0000","last_clean":"2024-04-25T10:13:23.120153+0000","last_became_active":"2024-04-25T10:13:22.139013+0000","last_became_peered":"2024-04-25T10:13:22.139013+0000","last_unstale":"2024-04-25T10:13:23.120153+0000","last_undegraded":"2024-04-25T10:13:23.120153+0000","last_fullsized":"2024-04-25T10:13:23.120153+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.772014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120219+0000","last_change":"2024-04-25T10:13:21.135789+0000","last_active":"2024-04-25T10:13:23.120219+0000","last_peered":"2024-04-25T10:13:23.120219+0000","last_clean":"2024-04-25T10:13:23.120219+0000","last_became_active":"2024-04-25T10:13:21.135342+0000","last_became_peered":"2024-04-25T10:13:21.135342+0000","last_unstale":"2024-04-25T10:13:23.120219+0000","last_undegraded":"2024-04-25T10:13:23.120219+0000","last_fullsized":"2024-04-25T10:13:23.120219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:22.870747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469284+0000","last_change":"2024-04-25T10:13:22.131588+0000","last_active":"2024-04-25T10:13:24.469284+0000","last_peered":"2024-04-25T10:13:24.469284+0000","last_clean":"2024-04-25T10:13:24.469284+0000","last_became_active":"2024-04-25T10:13:22.130293+0000","last_became_peered":"2024-04-25T10:13:22.130293+0000","last_unstale":"2024-04-25T10:13:24.469284+0000","last_undegraded":"2024-04-25T10:13:24.469284+0000","last_fullsized":"2024-04-25T10:13:24.469284+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.387388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.124643+0000","last_change":"2024-04-25T10:13:21.136901+0000","last_active":"2024-04-25T10:13:23.124643+0000","last_peered":"2024-04-25T10:13:23.124643+0000","last_clean":"2024-04-25T10:13:23.124643+0000","last_became_active":"2024-04-25T10:13:21.135921+0000","last_became_peered":"2024-04-25T10:13:21.135921+0000","last_unstale":"2024-04-25T10:13:23.124643+0000","last_undegraded":"2024-04-25T10:13:23.124643+0000","last_fullsized":"2024-04-25T10:13:23.124643+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698756+0000","last_change":"2024-04-25T10:13:22.175854+0000","last_active":"2024-04-25T10:13:24.698756+0000","last_peered":"2024-04-25T10:13:24.698756+0000","last_clean":"2024-04-25T10:13:24.698756+0000","last_became_active":"2024-04-25T10:13:22.174725+0000","last_became_peered":"2024-04-25T10:13:22.174725+0000","last_unstale":"2024-04-25T10:13:24.698756+0000","last_undegraded":"2024-04-25T10:13:24.698756+0000","last_fullsized":"2024-04-25T10:13:24.698756+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:30.954353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121716+0000","last_change":"2024-04-25T10:13:21.137409+0000","last_active":"2024-04-25T10:13:23.121716+0000","last_peered":"2024-04-25T10:13:23.121716+0000","last_clean":"2024-04-25T10:13:23.121716+0000","last_became_active":"2024-04-25T10:13:21.135819+0000","last_became_peered":"2024-04-25T10:13:21.135819+0000","last_unstale":"2024-04-25T10:13:23.121716+0000","last_undegraded":"2024-04-25T10:13:23.121716+0000","last_fullsized":"2024-04-25T10:13:23.121716+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832495+0000","last_change":"2024-04-25T10:13:22.157562+0000","last_active":"2024-04-25T10:13:24.832495+0000","last_peered":"2024-04-25T10:13:24.832495+0000","last_clean":"2024-04-25T10:13:24.832495+0000","last_became_active":"2024-04-25T10:13:22.157317+0000","last_became_peered":"2024-04-25T10:13:22.157317+0000","last_unstale":"2024-04-25T10:13:24.832495+0000","last_undegraded":"2024-04-25T10:13:24.832495+0000","last_fullsized":"2024-04-25T10:13:24.832495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:38.509711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120222+0000","last_change":"2024-04-25T10:13:21.136923+0000","last_active":"2024-04-25T10:13:23.120222+0000","last_peered":"2024-04-25T10:13:23.120222+0000","last_clean":"2024-04-25T10:13:23.120222+0000","last_became_active":"2024-04-25T10:13:21.136615+0000","last_became_peered":"2024-04-25T10:13:21.136615+0000","last_unstale":"2024-04-25T10:13:23.120222+0000","last_undegraded":"2024-04-25T10:13:23.120222+0000","last_fullsized":"2024-04-25T10:13:23.120222+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:25.701632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119357+0000","last_change":"2024-04-25T10:13:22.163933+0000","last_active":"2024-04-25T10:13:23.119357+0000","last_peered":"2024-04-25T10:13:23.119357+0000","last_clean":"2024-04-25T10:13:23.119357+0000","last_became_active":"2024-04-25T10:13:22.163616+0000","last_became_peered":"2024-04-25T10:13:22.163616+0000","last_unstale":"2024-04-25T10:13:23.119357+0000","last_undegraded":"2024-04-25T10:13:23.119357+0000","last_fullsized":"2024-04-25T10:13:23.119357+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:30.574918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121907+0000","last_change":"2024-04-25T10:13:21.122791+0000","last_active":"2024-04-25T10:13:23.121907+0000","last_peered":"2024-04-25T10:13:23.121907+0000","last_clean":"2024-04-25T10:13:23.121907+0000","last_became_active":"2024-04-25T10:13:21.122081+0000","last_became_peered":"2024-04-25T10:13:21.122081+0000","last_unstale":"2024-04-25T10:13:23.121907+0000","last_undegraded":"2024-04-25T10:13:23.121907+0000","last_fullsized":"2024-04-25T10:13:23.121907+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:25.894975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830741+0000","last_change":"2024-04-25T10:13:22.154950+0000","last_active":"2024-04-25T10:13:24.830741+0000","last_peered":"2024-04-25T10:13:24.830741+0000","last_clean":"2024-04-25T10:13:24.830741+0000","last_became_active":"2024-04-25T10:13:22.154590+0000","last_became_peered":"2024-04-25T10:13:22.154590+0000","last_unstale":"2024-04-25T10:13:24.830741+0000","last_undegraded":"2024-04-25T10:13:24.830741+0000","last_fullsized":"2024-04-25T10:13:24.830741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121758+0000","last_change":"2024-04-25T10:13:21.142854+0000","last_active":"2024-04-25T10:13:23.121758+0000","last_peered":"2024-04-25T10:13:23.121758+0000","last_clean":"2024-04-25T10:13:23.121758+0000","last_became_active":"2024-04-25T10:13:21.141977+0000","last_became_peered":"2024-04-25T10:13:21.141977+0000","last_unstale":"2024-04-25T10:13:23.121758+0000","last_undegraded":"2024-04-25T10:13:23.121758+0000","last_fullsized":"2024-04-25T10:13:23.121758+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:39.638106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697793+0000","last_change":"2024-04-25T10:13:22.171868+0000","last_active":"2024-04-25T10:13:24.697793+0000","last_peered":"2024-04-25T10:13:24.697793+0000","last_clean":"2024-04-25T10:13:24.697793+0000","last_became_active":"2024-04-25T10:13:22.171135+0000","last_became_peered":"2024-04-25T10:13:22.171135+0000","last_unstale":"2024-04-25T10:13:24.697793+0000","last_undegraded":"2024-04-25T10:13:24.697793+0000","last_fullsized":"2024-04-25T10:13:24.697793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:05.796548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119288+0000","last_change":"2024-04-25T10:13:21.135774+0000","last_active":"2024-04-25T10:13:23.119288+0000","last_peered":"2024-04-25T10:13:23.119288+0000","last_clean":"2024-04-25T10:13:23.119288+0000","last_became_active":"2024-04-25T10:13:21.135457+0000","last_became_peered":"2024-04-25T10:13:21.135457+0000","last_unstale":"2024-04-25T10:13:23.119288+0000","last_undegraded":"2024-04-25T10:13:23.119288+0000","last_fullsized":"2024-04-25T10:13:23.119288+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:44.136286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469550+0000","last_change":"2024-04-25T10:13:22.160706+0000","last_active":"2024-04-25T10:13:24.469550+0000","last_peered":"2024-04-25T10:13:24.469550+0000","last_clean":"2024-04-25T10:13:24.469550+0000","last_became_active":"2024-04-25T10:13:22.160403+0000","last_became_peered":"2024-04-25T10:13:22.160403+0000","last_unstale":"2024-04-25T10:13:24.469550+0000","last_undegraded":"2024-04-25T10:13:24.469550+0000","last_fullsized":"2024-04-25T10:13:24.469550+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:21.093042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697409+0000","last_change":"2024-04-25T10:13:21.158159+0000","last_active":"2024-04-25T10:13:24.697409+0000","last_peered":"2024-04-25T10:13:24.697409+0000","last_clean":"2024-04-25T10:13:24.697409+0000","last_became_active":"2024-04-25T10:13:21.157463+0000","last_became_peered":"2024-04-25T10:13:21.157463+0000","last_unstale":"2024-04-25T10:13:24.697409+0000","last_undegraded":"2024-04-25T10:13:24.697409+0000","last_fullsized":"2024-04-25T10:13:24.697409+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698391+0000","last_change":"2024-04-25T10:13:22.182871+0000","last_active":"2024-04-25T10:13:24.698391+0000","last_peered":"2024-04-25T10:13:24.698391+0000","last_clean":"2024-04-25T10:13:24.698391+0000","last_became_active":"2024-04-25T10:13:22.181637+0000","last_became_peered":"2024-04-25T10:13:22.181637+0000","last_unstale":"2024-04-25T10:13:24.698391+0000","last_undegraded":"2024-04-25T10:13:24.698391+0000","last_fullsized":"2024-04-25T10:13:24.698391+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830482+0000","last_change":"2024-04-25T10:13:21.155420+0000","last_active":"2024-04-25T10:13:24.830482+0000","last_peered":"2024-04-25T10:13:24.830482+0000","last_clean":"2024-04-25T10:13:24.830482+0000","last_became_active":"2024-04-25T10:13:21.154940+0000","last_became_peered":"2024-04-25T10:13:21.154940+0000","last_unstale":"2024-04-25T10:13:24.830482+0000","last_undegraded":"2024-04-25T10:13:24.830482+0000","last_fullsized":"2024-04-25T10:13:24.830482+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":361,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697618+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:13:24.697618+0000","last_peered":"2024-04-25T10:13:24.697618+0000","last_clean":"2024-04-25T10:13:24.697618+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:13:24.697618+0000","last_undegraded":"2024-04-25T10:13:24.697618+0000","last_fullsized":"2024-04-25T10:13:24.697618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1289847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":705977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":698863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":707007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":700349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1495040,"data_stored":698703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1499136,"data_stored":698725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2105344,"data_stored":1295785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:26.139 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:26.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.291+0000 7eff403f3640 1 Processor -- start 2024-04-25T10:13:26.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.291+0000 7eff403f3640 1 -- start start 2024-04-25T10:13:26.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff403f3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff3815a700 0x7eff3815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff403f3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7eff380b24a0 0x7eff380b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.294 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff403f3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff380b2090 con 0x7eff380b3f50 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff403f3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3815f200 con 0x7eff3815a700 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff403f3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff380b43e0 con 0x7eff380b24a0 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff3815a700 0x7eff3815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7eff3815a700 0x7eff3815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43714/0 (socket says 172.21.15.82:43714) 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 -- 172.21.15.82:0/4042963449 learned_addr learned my addr 172.21.15.82:0/4042963449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3d98e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7eff380b3f50 0x7eff380cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36822/0 (socket says 172.21.15.82:36822) 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 -- 172.21.15.82:0/4042963449 >> v2:172.21.15.196:3300/0 conn(0x7eff380b24a0 msgr2=0x7eff380b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 --2- 172.21.15.82:0/4042963449 >> v2:172.21.15.196:3300/0 conn(0x7eff380b24a0 0x7eff380b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 -- 172.21.15.82:0/4042963449 >> v1:172.21.15.82:6789/0 conn(0x7eff380b3f50 legacy=0x7eff380cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 -- 172.21.15.82:0/4042963449 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff380b46c0 con 0x7eff3815a700 2024-04-25T10:13:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3e18f640 1 --2- 172.21.15.82:0/4042963449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3815a700 0x7eff3815ecc0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7eff3806c980 tx=0x7eff3401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fdefeabb2ea924 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3d18d640 1 -- 172.21.15.82:0/4042963449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340b0070 con 0x7eff3815a700 2024-04-25T10:13:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3d18d640 1 -- 172.21.15.82:0/4042963449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff340ab600 con 0x7eff3815a700 2024-04-25T10:13:26.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.295+0000 7eff3d18d640 1 -- 172.21.15.82:0/4042963449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340ab9d0 con 0x7eff3815a700 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4042963449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3815a700 msgr2=0x7eff3815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- 172.21.15.82:0/4042963449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3815a700 0x7eff3815ecc0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7eff3806c980 tx=0x7eff3401c400 comp rx=0 tx=0).stop 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4042963449 shutdown_connections 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- 172.21.15.82:0/4042963449 >> v2:172.21.15.196:3300/0 conn(0x7eff380b24a0 0x7eff380b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- 172.21.15.82:0/4042963449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff3815a700 0x7eff3815ecc0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4042963449 >> 172.21.15.82:0/4042963449 conn(0x7eff380b11a0 msgr2=0x7eff380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4042963449 shutdown_connections 2024-04-25T10:13:26.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4042963449 wait complete. 2024-04-25T10:13:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 Processor -- start 2024-04-25T10:13:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- start start 2024-04-25T10:13:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff380b24a0 0x7eff3806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3815a700 0x7eff3806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380cd310 con 0x7eff381511f0 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff380ce600 con 0x7eff380b24a0 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff380cd490 con 0x7eff3815a700 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43722/0 (socket says 172.21.15.82:43722) 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 -- 172.21.15.82:0/4232165983 learned_addr learned my addr 172.21.15.82:0/4232165983 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e18f640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff380b24a0 0x7eff3806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 -- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3815a700 msgr2=0x7eff3806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3815a700 0x7eff3806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 -- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff380b24a0 msgr2=0x7eff3806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff380b24a0 0x7eff3806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 -- 172.21.15.82:0/4232165983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff38163b90 con 0x7eff381511f0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff3e990640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7eff3000b970 tx=0x7eff3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30019070 con 0x7eff381511f0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3000cb60 con 0x7eff381511f0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff38163e70 con 0x7eff381511f0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30014440 con 0x7eff381511f0 2024-04-25T10:13:26.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff38164430 con 0x7eff381511f0 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.299+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3804efa0 con 0x7eff381511f0 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.303+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff3001c070 con 0x7eff381511f0 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.303+0000 7eff2effd640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.303+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7eff30084e90 con 0x7eff381511f0 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.303+0000 7eff3e18f640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.303+0000 7eff3e18f640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff340ad000 tx=0x7eff34008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.306 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.307+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff30055da0 con 0x7eff381511f0 2024-04-25T10:13:26.460 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.459+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff380ae750 con 0x7eff1c066f60 2024-04-25T10:13:26.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.471+0000 7eff2effd640 1 -- 172.21.15.82:0/4232165983 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252495 (secure 0 0 0) 0x7eff380ae750 con 0x7eff1c066f60 2024-04-25T10:13:26.475 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:26.476 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:13:26.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 msgr2=0x7eff1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 0x7eff1c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff340ad000 tx=0x7eff34008040 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 msgr2=0x7eff381560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7eff3000b970 tx=0x7eff3000be30 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 shutdown_connections 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7eff1c066f60 0x7eff1c069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7eff3815a700 0x7eff3806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7eff381511f0 0x7eff381560a0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 --2- 172.21.15.82:0/4232165983 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7eff380b24a0 0x7eff3806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 >> 172.21.15.82:0/4232165983 conn(0x7eff380b11a0 msgr2=0x7eff380ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 shutdown_connections 2024-04-25T10:13:26.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.475+0000 7eff403f3640 1 -- 172.21.15.82:0/4232165983 wait complete. 2024-04-25T10:13:26.493 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-04-25T10:13:25.479358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262412,"kb_used_data":12940,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712308,"statfs":{"total":773094113280,"available":772825403392,"internally_reserved":0,"allocated":13250560,"data_stored":6795256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.372712"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122655+0000","last_change":"2024-04-25T10:13:22.140920+0000","last_active":"2024-04-25T10:13:23.122655+0000","last_peered":"2024-04-25T10:13:23.122655+0000","last_clean":"2024-04-25T10:13:23.122655+0000","last_became_active":"2024-04-25T10:13:22.140198+0000","last_became_peered":"2024-04-25T10:13:22.140198+0000","last_unstale":"2024-04-25T10:13:23.122655+0000","last_undegraded":"2024-04-25T10:13:23.122655+0000","last_fullsized":"2024-04-25T10:13:23.122655+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:54.475106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831056+0000","last_change":"2024-04-25T10:13:22.170359+0000","last_active":"2024-04-25T10:13:24.831056+0000","last_peered":"2024-04-25T10:13:24.831056+0000","last_clean":"2024-04-25T10:13:24.831056+0000","last_became_active":"2024-04-25T10:13:22.170005+0000","last_became_peered":"2024-04-25T10:13:22.170005+0000","last_unstale":"2024-04-25T10:13:24.831056+0000","last_undegraded":"2024-04-25T10:13:24.831056+0000","last_fullsized":"2024-04-25T10:13:24.831056+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:30.954353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831557+0000","last_change":"2024-04-25T10:13:22.168641+0000","last_active":"2024-04-25T10:13:24.831557+0000","last_peered":"2024-04-25T10:13:24.831557+0000","last_clean":"2024-04-25T10:13:24.831557+0000","last_became_active":"2024-04-25T10:13:22.168416+0000","last_became_peered":"2024-04-25T10:13:22.168416+0000","last_unstale":"2024-04-25T10:13:24.831557+0000","last_undegraded":"2024-04-25T10:13:24.831557+0000","last_fullsized":"2024-04-25T10:13:24.831557+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:28.671762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698420+0000","last_change":"2024-04-25T10:13:22.178046+0000","last_active":"2024-04-25T10:13:24.698420+0000","last_peered":"2024-04-25T10:13:24.698420+0000","last_clean":"2024-04-25T10:13:24.698420+0000","last_became_active":"2024-04-25T10:13:22.174113+0000","last_became_peered":"2024-04-25T10:13:22.174113+0000","last_unstale":"2024-04-25T10:13:24.698420+0000","last_undegraded":"2024-04-25T10:13:24.698420+0000","last_fullsized":"2024-04-25T10:13:24.698420+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121280+0000","last_change":"2024-04-25T10:13:22.164811+0000","last_active":"2024-04-25T10:13:23.121280+0000","last_peered":"2024-04-25T10:13:23.121280+0000","last_clean":"2024-04-25T10:13:23.121280+0000","last_became_active":"2024-04-25T10:13:22.164191+0000","last_became_peered":"2024-04-25T10:13:22.164191+0000","last_unstale":"2024-04-25T10:13:23.121280+0000","last_undegraded":"2024-04-25T10:13:23.121280+0000","last_fullsized":"2024-04-25T10:13:23.121280+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:10.076284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119281+0000","last_change":"2024-04-25T10:13:22.141772+0000","last_active":"2024-04-25T10:13:23.119281+0000","last_peered":"2024-04-25T10:13:23.119281+0000","last_clean":"2024-04-25T10:13:23.119281+0000","last_became_active":"2024-04-25T10:13:22.141514+0000","last_became_peered":"2024-04-25T10:13:22.141514+0000","last_unstale":"2024-04-25T10:13:23.119281+0000","last_undegraded":"2024-04-25T10:13:23.119281+0000","last_fullsized":"2024-04-25T10:13:23.119281+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:04.935575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119695+0000","last_change":"2024-04-25T10:13:22.175230+0000","last_active":"2024-04-25T10:13:23.119695+0000","last_peered":"2024-04-25T10:13:23.119695+0000","last_clean":"2024-04-25T10:13:23.119695+0000","last_became_active":"2024-04-25T10:13:22.174928+0000","last_became_peered":"2024-04-25T10:13:22.174928+0000","last_unstale":"2024-04-25T10:13:23.119695+0000","last_undegraded":"2024-04-25T10:13:23.119695+0000","last_fullsized":"2024-04-25T10:13:23.119695+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:03.284671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830407+0000","last_change":"2024-04-25T10:13:22.150837+0000","last_active":"2024-04-25T10:13:24.830407+0000","last_peered":"2024-04-25T10:13:24.830407+0000","last_clean":"2024-04-25T10:13:24.830407+0000","last_became_active":"2024-04-25T10:13:22.150515+0000","last_became_peered":"2024-04-25T10:13:22.150515+0000","last_unstale":"2024-04-25T10:13:24.830407+0000","last_undegraded":"2024-04-25T10:13:24.830407+0000","last_fullsized":"2024-04-25T10:13:24.830407+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698819+0000","last_change":"2024-04-25T10:13:22.170815+0000","last_active":"2024-04-25T10:13:24.698819+0000","last_peered":"2024-04-25T10:13:24.698819+0000","last_clean":"2024-04-25T10:13:24.698819+0000","last_became_active":"2024-04-25T10:13:22.170175+0000","last_became_peered":"2024-04-25T10:13:22.170175+0000","last_unstale":"2024-04-25T10:13:24.698819+0000","last_undegraded":"2024-04-25T10:13:24.698819+0000","last_fullsized":"2024-04-25T10:13:24.698819+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469594+0000","last_change":"2024-04-25T10:13:22.134687+0000","last_active":"2024-04-25T10:13:24.469594+0000","last_peered":"2024-04-25T10:13:24.469594+0000","last_clean":"2024-04-25T10:13:24.469594+0000","last_became_active":"2024-04-25T10:13:22.134484+0000","last_became_peered":"2024-04-25T10:13:22.134484+0000","last_unstale":"2024-04-25T10:13:24.469594+0000","last_undegraded":"2024-04-25T10:13:24.469594+0000","last_fullsized":"2024-04-25T10:13:24.469594+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:32.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122130+0000","last_change":"2024-04-25T10:13:22.161945+0000","last_active":"2024-04-25T10:13:23.122130+0000","last_peered":"2024-04-25T10:13:23.122130+0000","last_clean":"2024-04-25T10:13:23.122130+0000","last_became_active":"2024-04-25T10:13:22.161131+0000","last_became_peered":"2024-04-25T10:13:22.161131+0000","last_unstale":"2024-04-25T10:13:23.122130+0000","last_undegraded":"2024-04-25T10:13:23.122130+0000","last_fullsized":"2024-04-25T10:13:23.122130+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.387388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120039+0000","last_change":"2024-04-25T10:13:22.164368+0000","last_active":"2024-04-25T10:13:23.120039+0000","last_peered":"2024-04-25T10:13:23.120039+0000","last_clean":"2024-04-25T10:13:23.120039+0000","last_became_active":"2024-04-25T10:13:22.164133+0000","last_became_peered":"2024-04-25T10:13:22.164133+0000","last_unstale":"2024-04-25T10:13:23.120039+0000","last_undegraded":"2024-04-25T10:13:23.120039+0000","last_fullsized":"2024-04-25T10:13:23.120039+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:06.448170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469242+0000","last_change":"2024-04-25T10:13:22.160394+0000","last_active":"2024-04-25T10:13:24.469242+0000","last_peered":"2024-04-25T10:13:24.469242+0000","last_clean":"2024-04-25T10:13:24.469242+0000","last_became_active":"2024-04-25T10:13:22.160035+0000","last_became_peered":"2024-04-25T10:13:22.160035+0000","last_unstale":"2024-04-25T10:13:24.469242+0000","last_undegraded":"2024-04-25T10:13:24.469242+0000","last_fullsized":"2024-04-25T10:13:24.469242+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.105353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831383+0000","last_change":"2024-04-25T10:13:22.176353+0000","last_active":"2024-04-25T10:13:24.831383+0000","last_peered":"2024-04-25T10:13:24.831383+0000","last_clean":"2024-04-25T10:13:24.831383+0000","last_became_active":"2024-04-25T10:13:22.176093+0000","last_became_peered":"2024-04-25T10:13:22.176093+0000","last_unstale":"2024-04-25T10:13:24.831383+0000","last_undegraded":"2024-04-25T10:13:24.831383+0000","last_fullsized":"2024-04-25T10:13:24.831383+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:27.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120781+0000","last_change":"2024-04-25T10:13:22.161972+0000","last_active":"2024-04-25T10:13:23.120781+0000","last_peered":"2024-04-25T10:13:23.120781+0000","last_clean":"2024-04-25T10:13:23.120781+0000","last_became_active":"2024-04-25T10:13:22.160381+0000","last_became_peered":"2024-04-25T10:13:22.160381+0000","last_unstale":"2024-04-25T10:13:23.120781+0000","last_undegraded":"2024-04-25T10:13:23.120781+0000","last_fullsized":"2024-04-25T10:13:23.120781+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:43.716686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119148+0000","last_change":"2024-04-25T10:13:22.138350+0000","last_active":"2024-04-25T10:13:23.119148+0000","last_peered":"2024-04-25T10:13:23.119148+0000","last_clean":"2024-04-25T10:13:23.119148+0000","last_became_active":"2024-04-25T10:13:22.138032+0000","last_became_peered":"2024-04-25T10:13:22.138032+0000","last_unstale":"2024-04-25T10:13:23.119148+0000","last_undegraded":"2024-04-25T10:13:23.119148+0000","last_fullsized":"2024-04-25T10:13:23.119148+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:45.883913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469797+0000","last_change":"2024-04-25T10:13:22.155563+0000","last_active":"2024-04-25T10:13:24.469797+0000","last_peered":"2024-04-25T10:13:24.469797+0000","last_clean":"2024-04-25T10:13:24.469797+0000","last_became_active":"2024-04-25T10:13:22.155239+0000","last_became_peered":"2024-04-25T10:13:22.155239+0000","last_unstale":"2024-04-25T10:13:24.469797+0000","last_undegraded":"2024-04-25T10:13:24.469797+0000","last_fullsized":"2024-04-25T10:13:24.469797+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831458+0000","last_change":"2024-04-25T10:13:22.156706+0000","last_active":"2024-04-25T10:13:24.831458+0000","last_peered":"2024-04-25T10:13:24.831458+0000","last_clean":"2024-04-25T10:13:24.831458+0000","last_became_active":"2024-04-25T10:13:22.156432+0000","last_became_peered":"2024-04-25T10:13:22.156432+0000","last_unstale":"2024-04-25T10:13:24.831458+0000","last_undegraded":"2024-04-25T10:13:24.831458+0000","last_fullsized":"2024-04-25T10:13:24.831458+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:36.752369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121428+0000","last_change":"2024-04-25T10:13:22.178118+0000","last_active":"2024-04-25T10:13:23.121428+0000","last_peered":"2024-04-25T10:13:23.121428+0000","last_clean":"2024-04-25T10:13:23.121428+0000","last_became_active":"2024-04-25T10:13:22.177680+0000","last_became_peered":"2024-04-25T10:13:22.177680+0000","last_unstale":"2024-04-25T10:13:23.121428+0000","last_undegraded":"2024-04-25T10:13:23.121428+0000","last_fullsized":"2024-04-25T10:13:23.121428+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832113+0000","last_change":"2024-04-25T10:13:22.138209+0000","last_active":"2024-04-25T10:13:24.832113+0000","last_peered":"2024-04-25T10:13:24.832113+0000","last_clean":"2024-04-25T10:13:24.832113+0000","last_became_active":"2024-04-25T10:13:22.137892+0000","last_became_peered":"2024-04-25T10:13:22.137892+0000","last_unstale":"2024-04-25T10:13:24.832113+0000","last_undegraded":"2024-04-25T10:13:24.832113+0000","last_fullsized":"2024-04-25T10:13:24.832113+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:08.214176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698263+0000","last_change":"2024-04-25T10:13:22.172056+0000","last_active":"2024-04-25T10:13:24.698263+0000","last_peered":"2024-04-25T10:13:24.698263+0000","last_clean":"2024-04-25T10:13:24.698263+0000","last_became_active":"2024-04-25T10:13:22.171405+0000","last_became_peered":"2024-04-25T10:13:22.171405+0000","last_unstale":"2024-04-25T10:13:24.698263+0000","last_undegraded":"2024-04-25T10:13:24.698263+0000","last_fullsized":"2024-04-25T10:13:24.698263+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121512+0000","last_change":"2024-04-25T10:13:22.145457+0000","last_active":"2024-04-25T10:13:23.121512+0000","last_peered":"2024-04-25T10:13:23.121512+0000","last_clean":"2024-04-25T10:13:23.121512+0000","last_became_active":"2024-04-25T10:13:22.144567+0000","last_became_peered":"2024-04-25T10:13:22.144567+0000","last_unstale":"2024-04-25T10:13:23.121512+0000","last_undegraded":"2024-04-25T10:13:23.121512+0000","last_fullsized":"2024-04-25T10:13:23.121512+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:09.105353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698009+0000","last_change":"2024-04-25T10:13:22.174646+0000","last_active":"2024-04-25T10:13:24.698009+0000","last_peered":"2024-04-25T10:13:24.698009+0000","last_clean":"2024-04-25T10:13:24.698009+0000","last_became_active":"2024-04-25T10:13:22.173972+0000","last_became_peered":"2024-04-25T10:13:22.173972+0000","last_unstale":"2024-04-25T10:13:24.698009+0000","last_undegraded":"2024-04-25T10:13:24.698009+0000","last_fullsized":"2024-04-25T10:13:24.698009+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:59.496070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829777+0000","last_change":"2024-04-25T10:13:22.179273+0000","last_active":"2024-04-25T10:13:24.829777+0000","last_peered":"2024-04-25T10:13:24.829777+0000","last_clean":"2024-04-25T10:13:24.829777+0000","last_became_active":"2024-04-25T10:13:22.177205+0000","last_became_peered":"2024-04-25T10:13:22.177205+0000","last_unstale":"2024-04-25T10:13:24.829777+0000","last_undegraded":"2024-04-25T10:13:24.829777+0000","last_fullsized":"2024-04-25T10:13:24.829777+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:42.499470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830628+0000","last_change":"2024-04-25T10:13:22.165684+0000","last_active":"2024-04-25T10:13:24.830628+0000","last_peered":"2024-04-25T10:13:24.830628+0000","last_clean":"2024-04-25T10:13:24.830628+0000","last_became_active":"2024-04-25T10:13:22.164972+0000","last_became_peered":"2024-04-25T10:13:22.164972+0000","last_unstale":"2024-04-25T10:13:24.830628+0000","last_undegraded":"2024-04-25T10:13:24.830628+0000","last_fullsized":"2024-04-25T10:13:24.830628+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:59.496070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831640+0000","last_change":"2024-04-25T10:13:22.130217+0000","last_active":"2024-04-25T10:13:24.831640+0000","last_peered":"2024-04-25T10:13:24.831640+0000","last_clean":"2024-04-25T10:13:24.831640+0000","last_became_active":"2024-04-25T10:13:22.129934+0000","last_became_peered":"2024-04-25T10:13:22.129934+0000","last_unstale":"2024-04-25T10:13:24.831640+0000","last_undegraded":"2024-04-25T10:13:24.831640+0000","last_fullsized":"2024-04-25T10:13:24.831640+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:50:07.872588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121854+0000","last_change":"2024-04-25T10:13:22.171590+0000","last_active":"2024-04-25T10:13:23.121854+0000","last_peered":"2024-04-25T10:13:23.121854+0000","last_clean":"2024-04-25T10:13:23.121854+0000","last_became_active":"2024-04-25T10:13:22.170946+0000","last_became_peered":"2024-04-25T10:13:22.170946+0000","last_unstale":"2024-04-25T10:13:23.121854+0000","last_undegraded":"2024-04-25T10:13:23.121854+0000","last_fullsized":"2024-04-25T10:13:23.121854+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:42.499470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119944+0000","last_change":"2024-04-25T10:13:22.153736+0000","last_active":"2024-04-25T10:13:23.119944+0000","last_peered":"2024-04-25T10:13:23.119944+0000","last_clean":"2024-04-25T10:13:23.119944+0000","last_became_active":"2024-04-25T10:13:22.153476+0000","last_became_peered":"2024-04-25T10:13:22.153476+0000","last_unstale":"2024-04-25T10:13:23.119944+0000","last_undegraded":"2024-04-25T10:13:23.119944+0000","last_fullsized":"2024-04-25T10:13:23.119944+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:10.382084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832422+0000","last_change":"2024-04-25T10:13:22.133755+0000","last_active":"2024-04-25T10:13:24.832422+0000","last_peered":"2024-04-25T10:13:24.832422+0000","last_clean":"2024-04-25T10:13:24.832422+0000","last_became_active":"2024-04-25T10:13:22.133433+0000","last_became_peered":"2024-04-25T10:13:22.133433+0000","last_unstale":"2024-04-25T10:13:24.832422+0000","last_undegraded":"2024-04-25T10:13:24.832422+0000","last_fullsized":"2024-04-25T10:13:24.832422+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:22.231820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831429+0000","last_change":"2024-04-25T10:13:22.163180+0000","last_active":"2024-04-25T10:13:24.831429+0000","last_peered":"2024-04-25T10:13:24.831429+0000","last_clean":"2024-04-25T10:13:24.831429+0000","last_became_active":"2024-04-25T10:13:22.162882+0000","last_became_peered":"2024-04-25T10:13:22.162882+0000","last_unstale":"2024-04-25T10:13:24.831429+0000","last_undegraded":"2024-04-25T10:13:24.831429+0000","last_fullsized":"2024-04-25T10:13:24.831429+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:08.933078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829947+0000","last_change":"2024-04-25T10:13:22.151875+0000","last_active":"2024-04-25T10:13:24.829947+0000","last_peered":"2024-04-25T10:13:24.829947+0000","last_clean":"2024-04-25T10:13:24.829947+0000","last_became_active":"2024-04-25T10:13:22.151124+0000","last_became_peered":"2024-04-25T10:13:22.151124+0000","last_unstale":"2024-04-25T10:13:24.829947+0000","last_undegraded":"2024-04-25T10:13:24.829947+0000","last_fullsized":"2024-04-25T10:13:24.829947+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:50.426633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829756+0000","last_change":"2024-04-25T10:13:22.178679+0000","last_active":"2024-04-25T10:13:24.829756+0000","last_peered":"2024-04-25T10:13:24.829756+0000","last_clean":"2024-04-25T10:13:24.829756+0000","last_became_active":"2024-04-25T10:13:22.176484+0000","last_became_peered":"2024-04-25T10:13:22.176484+0000","last_unstale":"2024-04-25T10:13:24.829756+0000","last_undegraded":"2024-04-25T10:13:24.829756+0000","last_fullsized":"2024-04-25T10:13:24.829756+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:51.447660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469190+0000","last_change":"2024-04-25T10:13:22.134103+0000","last_active":"2024-04-25T10:13:24.469190+0000","last_peered":"2024-04-25T10:13:24.469190+0000","last_clean":"2024-04-25T10:13:24.469190+0000","last_became_active":"2024-04-25T10:13:22.133899+0000","last_became_peered":"2024-04-25T10:13:22.133899+0000","last_unstale":"2024-04-25T10:13:24.469190+0000","last_undegraded":"2024-04-25T10:13:24.469190+0000","last_fullsized":"2024-04-25T10:13:24.469190+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:10.076284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"120'4","reported_seq":18,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121263+0000","last_change":"2024-04-25T10:13:21.157367+0000","last_active":"2024-04-25T10:13:23.121263+0000","last_peered":"2024-04-25T10:13:23.121263+0000","last_clean":"2024-04-25T10:13:23.121263+0000","last_became_active":"2024-04-25T10:13:21.156537+0000","last_became_peered":"2024-04-25T10:13:21.156537+0000","last_unstale":"2024-04-25T10:13:23.121263+0000","last_undegraded":"2024-04-25T10:13:23.121263+0000","last_fullsized":"2024-04-25T10:13:23.121263+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119708+0000","last_change":"2024-04-25T10:13:22.140984+0000","last_active":"2024-04-25T10:13:23.119708+0000","last_peered":"2024-04-25T10:13:23.119708+0000","last_clean":"2024-04-25T10:13:23.119708+0000","last_became_active":"2024-04-25T10:13:22.140710+0000","last_became_peered":"2024-04-25T10:13:22.140710+0000","last_unstale":"2024-04-25T10:13:23.119708+0000","last_undegraded":"2024-04-25T10:13:23.119708+0000","last_fullsized":"2024-04-25T10:13:23.119708+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:56.599361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469587+0000","last_change":"2024-04-25T10:13:21.133173+0000","last_active":"2024-04-25T10:13:24.469587+0000","last_peered":"2024-04-25T10:13:24.469587+0000","last_clean":"2024-04-25T10:13:24.469587+0000","last_became_active":"2024-04-25T10:13:21.132811+0000","last_became_peered":"2024-04-25T10:13:21.132811+0000","last_unstale":"2024-04-25T10:13:24.469587+0000","last_undegraded":"2024-04-25T10:13:24.469587+0000","last_fullsized":"2024-04-25T10:13:24.469587+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:48.803866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830297+0000","last_change":"2024-04-25T10:13:22.155705+0000","last_active":"2024-04-25T10:13:24.830297+0000","last_peered":"2024-04-25T10:13:24.830297+0000","last_clean":"2024-04-25T10:13:24.830297+0000","last_became_active":"2024-04-25T10:13:22.154950+0000","last_became_peered":"2024-04-25T10:13:22.154950+0000","last_unstale":"2024-04-25T10:13:24.830297+0000","last_undegraded":"2024-04-25T10:13:24.830297+0000","last_fullsized":"2024-04-25T10:13:24.830297+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:55.710602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698616+0000","last_change":"2024-04-25T10:13:21.157108+0000","last_active":"2024-04-25T10:13:24.698616+0000","last_peered":"2024-04-25T10:13:24.698616+0000","last_clean":"2024-04-25T10:13:24.698616+0000","last_became_active":"2024-04-25T10:13:21.156170+0000","last_became_peered":"2024-04-25T10:13:21.156170+0000","last_unstale":"2024-04-25T10:13:24.698616+0000","last_undegraded":"2024-04-25T10:13:24.698616+0000","last_fullsized":"2024-04-25T10:13:24.698616+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698988+0000","last_change":"2024-04-25T10:13:22.178710+0000","last_active":"2024-04-25T10:13:24.698988+0000","last_peered":"2024-04-25T10:13:24.698988+0000","last_clean":"2024-04-25T10:13:24.698988+0000","last_became_active":"2024-04-25T10:13:22.178102+0000","last_became_peered":"2024-04-25T10:13:22.178102+0000","last_unstale":"2024-04-25T10:13:24.698988+0000","last_undegraded":"2024-04-25T10:13:24.698988+0000","last_fullsized":"2024-04-25T10:13:24.698988+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:08.933078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469305+0000","last_change":"2024-04-25T10:13:21.135597+0000","last_active":"2024-04-25T10:13:24.469305+0000","last_peered":"2024-04-25T10:13:24.469305+0000","last_clean":"2024-04-25T10:13:24.469305+0000","last_became_active":"2024-04-25T10:13:21.135265+0000","last_became_peered":"2024-04-25T10:13:21.135265+0000","last_unstale":"2024-04-25T10:13:24.469305+0000","last_undegraded":"2024-04-25T10:13:24.469305+0000","last_fullsized":"2024-04-25T10:13:24.469305+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:39.638106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469485+0000","last_change":"2024-04-25T10:13:22.130931+0000","last_active":"2024-04-25T10:13:24.469485+0000","last_peered":"2024-04-25T10:13:24.469485+0000","last_clean":"2024-04-25T10:13:24.469485+0000","last_became_active":"2024-04-25T10:13:22.130660+0000","last_became_peered":"2024-04-25T10:13:22.130660+0000","last_unstale":"2024-04-25T10:13:24.469485+0000","last_undegraded":"2024-04-25T10:13:24.469485+0000","last_fullsized":"2024-04-25T10:13:24.469485+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.602657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121236+0000","last_change":"2024-04-25T10:13:21.134313+0000","last_active":"2024-04-25T10:13:23.121236+0000","last_peered":"2024-04-25T10:13:23.121236+0000","last_clean":"2024-04-25T10:13:23.121236+0000","last_became_active":"2024-04-25T10:13:21.133430+0000","last_became_peered":"2024-04-25T10:13:21.133430+0000","last_unstale":"2024-04-25T10:13:23.121236+0000","last_undegraded":"2024-04-25T10:13:23.121236+0000","last_fullsized":"2024-04-25T10:13:23.121236+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:00.827119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120723+0000","last_change":"2024-04-25T10:13:22.162338+0000","last_active":"2024-04-25T10:13:23.120723+0000","last_peered":"2024-04-25T10:13:23.120723+0000","last_clean":"2024-04-25T10:13:23.120723+0000","last_became_active":"2024-04-25T10:13:22.161285+0000","last_became_peered":"2024-04-25T10:13:22.161285+0000","last_unstale":"2024-04-25T10:13:23.120723+0000","last_undegraded":"2024-04-25T10:13:23.120723+0000","last_fullsized":"2024-04-25T10:13:23.120723+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:02.670910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119155+0000","last_change":"2024-04-25T10:13:21.141367+0000","last_active":"2024-04-25T10:13:23.119155+0000","last_peered":"2024-04-25T10:13:23.119155+0000","last_clean":"2024-04-25T10:13:23.119155+0000","last_became_active":"2024-04-25T10:13:21.141007+0000","last_became_peered":"2024-04-25T10:13:21.141007+0000","last_unstale":"2024-04-25T10:13:23.119155+0000","last_undegraded":"2024-04-25T10:13:23.119155+0000","last_fullsized":"2024-04-25T10:13:23.119155+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:08:39.133706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831818+0000","last_change":"2024-04-25T10:13:22.135859+0000","last_active":"2024-04-25T10:13:24.831818+0000","last_peered":"2024-04-25T10:13:24.831818+0000","last_clean":"2024-04-25T10:13:24.831818+0000","last_became_active":"2024-04-25T10:13:22.135500+0000","last_became_peered":"2024-04-25T10:13:22.135500+0000","last_unstale":"2024-04-25T10:13:24.831818+0000","last_undegraded":"2024-04-25T10:13:24.831818+0000","last_fullsized":"2024-04-25T10:13:24.831818+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:26.474691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.829946+0000","last_change":"2024-04-25T10:13:21.140271+0000","last_active":"2024-04-25T10:13:24.829946+0000","last_peered":"2024-04-25T10:13:24.829946+0000","last_clean":"2024-04-25T10:13:24.829946+0000","last_became_active":"2024-04-25T10:13:21.139166+0000","last_became_peered":"2024-04-25T10:13:21.139166+0000","last_unstale":"2024-04-25T10:13:24.829946+0000","last_undegraded":"2024-04-25T10:13:24.829946+0000","last_fullsized":"2024-04-25T10:13:24.829946+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469518+0000","last_change":"2024-04-25T10:13:22.134349+0000","last_active":"2024-04-25T10:13:24.469518+0000","last_peered":"2024-04-25T10:13:24.469518+0000","last_clean":"2024-04-25T10:13:24.469518+0000","last_became_active":"2024-04-25T10:13:22.133511+0000","last_became_peered":"2024-04-25T10:13:22.133511+0000","last_unstale":"2024-04-25T10:13:24.469518+0000","last_undegraded":"2024-04-25T10:13:24.469518+0000","last_fullsized":"2024-04-25T10:13:24.469518+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:54.475106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119407+0000","last_change":"2024-04-25T10:13:21.135976+0000","last_active":"2024-04-25T10:13:23.119407+0000","last_peered":"2024-04-25T10:13:23.119407+0000","last_clean":"2024-04-25T10:13:23.119407+0000","last_became_active":"2024-04-25T10:13:21.135448+0000","last_became_peered":"2024-04-25T10:13:21.135448+0000","last_unstale":"2024-04-25T10:13:23.119407+0000","last_undegraded":"2024-04-25T10:13:23.119407+0000","last_fullsized":"2024-04-25T10:13:23.119407+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:45.481869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120592+0000","last_change":"2024-04-25T10:13:22.172294+0000","last_active":"2024-04-25T10:13:23.120592+0000","last_peered":"2024-04-25T10:13:23.120592+0000","last_clean":"2024-04-25T10:13:23.120592+0000","last_became_active":"2024-04-25T10:13:22.170734+0000","last_became_peered":"2024-04-25T10:13:22.170734+0000","last_unstale":"2024-04-25T10:13:23.120592+0000","last_undegraded":"2024-04-25T10:13:23.120592+0000","last_fullsized":"2024-04-25T10:13:23.120592+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:32.432027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122161+0000","last_change":"2024-04-25T10:13:21.136753+0000","last_active":"2024-04-25T10:13:23.122161+0000","last_peered":"2024-04-25T10:13:23.122161+0000","last_clean":"2024-04-25T10:13:23.122161+0000","last_became_active":"2024-04-25T10:13:21.135942+0000","last_became_peered":"2024-04-25T10:13:21.135942+0000","last_unstale":"2024-04-25T10:13:23.122161+0000","last_undegraded":"2024-04-25T10:13:23.122161+0000","last_fullsized":"2024-04-25T10:13:23.122161+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830431+0000","last_change":"2024-04-25T10:13:22.151962+0000","last_active":"2024-04-25T10:13:24.830431+0000","last_peered":"2024-04-25T10:13:24.830431+0000","last_clean":"2024-04-25T10:13:24.830431+0000","last_became_active":"2024-04-25T10:13:22.151288+0000","last_became_peered":"2024-04-25T10:13:22.151288+0000","last_unstale":"2024-04-25T10:13:24.830431+0000","last_undegraded":"2024-04-25T10:13:24.830431+0000","last_fullsized":"2024-04-25T10:13:24.830431+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:05.796548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830324+0000","last_change":"2024-04-25T10:13:21.142538+0000","last_active":"2024-04-25T10:13:24.830324+0000","last_peered":"2024-04-25T10:13:24.830324+0000","last_clean":"2024-04-25T10:13:24.830324+0000","last_became_active":"2024-04-25T10:13:21.141723+0000","last_became_peered":"2024-04-25T10:13:21.141723+0000","last_unstale":"2024-04-25T10:13:24.830324+0000","last_undegraded":"2024-04-25T10:13:24.830324+0000","last_fullsized":"2024-04-25T10:13:24.830324+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:55.291423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832138+0000","last_change":"2024-04-25T10:13:22.154863+0000","last_active":"2024-04-25T10:13:24.832138+0000","last_peered":"2024-04-25T10:13:24.832138+0000","last_clean":"2024-04-25T10:13:24.832138+0000","last_became_active":"2024-04-25T10:13:22.154548+0000","last_became_peered":"2024-04-25T10:13:22.154548+0000","last_unstale":"2024-04-25T10:13:24.832138+0000","last_undegraded":"2024-04-25T10:13:24.832138+0000","last_fullsized":"2024-04-25T10:13:24.832138+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:48.684867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"120'3","reported_seq":17,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830465+0000","last_change":"2024-04-25T10:13:21.149283+0000","last_active":"2024-04-25T10:13:24.830465+0000","last_peered":"2024-04-25T10:13:24.830465+0000","last_clean":"2024-04-25T10:13:24.830465+0000","last_became_active":"2024-04-25T10:13:21.148399+0000","last_became_peered":"2024-04-25T10:13:21.148399+0000","last_unstale":"2024-04-25T10:13:24.830465+0000","last_undegraded":"2024-04-25T10:13:24.830465+0000","last_fullsized":"2024-04-25T10:13:24.830465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:38.767138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830725+0000","last_change":"2024-04-25T10:13:22.153448+0000","last_active":"2024-04-25T10:13:24.830725+0000","last_peered":"2024-04-25T10:13:24.830725+0000","last_clean":"2024-04-25T10:13:24.830725+0000","last_became_active":"2024-04-25T10:13:22.144919+0000","last_became_peered":"2024-04-25T10:13:22.144919+0000","last_unstale":"2024-04-25T10:13:24.830725+0000","last_undegraded":"2024-04-25T10:13:24.830725+0000","last_fullsized":"2024-04-25T10:13:24.830725+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"120'6","reported_seq":20,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832597+0000","last_change":"2024-04-25T10:13:21.136638+0000","last_active":"2024-04-25T10:13:24.832597+0000","last_peered":"2024-04-25T10:13:24.832597+0000","last_clean":"2024-04-25T10:13:24.832597+0000","last_became_active":"2024-04-25T10:13:21.136274+0000","last_became_peered":"2024-04-25T10:13:21.136274+0000","last_unstale":"2024-04-25T10:13:24.832597+0000","last_undegraded":"2024-04-25T10:13:24.832597+0000","last_fullsized":"2024-04-25T10:13:24.832597+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:08.096788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120911+0000","last_change":"2024-04-25T10:13:22.169587+0000","last_active":"2024-04-25T10:13:23.120911+0000","last_peered":"2024-04-25T10:13:23.120911+0000","last_clean":"2024-04-25T10:13:23.120911+0000","last_became_active":"2024-04-25T10:13:22.168905+0000","last_became_peered":"2024-04-25T10:13:22.168905+0000","last_unstale":"2024-04-25T10:13:23.120911+0000","last_undegraded":"2024-04-25T10:13:23.120911+0000","last_fullsized":"2024-04-25T10:13:23.120911+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:49.588817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831430+0000","last_change":"2024-04-25T10:13:21.156529+0000","last_active":"2024-04-25T10:13:24.831430+0000","last_peered":"2024-04-25T10:13:24.831430+0000","last_clean":"2024-04-25T10:13:24.831430+0000","last_became_active":"2024-04-25T10:13:21.156141+0000","last_became_peered":"2024-04-25T10:13:21.156141+0000","last_unstale":"2024-04-25T10:13:24.831430+0000","last_undegraded":"2024-04-25T10:13:24.831430+0000","last_fullsized":"2024-04-25T10:13:24.831430+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469670+0000","last_change":"2024-04-25T10:13:22.178090+0000","last_active":"2024-04-25T10:13:24.469670+0000","last_peered":"2024-04-25T10:13:24.469670+0000","last_clean":"2024-04-25T10:13:24.469670+0000","last_became_active":"2024-04-25T10:13:22.177870+0000","last_became_peered":"2024-04-25T10:13:22.177870+0000","last_unstale":"2024-04-25T10:13:24.469670+0000","last_undegraded":"2024-04-25T10:13:24.469670+0000","last_fullsized":"2024-04-25T10:13:24.469670+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:49.588817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832307+0000","last_change":"2024-04-25T10:13:21.150192+0000","last_active":"2024-04-25T10:13:24.832307+0000","last_peered":"2024-04-25T10:13:24.832307+0000","last_clean":"2024-04-25T10:13:24.832307+0000","last_became_active":"2024-04-25T10:13:21.149784+0000","last_became_peered":"2024-04-25T10:13:21.149784+0000","last_unstale":"2024-04-25T10:13:24.832307+0000","last_undegraded":"2024-04-25T10:13:24.832307+0000","last_fullsized":"2024-04-25T10:13:24.832307+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.378823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832613+0000","last_change":"2024-04-25T10:13:22.146064+0000","last_active":"2024-04-25T10:13:24.832613+0000","last_peered":"2024-04-25T10:13:24.832613+0000","last_clean":"2024-04-25T10:13:24.832613+0000","last_became_active":"2024-04-25T10:13:22.145779+0000","last_became_peered":"2024-04-25T10:13:22.145779+0000","last_unstale":"2024-04-25T10:13:24.832613+0000","last_undegraded":"2024-04-25T10:13:24.832613+0000","last_fullsized":"2024-04-25T10:13:24.832613+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:33.128817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830811+0000","last_change":"2024-04-25T10:13:21.136420+0000","last_active":"2024-04-25T10:13:24.830811+0000","last_peered":"2024-04-25T10:13:24.830811+0000","last_clean":"2024-04-25T10:13:24.830811+0000","last_became_active":"2024-04-25T10:13:21.136121+0000","last_became_peered":"2024-04-25T10:13:21.136121+0000","last_unstale":"2024-04-25T10:13:24.830811+0000","last_undegraded":"2024-04-25T10:13:24.830811+0000","last_fullsized":"2024-04-25T10:13:24.830811+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830645+0000","last_change":"2024-04-25T10:13:22.149933+0000","last_active":"2024-04-25T10:13:24.830645+0000","last_peered":"2024-04-25T10:13:24.830645+0000","last_clean":"2024-04-25T10:13:24.830645+0000","last_became_active":"2024-04-25T10:13:22.149205+0000","last_became_peered":"2024-04-25T10:13:22.149205+0000","last_unstale":"2024-04-25T10:13:24.830645+0000","last_undegraded":"2024-04-25T10:13:24.830645+0000","last_fullsized":"2024-04-25T10:13:24.830645+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:12.769329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.470033+0000","last_change":"2024-04-25T10:13:21.139798+0000","last_active":"2024-04-25T10:13:24.470033+0000","last_peered":"2024-04-25T10:13:24.470033+0000","last_clean":"2024-04-25T10:13:24.470033+0000","last_became_active":"2024-04-25T10:13:21.139442+0000","last_became_peered":"2024-04-25T10:13:21.139442+0000","last_unstale":"2024-04-25T10:13:24.470033+0000","last_undegraded":"2024-04-25T10:13:24.470033+0000","last_fullsized":"2024-04-25T10:13:24.470033+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:38.282002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.122480+0000","last_change":"2024-04-25T10:13:22.145387+0000","last_active":"2024-04-25T10:13:23.122480+0000","last_peered":"2024-04-25T10:13:23.122480+0000","last_clean":"2024-04-25T10:13:23.122480+0000","last_became_active":"2024-04-25T10:13:22.144610+0000","last_became_peered":"2024-04-25T10:13:22.144610+0000","last_unstale":"2024-04-25T10:13:23.122480+0000","last_undegraded":"2024-04-25T10:13:23.122480+0000","last_fullsized":"2024-04-25T10:13:23.122480+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:49.602657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.831986+0000","last_change":"2024-04-25T10:13:21.132512+0000","last_active":"2024-04-25T10:13:24.831986+0000","last_peered":"2024-04-25T10:13:24.831986+0000","last_clean":"2024-04-25T10:13:24.831986+0000","last_became_active":"2024-04-25T10:13:21.132057+0000","last_became_peered":"2024-04-25T10:13:21.132057+0000","last_unstale":"2024-04-25T10:13:24.831986+0000","last_undegraded":"2024-04-25T10:13:24.831986+0000","last_fullsized":"2024-04-25T10:13:24.831986+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:25.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119770+0000","last_change":"2024-04-25T10:13:22.155284+0000","last_active":"2024-04-25T10:13:23.119770+0000","last_peered":"2024-04-25T10:13:23.119770+0000","last_clean":"2024-04-25T10:13:23.119770+0000","last_became_active":"2024-04-25T10:13:22.154998+0000","last_became_peered":"2024-04-25T10:13:22.154998+0000","last_unstale":"2024-04-25T10:13:23.119770+0000","last_undegraded":"2024-04-25T10:13:23.119770+0000","last_fullsized":"2024-04-25T10:13:23.119770+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:17.402902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"120'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698772+0000","last_change":"2024-04-25T10:13:21.158040+0000","last_active":"2024-04-25T10:13:24.698772+0000","last_peered":"2024-04-25T10:13:24.698772+0000","last_clean":"2024-04-25T10:13:24.698772+0000","last_became_active":"2024-04-25T10:13:21.157331+0000","last_became_peered":"2024-04-25T10:13:21.157331+0000","last_unstale":"2024-04-25T10:13:24.698772+0000","last_undegraded":"2024-04-25T10:13:24.698772+0000","last_fullsized":"2024-04-25T10:13:24.698772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:31.378823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830838+0000","last_change":"2024-04-25T10:13:22.177488+0000","last_active":"2024-04-25T10:13:24.830838+0000","last_peered":"2024-04-25T10:13:24.830838+0000","last_clean":"2024-04-25T10:13:24.830838+0000","last_became_active":"2024-04-25T10:13:22.176824+0000","last_became_peered":"2024-04-25T10:13:22.176824+0000","last_unstale":"2024-04-25T10:13:24.830838+0000","last_undegraded":"2024-04-25T10:13:24.830838+0000","last_fullsized":"2024-04-25T10:13:24.830838+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698461+0000","last_change":"2024-04-25T10:13:21.157947+0000","last_active":"2024-04-25T10:13:24.698461+0000","last_peered":"2024-04-25T10:13:24.698461+0000","last_clean":"2024-04-25T10:13:24.698461+0000","last_became_active":"2024-04-25T10:13:21.157130+0000","last_became_peered":"2024-04-25T10:13:21.157130+0000","last_unstale":"2024-04-25T10:13:24.698461+0000","last_undegraded":"2024-04-25T10:13:24.698461+0000","last_fullsized":"2024-04-25T10:13:24.698461+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119135+0000","last_change":"2024-04-25T10:13:22.171686+0000","last_active":"2024-04-25T10:13:23.119135+0000","last_peered":"2024-04-25T10:13:23.119135+0000","last_clean":"2024-04-25T10:13:23.119135+0000","last_became_active":"2024-04-25T10:13:22.171245+0000","last_became_peered":"2024-04-25T10:13:22.171245+0000","last_unstale":"2024-04-25T10:13:23.119135+0000","last_undegraded":"2024-04-25T10:13:23.119135+0000","last_fullsized":"2024-04-25T10:13:23.119135+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:24.894164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119316+0000","last_change":"2024-04-25T10:13:21.155110+0000","last_active":"2024-04-25T10:13:23.119316+0000","last_peered":"2024-04-25T10:13:23.119316+0000","last_clean":"2024-04-25T10:13:23.119316+0000","last_became_active":"2024-04-25T10:13:21.154755+0000","last_became_peered":"2024-04-25T10:13:21.154755+0000","last_unstale":"2024-04-25T10:13:23.119316+0000","last_undegraded":"2024-04-25T10:13:23.119316+0000","last_fullsized":"2024-04-25T10:13:23.119316+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:06.196866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830536+0000","last_change":"2024-04-25T10:13:22.158950+0000","last_active":"2024-04-25T10:13:24.830536+0000","last_peered":"2024-04-25T10:13:24.830536+0000","last_clean":"2024-04-25T10:13:24.830536+0000","last_became_active":"2024-04-25T10:13:22.158728+0000","last_became_peered":"2024-04-25T10:13:22.158728+0000","last_unstale":"2024-04-25T10:13:24.830536+0000","last_undegraded":"2024-04-25T10:13:24.830536+0000","last_fullsized":"2024-04-25T10:13:24.830536+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697922+0000","last_change":"2024-04-25T10:13:21.156709+0000","last_active":"2024-04-25T10:13:24.697922+0000","last_peered":"2024-04-25T10:13:24.697922+0000","last_clean":"2024-04-25T10:13:24.697922+0000","last_became_active":"2024-04-25T10:13:21.155827+0000","last_became_peered":"2024-04-25T10:13:21.155827+0000","last_unstale":"2024-04-25T10:13:24.697922+0000","last_undegraded":"2024-04-25T10:13:24.697922+0000","last_fullsized":"2024-04-25T10:13:24.697922+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:01.662345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832381+0000","last_change":"2024-04-25T10:13:22.155227+0000","last_active":"2024-04-25T10:13:24.832381+0000","last_peered":"2024-04-25T10:13:24.832381+0000","last_clean":"2024-04-25T10:13:24.832381+0000","last_became_active":"2024-04-25T10:13:22.154516+0000","last_became_peered":"2024-04-25T10:13:22.154516+0000","last_unstale":"2024-04-25T10:13:24.832381+0000","last_undegraded":"2024-04-25T10:13:24.832381+0000","last_fullsized":"2024-04-25T10:13:24.832381+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:07.205431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830992+0000","last_change":"2024-04-25T10:13:21.132768+0000","last_active":"2024-04-25T10:13:24.830992+0000","last_peered":"2024-04-25T10:13:24.830992+0000","last_clean":"2024-04-25T10:13:24.830992+0000","last_became_active":"2024-04-25T10:13:21.132275+0000","last_became_peered":"2024-04-25T10:13:21.132275+0000","last_unstale":"2024-04-25T10:13:24.830992+0000","last_undegraded":"2024-04-25T10:13:24.830992+0000","last_fullsized":"2024-04-25T10:13:24.830992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830114+0000","last_change":"2024-04-25T10:13:22.151164+0000","last_active":"2024-04-25T10:13:24.830114+0000","last_peered":"2024-04-25T10:13:24.830114+0000","last_clean":"2024-04-25T10:13:24.830114+0000","last_became_active":"2024-04-25T10:13:22.149322+0000","last_became_peered":"2024-04-25T10:13:22.149322+0000","last_unstale":"2024-04-25T10:13:24.830114+0000","last_undegraded":"2024-04-25T10:13:24.830114+0000","last_fullsized":"2024-04-25T10:13:24.830114+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:17.897835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697439+0000","last_change":"2024-04-25T10:13:21.156461+0000","last_active":"2024-04-25T10:13:24.697439+0000","last_peered":"2024-04-25T10:13:24.697439+0000","last_clean":"2024-04-25T10:13:24.697439+0000","last_became_active":"2024-04-25T10:13:21.155691+0000","last_became_peered":"2024-04-25T10:13:21.155691+0000","last_unstale":"2024-04-25T10:13:24.697439+0000","last_undegraded":"2024-04-25T10:13:24.697439+0000","last_fullsized":"2024-04-25T10:13:24.697439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120153+0000","last_change":"2024-04-25T10:13:22.139584+0000","last_active":"2024-04-25T10:13:23.120153+0000","last_peered":"2024-04-25T10:13:23.120153+0000","last_clean":"2024-04-25T10:13:23.120153+0000","last_became_active":"2024-04-25T10:13:22.139013+0000","last_became_peered":"2024-04-25T10:13:22.139013+0000","last_unstale":"2024-04-25T10:13:23.120153+0000","last_undegraded":"2024-04-25T10:13:23.120153+0000","last_fullsized":"2024-04-25T10:13:23.120153+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:23.772014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"120'2","reported_seq":16,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120219+0000","last_change":"2024-04-25T10:13:21.135789+0000","last_active":"2024-04-25T10:13:23.120219+0000","last_peered":"2024-04-25T10:13:23.120219+0000","last_clean":"2024-04-25T10:13:23.120219+0000","last_became_active":"2024-04-25T10:13:21.135342+0000","last_became_peered":"2024-04-25T10:13:21.135342+0000","last_unstale":"2024-04-25T10:13:23.120219+0000","last_undegraded":"2024-04-25T10:13:23.120219+0000","last_fullsized":"2024-04-25T10:13:23.120219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:22.870747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469284+0000","last_change":"2024-04-25T10:13:22.131588+0000","last_active":"2024-04-25T10:13:24.469284+0000","last_peered":"2024-04-25T10:13:24.469284+0000","last_clean":"2024-04-25T10:13:24.469284+0000","last_became_active":"2024-04-25T10:13:22.130293+0000","last_became_peered":"2024-04-25T10:13:22.130293+0000","last_unstale":"2024-04-25T10:13:24.469284+0000","last_undegraded":"2024-04-25T10:13:24.469284+0000","last_fullsized":"2024-04-25T10:13:24.469284+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:32.387388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"121'1","reported_seq":15,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.124643+0000","last_change":"2024-04-25T10:13:21.136901+0000","last_active":"2024-04-25T10:13:23.124643+0000","last_peered":"2024-04-25T10:13:23.124643+0000","last_clean":"2024-04-25T10:13:23.124643+0000","last_became_active":"2024-04-25T10:13:21.135921+0000","last_became_peered":"2024-04-25T10:13:21.135921+0000","last_unstale":"2024-04-25T10:13:23.124643+0000","last_undegraded":"2024-04-25T10:13:23.124643+0000","last_fullsized":"2024-04-25T10:13:23.124643+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:09.067719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698756+0000","last_change":"2024-04-25T10:13:22.175854+0000","last_active":"2024-04-25T10:13:24.698756+0000","last_peered":"2024-04-25T10:13:24.698756+0000","last_clean":"2024-04-25T10:13:24.698756+0000","last_became_active":"2024-04-25T10:13:22.174725+0000","last_became_peered":"2024-04-25T10:13:22.174725+0000","last_unstale":"2024-04-25T10:13:24.698756+0000","last_undegraded":"2024-04-25T10:13:24.698756+0000","last_fullsized":"2024-04-25T10:13:24.698756+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:30.954353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121716+0000","last_change":"2024-04-25T10:13:21.137409+0000","last_active":"2024-04-25T10:13:23.121716+0000","last_peered":"2024-04-25T10:13:23.121716+0000","last_clean":"2024-04-25T10:13:23.121716+0000","last_became_active":"2024-04-25T10:13:21.135819+0000","last_became_peered":"2024-04-25T10:13:21.135819+0000","last_unstale":"2024-04-25T10:13:23.121716+0000","last_undegraded":"2024-04-25T10:13:23.121716+0000","last_fullsized":"2024-04-25T10:13:23.121716+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:42.708121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.832495+0000","last_change":"2024-04-25T10:13:22.157562+0000","last_active":"2024-04-25T10:13:24.832495+0000","last_peered":"2024-04-25T10:13:24.832495+0000","last_clean":"2024-04-25T10:13:24.832495+0000","last_became_active":"2024-04-25T10:13:22.157317+0000","last_became_peered":"2024-04-25T10:13:22.157317+0000","last_unstale":"2024-04-25T10:13:24.832495+0000","last_undegraded":"2024-04-25T10:13:24.832495+0000","last_fullsized":"2024-04-25T10:13:24.832495+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:38.509711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.120222+0000","last_change":"2024-04-25T10:13:21.136923+0000","last_active":"2024-04-25T10:13:23.120222+0000","last_peered":"2024-04-25T10:13:23.120222+0000","last_clean":"2024-04-25T10:13:23.120222+0000","last_became_active":"2024-04-25T10:13:21.136615+0000","last_became_peered":"2024-04-25T10:13:21.136615+0000","last_unstale":"2024-04-25T10:13:23.120222+0000","last_undegraded":"2024-04-25T10:13:23.120222+0000","last_fullsized":"2024-04-25T10:13:23.120222+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:25.701632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119357+0000","last_change":"2024-04-25T10:13:22.163933+0000","last_active":"2024-04-25T10:13:23.119357+0000","last_peered":"2024-04-25T10:13:23.119357+0000","last_clean":"2024-04-25T10:13:23.119357+0000","last_became_active":"2024-04-25T10:13:22.163616+0000","last_became_peered":"2024-04-25T10:13:22.163616+0000","last_unstale":"2024-04-25T10:13:23.119357+0000","last_undegraded":"2024-04-25T10:13:23.119357+0000","last_fullsized":"2024-04-25T10:13:23.119357+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:30.574918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121907+0000","last_change":"2024-04-25T10:13:21.122791+0000","last_active":"2024-04-25T10:13:23.121907+0000","last_peered":"2024-04-25T10:13:23.121907+0000","last_clean":"2024-04-25T10:13:23.121907+0000","last_became_active":"2024-04-25T10:13:21.122081+0000","last_became_peered":"2024-04-25T10:13:21.122081+0000","last_unstale":"2024-04-25T10:13:23.121907+0000","last_undegraded":"2024-04-25T10:13:23.121907+0000","last_fullsized":"2024-04-25T10:13:23.121907+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:25.894975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830741+0000","last_change":"2024-04-25T10:13:22.154950+0000","last_active":"2024-04-25T10:13:24.830741+0000","last_peered":"2024-04-25T10:13:24.830741+0000","last_clean":"2024-04-25T10:13:24.830741+0000","last_became_active":"2024-04-25T10:13:22.154590+0000","last_became_peered":"2024-04-25T10:13:22.154590+0000","last_unstale":"2024-04-25T10:13:24.830741+0000","last_undegraded":"2024-04-25T10:13:24.830741+0000","last_fullsized":"2024-04-25T10:13:24.830741+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:28.202937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.121758+0000","last_change":"2024-04-25T10:13:21.142854+0000","last_active":"2024-04-25T10:13:23.121758+0000","last_peered":"2024-04-25T10:13:23.121758+0000","last_clean":"2024-04-25T10:13:23.121758+0000","last_became_active":"2024-04-25T10:13:21.141977+0000","last_became_peered":"2024-04-25T10:13:21.141977+0000","last_unstale":"2024-04-25T10:13:23.121758+0000","last_undegraded":"2024-04-25T10:13:23.121758+0000","last_fullsized":"2024-04-25T10:13:23.121758+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:39.638106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697793+0000","last_change":"2024-04-25T10:13:22.171868+0000","last_active":"2024-04-25T10:13:24.697793+0000","last_peered":"2024-04-25T10:13:24.697793+0000","last_clean":"2024-04-25T10:13:24.697793+0000","last_became_active":"2024-04-25T10:13:22.171135+0000","last_became_peered":"2024-04-25T10:13:22.171135+0000","last_unstale":"2024-04-25T10:13:24.697793+0000","last_undegraded":"2024-04-25T10:13:24.697793+0000","last_fullsized":"2024-04-25T10:13:24.697793+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:05.796548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:23.119288+0000","last_change":"2024-04-25T10:13:21.135774+0000","last_active":"2024-04-25T10:13:23.119288+0000","last_peered":"2024-04-25T10:13:23.119288+0000","last_clean":"2024-04-25T10:13:23.119288+0000","last_became_active":"2024-04-25T10:13:21.135457+0000","last_became_peered":"2024-04-25T10:13:21.135457+0000","last_unstale":"2024-04-25T10:13:23.119288+0000","last_undegraded":"2024-04-25T10:13:23.119288+0000","last_fullsized":"2024-04-25T10:13:23.119288+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:44.136286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.469550+0000","last_change":"2024-04-25T10:13:22.160706+0000","last_active":"2024-04-25T10:13:24.469550+0000","last_peered":"2024-04-25T10:13:24.469550+0000","last_clean":"2024-04-25T10:13:24.469550+0000","last_became_active":"2024-04-25T10:13:22.160403+0000","last_became_peered":"2024-04-25T10:13:22.160403+0000","last_unstale":"2024-04-25T10:13:24.469550+0000","last_undegraded":"2024-04-25T10:13:24.469550+0000","last_fullsized":"2024-04-25T10:13:24.469550+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:21.093042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697409+0000","last_change":"2024-04-25T10:13:21.158159+0000","last_active":"2024-04-25T10:13:24.697409+0000","last_peered":"2024-04-25T10:13:24.697409+0000","last_clean":"2024-04-25T10:13:24.697409+0000","last_became_active":"2024-04-25T10:13:21.157463+0000","last_became_peered":"2024-04-25T10:13:21.157463+0000","last_unstale":"2024-04-25T10:13:24.697409+0000","last_undegraded":"2024-04-25T10:13:24.697409+0000","last_fullsized":"2024-04-25T10:13:24.697409+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.698391+0000","last_change":"2024-04-25T10:13:22.182871+0000","last_active":"2024-04-25T10:13:24.698391+0000","last_peered":"2024-04-25T10:13:24.698391+0000","last_clean":"2024-04-25T10:13:24.698391+0000","last_became_active":"2024-04-25T10:13:22.181637+0000","last_became_peered":"2024-04-25T10:13:22.181637+0000","last_unstale":"2024-04-25T10:13:24.698391+0000","last_undegraded":"2024-04-25T10:13:24.698391+0000","last_fullsized":"2024-04-25T10:13:24.698391+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:21.105543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:21.105543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:07.933528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"120'8","reported_seq":22,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.830482+0000","last_change":"2024-04-25T10:13:21.155420+0000","last_active":"2024-04-25T10:13:24.830482+0000","last_peered":"2024-04-25T10:13:24.830482+0000","last_clean":"2024-04-25T10:13:24.830482+0000","last_became_active":"2024-04-25T10:13:21.154940+0000","last_became_peered":"2024-04-25T10:13:21.154940+0000","last_unstale":"2024-04-25T10:13:24.830482+0000","last_undegraded":"2024-04-25T10:13:24.830482+0000","last_fullsized":"2024-04-25T10:13:24.830482+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:20.096978+0000","last_clean_scrub_stamp":"2024-04-25T10:13:20.096978+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:53.033818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":361,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-25T10:13:24.697618+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:13:24.697618+0000","last_peered":"2024-04-25T10:13:24.697618+0000","last_clean":"2024-04-25T10:13:24.697618+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:13:24.697618+0000","last_undegraded":"2024-04-25T10:13:24.697618+0000","last_fullsized":"2024-04-25T10:13:24.697618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2105344,"data_stored":1289847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574990,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1511424,"data_stored":705977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574991,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32652,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339188,"statfs":{"total":96636764160,"available":96603328512,"internally_reserved":0,"allocated":1503232,"data_stored":698863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32672,"kb_used_data":1484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339168,"statfs":{"total":96636764160,"available":96603308032,"internally_reserved":0,"allocated":1519616,"data_stored":707007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574991,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1511424,"data_stored":700349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574990,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1495040,"data_stored":698703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574990,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1499136,"data_stored":698725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33240,"kb_used_data":2056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338600,"statfs":{"total":96636764160,"available":96602726400,"internally_reserved":0,"allocated":2105344,"data_stored":1295785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:26.533 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:13:26.533 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:13:26.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 Processor -- start 2024-04-25T10:13:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 -- start start 2024-04-25T10:13:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5480d07f0 0x7fc5480d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54806be60 con 0x7fc5480d1cd0 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54815de60 con 0x7fc54815da80 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc54e12c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54806c440 con 0x7fc5480d07f0 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc546ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5480d07f0 0x7fc5480d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc546ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc5480d07f0 0x7fc5480d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43734/0 (socket says 172.21.15.82:43734) 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.667+0000 7fc546ffd640 1 -- 172.21.15.82:0/4169457924 learned_addr learned my addr 172.21.15.82:0/4169457924 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc546ffd640 1 -- 172.21.15.82:0/4169457924 >> v1:172.21.15.196:6789/0 conn(0x7fc54815da80 legacy=0x7fc5480d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc546ffd640 1 -- 172.21.15.82:0/4169457924 >> v1:172.21.15.82:6789/0 conn(0x7fc5480d1cd0 legacy=0x7fc54806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc546ffd640 1 -- 172.21.15.82:0/4169457924 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480d2450 con 0x7fc5480d07f0 2024-04-25T10:13:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc546ffd640 1 --2- 172.21.15.82:0/4169457924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5480d4230 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc53801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0247ccd76a1c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc5467fc640 1 -- 172.21.15.82:0/4169457924 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5380ab640 con 0x7fc5480d07f0 2024-04-25T10:13:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc5467fc640 1 -- 172.21.15.82:0/4169457924 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5380b0070 con 0x7fc5480d07f0 2024-04-25T10:13:26.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc5467fc640 1 -- 172.21.15.82:0/4169457924 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5380aba00 con 0x7fc5480d07f0 2024-04-25T10:13:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 -- 172.21.15.82:0/4169457924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 msgr2=0x7fc5480d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 --2- 172.21.15.82:0/4169457924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5480d4230 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fc538006fd0 tx=0x7fc53801c510 comp rx=0 tx=0).stop 2024-04-25T10:13:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 -- 172.21.15.82:0/4169457924 shutdown_connections 2024-04-25T10:13:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 --2- 172.21.15.82:0/4169457924 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5480d4230 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 -- 172.21.15.82:0/4169457924 >> 172.21.15.82:0/4169457924 conn(0x7fc54808b000 msgr2=0x7fc5480d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 -- 172.21.15.82:0/4169457924 shutdown_connections 2024-04-25T10:13:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.671+0000 7fc54e12c640 1 -- 172.21.15.82:0/4169457924 wait complete. 2024-04-25T10:13:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 Processor -- start 2024-04-25T10:13:26.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- start start 2024-04-25T10:13:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d1cd0 0x7fc5480c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 0x7fc54815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54806d970 con 0x7fc5480d07f0 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54806d7f0 con 0x7fc54815da80 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54806daf0 con 0x7fc5480d1cd0 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43742/0 (socket says 172.21.15.82:43742) 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 -- 172.21.15.82:0/3669135877 learned_addr learned my addr 172.21.15.82:0/3669135877 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc547fff640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 0x7fc54815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 -- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d1cd0 msgr2=0x7fc5480c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc546ffd640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d1cd0 0x7fc5480c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d1cd0 0x7fc5480c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 -- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 msgr2=0x7fc54815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 0x7fc54815c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 -- 172.21.15.82:0/3669135877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54815b850 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc547fff640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 0x7fc54815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc5477fe640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c420 tx=0x7fc53000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5300052d0 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc530005470 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc530007260 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54815bb30 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54815c0f0 con 0x7fc5480d07f0 2024-04-25T10:13:26.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54804efa0 con 0x7fc5480d07f0 2024-04-25T10:13:26.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc530007400 con 0x7fc5480d07f0 2024-04-25T10:13:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc530084680 con 0x7fc5480d07f0 2024-04-25T10:13:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.675+0000 7fc546ffd640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:26.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.679+0000 7fc546ffd640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 0x7fc528069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5380af000 tx=0x7fc5380a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:26.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.679+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc530055500 con 0x7fc5480d07f0 2024-04-25T10:13:26.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.879+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5480d1520 con 0x7fc5480d07f0 2024-04-25T10:13:26.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc53ffff640 1 -- 172.21.15.82:0/3669135877 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc530054e00 con 0x7fc5480d07f0 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stdout:[client.0] 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stdout: key = AQDkKSpmUYYuKBAAxADrnFOJrHSGF557UJQapA== 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stdout: caps mds = "allow" 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stdout: caps mon = "allow r" 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:26.883 INFO:teuthology.orchestra.run.smithi082.stderr:updated caps for client.0 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 msgr2=0x7fc528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 0x7fc528069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fc5380af000 tx=0x7fc5380a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 msgr2=0x7fc5481534c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fc53000c420 tx=0x7fc53000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 shutdown_connections 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc528066f60 0x7fc528069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5480d1cd0 0x7fc5480c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc54815da80 0x7fc54815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 --2- 172.21.15.82:0/3669135877 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5480d07f0 0x7fc5481534c0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 >> 172.21.15.82:0/3669135877 conn(0x7fc54808b000 msgr2=0x7fc548160880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.883+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 shutdown_connections 2024-04-25T10:13:26.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:26.887+0000 7fc54e12c640 1 -- 172.21.15.82:0/3669135877 wait complete. 2024-04-25T10:13:26.900 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.007+0000 7fe1cda5a640 1 Processor -- start 2024-04-25T10:13:27.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.007+0000 7fe1cda5a640 1 -- start start 2024-04-25T10:13:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1c815da80 0x7fe1c80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe1c80d1cd0 0x7fe1c80d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c80d0620 0x7fe1c80ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c806c440 con 0x7fe1c815da80 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c806be60 con 0x7fe1c80d1cd0 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1cda5a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c815de60 con 0x7fe1c80d0620 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1c815da80 0x7fe1c80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe1c815da80 0x7fe1c80d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43746/0 (socket says 172.21.15.82:43746) 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3840584854 learned_addr learned my addr 172.21.15.82:0/3840584854 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1be7fc640 1 --2- 172.21.15.82:0/3840584854 >> v2:172.21.15.196:3300/0 conn(0x7fe1c80d1cd0 0x7fe1c80d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c77fe640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c80d0620 0x7fe1c80ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3840584854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c80d0620 msgr2=0x7fe1c80ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c80d0620 0x7fe1c80ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3840584854 >> v2:172.21.15.196:3300/0 conn(0x7fe1c80d1cd0 msgr2=0x7fe1c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3840584854 >> v2:172.21.15.196:3300/0 conn(0x7fe1c80d1cd0 0x7fe1c80d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.011 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3840584854 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80d2450 con 0x7fe1c815da80 2024-04-25T10:13:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c815da80 0x7fe1c80d4230 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009870 tx=0x7fe1b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422b0fb0297ae358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c67fc640 1 -- 172.21.15.82:0/3840584854 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80b0070 con 0x7fe1c815da80 2024-04-25T10:13:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c67fc640 1 -- 172.21.15.82:0/3840584854 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80ab600 con 0x7fe1c815da80 2024-04-25T10:13:27.012 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.011+0000 7fe1c67fc640 1 -- 172.21.15.82:0/3840584854 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80ab9d0 con 0x7fe1c815da80 2024-04-25T10:13:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3840584854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c815da80 msgr2=0x7fe1c80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c815da80 0x7fe1c80d4230 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009870 tx=0x7fe1b801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:27.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3840584854 shutdown_connections 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c80d0620 0x7fe1c80ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3840584854 >> v2:172.21.15.196:3300/0 conn(0x7fe1c80d1cd0 0x7fe1c80d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3840584854 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c815da80 0x7fe1c80d4230 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3840584854 >> 172.21.15.82:0/3840584854 conn(0x7fe1c808b000 msgr2=0x7fe1c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3840584854 shutdown_connections 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3840584854 wait complete. 2024-04-25T10:13:27.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 Processor -- start 2024-04-25T10:13:27.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- start start 2024-04-25T10:13:27.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 0x7fe1c81436a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 0x7fe1c80b7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c806ca50 con 0x7fe1c80d0620 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c806c8d0 con 0x7fe1c80d1cd0 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c806cbd0 con 0x7fe1c815da80 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43758/0 (socket says 172.21.15.82:43758) 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3677299692 learned_addr learned my addr 172.21.15.82:0/3677299692 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c77fe640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 0x7fe1c80b7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1be7fc640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 0x7fe1c81436a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 msgr2=0x7fe1c80b7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 0x7fe1c80b7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 msgr2=0x7fe1c81436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 0x7fe1c81436a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 -- 172.21.15.82:0/3677299692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80b8500 con 0x7fe1c80d0620 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c77fe640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 0x7fe1c80b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1be7fc640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 0x7fe1c81436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c6ffd640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fe1b801abd0 tx=0x7fe1b801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80ba020 con 0x7fe1c80d0620 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b80b6640 con 0x7fe1c80d0620 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c80b87e0 con 0x7fe1c80d0620 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80b0040 con 0x7fe1c80d0620 2024-04-25T10:13:27.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.015+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c80b8da0 con 0x7fe1c80d0620 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c804efa0 con 0x7fe1c80d0620 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe1b80bd020 con 0x7fe1c80d0620 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1c4ff9640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 0x7fe1a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1be7fc640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 0x7fe1a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1b811f340 con 0x7fe1c80d0620 2024-04-25T10:13:27.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.019+0000 7fe1be7fc640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 0x7fe1a8069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe1c81561e0 tx=0x7fe1b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.023+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1b80c4030 con 0x7fe1c80d0620 2024-04-25T10:13:27.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.199+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1c8067e40 con 0x7fe1c80d0620 2024-04-25T10:13:27.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1c4ff9640 1 -- 172.21.15.82:0/3677299692 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe1b8016110 con 0x7fe1c80d0620 2024-04-25T10:13:27.201 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:27.201 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":91,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:24.487528+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9492},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9492":{"gid":9492,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":3,"addr":"172.21.15.82:6837/781412207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":781412207},{"type":"v1","addr":"172.21.15.82:6837","nonce":781412207}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:27.202 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 91 2024-04-25T10:13:27.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 msgr2=0x7fe1a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 0x7fe1a8069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fe1c81561e0 tx=0x7fe1b0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:27.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 msgr2=0x7fe1c8155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fe1b801abd0 tx=0x7fe1b801ad60 comp rx=0 tx=0).stop 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 shutdown_connections 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe1a8066f60 0x7fe1a8069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1c815da80 0x7fe1c80b7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1c80d1cd0 0x7fe1c81436a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 --2- 172.21.15.82:0/3677299692 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe1c80d0620 0x7fe1c8155ca0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 >> 172.21.15.82:0/3677299692 conn(0x7fe1c808b000 msgr2=0x7fe1c80b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 shutdown_connections 2024-04-25T10:13:27.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.203+0000 7fe1cda5a640 1 -- 172.21.15.82:0/3677299692 wait complete. 2024-04-25T10:13:27.620 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:13:24.487528+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9492}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9492': {'gid': 9492, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.82:6837/781412207', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 781412207}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 781412207}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:27.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:27.621 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 Processor -- start 2024-04-25T10:13:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 -- start start 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff10815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff1080b3f50 0x7ff1080cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1080b2090 con 0x7ff1080ce430 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10815f200 con 0x7ff1080b3f50 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.751+0000 7ff11003d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080b43e0 con 0x7ff10815a700 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff1080ce430 0x7ff1080b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36842/0 (socket says 172.21.15.82:36842) 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 -- 172.21.15.82:0/3847742668 learned_addr learned my addr 172.21.15.82:0/3847742668 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/3847742668 >> v2:172.21.15.196:3300/0 conn(0x7ff1080b3f50 0x7ff1080cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff10815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240509004 0 0) 0x7ff1080b2090 con 0x7ff1080ce430 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0f4003530 con 0x7ff1080ce430 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 108282675 0 0) 0x7ff0f4003530 con 0x7ff1080ce430 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 msgr2=0x7ff10815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff10815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 >> v2:172.21.15.196:3300/0 conn(0x7ff1080b3f50 msgr2=0x7ff1080cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 --2- 172.21.15.82:0/3847742668 >> v2:172.21.15.196:3300/0 conn(0x7ff1080b3f50 0x7ff1080cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080b46c0 con 0x7ff1080ce430 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7ff100001680 con 0x7ff1080ce430 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 >> v1:172.21.15.82:6789/0 conn(0x7ff1080ce430 legacy=0x7ff1080b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff0f4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0f4003fb0 0x7ff0f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0f4004d50 0x7ff0f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff10815f200 con 0x7ff0f4004d50 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080b43e0 con 0x7ff0f4003fb0 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f400bb30 con 0x7ff10815a700 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0f4004d50 0x7ff0f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff0f4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0f4003fb0 0x7ff0f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 -- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 msgr2=0x7ff0f4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff10815a700 0x7ff0f4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 -- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0f4003fb0 msgr2=0x7ff0f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0f4003fb0 0x7ff0f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10e5da640 1 -- 172.21.15.82:0/3847742668 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4003530 con 0x7ff0f4004d50 2024-04-25T10:13:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.755+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff0f4003fb0 0x7ff0f4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff10e5da640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0f4004d50 0x7ff0f40051d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c2c0 tx=0x7ff10001cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97a180437985c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100007d60 con 0x7ff0f4004d50 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1000a9020 con 0x7ff0f4004d50 2024-04-25T10:13:27.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff10cdd7640 1 -- 172.21.15.82:0/3847742668 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100005ae0 con 0x7ff0f4004d50 2024-04-25T10:13:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- 172.21.15.82:0/3847742668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0f4004d50 msgr2=0x7ff0f40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 --2- 172.21.15.82:0/3847742668 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff0f4004d50 0x7ff0f40051d0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7ff10000c2c0 tx=0x7ff10001cd10 comp rx=0 tx=0).stop 2024-04-25T10:13:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff10e5da640 1 -- 172.21.15.82:0/3847742668 reap_dead start 2024-04-25T10:13:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- 172.21.15.82:0/3847742668 shutdown_connections 2024-04-25T10:13:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- 172.21.15.82:0/3847742668 >> 172.21.15.82:0/3847742668 conn(0x7ff1080b11a0 msgr2=0x7ff1080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- 172.21.15.82:0/3847742668 shutdown_connections 2024-04-25T10:13:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- 172.21.15.82:0/3847742668 wait complete. 2024-04-25T10:13:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 Processor -- start 2024-04-25T10:13:27.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.759+0000 7ff11003d640 1 -- start start 2024-04-25T10:13:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff108150fb0 0x7ff108158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 0x7ff1080b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080cd490 con 0x7ff108151390 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080cd310 con 0x7ff1080b0630 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1080b2bb0 con 0x7ff108150fb0 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43784/0 (socket says 172.21.15.82:43784) 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 -- 172.21.15.82:0/714841881 learned_addr learned my addr 172.21.15.82:0/714841881 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff108150fb0 0x7ff108158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10e5da640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 0x7ff1080b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 -- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff108150fb0 msgr2=0x7ff108158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff108150fb0 0x7ff108158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 -- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 msgr2=0x7ff1080b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 0x7ff1080b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 -- 172.21.15.82:0/714841881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1080ab780 con 0x7ff108151390 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10e5da640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 0x7ff1080b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010950 tx=0x7ff0f8010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 -- 172.21.15.82:0/714841881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800f040 con 0x7ff108151390 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 -- 172.21.15.82:0/714841881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800ec20 con 0x7ff108151390 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1080aba60 con 0x7ff108151390 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 -- 172.21.15.82:0/714841881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8004020 con 0x7ff108151390 2024-04-25T10:13:27.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1080ac020 con 0x7ff108151390 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 -- 172.21.15.82:0/714841881 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff0f80041c0 con 0x7ff108151390 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 0x7ff0ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 0x7ff0ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff0fe7fc640 1 -- 172.21.15.82:0/714841881 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0f8085680 con 0x7ff108151390 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.763+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 0x7ff0ec069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff1080cba30 tx=0x7ff10400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 Processor -- start 2024-04-25T10:13:27.765 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- start start 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bab8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327ba6f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b9565f0 con 0x56327bab9240 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b8bdda0 con 0x56327ba6ef30 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327b846150 con 0x56327bab8920 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43788/0 (socket says 172.21.15.82:43788) 2024-04-25T10:13:27.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 -- 172.21.15.82:0/535892768 learned_addr learned my addr 172.21.15.82:0/535892768 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bab8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 -- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 msgr2=0x56327bab8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10e5da640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327ba6f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bab8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 -- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 msgr2=0x56327ba6f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327ba6f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 -- 172.21.15.82:0/535892768 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327b85fca0 con 0x56327bab9240 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bab8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8083980 tx=0x7ff0f80045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bff5171be7b3f53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff0eb7fe640 1 -- 172.21.15.82:0/535892768 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f805ae60 con 0x56327bab9240 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff0eb7fe640 1 -- 172.21.15.82:0/535892768 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8059f40 con 0x56327bab9240 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff0eb7fe640 1 -- 172.21.15.82:0/535892768 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8021040 con 0x56327bab9240 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- 172.21.15.82:0/535892768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 msgr2=0x56327ba6e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8083980 tx=0x7ff0f80045f0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- 172.21.15.82:0/535892768 shutdown_connections 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327ba6f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba6e9f0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 --2- 172.21.15.82:0/535892768 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bab8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- 172.21.15.82:0/535892768 >> 172.21.15.82:0/535892768 conn(0x56327ba752a0 msgr2=0x56327ba6d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- 172.21.15.82:0/535892768 shutdown_connections 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- 172.21.15.82:0/535892768 wait complete. 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 -- start start 2024-04-25T10:13:27.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.767+0000 7ff1131aa000 1 Processor -- start 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327babfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b878990 con 0x56327bab9240 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b8b9e40 con 0x56327ba6ef30 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327b849900 con 0x56327bab8920 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 10 client.0 _ll_register_callbacks cb 0x56327b9711c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43802/0 (socket says 172.21.15.82:43802) 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 -- 172.21.15.82:0/676392169 learned_addr learned my addr 172.21.15.82:0/676392169 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 -- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 msgr2=0x56327bac0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 -- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 msgr2=0x56327babfcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327babfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327babfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 -- 172.21.15.82:0/676392169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56327b876630 con 0x56327bab9240 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10e5da640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff100098590 tx=0x7ff10001b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182df0617f49bb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff100001c90 con 0x56327bab9240 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f8059f40 con 0x56327bab9240 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1000b9cb0 con 0x56327bab9240 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff100000e90 con 0x56327bab9240 2024-04-25T10:13:27.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 10 client.9645 fetch_fsmap learned FSMap version 91 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x56327b8463a0 con 0x56327bab9240 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8059f40 con 0x56327bab9240 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff10806e9b0 con 0x56327bab9240 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 10 client.9645 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 10 client.9645 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 20 client.9645 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327ba734b0 0x56327ba73890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 4 client.9645 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 --> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56327b859a50 con 0x56327ba734b0 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327ba734b0 0x56327ba73890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 10 client.9645 ms_handle_connect on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327ba734b0 0x56327ba73890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30235e51837dacbc server_cookie=811681d054f14614 in_seq=0 out_seq=0 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/676392169 <== mds.0 v2:172.21.15.82:6835/781412207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56327b859a50 con 0x56327ba734b0 2024-04-25T10:13:27.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 10 client.9645 handle_command_reply: tid=0 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 client.9645 shutdown 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 msgr2=0x56327ba72aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7ff100098590 tx=0x7ff10001b550 comp rx=0 tx=0).stop 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 shutdown_connections 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327ba734b0 0x56327ba73890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327bab9240 0x56327ba72aa0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 --2- 172.21.15.82:0/676392169 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327babfcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 >> 172.21.15.82:0/676392169 conn(0x56327ba752a0 msgr2=0x56327ba6d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff0e8ff9640 0 client.9645 ms_handle_reset on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 shutdown_connections 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 1 -- 172.21.15.82:0/676392169 wait complete. 2024-04-25T10:13:27.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.771+0000 7ff1131aa000 20 client.9645 trim_cache size 0 max 16384 2024-04-25T10:13:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 Processor -- start 2024-04-25T10:13:27.787 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 -- start start 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327ba752a0 0x56327ba72330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327bab8920 0x56327bac0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108203450 con 0x56327ba6ef30 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff108202850 con 0x56327bab8920 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff108202e50 con 0x56327ba752a0 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43804/0 (socket says 172.21.15.82:43804) 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 -- 172.21.15.82:0/68369544 learned_addr learned my addr 172.21.15.82:0/68369544 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327ba752a0 0x56327ba72330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10e5da640 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327bab8920 0x56327bac0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 -- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327ba752a0 msgr2=0x56327ba72330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327ba752a0 0x56327ba72330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 -- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327bab8920 msgr2=0x56327bac0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327bab8920 0x56327bac0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 -- 172.21.15.82:0/68369544 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56327b93f600 con 0x56327ba6ef30 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff0f805dfd0 tx=0x7ff0f801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=269df9ca007d3d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff0eaffd640 1 -- 172.21.15.82:0/68369544 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8021040 con 0x56327ba6ef30 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff0eaffd640 1 -- 172.21.15.82:0/68369544 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8059f40 con 0x56327ba6ef30 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.787+0000 7ff0eaffd640 1 -- 172.21.15.82:0/68369544 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f805e500 con 0x56327ba6ef30 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/68369544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 msgr2=0x56327ba72870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.791 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7ff0f805dfd0 tx=0x7ff0f801e070 comp rx=0 tx=0).stop 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/68369544 shutdown_connections 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327bab8920 0x56327bac0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba6ef30 0x56327ba72870 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/68369544 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327ba752a0 0x56327ba72330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/68369544 >> 172.21.15.82:0/68369544 conn(0x56327bac01b0 msgr2=0x56327ba743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/68369544 shutdown_connections 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/68369544 wait complete. 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- start start 2024-04-25T10:13:27.792 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 Processor -- start 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327bb18f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b8c23d0 con 0x56327ba752a0 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x56327b979d50 con 0x56327ba6ef30 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x56327b858980 con 0x56327bab8920 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 10 client.0 _ll_register_callbacks cb 0x7ff0f8052090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43814/0 (socket says 172.21.15.82:43814) 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 -- 172.21.15.82:0/2790901532 learned_addr learned my addr 172.21.15.82:0/2790901532 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 -- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 msgr2=0x56327bac34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10e5da640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327bb18f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 -- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 msgr2=0x56327bb18f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327bb18f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 -- 172.21.15.82:0/2790901532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56327b8529b0 con 0x56327ba752a0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10d5d8640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010920 tx=0x7ff0f8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8958a4a10fe051f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8073070 con 0x56327ba752a0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0f806dd80 con 0x56327ba752a0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0f8059f40 con 0x56327ba752a0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0f8021040 con 0x56327ba752a0 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 10 client.9651 fetch_fsmap learned FSMap version 91 2024-04-25T10:13:27.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x56327b8650b0 con 0x56327ba752a0 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8059f40 con 0x56327ba752a0 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff0f8055240 con 0x56327ba752a0 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 10 client.9651 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 10 client.9651 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 20 client.9651 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327bab9240 0x56327bac3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 4 client.9651 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 --> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56327b8cb2e0 con 0x56327bab9240 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327bab9240 0x56327bac3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 10 client.9651 ms_handle_connect on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff10ddd9640 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327bab9240 0x56327bac3a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92b37c39c075f8ca server_cookie=cb08c41f2ce7eeb6 in_seq=0 out_seq=0 2024-04-25T10:13:27.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 1 -- 172.21.15.82:0/2790901532 <== mds.0 v2:172.21.15.82:6835/781412207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56327b8cb2e0 con 0x56327bab9240 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 10 client.9651 handle_command_reply: tid=0 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 client.9651 shutdown 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 msgr2=0x56327bb194a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010920 tx=0x7ff0f8000bd0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 shutdown_connections 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x56327bab9240 0x56327bac3a30 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x56327bab8920 0x56327bac34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x56327ba752a0 0x56327bb194a0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff0e8ff9640 0 client.9651 ms_handle_reset on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 --2- 172.21.15.82:0/2790901532 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x56327ba6ef30 0x56327bb18f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.791+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 >> 172.21.15.82:0/2790901532 conn(0x56327bac01b0 msgr2=0x56327ba74ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 shutdown_connections 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff1131aa000 1 -- 172.21.15.82:0/2790901532 wait complete. 2024-04-25T10:13:27.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff1131aa000 20 client.9651 trim_cache size 0 max 16384 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9492, 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 5.6801743099999999, 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 91, 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 121, 2024-04-25T10:13:27.800 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 121, 2024-04-25T10:13:27.801 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 10.052181952 2024-04-25T10:13:27.801 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:27.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 msgr2=0x7ff0ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 0x7ff0ec069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7ff1080cba30 tx=0x7ff10400b040 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 msgr2=0x7ff108159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010950 tx=0x7ff0f8010e10 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 shutdown_connections 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff0ec066f60 0x7ff0ec069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff1080b0630 0x7ff1080b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff108151390 0x7ff108159500 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 --2- 172.21.15.82:0/714841881 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff108150fb0 0x7ff108158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 >> 172.21.15.82:0/714841881 conn(0x7ff1080b11a0 msgr2=0x7ff1080cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 shutdown_connections 2024-04-25T10:13:27.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.795+0000 7ff11003d640 1 -- 172.21.15.82:0/714841881 wait complete. 2024-04-25T10:13:27.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:27.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 Processor -- start 2024-04-25T10:13:27.911 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 -- start start 2024-04-25T10:13:27.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7fec0815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec080b3f50 0x7fec080cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec080b2090 con 0x7fec080ce430 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec0815f200 con 0x7fec080b3f50 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0f60f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b43e0 con 0x7fec0815a700 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0dbac640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fec080ce430 0x7fec080b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36852/0 (socket says 172.21.15.82:36852) 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0dbac640 1 -- 172.21.15.82:0/3432637591 learned_addr learned my addr 172.21.15.82:0/3432637591 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7fec0815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7fec0cbaa640 1 --2- 172.21.15.82:0/3432637591 >> v2:172.21.15.196:3300/0 conn(0x7fec080b3f50 0x7fec080cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.911+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758021929 0 0) 0x7fec080b2090 con 0x7fec080ce430 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febf4003530 con 0x7fec080ce430 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 515907703 0 0) 0x7febf4003530 con 0x7fec080ce430 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 msgr2=0x7fec0815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7fec0815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 >> v2:172.21.15.196:3300/0 conn(0x7fec080b3f50 msgr2=0x7fec080cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 --2- 172.21.15.82:0/3432637591 >> v2:172.21.15.196:3300/0 conn(0x7fec080b3f50 0x7fec080cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec080b46c0 con 0x7fec080ce430 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fec00001a00 con 0x7fec080ce430 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7fec0815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 >> v1:172.21.15.82:6789/0 conn(0x7fec080ce430 legacy=0x7fec080b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7febf4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf4004810 0x7febf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7febf40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0815f200 con 0x7febf4003a30 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b43e0 con 0x7febf4004810 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf400bb50 con 0x7fec0815a700 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7febf4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 -- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 msgr2=0x7febf40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0dbac640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7febf40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0815a700 0x7febf40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 -- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf4004810 msgr2=0x7febf4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7febf4004810 0x7febf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 -- 172.21.15.82:0/3432637591 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4003530 con 0x7febf4003a30 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7febf4003e50 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fec040098d0 tx=0x7fec0401c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7901a37ed128599 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec0400f040 con 0x7febf4003a30 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec040b0070 con 0x7febf4003a30 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7febfffff640 1 -- 172.21.15.82:0/3432637591 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040ab800 con 0x7febf4003a30 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 -- 172.21.15.82:0/3432637591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 msgr2=0x7febf4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 --2- 172.21.15.82:0/3432637591 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7febf4003e50 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fec040098d0 tx=0x7fec0401c5a0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0dbac640 1 -- 172.21.15.82:0/3432637591 reap_dead start 2024-04-25T10:13:27.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 -- 172.21.15.82:0/3432637591 shutdown_connections 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 -- 172.21.15.82:0/3432637591 >> 172.21.15.82:0/3432637591 conn(0x7fec080b11a0 msgr2=0x7fec080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 -- 172.21.15.82:0/3432637591 shutdown_connections 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.915+0000 7fec0f60f640 1 -- 172.21.15.82:0/3432637591 wait complete. 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 Processor -- start 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- start start 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec08143d70 0x7fec0806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0806d5d0 0x7fec081e7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080cd490 con 0x7febf4003a30 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080cd310 con 0x7fec08143d70 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080b2bb0 con 0x7fec0806d5d0 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43834/0 (socket says 172.21.15.82:43834) 2024-04-25T10:13:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 -- 172.21.15.82:0/1844846996 learned_addr learned my addr 172.21.15.82:0/1844846996 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0cbaa640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec08143d70 0x7fec0806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0dbac640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0806d5d0 0x7fec081e7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 -- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0806d5d0 msgr2=0x7fec081e7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0806d5d0 0x7fec081e7c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 -- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec08143d70 msgr2=0x7fec0806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec08143d70 0x7fec0806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 -- 172.21.15.82:0/1844846996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec081e8950 con 0x7febf4003a30 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0d3ab640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fec040ad040 tx=0x7febf4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040b57c0 con 0x7febf4003a30 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec040b5960 con 0x7febf4003a30 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040b5d50 con 0x7febf4003a30 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec081e8c30 con 0x7febf4003a30 2024-04-25T10:13:27.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec081e91f0 con 0x7febf4003a30 2024-04-25T10:13:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec0804efa0 con 0x7febf4003a30 2024-04-25T10:13:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.919+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec040aa070 con 0x7febf4003a30 2024-04-25T10:13:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.923+0000 7febfdffb640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.923+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fec0411f650 con 0x7febf4003a30 2024-04-25T10:13:27.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.923+0000 7fec0cbaa640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.923+0000 7fec0cbaa640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 0x7febec069450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec0806dfa0 tx=0x7febf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:27.927+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec040162e0 con 0x7febf4003a30 2024-04-25T10:13:28.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fec08158630 con 0x7febf4003a30 2024-04-25T10:13:28.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7febfdffb640 1 -- 172.21.15.82:0/1844846996 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7fec040f04d0 con 0x7febf4003a30 2024-04-25T10:13:28.106 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:28.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 msgr2=0x7febec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 0x7febec069450 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fec0806dfa0 tx=0x7febf8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:28.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 msgr2=0x7fec081e99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fec040ad040 tx=0x7febf4005840 comp rx=0 tx=0).stop 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 shutdown_connections 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febec066f90 0x7febec069450 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec0806d5d0 0x7fec081e7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec08143d70 0x7fec0806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 --2- 172.21.15.82:0/1844846996 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7febf4003a30 0x7fec081e99f0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 >> 172.21.15.82:0/1844846996 conn(0x7fec080b11a0 msgr2=0x7fec080cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 shutdown_connections 2024-04-25T10:13:28.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:28.107+0000 7fec0f60f640 1 -- 172.21.15.82:0/1844846996 wait complete. 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:28.124 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:28.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.125 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:28.130 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41862sec preferred_lft 41862sec 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.131 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.132 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:28.133 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.133 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:28.133 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:28.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.133 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:28.186 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:13:28.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.187 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:28.238 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:28.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:28.239 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:13:28.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.312 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:13:28.358 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:28.358 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:28.358 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:28.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.358 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:13:28.414 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:13:28.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:28.414 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.461 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:28.462 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:28.462 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:28.509 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:28.566 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:28.615 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:28.616 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.617 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.618 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:28.666 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:13:28.666 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:13:28.711 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:28.726 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:28.773 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:28.773 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:28.773 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:28.774 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.775 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.776 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:28.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:28.777 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:28.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:13:28.791+0000 7f376fdce580 -1 init, newargv = 0x5567c9fd5c00 newargc=15 2024-04-25T10:13:28.791 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[54617]: starting ceph client 2024-04-25T10:13:28.797 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[54617]: starting fuse 2024-04-25T10:13:29.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:29.912 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:29.960 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:29.961 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:29.962 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:29.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:29.963 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:30.010 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:13:30.010 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:13:30.010 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:13:30.010 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:13:30.010 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:13:30.010 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.010 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:13:30.011 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:13:30.012 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:30.133 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.54617.asok 2024-04-25T10:13:30.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.144 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54617.asok 2024-04-25T10:13:30.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:30.144 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54617.asok status 2024-04-25T10:13:30.299 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:13:30.300 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:13:30.301 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "54617", 2024-04-25T10:13:30.301 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:13:30.301 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:30.301 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:13:30.301 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9666, 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:13:30.302 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 9666 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4291624572 2024-04-25T10:13:30.303 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 4291624572 2024-04-25T10:13:30.304 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:30.305 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.9666 192.168.0.1:0/4291624572", 2024-04-25T10:13:30.305 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/4291624572", 2024-04-25T10:13:30.305 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:13:30.305 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 91, 2024-04-25T10:13:30.305 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 121, 2024-04-25T10:13:30.306 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:13:30.306 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:13:30.306 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:13:30.306 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:30.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:30.313 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:13:30.313 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:30.321 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:13:30.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:30.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:30.321 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:30.379 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> parent = os.path.join(mount_point, "parent") 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> os.mkdir(parent) 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> flushed = os.path.join(parent, "flushed") 2024-04-25T10:13:30.380 DEBUG:teuthology.orchestra.run.smithi082:> os.mkdir(flushed) 2024-04-25T10:13:30.381 DEBUG:teuthology.orchestra.run.smithi082:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T10:13:30.381 DEBUG:teuthology.orchestra.run.smithi082:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T10:13:30.381 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:30.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:30.517 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./ 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./parent 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/charlie 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/alpha 2024-04-25T10:13:30.585 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/bravo 2024-04-25T10:13:30.585 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.585 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.586 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:30.586 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.586 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:13:30.586 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:30.693 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:13:30.694 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.701 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.702 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.702 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:30.702 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.702 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:13:30.702 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:30.808 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:13:30.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:13:30.818 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:30.920 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:13:30.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:13:30.930 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:31.035 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628280 2024-04-25T10:13:31.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:13:31.046 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:31.153 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627778 2024-04-25T10:13:31.156 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:13:31.163 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:31.268 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628279 2024-04-25T10:13:31.271 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:31.278 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:31.278 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:31.331 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.499+0000 7f16dfb24640 1 Processor -- start 2024-04-25T10:13:31.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.499+0000 7f16dfb24640 1 -- start start 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dfb24640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f16d815a700 0x7f16d815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dfb24640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80b2090 con 0x7f16d80b3f50 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dfb24640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80b43e0 con 0x7f16d80b24a0 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dfb24640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d815f200 con 0x7f16d815a700 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd0bf640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f16d80b3f50 0x7f16d80cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36862/0 (socket says 172.21.15.82:36862) 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd0bf640 1 -- 172.21.15.82:0/2979047410 learned_addr learned my addr 172.21.15.82:0/2979047410 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212599678 0 0) 0x7f16d80b2090 con 0x7f16d80b3f50 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16c4003530 con 0x7f16d80b3f50 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/2979047410 >> v2:172.21.15.196:3300/0 conn(0x7f16d815a700 0x7f16d815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3760974601 0 0) 0x7f16c4003530 con 0x7f16d80b3f50 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 >> v2:172.21.15.196:3300/0 conn(0x7f16d815a700 msgr2=0x7f16d815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 --2- 172.21.15.82:0/2979047410 >> v2:172.21.15.196:3300/0 conn(0x7f16d815a700 0x7f16d815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 >> v1:172.21.15.196:6789/0 conn(0x7f16d80b24a0 legacy=0x7f16d806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80b46c0 con 0x7f16d80b3f50 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f16c80016a0 con 0x7f16d80b3f50 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 >> v1:172.21.15.82:6789/0 conn(0x7f16d80b3f50 legacy=0x7f16d80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16c4003a30 0x7f16c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16c4004810 0x7f16c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815f200 con 0x7f16c40053d0 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c2b0 con 0x7f16c4004810 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c400c4b0 con 0x7f16c4003a30 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd0bf640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16c4004810 0x7f16c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16c4003a30 0x7f16c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 -- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16c4003a30 msgr2=0x7f16c4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16c4003a30 0x7f16c4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 -- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16c4004810 msgr2=0x7f16c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16c4004810 0x7f16c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16de0c1640 1 -- 172.21.15.82:0/2979047410 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c4003530 con 0x7f16c40053d0 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16c4003a30 0x7f16c4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:31.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.503+0000 7f16dd0bf640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16c4004810 0x7f16c4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16de0c1640 1 -- 172.21.15.82:0/2979047410 reap_dead start 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16de0c1640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16c400b9d0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f16d00044b0 tx=0x7f16d001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1f695889ff79cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00040b0 con 0x7f16c40053d0 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d00a9670 con 0x7f16c40053d0 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dc8be640 1 -- 172.21.15.82:0/2979047410 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d00a9a60 con 0x7f16c40053d0 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 msgr2=0x7f16c400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16c400b9d0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f16d00044b0 tx=0x7f16d001c670 comp rx=0 tx=0).stop 2024-04-25T10:13:31.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/2979047410 shutdown_connections 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 --2- 172.21.15.82:0/2979047410 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16c400b9d0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/2979047410 >> 172.21.15.82:0/2979047410 conn(0x7f16d80b11a0 msgr2=0x7f16d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/2979047410 shutdown_connections 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/2979047410 wait complete. 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 Processor -- start 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- start start 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8152af0 0x7f16d8153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 0x7f16d815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80cd490 con 0x7f16c40053d0 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80cd310 con 0x7f16d8153fb0 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d815ecc0 con 0x7f16d8152af0 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43864/0 (socket says 172.21.15.82:43864) 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 -- 172.21.15.82:0/3722124788 learned_addr learned my addr 172.21.15.82:0/3722124788 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16de0c1640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 0x7f16d815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd0bf640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8152af0 0x7f16d8153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 -- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8152af0 msgr2=0x7f16d8153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8152af0 0x7f16d8153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 -- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 msgr2=0x7f16d815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 0x7f16d815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 -- 172.21.15.82:0/3722124788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8154db0 con 0x7f16c40053d0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16de0c1640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 0x7f16d815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dd8c0640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e810 tx=0x7f16d400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d4017070 con 0x7f16c40053d0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d400c6d0 con 0x7f16c40053d0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d8155090 con 0x7f16c40053d0 2024-04-25T10:13:31.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.507+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d400f040 con 0x7f16c40053d0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8155650 con 0x7f16c40053d0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d804efa0 con 0x7f16c40053d0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f16d401a070 con 0x7f16c40053d0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16ce7fc640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 0x7f16bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16dd0bf640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 0x7f16bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16d4084d30 con 0x7f16c40053d0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.511+0000 7f16dd0bf640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 0x7f16bc069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f16c8001040 tx=0x7f16c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.515+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16d4055bb0 con 0x7f16c40053d0 2024-04-25T10:13:31.679 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.679+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16d806f260 con 0x7f16c40053d0 2024-04-25T10:13:31.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16ce7fc640 1 -- 172.21.15.82:0/3722124788 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f16d40554b0 con 0x7f16c40053d0 2024-04-25T10:13:31.682 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:31.683 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":121,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:23.113792+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 msgr2=0x7f16bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 0x7f16bc069450 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f16c8001040 tx=0x7f16c8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 msgr2=0x7f16d815d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e810 tx=0x7f16d400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 shutdown_connections 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f16bc066f90 0x7f16bc069450 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f16d8152af0 0x7f16d8153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f16d8153fb0 0x7f16d815bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 --2- 172.21.15.82:0/3722124788 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f16c40053d0 0x7f16d815d520 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 >> 172.21.15.82:0/3722124788 conn(0x7f16d80b11a0 msgr2=0x7f16d806e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 shutdown_connections 2024-04-25T10:13:31.686 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.683+0000 7f16dfb24640 1 -- 172.21.15.82:0/3722124788 wait complete. 2024-04-25T10:13:31.701 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:31.702 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:31.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 Processor -- start 2024-04-25T10:13:31.821 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 -- start start 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f68f815b420 0x7f68f80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68f80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f80cbd20 con 0x7f68f80cfad0 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f80cff20 con 0x7f68f815b420 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68ffb97640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cc930 con 0x7f68f815b800 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fe134640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f68f80cfad0 0x7f68f815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36878/0 (socket says 172.21.15.82:36878) 2024-04-25T10:13:31.824 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fe134640 1 -- 172.21.15.82:0/2470733649 learned_addr learned my addr 172.21.15.82:0/2470733649 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 --2- 172.21.15.82:0/2470733649 >> v2:172.21.15.196:3300/0 conn(0x7f68f815b420 0x7f68f80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529047182 0 0) 0x7f68f80cbd20 con 0x7f68f80cfad0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd132640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68f80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e4003530 con 0x7f68f80cfad0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2659169244 0 0) 0x7f68e4003530 con 0x7f68f80cfad0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 msgr2=0x7f68f80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68f80cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 >> v2:172.21.15.196:3300/0 conn(0x7f68f815b420 msgr2=0x7f68f80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 --2- 172.21.15.82:0/2470733649 >> v2:172.21.15.196:3300/0 conn(0x7f68f815b420 0x7f68f80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80cc000 con 0x7f68f80cfad0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f68e0001680 con 0x7f68f80cfad0 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd132640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68f80cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 >> v1:172.21.15.82:6789/0 conn(0x7f68f80cfad0 legacy=0x7f68f815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4003a30 0x7f68e4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cff20 con 0x7f68e4003a30 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cc930 con 0x7f68e4004d50 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e400ba00 con 0x7f68f815b800 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4003a30 0x7f68e4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd132640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fe134640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 -- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 msgr2=0x7f68e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 -- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4004d50 msgr2=0x7f68e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd933640 1 -- 172.21.15.82:0/2470733649 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4003530 con 0x7f68e4003a30 2024-04-25T10:13:31.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fd132640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f815b800 0x7f68e4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:31.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.823+0000 7f68fe134640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4004d50 0x7f68e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fd933640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4003a30 0x7f68e4003e50 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009fc0 tx=0x7f68e801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e0d29a66c54fba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e80b2070 con 0x7f68e4003a30 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e80a5c80 con 0x7f68e4003a30 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fc931640 1 -- 172.21.15.82:0/2470733649 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8009c10 con 0x7f68e4003a30 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- 172.21.15.82:0/2470733649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4003a30 msgr2=0x7f68e4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 --2- 172.21.15.82:0/2470733649 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68e4003a30 0x7f68e4003e50 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009fc0 tx=0x7f68e801cc90 comp rx=0 tx=0).stop 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 -- 172.21.15.82:0/2470733649 reap_dead start 2024-04-25T10:13:31.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- 172.21.15.82:0/2470733649 shutdown_connections 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- 172.21.15.82:0/2470733649 >> 172.21.15.82:0/2470733649 conn(0x7f68f806ae60 msgr2=0x7f68f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- 172.21.15.82:0/2470733649 shutdown_connections 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- 172.21.15.82:0/2470733649 wait complete. 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 Processor -- start 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- start start 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4003a30 0x7f68f80b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f80c3640 0x7f68f81f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80caa80 con 0x7f68f81f4e30 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80ca900 con 0x7f68e4003a30 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68ffb97640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80cadb0 con 0x7f68f80c3640 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43896/0 (socket says 172.21.15.82:43896) 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 -- 172.21.15.82:0/761716572 learned_addr learned my addr 172.21.15.82:0/761716572 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fd933640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4003a30 0x7f68f80b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fd132640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f80c3640 0x7f68f81f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 -- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f80c3640 msgr2=0x7f68f81f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f80c3640 0x7f68f81f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 -- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4003a30 msgr2=0x7f68f80b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.827+0000 7f68fe134640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4003a30 0x7f68f80b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68fe134640 1 -- 172.21.15.82:0/761716572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80c4090 con 0x7f68f81f4e30 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68fe134640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f68e00017f0 tx=0x7f68e000d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e00074e0 con 0x7f68f81f4e30 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e00047f0 con 0x7f68f81f4e30 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e0007680 con 0x7f68f81f4e30 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f80c4370 con 0x7f68f81f4e30 2024-04-25T10:13:31.832 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f80c4930 con 0x7f68f81f4e30 2024-04-25T10:13:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f804efa0 con 0x7f68f81f4e30 2024-04-25T10:13:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68e0004bc0 con 0x7f68f81f4e30 2024-04-25T10:13:31.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68fd933640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f68e00843c0 con 0x7f68f81f4e30 2024-04-25T10:13:31.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.831+0000 7f68fd933640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b0040 tx=0x7f68e8004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.835+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68e0055390 con 0x7f68f81f4e30 2024-04-25T10:13:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.995+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68f80b3ff0 con 0x7f68f81f4e30 2024-04-25T10:13:31.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68f67fc640 1 -- 172.21.15.82:0/761716572 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68e0054c90 con 0x7f68f81f4e30 2024-04-25T10:13:31.997 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 msgr2=0x7f68d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 0x7f68d8069450 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f68e80b0040 tx=0x7f68e8004ac0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 msgr2=0x7f68f80c5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f68e00017f0 tx=0x7f68e000d870 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 shutdown_connections 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f68d8066f90 0x7f68d8069450 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f68f80c3640 0x7f68f81f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f68f81f4e30 0x7f68f80c5130 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 --2- 172.21.15.82:0/761716572 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f68e4003a30 0x7f68f80b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 >> 172.21.15.82:0/761716572 conn(0x7f68f806ae60 msgr2=0x7f68f806d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 shutdown_connections 2024-04-25T10:13:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:31.999+0000 7f68ffb97640 1 -- 172.21.15.82:0/761716572 wait complete. 2024-04-25T10:13:32.016 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 Processor -- start 2024-04-25T10:13:32.133 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 -- start start 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff75c15ecb0 0x7ff75c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff75c0c9ef0 0x7ff75c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c0ca4c0 con 0x7ff75c0cf590 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0ca780 con 0x7ff75c15ecb0 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff76254a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c15f0f0 con 0x7ff75c0c9ef0 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff75c0cf590 0x7ff75c0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36884/0 (socket says 172.21.15.82:36884) 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff75c15ecb0 0x7ff75c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75bfff640 1 -- 172.21.15.82:0/2051113187 learned_addr learned my addr 172.21.15.82:0/2051113187 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff760ae7640 1 --2- 172.21.15.82:0/2051113187 >> v2:172.21.15.196:3300/0 conn(0x7ff75c0c9ef0 0x7ff75c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75affd640 1 -- 172.21.15.82:0/2051113187 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770403184 0 0) 0x7ff75c0ca4c0 con 0x7ff75c0cf590 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75affd640 1 -- 172.21.15.82:0/2051113187 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff744005740 con 0x7ff75c0cf590 2024-04-25T10:13:32.135 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 -- 172.21.15.82:0/2051113187 >> v2:172.21.15.196:3300/0 conn(0x7ff75c0c9ef0 msgr2=0x7ff75c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 --2- 172.21.15.82:0/2051113187 >> v2:172.21.15.196:3300/0 conn(0x7ff75c0c9ef0 0x7ff75c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 -- 172.21.15.82:0/2051113187 >> v1:172.21.15.82:6789/0 conn(0x7ff75c0cf590 legacy=0x7ff75c0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 -- 172.21.15.82:0/2051113187 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c0cd690 con 0x7ff75c15ecb0 2024-04-25T10:13:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75b7fe640 1 --2- 172.21.15.82:0/2051113187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c0c99b0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0b38c0 tx=0x7ff75001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515c7996d6f4c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.136 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75affd640 1 -- 172.21.15.82:0/2051113187 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500b1070 con 0x7ff75c15ecb0 2024-04-25T10:13:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75affd640 1 -- 172.21.15.82:0/2051113187 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500ac5f0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.135+0000 7ff75affd640 1 -- 172.21.15.82:0/2051113187 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500ac9c0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2051113187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 msgr2=0x7ff75c0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- 172.21.15.82:0/2051113187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c0c99b0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7ff75c0b38c0 tx=0x7ff75001c480 comp rx=0 tx=0).stop 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2051113187 shutdown_connections 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- 172.21.15.82:0/2051113187 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c0c99b0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- 172.21.15.82:0/2051113187 >> v2:172.21.15.196:3300/0 conn(0x7ff75c0c9ef0 0x7ff75c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2051113187 >> 172.21.15.82:0/2051113187 conn(0x7ff75c06ae60 msgr2=0x7ff75c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2051113187 shutdown_connections 2024-04-25T10:13:32.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2051113187 wait complete. 2024-04-25T10:13:32.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 Processor -- start 2024-04-25T10:13:32.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- start start 2024-04-25T10:13:32.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c0c9ef0 0x7ff75c0c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff75c0cf590 0x7ff75c153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0cc170 con 0x7ff75c15ecb0 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0cbff0 con 0x7ff75c0c9ef0 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0cc2f0 con 0x7ff75c0cf590 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43916/0 (socket says 172.21.15.82:43916) 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 -- 172.21.15.82:0/2121680583 learned_addr learned my addr 172.21.15.82:0/2121680583 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff75bfff640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c0c9ef0 0x7ff75c0c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff75b7fe640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff75c0cf590 0x7ff75c153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 -- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff75c0cf590 msgr2=0x7ff75c153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff75c0cf590 0x7ff75c153ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 -- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c0c9ef0 msgr2=0x7ff75c0c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c0c9ef0 0x7ff75c0c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 -- 172.21.15.82:0/2121680583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c072fc0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff760ae7640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff754007e40 tx=0x7ff75400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754014620 con 0x7ff75c15ecb0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff754019070 con 0x7ff75c15ecb0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0989a0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff754014a20 con 0x7ff75c15ecb0 2024-04-25T10:13:32.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.139+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c073260 con 0x7ff75c15ecb0 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c04efa0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff754014bc0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff758ff9640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 0x7ff748069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7540859f0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff75bfff640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 0x7ff748069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.143+0000 7ff75bfff640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 0x7ff748069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007fd0 tx=0x7ff74c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.146 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.147+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff754056900 con 0x7ff75c15ecb0 2024-04-25T10:13:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.299+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff75c0ca4c0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.299+0000 7ff758ff9640 1 -- 172.21.15.82:0/2121680583 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7540850a0 con 0x7ff75c15ecb0 2024-04-25T10:13:32.301 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 msgr2=0x7ff748069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 0x7ff748069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ff74c007fd0 tx=0x7ff74c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 msgr2=0x7ff75c154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff754007e40 tx=0x7ff75400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.303 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 shutdown_connections 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff748066f60 0x7ff748069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff75c0cf590 0x7ff75c153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff75c15ecb0 0x7ff75c154430 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 --2- 172.21.15.82:0/2121680583 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff75c0c9ef0 0x7ff75c0c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 >> 172.21.15.82:0/2121680583 conn(0x7ff75c06ae60 msgr2=0x7ff75c152bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 shutdown_connections 2024-04-25T10:13:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.303+0000 7ff76254a640 1 -- 172.21.15.82:0/2121680583 wait complete. 2024-04-25T10:13:32.319 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 Processor -- start 2024-04-25T10:13:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 -- start start 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f63140ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f6314073010 0x7f63140cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63140cbd20 con 0x7f63140cf590 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631415ba90 con 0x7f6314073010 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f631adda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140cff30 con 0x7f631415b6b0 2024-04-25T10:13:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f63140cf590 0x7f631415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36890/0 (socket says 172.21.15.82:36890) 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 -- 172.21.15.82:0/2648060728 learned_addr learned my addr 172.21.15.82:0/2648060728 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6319377640 1 --2- 172.21.15.82:0/2648060728 >> v2:172.21.15.196:3300/0 conn(0x7f6314073010 0x7f63140cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6318b76640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f63140ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127828016 0 0) 0x7f63140cbd20 con 0x7f63140cf590 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62fc003530 con 0x7f63140cf590 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879869713 0 0) 0x7f62fc003530 con 0x7f63140cf590 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 msgr2=0x7f63140ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f63140ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 >> v2:172.21.15.196:3300/0 conn(0x7f6314073010 msgr2=0x7f63140cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 --2- 172.21.15.82:0/2648060728 >> v2:172.21.15.196:3300/0 conn(0x7f6314073010 0x7f63140cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63140cc000 con 0x7f63140cf590 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6308001680 con 0x7f63140cf590 2024-04-25T10:13:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 >> v1:172.21.15.82:6789/0 conn(0x7f63140cf590 legacy=0x7f631415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f62fc003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f62fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62fc004d50 0x7f62fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631415ba90 con 0x7f62fc003fb0 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc00b9d0 con 0x7f62fc004d50 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc00bb80 con 0x7f631415b6b0 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f62fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6319377640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62fc004d50 0x7f62fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6318b76640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f62fc003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 -- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 msgr2=0x7f62fc003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415b6b0 0x7f62fc003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 -- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62fc004d50 msgr2=0x7f62fc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f62fc004d50 0x7f62fc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.443+0000 7f6313fff640 1 -- 172.21.15.82:0/2648060728 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc003530 con 0x7f62fc003fb0 2024-04-25T10:13:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f6313fff640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f62fc004810 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f630800c240 tx=0x7f630801f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17981f45aa21f5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308007d60 con 0x7f62fc003fb0 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308005b00 con 0x7f62fc003fb0 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f63137fe640 1 -- 172.21.15.82:0/2648060728 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308007070 con 0x7f62fc003fb0 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- 172.21.15.82:0/2648060728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 msgr2=0x7f62fc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 --2- 172.21.15.82:0/2648060728 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f62fc004810 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f630800c240 tx=0x7f630801f430 comp rx=0 tx=0).stop 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f6319377640 1 -- 172.21.15.82:0/2648060728 reap_dead start 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- 172.21.15.82:0/2648060728 shutdown_connections 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- 172.21.15.82:0/2648060728 >> 172.21.15.82:0/2648060728 conn(0x7f631406ae60 msgr2=0x7f631406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- 172.21.15.82:0/2648060728 shutdown_connections 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- 172.21.15.82:0/2648060728 wait complete. 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 Processor -- start 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.447+0000 7f631adda640 1 -- start start 2024-04-25T10:13:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415bc10 0x7f63140c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63140c3370 0x7f63140ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140cc8a0 con 0x7f62fc003fb0 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63140cc5c0 con 0x7f63140c3370 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63140cba30 con 0x7f631415bc10 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43926/0 (socket says 172.21.15.82:43926) 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 -- 172.21.15.82:0/656353219 learned_addr learned my addr 172.21.15.82:0/656353219 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6313fff640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415bc10 0x7f63140c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6319377640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63140c3370 0x7f63140ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 -- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415bc10 msgr2=0x7f63140c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415bc10 0x7f63140c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 -- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63140c3370 msgr2=0x7f63140ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63140c3370 0x7f63140ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.452 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 -- 172.21.15.82:0/656353219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631415c160 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6318b76640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6304011610 tx=0x7f6304011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304015870 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6304015a10 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631415c440 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6304015dc0 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63140ce0e0 con 0x7f62fc003fb0 2024-04-25T10:13:32.453 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631404efa0 con 0x7f62fc003fb0 2024-04-25T10:13:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f630400f480 con 0x7f62fc003fb0 2024-04-25T10:13:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63040859d0 con 0x7f62fc003fb0 2024-04-25T10:13:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6313fff640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.451+0000 7f6313fff640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f63080ae000 tx=0x7f6308001ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.456 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.455+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6304019030 con 0x7f62fc003fb0 2024-04-25T10:13:32.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.603+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63140cfa90 con 0x7f62fc003fb0 2024-04-25T10:13:32.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.603+0000 7f63117fa640 1 -- 172.21.15.82:0/656353219 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63040568e0 con 0x7f62fc003fb0 2024-04-25T10:13:32.603 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:32.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 msgr2=0x7f6300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 0x7f6300069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f63080ae000 tx=0x7f6308001ac0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 msgr2=0x7f63141f3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f6304011610 tx=0x7f6304011ad0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 shutdown_connections 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6300066f60 0x7f6300069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f631415bc10 0x7f63140c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f63140c3370 0x7f63140ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 --2- 172.21.15.82:0/656353219 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f62fc003fb0 0x7f63141f3200 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 >> 172.21.15.82:0/656353219 conn(0x7f631406ae60 msgr2=0x7f63140b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 shutdown_connections 2024-04-25T10:13:32.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.607+0000 7f631adda640 1 -- 172.21.15.82:0/656353219 wait complete. 2024-04-25T10:13:32.620 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:32.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 Processor -- start 2024-04-25T10:13:32.738 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 -- start start 2024-04-25T10:13:32.739 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 0x7fae3c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae3c0b3b30 0x7fae3c16b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c09a150 con 0x7fae3c0c44f0 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0cbc30 con 0x7fae3c0b3b30 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4336d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c09a430 con 0x7fae3c0cd0d0 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4190a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fae3c0b3b30 0x7fae3c16b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae41109640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fae3c0c44f0 0x7fae3c0c48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36894/0 (socket says 172.21.15.82:36894) 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae41109640 1 -- 172.21.15.82:0/691941902 learned_addr learned my addr 172.21.15.82:0/691941902 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae40908640 1 --2- 172.21.15.82:0/691941902 >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 0x7fae3c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4190a640 1 -- 172.21.15.82:0/691941902 >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 msgr2=0x7fae3c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4190a640 1 --2- 172.21.15.82:0/691941902 >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 0x7fae3c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4190a640 1 -- 172.21.15.82:0/691941902 >> v1:172.21.15.82:6789/0 conn(0x7fae3c0c44f0 legacy=0x7fae3c0c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae3bfff640 1 -- 172.21.15.82:0/691941902 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246693776 0 0) 0x7fae3c09a150 con 0x7fae3c0c44f0 2024-04-25T10:13:32.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae4190a640 1 -- 172.21.15.82:0/691941902 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c09a710 con 0x7fae3c0b3b30 2024-04-25T10:13:32.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.739+0000 7fae40908640 1 --2- 172.21.15.82:0/691941902 >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 0x7fae3c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:32.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4190a640 1 --2- 172.21.15.82:0/691941902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0b3b30 0x7fae3c16b850 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fae3c071db0 tx=0x7fae2c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824c3075a6d473a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae3bfff640 1 -- 172.21.15.82:0/691941902 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0b0070 con 0x7fae3c0b3b30 2024-04-25T10:13:32.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae3bfff640 1 -- 172.21.15.82:0/691941902 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c0ab600 con 0x7fae3c0b3b30 2024-04-25T10:13:32.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae3bfff640 1 -- 172.21.15.82:0/691941902 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c0ab9d0 con 0x7fae3c0b3b30 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- 172.21.15.82:0/691941902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0b3b30 msgr2=0x7fae3c16b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- 172.21.15.82:0/691941902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0b3b30 0x7fae3c16b850 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fae3c071db0 tx=0x7fae2c01c3f0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- 172.21.15.82:0/691941902 shutdown_connections 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- 172.21.15.82:0/691941902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0b3b30 0x7fae3c16b850 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- 172.21.15.82:0/691941902 >> v2:172.21.15.196:3300/0 conn(0x7fae3c0cd0d0 0x7fae3c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- 172.21.15.82:0/691941902 >> 172.21.15.82:0/691941902 conn(0x7fae3c0b3010 msgr2=0x7fae3c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- 172.21.15.82:0/691941902 shutdown_connections 2024-04-25T10:13:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- 172.21.15.82:0/691941902 wait complete. 2024-04-25T10:13:32.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 Processor -- start 2024-04-25T10:13:32.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- start start 2024-04-25T10:13:32.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae3c0b3b30 0x7fae3c0d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3c0cd0d0 0x7fae3c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c16e940 con 0x7fae3c0c44f0 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c16e7c0 con 0x7fae3c0cd0d0 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae4336d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c16eac0 con 0x7fae3c0b3b30 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae40908640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae40908640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43938/0 (socket says 172.21.15.82:43938) 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.743+0000 7fae40908640 1 -- 172.21.15.82:0/2673404306 learned_addr learned my addr 172.21.15.82:0/2673404306 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 -- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae3c0b3b30 msgr2=0x7fae3c0d0f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae3c0b3b30 0x7fae3c0d0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 -- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3c0cd0d0 msgr2=0x7fae3c1596e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3c0cd0d0 0x7fae3c1596e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 -- 172.21.15.82:0/2673404306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c158640 con 0x7fae3c0c44f0 2024-04-25T10:13:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae40908640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30007a30 con 0x7fae3c0c44f0 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30007bd0 con 0x7fae3c0c44f0 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c158920 con 0x7fae3c0c44f0 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300113f0 con 0x7fae3c0c44f0 2024-04-25T10:13:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c158ee0 con 0x7fae3c0c44f0 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c04efa0 con 0x7fae3c0c44f0 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae30011590 con 0x7fae3c0c44f0 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 0x7fae28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae30085700 con 0x7fae3c0c44f0 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae41109640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 0x7fae28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.747+0000 7fae41109640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 0x7fae28069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fae34001570 tx=0x7fae34007c40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.751+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae30015020 con 0x7fae3c0c44f0 2024-04-25T10:13:32.913 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.911+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae3c16bb70 con 0x7fae3c0c44f0 2024-04-25T10:13:32.914 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae39ffb640 1 -- 172.21.15.82:0/2673404306 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae300566f0 con 0x7fae3c0c44f0 2024-04-25T10:13:32.914 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 msgr2=0x7fae28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 0x7fae28069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fae34001570 tx=0x7fae34007c40 comp rx=0 tx=0).stop 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 msgr2=0x7fae3c156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 shutdown_connections 2024-04-25T10:13:32.916 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fae28066f90 0x7fae28069450 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fae3c0cd0d0 0x7fae3c1596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fae3c0c44f0 0x7fae3c156ae0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 --2- 172.21.15.82:0/2673404306 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fae3c0b3b30 0x7fae3c0d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 >> 172.21.15.82:0/2673404306 conn(0x7fae3c0b3010 msgr2=0x7fae3c0aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 shutdown_connections 2024-04-25T10:13:32.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:32.915+0000 7fae4336d640 1 -- 172.21.15.82:0/2673404306 wait complete. 2024-04-25T10:13:32.931 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:32.931 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.937 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 Processor -- start 2024-04-25T10:13:33.095 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 -- start start 2024-04-25T10:13:33.096 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0710144ca0 0x7f071006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 0x7f0710142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07100ccc70 con 0x7f07100ce510 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07100cc2b0 con 0x7f0710144ca0 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0717b6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071006e290 con 0x7f071006ec60 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715107640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0710144ca0 0x7f071006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715908640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f07100ce510 0x7f07100ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36896/0 (socket says 172.21.15.82:36896) 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715908640 1 -- 172.21.15.82:0/701839564 learned_addr learned my addr 172.21.15.82:0/701839564 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0716109640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 0x7f0710142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715107640 1 -- 172.21.15.82:0/701839564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 msgr2=0x7f0710142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715107640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 0x7f0710142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715107640 1 -- 172.21.15.82:0/701839564 >> v1:172.21.15.82:6789/0 conn(0x7f07100ce510 legacy=0x7f07100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0715107640 1 -- 172.21.15.82:0/701839564 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071006dc40 con 0x7f0710144ca0 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0714906640 1 -- 172.21.15.82:0/701839564 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490018537 0 0) 0x7f07100ccc70 con 0x7f07100ce510 2024-04-25T10:13:33.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.095+0000 7f0716109640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 0x7f0710142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0715107640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0710144ca0 0x7f071006e720 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f07000097c0 tx=0x7f070001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4238cae0b43cd0eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0714906640 1 -- 172.21.15.82:0/701839564 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000b5070 con 0x7f0710144ca0 2024-04-25T10:13:33.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0714906640 1 -- 172.21.15.82:0/701839564 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000a8990 con 0x7f0710144ca0 2024-04-25T10:13:33.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0714906640 1 -- 172.21.15.82:0/701839564 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07000a8d60 con 0x7f0710144ca0 2024-04-25T10:13:33.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 -- 172.21.15.82:0/701839564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0710144ca0 msgr2=0x7f071006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0710144ca0 0x7f071006e720 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f07000097c0 tx=0x7f070001ec90 comp rx=0 tx=0).stop 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 -- 172.21.15.82:0/701839564 shutdown_connections 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f071006ec60 0x7f0710142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 --2- 172.21.15.82:0/701839564 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0710144ca0 0x7f071006e720 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 -- 172.21.15.82:0/701839564 >> 172.21.15.82:0/701839564 conn(0x7f07100b11a0 msgr2=0x7f07100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 -- 172.21.15.82:0/701839564 shutdown_connections 2024-04-25T10:13:33.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.099+0000 7f0717b6c640 1 -- 172.21.15.82:0/701839564 wait complete. 2024-04-25T10:13:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 Processor -- start 2024-04-25T10:13:33.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- start start 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07100ce510 0x7f07100ca180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0710144ca0 0x7f07100ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07101451f0 con 0x7f071006ec60 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07100cc900 con 0x7f0710144ca0 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0710145370 con 0x7f07100ce510 2024-04-25T10:13:33.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43964/0 (socket says 172.21.15.82:43964) 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 -- 172.21.15.82:0/2854839906 learned_addr learned my addr 172.21.15.82:0/2854839906 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 -- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07100ce510 msgr2=0x7f07100ca180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07100ce510 0x7f07100ca180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 -- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0710144ca0 msgr2=0x7f07100ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0710144ca0 0x7f07100ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 -- 172.21.15.82:0/2854839906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071015b970 con 0x7f071006ec60 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715908640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f070c0011f0 tx=0x7f070c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c018070 con 0x7f071006ec60 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070c0074e0 con 0x7f071006ec60 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07101e3080 con 0x7f071006ec60 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c004a30 con 0x7f071006ec60 2024-04-25T10:13:33.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07101e3640 con 0x7f071006ec60 2024-04-25T10:13:33.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071004efa0 con 0x7f071006ec60 2024-04-25T10:13:33.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f070c01b070 con 0x7f071006ec60 2024-04-25T10:13:33.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f0715107640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.103+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f070c0844b0 con 0x7f071006ec60 2024-04-25T10:13:33.106 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.107+0000 7f0715107640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 0x7f06f4069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f07000b2000 tx=0x7f0700004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.111+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f070c0553c0 con 0x7f071006ec60 2024-04-25T10:13:33.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.259+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07100b0f80 con 0x7f071006ec60 2024-04-25T10:13:33.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.259+0000 7f07067fc640 1 -- 172.21.15.82:0/2854839906 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f070c054cc0 con 0x7f071006ec60 2024-04-25T10:13:33.260 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:33.260 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":121,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:23.113792+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:33.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 msgr2=0x7f06f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 0x7f06f4069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f07000b2000 tx=0x7f0700004740 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 msgr2=0x7f071015b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f070c0011f0 tx=0x7f070c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 shutdown_connections 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f06f4066f60 0x7f06f4069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07100ce510 0x7f07100ca180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0710144ca0 0x7f07100ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 --2- 172.21.15.82:0/2854839906 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f071006ec60 0x7f071015b430 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 >> 172.21.15.82:0/2854839906 conn(0x7f07100b11a0 msgr2=0x7f07100c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 shutdown_connections 2024-04-25T10:13:33.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.263+0000 7f0717b6c640 1 -- 172.21.15.82:0/2854839906 wait complete. 2024-04-25T10:13:33.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:33.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:33.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 Processor -- start 2024-04-25T10:13:33.402 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 -- start start 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c0cfca0 0x7f7b8c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b8c0cbf10 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b8c0cc1f0 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b90b09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0cc4d0 con 0x7f7b8c0cfca0 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43966/0 (socket says 172.21.15.82:43966) 2024-04-25T10:13:33.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 -- 172.21.15.82:0/3567331290 learned_addr learned my addr 172.21.15.82:0/3567331290 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b83fff640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c0cfca0 0x7f7b8c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 -- 172.21.15.82:0/3567331290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c0cfca0 msgr2=0x7f7b8c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c0cfca0 0x7f7b8c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 -- 172.21.15.82:0/3567331290 >> v1:172.21.15.196:6789/0 conn(0x7f7b8c0b3cf0 legacy=0x7f7b8c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.403+0000 7f7b8ad9d640 1 -- 172.21.15.82:0/3567331290 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c0cc7b0 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8ad9d640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009e50 tx=0x7f7b8401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a600a376518732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b89d9b640 1 -- 172.21.15.82:0/3567331290 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840a5700 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b89d9b640 1 -- 172.21.15.82:0/3567331290 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b840a58a0 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b89d9b640 1 -- 172.21.15.82:0/3567331290 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b840a5c70 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- 172.21.15.82:0/3567331290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c15e4f0 msgr2=0x7f7b8c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009e50 tx=0x7f7b8401ca30 comp rx=0 tx=0).stop 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- 172.21.15.82:0/3567331290 shutdown_connections 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c0cfca0 0x7f7b8c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3567331290 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c15e4f0 0x7f7b8c0caaf0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- 172.21.15.82:0/3567331290 >> 172.21.15.82:0/3567331290 conn(0x7f7b8c06ae60 msgr2=0x7f7b8c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- 172.21.15.82:0/3567331290 shutdown_connections 2024-04-25T10:13:33.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- 172.21.15.82:0/3567331290 wait complete. 2024-04-25T10:13:33.409 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 Processor -- start 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- start start 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b8c0cfca0 0x7f7b8c15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c15e4f0 0x7f7b8c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0cb9c0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.410 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0cb840 con 0x7f7b8c0cfca0 2024-04-25T10:13:33.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b90b09640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8c0cbb40 con 0x7f7b8c15e4f0 2024-04-25T10:13:33.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43976/0 (socket says 172.21.15.82:43976) 2024-04-25T10:13:33.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 -- 172.21.15.82:0/3388555739 learned_addr learned my addr 172.21.15.82:0/3388555739 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:33.411 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8ad9d640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c15e4f0 0x7f7b8c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b83fff640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b8c0cfca0 0x7f7b8c15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 -- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c15e4f0 msgr2=0x7f7b8c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c15e4f0 0x7f7b8c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 -- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b8c0cfca0 msgr2=0x7f7b8c15b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b8c0cfca0 0x7f7b8c15b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.412 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 -- 172.21.15.82:0/3388555739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b8c1603a0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b8a59c640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c450 tx=0x7f7b7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.407+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c016020 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7c0052d0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8c15bdf0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c007260 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8c15c3b0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8c04efa0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b7c007400 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b82ffd640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 0x7f7b6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b7c084ba0 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b83fff640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 0x7f7b6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.411+0000 7f7b83fff640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 0x7f7b6c069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c15afe0 tx=0x7f7b74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.415+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7b7c020020 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.575 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.575+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b8c0b3060 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.576 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.575+0000 7f7b82ffd640 1 -- 172.21.15.82:0/3388555739 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b7c005480 con 0x7f7b8c0b3cf0 2024-04-25T10:13:33.576 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 msgr2=0x7f7b6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 0x7f7b6c069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c15afe0 tx=0x7f7b74008040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 msgr2=0x7f7b8c15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c00c450 tx=0x7f7b7c00c910 comp rx=0 tx=0).stop 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 shutdown_connections 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7b6c066f60 0x7f7b6c069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7b8c15e4f0 0x7f7b8c0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7b8c0cfca0 0x7f7b8c15b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 --2- 172.21.15.82:0/3388555739 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7b8c0b3cf0 0x7f7b8c15fe60 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.579 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 >> 172.21.15.82:0/3388555739 conn(0x7f7b8c06ae60 msgr2=0x7f7b8c156cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 shutdown_connections 2024-04-25T10:13:33.580 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.579+0000 7f7b90b09640 1 -- 172.21.15.82:0/3388555739 wait complete. 2024-04-25T10:13:33.595 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:33.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.715+0000 7ffbdd074640 1 Processor -- start 2024-04-25T10:13:33.717 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 -- start start 2024-04-25T10:13:33.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159bd0 0x7ffbd80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbd8159fb0 0x7ffbd80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd80c9ef0 con 0x7ffbd80b26f0 2024-04-25T10:13:33.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd8072780 con 0x7ffbd8159fb0 2024-04-25T10:13:33.718 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbdd074640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8071b70 con 0x7ffbd8159bd0 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ffbd8159fb0 0x7ffbd80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd759e640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ffbd80b26f0 0x7ffbd80cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36898/0 (socket says 172.21.15.82:36898) 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd759e640 1 -- 172.21.15.82:0/420880971 learned_addr learned my addr 172.21.15.82:0/420880971 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 -- 172.21.15.82:0/420880971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159bd0 msgr2=0x7ffbd80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 --2- 172.21.15.82:0/420880971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159bd0 0x7ffbd80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 -- 172.21.15.82:0/420880971 >> v1:172.21.15.82:6789/0 conn(0x7ffbd80b26f0 legacy=0x7ffbd80cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 -- 172.21.15.82:0/420880971 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd8071e50 con 0x7ffbd8159fb0 2024-04-25T10:13:33.719 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd5d9b640 1 -- 172.21.15.82:0/420880971 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030043543 0 0) 0x7ffbd80c9ef0 con 0x7ffbd80b26f0 2024-04-25T10:13:33.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd659c640 1 --2- 172.21.15.82:0/420880971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd8159fb0 0x7ffbd80b21b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0097c0 tx=0x7ffbcc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bebcef1fd1068b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd5d9b640 1 -- 172.21.15.82:0/420880971 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0a7070 con 0x7ffbd8159fb0 2024-04-25T10:13:33.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd5d9b640 1 -- 172.21.15.82:0/420880971 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc0b0070 con 0x7ffbd8159fb0 2024-04-25T10:13:33.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.719+0000 7ffbd5d9b640 1 -- 172.21.15.82:0/420880971 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0ab890 con 0x7ffbd8159fb0 2024-04-25T10:13:33.721 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/420880971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd8159fb0 msgr2=0x7ffbd80b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- 172.21.15.82:0/420880971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd8159fb0 0x7ffbd80b21b0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0097c0 tx=0x7ffbcc01c470 comp rx=0 tx=0).stop 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/420880971 shutdown_connections 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- 172.21.15.82:0/420880971 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd8159fb0 0x7ffbd80b21b0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- 172.21.15.82:0/420880971 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159bd0 0x7ffbd80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/420880971 >> 172.21.15.82:0/420880971 conn(0x7ffbd806ae60 msgr2=0x7ffbd806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/420880971 shutdown_connections 2024-04-25T10:13:33.722 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/420880971 wait complete. 2024-04-25T10:13:33.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 Processor -- start 2024-04-25T10:13:33.723 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- start start 2024-04-25T10:13:33.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.724 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 0x7ffbd815a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159fb0 0x7ffbd80c2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8166980 con 0x7ffbd80b26f0 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd80cefd0 con 0x7ffbd8159bd0 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8166b00 con 0x7ffbd8159fb0 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:43992/0 (socket says 172.21.15.82:43992) 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 -- 172.21.15.82:0/4027271112 learned_addr learned my addr 172.21.15.82:0/4027271112 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd659c640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 0x7ffbd815a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 -- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159fb0 msgr2=0x7ffbd80c2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159fb0 0x7ffbd80c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 -- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 msgr2=0x7ffbd815a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 0x7ffbd815a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.725 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 -- 172.21.15.82:0/4027271112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd80c3370 con 0x7ffbd80b26f0 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd659c640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 0x7ffbd815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbd6d9d640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b550 tx=0x7ffbc800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8014070 con 0x7ffbd80b26f0 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8010890 con 0x7ffbd80b26f0 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8010c80 con 0x7ffbd80b26f0 2024-04-25T10:13:33.726 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd80c7e10 con 0x7ffbd80b26f0 2024-04-25T10:13:33.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.723+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd80c83d0 con 0x7ffbd80b26f0 2024-04-25T10:13:33.727 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd804efa0 con 0x7ffbd80b26f0 2024-04-25T10:13:33.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbc8017070 con 0x7ffbd80b26f0 2024-04-25T10:13:33.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbbf7fe640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 0x7ffbc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffbc8085080 con 0x7ffbd80b26f0 2024-04-25T10:13:33.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbd659c640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 0x7ffbc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.728 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.727+0000 7ffbd659c640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 0x7ffbc4069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af000 tx=0x7ffbcc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.731+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffbc8055e60 con 0x7ffbd80b26f0 2024-04-25T10:13:33.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffbd80cf9f0 con 0x7ffbd80b26f0 2024-04-25T10:13:33.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbbf7fe640 1 -- 172.21.15.82:0/4027271112 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffbc8010e20 con 0x7ffbd80b26f0 2024-04-25T10:13:33.878 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 msgr2=0x7ffbc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 0x7ffbc4069420 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc0af000 tx=0x7ffbcc0a7000 comp rx=0 tx=0).stop 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 msgr2=0x7ffbd8155230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b550 tx=0x7ffbc800ba10 comp rx=0 tx=0).stop 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 shutdown_connections 2024-04-25T10:13:33.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffbc4066f60 0x7ffbc4069420 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffbd8159fb0 0x7ffbd80c2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffbd8159bd0 0x7ffbd815a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 --2- 172.21.15.82:0/4027271112 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffbd80b26f0 0x7ffbd8155230 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 >> 172.21.15.82:0/4027271112 conn(0x7ffbd806ae60 msgr2=0x7ffbd80ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 shutdown_connections 2024-04-25T10:13:33.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:33.879+0000 7ffbdd074640 1 -- 172.21.15.82:0/4027271112 wait complete. 2024-04-25T10:13:33.895 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:34.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.019+0000 7f660080f640 1 Processor -- start 2024-04-25T10:13:34.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.019+0000 7f660080f640 1 -- start start 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f660080f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65fc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f660080f640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc15ff50 con 0x7f65fc0cf590 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f660080f640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65fc0cbd20 con 0x7f65fc073010 2024-04-25T10:13:34.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f660080f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc15ba90 con 0x7f65fc15b6b0 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f9d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f65fc0cf590 0x7f65fc15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:36912/0 (socket says 172.21.15.82:36912) 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f9d9b640 1 -- 172.21.15.82:0/1911046734 learned_addr learned my addr 172.21.15.82:0/1911046734 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65fc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301898471 0 0) 0x7f65fc15ff50 con 0x7f65fc0cf590 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e4003530 con 0x7f65fc0cf590 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418740023 0 0) 0x7f65e4003530 con 0x7f65fc0cf590 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 msgr2=0x7f65fc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65fc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 >> v1:172.21.15.196:6789/0 conn(0x7f65fc073010 legacy=0x7f65fc15f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0cc000 con 0x7f65fc0cf590 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f65f0001680 con 0x7f65fc0cf590 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 >> v1:172.21.15.82:6789/0 conn(0x7f65fc0cf590 legacy=0x7f65fc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65e4003a60 0x7f65e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e4004810 0x7f65e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc15ba90 con 0x7f65e4003a60 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400bad0 con 0x7f65e4004810 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e400bcd0 con 0x7f65fc15b6b0 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65e4003a60 0x7f65e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65f9d9b640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e4004810 0x7f65e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fad9d640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 -- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 msgr2=0x7f65e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 -- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e4004810 msgr2=0x7f65e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65e4004810 0x7f65e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 -- 172.21.15.82:0/1911046734 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e4003530 con 0x7f65e4003a60 2024-04-25T10:13:34.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fad9d640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65fc15b6b0 0x7f65e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.023+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65e4003a60 0x7f65e4003e80 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ae0 tx=0x7f65ec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418050a2985bc042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0b2070 con 0x7f65e4003a60 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65ec0a5530 con 0x7f65e4003a60 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65f959a640 1 -- 172.21.15.82:0/1911046734 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65ec0096b0 con 0x7f65e4003a60 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1911046734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65e4003a60 msgr2=0x7f65e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 --2- 172.21.15.82:0/1911046734 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65e4003a60 0x7f65e4003e80 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009ae0 tx=0x7f65ec01ca40 comp rx=0 tx=0).stop 2024-04-25T10:13:34.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 -- 172.21.15.82:0/1911046734 reap_dead start 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1911046734 shutdown_connections 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1911046734 >> 172.21.15.82:0/1911046734 conn(0x7f65fc06ae60 msgr2=0x7f65fc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1911046734 shutdown_connections 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1911046734 wait complete. 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 Processor -- start 2024-04-25T10:13:34.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- start start 2024-04-25T10:13:34.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65e4003a60 0x7f65fc0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65fc158f40 0x7f65fc0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0cfd80 con 0x7f65fc0c6740 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0cba30 con 0x7f65fc158f40 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65fc0cff00 con 0x7f65e4003a60 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44014/0 (socket says 172.21.15.82:44014) 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 -- 172.21.15.82:0/1374827681 learned_addr learned my addr 172.21.15.82:0/1374827681 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65e4003a60 0x7f65fc0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65f9d9b640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65fc158f40 0x7f65fc0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 -- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65e4003a60 msgr2=0x7f65fc0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65e4003a60 0x7f65fc0aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 -- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65fc158f40 msgr2=0x7f65fc0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65fc158f40 0x7f65fc0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 -- 172.21.15.82:0/1374827681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65fc0cd940 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65fad9d640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0d0180 tx=0x7f65f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4006d60 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f4003980 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65fc0cdc20 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f4003d70 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.027+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65fc0ce1e0 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65fc04efa0 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65f401b020 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f65eaffd640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 0x7f65dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65f4084cc0 con 0x7f65fc0c6740 2024-04-25T10:13:34.032 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 0x7f65dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.031+0000 7f65fa59c640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 0x7f65dc069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0af000 tx=0x7f65ec0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.035+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65f4055580 con 0x7f65fc0c6740 2024-04-25T10:13:34.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.179+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65fc166fa0 con 0x7f65fc0c6740 2024-04-25T10:13:34.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f65eaffd640 1 -- 172.21.15.82:0/1374827681 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65f4059620 con 0x7f65fc0c6740 2024-04-25T10:13:34.181 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:34.183 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 msgr2=0x7f65dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 0x7f65dc069420 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0af000 tx=0x7f65ec0046d0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 msgr2=0x7f65fc0afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f65fc0d0180 tx=0x7f65f400dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 shutdown_connections 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65dc066f60 0x7f65dc069420 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65fc0c6740 0x7f65fc0afa10 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65fc158f40 0x7f65fc0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 --2- 172.21.15.82:0/1374827681 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65e4003a60 0x7f65fc0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 >> 172.21.15.82:0/1374827681 conn(0x7f65fc06ae60 msgr2=0x7f65fc0ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 shutdown_connections 2024-04-25T10:13:34.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.183+0000 7f660080f640 1 -- 172.21.15.82:0/1374827681 wait complete. 2024-04-25T10:13:34.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:34.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.307+0000 7f57a0e77640 1 Processor -- start 2024-04-25T10:13:34.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.307+0000 7f57a0e77640 1 -- start start 2024-04-25T10:13:34.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f57a0e77640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f579c15a700 0x7f579c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.309 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f57a0e77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f57a0e77640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f579c0b2090 con 0x7f579c0ce430 2024-04-25T10:13:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f57a0e77640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f579c15f200 con 0x7f579c15a700 2024-04-25T10:13:34.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f57a0e77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b43e0 con 0x7f579c0b3f50 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f579a59c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f579c15a700 0x7f579c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f579ad9d640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f579c0ce430 0x7f579c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:41896/0 (socket says 172.21.15.82:41896) 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:33062/0 (socket says 172.21.15.82:33062) 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f579ad9d640 1 -- 172.21.15.82:0/319142624 learned_addr learned my addr 172.21.15.82:0/319142624 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 -- 172.21.15.82:0/319142624 >> v2:172.21.15.196:3300/0 conn(0x7f579c15a700 msgr2=0x7f579c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 --2- 172.21.15.82:0/319142624 >> v2:172.21.15.196:3300/0 conn(0x7f579c15a700 0x7f579c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 -- 172.21.15.82:0/319142624 >> v1:172.21.15.196:6789/0 conn(0x7f579c0ce430 legacy=0x7f579c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 -- 172.21.15.82:0/319142624 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c0b46c0 con 0x7f579c0b3f50 2024-04-25T10:13:34.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.311+0000 7f5799d9b640 1 --2- 172.21.15.82:0/319142624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57900097c0 tx=0x7f579001c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ad02304a24f2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f579959a640 1 -- 172.21.15.82:0/319142624 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900b0070 con 0x7f579c0b3f50 2024-04-25T10:13:34.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f579959a640 1 -- 172.21.15.82:0/319142624 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790004230 con 0x7f579c0b3f50 2024-04-25T10:13:34.313 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f579959a640 1 -- 172.21.15.82:0/319142624 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5790003620 con 0x7f579c0b3f50 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- 172.21.15.82:0/319142624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 msgr2=0x7f579c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- 172.21.15.82:0/319142624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f57900097c0 tx=0x7f579001c350 comp rx=0 tx=0).stop 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- 172.21.15.82:0/319142624 shutdown_connections 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- 172.21.15.82:0/319142624 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c0b3f50 0x7f579c0cdef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- 172.21.15.82:0/319142624 >> v2:172.21.15.196:3300/0 conn(0x7f579c15a700 0x7f579c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- 172.21.15.82:0/319142624 >> 172.21.15.82:0/319142624 conn(0x7f579c0b11a0 msgr2=0x7f579c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- 172.21.15.82:0/319142624 shutdown_connections 2024-04-25T10:13:34.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- 172.21.15.82:0/319142624 wait complete. 2024-04-25T10:13:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 Processor -- start 2024-04-25T10:13:34.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- start start 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f579c0ce430 0x7f579c071260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c1624d0 0x7f579c162960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0cd490 con 0x7f579c15a700 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0cd310 con 0x7f579c0ce430 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f57a0e77640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c0b2bb0 con 0x7f579c1624d0 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f5799d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.318 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f5799d9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:44020/0 (socket says 172.21.15.82:44020) 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f5799d9b640 1 -- 172.21.15.82:0/1299527028 learned_addr learned my addr 172.21.15.82:0/1299527028 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.315+0000 7f579a59c640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f579c0ce430 0x7f579c071260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f579ad9d640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c1624d0 0x7f579c162960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 -- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c1624d0 msgr2=0x7f579c162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c1624d0 0x7f579c162960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 -- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f579c0ce430 msgr2=0x7f579c071260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f579c0ce430 0x7f579c071260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 -- 172.21.15.82:0/1299527028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c1411a0 con 0x7f579c15a700 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f5799d9b640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f57900ae040 tx=0x7f57900190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900b9020 con 0x7f579c15a700 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5790003df0 con 0x7f579c15a700 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c141430 con 0x7f579c15a700 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57900b0040 con 0x7f579c15a700 2024-04-25T10:13:34.319 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c1419f0 con 0x7f579c15a700 2024-04-25T10:13:34.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c04efa0 con 0x7f579c15a700 2024-04-25T10:13:34.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57900bc020 con 0x7f579c15a700 2024-04-25T10:13:34.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 0x7f577c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f579a59c640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 0x7f577c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f579011f100 con 0x7f579c15a700 2024-04-25T10:13:34.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.319+0000 7f579a59c640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 0x7f577c069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f578c009af0 tx=0x7f578c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.323+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f57900c3030 con 0x7f579c15a700 2024-04-25T10:13:34.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.487+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f579c158900 con 0x7f579c15a700 2024-04-25T10:13:34.488 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.487+0000 7f578affd640 1 -- 172.21.15.82:0/1299527028 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5790016090 con 0x7f579c15a700 2024-04-25T10:13:34.488 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:13:34.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 msgr2=0x7f577c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.490 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 0x7f577c069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f578c009af0 tx=0x7f578c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 msgr2=0x7f579c0717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f57900ae040 tx=0x7f57900190f0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 shutdown_connections 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f577c066f60 0x7f577c069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f579c1624d0 0x7f579c162960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f579c15a700 0x7f579c0717a0 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 --2- 172.21.15.82:0/1299527028 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f579c0ce430 0x7f579c071260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 >> 172.21.15.82:0/1299527028 conn(0x7f579c0b11a0 msgr2=0x7f579c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 shutdown_connections 2024-04-25T10:13:34.491 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.491+0000 7f57a0e77640 1 -- 172.21.15.82:0/1299527028 wait complete. 2024-04-25T10:13:34.506 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:13:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.506 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:34.517 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[54617]: fuse finished with error 0 and tester_r 0 2024-04-25T10:13:34.518 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:34.583 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:13:34.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.583 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:34.587 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:13:34.588 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 Processor -- start 2024-04-25T10:13:34.740 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 -- start start 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f915815b6b0 0x7f91580ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 0x7f91580cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.741 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580cbd20 con 0x7f915815f130 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f915815ba90 con 0x7f915815b6b0 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f915e95e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91580cc000 con 0x7f91580cf590 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.739+0000 7f9157fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f915815b6b0 0x7f91580ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915cefb640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f915815f130 0x7f915815f560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50168/0 (socket says 172.21.15.82:50168) 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915cefb640 1 -- 172.21.15.82:0/4137852180 learned_addr learned my addr 172.21.15.82:0/4137852180 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f91577fe640 1 --2- 172.21.15.82:0/4137852180 >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 0x7f91580cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9157fff640 1 -- 172.21.15.82:0/4137852180 >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 msgr2=0x7f91580cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9157fff640 1 --2- 172.21.15.82:0/4137852180 >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 0x7f91580cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9157fff640 1 -- 172.21.15.82:0/4137852180 >> v1:172.21.15.82:6789/0 conn(0x7f915815f130 legacy=0x7f915815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9157fff640 1 -- 172.21.15.82:0/4137852180 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580cc2e0 con 0x7f915815b6b0 2024-04-25T10:13:34.742 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f91577fe640 1 --2- 172.21.15.82:0/4137852180 >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 0x7f91580cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9157fff640 1 --2- 172.21.15.82:0/4137852180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f91580ca3c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f91580b4430 tx=0x7f914801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ca64529fe5f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9156ffd640 1 -- 172.21.15.82:0/4137852180 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480b0070 con 0x7f915815b6b0 2024-04-25T10:13:34.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9156ffd640 1 -- 172.21.15.82:0/4137852180 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91480ab600 con 0x7f915815b6b0 2024-04-25T10:13:34.744 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f9156ffd640 1 -- 172.21.15.82:0/4137852180 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480ab9d0 con 0x7f915815b6b0 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 -- 172.21.15.82:0/4137852180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 msgr2=0x7f91580ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 --2- 172.21.15.82:0/4137852180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f91580ca3c0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f91580b4430 tx=0x7f914801c480 comp rx=0 tx=0).stop 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 -- 172.21.15.82:0/4137852180 shutdown_connections 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 --2- 172.21.15.82:0/4137852180 >> v2:172.21.15.196:3300/0 conn(0x7f91580cf590 0x7f91580cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 --2- 172.21.15.82:0/4137852180 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f91580ca3c0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 -- 172.21.15.82:0/4137852180 >> 172.21.15.82:0/4137852180 conn(0x7f915806ae60 msgr2=0x7f915806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 -- 172.21.15.82:0/4137852180 shutdown_connections 2024-04-25T10:13:34.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.743+0000 7f915e95e640 1 -- 172.21.15.82:0/4137852180 wait complete. 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 Processor -- start 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- start start 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91580cf590 0x7f91580c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 0x7f915806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580cadb0 con 0x7f915815b6b0 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580ca900 con 0x7f915815f130 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580caf30 con 0x7f91580cf590 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36694/0 (socket says 172.21.15.82:36694) 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 -- 172.21.15.82:0/2952969051 learned_addr learned my addr 172.21.15.82:0/2952969051 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915cefb640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 0x7f915806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 -- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91580cf590 msgr2=0x7f91580c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f9157fff640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91580cf590 0x7f91580c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91580cf590 0x7f91580c3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 -- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 msgr2=0x7f915806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 0x7f915806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 -- 172.21.15.82:0/2952969051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580c41d0 con 0x7f915815b6b0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915cefb640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 0x7f915806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f91577fe640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c7c0 tx=0x7f914c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f9154ff9640 1 -- 172.21.15.82:0/2952969051 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c007a30 con 0x7f915815b6b0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f9154ff9640 1 -- 172.21.15.82:0/2952969051 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c007bd0 con 0x7f915815b6b0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f915815ae90 con 0x7f915815b6b0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f9154ff9640 1 -- 172.21.15.82:0/2952969051 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c0113f0 con 0x7f915815b6b0 2024-04-25T10:13:34.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580c31b0 con 0x7f915815b6b0 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.747+0000 7f9154ff9640 1 -- 172.21.15.82:0/2952969051 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f914c011590 con 0x7f915815b6b0 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9154ff9640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 0x7f9144069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9154ff9640 1 -- 172.21.15.82:0/2952969051 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f914c085460 con 0x7f915815b6b0 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 0x7f9144069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 0x7f9144069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f91480ad000 tx=0x7f9148008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 Processor -- start 2024-04-25T10:13:34.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- start start 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f9688b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9689480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f94d5ff0 con 0x5596f96d29d0 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f945e150 con 0x5596f96d32f0 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f94d2090 con 0x5596f96890a0 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36706/0 (socket says 172.21.15.82:36706) 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 -- 172.21.15.82:0/1170175635 learned_addr learned my addr 172.21.15.82:0/1170175635 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f91577fe640 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f9688b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 -- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 msgr2=0x5596f9689480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9689480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 -- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 msgr2=0x5596f9688b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f9688b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 -- 172.21.15.82:0/1170175635 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f966c0d0 con 0x5596f96d29d0 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f91577fe640 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f9688b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9157fff640 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f914801a0b0 tx=0x7f91480a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ddd1f50291c205 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f913dffb640 1 -- 172.21.15.82:0/1170175635 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480c7070 con 0x5596f96d29d0 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f913dffb640 1 -- 172.21.15.82:0/1170175635 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91480b1040 con 0x5596f96d29d0 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f913dffb640 1 -- 172.21.15.82:0/1170175635 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480b7850 con 0x5596f96d29d0 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- 172.21.15.82:0/1170175635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 msgr2=0x5596f96d2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f914801a0b0 tx=0x7f91480a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- 172.21.15.82:0/1170175635 shutdown_connections 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9689480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f9688b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 --2- 172.21.15.82:0/1170175635 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96d2db0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- 172.21.15.82:0/1170175635 >> 172.21.15.82:0/1170175635 conn(0x5596f9685020 msgr2=0x5596f9687440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- 172.21.15.82:0/1170175635 shutdown_connections 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- 172.21.15.82:0/1170175635 wait complete. 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.751+0000 7f9161acb000 1 -- start start 2024-04-25T10:13:34.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 Processor -- start 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d29d0 0x5596f96b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f949a250 con 0x5596f96890a0 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f9490690 con 0x5596f96d32f0 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f945ea40 con 0x5596f96d29d0 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 10 client.0 _ll_register_callbacks cb 0x5596f9673e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36722/0 (socket says 172.21.15.82:36722) 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 -- 172.21.15.82:0/3345840449 learned_addr learned my addr 172.21.15.82:0/3345840449 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f915cefb640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 -- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d29d0 msgr2=0x5596f96b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f91577fe640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d29d0 0x5596f96b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d29d0 0x5596f96b7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 -- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 msgr2=0x5596f96b8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96b8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 -- 172.21.15.82:0/3345840449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f95831c0 con 0x5596f96890a0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f915cefb640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9157fff640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f914801a0b0 tx=0x7f914801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2937cb057a7506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91480d1060 con 0x5596f96890a0 2024-04-25T10:13:34.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91480b7c50 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91480b1040 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91480d7050 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 10 client.9750 fetch_fsmap learned FSMap version 91 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5596f957f170 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91480b1040 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91480c1070 con 0x5596f96890a0 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 10 client.9750 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 10 client.9750 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 20 client.9750 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f96e2c80 0x5596f96e5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 4 client.9750 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 --> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f9471c60 con 0x5596f96e2c80 2024-04-25T10:13:34.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f91577fe640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f96e2c80 0x5596f96e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 10 client.9750 ms_handle_connect on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f91577fe640 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f96e2c80 0x5596f96e5070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a513679aa5a2b8ad server_cookie=8b2cae751832a71f in_seq=0 out_seq=0 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 1 -- 172.21.15.82:0/3345840449 <== mds.0 v2:172.21.15.82:6835/781412207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596f9471c60 con 0x5596f96e2c80 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 10 client.9750 handle_command_reply: tid=0 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 client.9750 shutdown 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 msgr2=0x5596f96b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f914801a0b0 tx=0x7f914801acc0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 shutdown_connections 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f96e2c80 0x5596f96e5070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d29d0 0x5596f96b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96b8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 --2- 172.21.15.82:0/3345840449 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96890a0 0x5596f96b77f0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 >> 172.21.15.82:0/3345840449 conn(0x5596f9685020 msgr2=0x5596f9687ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9136ffd640 0 client.9750 ms_handle_reset on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:34.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 shutdown_connections 2024-04-25T10:13:34.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 1 -- 172.21.15.82:0/3345840449 wait complete. 2024-04-25T10:13:34.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.755+0000 7f9161acb000 20 client.9750 trim_cache size 0 max 16384 2024-04-25T10:13:34.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 Processor -- start 2024-04-25T10:13:34.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- start start 2024-04-25T10:13:34.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9688840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96e0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91582014d0 con 0x5596f96d29d0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9158200690 con 0x5596f96d32f0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9158098780 con 0x5596f96890a0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36736/0 (socket says 172.21.15.82:36736) 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 -- 172.21.15.82:0/2225214489 learned_addr learned my addr 172.21.15.82:0/2225214489 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 -- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 msgr2=0x5596f9688840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9688840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 -- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 msgr2=0x5596f96e0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 -- 172.21.15.82:0/2225214489 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f951cde0 con 0x5596f96d29d0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f914c05dd20 tx=0x7f914c083630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57be4cf9be678ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f913d7fa640 1 -- 172.21.15.82:0/2225214489 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01e040 con 0x5596f96d29d0 2024-04-25T10:13:34.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f913d7fa640 1 -- 172.21.15.82:0/2225214489 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c05e250 con 0x5596f96d29d0 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f913d7fa640 1 -- 172.21.15.82:0/2225214489 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c053e40 con 0x5596f96d29d0 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- 172.21.15.82:0/2225214489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 msgr2=0x5596f96dfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f914c05dd20 tx=0x7f914c083630 comp rx=0 tx=0).stop 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- 172.21.15.82:0/2225214489 shutdown_connections 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96d32f0 0x5596f96e0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96dfe40 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- 172.21.15.82:0/2225214489 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96890a0 0x5596f9688840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- 172.21.15.82:0/2225214489 >> 172.21.15.82:0/2225214489 conn(0x5596f9685020 msgr2=0x5596f96df730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- 172.21.15.82:0/2225214489 shutdown_connections 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- 172.21.15.82:0/2225214489 wait complete. 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- start start 2024-04-25T10:13:34.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 Processor -- start 2024-04-25T10:13:34.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96890a0 0x5596f96e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d32f0 0x5596f96e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f9470e00 con 0x5596f96d29d0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9158200160 con 0x5596f96890a0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f94d5300 con 0x5596f96d32f0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f9161acb000 10 client.0 _ll_register_callbacks cb 0x5596f9654040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36750/0 (socket says 172.21.15.82:36750) 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.771+0000 7f91577fe640 1 -- 172.21.15.82:0/1769990530 learned_addr learned my addr 172.21.15.82:0/1769990530 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 -- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d32f0 msgr2=0x5596f96e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d32f0 0x5596f96e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 -- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96890a0 msgr2=0x5596f96e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96890a0 0x5596f96e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 -- 172.21.15.82:0/1769990530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f94778e0 con 0x5596f96d29d0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f91577fe640 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c05c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5910955b3eacca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914c01e030 con 0x5596f96d29d0 2024-04-25T10:13:34.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f914c0850b0 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f914c05e250 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) v2 ==== 24+0+0 (secure 0 0 0) 0x7f914c05abb0 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 10 client.9756 fetch_fsmap learned FSMap version 91 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) v3 -- 0x5596f952f8f0 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914c05e250 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 91) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f914c082030 con 0x5596f96d29d0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 10 client.9756 fetch_fsmap finished waiting for FSMap version 91 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 10 client.9756 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 20 client.9756 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f9737f40 0x5596f973a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 4 client.9756 mds_command: new command op to 9492 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 --> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596f94e3530 con 0x5596f9737f40 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9157fff640 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f9737f40 0x5596f973a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9136ffd640 10 client.9756 ms_handle_connect on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:34.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.775+0000 7f9157fff640 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f9737f40 0x5596f973a330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=763c54be0ccf7a72 server_cookie=4f6592ce6d7d733b in_seq=0 out_seq=0 2024-04-25T10:13:34.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9136ffd640 1 -- 172.21.15.82:0/1769990530 <== mds.0 v2:172.21.15.82:6835/781412207 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596f94e3530 con 0x5596f9737f40 2024-04-25T10:13:34.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9136ffd640 10 client.9756 handle_command_reply: tid=0 2024-04-25T10:13:34.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 client.9756 shutdown 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 msgr2=0x5596f96e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f914c006fd0 tx=0x7f914c05c040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 shutdown_connections 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:6835/781412207,v1:172.21.15.82:6837/781412207] conn(0x5596f9737f40 0x5596f973a330 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596f96d32f0 0x5596f96e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9136ffd640 0 client.9756 ms_handle_reset on v2:172.21.15.82:6835/781412207 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596f96d29d0 0x5596f96e4f70 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 --2- 172.21.15.82:0/1769990530 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596f96890a0 0x5596f96e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 >> 172.21.15.82:0/1769990530 conn(0x5596f9685020 msgr2=0x5596f96dffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 shutdown_connections 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.791+0000 7f9161acb000 1 -- 172.21.15.82:0/1769990530 wait complete. 2024-04-25T10:13:34.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f9161acb000 20 client.9756 trim_cache size 0 max 16384 2024-04-25T10:13:34.795 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:34.795 INFO:teuthology.orchestra.run.smithi082.stdout: "message": "", 2024-04-25T10:13:34.795 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0 2024-04-25T10:13:34.795 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 msgr2=0x7f9144069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 0x7f9144069420 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f91480ad000 tx=0x7f9148008040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 msgr2=0x7f915806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7f914c00c7c0 tx=0x7f914c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 shutdown_connections 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9144066f60 0x7f9144069420 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f915815f130 0x7f915806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f915815b6b0 0x7f915806e360 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 --2- 172.21.15.82:0/2952969051 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f91580cf590 0x7f91580c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 >> 172.21.15.82:0/2952969051 conn(0x7f915806ae60 msgr2=0x7f91580b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 shutdown_connections 2024-04-25T10:13:34.797 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.795+0000 7f915e95e640 1 -- 172.21.15.82:0/2952969051 wait complete. 2024-04-25T10:13:34.808 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:34.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 Processor -- start 2024-04-25T10:13:34.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 -- start start 2024-04-25T10:13:34.920 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f51a80ce510 0x7f51a80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f51a806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a80cee30 con 0x7f51a806ec60 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a80cc2b0 con 0x7f51a80ce510 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51ad3e9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806e290 con 0x7f51a8144ca0 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f51a806ec60 0x7f51a8142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50172/0 (socket says 172.21.15.82:50172) 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 -- 172.21.15.82:0/1085307227 learned_addr learned my addr 172.21.15.82:0/1085307227 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/1085307227 >> v2:172.21.15.196:3300/0 conn(0x7f51a80ce510 0x7f51a80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a67fc640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f51a806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594784100 0 0) 0x7f51a80cee30 con 0x7f51a806ec60 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5194003530 con 0x7f51a806ec60 2024-04-25T10:13:34.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688538019 0 0) 0x7f5194003530 con 0x7f51a806ec60 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 msgr2=0x7f51a806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f51a806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 >> v2:172.21.15.196:3300/0 conn(0x7f51a80ce510 msgr2=0x7f51a80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 --2- 172.21.15.82:0/1085307227 >> v2:172.21.15.196:3300/0 conn(0x7f51a80ce510 0x7f51a80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80ccc70 con 0x7f51a806ec60 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f51a0001a00 con 0x7f51a806ec60 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 >> v1:172.21.15.82:6789/0 conn(0x7f51a806ec60 legacy=0x7f51a8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f5194003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194003fb0 0x7f5194004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5194004d50 0x7f51940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80cc2b0 con 0x7f5194004d50 2024-04-25T10:13:34.922 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806e290 con 0x7f5194003fb0 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519400ba80 con 0x7f51a8144ca0 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5194004d50 0x7f51940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a67fc640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194003fb0 0x7f5194004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 -- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 msgr2=0x7f5194003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f5194003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f5194003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 -- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194003fb0 msgr2=0x7f5194004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194003fb0 0x7f5194004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a77fe640 1 -- 172.21.15.82:0/1085307227 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194003530 con 0x7f5194004d50 2024-04-25T10:13:34.923 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.919+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a8144ca0 0x7f5194003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5194004d50 0x7f51940051d0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f51a00013e0 tx=0x7f51a001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355f2fafa2eb5146 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a000a4e0 con 0x7f5194004d50 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00ad650 con 0x7f5194004d50 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a5ffb640 1 -- 172.21.15.82:0/1085307227 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00ada40 con 0x7f5194004d50 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- 172.21.15.82:0/1085307227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5194004d50 msgr2=0x7f51940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/1085307227 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5194004d50 0x7f51940051d0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f51a00013e0 tx=0x7f51a001d340 comp rx=0 tx=0).stop 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 -- 172.21.15.82:0/1085307227 reap_dead start 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- 172.21.15.82:0/1085307227 shutdown_connections 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- 172.21.15.82:0/1085307227 >> 172.21.15.82:0/1085307227 conn(0x7f51a80b11a0 msgr2=0x7f51a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- 172.21.15.82:0/1085307227 shutdown_connections 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- 172.21.15.82:0/1085307227 wait complete. 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 Processor -- start 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- start start 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194004d50 0x7f51a815adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a80ad820 0x7f51a80adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a80cc900 con 0x7f51a80c4480 2024-04-25T10:13:34.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806df20 con 0x7f5194004d50 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51ad3e9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a8142d90 con 0x7f51a80ad820 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36764/0 (socket says 172.21.15.82:36764) 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 -- 172.21.15.82:0/3015713727 learned_addr learned my addr 172.21.15.82:0/3015713727 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194004d50 0x7f51a815adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a67fc640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a80ad820 0x7f51a80adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 -- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a80ad820 msgr2=0x7f51a80adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a80ad820 0x7f51a80adca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 -- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194004d50 msgr2=0x7f51a815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194004d50 0x7f51a815adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 -- 172.21.15.82:0/3015713727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a815b2f0 con 0x7f51a80c4480 2024-04-25T10:13:34.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.923+0000 7f51a77fe640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f51a001cdb0 tx=0x7f5194005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0002050 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a000a4c0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80c51f0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00017c0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80c57b0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a804efa0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51a00b5070 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 0x7f5188069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 0x7f5188069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51a011efa0 con 0x7f51a80c4480 2024-04-25T10:13:34.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.927+0000 7f51a6ffd640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 0x7f5188069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5198009a60 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:34.931+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51a00bd060 con 0x7f51a80c4480 2024-04-25T10:13:35.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f51a80b1cc0 con 0x7f51a80c4480 2024-04-25T10:13:35.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f519f7fe640 1 -- 172.21.15.82:0/3015713727 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) v1 ==== 53+0+73 (secure 0 0 0) 0x7f51a0017050 con 0x7f51a80c4480 2024-04-25T10:13:35.110 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 msgr2=0x7f5188069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 0x7f5188069450 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f5198009a60 tx=0x7f5198008040 comp rx=0 tx=0).stop 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 msgr2=0x7f51a80c4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f51a001cdb0 tx=0x7f5194005840 comp rx=0 tx=0).stop 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 shutdown_connections 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5188066f90 0x7f5188069450 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.112 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f51a80ad820 0x7f51a80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f51a80c4480 0x7f51a80c4900 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 --2- 172.21.15.82:0/3015713727 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f5194004d50 0x7f51a815adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 >> 172.21.15.82:0/3015713727 conn(0x7f51a80b11a0 msgr2=0x7f51a80b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 shutdown_connections 2024-04-25T10:13:35.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:35.111+0000 7f51ad3e9640 1 -- 172.21.15.82:0/3015713727 wait complete. 2024-04-25T10:13:35.127 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:13:35.127 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:35.127 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:35.128 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:35.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.128 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:13:35.134 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:35.134 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:35.134 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.135 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41855sec preferred_lft 41855sec 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:35.136 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:35.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.137 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:35.191 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:13:35.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.193 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:13:35.246 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:35.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.247 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:13:35.247 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:13:35.248 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:35.248 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:35.248 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:35.248 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:13:35.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.319 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:13:35.365 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:35.366 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:35.366 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:35.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:13:35.422 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:13:35.422 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:35.422 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:35.470 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:35.470 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:35.470 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:35.518 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:35.578 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:35.626 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:35.627 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:35.628 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:35.629 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:35.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.631 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:35.681 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:13:35.681 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:13:35.727 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:35.741 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:35.794 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:35.795 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:35.796 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:35.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.797 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:35.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:13:35.803+0000 7fdad0c71580 -1 init, newargv = 0x5601bb3e9180 newargc=15 2024-04-25T10:13:35.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55061]: starting ceph client 2024-04-25T10:13:35.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55061]: starting fuse 2024-04-25T10:13:36.909 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:36.926 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:36.974 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:36.975 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:36.976 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:36.977 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:36.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:36.978 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:37.025 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:13:37.026 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.026 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:13:37.027 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:13:37.028 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:13:37.028 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:13:37.028 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.028 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:13:37.028 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:37.151 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.55061.asok 2024-04-25T10:13:37.155 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:37.163 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55061.asok 2024-04-25T10:13:37.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.163 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55061.asok status 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "55061", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9768, 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 9768 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:37.313 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2006767943 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 2006767943 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.9768 192.168.0.1:0/2006767943", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/2006767943", 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:13:37.314 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 91, 2024-04-25T10:13:37.315 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 121, 2024-04-25T10:13:37.315 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:13:37.315 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:13:37.315 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:13:37.315 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:37.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.326 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:13:37.326 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:37.332 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:13:37.332 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:37.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.332 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:37.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.383 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T10:13:37.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:37.437 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T10:13:37.531 INFO:teuthology.orchestra.run.smithi082.stderr:8+0 records in 2024-04-25T10:13:37.532 INFO:teuthology.orchestra.run.smithi082.stderr:8+0 records out 2024-04-25T10:13:37.532 INFO:teuthology.orchestra.run.smithi082.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0440928 s, 190 MB/s 2024-04-25T10:13:37.533 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:37.534 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:37.534 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:37.534 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.534 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:13:37.534 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:37.600 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627779 2024-04-25T10:13:37.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:13:37.610 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:37.717 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627780 2024-04-25T10:13:37.720 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:37.727 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:37.727 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:37.775 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 Processor -- start 2024-04-25T10:13:37.950 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 -- start start 2024-04-25T10:13:37.951 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20d815b950 0x7f20d809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cf420 0x7f20d80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f20d80cfb80 0x7f20d80d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d815bd30 con 0x7f20d815b950 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d80cc0d0 con 0x7f20d80cfb80 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20e075d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d809ab10 con 0x7f20d80cf420 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20d815b950 0x7f20d809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f20d815b950 0x7f20d809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36780/0 (socket says 172.21.15.82:36780) 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 -- 172.21.15.82:0/1066171460 learned_addr learned my addr 172.21.15.82:0/1066171460 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20decfa640 1 --2- 172.21.15.82:0/1066171460 >> v2:172.21.15.196:3300/0 conn(0x7f20d80cfb80 0x7f20d80d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20ddcf8640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cf420 0x7f20d80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 -- 172.21.15.82:0/1066171460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cf420 msgr2=0x7f20d80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cf420 0x7f20d80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.952 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 -- 172.21.15.82:0/1066171460 >> v2:172.21.15.196:3300/0 conn(0x7f20d80cfb80 msgr2=0x7f20d80d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 --2- 172.21.15.82:0/1066171460 >> v2:172.21.15.196:3300/0 conn(0x7f20d80cfb80 0x7f20d80d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.953 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.951+0000 7f20de4f9640 1 -- 172.21.15.82:0/1066171460 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80cc370 con 0x7f20d815b950 2024-04-25T10:13:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20de4f9640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d809a480 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f20d80b3bd0 tx=0x7f20c801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1321e8b6e1e33f2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20dd4f7640 1 -- 172.21.15.82:0/1066171460 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80a5760 con 0x7f20d815b950 2024-04-25T10:13:37.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20dd4f7640 1 -- 172.21.15.82:0/1066171460 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c80a5900 con 0x7f20d815b950 2024-04-25T10:13:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20dd4f7640 1 -- 172.21.15.82:0/1066171460 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80a5cd0 con 0x7f20d815b950 2024-04-25T10:13:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 -- 172.21.15.82:0/1066171460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 msgr2=0x7f20d809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d809a480 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f20d80b3bd0 tx=0x7f20c801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 -- 172.21.15.82:0/1066171460 shutdown_connections 2024-04-25T10:13:37.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 --2- 172.21.15.82:0/1066171460 >> v2:172.21.15.196:3300/0 conn(0x7f20d80cfb80 0x7f20d80d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cf420 0x7f20d80cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 --2- 172.21.15.82:0/1066171460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d809a480 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 -- 172.21.15.82:0/1066171460 >> 172.21.15.82:0/1066171460 conn(0x7f20d80b30b0 msgr2=0x7f20d80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 -- 172.21.15.82:0/1066171460 shutdown_connections 2024-04-25T10:13:37.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.955+0000 7f20e075d640 1 -- 172.21.15.82:0/1066171460 wait complete. 2024-04-25T10:13:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 Processor -- start 2024-04-25T10:13:37.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- start start 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 0x7f20d8157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cfb80 0x7f20d81f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80cd390 con 0x7f20d815b950 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d806faa0 con 0x7f20d80cf420 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d80b25a0 con 0x7f20d80cfb80 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36788/0 (socket says 172.21.15.82:36788) 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 -- 172.21.15.82:0/2227452372 learned_addr learned my addr 172.21.15.82:0/2227452372 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20de4f9640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 0x7f20d8157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20ddcf8640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cfb80 0x7f20d81f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 -- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cfb80 msgr2=0x7f20d81f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cfb80 0x7f20d81f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 -- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 msgr2=0x7f20d8157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 0x7f20d8157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 -- 172.21.15.82:0/2227452372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d80b19d0 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20de4f9640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 0x7f20d8157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20decfa640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f20d000e960 tx=0x7f20d000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0017070 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d000c9a0 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d80b1c70 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d000cd70 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.959+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d80b2230 con 0x7f20d815b950 2024-04-25T10:13:37.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d804efa0 con 0x7f20d815b950 2024-04-25T10:13:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20d001a070 con 0x7f20d815b950 2024-04-25T10:13:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20d6ffd640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 0x7f20c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20d0085140 con 0x7f20d815b950 2024-04-25T10:13:37.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20de4f9640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 0x7f20c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.963+0000 7f20de4f9640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 0x7f20c4069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f20c80ae000 tx=0x7f20c80a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:37.967+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20d0055fc0 con 0x7f20d815b950 2024-04-25T10:13:38.127 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.127+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20d80d13a0 con 0x7f20d815b950 2024-04-25T10:13:38.129 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20d6ffd640 1 -- 172.21.15.82:0/2227452372 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f20d00558c0 con 0x7f20d815b950 2024-04-25T10:13:38.129 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:38.129 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":121,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:23.113792+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 msgr2=0x7f20c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 0x7f20c4069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f20c80ae000 tx=0x7f20c80a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 msgr2=0x7f20d80b0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f20d000e960 tx=0x7f20d000ee20 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 shutdown_connections 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f20c4066f60 0x7f20c4069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f20d80cfb80 0x7f20d81f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f20d815b950 0x7f20d80b0ea0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 --2- 172.21.15.82:0/2227452372 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f20d80cf420 0x7f20d8157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 >> 172.21.15.82:0/2227452372 conn(0x7f20d80b30b0 msgr2=0x7f20d815a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 shutdown_connections 2024-04-25T10:13:38.132 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.131+0000 7f20e075d640 1 -- 172.21.15.82:0/2227452372 wait complete. 2024-04-25T10:13:38.149 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:38.149 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:38.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 Processor -- start 2024-04-25T10:13:38.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 -- start start 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e380cf590 0x7f1e3815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1e380cf970 0x7f1e380c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380ca4c0 con 0x7f1e380c9ef0 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380ca780 con 0x7f1e380cf590 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e3db3d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380cd440 con 0x7f1e380cf970 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e380cf590 0x7f1e3815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f1e380cf590 0x7f1e3815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36800/0 (socket says 172.21.15.82:36800) 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 -- 172.21.15.82:0/2617388486 learned_addr learned my addr 172.21.15.82:0/2617388486 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e2ffff640 1 --2- 172.21.15.82:0/2617388486 >> v2:172.21.15.196:3300/0 conn(0x7f1e380cf970 0x7f1e380c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 -- 172.21.15.82:0/2617388486 >> v2:172.21.15.196:3300/0 conn(0x7f1e380cf970 msgr2=0x7f1e380c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 --2- 172.21.15.82:0/2617388486 >> v2:172.21.15.196:3300/0 conn(0x7f1e380cf970 0x7f1e380c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 -- 172.21.15.82:0/2617388486 >> v1:172.21.15.82:6789/0 conn(0x7f1e380c9ef0 legacy=0x7f1e380b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e36ffd640 1 -- 172.21.15.82:0/2617388486 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368697723 0 0) 0x7f1e380ca4c0 con 0x7f1e380c9ef0 2024-04-25T10:13:38.276 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.275+0000 7f1e377fe640 1 -- 172.21.15.82:0/2617388486 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380cd690 con 0x7f1e380cf590 2024-04-25T10:13:38.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e377fe640 1 --2- 172.21.15.82:0/2617388486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e3815ecb0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f1e380d00e0 tx=0x7f1e2801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c17a253571fc87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e36ffd640 1 -- 172.21.15.82:0/2617388486 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280b0070 con 0x7f1e380cf590 2024-04-25T10:13:38.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e36ffd640 1 -- 172.21.15.82:0/2617388486 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28004230 con 0x7f1e380cf590 2024-04-25T10:13:38.277 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e36ffd640 1 -- 172.21.15.82:0/2617388486 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003620 con 0x7f1e380cf590 2024-04-25T10:13:38.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- 172.21.15.82:0/2617388486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 msgr2=0x7f1e3815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/2617388486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e3815ecb0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f1e380d00e0 tx=0x7f1e2801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:38.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- 172.21.15.82:0/2617388486 shutdown_connections 2024-04-25T10:13:38.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/2617388486 >> v2:172.21.15.196:3300/0 conn(0x7f1e380cf970 0x7f1e380c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.278 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/2617388486 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e3815ecb0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- 172.21.15.82:0/2617388486 >> 172.21.15.82:0/2617388486 conn(0x7f1e3806ae60 msgr2=0x7f1e3806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- 172.21.15.82:0/2617388486 shutdown_connections 2024-04-25T10:13:38.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- 172.21.15.82:0/2617388486 wait complete. 2024-04-25T10:13:38.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 Processor -- start 2024-04-25T10:13:38.279 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- start start 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e380c9ef0 0x7f1e38098080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e380cf970 0x7f1e380729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e38167660 con 0x7f1e380cf590 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381674e0 con 0x7f1e380cf970 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e3db3d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e381677e0 con 0x7f1e380c9ef0 2024-04-25T10:13:38.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36810/0 (socket says 172.21.15.82:36810) 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 -- 172.21.15.82:0/4261387017 learned_addr learned my addr 172.21.15.82:0/4261387017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e377fe640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e380c9ef0 0x7f1e38098080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e37fff640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e380cf970 0x7f1e380729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 -- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e380c9ef0 msgr2=0x7f1e38098080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e380c9ef0 0x7f1e38098080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 -- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e380cf970 msgr2=0x7f1e380729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e380cf970 0x7f1e380729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 -- 172.21.15.82:0/4261387017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380985c0 con 0x7f1e380cf590 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e2ffff640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000fa40 tx=0x7f1e2000ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20007c90 con 0x7f1e380cf590 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.279+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e20007e30 con 0x7f1e380cf590 2024-04-25T10:13:38.282 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3806e4e0 con 0x7f1e380cf590 2024-04-25T10:13:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e20011650 con 0x7f1e380cf590 2024-04-25T10:13:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3806eaa0 con 0x7f1e380cf590 2024-04-25T10:13:38.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3804efa0 con 0x7f1e380cf590 2024-04-25T10:13:38.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e20019020 con 0x7f1e380cf590 2024-04-25T10:13:38.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e34ff9640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 0x7f1e18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e200873c0 con 0x7f1e380cf590 2024-04-25T10:13:38.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e377fe640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 0x7f1e18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.283+0000 7f1e377fe640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 0x7f1e18069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ad000 tx=0x7f1e28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.287+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1e20058240 con 0x7f1e380cf590 2024-04-25T10:13:38.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.443+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e380b4170 con 0x7f1e380cf590 2024-04-25T10:13:38.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.443+0000 7f1e34ff9640 1 -- 172.21.15.82:0/4261387017 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e20057b40 con 0x7f1e380cf590 2024-04-25T10:13:38.445 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 msgr2=0x7f1e18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 0x7f1e18069420 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1e280ad000 tx=0x7f1e28008040 comp rx=0 tx=0).stop 2024-04-25T10:13:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 msgr2=0x7f1e380724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.447 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f1e2000fa40 tx=0x7f1e2000ff00 comp rx=0 tx=0).stop 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 shutdown_connections 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1e18066f60 0x7f1e18069420 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1e380cf970 0x7f1e380729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1e380cf590 0x7f1e380724b0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 --2- 172.21.15.82:0/4261387017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1e380c9ef0 0x7f1e38098080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 >> 172.21.15.82:0/4261387017 conn(0x7f1e3806ae60 msgr2=0x7f1e380af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 shutdown_connections 2024-04-25T10:13:38.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.447+0000 7f1e3db3d640 1 -- 172.21.15.82:0/4261387017 wait complete. 2024-04-25T10:13:38.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:38.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.591+0000 7f78627c8640 1 Processor -- start 2024-04-25T10:13:38.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.591+0000 7f78627c8640 1 -- start start 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f78627c8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f785c15b6b0 0x7f785c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f78627c8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c0cf590 0x7f785c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f78627c8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785c0cbd20 con 0x7f785c15f130 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f78627c8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785c15ba90 con 0x7f785c15b6b0 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f78627c8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0cff30 con 0x7f785c0cf590 2024-04-25T10:13:38.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f785c15b6b0 0x7f785c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f7860d65640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f785c15f130 0x7f785c15f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50186/0 (socket says 172.21.15.82:50186) 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f785c15b6b0 0x7f785c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36820/0 (socket says 172.21.15.82:36820) 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f7860d65640 1 -- 172.21.15.82:0/1944858177 learned_addr learned my addr 172.21.15.82:0/1944858177 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 -- 172.21.15.82:0/1944858177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c0cf590 msgr2=0x7f785c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 --2- 172.21.15.82:0/1944858177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c0cf590 0x7f785c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 -- 172.21.15.82:0/1944858177 >> v1:172.21.15.82:6789/0 conn(0x7f785c15f130 legacy=0x7f785c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 -- 172.21.15.82:0/1944858177 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c0cc000 con 0x7f785c15b6b0 2024-04-25T10:13:38.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785affd640 1 -- 172.21.15.82:0/1944858177 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446518330 0 0) 0x7f785c0cbd20 con 0x7f785c15f130 2024-04-25T10:13:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785bfff640 1 --2- 172.21.15.82:0/1944858177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c0ca3c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d90 tx=0x7f785401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6836abbc99b0cd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785affd640 1 -- 172.21.15.82:0/1944858177 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540b2070 con 0x7f785c15b6b0 2024-04-25T10:13:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785affd640 1 -- 172.21.15.82:0/1944858177 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78540a5a00 con 0x7f785c15b6b0 2024-04-25T10:13:38.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.595+0000 7f785affd640 1 -- 172.21.15.82:0/1944858177 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78540a5dd0 con 0x7f785c15b6b0 2024-04-25T10:13:38.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1944858177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 msgr2=0x7f785c0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- 172.21.15.82:0/1944858177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c0ca3c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f7854009d90 tx=0x7f785401c940 comp rx=0 tx=0).stop 2024-04-25T10:13:38.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1944858177 shutdown_connections 2024-04-25T10:13:38.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- 172.21.15.82:0/1944858177 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c0cf590 0x7f785c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- 172.21.15.82:0/1944858177 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c0ca3c0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1944858177 >> 172.21.15.82:0/1944858177 conn(0x7f785c06ae60 msgr2=0x7f785c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1944858177 shutdown_connections 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1944858177 wait complete. 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 Processor -- start 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- start start 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f785c0cf590 0x7f785c1542c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 0x7f785c097c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0ca900 con 0x7f785c15b6b0 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0cba30 con 0x7f785c0cf590 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785c0cadb0 con 0x7f785c15f130 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36832/0 (socket says 172.21.15.82:36832) 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 -- 172.21.15.82:0/1407207513 learned_addr learned my addr 172.21.15.82:0/1407207513 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f7860d65640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 0x7f785c097c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785bfff640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f785c0cf590 0x7f785c1542c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 -- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 msgr2=0x7f785c097c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 0x7f785c097c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 -- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f785c0cf590 msgr2=0x7f785c1542c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f785c0cf590 0x7f785c1542c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 -- 172.21.15.82:0/1407207513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c154800 con 0x7f785c15b6b0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f7860d65640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 0x7f785c097c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f785b7fe640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f784400b580 tx=0x7f784400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844014070 con 0x7f785c15b6b0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844010890 con 0x7f785c15b6b0 2024-04-25T10:13:38.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c15d670 con 0x7f785c15b6b0 2024-04-25T10:13:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844010c80 con 0x7f785c15b6b0 2024-04-25T10:13:38.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.599+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c0c34d0 con 0x7f785c15b6b0 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c04efa0 con 0x7f785c15b6b0 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7844017070 con 0x7f785c15b6b0 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f7858ff9640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 0x7f783c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7844084f60 con 0x7f785c15b6b0 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f785bfff640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 0x7f783c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.604 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.603+0000 7f785bfff640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 0x7f783c069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f78540af000 tx=0x7f78540a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.607+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7844055df0 con 0x7f785c15b6b0 2024-04-25T10:13:38.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.759+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f785c166070 con 0x7f785c15b6b0 2024-04-25T10:13:38.761 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.759+0000 7f7858ff9640 1 -- 172.21.15.82:0/1407207513 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7844010e20 con 0x7f785c15b6b0 2024-04-25T10:13:38.761 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 msgr2=0x7f783c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 0x7f783c069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f78540af000 tx=0x7f78540a7070 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 msgr2=0x7f785c097730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f784400b580 tx=0x7f784400ba40 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 shutdown_connections 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f783c066f60 0x7f783c069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f785c15f130 0x7f785c097c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f785c15b6b0 0x7f785c097730 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 --2- 172.21.15.82:0/1407207513 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f785c0cf590 0x7f785c1542c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.763 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 >> 172.21.15.82:0/1407207513 conn(0x7f785c06ae60 msgr2=0x7f785c0af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 shutdown_connections 2024-04-25T10:13:38.764 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.763+0000 7f78627c8640 1 -- 172.21.15.82:0/1407207513 wait complete. 2024-04-25T10:13:38.778 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:38.843 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:38.846 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 Processor -- start 2024-04-25T10:13:38.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 -- start start 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc5580cf590 0x7fc5580cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5580cbd20 con 0x7fc55815f130 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55815ba90 con 0x7fc5580cf590 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc55e00e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580cff30 con 0x7fc55815b6b0 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:47456/0 (socket says 172.21.15.82:47456) 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 -- 172.21.15.82:0/2901047260 learned_addr learned my addr 172.21.15.82:0/2901047260 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc54effd640 1 --2- 172.21.15.82:0/2901047260 >> v2:172.21.15.196:3300/0 conn(0x7fc5580cf590 0x7fc5580cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 -- 172.21.15.82:0/2901047260 >> v2:172.21.15.196:3300/0 conn(0x7fc5580cf590 msgr2=0x7fc5580cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 --2- 172.21.15.82:0/2901047260 >> v2:172.21.15.196:3300/0 conn(0x7fc5580cf590 0x7fc5580cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 -- 172.21.15.82:0/2901047260 >> v1:172.21.15.196:6789/0 conn(0x7fc55815f130 legacy=0x7fc55815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.891+0000 7fc5577fe640 1 -- 172.21.15.82:0/2901047260 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580cc000 con 0x7fc55815b6b0 2024-04-25T10:13:38.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- 172.21.15.82:0/2901047260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc54801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf14c548dd1b96d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc556ffd640 1 -- 172.21.15.82:0/2901047260 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480a95f0 con 0x7fc55815b6b0 2024-04-25T10:13:38.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc556ffd640 1 -- 172.21.15.82:0/2901047260 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5480a9790 con 0x7fc55815b6b0 2024-04-25T10:13:38.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc556ffd640 1 -- 172.21.15.82:0/2901047260 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480a9b60 con 0x7fc55815b6b0 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/2901047260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 msgr2=0x7fc5580ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- 172.21.15.82:0/2901047260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc548009870 tx=0x7fc54801c530 comp rx=0 tx=0).stop 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/2901047260 shutdown_connections 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- 172.21.15.82:0/2901047260 >> v2:172.21.15.196:3300/0 conn(0x7fc5580cf590 0x7fc5580cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- 172.21.15.82:0/2901047260 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815b6b0 0x7fc5580ca3c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/2901047260 >> 172.21.15.82:0/2901047260 conn(0x7fc55806ae60 msgr2=0x7fc55806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/2901047260 shutdown_connections 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/2901047260 wait complete. 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 Processor -- start 2024-04-25T10:13:38.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- start start 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc55815b6b0 0x7fc5580c1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 0x7fc5580c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580ca900 con 0x7fc5580cf590 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580cba30 con 0x7fc55815b6b0 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580cadb0 con 0x7fc55815f130 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36848/0 (socket says 172.21.15.82:36848) 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 -- 172.21.15.82:0/609154760 learned_addr learned my addr 172.21.15.82:0/609154760 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc54effd640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc55815b6b0 0x7fc5580c1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc557fff640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 0x7fc5580c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 -- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 msgr2=0x7fc5580c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 0x7fc5580c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 -- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc55815b6b0 msgr2=0x7fc5580c1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc55815b6b0 0x7fc5580c1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 -- 172.21.15.82:0/609154760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580b0b80 con 0x7fc5580cf590 2024-04-25T10:13:38.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc557fff640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 0x7fc5580c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc5577fe640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc54801a160 tx=0x7fc54801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480a95f0 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5480b1820 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5580989e0 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5480b1c10 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.895+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5580c2240 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5580afb80 con 0x7fc5580cf590 2024-04-25T10:13:38.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5480bc450 con 0x7fc5580cf590 2024-04-25T10:13:38.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc554ff9640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 0x7fc538069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc54811fbb0 con 0x7fc5580cf590 2024-04-25T10:13:38.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc54effd640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 0x7fc538069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.899+0000 7fc54effd640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 0x7fc538069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc5581f7d70 tx=0x7fc540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.902 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:38.903+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5480ad020 con 0x7fc5580cf590 2024-04-25T10:13:39.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.047+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc55804efa0 con 0x7fc5580cf590 2024-04-25T10:13:39.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.047+0000 7fc554ff9640 1 -- 172.21.15.82:0/609154760 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc548123070 con 0x7fc5580cf590 2024-04-25T10:13:39.050 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 msgr2=0x7fc538069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 0x7fc538069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc5581f7d70 tx=0x7fc540008040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 msgr2=0x7fc5581547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7fc54801a160 tx=0x7fc54801ad70 comp rx=0 tx=0).stop 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 shutdown_connections 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc538066f60 0x7fc538069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc55815f130 0x7fc5580c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc55815b6b0 0x7fc5580c1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 --2- 172.21.15.82:0/609154760 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5580cf590 0x7fc5581547d0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 >> 172.21.15.82:0/609154760 conn(0x7fc55806ae60 msgr2=0x7fc5581540c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 shutdown_connections 2024-04-25T10:13:39.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.051+0000 7fc55e00e640 1 -- 172.21.15.82:0/609154760 wait complete. 2024-04-25T10:13:39.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:39.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 Processor -- start 2024-04-25T10:13:39.230 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 -- start start 2024-04-25T10:13:39.232 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 0x7f1b8406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b840cc130 0x7f1b840cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8406b5b0 con 0x7f1b8415ca90 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b8415ff70 con 0x7f1b840cc130 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b89a14640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8415f360 con 0x7f1b8406ac90 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 0x7f1b8406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b837fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f1b840cc130 0x7f1b840cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b827fc640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f1b8415ca90 0x7f1b840d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:57864/0 (socket says 172.21.15.82:57864) 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b82ffd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 0x7f1b8406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:47472/0 (socket says 172.21.15.82:47472) 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b827fc640 1 -- 172.21.15.82:0/701738713 learned_addr learned my addr 172.21.15.82:0/701738713 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b837fe640 1 -- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 msgr2=0x7f1b8406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b837fe640 1 --2- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 0x7f1b8406b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b837fe640 1 -- 172.21.15.82:0/701738713 >> v1:172.21.15.196:6789/0 conn(0x7f1b8415ca90 legacy=0x7f1b840d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.268 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.231+0000 7f1b837fe640 1 -- 172.21.15.82:0/701738713 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8415f640 con 0x7f1b840cc130 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b837fe640 1 --2- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b840cc130 0x7f1b840cf2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b740097c0 tx=0x7f1b7401c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39f0c6aef545ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b81ffb640 1 -- 172.21.15.82:0/701738713 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740a7070 con 0x7f1b840cc130 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b81ffb640 1 -- 172.21.15.82:0/701738713 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b740b0070 con 0x7f1b840cc130 2024-04-25T10:13:39.272 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b81ffb640 1 -- 172.21.15.82:0/701738713 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b740ab890 con 0x7f1b840cc130 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 -- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b840cc130 msgr2=0x7f1b840cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 --2- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b840cc130 0x7f1b840cf2f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1b740097c0 tx=0x7f1b7401c470 comp rx=0 tx=0).stop 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 -- 172.21.15.82:0/701738713 shutdown_connections 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 --2- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b840cc130 0x7f1b840cf2f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 --2- 172.21.15.82:0/701738713 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8406ac90 0x7f1b8406b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 -- 172.21.15.82:0/701738713 >> 172.21.15.82:0/701738713 conn(0x7f1b84099a70 msgr2=0x7f1b840d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 -- 172.21.15.82:0/701738713 shutdown_connections 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.235+0000 7f1b89a14640 1 -- 172.21.15.82:0/701738713 wait complete. 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 Processor -- start 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- start start 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8406ac90 0x7f1b84148980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.273 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8415ca90 0x7f1b84159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84159750 con 0x7f1b840cc130 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b841595d0 con 0x7f1b8406ac90 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b840d0040 con 0x7f1b8415ca90 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36850/0 (socket says 172.21.15.82:36850) 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 -- 172.21.15.82:0/1620530772 learned_addr learned my addr 172.21.15.82:0/1620530772 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b82ffd640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8406ac90 0x7f1b84148980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b837fe640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8415ca90 0x7f1b84159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 -- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8415ca90 msgr2=0x7f1b84159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8415ca90 0x7f1b84159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 -- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8406ac90 msgr2=0x7f1b84148980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.274 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8406ac90 0x7f1b84148980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 -- 172.21.15.82:0/1620530772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840c5cf0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b827fc640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c3a0 tx=0x7f1b7800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78017070 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78005cf0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b84149420 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b780072f0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b84148ec0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8406b7c0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.239+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b7801a070 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.243+0000 7f1b6bfff640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 0x7f1b70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.243+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b780844d0 con 0x7f1b840cc130 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.243+0000 7f1b82ffd640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 0x7f1b70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.243+0000 7f1b82ffd640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 0x7f1b70069450 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1b84159250 tx=0x7f1b7c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.275 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.247+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b780553e0 con 0x7f1b840cc130 2024-04-25T10:13:39.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b840b6480 con 0x7f1b840cc130 2024-04-25T10:13:39.414 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b6bfff640 1 -- 172.21.15.82:0/1620530772 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1b78054ce0 con 0x7f1b840cc130 2024-04-25T10:13:39.415 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 msgr2=0x7f1b70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 0x7f1b70069450 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1b84159250 tx=0x7f1b7c00b040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 msgr2=0x7f1b841598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.415+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800c3a0 tx=0x7f1b7800c860 comp rx=0 tx=0).stop 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 shutdown_connections 2024-04-25T10:13:39.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f1b70066f90 0x7f1b70069450 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f1b8415ca90 0x7f1b84159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f1b840cc130 0x7f1b841598f0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 --2- 172.21.15.82:0/1620530772 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f1b8406ac90 0x7f1b84148980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 >> 172.21.15.82:0/1620530772 conn(0x7f1b84099a70 msgr2=0x7f1b8415dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 shutdown_connections 2024-04-25T10:13:39.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.419+0000 7f1b89a14640 1 -- 172.21.15.82:0/1620530772 wait complete. 2024-04-25T10:13:39.432 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:39.432 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:39.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:39.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 Processor -- start 2024-04-25T10:13:39.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 -- start start 2024-04-25T10:13:39.592 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 0x7f8134166620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813406ec30 con 0x7f8134066e30 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813406f230 con 0x7f813406bca0 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f81390b4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81340669a0 con 0x7f81340af370 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8134066e30 0x7f813406c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50200/0 (socket says 172.21.15.82:50200) 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 -- 172.21.15.82:0/3209561943 learned_addr learned my addr 172.21.15.82:0/3209561943 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813359e640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 0x7f8134166620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064255678 0 0) 0x7f813406ec30 con 0x7f8134066e30 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f811c003530 con 0x7f8134066e30 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668362483 0 0) 0x7f811c003530 con 0x7f8134066e30 2024-04-25T10:13:39.593 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 msgr2=0x7f8134166620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 0x7f8134166620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 >> v1:172.21.15.196:6789/0 conn(0x7f813406bca0 legacy=0x7f81340cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813406f510 con 0x7f8134066e30 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8128001680 con 0x7f8134066e30 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 >> v1:172.21.15.82:6789/0 conn(0x7f8134066e30 legacy=0x7f813406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 0x7f811c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f811c003fc0 0x7f811c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c004d50 0x7f811c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81340669a0 con 0x7f811c003fc0 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811c00bc10 con 0x7f811c004d50 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811c00be10 con 0x7f81340af370 2024-04-25T10:13:39.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f811c003fc0 0x7f811c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 -- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 msgr2=0x7f811c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f81340af370 0x7f811c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813359e640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c004d50 0x7f811c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 -- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c004d50 msgr2=0x7f811c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c004d50 0x7f811c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.591+0000 7f813259c640 1 -- 172.21.15.82:0/3209561943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c003530 con 0x7f811c003fc0 2024-04-25T10:13:39.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f813259c640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f811c003fc0 0x7f811c004810 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f812800c240 tx=0x7f812801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0aea6f2f2151d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128007d20 con 0x7f811c003fc0 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81280a8020 con 0x7f811c003fc0 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f8131d9b640 1 -- 172.21.15.82:0/3209561943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280058e0 con 0x7f811c003fc0 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- 172.21.15.82:0/3209561943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f811c003fc0 msgr2=0x7f811c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 --2- 172.21.15.82:0/3209561943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f811c003fc0 0x7f811c004810 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f812800c240 tx=0x7f812801c940 comp rx=0 tx=0).stop 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f813359e640 1 -- 172.21.15.82:0/3209561943 reap_dead start 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- 172.21.15.82:0/3209561943 shutdown_connections 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- 172.21.15.82:0/3209561943 >> 172.21.15.82:0/3209561943 conn(0x7f8134071810 msgr2=0x7f8134071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- 172.21.15.82:0/3209561943 shutdown_connections 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- 172.21.15.82:0/3209561943 wait complete. 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 Processor -- start 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- start start 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c003fc0 0x7f81341521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 0x7f813414d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81341679e0 con 0x7f813414c5e0 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8134167860 con 0x7f811c003fc0 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.595+0000 7f81390b4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8134167b60 con 0x7f813413a560 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36874/0 (socket says 172.21.15.82:36874) 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 -- 172.21.15.82:0/3150266954 learned_addr learned my addr 172.21.15.82:0/3150266954 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f8132d9d640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c003fc0 0x7f81341521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813359e640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 0x7f813414d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 -- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 msgr2=0x7f813414d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 0x7f813414d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 -- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c003fc0 msgr2=0x7f81341521b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c003fc0 0x7f81341521b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 -- 172.21.15.82:0/3150266954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813413ad90 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813359e640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 0x7f813414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f813259c640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8128007040 tx=0x7f811c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280049f0 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8128004b90 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813413b070 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280070d0 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813414f770 con 0x7f813414c5e0 2024-04-25T10:13:39.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813404efa0 con 0x7f813414c5e0 2024-04-25T10:13:39.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8128007270 con 0x7f813414c5e0 2024-04-25T10:13:39.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 0x7f8120069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f812811e0a0 con 0x7f813414c5e0 2024-04-25T10:13:39.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f8132d9d640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 0x7f8120069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.599+0000 7f8132d9d640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 0x7f8120069450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f81240017d0 tx=0x7f8124009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.603+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f81280bd060 con 0x7f813414c5e0 2024-04-25T10:13:39.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.751+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8134072400 con 0x7f813414c5e0 2024-04-25T10:13:39.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f811b7fe640 1 -- 172.21.15.82:0/3150266954 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16964 (secure 0 0 0) 0x7f8128016970 con 0x7f813414c5e0 2024-04-25T10:13:39.754 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:39.754 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":121,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:23.113792+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:39.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 msgr2=0x7f8120069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 0x7f8120069450 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f81240017d0 tx=0x7f8124009040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 msgr2=0x7f81341384d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8128007040 tx=0x7f811c005840 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 shutdown_connections 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8120066f90 0x7f8120069450 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f813413a560 0x7f813414d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f813414c5e0 0x7f81341384d0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 --2- 172.21.15.82:0/3150266954 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f811c003fc0 0x7f81341521b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 >> 172.21.15.82:0/3150266954 conn(0x7f8134071810 msgr2=0x7f81340cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 shutdown_connections 2024-04-25T10:13:39.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.755+0000 7f81390b4640 1 -- 172.21.15.82:0/3150266954 wait complete. 2024-04-25T10:13:39.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:39.774 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 Processor -- start 2024-04-25T10:13:39.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 -- start start 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9be006d090 0x7f9be006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be006bff0 con 0x7f9be00b0380 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be006c2d0 con 0x7f9be00b5270 2024-04-25T10:13:39.887 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9be4bdd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be006cc00 con 0x7f9be006d090 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bddd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9be00b0380 0x7f9be00d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50204/0 (socket says 172.21.15.82:50204) 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bddd9b640 1 -- 172.21.15.82:0/3466111009 learned_addr learned my addr 172.21.15.82:0/3466111009 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bded9d640 1 --2- 172.21.15.82:0/3466111009 >> v2:172.21.15.82:3300/0 conn(0x7f9be006d090 0x7f9be006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bdd59a640 1 -- 172.21.15.82:0/3466111009 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716828486 0 0) 0x7f9be006bff0 con 0x7f9be00b0380 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bdd59a640 1 -- 172.21.15.82:0/3466111009 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bc8003530 con 0x7f9be00b0380 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bded9d640 1 -- 172.21.15.82:0/3466111009 >> v1:172.21.15.196:6789/0 conn(0x7f9be00b5270 legacy=0x7f9be00b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bded9d640 1 -- 172.21.15.82:0/3466111009 >> v1:172.21.15.82:6789/0 conn(0x7f9be00b0380 legacy=0x7f9be00d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.888 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.887+0000 7f9bded9d640 1 -- 172.21.15.82:0/3466111009 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be006c5b0 con 0x7f9be006d090 2024-04-25T10:13:39.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bded9d640 1 --2- 172.21.15.82:0/3466111009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be006d090 0x7f9be006d4f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009d90 tx=0x7f9bd801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46552a88e963b558 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bdd59a640 1 -- 172.21.15.82:0/3466111009 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80b2070 con 0x7f9be006d090 2024-04-25T10:13:39.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bdd59a640 1 -- 172.21.15.82:0/3466111009 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80a5900 con 0x7f9be006d090 2024-04-25T10:13:39.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bdd59a640 1 -- 172.21.15.82:0/3466111009 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a5cd0 con 0x7f9be006d090 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/3466111009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be006d090 msgr2=0x7f9be006d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/3466111009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be006d090 0x7f9be006d4f0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009d90 tx=0x7f9bd801c930 comp rx=0 tx=0).stop 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/3466111009 shutdown_connections 2024-04-25T10:13:39.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/3466111009 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be006d090 0x7f9be006d4f0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/3466111009 >> 172.21.15.82:0/3466111009 conn(0x7f9be008b000 msgr2=0x7f9be00d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/3466111009 shutdown_connections 2024-04-25T10:13:39.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/3466111009 wait complete. 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 Processor -- start 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- start start 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be006d090 0x7f9be00ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9be00b5270 0x7f9be0151cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be015de20 con 0x7f9be00b0380 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00b0790 con 0x7f9be006d090 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be015dfa0 con 0x7f9be00b5270 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36886/0 (socket says 172.21.15.82:36886) 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 -- 172.21.15.82:0/1351865780 learned_addr learned my addr 172.21.15.82:0/1351865780 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bded9d640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9be00b5270 0x7f9be0151cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bde59c640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be006d090 0x7f9be00ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 -- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9be00b5270 msgr2=0x7f9be0151cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9be00b5270 0x7f9be0151cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 -- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be006d090 msgr2=0x7f9be00ca830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be006d090 0x7f9be00ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 -- 172.21.15.82:0/1351865780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00cae10 con 0x7f9be00b0380 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bddd9b640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c730 tx=0x7f9bd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4017070 con 0x7f9be00b0380 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd4007270 con 0x7f9be00b0380 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be00cb0f0 con 0x7f9be00b0380 2024-04-25T10:13:39.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd4001d50 con 0x7f9be00b0380 2024-04-25T10:13:39.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.891+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00cb6b0 con 0x7f9be00b0380 2024-04-25T10:13:39.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004efa0 con 0x7f9be00b0380 2024-04-25T10:13:39.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bd401a070 con 0x7f9be00b0380 2024-04-25T10:13:39.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9bceffd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9bde59c640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9bd4084550 con 0x7f9be00b0380 2024-04-25T10:13:39.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.895+0000 7f9bde59c640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 0x7f9bc0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000a750 tx=0x7f9bd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:39.899+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bd40553d0 con 0x7f9be00b0380 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.055+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9be00d23b0 con 0x7f9be00b0380 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.055+0000 7f9bceffd640 1 -- 172.21.15.82:0/1351865780 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9bd405b0d0 con 0x7f9be00b0380 2024-04-25T10:13:40.056 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 msgr2=0x7f9bc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 0x7f9bc0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000a750 tx=0x7f9bd0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 msgr2=0x7f9be00b8f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400c730 tx=0x7f9bd400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 shutdown_connections 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9bc0066f60 0x7f9bc0069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9be00b5270 0x7f9be0151cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9be00b0380 0x7f9be00b8f60 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 --2- 172.21.15.82:0/1351865780 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9be006d090 0x7f9be00ca830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 >> 172.21.15.82:0/1351865780 conn(0x7f9be008b000 msgr2=0x7f9be00b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 shutdown_connections 2024-04-25T10:13:40.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.059+0000 7f9be4bdd640 1 -- 172.21.15.82:0/1351865780 wait complete. 2024-04-25T10:13:40.073 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:40.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 Processor -- start 2024-04-25T10:13:40.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 -- start start 2024-04-25T10:13:40.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0cf720 0x7fa42c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa42c0b2c60 0x7fa42c06e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa42c0d0040 con 0x7fa42c0af520 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c0cdd70 con 0x7fa42c0b2c60 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa434656640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0afa20 con 0x7fa42c0cf720 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa431bf1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa42c0b2c60 0x7fa42c06e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa432bf3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa42c0af520 0x7fa42c06e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:50212/0 (socket says 172.21.15.82:50212) 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa432bf3640 1 -- 172.21.15.82:0/2069767151 learned_addr learned my addr 172.21.15.82:0/2069767151 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa4323f2640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0cf720 0x7fa42c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa4313f0640 1 -- 172.21.15.82:0/2069767151 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865364000 0 0) 0x7fa42c0d0040 con 0x7fa42c0af520 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa4313f0640 1 -- 172.21.15.82:0/2069767151 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa418003530 con 0x7fa42c0af520 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa431bf1640 1 -- 172.21.15.82:0/2069767151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0cf720 msgr2=0x7fa42c0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa431bf1640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0cf720 0x7fa42c0cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa431bf1640 1 -- 172.21.15.82:0/2069767151 >> v1:172.21.15.82:6789/0 conn(0x7fa42c0af520 legacy=0x7fa42c06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.187+0000 7fa431bf1640 1 -- 172.21.15.82:0/2069767151 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0ccfb0 con 0x7fa42c0b2c60 2024-04-25T10:13:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa431bf1640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0b2c60 0x7fa42c06e170 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097f0 tx=0x7fa41c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e7a2611b0eeb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4313f0640 1 -- 172.21.15.82:0/2069767151 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0ab600 con 0x7fa42c0b2c60 2024-04-25T10:13:40.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4313f0640 1 -- 172.21.15.82:0/2069767151 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c0b0070 con 0x7fa42c0b2c60 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4313f0640 1 -- 172.21.15.82:0/2069767151 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c0ab9c0 con 0x7fa42c0b2c60 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- 172.21.15.82:0/2069767151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0b2c60 msgr2=0x7fa42c06e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0b2c60 0x7fa42c06e170 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0097f0 tx=0x7fa41c01c3e0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- 172.21.15.82:0/2069767151 shutdown_connections 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0b2c60 0x7fa42c06e170 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- 172.21.15.82:0/2069767151 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0cf720 0x7fa42c0cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- 172.21.15.82:0/2069767151 >> 172.21.15.82:0/2069767151 conn(0x7fa42c099a70 msgr2=0x7fa42c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- 172.21.15.82:0/2069767151 shutdown_connections 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- 172.21.15.82:0/2069767151 wait complete. 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 Processor -- start 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- start start 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0b2c60 0x7fa42c1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa42c0cf720 0x7fa42c14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c1611e0 con 0x7fa42c0af520 2024-04-25T10:13:40.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c0caac0 con 0x7fa42c0cf720 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa434656640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c06edc0 con 0x7fa42c0b2c60 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36904/0 (socket says 172.21.15.82:36904) 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 -- 172.21.15.82:0/1049776659 learned_addr learned my addr 172.21.15.82:0/1049776659 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 -- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0b2c60 msgr2=0x7fa42c1566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0b2c60 0x7fa42c1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 -- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa42c0cf720 msgr2=0x7fa42c14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa42c0cf720 0x7fa42c14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 -- 172.21.15.82:0/1049776659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c15ac40 con 0x7fa42c0af520 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa4323f2640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa42800d800 tx=0x7fa42800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42800b6c0 con 0x7fa42c0af520 2024-04-25T10:13:40.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa428019070 con 0x7fa42c0af520 2024-04-25T10:13:40.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.191+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42800ba60 con 0x7fa42c0af520 2024-04-25T10:13:40.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42c15c310 con 0x7fa42c0af520 2024-04-25T10:13:40.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c15c8d0 con 0x7fa42c0af520 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c04efa0 con 0x7fa42c0af520 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa42800bc00 con 0x7fa42c0af520 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa422ffd640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 0x7fa410069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa428085690 con 0x7fa42c0af520 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa431bf1640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 0x7fa410069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.195+0000 7fa431bf1640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 0x7fa410069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0af000 tx=0x7fa41c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.199 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.199+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa428056500 con 0x7fa42c0af520 2024-04-25T10:13:40.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.343+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa42c158d10 con 0x7fa42c0af520 2024-04-25T10:13:40.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa422ffd640 1 -- 172.21.15.82:0/1049776659 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa428055e00 con 0x7fa42c0af520 2024-04-25T10:13:40.346 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:40.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 msgr2=0x7fa410069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 0x7fa410069450 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa41c0af000 tx=0x7fa41c004740 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 msgr2=0x7fa42c0c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7fa42800d800 tx=0x7fa42800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 shutdown_connections 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa410066f90 0x7fa410069450 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa42c0b2c60 0x7fa42c1566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa42c0cf720 0x7fa42c14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 --2- 172.21.15.82:0/1049776659 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa42c0af520 0x7fa42c0c9fe0 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 >> 172.21.15.82:0/1049776659 conn(0x7fa42c099a70 msgr2=0x7fa42c0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 shutdown_connections 2024-04-25T10:13:40.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.347+0000 7fa434656640 1 -- 172.21.15.82:0/1049776659 wait complete. 2024-04-25T10:13:40.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 Processor -- start 2024-04-25T10:13:40.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 -- start start 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f00cf590 0x7f65f00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f65f0072000 0x7f65f0072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65f00cbd20 con 0x7f65f015b6b0 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f015ba90 con 0x7f65f00cf590 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65f4ac7640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65f00cc000 con 0x7f65f0072000 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f00cf590 0x7f65f00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f65f00cf590 0x7f65f00cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36906/0 (socket says 172.21.15.82:36906) 2024-04-25T10:13:40.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 -- 172.21.15.82:0/3776410632 learned_addr learned my addr 172.21.15.82:0/3776410632 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 -- 172.21.15.82:0/3776410632 >> v2:172.21.15.196:3300/0 conn(0x7f65f0072000 msgr2=0x7f65f0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65eed9d640 1 --2- 172.21.15.82:0/3776410632 >> v2:172.21.15.196:3300/0 conn(0x7f65f0072000 0x7f65f0072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 --2- 172.21.15.82:0/3776410632 >> v2:172.21.15.196:3300/0 conn(0x7f65f0072000 0x7f65f0072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 -- 172.21.15.82:0/3776410632 >> v1:172.21.15.196:6789/0 conn(0x7f65f015b6b0 legacy=0x7f65f00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 -- 172.21.15.82:0/3776410632 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f00cc2e0 con 0x7f65f00cf590 2024-04-25T10:13:40.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.475+0000 7f65edd9b640 1 --2- 172.21.15.82:0/3776410632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f00cf590 0x7f65f00cfa20 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f65f006f000 tx=0x7f65e401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da0f6256e7cb762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65ed59a640 1 -- 172.21.15.82:0/3776410632 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40b1070 con 0x7f65f00cf590 2024-04-25T10:13:40.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65ed59a640 1 -- 172.21.15.82:0/3776410632 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e40ac630 con 0x7f65f00cf590 2024-04-25T10:13:40.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65ed59a640 1 -- 172.21.15.82:0/3776410632 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40a8050 con 0x7f65f00cf590 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/3776410632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f00cf590 msgr2=0x7f65f00cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/3776410632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f00cf590 0x7f65f00cfa20 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f65f006f000 tx=0x7f65e401c4e0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/3776410632 shutdown_connections 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/3776410632 >> v2:172.21.15.196:3300/0 conn(0x7f65f0072000 0x7f65f0072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/3776410632 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f00cf590 0x7f65f00cfa20 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/3776410632 >> 172.21.15.82:0/3776410632 conn(0x7f65f006ae60 msgr2=0x7f65f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/3776410632 shutdown_connections 2024-04-25T10:13:40.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/3776410632 wait complete. 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 Processor -- start 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- start start 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65f0072000 0x7f65f00989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65f00cf590 0x7f65f0159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00caf30 con 0x7f65f015b6b0 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f00cadb0 con 0x7f65f00cf590 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65f01682a0 con 0x7f65f0072000 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36912/0 (socket says 172.21.15.82:36912) 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 -- 172.21.15.82:0/619047555 learned_addr learned my addr 172.21.15.82:0/619047555 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:40.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65edd9b640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65f00cf590 0x7f65f0159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65ee59c640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65f0072000 0x7f65f00989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 -- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65f0072000 msgr2=0x7f65f00989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65f0072000 0x7f65f00989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 -- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65f00cf590 msgr2=0x7f65f0159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65f00cf590 0x7f65f0159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 -- 172.21.15.82:0/619047555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f0098ef0 con 0x7f65f015b6b0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65eed9d640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f65e800cd70 tx=0x7f65e800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8003ba0 con 0x7f65f015b6b0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e8003d40 con 0x7f65f015b6b0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e8004240 con 0x7f65f015b6b0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.479+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f0158e50 con 0x7f65f015b6b0 2024-04-25T10:13:40.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f0159410 con 0x7f65f015b6b0 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f004efa0 con 0x7f65f015b6b0 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65e80043e0 con 0x7f65f015b6b0 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65deffd640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65e8085cb0 con 0x7f65f015b6b0 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65ee59c640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.483+0000 7f65ee59c640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 0x7f65d0069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f65e000a790 tx=0x7f65e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.487+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65e8056ba0 con 0x7f65f015b6b0 2024-04-25T10:13:40.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.631+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f65f00b3b20 con 0x7f65f015b6b0 2024-04-25T10:13:40.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65deffd640 1 -- 172.21.15.82:0/619047555 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65e80564a0 con 0x7f65f015b6b0 2024-04-25T10:13:40.634 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 msgr2=0x7f65d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 0x7f65d0069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f65e000a790 tx=0x7f65e0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 msgr2=0x7f65f0156d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f65e800cd70 tx=0x7f65e800a660 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 shutdown_connections 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f65d0066f60 0x7f65d0069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f65f015b6b0 0x7f65f0156d10 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f65f00cf590 0x7f65f0159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 --2- 172.21.15.82:0/619047555 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f65f0072000 0x7f65f00989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 >> 172.21.15.82:0/619047555 conn(0x7f65f006ae60 msgr2=0x7f65f00af850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 shutdown_connections 2024-04-25T10:13:40.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.635+0000 7f65f4ac7640 1 -- 172.21.15.82:0/619047555 wait complete. 2024-04-25T10:13:40.651 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:40.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 Processor -- start 2024-04-25T10:13:40.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 -- start start 2024-04-25T10:13:40.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92b415a700 0x7f92b415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f92b40b24a0 0x7f92b40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b40b2090 con 0x7f92b40b3f50 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b415f200 con 0x7f92b415a700 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b9209640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b40b43e0 con 0x7f92b40b24a0 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92b415a700 0x7f92b415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f92b415a700 0x7f92b415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36918/0 (socket says 172.21.15.82:36918) 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 -- 172.21.15.82:0/513079169 learned_addr learned my addr 172.21.15.82:0/513079169 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b359e640 1 --2- 172.21.15.82:0/513079169 >> v2:172.21.15.196:3300/0 conn(0x7f92b40b24a0 0x7f92b40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 -- 172.21.15.82:0/513079169 >> v2:172.21.15.196:3300/0 conn(0x7f92b40b24a0 msgr2=0x7f92b40b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 --2- 172.21.15.82:0/513079169 >> v2:172.21.15.196:3300/0 conn(0x7f92b40b24a0 0x7f92b40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 -- 172.21.15.82:0/513079169 >> v1:172.21.15.196:6789/0 conn(0x7f92b40b3f50 legacy=0x7f92b40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.771+0000 7f92b2d9d640 1 -- 172.21.15.82:0/513079169 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40b46c0 con 0x7f92b415a700 2024-04-25T10:13:40.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b2d9d640 1 --2- 172.21.15.82:0/513079169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b415a700 0x7f92b415ecc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f92b406bd20 tx=0x7f92a401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb35e3cc91327d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b259c640 1 -- 172.21.15.82:0/513079169 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40b0070 con 0x7f92b415a700 2024-04-25T10:13:40.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b259c640 1 -- 172.21.15.82:0/513079169 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a40ab600 con 0x7f92b415a700 2024-04-25T10:13:40.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b259c640 1 -- 172.21.15.82:0/513079169 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40ab9d0 con 0x7f92b415a700 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- 172.21.15.82:0/513079169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b415a700 msgr2=0x7f92b415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 --2- 172.21.15.82:0/513079169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b415a700 0x7f92b415ecc0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f92b406bd20 tx=0x7f92a401c3a0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- 172.21.15.82:0/513079169 shutdown_connections 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 --2- 172.21.15.82:0/513079169 >> v2:172.21.15.196:3300/0 conn(0x7f92b40b24a0 0x7f92b40b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 --2- 172.21.15.82:0/513079169 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b415a700 0x7f92b415ecc0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- 172.21.15.82:0/513079169 >> 172.21.15.82:0/513079169 conn(0x7f92b40b11a0 msgr2=0x7f92b40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- 172.21.15.82:0/513079169 shutdown_connections 2024-04-25T10:13:40.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- 172.21.15.82:0/513079169 wait complete. 2024-04-25T10:13:40.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 Processor -- start 2024-04-25T10:13:40.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.775+0000 7f92b9209640 1 -- start start 2024-04-25T10:13:40.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92b40b24a0 0x7f92b413e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b415a700 0x7f92b406d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40cd310 con 0x7f92b406d5d0 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40ce600 con 0x7f92b40b24a0 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40cd490 con 0x7f92b415a700 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:36924/0 (socket says 172.21.15.82:36924) 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 -- 172.21.15.82:0/2106543415 learned_addr learned my addr 172.21.15.82:0/2106543415 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:40.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92aa59c640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b415a700 0x7f92b406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b2d9d640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92b40b24a0 0x7f92b413e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 -- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b415a700 msgr2=0x7f92b406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b415a700 0x7f92b406d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 -- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92b40b24a0 msgr2=0x7f92b413e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92b40b24a0 0x7f92b413e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 -- 172.21.15.82:0/2106543415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b41533b0 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b359e640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00cc60 tx=0x7f92ac00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0039a0 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac003b40 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b40add80 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004020 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40ae340 con 0x7f92b406d5d0 2024-04-25T10:13:40.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404efa0 con 0x7f92b406d5d0 2024-04-25T10:13:40.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92ac0041c0 con 0x7f92b406d5d0 2024-04-25T10:13:40.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92ac085670 con 0x7f92b406d5d0 2024-04-25T10:13:40.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b2d9d640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.779+0000 7f92b2d9d640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 0x7f9294069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f92a40ad000 tx=0x7f92a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.784 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.783+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92ac014030 con 0x7f92b406d5d0 2024-04-25T10:13:40.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f92b40b1a40 con 0x7f92b406d5d0 2024-04-25T10:13:40.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92abfff640 1 -- 172.21.15.82:0/2106543415 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v121) v1 ==== 80+0+0 (secure 0 0 0) 0x7f92ac056460 con 0x7f92b406d5d0 2024-04-25T10:13:40.946 INFO:teuthology.orchestra.run.smithi082.stderr:listed 0 entries 2024-04-25T10:13:40.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 msgr2=0x7f9294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 0x7f9294069450 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f92a40ad000 tx=0x7f92a4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 msgr2=0x7f92b40ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00cc60 tx=0x7f92ac00a5c0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 shutdown_connections 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9294066f90 0x7f9294069450 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f92b415a700 0x7f92b406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f92b406d5d0 0x7f92b40ad110 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 --2- 172.21.15.82:0/2106543415 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f92b40b24a0 0x7f92b413e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 >> 172.21.15.82:0/2106543415 conn(0x7f92b40b11a0 msgr2=0x7f92b40cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 shutdown_connections 2024-04-25T10:13:40.949 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:40.947+0000 7f92b9209640 1 -- 172.21.15.82:0/2106543415 wait complete. 2024-04-25T10:13:40.963 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:13:40.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:40.964 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:40.974 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55061]: fuse finished with error 0 and tester_r 0 2024-04-25T10:13:40.976 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:47.077 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:13:47.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:47.078 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:47.085 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:47.085 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:47.134 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:47.134 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi082.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:47.134 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:13:47.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:47.134 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:47.181 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T10:13:47.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:13:47.182 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 Processor -- start 2024-04-25T10:13:47.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 -- start start 2024-04-25T10:13:47.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3a0144ca0 0x7fe3a006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a00cee30 con 0x7fe3a006ec60 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a00cc2b0 con 0x7fe3a0144ca0 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a6c51640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a006e290 con 0x7fe3a00ce510 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3a0144ca0 0x7fe3a006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe3a0144ca0 0x7fe3a006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51236/0 (socket says 172.21.15.82:51236) 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a51ee640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe3a006ec60 0x7fe3a0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47164/0 (socket says 172.21.15.82:47164) 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 -- 172.21.15.82:0/2124587688 learned_addr learned my addr 172.21.15.82:0/2124587688 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:47.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe3a49ed640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 -- 172.21.15.82:0/2124587688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 msgr2=0x7fe3a00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 -- 172.21.15.82:0/2124587688 >> v1:172.21.15.82:6789/0 conn(0x7fe3a006ec60 legacy=0x7fe3a0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39ffff640 1 -- 172.21.15.82:0/2124587688 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a00ccc70 con 0x7fe3a0144ca0 2024-04-25T10:13:47.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.355+0000 7fe39f7fe640 1 -- 172.21.15.82:0/2124587688 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070583229 0 0) 0x7fe3a00cee30 con 0x7fe3a006ec60 2024-04-25T10:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe39ffff640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a0144ca0 0x7fe3a006e720 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fe394009870 tx=0x7fe39401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb8c445afda0354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe39f7fe640 1 -- 172.21.15.82:0/2124587688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940b5070 con 0x7fe3a0144ca0 2024-04-25T10:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe39f7fe640 1 -- 172.21.15.82:0/2124587688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3940a8a50 con 0x7fe3a0144ca0 2024-04-25T10:13:47.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe39f7fe640 1 -- 172.21.15.82:0/2124587688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940a8e20 con 0x7fe3a0144ca0 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2124587688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a0144ca0 msgr2=0x7fe3a006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a0144ca0 0x7fe3a006e720 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fe394009870 tx=0x7fe39401ed50 comp rx=0 tx=0).stop 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2124587688 shutdown_connections 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a0144ca0 0x7fe3a006e720 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2124587688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a00ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2124587688 >> 172.21.15.82:0/2124587688 conn(0x7fe3a00b11a0 msgr2=0x7fe3a00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2124587688 shutdown_connections 2024-04-25T10:13:47.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.359+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2124587688 wait complete. 2024-04-25T10:13:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 Processor -- start 2024-04-25T10:13:47.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- start start 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a0143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3a0144ca0 0x7fe3a01435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a00cc900 con 0x7fe3a006ec60 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a006df20 con 0x7fe3a0144ca0 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a0142d90 con 0x7fe3a00ce510 2024-04-25T10:13:47.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51244/0 (socket says 172.21.15.82:51244) 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 -- 172.21.15.82:0/2697292640 learned_addr learned my addr 172.21.15.82:0/2697292640 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a51ee640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3a0144ca0 0x7fe3a01435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 -- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 msgr2=0x7fe3a0143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a0143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 -- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3a0144ca0 msgr2=0x7fe3a01435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3a0144ca0 0x7fe3a01435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 -- 172.21.15.82:0/2697292640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a01e32c0 con 0x7fe3a006ec60 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a49ed640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c820 tx=0x7fe39000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390015070 con 0x7fe3a006ec60 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390004240 con 0x7fe3a006ec60 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000ce10 con 0x7fe3a006ec60 2024-04-25T10:13:47.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a01e35a0 con 0x7fe3a006ec60 2024-04-25T10:13:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a01e3b60 con 0x7fe3a006ec60 2024-04-25T10:13:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a004efa0 con 0x7fe3a006ec60 2024-04-25T10:13:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe390007830 con 0x7fe3a006ec60 2024-04-25T10:13:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 0x7fe38c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39ffff640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 0x7fe38c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.363+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe3900850a0 con 0x7fe3a006ec60 2024-04-25T10:13:47.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.367+0000 7fe39ffff640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 0x7fe38c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe3940b2000 tx=0x7fe394004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.371+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe390055f30 con 0x7fe3a006ec60 2024-04-25T10:13:47.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.563+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe3a01445d0 con 0x7fe3a006ec60 2024-04-25T10:13:47.562 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:13:47.563+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:47.567 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.567+0000 7fe39d7fa640 1 -- 172.21.15.82:0/2697292640 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe390055830 con 0x7fe3a006ec60 2024-04-25T10:13:47.567 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 msgr2=0x7fe38c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 0x7fe38c069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe3940b2000 tx=0x7fe394004740 comp rx=0 tx=0).stop 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 msgr2=0x7fe3a013ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fe39000c820 tx=0x7fe39000cce0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 shutdown_connections 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe38c066f60 0x7fe38c069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe3a00ce510 0x7fe3a0143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe3a0144ca0 0x7fe3a01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 --2- 172.21.15.82:0/2697292640 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe3a006ec60 0x7fe3a013ec40 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 >> 172.21.15.82:0/2697292640 conn(0x7fe3a00b11a0 msgr2=0x7fe3a00ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 shutdown_connections 2024-04-25T10:13:47.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.571+0000 7fe3a6c51640 1 -- 172.21.15.82:0/2697292640 wait complete. 2024-04-25T10:13:47.580 INFO:tasks.ceph.mds.c.smithi082.stderr: -7804> 2024-04-25T10:13:23.107+0000 7f711e4c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:47.590 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:47.592 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:47.614 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T10:13:47.614 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T10:13:47.615 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:47.616 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:13:47.617 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T10:13:47.618 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:47.619 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T10:13:47.620 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:13:47.621 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:47.622 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:47.623 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T10:13:47.624 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = False 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.625 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:47.626 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T10:13:47.627 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T10:13:47.628 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.629 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.630 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T10:13:47.631 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.632 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.633 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.634 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T10:13:47.635 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = False 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.636 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.637 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:13:47.639 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:47.682 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:13:47.703 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T10:13:47.703 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T10:13:47.703 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:47.703 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.704 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:47.705 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.706 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.707 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T10:13:47.708 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T10:13:47.709 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T10:13:47.710 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T10:13:47.711 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.712 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T10:13:47.713 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = False 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = False 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T10:13:47.714 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:13:47.715 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T10:13:47.716 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:13:47.717 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T10:13:47.718 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.719 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T10:13:47.720 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T10:13:47.721 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:13:47.722 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:13:47.723 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T10:13:47.724 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = False 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = False 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:13:47.725 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:13:47.727 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T10:13:47.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 Processor -- start 2024-04-25T10:13:47.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 -- start start 2024-04-25T10:13:47.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 0x560c9440e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 0x560c943380c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c94367d30 con 0x560c943f6260 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c942df9f0 con 0x560c943f6640 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd326d91080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c94367eb0 con 0x560c943655f0 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51284/0 (socket says 172.21.15.82:51284) 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 -- 172.21.15.82:0/3796742233 learned_addr learned my addr 172.21.15.82:0/3796742233 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3250b0640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 0x560c943380c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3248af640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 0x560c9440e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 -- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 msgr2=0x560c9440e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 0x560c9440e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 -- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 msgr2=0x560c943380c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 0x560c943380c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 -- 172.21.15.82:0/3796742233 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c944131a0 con 0x560c943f6260 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3248af640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 0x560c9440e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3250b0640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 0x560c943380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3240ae640 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fd31800b300 tx=0x7fd31800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d50b564a6657c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3230ac640 1 -- 172.21.15.82:0/3796742233 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800d510 con 0x560c943f6260 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3230ac640 1 -- 172.21.15.82:0/3796742233 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318008070 con 0x560c943f6260 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.803+0000 7fd3230ac640 1 -- 172.21.15.82:0/3796742233 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800d8d0 con 0x560c943f6260 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/3796742233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 msgr2=0x560c943f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7fd31800b300 tx=0x7fd31800b7c0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/3796742233 shutdown_connections 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943f6640 0x560c943380c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c943f20d0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- 172.21.15.82:0/3796742233 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943655f0 0x560c9440e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/3796742233 >> 172.21.15.82:0/3796742233 conn(0x560c94363ac0 msgr2=0x560c943f5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/3796742233 shutdown_connections 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/3796742233 wait complete. 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 Processor -- start 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- start start 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943655f0 0x560c943385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c944a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943f6640 0x560c944abb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c944136e0 con 0x560c943f6260 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c94413440 con 0x560c943655f0 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560c9433b4a0 con 0x560c943f6640 2024-04-25T10:13:47.811 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c944a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c944a8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51294/0 (socket says 172.21.15.82:51294) 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 -- 172.21.15.82:0/4178565902 learned_addr learned my addr 172.21.15.82:0/4178565902 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3248af640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943655f0 0x560c943385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3250b0640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943f6640 0x560c944abb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 -- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943f6640 msgr2=0x560c944abb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943f6640 0x560c944abb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 -- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943655f0 msgr2=0x560c943385d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943655f0 0x560c943385d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 -- 172.21.15.82:0/4178565902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c9433be40 con 0x560c943f6260 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3248af640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560c943655f0 0x560c943385d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3250b0640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560c943f6640 0x560c944abb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:47.812 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3240ae640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c944a8f30 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd318000f70 tx=0x7fd318000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3220aa640 1 -- 172.21.15.82:0/4178565902 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31801c650 con 0x560c943f6260 2024-04-25T10:13:47.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3220aa640 1 -- 172.21.15.82:0/4178565902 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318021020 con 0x560c943f6260 2024-04-25T10:13:47.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560c944116b0 con 0x560c943f6260 2024-04-25T10:13:47.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd3220aa640 1 -- 172.21.15.82:0/4178565902 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31801c810 con 0x560c943f6260 2024-04-25T10:13:47.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.807+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c94411bf0 con 0x560c943f6260 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3220aa640 1 -- 172.21.15.82:0/4178565902 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd318017070 con 0x560c943f6260 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3220aa640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd310065970 0x7fd310067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3220aa640 1 -- 172.21.15.82:0/4178565902 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd318086000 con 0x560c943f6260 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3248af640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd310065970 0x7fd310067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd326d91080 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x560c9437ebc0 0x560c94412330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e122) v9 -- 0x560c94412870 con 0x560c9437ebc0 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3250b0640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x560c9437ebc0 0x560c94412330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3248af640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd310065970 0x7fd310067e30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd3140076d0 tx=0x7fd314009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3250b0640 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x560c9437ebc0 0x560c94412330 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd3250b0640 1 -- 172.21.15.82:0/4178565902 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v122'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd31c0b4040 con 0x560c9437ebc0 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x560c9437ebc0 msgr2=0x560c94412330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.811+0000 7fd326d91080 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x560c9437ebc0 0x560c94412330 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd310065970 msgr2=0x7fd310067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd310065970 0x7fd310067e30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fd3140076d0 tx=0x7fd314009040 comp rx=0 tx=0).stop 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 msgr2=0x560c944a8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 --2- 172.21.15.82:0/4178565902 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560c943f6260 0x560c944a8f30 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd318000f70 tx=0x7fd318000ee0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd3250b0640 1 -- 172.21.15.82:0/4178565902 reap_dead start 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 shutdown_connections 2024-04-25T10:13:47.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 >> 172.21.15.82:0/4178565902 conn(0x560c94363ac0 msgr2=0x560c943f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 shutdown_connections 2024-04-25T10:13:47.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.815+0000 7fd326d91080 1 -- 172.21.15.82:0/4178565902 wait complete. 2024-04-25T10:13:47.818 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:13:47.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 Processor -- start 2024-04-25T10:13:47.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 -- start start 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8ef415ecf0 0x7f8ef40cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ef406bf00 0x7f8ef406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef406cc70 con 0x7f8ef40cfec0 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef415f0d0 con 0x7f8ef406bf00 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8efa490640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40ce090 con 0x7f8ef415ecf0 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef8a2d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8ef406bf00 0x7f8ef406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8ef40cfec0 0x7f8ef40d0370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47188/0 (socket says 172.21.15.82:47188) 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef37fe640 1 -- 172.21.15.82:0/2083243464 learned_addr learned my addr 172.21.15.82:0/2083243464 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef3fff640 1 --2- 172.21.15.82:0/2083243464 >> v2:172.21.15.196:3300/0 conn(0x7f8ef415ecf0 0x7f8ef40cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef8a2d640 1 -- 172.21.15.82:0/2083243464 >> v2:172.21.15.196:3300/0 conn(0x7f8ef415ecf0 msgr2=0x7f8ef40cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef8a2d640 1 --2- 172.21.15.82:0/2083243464 >> v2:172.21.15.196:3300/0 conn(0x7f8ef415ecf0 0x7f8ef40cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef8a2d640 1 -- 172.21.15.82:0/2083243464 >> v1:172.21.15.82:6789/0 conn(0x7f8ef40cfec0 legacy=0x7f8ef40d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef8a2d640 1 -- 172.21.15.82:0/2083243464 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40ce370 con 0x7f8ef406bf00 2024-04-25T10:13:47.984 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.983+0000 7f8ef2ffd640 1 -- 172.21.15.82:0/2083243464 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204053215 0 0) 0x7f8ef406cc70 con 0x7f8ef40cfec0 2024-04-25T10:13:47.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8ef8a2d640 1 --2- 172.21.15.82:0/2083243464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef406bf00 0x7f8ef406c310 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40d28a0 tx=0x7f8eec01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c353841c351865 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8ef2ffd640 1 -- 172.21.15.82:0/2083243464 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0b2070 con 0x7f8ef406bf00 2024-04-25T10:13:47.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8ef2ffd640 1 -- 172.21.15.82:0/2083243464 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec0a59c0 con 0x7f8ef406bf00 2024-04-25T10:13:47.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8ef2ffd640 1 -- 172.21.15.82:0/2083243464 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0a5d90 con 0x7f8ef406bf00 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- 172.21.15.82:0/2083243464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef406bf00 msgr2=0x7f8ef406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- 172.21.15.82:0/2083243464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef406bf00 0x7f8ef406c310 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40d28a0 tx=0x7f8eec01c900 comp rx=0 tx=0).stop 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- 172.21.15.82:0/2083243464 shutdown_connections 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- 172.21.15.82:0/2083243464 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef406bf00 0x7f8ef406c310 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- 172.21.15.82:0/2083243464 >> v2:172.21.15.196:3300/0 conn(0x7f8ef415ecf0 0x7f8ef40cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- 172.21.15.82:0/2083243464 >> 172.21.15.82:0/2083243464 conn(0x7f8ef40b52a0 msgr2=0x7f8ef40b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- 172.21.15.82:0/2083243464 shutdown_connections 2024-04-25T10:13:47.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- 172.21.15.82:0/2083243464 wait complete. 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 Processor -- start 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- start start 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef406bf00 0x7f8ef40cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ef415ecf0 0x7f8ef415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40d2220 con 0x7f8ef40cfec0 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40d20a0 con 0x7f8ef406bf00 2024-04-25T10:13:47.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.987+0000 7f8efa490640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef40d23a0 con 0x7f8ef415ecf0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51312/0 (socket says 172.21.15.82:51312) 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 -- 172.21.15.82:0/1813690454 learned_addr learned my addr 172.21.15.82:0/1813690454 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef3fff640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef406bf00 0x7f8ef40cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef8a2d640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ef415ecf0 0x7f8ef415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 -- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ef415ecf0 msgr2=0x7f8ef415adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ef415ecf0 0x7f8ef415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 -- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef406bf00 msgr2=0x7f8ef40cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef406bf00 0x7f8ef40cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 -- 172.21.15.82:0/1813690454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef415ce30 con 0x7f8ef40cfec0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef37fe640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c520 tx=0x7f8ee800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8018070 con 0x7f8ef40cfec0 2024-04-25T10:13:47.991 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee80074e0 con 0x7f8ef40cfec0 2024-04-25T10:13:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef415d690 con 0x7f8ef40cfec0 2024-04-25T10:13:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee80049f0 con 0x7f8ef40cfec0 2024-04-25T10:13:47.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef415d3b0 con 0x7f8ef40cfec0 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4067c40 con 0x7f8ef40cfec0 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ee801b070 con 0x7f8ef40cfec0 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 0x7f8ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8ee8084580 con 0x7f8ef40cfec0 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef3fff640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 0x7f8ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.991+0000 7f8ef3fff640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 0x7f8ee0069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8ef415ac50 tx=0x7f8ee4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:47.995+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ee8055440 con 0x7f8ef40cfec0 2024-04-25T10:13:48.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.191+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f8ef40d1670 con 0x7f8ef40cfec0 2024-04-25T10:13:48.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8ef0ff9640 1 -- 172.21.15.82:0/1813690454 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v93) v1 ==== 162+0+0 (secure 0 0 0) 0x7f8ee8054d40 con 0x7f8ef40cfec0 2024-04-25T10:13:48.605 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 msgr2=0x7f8ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 0x7f8ee0069420 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8ef415ac50 tx=0x7f8ee4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 msgr2=0x7f8ef40c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800c520 tx=0x7f8ee800c9e0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 shutdown_connections 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8ee0066f60 0x7f8ee0069420 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8ef415ecf0 0x7f8ef415adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8ef40cfec0 0x7f8ef40c9070 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 --2- 172.21.15.82:0/1813690454 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8ef406bf00 0x7f8ef40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 >> 172.21.15.82:0/1813690454 conn(0x7f8ef40b52a0 msgr2=0x7f8ef40d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 shutdown_connections 2024-04-25T10:13:48.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.607+0000 7f8efa490640 1 -- 172.21.15.82:0/1813690454 wait complete. 2024-04-25T10:13:48.637 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:48.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 Processor -- start 2024-04-25T10:13:48.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 -- start start 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6920071130 0x7f6920071590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692015f240 con 0x7f692015bf00 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692015f520 con 0x7f692015edc0 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f692551c640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6920071ad0 con 0x7f6920071130 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f69167fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6920071130 0x7f6920071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f691f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f692015bf00 0x7f692015c370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47190/0 (socket says 172.21.15.82:47190) 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f69167fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6920071130 0x7f6920071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51324/0 (socket says 172.21.15.82:51324) 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f691f7fe640 1 -- 172.21.15.82:0/1778366137 learned_addr learned my addr 172.21.15.82:0/1778366137 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:48.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f69167fc640 1 -- 172.21.15.82:0/1778366137 >> v1:172.21.15.196:6789/0 conn(0x7f692015edc0 legacy=0x7f6920144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f69167fc640 1 -- 172.21.15.82:0/1778366137 >> v1:172.21.15.82:6789/0 conn(0x7f692015bf00 legacy=0x7f692015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f69167fc640 1 -- 172.21.15.82:0/1778366137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692015f800 con 0x7f6920071130 2024-04-25T10:13:48.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.751+0000 7f691e7fc640 1 -- 172.21.15.82:0/1778366137 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100427279 0 0) 0x7f692015f240 con 0x7f692015bf00 2024-04-25T10:13:48.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- 172.21.15.82:0/1778366137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6920071130 0x7f6920071590 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb5392e9ca6fc55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f691e7fc640 1 -- 172.21.15.82:0/1778366137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080b0070 con 0x7f6920071130 2024-04-25T10:13:48.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f691e7fc640 1 -- 172.21.15.82:0/1778366137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6908004230 con 0x7f6920071130 2024-04-25T10:13:48.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f691e7fc640 1 -- 172.21.15.82:0/1778366137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908003620 con 0x7f6920071130 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1778366137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6920071130 msgr2=0x7f6920071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 --2- 172.21.15.82:0/1778366137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6920071130 0x7f6920071590 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f69080097c0 tx=0x7f690801c460 comp rx=0 tx=0).stop 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1778366137 shutdown_connections 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 --2- 172.21.15.82:0/1778366137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6920071130 0x7f6920071590 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1778366137 >> 172.21.15.82:0/1778366137 conn(0x7f6920066fa0 msgr2=0x7f692006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1778366137 shutdown_connections 2024-04-25T10:13:48.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1778366137 wait complete. 2024-04-25T10:13:48.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 Processor -- start 2024-04-25T10:13:48.758 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- start start 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6920071130 0x7f69201419f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f692015edc0 0x7f6920143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69201451a0 con 0x7f692015bf00 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6920145020 con 0x7f692015edc0 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69201695f0 con 0x7f6920071130 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51326/0 (socket says 172.21.15.82:51326) 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 -- 172.21.15.82:0/1373067433 learned_addr learned my addr 172.21.15.82:0/1373067433 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 -- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6920071130 msgr2=0x7f69201419f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6920071130 0x7f69201419f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 -- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f692015edc0 msgr2=0x7f6920143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f692015edc0 0x7f6920143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 -- 172.21.15.82:0/1373067433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920140c30 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f69167fc640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f69080ae040 tx=0x7f6908019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080b0040 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69080bc070 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6920140f10 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080b5480 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.755+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692013b830 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6920067c40 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69080b3070 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f6917fff640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 0x7f6900069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f691effd640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 0x7f6900069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f690811f110 con 0x7f692015bf00 2024-04-25T10:13:48.760 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.759+0000 7f691effd640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 0x7f6900069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6920159790 tx=0x7f6910009230 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.762 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.763+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6908016050 con 0x7f692015bf00 2024-04-25T10:13:48.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69200afdc0 con 0x7f692015bf00 2024-04-25T10:13:48.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f6917fff640 1 -- 172.21.15.82:0/1373067433 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f69080aa3a0 con 0x7f692015bf00 2024-04-25T10:13:48.942 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:48.942 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":94,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:48.611616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9849":{"gid":9849,"name":"c","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/3303693719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3303693719},{"type":"v1","addr":"172.21.15.82:6837","nonce":3303693719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:48.942 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 94 2024-04-25T10:13:48.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 msgr2=0x7f6900069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.944 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 0x7f6900069450 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f6920159790 tx=0x7f6910009230 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 msgr2=0x7f692013b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f69080ae040 tx=0x7f6908019200 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 shutdown_connections 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6900066f90 0x7f6900069450 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f692015edc0 0x7f6920143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f692015bf00 0x7f692013b2f0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 --2- 172.21.15.82:0/1373067433 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6920071130 0x7f69201419f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 >> 172.21.15.82:0/1373067433 conn(0x7f6920066fa0 msgr2=0x7f692006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 shutdown_connections 2024-04-25T10:13:48.945 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:48.943+0000 7f692551c640 1 -- 172.21.15.82:0/1373067433 wait complete. 2024-04-25T10:13:49.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 94, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:13:48.611616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9849}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9849': {'gid': 9849, 'name': 'c', 'rank': 0, 'incarnation': 94, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/3303693719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 3303693719}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3303693719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:49.362 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9849:up:replay 2024-04-25T10:13:50.363 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:50.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 Processor -- start 2024-04-25T10:13:50.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 -- start start 2024-04-25T10:13:50.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe268071e90 0x7fe2680b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26806e260 0x7fe26806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2680b3010 con 0x7fe268143f50 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26815ef10 con 0x7fe268071e90 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26e74b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680b2dc0 con 0x7fe26806e260 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe268071e90 0x7fe2680b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe268071e90 0x7fe2680b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51328/0 (socket says 172.21.15.82:51328) 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 -- 172.21.15.82:0/3341739526 learned_addr learned my addr 172.21.15.82:0/3341739526 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 -- 172.21.15.82:0/3341739526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26806e260 msgr2=0x7fe26806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe26cce8640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26806e260 0x7fe26806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26806e260 0x7fe26806e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 -- 172.21.15.82:0/3341739526 >> v1:172.21.15.196:6789/0 conn(0x7fe268143f50 legacy=0x7fe26815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 -- 172.21.15.82:0/3341739526 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680b32f0 con 0x7fe268071e90 2024-04-25T10:13:50.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe2677fe640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe268071e90 0x7fe2680b3dd0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dda0697126789b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe266ffd640 1 -- 172.21.15.82:0/3341739526 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2580b0070 con 0x7fe268071e90 2024-04-25T10:13:50.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe266ffd640 1 -- 172.21.15.82:0/3341739526 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580ab600 con 0x7fe268071e90 2024-04-25T10:13:50.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.495+0000 7fe266ffd640 1 -- 172.21.15.82:0/3341739526 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2580ab9d0 con 0x7fe268071e90 2024-04-25T10:13:50.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/3341739526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe268071e90 msgr2=0x7fe2680b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe268071e90 0x7fe2680b3dd0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fe258009870 tx=0x7fe25801c520 comp rx=0 tx=0).stop 2024-04-25T10:13:50.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/3341739526 shutdown_connections 2024-04-25T10:13:50.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe26806e260 0x7fe26806e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- 172.21.15.82:0/3341739526 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe268071e90 0x7fe2680b3dd0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/3341739526 >> 172.21.15.82:0/3341739526 conn(0x7fe268066fa0 msgr2=0x7fe26806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/3341739526 shutdown_connections 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/3341739526 wait complete. 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 Processor -- start 2024-04-25T10:13:50.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- start start 2024-04-25T10:13:50.500 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe268071e90 0x7fe2680ac890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe268143f50 0x7fe2680acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26815b6a0 con 0x7fe26806e260 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26815b520 con 0x7fe268071e90 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26815b820 con 0x7fe268143f50 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51332/0 (socket says 172.21.15.82:51332) 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 -- 172.21.15.82:0/2556074720 learned_addr learned my addr 172.21.15.82:0/2556074720 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe2677fe640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe268071e90 0x7fe2680ac890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 -- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe268143f50 msgr2=0x7fe2680acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe268143f50 0x7fe2680acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 -- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe268071e90 msgr2=0x7fe2680ac890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe268071e90 0x7fe2680ac890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 -- 172.21.15.82:0/2556074720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26813f640 con 0x7fe26806e260 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe267fff640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00b440 tx=0x7fe25c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c015020 con 0x7fe26806e260 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c011890 con 0x7fe26806e260 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2680ad400 con 0x7fe26806e260 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c011c80 con 0x7fe26806e260 2024-04-25T10:13:50.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.499+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2680ad9c0 con 0x7fe26806e260 2024-04-25T10:13:50.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe268067c40 con 0x7fe26806e260 2024-04-25T10:13:50.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe25c018020 con 0x7fe26806e260 2024-04-25T10:13:50.503 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe264ff9640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 0x7fe254069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe25c084fb0 con 0x7fe26806e260 2024-04-25T10:13:50.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe2677fe640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 0x7fe254069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.503+0000 7fe2677fe640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 0x7fe254069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ad000 tx=0x7fe2580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.507+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe25c01f030 con 0x7fe26806e260 2024-04-25T10:13:50.699 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.699+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe26815a5b0 con 0x7fe26806e260 2024-04-25T10:13:50.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.699+0000 7fe264ff9640 1 -- 172.21.15.82:0/2556074720 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fe25c00d040 con 0x7fe26806e260 2024-04-25T10:13:50.700 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:50.701 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":96,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:49.626820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9849":{"gid":9849,"name":"c","rank":0,"incarnation":94,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/3303693719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3303693719},{"type":"v1","addr":"172.21.15.82:6837","nonce":3303693719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:50.701 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 96 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 msgr2=0x7fe254069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 0x7fe254069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe2580ad000 tx=0x7fe2580a6000 comp rx=0 tx=0).stop 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 msgr2=0x7fe2681f9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fe25c00b440 tx=0x7fe25c00b900 comp rx=0 tx=0).stop 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 shutdown_connections 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe254066f60 0x7fe254069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe268143f50 0x7fe2680acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe268071e90 0x7fe2680ac890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 --2- 172.21.15.82:0/2556074720 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe26806e260 0x7fe2681f9490 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 >> 172.21.15.82:0/2556074720 conn(0x7fe268066fa0 msgr2=0x7fe2680afb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 shutdown_connections 2024-04-25T10:13:50.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:50.703+0000 7fe26e74b640 1 -- 172.21.15.82:0/2556074720 wait complete. 2024-04-25T10:13:51.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:13:49.626820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9849}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9849': {'gid': 9849, 'name': 'c', 'rank': 0, 'incarnation': 94, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/3303693719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 3303693719}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3303693719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:51.123 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9849:up:rejoin 2024-04-25T10:13:52.124 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 Processor -- start 2024-04-25T10:13:52.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 -- start start 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 0x7f3f00144910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f00071840 0x7f3f0015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0015f240 con 0x7f3f00144e50 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0015f520 con 0x7f3f00071840 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f07c23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015f800 con 0x7f3f0015edc0 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f061c0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3f00071840 0x7f3f0015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f051be640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3f00144e50 0x7f3f00071130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:47192/0 (socket says 172.21.15.82:47192) 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f051be640 1 -- 172.21.15.82:0/1245530825 learned_addr learned my addr 172.21.15.82:0/1245530825 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f061c0640 1 -- 172.21.15.82:0/1245530825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 msgr2=0x7f3f00144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f059bf640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 0x7f3f00144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f049bd640 1 -- 172.21.15.82:0/1245530825 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128305306 0 0) 0x7f3f0015f240 con 0x7f3f00144e50 2024-04-25T10:13:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f061c0640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 0x7f3f00144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f061c0640 1 -- 172.21.15.82:0/1245530825 >> v1:172.21.15.82:6789/0 conn(0x7f3f00144e50 legacy=0x7f3f00071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f061c0640 1 -- 172.21.15.82:0/1245530825 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0015fae0 con 0x7f3f00071840 2024-04-25T10:13:52.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.255+0000 7f3f059bf640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 0x7f3f00144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f00071840 0x7f3f0015bf00 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009d90 tx=0x7f3ef801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de02140d3ce4016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f049bd640 1 -- 172.21.15.82:0/1245530825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80b2070 con 0x7f3f00071840 2024-04-25T10:13:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f049bd640 1 -- 172.21.15.82:0/1245530825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80a5940 con 0x7f3f00071840 2024-04-25T10:13:52.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f049bd640 1 -- 172.21.15.82:0/1245530825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80a5d10 con 0x7f3f00071840 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/1245530825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f00071840 msgr2=0x7f3f0015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f00071840 0x7f3f0015bf00 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009d90 tx=0x7f3ef801c970 comp rx=0 tx=0).stop 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/1245530825 shutdown_connections 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f00071840 0x7f3f0015bf00 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- 172.21.15.82:0/1245530825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f0015edc0 0x7f3f00144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/1245530825 >> 172.21.15.82:0/1245530825 conn(0x7f3f00066fa0 msgr2=0x7f3f0006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/1245530825 shutdown_connections 2024-04-25T10:13:52.258 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/1245530825 wait complete. 2024-04-25T10:13:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 Processor -- start 2024-04-25T10:13:52.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- start start 2024-04-25T10:13:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 0x7f3f00138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 0x7f3f00141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015cfe0 con 0x7f3f0015edc0 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015ce60 con 0x7f3f00071840 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0015d160 con 0x7f3f00144e50 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51344/0 (socket says 172.21.15.82:51344) 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 -- 172.21.15.82:0/2044500770 learned_addr learned my addr 172.21.15.82:0/2044500770 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f059bf640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 0x7f3f00138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f051be640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 0x7f3f00141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 -- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 msgr2=0x7f3f00141aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 0x7f3f00141aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 -- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 msgr2=0x7f3f00138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 0x7f3f00138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 -- 172.21.15.82:0/2044500770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000b4450 con 0x7f3f0015edc0 2024-04-25T10:13:52.262 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f059bf640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 0x7f3f00138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f051be640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 0x7f3f00141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f061c0640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef801b020 tx=0x7f3ef8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80b2050 con 0x7f3f0015edc0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef80be070 con 0x7f3f0015edc0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef80adb50 con 0x7f3f0015edc0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.259+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f0013a4a0 con 0x7f3f0015edc0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f00141fe0 con 0x7f3f0015edc0 2024-04-25T10:13:52.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f00067c40 con 0x7f3f0015edc0 2024-04-25T10:13:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef80adcf0 con 0x7f3f0015edc0 2024-04-25T10:13:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3ef67fc640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3ef8120440 con 0x7f3f0015edc0 2024-04-25T10:13:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3f059bf640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.263+0000 7f3f059bf640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 0x7f3ee4069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3efc008890 tx=0x7f3efc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.267+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef8016560 con 0x7f3f0015edc0 2024-04-25T10:13:52.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.459+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f0006b9f0 con 0x7f3f0015edc0 2024-04-25T10:13:52.459 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.459+0000 7f3ef67fc640 1 -- 172.21.15.82:0/2044500770 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f3ef80ab340 con 0x7f3f0015edc0 2024-04-25T10:13:52.460 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:52.460 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:50.635981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9849":{"gid":9849,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/3303693719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3303693719},{"type":"v1","addr":"172.21.15.82:6837","nonce":3303693719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:52.460 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 97 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 msgr2=0x7f3ee4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 0x7f3ee4069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f3efc008890 tx=0x7f3efc008040 comp rx=0 tx=0).stop 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 msgr2=0x7f3f000b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef801b020 tx=0x7f3ef8019710 comp rx=0 tx=0).stop 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 shutdown_connections 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3ee4066f60 0x7f3ee4069420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3f00144e50 0x7f3f00141aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3f0015edc0 0x7f3f000b1380 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 --2- 172.21.15.82:0/2044500770 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3f00071840 0x7f3f00138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 >> 172.21.15.82:0/2044500770 conn(0x7f3f00066fa0 msgr2=0x7f3f000b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 shutdown_connections 2024-04-25T10:13:52.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:52.463+0000 7f3f07c23640 1 -- 172.21.15.82:0/2044500770 wait complete. 2024-04-25T10:13:52.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:13:50.635981+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 122, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9849}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9849': {'gid': 9849, 'name': 'c', 'rank': 0, 'incarnation': 94, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/3303693719', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 3303693719}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 3303693719}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:13:52.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:13:52.882 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:13:53.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 Processor -- start 2024-04-25T10:13:53.014 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 -- start start 2024-04-25T10:13:53.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc4d8147080 0x7fc4d815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.015 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4d8071130 0x7fc4d8071580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d8147460 con 0x7fc4d815eaf0 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d806e360 con 0x7fc4d8071130 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4df3d9640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d8071ac0 con 0x7fc4d8147080 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4d8071130 0x7fc4d8071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc4d8071130 0x7fc4d8071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51356/0 (socket says 172.21.15.82:51356) 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 -- 172.21.15.82:0/3992483789 learned_addr learned my addr 172.21.15.82:0/3992483789 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 -- 172.21.15.82:0/3992483789 >> v2:172.21.15.196:3300/0 conn(0x7fc4d8147080 msgr2=0x7fc4d815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 --2- 172.21.15.82:0/3992483789 >> v2:172.21.15.196:3300/0 conn(0x7fc4d8147080 0x7fc4d815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 -- 172.21.15.82:0/3992483789 >> v1:172.21.15.196:6789/0 conn(0x7fc4d815eaf0 legacy=0x7fc4d806d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.016 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.015+0000 7fc4dd976640 1 -- 172.21.15.82:0/3992483789 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80adaf0 con 0x7fc4d8071130 2024-04-25T10:13:53.017 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4dd976640 1 --2- 172.21.15.82:0/3992483789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8071130 0x7fc4d8071580 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80725b0 tx=0x7fc4d001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660ba5b5752b77e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4cffff640 1 -- 172.21.15.82:0/3992483789 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a5780 con 0x7fc4d8071130 2024-04-25T10:13:53.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4cffff640 1 -- 172.21.15.82:0/3992483789 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00a5920 con 0x7fc4d8071130 2024-04-25T10:13:53.018 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4cffff640 1 -- 172.21.15.82:0/3992483789 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a5cf0 con 0x7fc4d8071130 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3992483789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8071130 msgr2=0x7fc4d8071580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3992483789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8071130 0x7fc4d8071580 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fc4d80725b0 tx=0x7fc4d001c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3992483789 shutdown_connections 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3992483789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d8071130 0x7fc4d8071580 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3992483789 >> v2:172.21.15.196:3300/0 conn(0x7fc4d8147080 0x7fc4d815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3992483789 >> 172.21.15.82:0/3992483789 conn(0x7fc4d8066fa0 msgr2=0x7fc4d806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3992483789 shutdown_connections 2024-04-25T10:13:53.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3992483789 wait complete. 2024-04-25T10:13:53.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 Processor -- start 2024-04-25T10:13:53.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- start start 2024-04-25T10:13:53.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4d8071130 0x7fc4d8143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 0x7fc4d80ce440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80aeac0 con 0x7fc4d815eaf0 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80add90 con 0x7fc4d8147080 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.019+0000 7fc4df3d9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80aec40 con 0x7fc4d8071130 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51372/0 (socket says 172.21.15.82:51372) 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 -- 172.21.15.82:0/3127927627 learned_addr learned my addr 172.21.15.82:0/3127927627 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 -- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4d8071130 msgr2=0x7fc4d8143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4d8071130 0x7fc4d8143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 -- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 msgr2=0x7fc4d80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dc974640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 0x7fc4d80ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 0x7fc4d80ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 -- 172.21.15.82:0/3127927627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d806e9a0 con 0x7fc4d815eaf0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dc974640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 0x7fc4d80ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd976640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001a5e0 tx=0x7fc4d0019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 -- 172.21.15.82:0/3127927627 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00b8670 con 0x7fc4d815eaf0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 -- 172.21.15.82:0/3127927627 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00be070 con 0x7fc4d815eaf0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d8141180 con 0x7fc4d815eaf0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 -- 172.21.15.82:0/3127927627 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00b8a70 con 0x7fc4d815eaf0 2024-04-25T10:13:53.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d813b1f0 con 0x7fc4d815eaf0 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 -- 172.21.15.82:0/3127927627 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4d00b8c10 con 0x7fc4d815eaf0 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4cdffb640 1 -- 172.21.15.82:0/3127927627 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc4d01203b0 con 0x7fc4d815eaf0 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd175640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4dd175640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 0x7fc4bc069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009730 tx=0x7fc4d4006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4e2546000 1 Processor -- start 2024-04-25T10:13:53.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.023+0000 7fc4e2546000 1 -- start start 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a50d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a069f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb18ee5f0 con 0x55bfb1a06f30 2024-04-25T10:13:53.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb1855da0 con 0x55bfb1a51240 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb17de150 con 0x55bfb1a50920 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51386/0 (socket says 172.21.15.82:51386) 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1945405948 learned_addr learned my addr 172.21.15.82:0/1945405948 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a50d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dc974640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a069f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 msgr2=0x55bfb1a50d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a50d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 msgr2=0x55bfb1a069f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a069f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1945405948 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb17f7ca0 con 0x55bfb1a06f30 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00195c0 tx=0x7fc4d00f0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de421e5114a3ec88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4c2ffd640 1 -- 172.21.15.82:0/1945405948 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00f0ae0 con 0x55bfb1a06f30 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4c2ffd640 1 -- 172.21.15.82:0/1945405948 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00f4c30 con 0x55bfb1a06f30 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4c2ffd640 1 -- 172.21.15.82:0/1945405948 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00f91f0 con 0x55bfb1a06f30 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- 172.21.15.82:0/1945405948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 msgr2=0x55bfb1a07310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00195c0 tx=0x7fc4d00f0440 comp rx=0 tx=0).stop 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- 172.21.15.82:0/1945405948 shutdown_connections 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a06f30 0x55bfb1a07310 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a069f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1945405948 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a50d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- 172.21.15.82:0/1945405948 >> 172.21.15.82:0/1945405948 conn(0x55bfb1a0d2a0 msgr2=0x55bfb1a052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- 172.21.15.82:0/1945405948 shutdown_connections 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- 172.21.15.82:0/1945405948 wait complete. 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- start start 2024-04-25T10:13:53.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 Processor -- start 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 0x55bfb1a57cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a58230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb1810990 con 0x55bfb1a51240 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb1851e40 con 0x55bfb1a06f30 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb17e1900 con 0x55bfb1a50920 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4e2546000 10 client.0 _ll_register_callbacks cb 0x55bfb19091c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51396/0 (socket says 172.21.15.82:51396) 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1824151642 learned_addr learned my addr 172.21.15.82:0/1824151642 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 0x55bfb1a57cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dc974640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a58230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 msgr2=0x55bfb1a58230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a58230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 msgr2=0x55bfb1a57cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 0x55bfb1a57cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 -- 172.21.15.82:0/1824151642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb180e630 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 0x55bfb1a57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.027+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001c9f0 tx=0x7fc4d0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2eba3002cabf43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d011e1b0 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc4d011e350 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d00f4c30 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d00f0d40 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 10 client.9906 fetch_fsmap learned FSMap version 97 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x55bfb17de3a0 con 0x55bfb1a51240 2024-04-25T10:13:53.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00f4c30 con 0x55bfb1a51240 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4d00f8d60 con 0x55bfb1a51240 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 10 client.9906 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 10 client.9906 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 20 client.9906 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a0b4b0 0x55bfb1a0b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 4 client.9906 mds_command: new command op to 9849 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 --> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb17f1a50 con 0x55bfb1a0b4b0 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a0b4b0 0x55bfb1a0b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 10 client.9906 ms_handle_connect on v2:172.21.15.82:6835/3303693719 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a0b4b0 0x55bfb1a0b890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f70d4b2d37783d5 server_cookie=3b8b5cc286c11819 in_seq=0 out_seq=0 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1824151642 <== mds.0 v2:172.21.15.82:6835/3303693719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfb17f1a50 con 0x55bfb1a0b4b0 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 10 client.9906 handle_command_reply: tid=0 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 client.9906 shutdown 2024-04-25T10:13:53.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 msgr2=0x55bfb1a0aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001c9f0 tx=0x7fc4d0009810 comp rx=0 tx=0).stop 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 shutdown_connections 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a0b4b0 0x55bfb1a0b890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a50920 0x55bfb1a58230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1a0aaa0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4b7fff640 0 client.9906 ms_handle_reset on v2:172.21.15.82:6835/3303693719 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1824151642 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a06f30 0x55bfb1a57cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 >> 172.21.15.82:0/1824151642 conn(0x55bfb1a0d2a0 msgr2=0x55bfb1a059e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 shutdown_connections 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 1 -- 172.21.15.82:0/1824151642 wait complete. 2024-04-25T10:13:53.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.031+0000 7fc4e2546000 20 client.9906 trim_cache size 0 max 16384 2024-04-25T10:13:53.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 Processor -- start 2024-04-25T10:13:53.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- start start 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1aab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a50920 0x55bfb1aabd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d8202340 con 0x55bfb1a51240 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d81fabd0 con 0x55bfb1a50920 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80cc630 con 0x55bfb1a06f30 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51412/0 (socket says 172.21.15.82:51412) 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 -- 172.21.15.82:0/1973951816 learned_addr learned my addr 172.21.15.82:0/1973951816 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 -- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 msgr2=0x55bfb1aab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1aab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 -- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a50920 msgr2=0x55bfb1aabd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a50920 0x55bfb1aabd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 -- 172.21.15.82:0/1973951816 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb1a51070 con 0x55bfb1a51240 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00f0640 tx=0x7fc4d0105000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347fde659bc4f2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4c27fc640 1 -- 172.21.15.82:0/1973951816 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00f4c30 con 0x55bfb1a51240 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4c27fc640 1 -- 172.21.15.82:0/1973951816 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00f0ae0 con 0x55bfb1a51240 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4c27fc640 1 -- 172.21.15.82:0/1973951816 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00f8d60 con 0x55bfb1a51240 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- 172.21.15.82:0/1973951816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 msgr2=0x55bfb1aac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7fc4d00f0640 tx=0x7fc4d0105000 comp rx=0 tx=0).stop 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- 172.21.15.82:0/1973951816 shutdown_connections 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a51240 0x55bfb1aac280 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a50920 0x55bfb1aabd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1973951816 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1aab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- 172.21.15.82:0/1973951816 >> 172.21.15.82:0/1973951816 conn(0x55bfb1a0d2a0 msgr2=0x55bfb1a0a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- 172.21.15.82:0/1973951816 shutdown_connections 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- 172.21.15.82:0/1973951816 wait complete. 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 -- start start 2024-04-25T10:13:53.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.047+0000 7fc4e2546000 1 Processor -- start 2024-04-25T10:13:53.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1a93fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a65d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb185a3d0 con 0x55bfb1a50920 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb1911d50 con 0x55bfb1a51240 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb17f0980 con 0x55bfb1a06f30 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 10 client.0 _ll_register_callbacks cb 0x7fc4d0009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51422/0 (socket says 172.21.15.82:51422) 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 -- 172.21.15.82:0/1319612396 learned_addr learned my addr 172.21.15.82:0/1319612396 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 -- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 msgr2=0x55bfb1a93fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1a93fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 -- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 msgr2=0x55bfb1a65d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dd976640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a65d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a65d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 -- 172.21.15.82:0/1319612396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb18d7600 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dc974640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ee80 tx=0x7fc4c800c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b27a49902ec2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8016070 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc4c8012650 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d00f0ae0 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c8021050 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 10 client.9912 fetch_fsmap learned FSMap version 97 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x55bfb17ea9b0 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00f0ae0 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4c8024020 con 0x55bfb1a50920 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 10 client.9912 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T10:13:53.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 10 client.9912 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 20 client.9912 populate_metadata read hostname 'smithi082' 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a66290 0x55bfb1a66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 4 client.9912 mds_command: new command op to 9849 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 --> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bfb18632e0 con 0x55bfb1a66290 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a66290 0x55bfb1a66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4dd175640 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a66290 0x55bfb1a66670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d61f673af6ea742e server_cookie=e83372e9f2b0b5f5 in_seq=0 out_seq=0 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 10 client.9912 ms_handle_connect on v2:172.21.15.82:6835/3303693719 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 1 -- 172.21.15.82:0/1319612396 <== mds.0 v2:172.21.15.82:6835/3303693719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55bfb18632e0 con 0x55bfb1a66290 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 10 client.9912 handle_command_reply: tid=0 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 client.9912 shutdown 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 msgr2=0x55bfb1a65810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ee80 tx=0x7fc4c800c5e0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 shutdown_connections 2024-04-25T10:13:53.056 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x55bfb1a66290 0x55bfb1a66670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55bfb1a51240 0x55bfb1a65d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55bfb1a50920 0x55bfb1a65810 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4b7fff640 0 client.9912 ms_handle_reset on v2:172.21.15.82:6835/3303693719 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 --2- 172.21.15.82:0/1319612396 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55bfb1a06f30 0x55bfb1a93fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 >> 172.21.15.82:0/1319612396 conn(0x55bfb1a0d2a0 msgr2=0x55bfb1a60ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 shutdown_connections 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 1 -- 172.21.15.82:0/1319612396 wait complete. 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4e2546000 20 client.9912 trim_cache size 0 max 16384 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:53.057 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9849, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4371680070000004, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 97, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 122, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 122, 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 5.413093409 2024-04-25T10:13:53.058 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 msgr2=0x7fc4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 0x7fc4bc069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009730 tx=0x7fc4d4006ca0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 msgr2=0x7fc4d80b0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7fc4d001a5e0 tx=0x7fc4d0019750 comp rx=0 tx=0).stop 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 shutdown_connections 2024-04-25T10:13:53.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc4bc066f60 0x7fc4bc069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc4d815eaf0 0x7fc4d80b0180 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc4d8147080 0x7fc4d80ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 --2- 172.21.15.82:0/3127927627 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc4d8071130 0x7fc4d8143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 >> 172.21.15.82:0/3127927627 conn(0x7fc4d8066fa0 msgr2=0x7fc4d815de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 shutdown_connections 2024-04-25T10:13:53.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.051+0000 7fc4df3d9640 1 -- 172.21.15.82:0/3127927627 wait complete. 2024-04-25T10:13:53.068 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:13:53.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.183+0000 7f55fa804640 1 Processor -- start 2024-04-25T10:13:53.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.183+0000 7f55fa804640 1 -- start start 2024-04-25T10:13:53.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55fa804640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f415ec40 0x7f55f40d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55fa804640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f55f409a910 0x7f55f406c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55fa804640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40cf910 con 0x7f55f4099a20 2024-04-25T10:13:53.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55fa804640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f40cfbf0 con 0x7f55f409a910 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55fa804640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40cfed0 con 0x7f55f415ec40 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f55f409a910 0x7f55f406c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f55f409a910 0x7f55f406c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51436/0 (socket says 172.21.15.82:51436) 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 -- 172.21.15.82:0/2737344248 learned_addr learned my addr 172.21.15.82:0/2737344248 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f3fff640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f415ec40 0x7f55f40d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 -- 172.21.15.82:0/2737344248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f415ec40 msgr2=0x7f55f40d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f415ec40 0x7f55f40d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 -- 172.21.15.82:0/2737344248 >> v1:172.21.15.196:6789/0 conn(0x7f55f4099a20 legacy=0x7f55f415f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 -- 172.21.15.82:0/2737344248 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40d01b0 con 0x7f55f409a910 2024-04-25T10:13:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f37fe640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f409a910 0x7f55f406c7b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009870 tx=0x7f55e401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c087bbd41c5d1d42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f2ffd640 1 -- 172.21.15.82:0/2737344248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40ab430 con 0x7f55f409a910 2024-04-25T10:13:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f2ffd640 1 -- 172.21.15.82:0/2737344248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e40b0070 con 0x7f55f409a910 2024-04-25T10:13:53.188 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.187+0000 7f55f2ffd640 1 -- 172.21.15.82:0/2737344248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e40ab7f0 con 0x7f55f409a910 2024-04-25T10:13:53.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2737344248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f409a910 msgr2=0x7f55f406c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f409a910 0x7f55f406c7b0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009870 tx=0x7f55e401c560 comp rx=0 tx=0).stop 2024-04-25T10:13:53.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2737344248 shutdown_connections 2024-04-25T10:13:53.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f409a910 0x7f55f406c7b0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- 172.21.15.82:0/2737344248 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f415ec40 0x7f55f40d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2737344248 >> 172.21.15.82:0/2737344248 conn(0x7f55f40d15e0 msgr2=0x7f55f40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2737344248 shutdown_connections 2024-04-25T10:13:53.190 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2737344248 wait complete. 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 Processor -- start 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- start start 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4099a20 0x7f55f4154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f409a910 0x7f55f415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f406d160 con 0x7f55f415ec40 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f406cfe0 con 0x7f55f4099a20 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f415ffa0 con 0x7f55f409a910 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51444/0 (socket says 172.21.15.82:51444) 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 -- 172.21.15.82:0/2056323608 learned_addr learned my addr 172.21.15.82:0/2056323608 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f3fff640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4099a20 0x7f55f4154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f37fe640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f409a910 0x7f55f415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 -- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f409a910 msgr2=0x7f55f415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f409a910 0x7f55f415ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 -- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4099a20 msgr2=0x7f55f4154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4099a20 0x7f55f4154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 -- 172.21.15.82:0/2056323608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f40c9580 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f8da1640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c730 tx=0x7f55ec00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec016020 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec006d60 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40c9860 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec003bc0 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.191+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40c9e20 con 0x7f55f415ec40 2024-04-25T10:13:53.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4068350 con 0x7f55f415ec40 2024-04-25T10:13:53.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55ec0050d0 con 0x7f55f415ec40 2024-04-25T10:13:53.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55f0ff9640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 0x7f55e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55ec084e50 con 0x7f55f415ec40 2024-04-25T10:13:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55f3fff640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 0x7f55e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.195+0000 7f55f3fff640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 0x7f55e0069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f55e800b9d0 tx=0x7f55e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.199+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55ec020020 con 0x7f55f415ec40 2024-04-25T10:13:53.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.387+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f55f40d08b0 con 0x7f55f415ec40 2024-04-25T10:13:53.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.387+0000 7f55f0ff9640 1 -- 172.21.15.82:0/2056323608 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7f55ec055d10 con 0x7f55f415ec40 2024-04-25T10:13:53.388 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 msgr2=0x7f55e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 0x7f55e0069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f55e800b9d0 tx=0x7f55e8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 msgr2=0x7f55f41563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00c730 tx=0x7f55ec00cb30 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 shutdown_connections 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f55e0066f60 0x7f55e0069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f55f409a910 0x7f55f415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f55f415ec40 0x7f55f41563d0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 --2- 172.21.15.82:0/2056323608 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f55f4099a20 0x7f55f4154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 >> 172.21.15.82:0/2056323608 conn(0x7f55f40d15e0 msgr2=0x7f55f40d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 shutdown_connections 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:53.391+0000 7f55fa804640 1 -- 172.21.15.82:0/2056323608 wait complete. 2024-04-25T10:13:53.405 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:13:53.406 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:53.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.406 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.416 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41836sec preferred_lft 41836sec 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:13:53.417 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.418 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:13:53.419 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:53.419 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:13:53.419 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:13:53.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.419 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:13:53.468 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:13:53.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.469 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:13:53.520 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:13:53.520 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:13:53.520 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:13:53.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.521 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:13:53.522 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:13:53.522 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:13:53.522 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:13:53.522 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:13:53.522 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:13:53.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.594 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:13:53.618 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:13:53.615+0000 7f8277320640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:53.623 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:13:53.623 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:13:53.623 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:53.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.624 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:13:53.681 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:13:53.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:53.682 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:53.730 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:13:53.730 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:13:53.730 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:53.777 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:53.836 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:53.890 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:53.891 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:53.892 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:53.893 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:53.893 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:53.893 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:53.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:53.894 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:53.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:13:53.942 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:13:53.986 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:54.001 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:54.050 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:54.051 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:54.052 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:54.053 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:54.053 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:54.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:54.054 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:54.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:13:54.063+0000 7f4984bfa580 -1 init, newargv = 0x5632e5600290 newargc=15 2024-04-25T10:13:54.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55702]: starting ceph client 2024-04-25T10:13:54.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55702]: starting fuse 2024-04-25T10:13:55.166 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:13:55.183 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:13:55.233 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:13:55.234 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:13:55.235 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:13:55.236 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:55.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:55.238 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:13:55.286 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:13:55.286 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.287 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:13:55.288 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.289 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:13:55.289 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:55.410 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.55702.asok 2024-04-25T10:13:55.413 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:55.420 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55702.asok 2024-04-25T10:13:55.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:55.420 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55702.asok status 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:13:55.573 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "55702", 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 9924, 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 9924 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:55.574 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1792367685 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1792367685 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.9924 192.168.0.1:0/1792367685", 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/1792367685", 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 97, 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 122, 2024-04-25T10:13:55.575 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:13:55.576 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:13:55.576 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:13:55.576 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:13:55.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:55.586 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:13:55.586 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:55.592 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:13:55.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:55.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:55.592 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:55.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:55.642 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:13:55.704 INFO:teuthology.orchestra.run.smithi082.stdout:./ 2024-04-25T10:13:55.704 INFO:teuthology.orchestra.run.smithi082.stdout:./parent 2024-04-25T10:13:55.704 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed 2024-04-25T10:13:55.704 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/charlie 2024-04-25T10:13:55.704 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/alpha 2024-04-25T10:13:55.705 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/unflushed 2024-04-25T10:13:55.705 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/unflushed/jfile 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:13:55.706 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:55.810 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:13:55.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:55.820 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:55.821 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:55.821 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:55.821 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.821 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:13:55.821 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:55.922 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:13:55.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:13:55.931 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:56.027 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:13:56.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:13:56.036 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:56.132 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628280 2024-04-25T10:13:56.134 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:13:56.142 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:56.244 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627778 2024-04-25T10:13:56.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:13:56.254 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:56.358 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627779 2024-04-25T10:13:56.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:56.367 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:13:56.367 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:13:56.368 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:13:56.368 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:13:56.368 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:13:56.368 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:13:56.468 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627780 2024-04-25T10:13:56.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:13:56.478 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:56.478 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:56.526 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:56.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 Processor -- start 2024-04-25T10:13:56.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 -- start start 2024-04-25T10:13:56.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 0x7fcb880b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb8806e260 0x7fcb8806e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb880b3010 con 0x7fcb88143f50 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8815ef10 con 0x7fcb8806e260 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8cde5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b2dc0 con 0x7fcb88071e90 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fcb88143f50 0x7fcb8815e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57858/0 (socket says 172.21.15.82:57858) 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 -- 172.21.15.82:0/504186708 learned_addr learned my addr 172.21.15.82:0/504186708 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb86d9d640 1 --2- 172.21.15.82:0/504186708 >> v2:172.21.15.196:3300/0 conn(0x7fcb8806e260 0x7fcb8806e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477624306 0 0) 0x7fcb880b3010 con 0x7fcb88143f50 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb70003530 con 0x7fcb88143f50 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb85d9b640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 0x7fcb880b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649439537 0 0) 0x7fcb70003530 con 0x7fcb88143f50 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 msgr2=0x7fcb880b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.701 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 0x7fcb880b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 >> v2:172.21.15.196:3300/0 conn(0x7fcb8806e260 msgr2=0x7fcb8806e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 --2- 172.21.15.82:0/504186708 >> v2:172.21.15.196:3300/0 conn(0x7fcb8806e260 0x7fcb8806e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880b32f0 con 0x7fcb88143f50 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fcb78001680 con 0x7fcb88143f50 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 >> v1:172.21.15.82:6789/0 conn(0x7fcb88143f50 legacy=0x7fcb8815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb700039c0 0x7fcb70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 0x7fcb70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb70004d50 0x7fcb700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880b2dc0 con 0x7fcb700039c0 2024-04-25T10:13:56.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7000b9d0 con 0x7fcb70004d50 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7000bb50 con 0x7fcb88071e90 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb700039c0 0x7fcb70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 -- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 msgr2=0x7fcb70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb88071e90 0x7fcb70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 -- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb70004d50 msgr2=0x7fcb700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb70004d50 0x7fcb700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.699+0000 7fcb8659c640 1 -- 172.21.15.82:0/504186708 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb70003530 con 0x7fcb700039c0 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8659c640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb700039c0 0x7fcb70003de0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c2c0 tx=0x7fcb7801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b8bf4f335084f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78007d60 con 0x7fcb700039c0 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb780a9020 con 0x7fcb700039c0 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8559a640 1 -- 172.21.15.82:0/504186708 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78005ae0 con 0x7fcb700039c0 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- 172.21.15.82:0/504186708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb700039c0 msgr2=0x7fcb70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/504186708 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb700039c0 0x7fcb70003de0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c2c0 tx=0x7fcb7801cd40 comp rx=0 tx=0).stop 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb86d9d640 1 -- 172.21.15.82:0/504186708 reap_dead start 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- 172.21.15.82:0/504186708 shutdown_connections 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- 172.21.15.82:0/504186708 >> 172.21.15.82:0/504186708 conn(0x7fcb88066fa0 msgr2=0x7fcb8806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:56.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- 172.21.15.82:0/504186708 shutdown_connections 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- 172.21.15.82:0/504186708 wait complete. 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 Processor -- start 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.703+0000 7fcb8cde5640 1 -- start start 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88159f90 0x7fcb88141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 0x7fcb88142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815b6a0 con 0x7fcb881f7c10 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815b520 con 0x7fcb88159f90 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8815b820 con 0x7fcb8815a370 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42138/0 (socket says 172.21.15.82:42138) 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 -- 172.21.15.82:0/3555279186 learned_addr learned my addr 172.21.15.82:0/3555279186 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:56.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8659c640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88159f90 0x7fcb88141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb85d9b640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 0x7fcb88142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 -- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 msgr2=0x7fcb88142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 0x7fcb88142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 -- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88159f90 msgr2=0x7fcb88141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88159f90 0x7fcb88141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 -- 172.21.15.82:0/3555279186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb881f8180 con 0x7fcb881f7c10 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb85d9b640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 0x7fcb88142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb86d9d640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fcb88144430 tx=0x7fcb8000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8000bb10 con 0x7fcb881f7c10 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8001a070 con 0x7fcb881f7c10 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb881f8460 con 0x7fcb881f7c10 2024-04-25T10:13:56.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb80014400 con 0x7fcb881f7c10 2024-04-25T10:13:56.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb881f8a20 con 0x7fcb881f7c10 2024-04-25T10:13:56.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb88067c40 con 0x7fcb881f7c10 2024-04-25T10:13:56.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb80013070 con 0x7fcb881f7c10 2024-04-25T10:13:56.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 0x7fcb68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fcb800852f0 con 0x7fcb881f7c10 2024-04-25T10:13:56.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8659c640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 0x7fcb68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.707+0000 7fcb8659c640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 0x7fcb68069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb780aa000 tx=0x7fcb78001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.711+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb80024020 con 0x7fcb881f7c10 2024-04-25T10:13:56.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.875+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb88164ba0 con 0x7fcb881f7c10 2024-04-25T10:13:56.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb76ffd640 1 -- 172.21.15.82:0/3555279186 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17091 (secure 0 0 0) 0x7fcb80056120 con 0x7fcb881f7c10 2024-04-25T10:13:56.878 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:56.878 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":122,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:47.565030+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 msgr2=0x7fcb68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 0x7fcb68069450 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcb780aa000 tx=0x7fcb78001a50 comp rx=0 tx=0).stop 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 msgr2=0x7fcb880ce230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fcb88144430 tx=0x7fcb8000b480 comp rx=0 tx=0).stop 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 shutdown_connections 2024-04-25T10:13:56.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb68066f90 0x7fcb68069450 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb8815a370 0x7fcb88142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb881f7c10 0x7fcb880ce230 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 --2- 172.21.15.82:0/3555279186 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb88159f90 0x7fcb88141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 >> 172.21.15.82:0/3555279186 conn(0x7fcb88066fa0 msgr2=0x7fcb88071130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 shutdown_connections 2024-04-25T10:13:56.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:56.879+0000 7fcb8cde5640 1 -- 172.21.15.82:0/3555279186 wait complete. 2024-04-25T10:13:56.897 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:56.897 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:57.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.023+0000 7f9e16371640 1 Processor -- start 2024-04-25T10:13:57.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.023+0000 7f9e16371640 1 -- start start 2024-04-25T10:13:57.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e16371640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e1009a130 0x7f9e100d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e16371640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e100cf910 con 0x7f9e1009a910 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e16371640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e100cfbf0 con 0x7f9e1015ec40 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e16371640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e100cfed0 con 0x7f9e1009a130 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e1490e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e1009a130 0x7f9e100d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e1009a910 0x7f9e10099a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57870/0 (socket says 172.21.15.82:57870) 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0f7fe640 1 -- 172.21.15.82:0/3711312751 learned_addr learned my addr 172.21.15.82:0/3711312751 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:57.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e1490e640 1 -- 172.21.15.82:0/3711312751 >> v1:172.21.15.196:6789/0 conn(0x7f9e1015ec40 legacy=0x7f9e1006c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e1490e640 1 -- 172.21.15.82:0/3711312751 >> v1:172.21.15.82:6789/0 conn(0x7f9e1009a910 legacy=0x7f9e10099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0effd640 1 -- 172.21.15.82:0/3711312751 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2119622852 0 0) 0x7f9e100cf910 con 0x7f9e1009a910 2024-04-25T10:13:57.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e1490e640 1 -- 172.21.15.82:0/3711312751 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100d01b0 con 0x7f9e1009a130 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e1490e640 1 --2- 172.21.15.82:0/3711312751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a130 0x7f9e100d06d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009d90 tx=0x7f9e0801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1601e0215433ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0effd640 1 -- 172.21.15.82:0/3711312751 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080b2070 con 0x7f9e1009a130 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0effd640 1 -- 172.21.15.82:0/3711312751 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e080a5900 con 0x7f9e1009a130 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.027+0000 7f9e0effd640 1 -- 172.21.15.82:0/3711312751 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e080a5cd0 con 0x7f9e1009a130 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3711312751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a130 msgr2=0x7f9e100d06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 --2- 172.21.15.82:0/3711312751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a130 0x7f9e100d06d0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009d90 tx=0x7f9e0801c930 comp rx=0 tx=0).stop 2024-04-25T10:13:57.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3711312751 shutdown_connections 2024-04-25T10:13:57.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 --2- 172.21.15.82:0/3711312751 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a130 0x7f9e100d06d0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3711312751 >> 172.21.15.82:0/3711312751 conn(0x7f9e100d15e0 msgr2=0x7f9e100d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3711312751 shutdown_connections 2024-04-25T10:13:57.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3711312751 wait complete. 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 Processor -- start 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- start start 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e1009a130 0x7f9e100b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e1015ec40 0x7f9e10155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10160020 con 0x7f9e1009a910 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1015fea0 con 0x7f9e1015ec40 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e10150e60 con 0x7f9e1009a130 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42162/0 (socket says 172.21.15.82:42162) 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 -- 172.21.15.82:0/3547235601 learned_addr learned my addr 172.21.15.82:0/3547235601 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e1490e640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e1015ec40 0x7f9e10155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0ffff640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e1009a130 0x7f9e100b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 -- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e1009a130 msgr2=0x7f9e100b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e1009a130 0x7f9e100b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 -- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e1015ec40 msgr2=0x7f9e10155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e1015ec40 0x7f9e10155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 -- 172.21.15.82:0/3547235601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10151140 con 0x7f9e1009a910 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0f7fe640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c520 tx=0x7f9e0000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00018070 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e000074e0 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e000049f0 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e10151420 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.031+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e10151cc0 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e101515f0 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e00004b90 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e0cff9640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 0x7f9df0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e0ffff640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 0x7f9df0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9e000844e0 con 0x7f9e1009a910 2024-04-25T10:13:57.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.035+0000 7f9e0ffff640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 0x7f9df0069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9df8003d60 tx=0x7f9df8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.039+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e00054cb0 con 0x7f9e1009a910 2024-04-25T10:13:57.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9e10068350 con 0x7f9e1009a910 2024-04-25T10:13:57.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e0cff9640 1 -- 172.21.15.82:0/3547235601 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e00058d50 con 0x7f9e1009a910 2024-04-25T10:13:57.198 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:57.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 msgr2=0x7f9df0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 0x7f9df0069420 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f9df8003d60 tx=0x7f9df8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 msgr2=0x7f9e10155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c520 tx=0x7f9e0000c9e0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 shutdown_connections 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9df0066f60 0x7f9df0069420 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e1015ec40 0x7f9e10155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e1009a910 0x7f9e10155670 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 --2- 172.21.15.82:0/3547235601 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e1009a130 0x7f9e100b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 >> 172.21.15.82:0/3547235601 conn(0x7f9e100d15e0 msgr2=0x7f9e10099a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 shutdown_connections 2024-04-25T10:13:57.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.199+0000 7f9e16371640 1 -- 172.21.15.82:0/3547235601 wait complete. 2024-04-25T10:13:57.217 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:57.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 Processor -- start 2024-04-25T10:13:57.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 -- start start 2024-04-25T10:13:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faa280b1f20 0x7faa280b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2809ab90 con 0x7faa280b23b0 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa280cc2f0 con 0x7faa280b1f20 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa2dc73640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa280b2b20 con 0x7faa280cce90 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa277fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faa280b1f20 0x7faa280b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa26ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faa280b23b0 0x7faa280cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57874/0 (socket says 172.21.15.82:57874) 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa26ffd640 1 -- 172.21.15.82:0/569566658 learned_addr learned my addr 172.21.15.82:0/569566658 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa27fff640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa277fe640 1 -- 172.21.15.82:0/569566658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 msgr2=0x7faa280cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa277fe640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa277fe640 1 -- 172.21.15.82:0/569566658 >> v1:172.21.15.82:6789/0 conn(0x7faa280b23b0 legacy=0x7faa280cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa267fc640 1 -- 172.21.15.82:0/569566658 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127114856 0 0) 0x7faa2809ab90 con 0x7faa280b23b0 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa277fe640 1 -- 172.21.15.82:0/569566658 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2809a5d0 con 0x7faa280b1f20 2024-04-25T10:13:57.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.335+0000 7faa27fff640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa277fe640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280b4430 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7faa180097c0 tx=0x7faa1801c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b546713772d483 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa267fc640 1 -- 172.21.15.82:0/569566658 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180b0070 con 0x7faa280b1f20 2024-04-25T10:13:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa267fc640 1 -- 172.21.15.82:0/569566658 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa180ab600 con 0x7faa280b1f20 2024-04-25T10:13:57.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa267fc640 1 -- 172.21.15.82:0/569566658 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180ab9d0 con 0x7faa280b1f20 2024-04-25T10:13:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- 172.21.15.82:0/569566658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 msgr2=0x7faa280b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280b4430 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7faa180097c0 tx=0x7faa1801c150 comp rx=0 tx=0).stop 2024-04-25T10:13:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- 172.21.15.82:0/569566658 shutdown_connections 2024-04-25T10:13:57.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- 172.21.15.82:0/569566658 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280b4430 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- 172.21.15.82:0/569566658 >> 172.21.15.82:0/569566658 conn(0x7faa280d15e0 msgr2=0x7faa280d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- 172.21.15.82:0/569566658 shutdown_connections 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- 172.21.15.82:0/569566658 wait complete. 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 Processor -- start 2024-04-25T10:13:57.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- start start 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 0x7faa28157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28168910 con 0x7faa280b1f20 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28168790 con 0x7faa280b23b0 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa2dc73640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa28168a90 con 0x7faa280cce90 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa277fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa277fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42172/0 (socket says 172.21.15.82:42172) 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.339+0000 7faa277fe640 1 -- 172.21.15.82:0/3386730799 learned_addr learned my addr 172.21.15.82:0/3386730799 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa26ffd640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 0x7faa28157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 -- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 msgr2=0x7faa280ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 -- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 msgr2=0x7faa28157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 0x7faa28157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 -- 172.21.15.82:0/3386730799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa280c76e0 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa26ffd640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 0x7faa28157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa277fe640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7faa180ae040 tx=0x7faa1801a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa180b0040 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa18004030 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa280c7f40 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa18004400 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2806db90 con 0x7faa280b1f20 2024-04-25T10:13:57.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa28068350 con 0x7faa280b1f20 2024-04-25T10:13:57.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa18003620 con 0x7faa280b1f20 2024-04-25T10:13:57.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 0x7faa14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faa1811f8b0 con 0x7faa280b1f20 2024-04-25T10:13:57.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa26ffd640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 0x7faa14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.343+0000 7faa26ffd640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 0x7faa14069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faa280c5040 tx=0x7faa1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.347+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa18016090 con 0x7faa280b1f20 2024-04-25T10:13:57.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faa280d22f0 con 0x7faa280b1f20 2024-04-25T10:13:57.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa0ffff640 1 -- 172.21.15.82:0/3386730799 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7faa180f06e0 con 0x7faa280b1f20 2024-04-25T10:13:57.499 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 msgr2=0x7faa14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 0x7faa14069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7faa280c5040 tx=0x7faa1c009040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 msgr2=0x7faa280caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7faa180ae040 tx=0x7faa1801a990 comp rx=0 tx=0).stop 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 shutdown_connections 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faa14066f60 0x7faa14069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faa280cce90 0x7faa280ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faa280b23b0 0x7faa28157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.501 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.499+0000 7faa2dc73640 1 --2- 172.21.15.82:0/3386730799 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faa280b1f20 0x7faa280caa60 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.503+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 >> 172.21.15.82:0/3386730799 conn(0x7faa280d15e0 msgr2=0x7faa280b6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.503+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 shutdown_connections 2024-04-25T10:13:57.502 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.503+0000 7faa2dc73640 1 -- 172.21.15.82:0/3386730799 wait complete. 2024-04-25T10:13:57.515 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 Processor -- start 2024-04-25T10:13:57.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 -- start start 2024-04-25T10:13:57.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bec15dc10 0x7f3bec071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec071960 con 0x7f3bec143870 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec071c40 con 0x7f3bec15d830 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bf0de8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec071f20 con 0x7f3bec15dc10 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3be1d9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3bec15dc10 0x7f3bec071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bead9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f3bec143870 0x7f3bec06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57890/0 (socket says 172.21.15.82:57890) 2024-04-25T10:13:57.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3bead9d640 1 -- 172.21.15.82:0/4110490637 learned_addr learned my addr 172.21.15.82:0/4110490637 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3be1d9b640 1 -- 172.21.15.82:0/4110490637 >> v1:172.21.15.196:6789/0 conn(0x7f3bec15d830 legacy=0x7f3bec15aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3be1d9b640 1 -- 172.21.15.82:0/4110490637 >> v1:172.21.15.82:6789/0 conn(0x7f3bec143870 legacy=0x7f3bec06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3be9d9b640 1 -- 172.21.15.82:0/4110490637 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512515993 0 0) 0x7f3bec071960 con 0x7f3bec143870 2024-04-25T10:13:57.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.635+0000 7f3be1d9b640 1 -- 172.21.15.82:0/4110490637 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec071650 con 0x7f3bec15dc10 2024-04-25T10:13:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3be1d9b640 1 --2- 172.21.15.82:0/4110490637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec15dc10 0x7f3bec071110 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400ba90 tx=0x7f3bd401ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a28c2b9ab6e36b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3be9d9b640 1 -- 172.21.15.82:0/4110490637 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40b4070 con 0x7f3bec15dc10 2024-04-25T10:13:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3be9d9b640 1 -- 172.21.15.82:0/4110490637 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd40a78e0 con 0x7f3bec15dc10 2024-04-25T10:13:57.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3be9d9b640 1 -- 172.21.15.82:0/4110490637 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd40a7cb0 con 0x7f3bec15dc10 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- 172.21.15.82:0/4110490637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec15dc10 msgr2=0x7f3bec071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/4110490637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec15dc10 0x7f3bec071110 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400ba90 tx=0x7f3bd401ea10 comp rx=0 tx=0).stop 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- 172.21.15.82:0/4110490637 shutdown_connections 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/4110490637 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec15dc10 0x7f3bec071110 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- 172.21.15.82:0/4110490637 >> 172.21.15.82:0/4110490637 conn(0x7f3bec066fa0 msgr2=0x7f3bec06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- 172.21.15.82:0/4110490637 shutdown_connections 2024-04-25T10:13:57.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- 172.21.15.82:0/4110490637 wait complete. 2024-04-25T10:13:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 Processor -- start 2024-04-25T10:13:57.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.639+0000 7f3bf0de8640 1 -- start start 2024-04-25T10:13:57.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 0x7f3bec0adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 0x7f3bec13c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b3cc0 con 0x7f3bec143870 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec06eb60 con 0x7f3bec15a030 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b3e40 con 0x7f3bec15d830 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42192/0 (socket says 172.21.15.82:42192) 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 -- 172.21.15.82:0/2421569310 learned_addr learned my addr 172.21.15.82:0/2421569310 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bead9d640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 0x7f3bec13c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be1d9b640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 0x7f3bec0adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 -- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 msgr2=0x7f3bec0adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 0x7f3bec0adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 -- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 msgr2=0x7f3bec13c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 0x7f3bec13c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 -- 172.21.15.82:0/2421569310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0af880 con 0x7f3bec143870 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be1d9b640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 0x7f3bec0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bead9d640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 0x7f3bec13c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bea59c640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c450 tx=0x7f3bdc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc012830 con 0x7f3bec143870 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc017070 con 0x7f3bec143870 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec1377f0 con 0x7f3bec143870 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc012c30 con 0x7f3bec143870 2024-04-25T10:13:57.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0afe20 con 0x7f3bec143870 2024-04-25T10:13:57.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec067c40 con 0x7f3bec143870 2024-04-25T10:13:57.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bdc01c460 con 0x7f3bec143870 2024-04-25T10:13:57.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3bdc084fe0 con 0x7f3bec143870 2024-04-25T10:13:57.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.643+0000 7f3be1d9b640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.647+0000 7f3be1d9b640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 0x7f3bcc069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40b1000 tx=0x7f3bd4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.647+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bdc055db0 con 0x7f3bec143870 2024-04-25T10:13:57.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.799+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bec15a5b0 con 0x7f3bec143870 2024-04-25T10:13:57.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.799+0000 7f3be37fe640 1 -- 172.21.15.82:0/2421569310 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bdc0556b0 con 0x7f3bec143870 2024-04-25T10:13:57.799 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 msgr2=0x7f3bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 0x7f3bcc069420 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f3bd40b1000 tx=0x7f3bd4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 msgr2=0x7f3bec1f61a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc00c450 tx=0x7f3bdc00c910 comp rx=0 tx=0).stop 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 shutdown_connections 2024-04-25T10:13:57.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3bcc066f60 0x7f3bcc069420 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3bec15d830 0x7f3bec0adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3bec15a030 0x7f3bec13c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 --2- 172.21.15.82:0/2421569310 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3bec143870 0x7f3bec1f61a0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 >> 172.21.15.82:0/2421569310 conn(0x7f3bec066fa0 msgr2=0x7f3bec0cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 shutdown_connections 2024-04-25T10:13:57.802 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.803+0000 7f3bf0de8640 1 -- 172.21.15.82:0/2421569310 wait complete. 2024-04-25T10:13:57.816 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:57.925 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 Processor -- start 2024-04-25T10:13:57.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 -- start start 2024-04-25T10:13:57.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f78a806cef0 0x7f78a815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a815f050 con 0x7f78a80d32d0 2024-04-25T10:13:57.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a806c410 con 0x7f78a80ceeb0 2024-04-25T10:13:57.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78ad5de640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a80b6ae0 con 0x7f78a806cef0 2024-04-25T10:13:57.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a6ffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78a80d32d0 0x7f78a80ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57896/0 (socket says 172.21.15.82:57896) 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a6ffd640 1 -- 172.21.15.82:0/2683119243 learned_addr learned my addr 172.21.15.82:0/2683119243 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559774930 0 0) 0x7f78a815f050 con 0x7f78a80d32d0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7890003530 con 0x7f78a80d32d0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 --2- 172.21.15.82:0/2683119243 >> v2:172.21.15.196:3300/0 conn(0x7f78a806cef0 0x7f78a815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798735854 0 0) 0x7f7890003530 con 0x7f78a80d32d0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 >> v2:172.21.15.196:3300/0 conn(0x7f78a806cef0 msgr2=0x7f78a815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 --2- 172.21.15.82:0/2683119243 >> v2:172.21.15.196:3300/0 conn(0x7f78a806cef0 0x7f78a815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 >> v1:172.21.15.196:6789/0 conn(0x7f78a80ceeb0 legacy=0x7f78a815e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a806c6f0 con 0x7f78a80d32d0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7898001680 con 0x7f78a80d32d0 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 >> v1:172.21.15.82:6789/0 conn(0x7f78a80d32d0 legacy=0x7f78a80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900039c0 0x7f7890003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890004810 0x7f7890004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 0x7f789000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000c050 con 0x7f78900051d0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789000c200 con 0x7f7890004810 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789000c400 con 0x7f78900039c0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 0x7f789000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a67fc640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890004810 0x7f7890004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a6ffd640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900039c0 0x7f7890003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 -- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900039c0 msgr2=0x7f7890003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900039c0 0x7f7890003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 -- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890004810 msgr2=0x7f7890004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890004810 0x7f7890004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a77fe640 1 -- 172.21.15.82:0/2683119243 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890003530 con 0x7f78900051d0 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a67fc640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7890004810 0x7f7890004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.927+0000 7f78a6ffd640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78900039c0 0x7f7890003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78a77fe640 1 -- 172.21.15.82:0/2683119243 reap_dead start 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78a77fe640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 0x7f789000b9d0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009ae0 tx=0x7f78a001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cc2f69e149a905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a00096b0 con 0x7f78900051d0 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a0004020 con 0x7f78900051d0 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78a5ffb640 1 -- 172.21.15.82:0/2683119243 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004410 con 0x7f78900051d0 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 msgr2=0x7f789000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 0x7f789000b9d0 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f78a0009ae0 tx=0x7f78a001cab0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- 172.21.15.82:0/2683119243 shutdown_connections 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 --2- 172.21.15.82:0/2683119243 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78900051d0 0x7f789000b9d0 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- 172.21.15.82:0/2683119243 >> 172.21.15.82:0/2683119243 conn(0x7f78a80d39f0 msgr2=0x7f78a80d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- 172.21.15.82:0/2683119243 shutdown_connections 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- 172.21.15.82:0/2683119243 wait complete. 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 Processor -- start 2024-04-25T10:13:57.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.931+0000 7f78ad5de640 1 -- start start 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900051d0 0x7f78a8152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78a80c8f00 0x7f78a80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80d36b0 con 0x7f78a81e8680 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a806c1c0 con 0x7f78900051d0 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a80ce7a0 con 0x7f78a80c8f00 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42198/0 (socket says 172.21.15.82:42198) 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 -- 172.21.15.82:0/3829070953 learned_addr learned my addr 172.21.15.82:0/3829070953 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:57.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 -- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78a80c8f00 msgr2=0x7f78a80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78a80c8f00 0x7f78a80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 -- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900051d0 msgr2=0x7f78a8152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900051d0 0x7f78a8152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 -- 172.21.15.82:0/3829070953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a80c98c0 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a67fc640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f789c00d8b0 tx=0x7f789c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c003e00 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c017070 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a80b7e00 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c00b8f0 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a80b83c0 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a8067c40 con 0x7f78a81e8680 2024-04-25T10:13:57.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f789c00ba90 con 0x7f78a81e8680 2024-04-25T10:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 0x7f7894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f789c0854d0 con 0x7f78a81e8680 2024-04-25T10:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a6ffd640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 0x7f7894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.935+0000 7f78a6ffd640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 0x7f7894069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f78980017f0 tx=0x7f7898009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:57.939+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f789c0562a0 con 0x7f78a81e8680 2024-04-25T10:13:58.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78a80b56c0 con 0x7f78a81e8680 2024-04-25T10:13:58.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f788f7fe640 1 -- 172.21.15.82:0/3829070953 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) v1 ==== 80+0+120 (secure 0 0 0) 0x7f789c055ba0 con 0x7f78a81e8680 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6835/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:13:58.099 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 msgr2=0x7f7894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 0x7f7894069420 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f78980017f0 tx=0x7f7898009040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 msgr2=0x7f78a81e8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f789c00d8b0 tx=0x7f789c00dd70 comp rx=0 tx=0).stop 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 shutdown_connections 2024-04-25T10:13:58.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7894066f60 0x7f7894069420 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78a80c8f00 0x7f78a80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78a81e8680 0x7f78a81e8b00 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.103 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 --2- 172.21.15.82:0/3829070953 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78900051d0 0x7f78a8152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 >> 172.21.15.82:0/3829070953 conn(0x7f78a80d39f0 msgr2=0x7f78a815f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 shutdown_connections 2024-04-25T10:13:58.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.099+0000 7f78ad5de640 1 -- 172.21.15.82:0/3829070953 wait complete. 2024-04-25T10:13:58.119 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:13:58.119 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:58.125 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 Processor -- start 2024-04-25T10:13:58.281 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 -- start start 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc63815ecf0 0x7fc6380cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 0x7fc6380d0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6380d08b0 con 0x7fc63806bf00 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63815f0d0 con 0x7fc63815ecf0 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc63d3e8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63806ca20 con 0x7fc6380cfec0 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc63815ecf0 0x7fc6380cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fc63815ecf0 0x7fc6380cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42212/0 (socket says 172.21.15.82:42212) 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 -- 172.21.15.82:0/635917411 learned_addr learned my addr 172.21.15.82:0/635917411 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc6367fc640 1 --2- 172.21.15.82:0/635917411 >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 0x7fc6380d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 -- 172.21.15.82:0/635917411 >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 msgr2=0x7fc6380d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 --2- 172.21.15.82:0/635917411 >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 0x7fc6380d0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.283 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 -- 172.21.15.82:0/635917411 >> v1:172.21.15.196:6789/0 conn(0x7fc63806bf00 legacy=0x7fc63806c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc636ffd640 1 -- 172.21.15.82:0/635917411 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63806cc70 con 0x7fc63815ecf0 2024-04-25T10:13:58.284 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.283+0000 7fc6367fc640 1 --2- 172.21.15.82:0/635917411 >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 0x7fc6380d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc636ffd640 1 --2- 172.21.15.82:0/635917411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc63815ecf0 0x7fc6380cee50 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d28a0 tx=0x7fc62c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0448b5cc7244601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc635ffb640 1 -- 172.21.15.82:0/635917411 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0b0070 con 0x7fc63815ecf0 2024-04-25T10:13:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc635ffb640 1 -- 172.21.15.82:0/635917411 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc62c0ab600 con 0x7fc63815ecf0 2024-04-25T10:13:58.285 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc635ffb640 1 -- 172.21.15.82:0/635917411 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62c0ab9d0 con 0x7fc63815ecf0 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- 172.21.15.82:0/635917411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc63815ecf0 msgr2=0x7fc6380cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/635917411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc63815ecf0 0x7fc6380cee50 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d28a0 tx=0x7fc62c01c2d0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- 172.21.15.82:0/635917411 shutdown_connections 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/635917411 >> v2:172.21.15.196:3300/0 conn(0x7fc6380cfec0 0x7fc6380d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/635917411 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc63815ecf0 0x7fc6380cee50 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- 172.21.15.82:0/635917411 >> 172.21.15.82:0/635917411 conn(0x7fc6380b52a0 msgr2=0x7fc6380b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.287 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- 172.21.15.82:0/635917411 shutdown_connections 2024-04-25T10:13:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- 172.21.15.82:0/635917411 wait complete. 2024-04-25T10:13:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 Processor -- start 2024-04-25T10:13:58.288 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.287+0000 7fc63d3e8640 1 -- start start 2024-04-25T10:13:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63806bf00 0x7fc6380b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63815ecf0 0x7fc6380cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d20a0 con 0x7fc6380cfec0 2024-04-25T10:13:58.290 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380ce930 con 0x7fc63815ecf0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380d2220 con 0x7fc63806bf00 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42218/0 (socket says 172.21.15.82:42218) 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 -- 172.21.15.82:0/1184118901 learned_addr learned my addr 172.21.15.82:0/1184118901 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc636ffd640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63806bf00 0x7fc6380b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6377fe640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63815ecf0 0x7fc6380cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 -- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63806bf00 msgr2=0x7fc6380b7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63806bf00 0x7fc6380b7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 -- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63815ecf0 msgr2=0x7fc6380cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63815ecf0 0x7fc6380cc6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 -- 172.21.15.82:0/1184118901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6380cb650 con 0x7fc6380cfec0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc6367fc640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c7f0 tx=0x7fc62800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.291 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628007a30 con 0x7fc6380cfec0 2024-04-25T10:13:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628007bd0 con 0x7fc6380cfec0 2024-04-25T10:13:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6380cb930 con 0x7fc6380cfec0 2024-04-25T10:13:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280113f0 con 0x7fc6380cfec0 2024-04-25T10:13:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6380cbef0 con 0x7fc6380cfec0 2024-04-25T10:13:58.292 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc638067c40 con 0x7fc6380cfec0 2024-04-25T10:13:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc628011590 con 0x7fc6380cfec0 2024-04-25T10:13:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 0x7fc624069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc6280854e0 con 0x7fc6380cfec0 2024-04-25T10:13:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc636ffd640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 0x7fc624069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.293 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.291+0000 7fc636ffd640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 0x7fc624069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0ad000 tx=0x7fc62c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.295 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.295+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc628015020 con 0x7fc6380cfec0 2024-04-25T10:13:58.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.443+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6380d1670 con 0x7fc6380cfec0 2024-04-25T10:13:58.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc61f7fe640 1 -- 172.21.15.82:0/1184118901 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+17091 (secure 0 0 0) 0x7fc628056280 con 0x7fc6380cfec0 2024-04-25T10:13:58.446 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:13:58.446 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":122,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:13:47.565030+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 msgr2=0x7fc624069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 0x7fc624069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0ad000 tx=0x7fc62c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 msgr2=0x7fc6380cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fc62800c7f0 tx=0x7fc62800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 shutdown_connections 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc624066f60 0x7fc624069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc63815ecf0 0x7fc6380cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc6380cfec0 0x7fc6380cd3d0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 --2- 172.21.15.82:0/1184118901 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc63806bf00 0x7fc6380b7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 >> 172.21.15.82:0/1184118901 conn(0x7fc6380b52a0 msgr2=0x7fc638157ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 shutdown_connections 2024-04-25T10:13:58.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.447+0000 7fc63d3e8640 1 -- 172.21.15.82:0/1184118901 wait complete. 2024-04-25T10:13:58.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:58.464 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:58.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 Processor -- start 2024-04-25T10:13:58.585 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 -- start start 2024-04-25T10:13:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7faef415c980 0x7faef415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faef4071110 0x7faef4071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faef40cda10 con 0x7faef4144e40 2024-04-25T10:13:58.587 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef415f260 con 0x7faef4071110 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faefa063640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faef415f820 con 0x7faef415c980 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef3fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7faef4071110 0x7faef4071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7faef4144e40 0x7faef40cd130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57904/0 (socket says 172.21.15.82:57904) 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef37fe640 1 -- 172.21.15.82:0/280561456 learned_addr learned my addr 172.21.15.82:0/280561456 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef2ffd640 1 --2- 172.21.15.82:0/280561456 >> v2:172.21.15.196:3300/0 conn(0x7faef415c980 0x7faef415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.588 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef3fff640 1 -- 172.21.15.82:0/280561456 >> v2:172.21.15.196:3300/0 conn(0x7faef415c980 msgr2=0x7faef415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef3fff640 1 --2- 172.21.15.82:0/280561456 >> v2:172.21.15.196:3300/0 conn(0x7faef415c980 0x7faef415fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef3fff640 1 -- 172.21.15.82:0/280561456 >> v1:172.21.15.82:6789/0 conn(0x7faef4144e40 legacy=0x7faef40cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef27fc640 1 -- 172.21.15.82:0/280561456 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17051013 0 0) 0x7faef40cda10 con 0x7faef4144e40 2024-04-25T10:13:58.589 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.587+0000 7faef3fff640 1 -- 172.21.15.82:0/280561456 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef40cdcf0 con 0x7faef4071110 2024-04-25T10:13:58.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef3fff640 1 --2- 172.21.15.82:0/280561456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4071110 0x7faef4071540 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7faef40721f0 tx=0x7faeec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c9d1abb4a2b997 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef27fc640 1 -- 172.21.15.82:0/280561456 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b2070 con 0x7faef4071110 2024-04-25T10:13:58.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef27fc640 1 -- 172.21.15.82:0/280561456 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec0a57f0 con 0x7faef4071110 2024-04-25T10:13:58.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef27fc640 1 -- 172.21.15.82:0/280561456 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec0a5bc0 con 0x7faef4071110 2024-04-25T10:13:58.590 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/280561456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4071110 msgr2=0x7faef4071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- 172.21.15.82:0/280561456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4071110 0x7faef4071540 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7faef40721f0 tx=0x7faeec01c930 comp rx=0 tx=0).stop 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/280561456 shutdown_connections 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- 172.21.15.82:0/280561456 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4071110 0x7faef4071540 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- 172.21.15.82:0/280561456 >> v2:172.21.15.196:3300/0 conn(0x7faef415c980 0x7faef415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/280561456 >> 172.21.15.82:0/280561456 conn(0x7faef4066fa0 msgr2=0x7faef406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/280561456 shutdown_connections 2024-04-25T10:13:58.591 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/280561456 wait complete. 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 Processor -- start 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- start start 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4071110 0x7faef413b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faef415c980 0x7faef4141dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef415d9d0 con 0x7faef4144e40 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faef415d850 con 0x7faef4071110 2024-04-25T10:13:58.594 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faef41601f0 con 0x7faef415c980 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42246/0 (socket says 172.21.15.82:42246) 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 -- 172.21.15.82:0/3866188733 learned_addr learned my addr 172.21.15.82:0/3866188733 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef37fe640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4071110 0x7faef413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef3fff640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faef415c980 0x7faef4141dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 -- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faef415c980 msgr2=0x7faef4141dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faef415c980 0x7faef4141dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 -- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4071110 msgr2=0x7faef413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4071110 0x7faef413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 -- 172.21.15.82:0/3866188733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4071e60 con 0x7faef4144e40 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faef2ffd640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7faee800d5f0 tx=0x7faee800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80125f0 con 0x7faef4144e40 2024-04-25T10:13:58.595 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faee8019070 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef4142de0 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.591+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee80129b0 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faef4142820 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faef4067c40 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faee8012b50 con 0x7faef4144e40 2024-04-25T10:13:58.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faedbfff640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 0x7faee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faef37fe640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 0x7faee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7faee8085530 con 0x7faef4144e40 2024-04-25T10:13:58.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.595+0000 7faef37fe640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 0x7faee0069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faee4000f70 tx=0x7faee4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.598 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.599+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faee80563f0 con 0x7faef4144e40 2024-04-25T10:13:58.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faef4072220 con 0x7faef4144e40 2024-04-25T10:13:58.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faedbfff640 1 -- 172.21.15.82:0/3866188733 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7faee8055cf0 con 0x7faef4144e40 2024-04-25T10:13:58.754 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 msgr2=0x7faee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 0x7faee0069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7faee4000f70 tx=0x7faee4007b60 comp rx=0 tx=0).stop 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 msgr2=0x7faef4143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7faee800d5f0 tx=0x7faee800dab0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 shutdown_connections 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7faee0066f60 0x7faee0069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7faef415c980 0x7faef4141dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7faef4144e40 0x7faef4143b30 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 --2- 172.21.15.82:0/3866188733 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7faef4071110 0x7faef413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 >> 172.21.15.82:0/3866188733 conn(0x7faef4066fa0 msgr2=0x7faef406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 shutdown_connections 2024-04-25T10:13:58.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.755+0000 7faefa063640 1 -- 172.21.15.82:0/3866188733 wait complete. 2024-04-25T10:13:58.771 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 Processor -- start 2024-04-25T10:13:58.881 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 -- start start 2024-04-25T10:13:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c700cee50 0x7f6c7006c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7006cc70 con 0x7f6c7015ecf0 2024-04-25T10:13:58.882 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c700ce090 con 0x7f6c700d05d0 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c787c9640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7015f0d0 con 0x7f6c700cee50 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76d66640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f6c700cee50 0x7f6c7006c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76565640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6c7015ecf0 0x7f6c700cfec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57918/0 (socket says 172.21.15.82:57918) 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76565640 1 -- 172.21.15.82:0/2146686137 learned_addr learned my addr 172.21.15.82:0/2146686137 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76d66640 1 -- 172.21.15.82:0/2146686137 >> v1:172.21.15.196:6789/0 conn(0x7f6c700d05d0 legacy=0x7f6c7006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76d66640 1 -- 172.21.15.82:0/2146686137 >> v1:172.21.15.82:6789/0 conn(0x7f6c7015ecf0 legacy=0x7f6c700cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76d66640 1 -- 172.21.15.82:0/2146686137 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700ce370 con 0x7f6c700cee50 2024-04-25T10:13:58.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c75563640 1 -- 172.21.15.82:0/2146686137 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428691376 0 0) 0x7f6c7006cc70 con 0x7f6c7015ecf0 2024-04-25T10:13:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.883+0000 7f6c76d66640 1 --2- 172.21.15.82:0/2146686137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c7006c610 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009d90 tx=0x7f6c6801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51535f95ac047bbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.884 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c75563640 1 -- 172.21.15.82:0/2146686137 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680b2070 con 0x7f6c700cee50 2024-04-25T10:13:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c75563640 1 -- 172.21.15.82:0/2146686137 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c680a58e0 con 0x7f6c700cee50 2024-04-25T10:13:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c75563640 1 -- 172.21.15.82:0/2146686137 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680a5cb0 con 0x7f6c700cee50 2024-04-25T10:13:58.885 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- 172.21.15.82:0/2146686137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 msgr2=0x7f6c7006c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 --2- 172.21.15.82:0/2146686137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c7006c610 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f6c68009d90 tx=0x7f6c6801c930 comp rx=0 tx=0).stop 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- 172.21.15.82:0/2146686137 shutdown_connections 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 --2- 172.21.15.82:0/2146686137 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c7006c610 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- 172.21.15.82:0/2146686137 >> 172.21.15.82:0/2146686137 conn(0x7f6c700b52a0 msgr2=0x7f6c700b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- 172.21.15.82:0/2146686137 shutdown_connections 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- 172.21.15.82:0/2146686137 wait complete. 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 Processor -- start 2024-04-25T10:13:58.886 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- start start 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c700d05d0 0x7f6c70153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 0x7f6c70153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7016ba00 con 0x7f6c700cee50 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7016b880 con 0x7f6c7015ecf0 2024-04-25T10:13:58.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c787c9640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7016bb80 con 0x7f6c700d05d0 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42274/0 (socket says 172.21.15.82:42274) 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 -- 172.21.15.82:0/4150399292 learned_addr learned my addr 172.21.15.82:0/4150399292 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76d66640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 0x7f6c70153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c75d64640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c700d05d0 0x7f6c70153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 -- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c700d05d0 msgr2=0x7f6c70153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c700d05d0 0x7f6c70153570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 -- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 msgr2=0x7f6c70153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 0x7f6c70153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 -- 172.21.15.82:0/4150399292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700cb310 con 0x7f6c700cee50 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76d66640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 0x7f6c70153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c76565640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c4b0 tx=0x7f6c6c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c018070 con 0x7f6c700cee50 2024-04-25T10:13:58.890 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c007500 con 0x7f6c700cee50 2024-04-25T10:13:58.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.887+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004a30 con 0x7f6c700cee50 2024-04-25T10:13:58.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c700cb5b0 con 0x7f6c700cee50 2024-04-25T10:13:58.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c700cbb70 con 0x7f6c700cee50 2024-04-25T10:13:58.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c70067c40 con 0x7f6c700cee50 2024-04-25T10:13:58.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c6c004bd0 con 0x7f6c700cee50 2024-04-25T10:13:58.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c66ffd640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c75d64640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6c6c0843f0 con 0x7f6c700cee50 2024-04-25T10:13:58.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.891+0000 7f6c75d64640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 0x7f6c54069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c700cd920 tx=0x7f6c60009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:58.895+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c6c055220 con 0x7f6c700cee50 2024-04-25T10:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.039+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6c700d1670 con 0x7f6c700cee50 2024-04-25T10:13:59.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.039+0000 7f6c66ffd640 1 -- 172.21.15.82:0/4150399292 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v122) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6c6c054b20 con 0x7f6c700cee50 2024-04-25T10:13:59.040 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 msgr2=0x7f6c54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 0x7f6c54069420 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6c700cd920 tx=0x7f6c60009040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 msgr2=0x7f6c700c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c4b0 tx=0x7f6c6c00c970 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 shutdown_connections 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6c54066f60 0x7f6c54069420 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6c700d05d0 0x7f6c70153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6c7015ecf0 0x7f6c70153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 --2- 172.21.15.82:0/4150399292 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6c700cee50 0x7f6c700c9150 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 >> 172.21.15.82:0/4150399292 conn(0x7f6c700b52a0 msgr2=0x7f6c700b5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 shutdown_connections 2024-04-25T10:13:59.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.043+0000 7f6c787c9640 1 -- 172.21.15.82:0/4150399292 wait complete. 2024-04-25T10:13:59.057 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:59.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.175+0000 7f9e57dd0640 1 Processor -- start 2024-04-25T10:13:59.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.175+0000 7f9e57dd0640 1 -- start start 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e57dd0640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 0x7f9e5015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e57dd0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e50071110 0x7f9e50071540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e57dd0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e500cda10 con 0x7f9e50144e40 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e57dd0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5015f260 con 0x7f9e50071110 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e57dd0640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5015f820 con 0x7f9e5015c980 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9e50071110 0x7f9e50071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e55b6c640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f9e50144e40 0x7f9e500cd130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:57920/0 (socket says 172.21.15.82:57920) 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e55b6c640 1 -- 172.21.15.82:0/2744885257 learned_addr learned my addr 172.21.15.82:0/2744885257 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5536b640 1 --2- 172.21.15.82:0/2744885257 >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 0x7f9e5015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 -- 172.21.15.82:0/2744885257 >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 msgr2=0x7f9e5015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 --2- 172.21.15.82:0/2744885257 >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 0x7f9e5015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 -- 172.21.15.82:0/2744885257 >> v1:172.21.15.82:6789/0 conn(0x7f9e50144e40 legacy=0x7f9e500cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 -- 172.21.15.82:0/2744885257 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e500cdcf0 con 0x7f9e50071110 2024-04-25T10:13:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e54b6a640 1 -- 172.21.15.82:0/2744885257 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983529842 0 0) 0x7f9e500cda10 con 0x7f9e50144e40 2024-04-25T10:13:59.179 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5536b640 1 --2- 172.21.15.82:0/2744885257 >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 0x7f9e5015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e5636d640 1 --2- 172.21.15.82:0/2744885257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e50071540 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f9e500721f0 tx=0x7f9e4801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73431f0f2a4d65cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e54b6a640 1 -- 172.21.15.82:0/2744885257 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480b2070 con 0x7f9e50071110 2024-04-25T10:13:59.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e54b6a640 1 -- 172.21.15.82:0/2744885257 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480a57f0 con 0x7f9e50071110 2024-04-25T10:13:59.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.179+0000 7f9e54b6a640 1 -- 172.21.15.82:0/2744885257 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e480a5bc0 con 0x7f9e50071110 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/2744885257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 msgr2=0x7f9e50071540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/2744885257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e50071540 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7f9e500721f0 tx=0x7f9e4801c930 comp rx=0 tx=0).stop 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/2744885257 shutdown_connections 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/2744885257 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e50071540 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/2744885257 >> v2:172.21.15.196:3300/0 conn(0x7f9e5015c980 0x7f9e5015fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/2744885257 >> 172.21.15.82:0/2744885257 conn(0x7f9e50066fa0 msgr2=0x7f9e5006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/2744885257 shutdown_connections 2024-04-25T10:13:59.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/2744885257 wait complete. 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 Processor -- start 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- start start 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e50144e40 0x7f9e50165ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5015c980 0x7f9e50164170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5015d9d0 con 0x7f9e50071110 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5015d850 con 0x7f9e5015c980 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e501601f0 con 0x7f9e50144e40 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.184 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42294/0 (socket says 172.21.15.82:42294) 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 -- 172.21.15.82:0/1438388317 learned_addr learned my addr 172.21.15.82:0/1438388317 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e5636d640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5015c980 0x7f9e50164170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 -- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e50144e40 msgr2=0x7f9e50165ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e5536b640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e50144e40 0x7f9e50165ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e50144e40 0x7f9e50165ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 -- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5015c980 msgr2=0x7f9e50164170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5015c980 0x7f9e50164170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 -- 172.21.15.82:0/1438388317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e50071e60 con 0x7f9e50071110 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e55b6c640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0011f0 tx=0x7f9e4c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c018070 con 0x7f9e50071110 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c0074e0 con 0x7f9e50071110 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e50164900 con 0x7f9e50071110 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c004a10 con 0x7f9e50071110 2024-04-25T10:13:59.185 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e50164ec0 con 0x7f9e50071110 2024-04-25T10:13:59.186 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.183+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e50067c40 con 0x7f9e50071110 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.187+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9e4c01b070 con 0x7f9e50071110 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.187+0000 7f9e467fc640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.187+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9e4c084530 con 0x7f9e50071110 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.187+0000 7f9e5536b640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.187 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.187+0000 7f9e5536b640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 0x7f9e34069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006fd0 tx=0x7f9e40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.191+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9e4c055360 con 0x7f9e50071110 2024-04-25T10:13:59.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e50072220 con 0x7f9e50071110 2024-04-25T10:13:59.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e467fc640 1 -- 172.21.15.82:0/1438388317 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v122) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e4c054c60 con 0x7f9e50071110 2024-04-25T10:13:59.335 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:13:59.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 msgr2=0x7f9e34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 0x7f9e34069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f9e40006fd0 tx=0x7f9e40008040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 msgr2=0x7f9e5013b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0011f0 tx=0x7f9e4c00c9b0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 shutdown_connections 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9e34066f60 0x7f9e34069420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.335+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9e50144e40 0x7f9e50165ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.339+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9e5015c980 0x7f9e50164170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.339+0000 7f9e57dd0640 1 --2- 172.21.15.82:0/1438388317 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9e50071110 0x7f9e5013b030 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.339+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 >> 172.21.15.82:0/1438388317 conn(0x7f9e50066fa0 msgr2=0x7f9e5006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.339+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 shutdown_connections 2024-04-25T10:13:59.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.339+0000 7f9e57dd0640 1 -- 172.21.15.82:0/1438388317 wait complete. 2024-04-25T10:13:59.352 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:59.465 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 Processor -- start 2024-04-25T10:13:59.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 -- start start 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7ed015e9f0 0x7f7ed00cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed015f050 con 0x7f7ed00ce7a0 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed00b6ae0 con 0x7f7ed015e9f0 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ed5d6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed006c1c0 con 0x7f7ed00d32d0 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3301/0 says I am v2:172.21.15.82:33140/0 (socket says 172.21.15.82:33140) 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 -- 172.21.15.82:0/385848138 learned_addr learned my addr 172.21.15.82:0/385848138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:59.467 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecffff640 1 --2- 172.21.15.82:0/385848138 >> v2:172.21.15.196:3300/0 conn(0x7f7ed015e9f0 0x7f7ed00cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 -- 172.21.15.82:0/385848138 >> v2:172.21.15.196:3300/0 conn(0x7f7ed015e9f0 msgr2=0x7f7ed00cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 --2- 172.21.15.82:0/385848138 >> v2:172.21.15.196:3300/0 conn(0x7f7ed015e9f0 0x7f7ed00cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 -- 172.21.15.82:0/385848138 >> v1:172.21.15.196:6789/0 conn(0x7f7ed00ce7a0 legacy=0x7f7ed015e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.468 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.467+0000 7f7ecf7fe640 1 -- 172.21.15.82:0/385848138 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed006c410 con 0x7f7ed00d32d0 2024-04-25T10:13:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ecf7fe640 1 --2- 172.21.15.82:0/385848138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009870 tx=0x7f7ec001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1a3a1ad753baf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ece7fc640 1 -- 172.21.15.82:0/385848138 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00b5070 con 0x7f7ed00d32d0 2024-04-25T10:13:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ece7fc640 1 -- 172.21.15.82:0/385848138 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec00a8a50 con 0x7f7ed00d32d0 2024-04-25T10:13:59.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ece7fc640 1 -- 172.21.15.82:0/385848138 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec00a8e20 con 0x7f7ed00d32d0 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/385848138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 msgr2=0x7f7ed006cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/385848138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009870 tx=0x7f7ec001ed50 comp rx=0 tx=0).stop 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/385848138 shutdown_connections 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/385848138 >> v2:172.21.15.196:3300/0 conn(0x7f7ed015e9f0 0x7f7ed00cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/385848138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed006cef0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/385848138 >> 172.21.15.82:0/385848138 conn(0x7f7ed00d39f0 msgr2=0x7f7ed00d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/385848138 shutdown_connections 2024-04-25T10:13:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/385848138 wait complete. 2024-04-25T10:13:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 Processor -- start 2024-04-25T10:13:59.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.471+0000 7f7ed5d6c640 1 -- start start 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed00ce7a0 0x7f7ed00cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed00c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00d36b0 con 0x7f7ed015e9f0 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed006c9d0 con 0x7f7ed00ce7a0 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed00d1bc0 con 0x7f7ed00d32d0 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.475 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:42310/0 (socket says 172.21.15.82:42310) 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 -- 172.21.15.82:0/1785408779 learned_addr learned my addr 172.21.15.82:0/1785408779 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecf7fe640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed00ce7a0 0x7f7ed00cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eceffd640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed00c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 -- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 msgr2=0x7f7ed00c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed00c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 -- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed00ce7a0 msgr2=0x7f7ed00cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed00ce7a0 0x7f7ed00cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 -- 172.21.15.82:0/1785408779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0158d70 con 0x7f7ed015e9f0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecffff640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009d30 tx=0x7f7ec800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8018020 con 0x7f7ed015e9f0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8014480 con 0x7f7ed015e9f0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed0159010 con 0x7f7ed015e9f0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8014870 con 0x7f7ed015e9f0 2024-04-25T10:13:59.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed01595d0 con 0x7f7ed015e9f0 2024-04-25T10:13:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed0067c40 con 0x7f7ed015e9f0 2024-04-25T10:13:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ec801b020 con 0x7f7ed015e9f0 2024-04-25T10:13:59.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 0x7f7ebc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7ec8085ba0 con 0x7f7ed015e9f0 2024-04-25T10:13:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.475+0000 7f7ecf7fe640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 0x7f7ebc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.479+0000 7f7ecf7fe640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 0x7f7ebc069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b2000 tx=0x7f7ec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.479+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ec8022070 con 0x7f7ed015e9f0 2024-04-25T10:13:59.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.647+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7ed015b9e0 con 0x7f7ed015e9f0 2024-04-25T10:13:59.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.647+0000 7f7eb7fff640 1 -- 172.21.15.82:0/1785408779 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v122) v1 ==== 80+0+120 (secure 0 0 0) 0x7f7ec8010040 con 0x7f7ed015e9f0 2024-04-25T10:13:59.649 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:13:59.649 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6835/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:13:59.649 INFO:teuthology.orchestra.run.smithi082.stderr:listed 2 entries 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 msgr2=0x7f7ebc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 0x7f7ebc069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7ec00b2000 tx=0x7f7ec0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 msgr2=0x7f7ed00c79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009d30 tx=0x7f7ec800c490 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 shutdown_connections 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7ebc066f60 0x7f7ebc069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7ed00d32d0 0x7f7ed00c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7ed015e9f0 0x7f7ed00c79d0 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 --2- 172.21.15.82:0/1785408779 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7ed00ce7a0 0x7f7ed00cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 >> 172.21.15.82:0/1785408779 conn(0x7f7ed00d39f0 msgr2=0x7f7ed015bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 shutdown_connections 2024-04-25T10:13:59.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:13:59.651+0000 7f7ed5d6c640 1 -- 172.21.15.82:0/1785408779 wait complete. 2024-04-25T10:13:59.666 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:13:59.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:59.667 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:59.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[55702]: fuse finished with error 0 and tester_r 0 2024-04-25T10:13:59.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:05.781 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:14:05.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:05.781 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:05.789 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:05.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.963+0000 7fe781102640 1 Processor -- start 2024-04-25T10:14:05.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.963+0000 7fe781102640 1 -- start start 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe781102640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe77c071e90 0x7fe77c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe781102640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 0x7fe77c06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe781102640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77c0b3010 con 0x7fe77c143f50 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe781102640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77c15ef10 con 0x7fe77c071e90 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe781102640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0b2dc0 con 0x7fe77c06e260 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77ad9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe77c143f50 0x7fe77c15e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59144/0 (socket says 172.21.15.82:59144) 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77ad9d640 1 -- 172.21.15.82:0/816441381 learned_addr learned my addr 172.21.15.82:0/816441381 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 --2- 172.21.15.82:0/816441381 >> v2:172.21.15.196:3300/0 conn(0x7fe77c071e90 0x7fe77c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77b59e640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 0x7fe77c06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773679919 0 0) 0x7fe77c0b3010 con 0x7fe77c143f50 2024-04-25T10:14:05.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe764003530 con 0x7fe77c143f50 2024-04-25T10:14:05.967 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987966423 0 0) 0x7fe764003530 con 0x7fe77c143f50 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 msgr2=0x7fe77c06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 0x7fe77c06e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 >> v2:172.21.15.196:3300/0 conn(0x7fe77c071e90 msgr2=0x7fe77c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 --2- 172.21.15.82:0/816441381 >> v2:172.21.15.196:3300/0 conn(0x7fe77c071e90 0x7fe77c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c0b32f0 con 0x7fe77c143f50 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe76c001680 con 0x7fe77c143f50 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 >> v1:172.21.15.82:6789/0 conn(0x7fe77c143f50 legacy=0x7fe77c15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 0x7fe7640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe764004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe764004d50 0x7fe7640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0b2dc0 con 0x7fe764003f40 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76400b9d0 con 0x7fe764004d50 2024-04-25T10:14:05.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76400bb50 con 0x7fe77c06e260 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe764004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77b59e640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe764004d50 0x7fe7640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 -- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 msgr2=0x7fe7640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c06e260 0x7fe7640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 -- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe764004d50 msgr2=0x7fe7640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe764004d50 0x7fe7640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 -- 172.21.15.82:0/816441381 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764003530 con 0x7fe764003f40 2024-04-25T10:14:05.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77b59e640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe764004d50 0x7fe7640051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:05.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe77a59c640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe764004810 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fe770009ab0 tx=0x7fe77001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f55800a4f3ed184 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700b2070 con 0x7fe764003f40 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700a5680 con 0x7fe764003f40 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.967+0000 7fe779d9b640 1 -- 172.21.15.82:0/816441381 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7700096b0 con 0x7fe764003f40 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- 172.21.15.82:0/816441381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 msgr2=0x7fe764004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 --2- 172.21.15.82:0/816441381 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe764004810 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7fe770009ab0 tx=0x7fe77001ca30 comp rx=0 tx=0).stop 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77b59e640 1 -- 172.21.15.82:0/816441381 reap_dead start 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- 172.21.15.82:0/816441381 shutdown_connections 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- 172.21.15.82:0/816441381 >> 172.21.15.82:0/816441381 conn(0x7fe77c066fa0 msgr2=0x7fe77c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- 172.21.15.82:0/816441381 shutdown_connections 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- 172.21.15.82:0/816441381 wait complete. 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 Processor -- start 2024-04-25T10:14:05.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- start start 2024-04-25T10:14:05.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 0x7fe77c0a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c1e4ff0 0x7fe77c1e5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c06e6e0 con 0x7fe764003f40 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c0b3940 con 0x7fe77c0a96c0 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe781102640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe77c06e860 con 0x7fe77c1e4ff0 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 0x7fe77c0a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77a59c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 0x7fe77c0a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:47472/0 (socket says 172.21.15.82:47472) 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77b59e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c1e4ff0 0x7fe77c1e5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77a59c640 1 -- 172.21.15.82:0/451467529 learned_addr learned my addr 172.21.15.82:0/451467529 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41556/0 (socket says 172.21.15.82:41556) 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 -- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c1e4ff0 msgr2=0x7fe77c1e5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c1e4ff0 0x7fe77c1e5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 -- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 msgr2=0x7fe77c0a9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 0x7fe77c0a9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 -- 172.21.15.82:0/451467529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77c1e68a0 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe77ad9d640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe76c004880 tx=0x7fe76c00f820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c007d00 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76c00a0d0 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.971+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00a4c0 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77c1e5d20 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77c1e62e0 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe77c0ac820 con 0x7fe764003f40 2024-04-25T10:14:05.975 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe76c014070 con 0x7fe764003f40 2024-04-25T10:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe7637fe640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 0x7fe768069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe76c087a20 con 0x7fe764003f40 2024-04-25T10:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe77a59c640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 0x7fe768069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:05.976 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.975+0000 7fe77a59c640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 0x7fe768069450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe7700046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:05.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:05.979+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe76c00fe20 con 0x7fe764003f40 2024-04-25T10:14:06.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.167+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe77c067c40 con 0x7fe764003f40 2024-04-25T10:14:06.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.167+0000 7fe7637fe640 1 -- 172.21.15.82:0/451467529 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fe76c058200 con 0x7fe764003f40 2024-04-25T10:14:06.169 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:06.169 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:13:50.635981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":122,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9849},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9849":{"gid":9849,"name":"c","rank":0,"incarnation":94,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/3303693719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3303693719},{"type":"v1","addr":"172.21.15.82:6837","nonce":3303693719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:06.169 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 97 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 msgr2=0x7fe768069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 0x7fe768069450 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fe7700b0040 tx=0x7fe7700046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 msgr2=0x7fe77c0aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7fe76c004880 tx=0x7fe76c00f820 comp rx=0 tx=0).stop 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 shutdown_connections 2024-04-25T10:14:06.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe768066f90 0x7fe768069450 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe77c1e4ff0 0x7fe77c1e5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe77c0a96c0 0x7fe77c0a9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 --2- 172.21.15.82:0/451467529 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe764003f40 0x7fe77c0aa920 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 >> 172.21.15.82:0/451467529 conn(0x7fe77c066fa0 msgr2=0x7fe77c0b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 shutdown_connections 2024-04-25T10:14:06.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.171+0000 7fe781102640 1 -- 172.21.15.82:0/451467529 wait complete. 2024-04-25T10:14:06.187 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json tag path /parent mytag123 2024-04-25T10:14:06.311 INFO:teuthology.orchestra.run.smithi082.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T10:14:06.311 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T10:14:06.312 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.347+0000 7ff867c9a080 1 Processor -- start 2024-04-25T10:14:06.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.347+0000 7ff867c9a080 1 -- start start 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1c46070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 0x561ae1b8b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ae1bbad30 con 0x561ae1bb7ee0 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ae1b329f0 con 0x561ae1c49260 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ae1bbaeb0 con 0x561ae1c49640 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41578/0 (socket says 172.21.15.82:41578) 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/332373341 learned_addr learned my addr 172.21.15.82:0/332373341 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff864fb7640 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1c46070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 msgr2=0x561ae1b8b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 0x561ae1b8b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 msgr2=0x561ae1c46070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1c46070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/332373341 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ae1c465b0 con 0x561ae1bb7ee0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff864fb7640 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1c46070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff85800a720 tx=0x7ff85800abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ade6b2c155b46a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8577fe640 1 -- 172.21.15.82:0/332373341 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85801a070 con 0x561ae1bb7ee0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8577fe640 1 -- 172.21.15.82:0/332373341 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85800ae30 con 0x561ae1bb7ee0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8577fe640 1 -- 172.21.15.82:0/332373341 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858008b90 con 0x561ae1bb7ee0 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- 172.21.15.82:0/332373341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 msgr2=0x561ae1c61ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7ff85800a720 tx=0x7ff85800abe0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- 172.21.15.82:0/332373341 shutdown_connections 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 0x561ae1b8b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1c46070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- 172.21.15.82:0/332373341 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c61ee0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- 172.21.15.82:0/332373341 >> 172.21.15.82:0/332373341 conn(0x561ae1bb6ac0 msgr2=0x561ae1c48300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- 172.21.15.82:0/332373341 shutdown_connections 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- 172.21.15.82:0/332373341 wait complete. 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 Processor -- start 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- start start 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c45370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1b8b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 0x561ae1b8b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ae1c636e0 con 0x561ae1bb7ee0 2024-04-25T10:14:06.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ae1c60fc0 con 0x561ae1c49260 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff867c9a080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ae1c61260 con 0x561ae1c49640 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c45370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c45370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41582/0 (socket says 172.21.15.82:41582) 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/3419815014 learned_addr learned my addr 172.21.15.82:0/3419815014 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff864fb7640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1b8b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 msgr2=0x561ae1b8b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x561ae1c49640 0x561ae1b8b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 msgr2=0x561ae1b8b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1b8b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 -- 172.21.15.82:0/3419815014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561ae1bd2880 con 0x561ae1bb7ee0 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff864fb7640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x561ae1c49260 0x561ae1b8b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8657b8640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c45370 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff858008c40 tx=0x7ff858003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.351+0000 7ff8567fc640 1 -- 172.21.15.82:0/3419815014 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85801a040 con 0x561ae1bb7ee0 2024-04-25T10:14:06.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff8567fc640 1 -- 172.21.15.82:0/3419815014 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff858008e20 con 0x561ae1bb7ee0 2024-04-25T10:14:06.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ae1bd1960 con 0x561ae1bb7ee0 2024-04-25T10:14:06.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff8567fc640 1 -- 172.21.15.82:0/3419815014 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff858020650 con 0x561ae1bb7ee0 2024-04-25T10:14:06.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ae1bd1ea0 con 0x561ae1bb7ee0 2024-04-25T10:14:06.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff8567fc640 1 -- 172.21.15.82:0/3419815014 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8580207f0 con 0x561ae1bb7ee0 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff8567fc640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff848065970 0x7ff848067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff8567fc640 1 -- 172.21.15.82:0/3419815014 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff8580853c0 con 0x561ae1bb7ee0 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff864fb7640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff848065970 0x7ff848067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x561ae1bd06e0 0x561ae1bd0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x561ae1c458b0 con 0x561ae1bd06e0 2024-04-25T10:14:06.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff865fb9640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x561ae1bd06e0 0x561ae1bd0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff864fb7640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff848065970 0x7ff848067e30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0ac000 tx=0x7ff85c0a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff865fb9640 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x561ae1bd06e0 0x561ae1bd0ac0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff865fb9640 1 -- 172.21.15.82:0/3419815014 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff860006040 con 0x561ae1bd06e0 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x561ae1bd06e0 msgr2=0x561ae1bd0ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x561ae1bd06e0 0x561ae1bd0ac0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff848065970 msgr2=0x7ff848067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff848065970 0x7ff848067e30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0ac000 tx=0x7ff85c0a5000 comp rx=0 tx=0).stop 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 msgr2=0x561ae1c45370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 --2- 172.21.15.82:0/3419815014 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x561ae1bb7ee0 0x561ae1c45370 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7ff858008c40 tx=0x7ff858003030 comp rx=0 tx=0).stop 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff865fb9640 1 -- 172.21.15.82:0/3419815014 reap_dead start 2024-04-25T10:14:06.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 shutdown_connections 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 >> 172.21.15.82:0/3419815014 conn(0x561ae1bb6ac0 msgr2=0x561ae1c66170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 shutdown_connections 2024-04-25T10:14:06.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.355+0000 7ff867c9a080 1 -- 172.21.15.82:0/3419815014 wait complete. 2024-04-25T10:14:06.362 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T10:14:06.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 Processor -- start 2024-04-25T10:14:06.437 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- start start 2024-04-25T10:14:06.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1225692e0 0x55c12253fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c12256c180 0x55c12256b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1224e29f0 con 0x55c1225696c0 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1224e6950 con 0x55c1225692e0 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1225874f0 con 0x55c12256c180 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41590/0 (socket says 172.21.15.82:41590) 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 -- 172.21.15.82:0/1361414686 learned_addr learned my addr 172.21.15.82:0/1361414686 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7664cdb640 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c12256c180 0x55c12256b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76644da640 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1225692e0 0x55c12253fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 -- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c12256c180 msgr2=0x55c12256b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c12256c180 0x55c12256b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 -- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1225692e0 msgr2=0x55c12253fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1225692e0 0x55c12253fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 -- 172.21.15.82:0/1361414686 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1225f8950 con 0x55c1225696c0 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f765800b2d0 tx=0x7f765800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9f00156f015343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7662cd7640 1 -- 172.21.15.82:0/1361414686 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800a040 con 0x55c1225696c0 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7662cd7640 1 -- 172.21.15.82:0/1361414686 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765800f620 con 0x55c1225696c0 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f7662cd7640 1 -- 172.21.15.82:0/1361414686 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765800fa10 con 0x55c1225696c0 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- 172.21.15.82:0/1361414686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 msgr2=0x55c12256b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7f765800b2d0 tx=0x7f765800b790 comp rx=0 tx=0).stop 2024-04-25T10:14:06.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- 172.21.15.82:0/1361414686 shutdown_connections 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c12256c180 0x55c12256b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225696c0 0x55c12256b100 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 --2- 172.21.15.82:0/1361414686 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c1225692e0 0x55c12253fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- 172.21.15.82:0/1361414686 >> 172.21.15.82:0/1361414686 conn(0x55c122567c50 msgr2=0x55c12253c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- 172.21.15.82:0/1361414686 shutdown_connections 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- 172.21.15.82:0/1361414686 wait complete. 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 Processor -- start 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.439+0000 7f76669bc080 1 -- start start 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 0x55c12257f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1225696c0 0x55c12261a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c12256c180 0x55c12253e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12253f790 con 0x55c1225692e0 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c12253f4f0 con 0x55c12256c180 2024-04-25T10:14:06.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c12253fa30 con 0x55c1225696c0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 0x55c12257f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 0x55c12257f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41598/0 (socket says 172.21.15.82:41598) 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 -- 172.21.15.82:0/1359809851 learned_addr learned my addr 172.21.15.82:0/1359809851 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7664cdb640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c12256c180 0x55c12253e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1225696c0 0x55c12261a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 -- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1225696c0 msgr2=0x55c12261a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55c1225696c0 0x55c12261a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 -- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c12256c180 msgr2=0x55c12253e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55c12256c180 0x55c12253e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 -- 172.21.15.82:0/1359809851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c12257e4e0 con 0x55c1225692e0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76644da640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 0x55c12257f400 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f765400cf30 tx=0x7f765400a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 -- 172.21.15.82:0/1359809851 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400e660 con 0x55c1225692e0 2024-04-25T10:14:06.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 -- 172.21.15.82:0/1359809851 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f765400e800 con 0x55c1225692e0 2024-04-25T10:14:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c12257e780 con 0x55c1225692e0 2024-04-25T10:14:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 -- 172.21.15.82:0/1359809851 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f765400ebb0 con 0x55c1225692e0 2024-04-25T10:14:06.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c12257ecc0 con 0x55c1225692e0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 -- 172.21.15.82:0/1359809851 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f765401f3f0 con 0x55c1225692e0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7650065970 0x7f7650067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7661cd5640 1 -- 172.21.15.82:0/1359809851 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f7654084480 con 0x55c1225692e0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7650065970 0x7f7650067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55c12253eef0 0x55c12257cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55c12257d490 con 0x55c12253eef0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7664cdb640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55c12253eef0 0x55c12257cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7663cd9640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7650065970 0x7f7650067e30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7658001360 tx=0x7f7658009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7664cdb640 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55c12253eef0 0x55c12257cf50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7664cdb640 1 -- 172.21.15.82:0/1359809851 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f765c0ab620 con 0x55c12253eef0 2024-04-25T10:14:06.448 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55c12253eef0 msgr2=0x55c12257cf50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55c12253eef0 0x55c12257cf50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7650065970 msgr2=0x7f7650067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7650065970 0x7f7650067e30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f7658001360 tx=0x7f7658009000 comp rx=0 tx=0).stop 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 msgr2=0x55c12257f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 --2- 172.21.15.82:0/1359809851 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55c1225692e0 0x55c12257f400 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f765400cf30 tx=0x7f765400a780 comp rx=0 tx=0).stop 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f7664cdb640 1 -- 172.21.15.82:0/1359809851 reap_dead start 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 shutdown_connections 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.443+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 >> 172.21.15.82:0/1359809851 conn(0x55c122567c50 msgr2=0x55c1225f7640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.447+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 shutdown_connections 2024-04-25T10:14:06.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.447+0000 7f76669bc080 1 -- 172.21.15.82:0/1359809851 wait complete. 2024-04-25T10:14:06.450 INFO:teuthology.orchestra.run.smithi082.stdout:mytag123 2024-04-25T10:14:06.450 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 Processor -- start 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- start start 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acdc690 0x560f0ad8a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0ad81070 0x560f0ad82c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad81ff0 con 0x560f0acf4b60 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad81e70 con 0x560f0acdc690 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad82170 con 0x560f0ad81070 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41612/0 (socket says 172.21.15.82:41612) 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 -- 172.21.15.82:0/5155732 learned_addr learned my addr 172.21.15.82:0/5155732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4b3b1640 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acdc690 0x560f0ad8a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 -- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0ad81070 msgr2=0x560f0ad82c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4bbb2640 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0ad81070 0x560f0ad82c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0ad81070 0x560f0ad82c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 -- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acdc690 msgr2=0x560f0ad8a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acdc690 0x560f0ad8a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 -- 172.21.15.82:0/5155732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0ad801e0 con 0x560f0acf4b60 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4abb0640 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7efd40009640 tx=0x7efd40009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46933a7d3d7e2e1f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd49bae640 1 -- 172.21.15.82:0/5155732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40006660 con 0x560f0acf4b60 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd49bae640 1 -- 172.21.15.82:0/5155732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40006800 con 0x560f0acf4b60 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd49bae640 1 -- 172.21.15.82:0/5155732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40006bf0 con 0x560f0acf4b60 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- 172.21.15.82:0/5155732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 msgr2=0x560f0acaf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7efd40009640 tx=0x7efd40009b00 comp rx=0 tx=0).stop 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- 172.21.15.82:0/5155732 shutdown_connections 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0ad81070 0x560f0ad82c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0acf4b60 0x560f0acaf2d0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 --2- 172.21.15.82:0/5155732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acdc690 0x560f0ad8a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- 172.21.15.82:0/5155732 >> 172.21.15.82:0/5155732 conn(0x560f0acd6ab0 msgr2=0x560f0ad8b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- 172.21.15.82:0/5155732 shutdown_connections 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.531+0000 7efd4d893080 1 -- 172.21.15.82:0/5155732 wait complete. 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 Processor -- start 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- start start 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0acdc690 0x560f0ad725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acf4b60 0x560f0ad72f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 0x560f0ad7f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad80720 con 0x560f0ad81070 2024-04-25T10:14:06.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad80480 con 0x560f0acf4b60 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f0ad7d3f0 con 0x560f0acdc690 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 0x560f0ad7f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 0x560f0ad7f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41614/0 (socket says 172.21.15.82:41614) 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 -- 172.21.15.82:0/1634202059 learned_addr learned my addr 172.21.15.82:0/1634202059 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4abb0640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acf4b60 0x560f0ad72f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4b3b1640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0acdc690 0x560f0ad725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 -- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0acdc690 msgr2=0x560f0ad725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x560f0acdc690 0x560f0ad725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 -- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acf4b60 msgr2=0x560f0ad72f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acf4b60 0x560f0ad72f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 -- 172.21.15.82:0/1634202059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f0ad7bd60 con 0x560f0ad81070 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4abb0640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x560f0acf4b60 0x560f0ad72f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4bbb2640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 0x560f0ad7f020 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7efd4400a420 tx=0x7efd4400a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 -- 172.21.15.82:0/1634202059 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44015070 con 0x560f0ad81070 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 -- 172.21.15.82:0/1634202059 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd44011870 con 0x560f0ad81070 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560f0ad7b0e0 con 0x560f0ad81070 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 -- 172.21.15.82:0/1634202059 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44011c60 con 0x560f0ad81070 2024-04-25T10:14:06.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f0ad7b620 con 0x560f0ad81070 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 -- 172.21.15.82:0/1634202059 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd44018070 con 0x560f0ad81070 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd38065970 0x7efd38067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd48bac640 1 -- 172.21.15.82:0/1634202059 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efd44083f10 con 0x560f0ad81070 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4b3b1640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd38065970 0x7efd38067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x560f0ae1cc10 0x560f0ae1cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.535+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x560f0ae1d530 con 0x560f0ae1cc10 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4b3b1640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd38065970 0x7efd38067e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0ac000 tx=0x7efd3c0a5000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4abb0640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x560f0ae1cc10 0x560f0ae1cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4abb0640 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x560f0ae1cc10 0x560f0ae1cff0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4abb0640 1 -- 172.21.15.82:0/1634202059 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7efd4001b040 con 0x560f0ae1cc10 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x560f0ae1cc10 msgr2=0x560f0ae1cff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x560f0ae1cc10 0x560f0ae1cff0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd38065970 msgr2=0x7efd38067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd38065970 0x7efd38067e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0ac000 tx=0x7efd3c0a5000 comp rx=0 tx=0).stop 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 msgr2=0x560f0ad7f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 --2- 172.21.15.82:0/1634202059 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x560f0ad81070 0x560f0ad7f020 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7efd4400a420 tx=0x7efd4400a8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4bbb2640 1 -- 172.21.15.82:0/1634202059 reap_dead start 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 shutdown_connections 2024-04-25T10:14:06.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 >> 172.21.15.82:0/1634202059 conn(0x560f0acd6ab0 msgr2=0x560f0ad8b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 shutdown_connections 2024-04-25T10:14:06.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.539+0000 7efd4d893080 1 -- 172.21.15.82:0/1634202059 wait complete. 2024-04-25T10:14:06.543 INFO:teuthology.orchestra.run.smithi082.stdout:mytag123 2024-04-25T10:14:06.544 DEBUG:teuthology.orchestra.run.smithi082:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T10:14:06.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 Processor -- start 2024-04-25T10:14:06.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- start start 2024-04-25T10:14:06.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a48332d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4832b8380 0x55a48332f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4832189f0 con 0x55a483273570 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a48321c950 con 0x55a4832b8380 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a483347f60 con 0x7efd24002cb0 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41638/0 (socket says 172.21.15.82:41638) 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 -- 172.21.15.82:0/2599221781 learned_addr learned my addr 172.21.15.82:0/2599221781 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 -- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 msgr2=0x55a48332d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a48332d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 -- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4832b8380 msgr2=0x55a48332f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4832b8380 0x55a48332f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 -- 172.21.15.82:0/2599221781 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4832747e0 con 0x55a483273570 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3aade640 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7efd300af040 tx=0x7efd30019e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bf39608da3a270 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd39adc640 1 -- 172.21.15.82:0/2599221781 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd300b1040 con 0x55a483273570 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd39adc640 1 -- 172.21.15.82:0/2599221781 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd300ba070 con 0x55a483273570 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd39adc640 1 -- 172.21.15.82:0/2599221781 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30006c40 con 0x55a483273570 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- 172.21.15.82:0/2599221781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 msgr2=0x55a4832b7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7efd300af040 tx=0x7efd30019e80 comp rx=0 tx=0).stop 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- 172.21.15.82:0/2599221781 shutdown_connections 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4832b8380 0x55a48332f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a4832b7e40 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/2599221781 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a48332d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- 172.21.15.82:0/2599221781 >> 172.21.15.82:0/2599221781 conn(0x55a48329c230 msgr2=0x55a483273dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- 172.21.15.82:0/2599221781 shutdown_connections 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- 172.21.15.82:0/2599221781 wait complete. 2024-04-25T10:14:06.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 Processor -- start 2024-04-25T10:14:06.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- start start 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a48332f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a483346d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4833472b0 0x55a48332c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a483274a80 con 0x55a483273570 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4832b7700 con 0x55a4833472b0 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3d7c1080 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4832b79a0 con 0x7efd24002cb0 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.623+0000 7efd3b2df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a48332f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a48332f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41652/0 (socket says 172.21.15.82:41652) 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 -- 172.21.15.82:0/1525526405 learned_addr learned my addr 172.21.15.82:0/1525526405 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3bae0640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4833472b0 0x55a48332c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3aade640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a483346d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 -- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 msgr2=0x55a483346d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7efd24002cb0 0x55a483346d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 -- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4833472b0 msgr2=0x55a48332c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55a4833472b0 0x55a48332c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 -- 172.21.15.82:0/1525526405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4833db3c0 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3b2df640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a48332f890 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c430 tx=0x7efd3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 -- 172.21.15.82:0/1525526405 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34003660 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 -- 172.21.15.82:0/1525526405 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd340062e0 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a4833db660 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 -- 172.21.15.82:0/1525526405 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34003a40 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4833dbba0 con 0x55a483273570 2024-04-25T10:14:06.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 -- 172.21.15.82:0/1525526405 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd3401a020 con 0x55a483273570 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd28065970 0x7efd28067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd38ada640 1 -- 172.21.15.82:0/1525526405 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efd34083cd0 con 0x55a483273570 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3aade640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd28065970 0x7efd28067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a4833d31d0 0x55a4833d35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e122) v9 -- 0x55a4833d3af0 con 0x55a4833d31d0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3bae0640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a4833d31d0 0x55a4833d35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3aade640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd28065970 0x7efd28067e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efd300b9000 tx=0x7efd30002fc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3bae0640 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a4833d31d0 0x55a4833d35b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3bae0640 1 -- 172.21.15.82:0/1525526405 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7efd2c00d040 con 0x55a4833d31d0 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a4833d31d0 msgr2=0x55a4833d35b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55a4833d31d0 0x55a4833d35b0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd28065970 msgr2=0x7efd28067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7efd28065970 0x7efd28067e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7efd300b9000 tx=0x7efd30002fc0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 msgr2=0x55a48332f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 --2- 172.21.15.82:0/1525526405 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55a483273570 0x55a48332f890 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7efd3400c430 tx=0x7efd3400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3bae0640 1 -- 172.21.15.82:0/1525526405 reap_dead start 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 shutdown_connections 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 >> 172.21.15.82:0/1525526405 conn(0x55a48329c230 msgr2=0x55a483273950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 shutdown_connections 2024-04-25T10:14:06.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.627+0000 7efd3d7c1080 1 -- 172.21.15.82:0/1525526405 wait complete. 2024-04-25T10:14:06.634 INFO:teuthology.orchestra.run.smithi082.stdout:mytag123 2024-04-25T10:14:06.634 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:14:06.634 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T10:14:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 Processor -- start 2024-04-25T10:14:06.795 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 -- start start 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f227015a6a0 0x7f22700b3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700b3ad0 con 0x7f227015aa80 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22700b3db0 con 0x7f22700b0f90 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f2274c8d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22700b4090 con 0x7f227015a6a0 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f226dd9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f227015aa80 0x7f227015e380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59162/0 (socket says 172.21.15.82:59162) 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f226dd9b640 1 -- 172.21.15.82:0/1570419773 learned_addr learned my addr 172.21.15.82:0/1570419773 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:06.798 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f226e59c640 1 --2- 172.21.15.82:0/1570419773 >> v2:172.21.15.196:3300/0 conn(0x7f227015a6a0 0x7f22700b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609347297 0 0) 0x7f22700b3ad0 con 0x7f227015aa80 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.795+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f225c003530 con 0x7f227015aa80 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3848544127 0 0) 0x7f225c003530 con 0x7f227015aa80 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 >> v2:172.21.15.196:3300/0 conn(0x7f227015a6a0 msgr2=0x7f22700b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 --2- 172.21.15.82:0/1570419773 >> v2:172.21.15.196:3300/0 conn(0x7f227015a6a0 0x7f22700b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 >> v1:172.21.15.196:6789/0 conn(0x7f22700b0f90 legacy=0x7f22700ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22700b4370 con 0x7f227015aa80 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2258001680 con 0x7f227015aa80 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 >> v1:172.21.15.82:6789/0 conn(0x7f227015aa80 legacy=0x7f227015e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f225c0039c0 0x7f225c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c004810 0x7f225c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 0x7f225c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b4090 con 0x7f225c0051d0 2024-04-25T10:14:06.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00c080 con 0x7f225c004810 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c00c280 con 0x7f225c0039c0 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 0x7f225c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226e59c640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f225c0039c0 0x7f225c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226dd9b640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c004810 0x7f225c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 -- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f225c0039c0 msgr2=0x7f225c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f225c0039c0 0x7f225c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 -- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c004810 msgr2=0x7f225c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c004810 0x7f225c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 -- 172.21.15.82:0/1570419773 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c003530 con 0x7f225c0051d0 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 -- 172.21.15.82:0/1570419773 reap_dead start 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226ed9d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 0x7f225c00b9d0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f22700723e0 tx=0x7f226001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad5d872efcddcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22600013a0 con 0x7f225c0051d0 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22600052d0 con 0x7f225c0051d0 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f226d59a640 1 -- 172.21.15.82:0/1570419773 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2260003840 con 0x7f225c0051d0 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 -- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 msgr2=0x7f225c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 0x7f225c00b9d0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f22700723e0 tx=0x7f226001c3c0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 -- 172.21.15.82:0/1570419773 shutdown_connections 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 --2- 172.21.15.82:0/1570419773 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f225c0051d0 0x7f225c00b9d0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 -- 172.21.15.82:0/1570419773 >> 172.21.15.82:0/1570419773 conn(0x7f22700b5930 msgr2=0x7f22700b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 -- 172.21.15.82:0/1570419773 shutdown_connections 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.799+0000 7f2274c8d640 1 -- 172.21.15.82:0/1570419773 wait complete. 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 Processor -- start 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- start start 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c0051d0 0x7f227015d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227015c050 0x7f22701586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b3590 con 0x7f22701f3a90 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b3410 con 0x7f225c0051d0 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22700b3710 con 0x7f227015c050 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41668/0 (socket says 172.21.15.82:41668) 2024-04-25T10:14:06.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 -- 172.21.15.82:0/579560069 learned_addr learned my addr 172.21.15.82:0/579560069 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226e59c640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c0051d0 0x7f227015d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226ed9d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227015c050 0x7f22701586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 -- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227015c050 msgr2=0x7f22701586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227015c050 0x7f22701586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 -- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c0051d0 msgr2=0x7f227015d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c0051d0 0x7f227015d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 -- 172.21.15.82:0/579560069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227015afb0 con 0x7f22701f3a90 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f226dd9b640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f225800e820 tx=0x7f225800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2266ffd640 1 -- 172.21.15.82:0/579560069 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258019070 con 0x7f22701f3a90 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2266ffd640 1 -- 172.21.15.82:0/579560069 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800c870 con 0x7f22701f3a90 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2266ffd640 1 -- 172.21.15.82:0/579560069 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225800cc40 con 0x7f22701f3a90 2024-04-25T10:14:06.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f227015b290 con 0x7f22701f3a90 2024-04-25T10:14:06.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.803+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227015b850 con 0x7f22701f3a90 2024-04-25T10:14:06.813 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2266ffd640 1 -- 172.21.15.82:0/579560069 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f225801d3f0 con 0x7f22701f3a90 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2266ffd640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2266ffd640 1 -- 172.21.15.82:0/579560069 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f22580849e0 con 0x7f22701f3a90 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226e59c640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226e59c640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 0x7f2250069450 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2270156340 tx=0x7f226800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 Processor -- start 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 -- start start 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680a60a0 0x5596680a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559667ef2ff0 con 0x5596680f02f0 2024-04-25T10:14:06.814 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559667e7b150 con 0x5596680ef9d0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559667eef090 con 0x5596680a60a0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41676/0 (socket says 172.21.15.82:41676) 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 -- 172.21.15.82:0/4050680732 learned_addr learned my addr 172.21.15.82:0/4050680732 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226e59c640 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 -- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680a60a0 msgr2=0x5596680a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680a60a0 0x5596680a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 -- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 msgr2=0x5596680efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 -- 172.21.15.82:0/4050680732 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596680890d0 con 0x5596680f02f0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f226dd9b640 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f225805d380 tx=0x7f2258055960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eaff629c9d5fd74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f224ffff640 1 -- 172.21.15.82:0/4050680732 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225805d8b0 con 0x5596680f02f0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f224ffff640 1 -- 172.21.15.82:0/4050680732 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800cde0 con 0x5596680f02f0 2024-04-25T10:14:06.815 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.807+0000 7f224ffff640 1 -- 172.21.15.82:0/4050680732 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258051810 con 0x5596680f02f0 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/4050680732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 msgr2=0x5596680a5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f225805d380 tx=0x7f2258055960 comp rx=0 tx=0).stop 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/4050680732 shutdown_connections 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680a60a0 0x5596680a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680f02f0 0x5596680a5b60 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- 172.21.15.82:0/4050680732 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/4050680732 >> 172.21.15.82:0/4050680732 conn(0x5596680a2020 msgr2=0x5596680a4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/4050680732 shutdown_connections 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/4050680732 wait complete. 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- start start 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 Processor -- start 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.816 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x5596680d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559667eb7250 con 0x5596680a60a0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559667ead690 con 0x5596680ef9d0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559667e7ba40 con 0x5596680f02f0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 10 client.0 _ll_register_callbacks cb 0x559668090e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41680/0 (socket says 172.21.15.82:41680) 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 -- 172.21.15.82:0/645135226 learned_addr learned my addr 172.21.15.82:0/645135226 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226dd9b640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 -- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 msgr2=0x5596680d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226ed9d640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x5596680d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x5596680d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 -- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 msgr2=0x5596680d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.817 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 -- 172.21.15.82:0/645135226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559667fa01c0 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226dd9b640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226e59c640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f2268008010 tx=0x7f2268004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce51c66a7c2f3e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268019c10 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2268019db0 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f225800cde0 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2268030050 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 10 client.10053 fetch_fsmap learned FSMap version 97 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x559667f9c170 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225800cde0 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2268033020 con 0x5596680a60a0 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 10 client.10053 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 10 client.10053 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 20 client.10053 populate_metadata read hostname 'smithi082' 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x5596680ffc80 0x559668102070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 4 client.10053 mds_command: new command op to 9849 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 --> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559667e8ec60 con 0x5596680ffc80 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226dd9b640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x5596680ffc80 0x559668102070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 10 client.10053 ms_handle_connect on v2:172.21.15.82:6835/3303693719 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f226dd9b640 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x5596680ffc80 0x559668102070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21761dc01d5e7f1f server_cookie=e866e8bfd6de905 in_seq=0 out_seq=0 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 1 -- 172.21.15.82:0/645135226 <== mds.0 v2:172.21.15.82:6835/3303693719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559667e8ec60 con 0x5596680ffc80 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f224d7fa640 10 client.10053 handle_command_reply: tid=0 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.811+0000 7f2277dfa000 1 client.10053 shutdown 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 msgr2=0x5596680d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.819 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f2268008010 tx=0x7f2268004a50 comp rx=0 tx=0).stop 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 shutdown_connections 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x5596680ffc80 0x559668102070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x5596680d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f224d7fa640 0 client.10053 ms_handle_reset on v2:172.21.15.82:6835/3303693719 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x5596680d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 --2- 172.21.15.82:0/645135226 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x5596680d47f0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 >> 172.21.15.82:0/645135226 conn(0x5596680a2020 msgr2=0x5596680a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 shutdown_connections 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 1 -- 172.21.15.82:0/645135226 wait complete. 2024-04-25T10:14:06.820 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.815+0000 7f2277dfa000 20 client.10053 trim_cache size 0 max 16384 2024-04-25T10:14:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 Processor -- start 2024-04-25T10:14:06.826 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- start start 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x559668154e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x559668155380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22702007b0 con 0x5596680a60a0 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701ff970 con 0x5596680ef9d0 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22701feda0 con 0x5596680f02f0 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41684/0 (socket says 172.21.15.82:41684) 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 -- 172.21.15.82:0/3834596360 learned_addr learned my addr 172.21.15.82:0/3834596360 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 -- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 msgr2=0x559668155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x559668155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 -- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 msgr2=0x559668154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x559668154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 -- 172.21.15.82:0/3834596360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559667f39de0 con 0x5596680a60a0 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f226e59c640 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f22680070b0 tx=0x7f2268017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a78d3b6db46317b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f224f7fe640 1 -- 172.21.15.82:0/3834596360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268016640 con 0x5596680a60a0 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f224f7fe640 1 -- 172.21.15.82:0/3834596360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22680168d0 con 0x5596680a60a0 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f224f7fe640 1 -- 172.21.15.82:0/3834596360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268016c80 con 0x5596680a60a0 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- 172.21.15.82:0/3834596360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 msgr2=0x559668154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f22680070b0 tx=0x7f2268017000 comp rx=0 tx=0).stop 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- 172.21.15.82:0/3834596360 shutdown_connections 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680f02f0 0x559668155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.829 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680ef9d0 0x559668154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 --2- 172.21.15.82:0/3834596360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668154900 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- 172.21.15.82:0/3834596360 >> 172.21.15.82:0/3834596360 conn(0x5596680a2020 msgr2=0x5596680fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- 172.21.15.82:0/3834596360 shutdown_connections 2024-04-25T10:14:06.830 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- 172.21.15.82:0/3834596360 wait complete. 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.827+0000 7f2277dfa000 1 -- start start 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 Processor -- start 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680ef9d0 0x5596681028c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680f02f0 0x559668102e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559667e8de00 con 0x5596680a60a0 2024-04-25T10:14:06.833 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22701ff370 con 0x5596680f02f0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559667ef2300 con 0x5596680ef9d0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 10 client.0 _ll_register_callbacks cb 0x559668071040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41690/0 (socket says 172.21.15.82:41690) 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 -- 172.21.15.82:0/1614095514 learned_addr learned my addr 172.21.15.82:0/1614095514 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226ed9d640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680f02f0 0x559668102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226dd9b640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680ef9d0 0x5596681028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 -- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680ef9d0 msgr2=0x5596681028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680ef9d0 0x5596681028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 -- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680f02f0 msgr2=0x559668102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680f02f0 0x559668102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 -- 172.21.15.82:0/1614095514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559667e948e0 con 0x5596680a60a0 2024-04-25T10:14:06.834 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226e59c640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f90 tx=0x7f2268004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2916c29e887916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268019850 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f226803d420 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22680168d0 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f226804d050 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 10 client.10059 fetch_fsmap learned FSMap version 97 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x559667f4c8f0 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22680168d0 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2268050020 con 0x5596680a60a0 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 10 client.10059 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 10 client.10059 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 20 client.10059 populate_metadata read hostname 'smithi082' 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x559668103700 0x559668160770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 4 client.10059 mds_command: new command op to 9849 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 --> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559667f00530 con 0x559668103700 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226dd9b640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x559668103700 0x559668160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f224d7fa640 10 client.10059 ms_handle_connect on v2:172.21.15.82:6835/3303693719 2024-04-25T10:14:06.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.831+0000 7f226dd9b640 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x559668103700 0x559668160770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7504d1fd4adb2c0e server_cookie=63a345ce59ec45fe in_seq=0 out_seq=0 2024-04-25T10:14:06.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f224d7fa640 1 -- 172.21.15.82:0/1614095514 <== mds.0 v2:172.21.15.82:6835/3303693719 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559667f00530 con 0x559668103700 2024-04-25T10:14:06.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f224d7fa640 10 client.10059 handle_command_reply: tid=0 2024-04-25T10:14:06.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 client.10059 shutdown 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 msgr2=0x559668102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7f2268000f90 tx=0x7f2268004ac0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 shutdown_connections 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:6835/3303693719,v1:172.21.15.82:6837/3303693719] conn(0x559668103700 0x559668160770 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5596680ef9d0 0x5596681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5596680f02f0 0x559668102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f224d7fa640 0 client.10059 ms_handle_reset on v2:172.21.15.82:6835/3303693719 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 --2- 172.21.15.82:0/1614095514 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5596680a60a0 0x559668102380 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 >> 172.21.15.82:0/1614095514 conn(0x5596680a2020 msgr2=0x5596680ee0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 shutdown_connections 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 1 -- 172.21.15.82:0/1614095514 wait complete. 2024-04-25T10:14:06.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2277dfa000 20 client.10059 trim_cache size 0 max 16384 2024-04-25T10:14:06.849 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:06.849 INFO:teuthology.orchestra.run.smithi082.stdout: "message": "", 2024-04-25T10:14:06.849 INFO:teuthology.orchestra.run.smithi082.stdout: "return_code": 0 2024-04-25T10:14:06.849 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 msgr2=0x7f2250069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 0x7f2250069450 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f2270156340 tx=0x7f226800b040 comp rx=0 tx=0).stop 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 msgr2=0x7f2270154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f225800e820 tx=0x7f225800ece0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 shutdown_connections 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2250066f90 0x7f2250069450 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f227015c050 0x7f22701586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22701f3a90 0x7f2270154fe0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 --2- 172.21.15.82:0/579560069 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f225c0051d0 0x7f227015d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 >> 172.21.15.82:0/579560069 conn(0x7f22700b5930 msgr2=0x7f22700b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 shutdown_connections 2024-04-25T10:14:06.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.847+0000 7f2274c8d640 1 -- 172.21.15.82:0/579560069 wait complete. 2024-04-25T10:14:06.863 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 Processor -- start 2024-04-25T10:14:06.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 -- start start 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fb95c071590 0x7fb95c15c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c15f240 con 0x7fb95c071130 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95c15f520 con 0x7fb95c15edc0 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb96199e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95c15f800 con 0x7fb95c071590 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb9527fc640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb95c071130 0x7fb95c15bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59172/0 (socket says 172.21.15.82:59172) 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb9527fc640 1 -- 172.21.15.82:0/3664074688 learned_addr learned my addr 172.21.15.82:0/3664074688 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2477664936 0 0) 0x7fb95c15f240 con 0x7fb95c071130 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb948003530 con 0x7fb95c071130 2024-04-25T10:14:06.972 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95b7fe640 1 --2- 172.21.15.82:0/3664074688 >> v2:172.21.15.196:3300/0 conn(0x7fb95c071590 0x7fb95c15c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2188202243 0 0) 0x7fb948003530 con 0x7fb95c071130 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 >> v2:172.21.15.196:3300/0 conn(0x7fb95c071590 msgr2=0x7fb95c15c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 --2- 172.21.15.82:0/3664074688 >> v2:172.21.15.196:3300/0 conn(0x7fb95c071590 0x7fb95c15c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 >> v1:172.21.15.196:6789/0 conn(0x7fb95c15edc0 legacy=0x7fb95c144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c15fae0 con 0x7fb95c071130 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb944001680 con 0x7fb95c071130 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 >> v1:172.21.15.82:6789/0 conn(0x7fb95c071130 legacy=0x7fb95c15bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 0x7fb948003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948004810 0x7fb948004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480051d0 0x7fb94800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c15f800 con 0x7fb9480039c0 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94800c120 con 0x7fb948004810 2024-04-25T10:14:06.973 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94800c320 con 0x7fb9480051d0 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 0x7fb948003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb9527fc640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948004810 0x7fb948004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95b7fe640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480051d0 0x7fb94800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 -- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480051d0 msgr2=0x7fb94800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480051d0 0x7fb94800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 -- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948004810 msgr2=0x7fb948004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb948004810 0x7fb948004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95affd640 1 -- 172.21.15.82:0/3664074688 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb948003530 con 0x7fb9480039c0 2024-04-25T10:14:06.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.971+0000 7fb95b7fe640 1 -- 172.21.15.82:0/3664074688 reap_dead start 2024-04-25T10:14:06.977 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95affd640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 0x7fb948003de0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb94c001780 tx=0x7fb94c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a7d1858aba9f87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0013a0 con 0x7fb9480039c0 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0052f0 con 0x7fb9480039c0 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95a7fc640 1 -- 172.21.15.82:0/3664074688 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c003840 con 0x7fb9480039c0 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 msgr2=0x7fb948003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 0x7fb948003de0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fb94c001780 tx=0x7fb94c01c420 comp rx=0 tx=0).stop 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- 172.21.15.82:0/3664074688 shutdown_connections 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 --2- 172.21.15.82:0/3664074688 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9480039c0 0x7fb948003de0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- 172.21.15.82:0/3664074688 >> 172.21.15.82:0/3664074688 conn(0x7fb95c066fa0 msgr2=0x7fb95c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- 172.21.15.82:0/3664074688 shutdown_connections 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- 172.21.15.82:0/3664074688 wait complete. 2024-04-25T10:14:06.978 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 Processor -- start 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- start start 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480039c0 0x7fb95c0aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 0x7fb95c0c4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06d2d0 con 0x7fb95c0c4970 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06d150 con 0x7fb95c0ac060 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb96199e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c06d450 con 0x7fb9480039c0 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41700/0 (socket says 172.21.15.82:41700) 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 -- 172.21.15.82:0/323163138 learned_addr learned my addr 172.21.15.82:0/323163138 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb9527fc640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 0x7fb95c0c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95affd640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480039c0 0x7fb95c0aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.979 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 -- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480039c0 msgr2=0x7fb95c0aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480039c0 0x7fb95c0aa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 -- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 msgr2=0x7fb95c0c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 0x7fb95c0c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb95b7fe640 1 -- 172.21.15.82:0/323163138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95c0aafc0 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.975+0000 7fb9527fc640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 0x7fb95c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb95b7fe640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fb9540049e0 tx=0x7fb954009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954019070 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb954007e00 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c0ab2a0 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb954014640 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c0ab860 con 0x7fb95c0c4970 2024-04-25T10:14:06.980 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c067c40 con 0x7fb95c0c4970 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb95401c070 con 0x7fb95c0c4970 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb954084dc0 con 0x7fb95c0c4970 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb95affd640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:06.981 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.979+0000 7fb95affd640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 0x7fb93c069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb94c001750 tx=0x7fb94c0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:06.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:06.983+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb954055c80 con 0x7fb95c0c4970 2024-04-25T10:14:07.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.175+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb95c06b920 con 0x7fb95c0c4970 2024-04-25T10:14:07.175 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:14:07.175+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:07.177 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb958ff9640 1 -- 172.21.15.82:0/323163138 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb954055580 con 0x7fb95c0c4970 2024-04-25T10:14:07.177 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 msgr2=0x7fb93c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 0x7fb93c069450 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb94c001750 tx=0x7fb94c0011a0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 msgr2=0x7fb95c0c4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7fb9540049e0 tx=0x7fb954009dc0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 shutdown_connections 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb93c066f90 0x7fb93c069450 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb95c0c4970 0x7fb95c0c4df0 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.180 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb95c0ac060 0x7fb95c0c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 --2- 172.21.15.82:0/323163138 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9480039c0 0x7fb95c0aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.179+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 >> 172.21.15.82:0/323163138 conn(0x7fb95c066fa0 msgr2=0x7fb95c073570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.183+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 shutdown_connections 2024-04-25T10:14:07.181 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.183+0000 7fb96199e640 1 -- 172.21.15.82:0/323163138 wait complete. 2024-04-25T10:14:07.185 INFO:tasks.ceph.mds.c.smithi082.stderr: -2315> 2024-04-25T10:13:53.615+0000 7f8277320640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:07.200 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T10:14:07.217 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:14:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 -- start start 2024-04-25T10:14:07.236 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 Processor -- start 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887c83800 0x55e887e3c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 0x55e887e3cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e180 con 0x55e887c83000 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.235+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e300 con 0x55e887c83400 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e480 con 0x55e887c83800 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41728/0 (socket says 172.21.15.82:41728) 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 -- 172.21.15.82:0/3880086088 learned_addr learned my addr 172.21.15.82:0/3880086088 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 0x55e887e3cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d05b3640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887c83800 0x55e887e3c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 -- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887c83800 msgr2=0x55e887e3c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887c83800 0x55e887e3c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 -- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 msgr2=0x55e887e3cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 0x55e887e3cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 -- 172.21.15.82:0/3880086088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e888a30960 con 0x55e887c83000 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 0x55e887e3cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d0db4640 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x55e888af3350 tx=0x55e887d45020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaf527ccde5e749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cddae640 1 -- 172.21.15.82:0/3880086088 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb84e0 con 0x55e887c83000 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cddae640 1 -- 172.21.15.82:0/3880086088 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e887ccce00 con 0x55e887c83000 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cddae640 1 -- 172.21.15.82:0/3880086088 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb8b60 con 0x55e887c83000 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e888a30d20 con 0x55e887c83000 2024-04-25T10:14:07.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e888a30f00 con 0x55e887c83000 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 4 waiting for MDS map... 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cddae640 1 -- 172.21.15.82:0/3880086088 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55e888b9e600 con 0x55e887c83000 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cddae640 1 -- 172.21.15.82:0/3880086088 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 2723+0+0 (secure 0 0 0) 0x55e887cd9180 con 0x55e887c83000 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 4 Got MDS map 98 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 10 main: JournalTool::main 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 4 main: JournalTool: connecting to RADOS... 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 Processor -- start 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- start start 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e600 con 0x55e888ba5400 2024-04-25T10:14:07.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e780 con 0x55e888ba4c00 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e900 con 0x55e888ba5000 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41730/0 (socket says 172.21.15.82:41730) 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 -- 172.21.15.82:0/2387256278 learned_addr learned my addr 172.21.15.82:0/2387256278 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d05b3640 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 -- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 msgr2=0x55e887e3db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 -- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 msgr2=0x55e887e3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 -- 172.21.15.82:0/2387256278 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e888a30d20 con 0x55e888ba5400 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x55e888bb06c0 tx=0x55e888c0a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd73b0f5b7027cc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cbdaa640 1 -- 172.21.15.82:0/2387256278 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb84e0 con 0x55e888ba5400 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cbdaa640 1 -- 172.21.15.82:0/2387256278 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e887ccc540 con 0x55e888ba5400 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57cbdaa640 1 -- 172.21.15.82:0/2387256278 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb8b60 con 0x55e888ba5400 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/2387256278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 msgr2=0x55e887e3e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x55e888bb06c0 tx=0x55e888c0a0c0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/2387256278 shutdown_connections 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e100 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 --2- 172.21.15.82:0/2387256278 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/2387256278 >> 172.21.15.82:0/2387256278 conn(0x55e887c82c00 msgr2=0x55e887cb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/2387256278 shutdown_connections 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.239+0000 7f57d1df4400 1 -- 172.21.15.82:0/2387256278 wait complete. 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 Processor -- start 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- start start 2024-04-25T10:14:07.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8fc80 con 0x55e888ba5400 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8fb00 con 0x55e888ba4c00 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8f980 con 0x55e888ba5000 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41734/0 (socket says 172.21.15.82:41734) 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1645350707 learned_addr learned my addr 172.21.15.82:0/1645350707 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 msgr2=0x55e887e3ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 msgr2=0x55e887e3f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1645350707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e888a31c20 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x55e888bb1380 tx=0x55e888c0a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 -- 172.21.15.82:0/1645350707 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb8000 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 -- 172.21.15.82:0/1645350707 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e887cccc40 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 -- 172.21.15.82:0/1645350707 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb84e0 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e888a31e00 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e887fc21e0 con 0x55e888ba5400 2024-04-25T10:14:07.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 4 main: JournalTool: resolving pool 17 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 -- 172.21.15.82:0/1645350707 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55e888becc00 con 0x55e888ba5400 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 0x55e887e3f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57ca5a7640 1 -- 172.21.15.82:0/1645350707 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55e887faa400 con 0x55e888ba5400 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 0x55e887e3f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 4 main: JournalTool: creating IoCtx.. 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.243+0000 7f57d1df4400 4 main: Executing for rank 0 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 0x55e887e3f700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55e887cd37d0 tx=0x55e888a4e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- start start 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 Processor -- start 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887fb3000 0x55e887e3db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887fb3400 0x55e887e3d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8f680 con 0x55e888ba5c00 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e600 con 0x55e887fb3000 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e888b8e780 con 0x55e887fb3400 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41742/0 (socket says 172.21.15.82:41742) 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 -- 172.21.15.82:0/1615940384 learned_addr learned my addr 172.21.15.82:0/1615940384 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887fb3000 0x55e887e3db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887fb3400 msgr2=0x55e887e3d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887fb3400 0x55e887e3d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887fb3000 msgr2=0x55e887e3db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887fb3000 0x55e887e3db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e887fc2d20 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d0db4640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x55e888065590 tx=0x55e887d3aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72973b4c0310c5c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57c75a1640 1 -- 172.21.15.82:0/1615940384 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887cb8340 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57c75a1640 1 -- 172.21.15.82:0/1615940384 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e887ccca80 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57c75a1640 1 -- 172.21.15.82:0/1615940384 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e887ff21a0 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e887fc2f00 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e887fc30e0 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 4 waiting for MDS map... 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57c75a1640 1 -- 172.21.15.82:0/1615940384 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55e888108200 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57c75a1640 1 -- 172.21.15.82:0/1615940384 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 2723+0+0 (secure 0 0 0) 0x55e887cd9880 con 0x55e888ba5c00 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 4 Got MDS map 98 2024-04-25T10:14:07.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e887fb3800 0x55e888118000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e887fb3c00 con 0x55e887fb3800 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e887fb3800 0x55e888118000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.247+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e887fb3800 0x55e888118000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=2bc6d786e7c5123a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.251+0000 7f57d05b3640 1 -- 172.21.15.82:0/1615940384 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v123'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe000 con 0x55e887fb3800 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.251+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55e88810b400 0x55e888118580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.251+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e88810b800 con 0x55e88810b400 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.251+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55e88810b400 0x55e888118580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.251+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55e88810b400 0x55e888118580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=d1772cb808587c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1615940384 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v123'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe500 con 0x55e88810b400 2024-04-25T10:14:07.257 INFO:teuthology.orchestra.run.smithi082.stdout:writing EResetJournal entry 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e88810bc00 con 0x55e88810b400 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 0x55e888118b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888130400 con 0x55e888130000 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55e888130800 0x55e888119080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888130c00 con 0x55e888130800 2024-04-25T10:14:07.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 0x55e888119600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888131400 con 0x55e888131000 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888131800 con 0x55e888130800 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888131c00 con 0x55e887fb3800 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55e888130800 0x55e888119080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d0db4640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 0x55e888118b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d05b3640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55e888130800 0x55e888119080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=17664e711db6d951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 0x55e888119600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57d0db4640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 0x55e888118b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=e9bd5ef2f8578d23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.255+0000 7f57cfdb2640 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 0x55e888119600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=1974dd0028df7f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57d05b3640 1 -- 172.21.15.82:0/1615940384 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(8 200.00000006 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe000 con 0x55e887fb3800 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57d05b3640 1 -- 172.21.15.82:0/1615940384 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(7 200.00000005 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe000 con 0x55e888130800 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1615940384 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(3 200.00000001 [delete] v123'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe500 con 0x55e88810b400 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57d05b3640 1 -- 172.21.15.82:0/1615940384 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(5 200.00000003 [delete] v123'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe000 con 0x55e888130800 2024-04-25T10:14:07.260 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57d0db4640 1 -- 172.21.15.82:0/1615940384 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(4 200.00000002 [delete] v123'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe780 con 0x55e888130000 2024-04-25T10:14:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57c5d9e640 1 -- 172.21.15.82:0/1615940384 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55e888a65400 con 0x55e88810b400 2024-04-25T10:14:07.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.259+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1615940384 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(6 200.00000004 [delete] v123'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe500 con 0x55e888131000 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1615940384 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v123'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55e887dfe500 con 0x55e88810b400 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55e888130800 msgr2=0x55e888119080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55e888130800 0x55e888119080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e887fb3800 msgr2=0x55e888118000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55e887fb3800 0x55e888118000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55e88810b400 msgr2=0x55e888118580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55e88810b400 0x55e888118580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 msgr2=0x55e888118b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 0x55e888118b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57cfdb2640 1 -- 172.21.15.82:0/1615940384 reap_dead start 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 msgr2=0x55e888119600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 0x55e888119600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 msgr2=0x55e887e3e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x55e888065590 tx=0x55e887d3aba0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 shutdown_connections 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55e888130000 0x55e888118b00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55e888131000 0x55e888119600 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1615940384 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5c00 0x55e887e3e100 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 >> 172.21.15.82:0/1615940384 conn(0x55e888ba5800 msgr2=0x55e8880fa000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 shutdown_connections 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1615940384 wait complete. 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 msgr2=0x55e887e3f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 0x55e887e3f700 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x55e887cd37d0 tx=0x55e888a4e000 comp rx=0 tx=0).stop 2024-04-25T10:14:07.265 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 msgr2=0x55e887e3e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x55e888bb1380 tx=0x55e888c0a000 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 shutdown_connections 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55e887fb2c00 0x55e887e3f700 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e888ba5000 0x55e887e3ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e888ba5400 0x55e887e3e680 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/1645350707 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e888ba4c00 0x55e887e3f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 >> 172.21.15.82:0/1645350707 conn(0x55e887c82c00 msgr2=0x55e887cb1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 shutdown_connections 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/1645350707 wait complete. 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 msgr2=0x55e887e3d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x55e888af3350 tx=0x55e887d45020 comp rx=0 tx=0).stop 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 shutdown_connections 2024-04-25T10:14:07.266 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55e887c83000 0x55e887e3d080 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55e887c83400 0x55e887e3cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 --2- 172.21.15.82:0/3880086088 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55e887c83800 0x55e887e3c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 >> 172.21.15.82:0/3880086088 conn(0x55e887c83c00 msgr2=0x55e887cb1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 shutdown_connections 2024-04-25T10:14:07.267 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.263+0000 7f57d1df4400 1 -- 172.21.15.82:0/3880086088 wait complete. 2024-04-25T10:14:07.268 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.068102, result: writing EResetJournal entry 2024-04-25T10:14:07.268 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T10:14:07.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- start start 2024-04-25T10:14:07.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 Processor -- start 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b27663800 0x564b2781c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b27663400 0x564b2781cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a180 con 0x564b27663000 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a300 con 0x564b27663800 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a480 con 0x564b27663400 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41758/0 (socket says 172.21.15.82:41758) 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 -- 172.21.15.82:0/1762354212 learned_addr learned my addr 172.21.15.82:0/1762354212 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 -- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b27663400 msgr2=0x564b2781cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b27663400 0x564b2781cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 -- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b27663800 msgr2=0x564b2781c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b27663800 0x564b2781c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 -- 172.21.15.82:0/1762354212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b276881e0 con 0x564b27663000 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331df96640 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x564b284a70e0 tx=0x564b27760000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4c6aae3eaee211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331af90640 1 -- 172.21.15.82:0/1762354212 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276989c0 con 0x564b27663000 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331af90640 1 -- 172.21.15.82:0/1762354212 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b276ace00 con 0x564b27663000 2024-04-25T10:14:07.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331af90640 1 -- 172.21.15.82:0/1762354212 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276984e0 con 0x564b27663000 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b276892c0 con 0x564b27663000 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564b276883c0 con 0x564b27663000 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 4 waiting for MDS map... 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331af90640 1 -- 172.21.15.82:0/1762354212 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x564b2856c200 con 0x564b27663000 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331af90640 1 -- 172.21.15.82:0/1762354212 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 2723+0+0 (secure 0 0 0) 0x564b276b9180 con 0x564b27663000 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 4 Got MDS map 98 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 4 datascan.main: Using metadata pool output 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 Processor -- start 2024-04-25T10:14:07.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- start start 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b28437c00 0x564b2781db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b285d6000 0x564b2781e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a600 con 0x564b28437800 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a780 con 0x564b28437c00 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b2855a900 con 0x564b285d6000 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41760/0 (socket says 172.21.15.82:41760) 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 -- 172.21.15.82:0/1339988294 learned_addr learned my addr 172.21.15.82:0/1339988294 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 -- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b285d6000 msgr2=0x564b2781e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b285d6000 0x564b2781e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 -- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b28437c00 msgr2=0x564b2781db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331d795640 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b28437c00 0x564b2781db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b28437c00 0x564b2781db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.339+0000 7f331cf94640 1 -- 172.21.15.82:0/1339988294 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b276881e0 con 0x564b28437800 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331cf94640 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x564b2842cc30 tx=0x564b285e2060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c6c1c4d89f399d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3318f8c640 1 -- 172.21.15.82:0/1339988294 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276989c0 con 0x564b28437800 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3318f8c640 1 -- 172.21.15.82:0/1339988294 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b276aca80 con 0x564b28437800 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3318f8c640 1 -- 172.21.15.82:0/1339988294 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276984e0 con 0x564b28437800 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/1339988294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 msgr2=0x564b2781d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x564b2842cc30 tx=0x564b285e2060 comp rx=0 tx=0).stop 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/1339988294 shutdown_connections 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b285d6000 0x564b2781e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b28437c00 0x564b2781db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1339988294 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781d600 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/1339988294 >> 172.21.15.82:0/1339988294 conn(0x564b27662c00 msgr2=0x564b27691800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/1339988294 shutdown_connections 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/1339988294 wait complete. 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 Processor -- start 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- start start 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b28437c00 0x564b2781f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b285d7800 0x564b2781e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855bc80 con 0x564b28437800 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b2855bb00 con 0x564b285d7800 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b2855b980 con 0x564b28437c00 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41772/0 (socket says 172.21.15.82:41772) 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 learned_addr learned my addr 172.21.15.82:0/2344646220 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b28437c00 msgr2=0x564b2781f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b28437c00 0x564b2781f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b285d7800 0x564b2781e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b285d7800 msgr2=0x564b2781e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b285d7800 0x564b2781e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b276894a0 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781ec00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x564b2842d920 tx=0x564b285e2240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3317789640 1 -- 172.21.15.82:0/2344646220 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276989c0 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3317789640 1 -- 172.21.15.82:0/2344646220 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564b276ac540 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564b27689a40 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3317789640 1 -- 172.21.15.82:0/2344646220 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564b276984e0 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b27689c20 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f331efd68c0 4 datascan.init: resolving metadata pool 17 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.343+0000 7f3317789640 1 -- 172.21.15.82:0/2344646220 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x564b285e8c00 con 0x564b28437800 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f3317789640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x564b285e5400 0x564b2781f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f3317789640 1 -- 172.21.15.82:0/2344646220 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x564b285e7000 con 0x564b28437800 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331efd68c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331efd68c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331efd68c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x564b285e5400 0x564b2781f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x564b285d7400 0x564b2781e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d7000 con 0x564b285d7400 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x564b285d7400 0x564b2781e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x564b285e5400 0x564b2781f700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x564b2842cdb0 tx=0x564b285e20c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x564b285d7400 0x564b2781e100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331878b640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x564b285e5800 0x564b27a82000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e5c00 con 0x564b285e5800 2024-04-25T10:14:07.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x564b285e5800 0x564b27a82000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.347+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x564b285e5800 0x564b27a82000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.351+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.351+0000 7f3318f8c640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x564b27a92000 0x564b27a82580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.351+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92400 con 0x564b27a92000 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.351+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x564b27a92000 0x564b27a82580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.351+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x564b27a92000 0x564b27a82580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27a92000 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92c00 con 0x564b285d7400 2024-04-25T10:14:07.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92800 con 0x564b285d7400 2024-04-25T10:14:07.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93000 con 0x564b285e5800 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f3318f8c640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x564b27a92c00 0x564b27a82b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d7000 con 0x564b27a92c00 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x564b27a92c00 0x564b27a82b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x564b27a92c00 0x564b27a82b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331878b640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x564b27a93400 0x564b27a83600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92800 con 0x564b27a93400 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x564b27a93400 0x564b27a83600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x564b27a93400 0x564b27a83600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e4400 con 0x564b285d7400 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93800 con 0x564b285d7400 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(10 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.355+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa000 con 0x564b27a92000 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27a92000 2024-04-25T10:14:07.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93c00 con 0x564b285d7400 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa400 con 0x564b285e5800 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(13 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331878b640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x564b27aaa400 0x564b27a83b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93000 con 0x564b27aaa400 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x564b27aaa400 0x564b27a83b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331d795640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x564b27aaa400 0x564b27a83b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa800 con 0x564b27a92c00 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(15 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331878b640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x564b27a92800 0x564b27a84100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e5c00 con 0x564b27a92800 2024-04-25T10:14:07.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x564b27a92800 0x564b27a84100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.359+0000 7f331cf94640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x564b27a92800 0x564b27a84100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaac00 con 0x564b27a92800 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b28437400 con 0x564b27a93400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa800 con 0x564b27aaa400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(19 [pgnls start_epoch 123 out=81b] v122'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6c00 con 0x564b27aaa400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331efd68c0 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T10:14:07.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6800 con 0x564b27aaa400 2024-04-25T10:14:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(21 10000000002.00000000 [call] v123'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6400 con 0x564b27a93400 2024-04-25T10:14:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.364 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93800 con 0x564b27a93400 2024-04-25T10:14:07.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(23 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331878b640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x564b27aaa000 0x564b27a84680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.363+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa800 con 0x564b27aaa000 2024-04-25T10:14:07.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x564b27aaa000 0x564b27a84680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.365 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331df96640 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x564b27aaa000 0x564b27a84680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(24 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93c00 con 0x564b27a93400 2024-04-25T10:14:07.366 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 5 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab000 con 0x564b285d7400 2024-04-25T10:14:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(26 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e4400 con 0x564b27a92800 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(27 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab400 con 0x564b27aaa400 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 5 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d7000 con 0x564b27a93400 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.368 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92400 con 0x564b27a92c00 2024-04-25T10:14:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=81b] v121'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6000 con 0x564b27a92c00 2024-04-25T10:14:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331efd68c0 10 datascan.operator(): handling object 1099511628279.0 2024-04-25T10:14:07.369 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.367+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaac00 con 0x564b27a92c00 2024-04-25T10:14:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v123'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6000 con 0x564b285d7400 2024-04-25T10:14:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(33 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.370 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93000 con 0x564b27a93400 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 7 ==== osd_op_reply(34 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab000 con 0x564b285e5800 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(35 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc000 con 0x564b27a93400 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 8 ==== osd_op_reply(36 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.371 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa800 con 0x564b27a93400 2024-04-25T10:14:07.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 9 ==== osd_op_reply(37 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc400 con 0x564b285e5800 2024-04-25T10:14:07.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(38 [pgnls start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.372 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab800 con 0x564b27a92c00 2024-04-25T10:14:07.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(39 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.371+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc800 con 0x564b27aaa000 2024-04-25T10:14:07.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(40 [pgnls start_epoch 123 out=81b] v121'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.373 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6400 con 0x564b27aaa000 2024-04-25T10:14:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331efd68c0 10 datascan.operator(): handling object 1099511627780.1 2024-04-25T10:14:07.374 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d7000 con 0x564b27a92c00 2024-04-25T10:14:07.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(42 10000000004.00000000 [call] v123'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92400 con 0x564b27a93400 2024-04-25T10:14:07.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 10 ==== osd_op_reply(43 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.376 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.375+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abcc00 con 0x564b27a92800 2024-04-25T10:14:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(44 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.377 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aabc00 con 0x564b27aaa000 2024-04-25T10:14:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(45 [pgnls start_epoch 123 out=81b] v122'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93800 con 0x564b27aaa000 2024-04-25T10:14:07.378 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331efd68c0 10 datascan.operator(): handling object 1099511628280.0 2024-04-25T10:14:07.379 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.379+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93c00 con 0x564b27aaa000 2024-04-25T10:14:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v123'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.381 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b28437400 con 0x564b285d7400 2024-04-25T10:14:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 9 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc000 con 0x564b27a92c00 2024-04-25T10:14:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(49 [pgnls start_epoch 123 out=81b] v123'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e5c00 con 0x564b27a92c00 2024-04-25T10:14:07.382 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331efd68c0 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T10:14:07.383 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.383+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285e4400 con 0x564b27a92c00 2024-04-25T10:14:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 10 ==== osd_op_reply(51 10000000004.00000000 [call] v123'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.384 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abcc00 con 0x564b27a92000 2024-04-25T10:14:07.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27a92000 2024-04-25T10:14:07.385 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc400 con 0x564b285e5800 2024-04-25T10:14:07.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(53 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd000 con 0x564b27aaa000 2024-04-25T10:14:07.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 7 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.386 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc800 con 0x564b27aaa000 2024-04-25T10:14:07.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 8 ==== osd_op_reply(55 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.387 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd400 con 0x564b27a92000 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27a92000 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6400 con 0x564b285d7400 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.0 v2:172.21.15.82:6800/2082960483 10 ==== osd_op_reply(57 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b285d7400 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.387+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd800 con 0x564b27a93400 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.3 v2:172.21.15.82:6824/3269667522 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27a93400 2024-04-25T10:14:07.388 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab000 con 0x564b27a92c00 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc400 con 0x564b27aaa400 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aaa800 con 0x564b27a92800 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aabc00 con 0x564b27a92c00 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 12 ==== osd_op_reply(62 [pgnls start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.390 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93000 con 0x564b285e5800 2024-04-25T10:14:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27aab400 con 0x564b27a92800 2024-04-25T10:14:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 6 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd000 con 0x564b27a92c00 2024-04-25T10:14:07.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 13 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6c00 con 0x564b27aaa000 2024-04-25T10:14:07.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 9 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.391+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abc800 con 0x564b27aaa400 2024-04-25T10:14:07.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 <== osd.2 v2:172.21.15.82:6803/3086763100 7 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716780 con 0x564b27aaa400 2024-04-25T10:14:07.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b285d6800 con 0x564b285e5800 2024-04-25T10:14:07.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd800 con 0x564b285e5800 2024-04-25T10:14:07.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 9 ==== osd_op_reply(69 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a92400 con 0x564b27a92800 2024-04-25T10:14:07.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 7 ==== osd_op_reply(70 [pgnls start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abcc00 con 0x564b285e5800 2024-04-25T10:14:07.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.5 v2:172.21.15.196:6800/1745719129 10 ==== osd_op_reply(71 [pgnls start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b285e5800 2024-04-25T10:14:07.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93800 con 0x564b27a92c00 2024-04-25T10:14:07.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.1 v2:172.21.15.82:6816/1010291016 14 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92c00 2024-04-25T10:14:07.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.395+0000 7f3318f8c640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27abd400 con 0x564b27a92800 2024-04-25T10:14:07.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331cf94640 1 -- 172.21.15.82:0/2344646220 <== osd.7 v2:172.21.15.196:6822/2552340661 8 ==== osd_op_reply(73 [pgnls start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716000 con 0x564b27a92800 2024-04-25T10:14:07.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331878b640 1 -- 172.21.15.82:0/2344646220 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x564b27a93c00 con 0x564b27aaa000 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331df96640 1 -- 172.21.15.82:0/2344646220 <== osd.4 v2:172.21.15.196:6816/2164040014 10 ==== osd_op_reply(74 [pgnls start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564b27716500 con 0x564b27aaa000 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x564b285d7400 msgr2=0x564b2781e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x564b285d7400 0x564b2781e100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x564b27a92c00 msgr2=0x564b27a82b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x564b27a92c00 0x564b27a82b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x564b27aaa400 msgr2=0x564b27a83b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x564b27aaa400 0x564b27a83b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x564b27a93400 msgr2=0x564b27a83600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x564b27a93400 0x564b27a83600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x564b27aaa000 msgr2=0x564b27a84680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 reap_dead start 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x564b27aaa000 0x564b27a84680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x564b285e5800 msgr2=0x564b27a82000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x564b285e5800 0x564b27a82000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 reap_dead start 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x564b27a92000 msgr2=0x564b27a82580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x564b27a92000 0x564b27a82580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x564b27a92800 msgr2=0x564b27a84100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x564b27a92800 0x564b27a84100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x564b285e5400 msgr2=0x564b2781f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x564b285e5400 0x564b2781f700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x564b2842cdb0 tx=0x564b285e20c0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 msgr2=0x564b2781ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/2344646220 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b28437800 0x564b2781ec00 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x564b2842d920 tx=0x564b285e2240 comp rx=0 tx=0).stop 2024-04-25T10:14:07.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331d795640 1 -- 172.21.15.82:0/2344646220 reap_dead start 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 shutdown_connections 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 >> 172.21.15.82:0/2344646220 conn(0x564b27662c00 msgr2=0x564b27691800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 shutdown_connections 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/2344646220 wait complete. 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 msgr2=0x564b2781d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x564b284a70e0 tx=0x564b27760000 comp rx=0 tx=0).stop 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 shutdown_connections 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x564b27663400 0x564b2781cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x564b27663000 0x564b2781d080 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 --2- 172.21.15.82:0/1762354212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x564b27663800 0x564b2781c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 >> 172.21.15.82:0/1762354212 conn(0x564b27663c00 msgr2=0x564b27691000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 shutdown_connections 2024-04-25T10:14:07.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.399+0000 7f331efd68c0 1 -- 172.21.15.82:0/1762354212 wait complete. 2024-04-25T10:14:07.402 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.134377, result: 2024-04-25T10:14:07.403 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T10:14:07.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- start start 2024-04-25T10:14:07.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 Processor -- start 2024-04-25T10:14:07.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 0x55d0bbeaa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 0x55d0bbeab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0180 con 0x55d0bbcf1400 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0300 con 0x55d0bbcf1000 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0480 con 0x55d0bbcf1800 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41784/0 (socket says 172.21.15.82:41784) 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 -- 172.21.15.82:0/2324847980 learned_addr learned my addr 172.21.15.82:0/2324847980 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc660bbb640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 0x55d0bbeab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc6603ba640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 0x55d0bbeaa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 -- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 msgr2=0x55d0bbeaa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 0x55d0bbeaa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 -- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 msgr2=0x55d0bbeab080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 0x55d0bbeab080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 -- 172.21.15.82:0/2324847980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bbd161e0 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc6603ba640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 0x55d0bbeaa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc660bbb640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 0x55d0bbeab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x55d0bcb35890 tx=0x55d0bbdcb2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcf551e0a810eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65dbb5640 1 -- 172.21.15.82:0/2324847980 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd264e0 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65dbb5640 1 -- 172.21.15.82:0/2324847980 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0bbd3a540 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65dbb5640 1 -- 172.21.15.82:0/2324847980 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd26b60 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0bbd172c0 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d0bbd17a40 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 4 waiting for MDS map... 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65dbb5640 1 -- 172.21.15.82:0/2324847980 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d0bcc02600 con 0x55d0bbcf1400 2024-04-25T10:14:07.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc65dbb5640 1 -- 172.21.15.82:0/2324847980 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 2723+0+0 (secure 0 0 0) 0x55d0bbd47180 con 0x55d0bbcf1400 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 4 Got MDS map 98 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 4 datascan.main: Using metadata pool output 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.483+0000 7fc661bfb8c0 1 Processor -- start 2024-04-25T10:14:07.487 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- start start 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc08c00 0x55d0bbeab600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeabb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0600 con 0x55d0bcc09400 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0780 con 0x55d0bcc08c00 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf0900 con 0x55d0bcc09000 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41786/0 (socket says 172.21.15.82:41786) 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 -- 172.21.15.82:0/963662323 learned_addr learned my addr 172.21.15.82:0/963662323 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc08c00 0x55d0bbeab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc6603ba640 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeabb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 -- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 msgr2=0x55d0bbeabb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeabb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 -- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc08c00 msgr2=0x55d0bbeab600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc08c00 0x55d0bbeab600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 -- 172.21.15.82:0/963662323 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bbd17c20 con 0x55d0bcc09400 2024-04-25T10:14:07.494 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x55d0bcc1ea80 tx=0x55d0bbdaad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5639b48e745d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65bbb1640 1 -- 172.21.15.82:0/963662323 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd26340 con 0x55d0bcc09400 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65bbb1640 1 -- 172.21.15.82:0/963662323 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0bbd3ae00 con 0x55d0bcc09400 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65bbb1640 1 -- 172.21.15.82:0/963662323 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd264e0 con 0x55d0bcc09400 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/963662323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 msgr2=0x55d0bbeac100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x55d0bcc1ea80 tx=0x55d0bbdaad20 comp rx=0 tx=0).stop 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/963662323 shutdown_connections 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeabb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc09400 0x55d0bbeac100 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/963662323 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc08c00 0x55d0bbeab600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/963662323 >> 172.21.15.82:0/963662323 conn(0x55d0bbcf0c00 msgr2=0x55d0bbd1f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/963662323 shutdown_connections 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/963662323 wait complete. 2024-04-25T10:14:07.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 Processor -- start 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- start start 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeacc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc09400 0x55d0bbeac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf1c80 con 0x55d0bcc08c00 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf1b00 con 0x55d0bcc09400 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc661bfb8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d0bcbf1980 con 0x55d0bcc09000 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41798/0 (socket says 172.21.15.82:41798) 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 learned_addr learned my addr 172.21.15.82:0/3701294039 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeacc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc09400 0x55d0bbeac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 msgr2=0x55d0bbeacc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bcc09000 0x55d0bbeacc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc09400 msgr2=0x55d0bbeac680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bcc09400 0x55d0bbeac680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bbd161e0 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.487+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x55d0bcc1f740 tx=0x55d0bbdee000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 -- 172.21.15.82:0/3701294039 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd269c0 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 -- 172.21.15.82:0/3701294039 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d0bbd3ac40 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d0bbd172c0 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 -- 172.21.15.82:0/3701294039 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d0bbd26340 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0bbd17e00 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 -- 172.21.15.82:0/3701294039 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55d0bcc78c00 con 0x55d0bcc08c00 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 0x55d0bbead700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.497 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65a3ae640 1 -- 172.21.15.82:0/3701294039 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x55d0bc02c600 con 0x55d0bcc08c00 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 0x55d0bbead700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55d0bcc09800 0x55d0bbeac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc09c00 con 0x55d0bcc09800 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55d0bcc09800 0x55d0bbeac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 0x55d0bbead700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55d0bbdccea0 tx=0x55d0bc116000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55d0bcc09800 0x55d0bbeac100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv35 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55d0bc10f000 0x55d0bbeabb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10f400 con 0x55d0bc10f000 2024-04-25T10:14:07.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55d0bc10f000 0x55d0bbeabb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.491+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55d0bc10f000 0x55d0bbeabb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55d0bc10f800 0x55d0bbeab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10fc00 con 0x55d0bc10f800 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55d0bc10f800 0x55d0bbeab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55d0bc10f800 0x55d0bbeab600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65b3b0640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55d0bc130000 0x55d0bc136000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130400 con 0x55d0bc130000 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55d0bc130000 0x55d0bc136000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55d0bc130000 0x55d0bc136000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130c00 con 0x55d0bc10f000 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.495+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bc10f800 2024-04-25T10:14:07.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bc10f800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bc130000 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55d0bc131400 0x55d0bc136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08400 con 0x55d0bc131400 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55d0bc131400 0x55d0bc136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55d0bc131400 0x55d0bc136580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10ec00 con 0x55d0bcc09800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08800 con 0x55d0bc10f800 2024-04-25T10:14:07.505 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10fc00 con 0x55d0bc10f800 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc09c00 con 0x55d0bc10f000 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08400 con 0x55d0bc10f800 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.499+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130c00 con 0x55d0bc130000 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65b3b0640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55d0bcc09c00 0x55d0bc136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10f400 con 0x55d0bcc09c00 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55d0bcc09c00 0x55d0bc136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc6603ba640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55d0bcc09c00 0x55d0bc136b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130400 con 0x55d0bc131400 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65b3b0640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55d0bcac5400 0x55d0bc137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131800 con 0x55d0bcac5400 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55d0bcac5400 0x55d0bc137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65fbb9640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55d0bcac5400 0x55d0bc137080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.507 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bcac5400 2024-04-25T10:14:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.503+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131c00 con 0x55d0bcc09800 2024-04-25T10:14:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bcc09c00 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 123 out=81b] v123'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148000 con 0x55d0bc130000 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148000 con 0x55d0bc10f000 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148400 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130c00 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148800 con 0x55d0bc130000 2024-04-25T10:14:07.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131800 con 0x55d0bc131400 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.507+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148c00 con 0x55d0bcc09800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149000 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131c00 con 0x55d0bc10f000 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149400 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.513 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10ec00 con 0x55d0bc130000 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149800 con 0x55d0bcc09c00 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148400 con 0x55d0bc131400 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.514 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149c00 con 0x55d0bcac5400 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.511+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149000 con 0x55d0bcac5400 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e000 con 0x55d0bcc09800 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149400 con 0x55d0bcc09c00 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 123 out=49b] v123'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e400 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 123 out=49b] v121'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65bbb1640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55d0bc14e800 0x55d0bc137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14ec00 con 0x55d0bc14e800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55d0bc14e800 0x55d0bc137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc660bbb640 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55d0bc14e800 0x55d0bc137600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130c00 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f000 con 0x55d0bc10f800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.518 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.515+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bcac5400 2024-04-25T10:14:07.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f400 con 0x55d0bcc09c00 2024-04-25T10:14:07.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.519 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148000 con 0x55d0bcc09800 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f800 con 0x55d0bc131400 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 123 out=81b] v123'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 10 datascan.operator(): handling object 100000001f7.0 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10f400 con 0x55d0bc131400 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.522 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08800 con 0x55d0bc10f800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08400 con 0x55d0bcc09800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.get_frag_of: frag is * 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14ec00 con 0x55d0bc10f800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.523 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 10 datascan.find_or_create_dirfrag: read_version = 8 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e000 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.519+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e400 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v123'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc661bfb8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14fc00 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f000 con 0x55d0bcc09800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.524 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148400 con 0x55d0bc130000 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f400 con 0x55d0bcc09800 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148800 con 0x55d0bcc09800 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14ec00 con 0x55d0bc130000 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 123 out=49b] v121'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.527 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148c00 con 0x55d0bc131400 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.523+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e000 con 0x55d0bc14e800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 123 out=49b] v121'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10ec00 con 0x55d0bcc09800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08800 con 0x55d0bcac5400 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130400 con 0x55d0bc14e800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 123 out=49b] v123'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149800 con 0x55d0bc10f800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.528 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131800 con 0x55d0bc131400 2024-04-25T10:14:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 123 out=49b] v123'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.529 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149400 con 0x55d0bc10f000 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.527+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131c00 con 0x55d0bc130000 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148c00 con 0x55d0bc14e800 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.530 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f800 con 0x55d0bc14e800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bcc08400 con 0x55d0bc10f000 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.6 v2:172.21.15.196:6801/1119498737 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc10f000 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14e000 con 0x55d0bc10f800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.0 v2:172.21.15.82:6800/2082960483 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bc10f800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130400 con 0x55d0bcc09800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.3 v2:172.21.15.82:6824/3269667522 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcc09800 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f000 con 0x55d0bc131400 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.532 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131800 con 0x55d0bcc09c00 2024-04-25T10:14:07.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 123 out=49b] v121'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.533 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.531+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f400 con 0x55d0bcac5400 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 123 out=49b] v121'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bc131400 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 123 out=49b] v121'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148000 con 0x55d0bc130000 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.534 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130c00 con 0x55d0bcac5400 2024-04-25T10:14:07.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148400 con 0x55d0bc131400 2024-04-25T10:14:07.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.535 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bc14e800 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 123 out=49b] v121'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc149400 con 0x55d0bcc09c00 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 <== osd.2 v2:172.21.15.82:6803/3086763100 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 123 out=49b] v121'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4780 con 0x55d0bcc09c00 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10f400 con 0x55d0bc130000 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 123 out=49b] v121'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.536 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.535+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148c00 con 0x55d0bc130000 2024-04-25T10:14:07.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc10fc00 con 0x55d0bcac5400 2024-04-25T10:14:07.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 123 out=49b] v121'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc148400 con 0x55d0bc130000 2024-04-25T10:14:07.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.5 v2:172.21.15.196:6800/1745719129 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 123 out=49b] v121'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bc130000 2024-04-25T10:14:07.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc131000 con 0x55d0bc131400 2024-04-25T10:14:07.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.1 v2:172.21.15.82:6816/1010291016 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 123 out=49b] v121'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc131400 2024-04-25T10:14:07.538 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65b3b0640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc14f800 con 0x55d0bcac5400 2024-04-25T10:14:07.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65fbb9640 1 -- 172.21.15.82:0/3701294039 <== osd.7 v2:172.21.15.196:6822/2552340661 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 123 out=49b] v121'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4000 con 0x55d0bcac5400 2024-04-25T10:14:07.539 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc65bbb1640 1 -- 172.21.15.82:0/3701294039 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 123 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x55d0bc130800 con 0x55d0bc14e800 2024-04-25T10:14:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc660bbb640 1 -- 172.21.15.82:0/3701294039 <== osd.4 v2:172.21.15.196:6816/2164040014 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 123 out=49b] v121'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55d0bbda4500 con 0x55d0bc14e800 2024-04-25T10:14:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55d0bc10f800 msgr2=0x55d0bbeab600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x55d0bc10f800 0x55d0bbeab600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55d0bc131400 msgr2=0x55d0bc136580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x55d0bc131400 0x55d0bc136580 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55d0bcc09c00 msgr2=0x55d0bc136b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x55d0bcc09c00 0x55d0bc136b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55d0bcc09800 msgr2=0x55d0bbeac100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x55d0bcc09800 0x55d0bbeac100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55d0bc14e800 msgr2=0x55d0bc137600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x55d0bc14e800 0x55d0bc137600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 reap_dead start 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55d0bc130000 msgr2=0x55d0bc136000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x55d0bc130000 0x55d0bc136000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55d0bc10f000 msgr2=0x55d0bbeabb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x55d0bc10f000 0x55d0bbeabb80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55d0bcac5400 msgr2=0x55d0bc137080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x55d0bcac5400 0x55d0bc137080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 reap_dead start 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc6603ba640 1 -- 172.21.15.82:0/3701294039 reap_dead start 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 msgr2=0x55d0bbead700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 0x55d0bbead700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55d0bbdccea0 tx=0x55d0bc116000 comp rx=0 tx=0).stop 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 msgr2=0x55d0bbead180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x55d0bcc1f740 tx=0x55d0bbdee000 comp rx=0 tx=0).stop 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 shutdown_connections 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x55d0bc10e000 0x55d0bbead700 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/3701294039 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bcc08c00 0x55d0bbead180 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 >> 172.21.15.82:0/3701294039 conn(0x55d0bbcf0c00 msgr2=0x55d0bbd1f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 shutdown_connections 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.539+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/3701294039 wait complete. 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 msgr2=0x55d0bbeaab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x55d0bcb35890 tx=0x55d0bbdcb2c0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 shutdown_connections 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x55d0bbcf1000 0x55d0bbeab080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x55d0bbcf1400 0x55d0bbeaab00 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 --2- 172.21.15.82:0/2324847980 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x55d0bbcf1800 0x55d0bbeaa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 >> 172.21.15.82:0/2324847980 conn(0x55d0bbcf1c00 msgr2=0x55d0bbd1f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 shutdown_connections 2024-04-25T10:14:07.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.543+0000 7fc661bfb8c0 1 -- 172.21.15.82:0/2324847980 wait complete. 2024-04-25T10:14:07.545 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.142375, result: 2024-04-25T10:14:07.545 DEBUG:teuthology.orchestra.run.smithi082:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T10:14:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 -- start start 2024-04-25T10:14:07.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 Processor -- start 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590aba61c00 0x5590abc1ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590aba61800 0x5590abc1bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974000 con 0x5590ac836800 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974180 con 0x5590aba61800 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974300 con 0x5590aba61c00 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41822/0 (socket says 172.21.15.82:41822) 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 -- 172.21.15.82:0/3725188251 learned_addr learned my addr 172.21.15.82:0/3725188251 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b73d78640 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590aba61800 0x5590abc1bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74579640 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590aba61c00 0x5590abc1ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 -- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590aba61c00 msgr2=0x5590abc1ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590aba61c00 0x5590abc1ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 -- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590aba61800 msgr2=0x5590abc1bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590aba61800 0x5590abc1bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 -- 172.21.15.82:0/3725188251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590aba861e0 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x5590ac8b7920 tx=0x5590abb3aea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d851b41956ca8a40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.623+0000 7f6b71d74640 1 -- 172.21.15.82:0/3725188251 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba961a0 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b71d74640 1 -- 172.21.15.82:0/3725188251 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590abaaac40 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b71d74640 1 -- 172.21.15.82:0/3725188251 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba964e0 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590aba87e00 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5590ac992000 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 4 waiting for MDS map... 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b71d74640 1 -- 172.21.15.82:0/3725188251 <== mon.0 v2:172.21.15.82:3300/0 4 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x5590ac988600 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b71d74640 1 -- 172.21.15.82:0/3725188251 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 98) v1 ==== 2723+0+0 (secure 0 0 0) 0x5590abab7180 con 0x5590ac836800 2024-04-25T10:14:07.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 4 Got MDS map 98 2024-04-25T10:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 4 datascan.main: Using metadata pool output 2024-04-25T10:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 4 datascan.main: connecting to RADOS... 2024-04-25T10:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 Processor -- start 2024-04-25T10:14:07.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- start start 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974480 con 0x5590ac98cc00 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974600 con 0x5590ac98d000 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590ac974780 con 0x5590ac98c800 2024-04-25T10:14:07.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41824/0 (socket says 172.21.15.82:41824) 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 -- 172.21.15.82:0/4057323697 learned_addr learned my addr 172.21.15.82:0/4057323697 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b73d78640 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 -- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 msgr2=0x5590abc1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 -- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98d000 msgr2=0x5590abc1c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 -- 172.21.15.82:0/4057323697 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ac9921e0 con 0x5590ac98cc00 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b74579640 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a2b70 tx=0x5590ac9b0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6be9c385d0ea61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b6fd70640 1 -- 172.21.15.82:0/4057323697 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba961a0 con 0x5590ac98cc00 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b6fd70640 1 -- 172.21.15.82:0/4057323697 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590abaaaa80 con 0x5590ac98cc00 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b6fd70640 1 -- 172.21.15.82:0/4057323697 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba964e0 con 0x5590ac98cc00 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/4057323697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 msgr2=0x5590abc1c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a2b70 tx=0x5590ac9b0c00 comp rx=0 tx=0).stop 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/4057323697 shutdown_connections 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/4057323697 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/4057323697 >> 172.21.15.82:0/4057323697 conn(0x5590ac836000 msgr2=0x5590aba8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/4057323697 shutdown_connections 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.627+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/4057323697 wait complete. 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 Processor -- start 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- start start 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac975e00 con 0x5590ac98d000 2024-04-25T10:14:07.635 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ac975c80 con 0x5590ac98cc00 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590ac975b00 con 0x5590ac98c800 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41840/0 (socket says 172.21.15.82:41840) 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 learned_addr learned my addr 172.21.15.82:0/2644718913 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 msgr2=0x5590abc1a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590ac98c800 0x5590abc1a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98cc00 msgr2=0x5590abc1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ac9923c0 con 0x5590ac98d000 2024-04-25T10:14:07.636 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590ac98cc00 0x5590abc1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a3860 tx=0x5590ac824480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 -- 172.21.15.82:0/2644718913 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba96b60 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 -- 172.21.15.82:0/2644718913 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590abaaa540 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5590ac9925a0 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 -- 172.21.15.82:0/2644718913 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5590aba961a0 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590aba87c20 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 -- 172.21.15.82:0/2644718913 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5590ac986c00 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 0x5590abc1d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b6e56d640 1 -- 172.21.15.82:0/2644718913 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x5590ac984a00 con 0x5590ac98d000 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 0x5590abc1d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 4 datascan.main: resolving metadata pool 17 2024-04-25T10:14:07.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5590ac98d400 0x5590abc1cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590ac98d800 con 0x5590ac98d400 2024-04-25T10:14:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.631+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5590ac98d400 0x5590abc1cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 0x5590abc1d700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a2b70 tx=0x5590ac8242a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5590ac98d400 0x5590abc1cc00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== osd_op_reply(1 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5590ac98d800 0x5590abc1b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590ac837c00 con 0x5590ac98d800 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5590ac98d800 0x5590abc1b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5590ac98d800 0x5590abc1b600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== osd_op_reply(2 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590ac837c00 con 0x5590ac98d400 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== osd_op_reply(3 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5590ac837c00 0x5590abe98000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9cc00 con 0x5590ac837c00 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5590ac837c00 0x5590abe98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5590ac837c00 0x5590abe98000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== osd_op_reply(4 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.635+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9cc00 con 0x5590ac837c00 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== osd_op_reply(5 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9cc00 con 0x5590ac98d800 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== osd_op_reply(6 [pgnls start_epoch 123 out=149b] v123'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9cc00 con 0x5590ac98d400 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 3 ==== osd_op_reply(7 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5590abd9cc00 0x5590abe98580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d000 con 0x5590abd9cc00 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5590abd9cc00 0x5590abe98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5590abd9cc00 0x5590abe98580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== osd_op_reply(8 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5590abd9d000 0x5590abe98b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d400 con 0x5590abd9d000 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5590abd9d000 0x5590abe98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5590abd9d000 0x5590abe98b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== osd_op_reply(9 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.639+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d400 con 0x5590ac98d800 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 3 ==== osd_op_reply(10 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d400 con 0x5590ac98d400 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 4 ==== osd_op_reply(11 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d400 con 0x5590ac837c00 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 3 ==== osd_op_reply(12 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5590abd9d400 0x5590abe99080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9c400 con 0x5590abd9d400 2024-04-25T10:14:07.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5590abd9d400 0x5590abe99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5590abd9d400 0x5590abe99080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== osd_op_reply(13 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5590abd9c400 0x5590abe99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9dc00 con 0x5590abd9c400 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5590abd9c400 0x5590abe99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.643+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5590abd9c400 0x5590abe99600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== osd_op_reply(14 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5590abd9dc00 0x5590abe9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9dc00 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5590abd9dc00 0x5590abe9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5590abd9dc00 0x5590abe9a100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== osd_op_reply(15 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9dc00 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9cc00 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== osd_op_reply(16 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac837c00 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 4 ==== osd_op_reply(17 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d800 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 4 ==== osd_op_reply(18 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d800 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 5 ==== osd_op_reply(19 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9c400 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== osd_op_reply(20 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9d400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== osd_op_reply(21 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.647+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9c400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 3 ==== osd_op_reply(22 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 5 ==== osd_op_reply(23 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.650 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d400 2024-04-25T10:14:07.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 6 ==== osd_op_reply(24 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9d000 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== osd_op_reply(25 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d800 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 6 ==== osd_op_reply(26 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac837c00 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 5 ==== osd_op_reply(27 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.651+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9d000 2024-04-25T10:14:07.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 3 ==== osd_op_reply(28 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac837c00 2024-04-25T10:14:07.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 6 ==== osd_op_reply(29 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9d400 2024-04-25T10:14:07.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 3 ==== osd_op_reply(30 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9cc00 2024-04-25T10:14:07.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 3 ==== osd_op_reply(31 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.654 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590abd9d400 2024-04-25T10:14:07.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 4 ==== osd_op_reply(32 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T10:14:07.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8000 con 0x5590ac98d800 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac98d800 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T10:14:07.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8800 con 0x5590ac98d800 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T10:14:07.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.655+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9000 con 0x5590ac98d800 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9400 con 0x5590ac98d800 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8000 con 0x5590abd9cc00 2024-04-25T10:14:07.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac98d800 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8800 con 0x5590abd9dc00 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9dc00 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T10:14:07.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9800 con 0x5590abd9c400 2024-04-25T10:14:07.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9c00 con 0x5590ac98d400 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590ac98c400 con 0x5590ac98d800 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.663 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.659+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9000 con 0x5590ac837c00 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T10:14:07.664 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb9400 con 0x5590abd9d400 2024-04-25T10:14:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac98d400 2024-04-25T10:14:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 8 ==== osd_op_reply(48 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac98d800 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 15 ==== osd_op_reply(49 [pgnls start_epoch 123 out=177b] v122'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac98d400 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 9 ==== osd_op_reply(50 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8400 con 0x5590ac837c00 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 9 ==== osd_op_reply(51 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 10 ==== osd_op_reply(52 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.663+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 16 ==== osd_op_reply(53 [pgnls start_epoch 123 out=149b] v123'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d400 2024-04-25T10:14:07.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 10 ==== osd_op_reply(54 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9cc00 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 5 ==== osd_op_reply(55 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d000 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 4 ==== osd_op_reply(56 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 17 ==== osd_op_reply(57 [pgnls start_epoch 123 out=81b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d400 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 11 ==== osd_op_reply(58 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 11 ==== osd_op_reply(59 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.667+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d400 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 6 ==== osd_op_reply(60 [pgnls start_epoch 123 out=77b] v122'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9c400 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 5 ==== osd_op_reply(61 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9dc00 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.2 v2:172.21.15.82:6803/3086763100 3 ==== osd_op_reply(62 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9dc00 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9cc00 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 6 ==== osd_op_reply(63 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.670 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 12 ==== osd_op_reply(64 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 18 ==== osd_op_reply(65 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 19 ==== osd_op_reply(66 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9c400 2024-04-25T10:14:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 6 ==== osd_op_reply(67 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d400 2024-04-25T10:14:07.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 7 ==== osd_op_reply(68 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.671+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9c400 2024-04-25T10:14:07.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 7 ==== osd_op_reply(69 [pgnls start_epoch 123 out=97b] v122'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d400 2024-04-25T10:14:07.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 12 ==== osd_op_reply(70 [pgnls start_epoch 123 out=74b] v120'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.673 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d400 2024-04-25T10:14:07.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 13 ==== osd_op_reply(71 [pgnls start_epoch 123 out=73b] v122'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.674 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d000 2024-04-25T10:14:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 5 ==== osd_op_reply(72 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac98d800 2024-04-25T10:14:07.675 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 20 ==== osd_op_reply(73 [pgnls start_epoch 123 out=98b] v122'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 13 ==== osd_op_reply(74 [pgnls start_epoch 123 out=76b] v123'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d000 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.4 v2:172.21.15.196:6816/2164040014 6 ==== osd_op_reply(75 [pgnls start_epoch 123 out=49b] v123'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9d000 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.675+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590ac837c00 2024-04-25T10:14:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 14 ==== osd_op_reply(76 [pgnls start_epoch 123 out=105b] v123'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d400 2024-04-25T10:14:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 8 ==== osd_op_reply(77 [pgnls start_epoch 123 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.677 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9cc00 2024-04-25T10:14:07.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 7 ==== osd_op_reply(78 [pgnls start_epoch 123 out=73b] v123'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.678 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 123 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8c00 con 0x5590abd9d400 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 9 ==== osd_op_reply(79 [pgnls start_epoch 123 out=101b] v123'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8800 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe000 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe400 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe800 con 0x5590ac98d800 2024-04-25T10:14:07.680 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T10:14:07.681 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebec00 con 0x5590ac98d800 2024-04-25T10:14:07.682 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.679+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebf000 con 0x5590ac98d800 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv37 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebf400 con 0x5590abd9cc00 2024-04-25T10:14:07.683 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.1 v2:172.21.15.82:6816/1010291016 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9cc00 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebf800 con 0x5590ac98d800 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebfc00 con 0x5590abd9dc00 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.2 v2:172.21.15.82:6803/3086763100 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70500 con 0x5590abd9dc00 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe000 con 0x5590abd9c400 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.5 v2:172.21.15.196:6800/1745719129 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590abd9c400 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T10:14:07.684 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T10:14:07.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T10:14:07.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T10:14:07.685 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe400 con 0x5590ac98d400 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe800 con 0x5590ac98d800 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.687 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.683+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebec00 con 0x5590ac837c00 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abeb8800 con 0x5590ac837c00 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 20 datascan.scan_links: injected ino detected 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T10:14:07.688 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590ac98c400 con 0x5590abd9d400 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.6 v2:172.21.15.196:6801/1119498737 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x5590abb70780 con 0x5590abd9d400 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebec00 con 0x5590ac837c00 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 20 datascan.scan_links: first is now 2 2024-04-25T10:14:07.689 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.687+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebf400 con 0x5590ac837c00 2024-04-25T10:14:07.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 <== osd.0 v2:172.21.15.82:6800/2082960483 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v123'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5590abb70780 con 0x5590ac837c00 2024-04-25T10:14:07.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T10:14:07.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 10 datascan.scan_links: updating inotable 2024-04-25T10:14:07.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe000 con 0x5590ac98d800 2024-04-25T10:14:07.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-25T10:14:07.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-25T10:14:07.691 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.691+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abebe400 con 0x5590ac98d800 2024-04-25T10:14:07.692 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b74d7a640 1 -- 172.21.15.82:0/2644718913 <== osd.3 v2:172.21.15.82:6824/3269667522 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v123'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5590abb70500 con 0x5590ac98d800 2024-04-25T10:14:07.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 10 datascan.scan_links: updating snaptable 2024-04-25T10:14:07.693 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e123) v9 -- 0x5590abd9d800 con 0x5590ac98d400 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b73d78640 1 -- 172.21.15.82:0/2644718913 <== osd.7 v2:172.21.15.196:6822/2552340661 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x5590abb70000 con 0x5590ac98d400 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5590ac837c00 msgr2=0x5590abe98000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x5590ac837c00 0x5590abe98000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5590abd9cc00 msgr2=0x5590abe98580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x5590abd9cc00 0x5590abe98580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5590abd9dc00 msgr2=0x5590abe9a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x5590abd9dc00 0x5590abe9a100 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5590ac98d800 msgr2=0x5590abc1b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 reap_dead start 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x5590ac98d800 0x5590abc1b600 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.696 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5590abd9d000 msgr2=0x5590abe98b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x5590abd9d000 0x5590abe98b00 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5590abd9c400 msgr2=0x5590abe99600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x5590abd9c400 0x5590abe99600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5590abd9d400 msgr2=0x5590abe99080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x5590abd9d400 0x5590abe99080 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5590ac98d400 msgr2=0x5590abc1cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x5590ac98d400 0x5590abc1cc00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b74579640 1 -- 172.21.15.82:0/2644718913 reap_dead start 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 msgr2=0x5590abc1d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 0x5590abc1d700 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a2b70 tx=0x5590ac8242a0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 msgr2=0x5590abc1c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x5590ac9a3860 tx=0x5590ac824480 comp rx=0 tx=0).stop 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 shutdown_connections 2024-04-25T10:14:07.697 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x5590abd9c800 0x5590abc1d700 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/2644718913 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac98d000 0x5590abc1c680 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 >> 172.21.15.82:0/2644718913 conn(0x5590ac836000 msgr2=0x5590aba8d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 shutdown_connections 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/2644718913 wait complete. 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 msgr2=0x5590abc1b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x5590ac8b7920 tx=0x5590abb3aea0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 shutdown_connections 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x5590ac836800 0x5590abc1b080 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x5590aba61800 0x5590abc1bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 --2- 172.21.15.82:0/3725188251 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x5590aba61c00 0x5590abc1ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 >> 172.21.15.82:0/3725188251 conn(0x5590aba60c00 msgr2=0x5590aba8e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 shutdown_connections 2024-04-25T10:14:07.698 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:07.695+0000 7f6b75dba8c0 1 -- 172.21.15.82:0/3725188251 wait complete. 2024-04-25T10:14:07.699 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.154187, result: 2024-04-25T10:14:07.700 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:07.744 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:07.770 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T10:14:07.770 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T10:14:07.770 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.771 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:07.772 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T10:14:07.773 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.774 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.775 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:07.776 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.777 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.779 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T10:14:07.780 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = False 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.781 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.782 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T10:14:07.782 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T10:14:07.782 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:07.782 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T10:14:07.782 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.783 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:07.784 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T10:14:07.785 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.786 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T10:14:07.787 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.788 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:07.789 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T10:14:07.790 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T10:14:07.791 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = False 2024-04-25T10:14:07.792 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T10:14:07.793 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-04-25T10:14:07.793 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:14:07.793 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.793 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.795 DEBUG:teuthology.orchestra.run.smithi082:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:07.842 DEBUG:teuthology.orchestra.run.smithi196:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout:[global] 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: chdir = "" 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: auth supported = cephx 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: filestore xattr use omap = true 2024-04-25T10:14:07.867 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: auth debug = true 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on old message = true 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: ms die on bug = true 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.868 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default size = 2 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd allow pg remap = true 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon warn on too few osds = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.869 INFO:teuthology.orchestra.run.smithi082.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: osd default data pool replay window = 5 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: mon allow pool delete = true 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: mon cluster log file level = debug 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: debug asserts on shutdown = true 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: mon health detail to clog = false 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.870 INFO:teuthology.orchestra.run.smithi082.stdout:[osd] 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd journal size = 100 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd scrub max interval = 600 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd recover clone overlap = true 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug shutdown = true 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug op order = true 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug trim objects = true 2024-04-25T10:14:07.871 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: osd open classes on start = true 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug pg log writeout = true 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: osd map max advance = 10 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: journal zero on create = true 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.872 INFO:teuthology.orchestra.run.smithi082.stdout: bdev debug aio = true 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: osd debug misdirected ops = true 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore block size = 96636764160 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: bluestore fsck on mount = True 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluefs = 20 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: debug bluestore = 20 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: debug osd = 20 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: debug rocksdb = 10 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: osd objectstore = bluestore 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:14:07.873 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout:[mgr] 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug mgr = 20 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout:[mon] 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug mon = 20 2024-04-25T10:14:07.874 INFO:teuthology.orchestra.run.smithi082.stdout: debug paxos = 20 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: debug auth = 20 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon data avail warn = 5 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon osd prime pg temp = true 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:07.875 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: # 1m isn't quite enough 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: mon op complaint time = 120 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout:[client] 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: rgw cache enabled = true 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable ops log = true 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: rgw enable usage log = true 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:07.876 INFO:teuthology.orchestra.run.smithi082.stdout: client mount timeout = 600 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: debug client = 20 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: pid file = "" 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.a] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.b] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mon.c] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.a] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.c] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.b] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mds.d] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout:[mds] 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds = 20 2024-04-25T10:14:07.877 INFO:teuthology.orchestra.run.smithi082.stdout: debug mds balancer = 20 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: debug ms = 1 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug frag = True 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: mds debug scatterstat = True 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: mds op complaint time = 180 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: mds verify scatter = True 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: osd op complaint time = 180 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.878 INFO:teuthology.orchestra.run.smithi082.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.879 INFO:teuthology.orchestra.run.smithi196.stdout:[global] 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: chdir = "" 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: auth supported = cephx 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: filestore xattr use omap = true 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: mon clock drift allowed = 1.000 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: osd crush chooseleaf type = 0 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: auth debug = true 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on old message = true 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi196.stdout: ms die on bug = true 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon pg warn max object skew = 0 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default size = 2 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow primary affinity = true 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd allow pg remap = true 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on no sortbitwise = false 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon warn on too few osds = false 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon_allow_pool_size_one = true 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: osd default data pool replay window = 5 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon allow pool delete = true 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.882 INFO:teuthology.orchestra.run.smithi196.stdout: mon cluster log file level = debug 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: debug asserts on shutdown = true 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: mon health detail to clog = false 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: mon host = "172.21.15.82,172.21.15.196,[v2:172.21.15.196:3301,v1:172.21.15.196:6790]" 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: fsid = 923528c9-0d4c-4379-be0f-2d1b434d3c6c 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout:[osd] 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd journal size = 100 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub load threshold = 5.0 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd scrub max interval = 600 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd recover clone overlap = true 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: osd recovery max chunk = 1048576 2024-04-25T10:14:07.883 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug shutdown = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug op order = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug verify stray on activate = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug trim objects = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd open classes on start = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug pg log writeout = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: osd map max advance = 10 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: journal zero on create = true 2024-04-25T10:14:07.884 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: filestore apply finisher threads = 3 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: bdev debug aio = true 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: osd debug misdirected ops = true 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore block size = 96636764160 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: bluestore fsck on mount = True 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluefs = 20 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: debug bluestore = 20 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: debug osd = 20 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: debug rocksdb = 10 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:14:07.885 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd full ratio = 0.9 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: osd objectstore = bluestore 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout:[mgr] 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: debug mgr = 20 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: mgr/telemetry/nag = false 2024-04-25T10:14:07.886 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout:[mon] 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: debug mon = 20 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: debug paxos = 20 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: debug auth = 20 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon data avail warn = 5 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon mgr mkfs grace = 240 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd reporter subtree level = osd 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon osd prime pg temp = true 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:14:07.887 INFO:teuthology.orchestra.run.smithi196.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: # 1m isn't quite enough 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: mon op complaint time = 120 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout:[client] 2024-04-25T10:14:07.888 INFO:teuthology.orchestra.run.smithi196.stdout: rgw cache enabled = true 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable ops log = true 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: rgw enable usage log = true 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: client mount timeout = 600 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: debug client = 20 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: pid file = "" 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.a] 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.b] 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout:[mon.c] 2024-04-25T10:14:07.889 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.a] 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.c] 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.b] 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout:[mds.d] 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout:[mds] 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds = 20 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: debug mds balancer = 20 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: debug ms = 1 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug frag = True 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: mds debug scatterstat = True 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: mds op complaint time = 180 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: mds verify scatter = True 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: osd op complaint time = 180 2024-04-25T10:14:07.890 INFO:teuthology.orchestra.run.smithi196.stdout: rados mon op timeout = 900 2024-04-25T10:14:07.891 INFO:teuthology.orchestra.run.smithi196.stdout: rados osd op timeout = 900 2024-04-25T10:14:07.891 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:14:08.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 Processor -- start 2024-04-25T10:14:08.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 -- start start 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 0x7f0dac0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0dac072000 0x7f0dac072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dac0cbd20 con 0x7f0dac15b6b0 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dac15ba90 con 0x7f0dac072000 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db4137640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0cff30 con 0x7f0dac0cf590 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db26d4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0dac072000 0x7f0dac072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db1ed3640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f0dac15b6b0 0x7f0dac0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59218/0 (socket says 172.21.15.82:59218) 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db1ed3640 1 -- 172.21.15.82:0/4064164871 learned_addr learned my addr 172.21.15.82:0/4064164871 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:08.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db16d2640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 0x7f0dac0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db26d4640 1 -- 172.21.15.82:0/4064164871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 msgr2=0x7f0dac0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db26d4640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 0x7f0dac0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db26d4640 1 -- 172.21.15.82:0/4064164871 >> v1:172.21.15.82:6789/0 conn(0x7f0dac15b6b0 legacy=0x7f0dac0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db0ed1640 1 -- 172.21.15.82:0/4064164871 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947170715 0 0) 0x7f0dac0cbd20 con 0x7f0dac15b6b0 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db26d4640 1 -- 172.21.15.82:0/4064164871 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0cc000 con 0x7f0dac072000 2024-04-25T10:14:08.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.050+0000 7f0db16d2640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 0x7f0dac0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:08.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac072000 0x7f0dac072470 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d90 tx=0x7f0da401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f69d4ad9c6ea14d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db0ed1640 1 -- 172.21.15.82:0/4064164871 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40b2070 con 0x7f0dac072000 2024-04-25T10:14:08.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db0ed1640 1 -- 172.21.15.82:0/4064164871 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da40a5a00 con 0x7f0dac072000 2024-04-25T10:14:08.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db0ed1640 1 -- 172.21.15.82:0/4064164871 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40a5dd0 con 0x7f0dac072000 2024-04-25T10:14:08.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- 172.21.15.82:0/4064164871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac072000 msgr2=0x7f0dac072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac072000 0x7f0dac072470 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d90 tx=0x7f0da401c940 comp rx=0 tx=0).stop 2024-04-25T10:14:08.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- 172.21.15.82:0/4064164871 shutdown_connections 2024-04-25T10:14:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac072000 0x7f0dac072470 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- 172.21.15.82:0/4064164871 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac0cf590 0x7f0dac0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- 172.21.15.82:0/4064164871 >> 172.21.15.82:0/4064164871 conn(0x7f0dac06ae60 msgr2=0x7f0dac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- 172.21.15.82:0/4064164871 shutdown_connections 2024-04-25T10:14:08.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- 172.21.15.82:0/4064164871 wait complete. 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 Processor -- start 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- start start 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac072000 0x7f0dac1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dac0cf590 0x7f0dac155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0cadb0 con 0x7f0dac15b6b0 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0cba30 con 0x7f0dac0cf590 2024-04-25T10:14:08.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db4137640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dac0caf30 con 0x7f0dac072000 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41856/0 (socket says 172.21.15.82:41856) 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 -- 172.21.15.82:0/1565464212 learned_addr learned my addr 172.21.15.82:0/1565464212 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 -- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac072000 msgr2=0x7f0dac1f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac072000 0x7f0dac1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 -- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dac0cf590 msgr2=0x7f0dac155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dac0cf590 0x7f0dac155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.054+0000 7f0db26d4640 1 -- 172.21.15.82:0/1565464212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dac0ce310 con 0x7f0dac15b6b0 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db26d4640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d90 tx=0x7f0da40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40b2050 con 0x7f0dac15b6b0 2024-04-25T10:14:08.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da40b3040 con 0x7f0dac15b6b0 2024-04-25T10:14:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dac0ce5f0 con 0x7f0dac15b6b0 2024-04-25T10:14:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40ada50 con 0x7f0dac15b6b0 2024-04-25T10:14:08.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dac0cebb0 con 0x7f0dac15b6b0 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dac04efa0 con 0x7f0dac15b6b0 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0da40c0020 con 0x7f0dac15b6b0 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 0x7f0d90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db1ed3640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 0x7f0d90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f0da411ffb0 con 0x7f0dac15b6b0 2024-04-25T10:14:08.060 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.058+0000 7f0db1ed3640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 0x7f0d90069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0015a0 tx=0x7f0d9c007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:08.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.062+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0da4016560 con 0x7f0dac15b6b0 2024-04-25T10:14:08.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:08.258+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0dac165760 con 0x7f0dac15b6b0 2024-04-25T10:14:08.886 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:08.889 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.190+0000 7f0da27fc640 1 -- 172.21.15.82:0/1565464212 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v100) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0da40f0d80 con 0x7f0dac15b6b0 2024-04-25T10:14:09.192 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:14:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 msgr2=0x7f0d90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 0x7f0d90069420 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0015a0 tx=0x7f0d9c007970 comp rx=0 tx=0).stop 2024-04-25T10:14:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 msgr2=0x7f0dac164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009d90 tx=0x7f0da40196e0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 shutdown_connections 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0d90066f60 0x7f0d90069420 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0dac15b6b0 0x7f0dac164670 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0dac0cf590 0x7f0dac155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 --2- 172.21.15.82:0/1565464212 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0dac072000 0x7f0dac1f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 >> 172.21.15.82:0/1565464212 conn(0x7f0dac06ae60 msgr2=0x7f0dac0b3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.197 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 shutdown_connections 2024-04-25T10:14:09.198 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.194+0000 7f0db4137640 1 -- 172.21.15.82:0/1565464212 wait complete. 2024-04-25T10:14:09.221 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:09.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.338+0000 7f2453238640 1 Processor -- start 2024-04-25T10:14:09.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.338+0000 7f2453238640 1 -- start start 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2453238640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f244c0ce510 0x7f244c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2453238640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c0cee30 con 0x7f244c144ca0 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2453238640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244c0ccc70 con 0x7f244c1451f0 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2453238640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244c0cc2b0 con 0x7f244c0ce510 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f244c144ca0 0x7f244c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59220/0 (socket says 172.21.15.82:59220) 2024-04-25T10:14:09.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 -- 172.21.15.82:0/3466382373 learned_addr learned my addr 172.21.15.82:0/3466382373 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:09.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3466382373 >> v2:172.21.15.196:3300/0 conn(0x7f244c0ce510 0x7f244c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1858505636 0 0) 0x7f244c0cee30 con 0x7f244c144ca0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2434003530 con 0x7f244c144ca0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 722097403 0 0) 0x7f2434003530 con 0x7f244c144ca0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 >> v2:172.21.15.196:3300/0 conn(0x7f244c0ce510 msgr2=0x7f244c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 --2- 172.21.15.82:0/3466382373 >> v2:172.21.15.196:3300/0 conn(0x7f244c0ce510 0x7f244c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 >> v1:172.21.15.196:6789/0 conn(0x7f244c1451f0 legacy=0x7f244c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c06dc40 con 0x7f244c144ca0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f2440001680 con 0x7f244c144ca0 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 >> v1:172.21.15.82:6789/0 conn(0x7f244c144ca0 legacy=0x7f244c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 0x7f2434004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0cc2b0 con 0x7f2434004810 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243400c0e0 con 0x7f24340051d0 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243400c2e0 con 0x7f2434003a30 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 0x7f2434004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 -- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434003a30 msgr2=0x7f2434003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 -- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24340051d0 msgr2=0x7f243400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f24340051d0 0x7f243400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f24517d5640 1 -- 172.21.15.82:0/3466382373 reap_dead start 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 -- 172.21.15.82:0/3466382373 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434003530 con 0x7f2434004810 2024-04-25T10:14:09.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434003a30 0x7f2434003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.342+0000 7f244bfff640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 0x7f2434004c90 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f244c06f4c0 tx=0x7f244001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9b0719e375cf51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440007d40 con 0x7f2434004810 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24400a9020 con 0x7f2434004810 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244b7fe640 1 -- 172.21.15.82:0/3466382373 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24400a5b30 con 0x7f2434004810 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 msgr2=0x7f2434004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 0x7f2434004c90 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f244c06f4c0 tx=0x7f244001ccc0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3466382373 shutdown_connections 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 --2- 172.21.15.82:0/3466382373 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2434004810 0x7f2434004c90 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3466382373 >> 172.21.15.82:0/3466382373 conn(0x7f244c0b11a0 msgr2=0x7f244c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3466382373 shutdown_connections 2024-04-25T10:14:09.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3466382373 wait complete. 2024-04-25T10:14:09.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 Processor -- start 2024-04-25T10:14:09.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- start start 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f244c1e0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 0x7f244c143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0cc900 con 0x7f244c0ac720 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244c06e290 con 0x7f244c143600 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244c0ce8f0 con 0x7f2434004810 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41866/0 (socket says 172.21.15.82:41866) 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 -- 172.21.15.82:0/3296458010 learned_addr learned my addr 172.21.15.82:0/3296458010 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f24517d5640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 0x7f244c143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f244c1e0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 -- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 msgr2=0x7f244c1e0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f244c1e0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 -- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 msgr2=0x7f244c143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 0x7f244c143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 -- 172.21.15.82:0/3296458010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f244c0ace90 con 0x7f244c0ac720 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f24517d5640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 0x7f244c143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f244bfff640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f2440001980 tx=0x7f2434005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24400a5d40 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24400b2070 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244c0ad170 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440004a10 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.346+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244c0ad730 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244c04efa0 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2440004bb0 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f24497fa640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 0x7f24380693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f244011ed80 con 0x7f244c0ac720 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 0x7f24380693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:09.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.350+0000 7f2450fd4640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 0x7f24380693c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f244c050870 tx=0x7f243c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:09.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.354+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f24400169d0 con 0x7f244c0ac720 2024-04-25T10:14:09.546 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.546+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f244c141f60 con 0x7f244c0ac720 2024-04-25T10:14:09.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.546+0000 7f24497fa640 1 -- 172.21.15.82:0/3296458010 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f24400efb50 con 0x7f244c0ac720 2024-04-25T10:14:09.547 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:09.547 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":101,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:09.198231+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:replay","state_seq":1,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:09.547 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 101 2024-04-25T10:14:09.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 msgr2=0x7f24380693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 0x7f24380693c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f244c050870 tx=0x7f243c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 msgr2=0x7f244c13de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7f2440001980 tx=0x7f2434005840 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 shutdown_connections 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2438066f00 0x7f24380693c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f244c143600 0x7f244c143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f244c0ac720 0x7f244c13de10 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 --2- 172.21.15.82:0/3296458010 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2434004810 0x7f244c1e0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 >> 172.21.15.82:0/3296458010 conn(0x7f244c0b11a0 msgr2=0x7f244c06d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 shutdown_connections 2024-04-25T10:14:09.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:09.550+0000 7f2453238640 1 -- 172.21.15.82:0/3296458010 wait complete. 2024-04-25T10:14:09.970 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:14:09.198231+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10068}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10068': {'gid': 10068, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.82:6837/2659325772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 2659325772}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2659325772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:09.970 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10068:up:replay 2024-04-25T10:14:10.971 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:11.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.102+0000 7f8b3b3c4640 1 Processor -- start 2024-04-25T10:14:11.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.102+0000 7f8b3b3c4640 1 -- start start 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b3b3c4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f8b34071fd0 0x7f8b3415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b3b3c4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3415ae30 con 0x7f8b34159ea0 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b3b3c4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b3415b110 con 0x7f8b3415a280 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b3b3c4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b340b2990 con 0x7f8b34071fd0 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8b34159ea0 0x7f8b340b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59228/0 (socket says 172.21.15.82:59228) 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 -- 172.21.15.82:0/1593889119 learned_addr learned my addr 172.21.15.82:0/1593889119 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39961640 1 --2- 172.21.15.82:0/1593889119 >> v2:172.21.15.82:3300/0 conn(0x7f8b34071fd0 0x7f8b3415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849796304 0 0) 0x7f8b3415ae30 con 0x7f8b34159ea0 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b20003530 con 0x7f8b34159ea0 2024-04-25T10:14:11.107 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827612812 0 0) 0x7f8b20003530 con 0x7f8b34159ea0 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 >> v2:172.21.15.82:3300/0 conn(0x7f8b34071fd0 msgr2=0x7f8b3415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 --2- 172.21.15.82:0/1593889119 >> v2:172.21.15.82:3300/0 conn(0x7f8b34071fd0 0x7f8b3415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 >> v1:172.21.15.196:6789/0 conn(0x7f8b3415a280 legacy=0x7f8b340718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b340b2620 con 0x7f8b34159ea0 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f8b30001680 con 0x7f8b34159ea0 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 >> v1:172.21.15.82:6789/0 conn(0x7f8b34159ea0 legacy=0x7f8b340b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b20003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b20004810 0x7f8b20004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b200051d0 0x7f8b2000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b340b2990 con 0x7f8b20003a30 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c100 con 0x7f8b200051d0 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b2000c300 con 0x7f8b20004810 2024-04-25T10:14:11.108 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b20003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b3895f640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b20004810 0x7f8b20004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 -- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b20004810 msgr2=0x7f8b20004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b20004810 0x7f8b20004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 -- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b200051d0 msgr2=0x7f8b2000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39961640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b200051d0 0x7f8b2000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b200051d0 0x7f8b2000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39961640 1 -- 172.21.15.82:0/1593889119 reap_dead start 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.106+0000 7f8b39160640 1 -- 172.21.15.82:0/1593889119 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b20003530 con 0x7f8b20003a30 2024-04-25T10:14:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b20003e50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c1c0 tx=0x7f8b3001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c25120a7af4f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007d00 con 0x7f8b20003a30 2024-04-25T10:14:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b300a9020 con 0x7f8b20003a30 2024-04-25T10:14:11.110 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b2bfff640 1 -- 172.21.15.82:0/1593889119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30005ae0 con 0x7f8b20003a30 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 msgr2=0x7f8b20003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b20003e50 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000c1c0 tx=0x7f8b3001cc20 comp rx=0 tx=0).stop 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/1593889119 shutdown_connections 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/1593889119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b20003e50 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/1593889119 >> 172.21.15.82:0/1593889119 conn(0x7f8b340b11a0 msgr2=0x7f8b340b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/1593889119 shutdown_connections 2024-04-25T10:14:11.111 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/1593889119 wait complete. 2024-04-25T10:14:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 Processor -- start 2024-04-25T10:14:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- start start 2024-04-25T10:14:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.113 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34146050 0x7f8b341464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b34153c60 0x7f8b341540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406d420 con 0x7f8b20003a30 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406d2a0 con 0x7f8b34146050 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3b3c4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406d5a0 con 0x7f8b34153c60 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41878/0 (socket says 172.21.15.82:41878) 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 -- 172.21.15.82:0/34178073 learned_addr learned my addr 172.21.15.82:0/34178073 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39961640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b34153c60 0x7f8b341540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b3895f640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34146050 0x7f8b341464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 -- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b34153c60 msgr2=0x7f8b341540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b34153c60 0x7f8b341540e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.114 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 -- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34146050 msgr2=0x7f8b341464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34146050 0x7f8b341464d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 -- 172.21.15.82:0/34178073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34147370 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.110+0000 7f8b39160640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8b3001ca50 tx=0x7f8b20005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b300b4070 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b30000ee0 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b34146ad0 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b30007d00 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b34147090 con 0x7f8b20003a30 2024-04-25T10:14:11.115 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3404efa0 con 0x7f8b20003a30 2024-04-25T10:14:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8b300bf060 con 0x7f8b20003a30 2024-04-25T10:14:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 0x7f8b18069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b3895f640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 0x7f8b18069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8b3011dcf0 con 0x7f8b20003a30 2024-04-25T10:14:11.116 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.114+0000 7f8b3895f640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 0x7f8b18069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8b24007fd0 tx=0x7f8b24009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:11.119 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.118+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8b30122070 con 0x7f8b20003a30 2024-04-25T10:14:11.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.306+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b340b22e0 con 0x7f8b20003a30 2024-04-25T10:14:11.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.306+0000 7f8b29ffb640 1 -- 172.21.15.82:0/34178073 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f8b30016970 con 0x7f8b20003a30 2024-04-25T10:14:11.308 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:11.308 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:10.216909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:11.309 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 103 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 msgr2=0x7f8b18069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 0x7f8b18069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8b24007fd0 tx=0x7f8b24009040 comp rx=0 tx=0).stop 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 msgr2=0x7f8b340cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8b3001ca50 tx=0x7f8b20005840 comp rx=0 tx=0).stop 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 shutdown_connections 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8b18066f90 0x7f8b18069450 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8b34153c60 0x7f8b341540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8b34146050 0x7f8b341464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 --2- 172.21.15.82:0/34178073 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8b20003a30 0x7f8b340cd920 secure :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8b3001ca50 tx=0x7f8b20005840 comp rx=0 tx=0).stop 2024-04-25T10:14:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 >> 172.21.15.82:0/34178073 conn(0x7f8b340b11a0 msgr2=0x7f8b3415b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 shutdown_connections 2024-04-25T10:14:11.312 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:11.310+0000 7f8b3b3c4640 1 -- 172.21.15.82:0/34178073 wait complete. 2024-04-25T10:14:11.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:14:10.216909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10068}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10068': {'gid': 10068, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.82:6837/2659325772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 2659325772}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2659325772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:11.730 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_10068:up:rejoin 2024-04-25T10:14:12.732 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:12.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.858+0000 7f78d68d0640 1 Processor -- start 2024-04-25T10:14:12.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.858+0000 7f78d68d0640 1 -- start start 2024-04-25T10:14:12.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d68d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78d006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d68d0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00b2090 con 0x7f78d00b3f50 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d68d0640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00b43e0 con 0x7f78d015a700 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d68d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b46c0 con 0x7f78d015ecc0 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78cf7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f78d00b3f50 0x7f78d00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59232/0 (socket says 172.21.15.82:59232) 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78cf7fe640 1 -- 172.21.15.82:0/3866048638 learned_addr learned my addr 172.21.15.82:0/3866048638 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3496386224 0 0) 0x7f78d00b2090 con 0x7f78d00b3f50 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78d006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b8003530 con 0x7f78d00b3f50 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055740392 0 0) 0x7f78b8003530 con 0x7f78d00b3f50 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 msgr2=0x7f78d006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78d006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 >> v1:172.21.15.196:6789/0 conn(0x7f78d015a700 legacy=0x7f78d00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00cd6f0 con 0x7f78d00b3f50 2024-04-25T10:14:12.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f78c40016a0 con 0x7f78d00b3f50 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 >> v1:172.21.15.82:6789/0 conn(0x7f78d00b3f50 legacy=0x7f78d00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78b8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8004090 0x7f78b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8004e70 0x7f78b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00b46c0 con 0x7f78b8004e70 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b8005690 con 0x7f78b8004090 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b800bf40 con 0x7f78d015ecc0 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8004e70 0x7f78b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78cf7fe640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8004090 0x7f78b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78cffff640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78b8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 msgr2=0x7f78b8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d015ecc0 0x7f78b8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8004090 msgr2=0x7f78b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:12.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78b8004090 0x7f78b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.862+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3866048638 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b8003530 con 0x7f78b8004e70 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8004e70 0x7f78b800b9d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007360 tx=0x7f78c801ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31b02fba055ab008 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80b2070 con 0x7f78b8004e70 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80a5c50 con 0x7f78b8004e70 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78ceffd640 1 -- 172.21.15.82:0/3866048638 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8009be0 con 0x7f78b8004e70 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- 172.21.15.82:0/3866048638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8004e70 msgr2=0x7f78b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3866048638 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78b8004e70 0x7f78b800b9d0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7f78c8007360 tx=0x7f78c801ce10 comp rx=0 tx=0).stop 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3866048638 reap_dead start 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- 172.21.15.82:0/3866048638 shutdown_connections 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- 172.21.15.82:0/3866048638 >> 172.21.15.82:0/3866048638 conn(0x7f78d00b11a0 msgr2=0x7f78d00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- 172.21.15.82:0/3866048638 shutdown_connections 2024-04-25T10:14:12.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- 172.21.15.82:0/3866048638 wait complete. 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 Processor -- start 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.866+0000 7f78d68d0640 1 -- start start 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 0x7f78d013f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d013fb90 0x7f78d0157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015f220 con 0x7f78d01582b0 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015f0a0 con 0x7f78d01638e0 2024-04-25T10:14:12.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00cd310 con 0x7f78d013fb90 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41898/0 (socket says 172.21.15.82:41898) 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3062313480 learned_addr learned my addr 172.21.15.82:0/3062313480 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78cffff640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 0x7f78d013f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78cf7fe640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d013fb90 0x7f78d0157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d013fb90 msgr2=0x7f78d0157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d013fb90 0x7f78d0157d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 msgr2=0x7f78d013f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 0x7f78d013f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:12.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 -- 172.21.15.82:0/3062313480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0156cd0 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78cffff640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 0x7f78d013f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d4e6d640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f78c8002240 tx=0x7f78b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80bb070 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c80a5a50 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d0156fb0 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c80b2040 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d0157570 con 0x7f78d01582b0 2024-04-25T10:14:12.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d004efa0 con 0x7f78d01582b0 2024-04-25T10:14:12.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78c80be070 con 0x7f78d01582b0 2024-04-25T10:14:12.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 0x7f78bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:12.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f78c811ea20 con 0x7f78d01582b0 2024-04-25T10:14:12.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78cffff640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 0x7f78bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:12.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.870+0000 7f78cffff640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 0x7f78bc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f78c00016a0 tx=0x7f78c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:12.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:12.874+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78c8016b50 con 0x7f78d01582b0 2024-04-25T10:14:13.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.066+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78d015a1b0 con 0x7f78d01582b0 2024-04-25T10:14:13.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.066+0000 7f78ccff9640 1 -- 172.21.15.82:0/3062313480 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f78c80ef110 con 0x7f78d01582b0 2024-04-25T10:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:13.068 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:11.226477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:13.068 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 104 2024-04-25T10:14:13.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 msgr2=0x7f78bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 0x7f78bc069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f78c00016a0 tx=0x7f78c0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 msgr2=0x7f78d0158870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f78c8002240 tx=0x7f78b8005840 comp rx=0 tx=0).stop 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 shutdown_connections 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f78bc066f60 0x7f78bc069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f78d013fb90 0x7f78d0157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f78d01582b0 0x7f78d0158870 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 --2- 172.21.15.82:0/3062313480 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f78d01638e0 0x7f78d013f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 >> 172.21.15.82:0/3062313480 conn(0x7f78d00b11a0 msgr2=0x7f78d0151d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 shutdown_connections 2024-04-25T10:14:13.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.070+0000 7f78d68d0640 1 -- 172.21.15.82:0/3062313480 wait complete. 2024-04-25T10:14:13.198 INFO:tasks.ceph.mds.c.smithi082.stderr:2024-04-25T10:14:13.198+0000 7fa0a5943640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:13.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:21.485141+0000', 'modified': '2024-04-25T10:14:11.226477+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 123, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10068}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10068': {'gid': 10068, 'name': 'c', 'rank': 0, 'incarnation': 101, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.82:6837/2659325772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.82:6835', 'nonce': 2659325772}, {'type': 'v1', 'addr': '172.21.15.82:6837', 'nonce': 2659325772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:13.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:13.490 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:14:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 Processor -- start 2024-04-25T10:14:13.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 -- start start 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d9c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f4d9c0d1cd0 0x7f4d9c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c06be60 con 0x7f4d9c15da80 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d9c06c440 con 0x7f4d9c0d1cd0 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4da1410640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c15de60 con 0x7f4d9c0d07f0 2024-04-25T10:14:13.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d9affd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f4d9c15da80 0x7f4d9c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59242/0 (socket says 172.21.15.82:59242) 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d9affd640 1 -- 172.21.15.82:0/3751188743 learned_addr learned my addr 172.21.15.82:0/3751188743 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d9c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/3751188743 >> v2:172.21.15.196:3300/0 conn(0x7f4d9c0d1cd0 0x7f4d9c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761791215 0 0) 0x7f4d9c06be60 con 0x7f4d9c15da80 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d84003530 con 0x7f4d9c15da80 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050955832 0 0) 0x7f4d84003530 con 0x7f4d9c15da80 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 msgr2=0x7f4d9c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d9c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 >> v2:172.21.15.196:3300/0 conn(0x7f4d9c0d1cd0 msgr2=0x7f4d9c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 --2- 172.21.15.82:0/3751188743 >> v2:172.21.15.196:3300/0 conn(0x7f4d9c0d1cd0 0x7f4d9c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c0d2450 con 0x7f4d9c15da80 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4d90001680 con 0x7f4d9c15da80 2024-04-25T10:14:13.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 >> v1:172.21.15.82:6789/0 conn(0x7f4d9c15da80 legacy=0x7f4d9c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d84003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d84004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d84004d50 0x7f4d840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06c440 con 0x7f4d84003a30 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c15de60 con 0x7f4d84004d50 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.622+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8400b9d0 con 0x7f4d9c0d07f0 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d84003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d84004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d84004d50 0x7f4d840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 -- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 msgr2=0x7f4d84004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d84004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 -- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d84004d50 msgr2=0x7f4d840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d84004d50 0x7f4d840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 -- 172.21.15.82:0/3751188743 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d84003530 con 0x7f4d84003a30 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c0d07f0 0x7f4d84004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d84004d50 0x7f4d840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9affd640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d84003e50 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c2c0 tx=0x7f4d9001cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843548627b73a721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90007d60 con 0x7f4d84003a30 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900a9020 con 0x7f4d84003a30 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d99ffb640 1 -- 172.21.15.82:0/3751188743 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90005ae0 con 0x7f4d84003a30 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 -- 172.21.15.82:0/3751188743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 msgr2=0x7f4d84003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 --2- 172.21.15.82:0/3751188743 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d84003e50 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7f4d9000c2c0 tx=0x7f4d9001cd10 comp rx=0 tx=0).stop 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/3751188743 reap_dead start 2024-04-25T10:14:13.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 -- 172.21.15.82:0/3751188743 shutdown_connections 2024-04-25T10:14:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 -- 172.21.15.82:0/3751188743 >> 172.21.15.82:0/3751188743 conn(0x7f4d9c08b000 msgr2=0x7f4d9c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 -- 172.21.15.82:0/3751188743 shutdown_connections 2024-04-25T10:14:13.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.626+0000 7f4da1410640 1 -- 172.21.15.82:0/3751188743 wait complete. 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 Processor -- start 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- start start 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c1572c0 0x7f4d9c162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d9c1f6710 0x7f4d9c0cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06d7a0 con 0x7f4d84003a30 2024-04-25T10:14:13.632 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06d620 con 0x7f4d9c1f6710 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06d920 con 0x7f4d9c1572c0 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41910/0 (socket says 172.21.15.82:41910) 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 -- 172.21.15.82:0/933762101 learned_addr learned my addr 172.21.15.82:0/933762101 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d9c1f6710 0x7f4d9c0cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c1572c0 0x7f4d9c162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 -- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c1572c0 msgr2=0x7f4d9c162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c1572c0 0x7f4d9c162050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 -- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d9c1f6710 msgr2=0x7f4d9c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d9c1f6710 0x7f4d9c0cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 -- 172.21.15.82:0/933762101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c0cb9b0 con 0x7f4d84003a30 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d9affd640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007040 tx=0x7f4d84005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d837fe640 1 -- 172.21.15.82:0/933762101 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d90001c90 con 0x7f4d84003a30 2024-04-25T10:14:13.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d837fe640 1 -- 172.21.15.82:0/933762101 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900b3070 con 0x7f4d84003a30 2024-04-25T10:14:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c157880 con 0x7f4d84003a30 2024-04-25T10:14:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4d837fe640 1 -- 172.21.15.82:0/933762101 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900a5cb0 con 0x7f4d84003a30 2024-04-25T10:14:13.634 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.630+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c0cbf30 con 0x7f4d84003a30 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d837fe640 1 -- 172.21.15.82:0/933762101 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d900047f0 con 0x7f4d84003a30 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d837fe640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 0x7f4d88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d837fe640 1 -- 172.21.15.82:0/933762101 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4d9011eb10 con 0x7f4d84003a30 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 0x7f4d88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 0x7f4d88069450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c160460 tx=0x7f4d8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 Processor -- start 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- start start 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbd920 0x559b0ffbdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ff74310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fe5b5f0 con 0x559b0ffbe240 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fdc2da0 con 0x559b0ff73f30 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b0fd4b150 con 0x559b0ffbd920 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41924/0 (socket says 172.21.15.82:41924) 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/3700537610 learned_addr learned my addr 172.21.15.82:0/3700537610 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9affd640 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbd920 0x559b0ffbdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbd920 msgr2=0x559b0ffbdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbd920 0x559b0ffbdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 msgr2=0x559b0ff74310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ff74310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ff74310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.638 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/3700537610 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0fd64ca0 con 0x559b0ffbe240 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006e30 tx=0x7f4d8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca88d1a80ed8f2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d80ff9640 1 -- 172.21.15.82:0/3700537610 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c011d70 con 0x559b0ffbe240 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d80ff9640 1 -- 172.21.15.82:0/3700537610 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c026070 con 0x559b0ffbe240 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4d80ff9640 1 -- 172.21.15.82:0/3700537610 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c0067f0 con 0x559b0ffbe240 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- 172.21.15.82:0/3700537610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 msgr2=0x559b0ff739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c006e30 tx=0x7f4d8c014000 comp rx=0 tx=0).stop 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 -- 172.21.15.82:0/3700537610 shutdown_connections 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.634+0000 7f4da457d000 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ff74310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbe240 0x559b0ff739f0 unknown :-1 s=CLOSED pgs=2211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- 172.21.15.82:0/3700537610 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbd920 0x559b0ffbdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/3700537610 >> 172.21.15.82:0/3700537610 conn(0x559b0ff7a2a0 msgr2=0x559b0ff722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/3700537610 shutdown_connections 2024-04-25T10:14:13.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/3700537610 wait complete. 2024-04-25T10:14:13.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- start start 2024-04-25T10:14:13.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 Processor -- start 2024-04-25T10:14:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffc4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ff77aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fd7d990 con 0x559b0ffbd920 2024-04-25T10:14:13.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fdbee40 con 0x559b0ff73f30 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b0fd4e900 con 0x559b0ffbe240 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 10 client.0 _ll_register_callbacks cb 0x559b0fe761c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41930/0 (socket says 172.21.15.82:41930) 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/1213878626 learned_addr learned my addr 172.21.15.82:0/1213878626 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ff77aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9affd640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffc4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 msgr2=0x559b0ff77aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ff77aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 msgr2=0x559b0ffc4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffc4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 -- 172.21.15.82:0/1213878626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b0fd7b630 con 0x559b0ffbd920 2024-04-25T10:14:13.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c023040 tx=0x7f4d8c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd7e795efee8bed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c021920 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d8c026070 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4d8c036d30 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d8c03d050 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 10 client.10170 fetch_fsmap learned FSMap version 104 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x559b0fd4b3a0 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c026070 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d8c040020 con 0x559b0ffbd920 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 10 client.10170 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 10 client.10170 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 20 client.10170 populate_metadata read hostname 'smithi082' 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ff784b0 0x559b0ff78890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 4 client.10170 mds_command: new command op to 10068 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 --> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b0fd5ea50 con 0x559b0ff784b0 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9affd640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ff784b0 0x559b0ff78890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d79ffb640 10 client.10170 ms_handle_connect on v2:172.21.15.82:6835/2659325772 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.638+0000 7f4d9affd640 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ff784b0 0x559b0ff78890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44bcbd1f735716b6 server_cookie=cbf52296659264b4 in_seq=0 out_seq=0 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4d79ffb640 1 -- 172.21.15.82:0/1213878626 <== mds.0 v2:172.21.15.82:6835/2659325772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559b0fd5ea50 con 0x559b0ff784b0 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4d79ffb640 10 client.10170 handle_command_reply: tid=0 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 client.10170 shutdown 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 msgr2=0x559b0ffc5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7f4d8c023040 tx=0x7f4d8c004740 comp rx=0 tx=0).stop 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 shutdown_connections 2024-04-25T10:14:13.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ff77aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ff784b0 0x559b0ff78890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b0ffbd920 0x559b0ffc5230 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 --2- 172.21.15.82:0/1213878626 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffc4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 >> 172.21.15.82:0/1213878626 conn(0x559b0ff7a2a0 msgr2=0x559b0ff729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4d79ffb640 0 client.10170 ms_handle_reset on v2:172.21.15.82:6835/2659325772 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 shutdown_connections 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 1 -- 172.21.15.82:0/1213878626 wait complete. 2024-04-25T10:14:13.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.642+0000 7f4da457d000 20 client.10170 trim_cache size 0 max 16384 2024-04-25T10:14:13.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 Processor -- start 2024-04-25T10:14:13.657 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- start start 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b10019040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b10019580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c202d80 con 0x559b10019ac0 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c202180 con 0x559b0ff73f30 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c202780 con 0x559b0ffbe240 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41942/0 (socket says 172.21.15.82:41942) 2024-04-25T10:14:13.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/1607050099 learned_addr learned my addr 172.21.15.82:0/1607050099 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9affd640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b10019040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b10019580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 msgr2=0x559b10019580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b10019580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 msgr2=0x559b10019040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b10019040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/1607050099 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b0fe44600 con 0x559b10019ac0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9affd640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b10019040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c06bc80 tx=0x7f4d94009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd9bef8a1ae1515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d7bfff640 1 -- 172.21.15.82:0/1607050099 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9401b020 con 0x559b10019ac0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d7bfff640 1 -- 172.21.15.82:0/1607050099 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94004780 con 0x559b10019ac0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4d7bfff640 1 -- 172.21.15.82:0/1607050099 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94017690 con 0x559b10019ac0 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- 172.21.15.82:0/1607050099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 msgr2=0x559b0ffc6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c06bc80 tx=0x7f4d94009750 comp rx=0 tx=0).stop 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- 172.21.15.82:0/1607050099 shutdown_connections 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b10019580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffc6920 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 --2- 172.21.15.82:0/1607050099 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b10019040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- 172.21.15.82:0/1607050099 >> 172.21.15.82:0/1607050099 conn(0x559b0ff7a2a0 msgr2=0x559b0ff77220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- 172.21.15.82:0/1607050099 shutdown_connections 2024-04-25T10:14:13.661 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- 172.21.15.82:0/1607050099 wait complete. 2024-04-25T10:14:13.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 -- start start 2024-04-25T10:14:13.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.658+0000 7f4da457d000 1 Processor -- start 2024-04-25T10:14:13.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.665 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ffd9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fdc73d0 con 0x559b10019ac0 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b0fe7ed50 con 0x559b0ff73f30 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x559b0fd5d980 con 0x559b0ffbe240 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 10 client.0 _ll_register_callbacks cb 0x7f4d8c009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41952/0 (socket says 172.21.15.82:41952) 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/2068771644 learned_addr learned my addr 172.21.15.82:0/2068771644 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9affd640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9a7fc640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ffd9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 msgr2=0x559b0ffd9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ffd9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.666 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 msgr2=0x559b0ffca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 -- 172.21.15.82:0/2068771644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559b0fd579b0 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9affd640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9b7fe640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4d9401a040 tx=0x7f4d94004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ef475c72b14c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.82:3300/0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94017cf0 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d94004780 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mon.0 v2:172.21.15.82:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4d9402c480 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d94034050 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 10 client.10176 fetch_fsmap learned FSMap version 104 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x559b0fd6a0b0 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mon.0 v2:172.21.15.82:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94004780 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mon.0 v2:172.21.15.82:3300/0 5 ==== fsmap(e 104) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d94037020 con 0x559b10019ac0 2024-04-25T10:14:13.667 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 10 client.10176 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 10 client.10176 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 20 client.10176 populate_metadata read hostname 'smithi082' 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ffca920 0x559b0ffd9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 4 client.10176 mds_command: new command op to 10068 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 --> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559b0fdd02e0 con 0x559b0ffca920 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9affd640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ffca920 0x559b0ffd9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 10 client.10176 ms_handle_connect on v2:172.21.15.82:6835/2659325772 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d9affd640 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ffca920 0x559b0ffd9e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511374c66462aea4 server_cookie=3d4fd5a61368b7d2 in_seq=0 out_seq=0 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 1 -- 172.21.15.82:0/2068771644 <== mds.0 v2:172.21.15.82:6835/2659325772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x559b0fdd02e0 con 0x559b0ffca920 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 10 client.10176 handle_command_reply: tid=0 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 client.10176 shutdown 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 msgr2=0x559b0ffd9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.668 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4d9401a040 tx=0x7f4d94004a50 comp rx=0 tx=0).stop 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 shutdown_connections 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x559b0ffbe240 0x559b0ffd9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:6835/2659325772,v1:172.21.15.82:6837/2659325772] conn(0x559b0ffca920 0x559b0ffd9e90 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x559b10019ac0 0x559b0ffd9950 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 --2- 172.21.15.82:0/2068771644 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x559b0ff73f30 0x559b0ffca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4d79ffb640 0 client.10176 ms_handle_reset on v2:172.21.15.82:6835/2659325772 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 >> 172.21.15.82:0/2068771644 conn(0x559b0ff7a2a0 msgr2=0x559b0ffc58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 shutdown_connections 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 1 -- 172.21.15.82:0/2068771644 wait complete. 2024-04-25T10:14:13.669 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da457d000 20 client.10176 trim_cache size 0 max 16384 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "cluster_fsid": "923528c9-0d4c-4379-be0f-2d1b434d3c6c", 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "whoami": 0, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 10068, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "want_state": "up:active", 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "state": "up:active", 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs", 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "rank_uptime": 4.4641212870000002, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "mdsmap_epoch": 104, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch": 123, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "osdmap_epoch_barrier": 123, 2024-04-25T10:14:13.670 INFO:teuthology.orchestra.run.smithi082.stdout: "uptime": 6.4442593229999998 2024-04-25T10:14:13.671 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:13.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 msgr2=0x7f4d88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 0x7f4d88069450 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c160460 tx=0x7f4d8c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.671 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 msgr2=0x7f4d9c152d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7f4d90007040 tx=0x7f4d84005840 comp rx=0 tx=0).stop 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 shutdown_connections 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4d88066f90 0x7f4d88069450 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4d9c1572c0 0x7f4d9c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4d9c1f6710 0x7f4d9c0cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 --2- 172.21.15.82:0/933762101 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f4d84003a30 0x7f4d9c152d90 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 >> 172.21.15.82:0/933762101 conn(0x7f4d9c08b000 msgr2=0x7f4d9c0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 shutdown_connections 2024-04-25T10:14:13.672 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.662+0000 7f4da1410640 1 -- 172.21.15.82:0/933762101 wait complete. 2024-04-25T10:14:13.682 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:13.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.794+0000 7fc8f06f3640 1 Processor -- start 2024-04-25T10:14:13.796 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.794+0000 7fc8f06f3640 1 -- start start 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8f06f3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8e8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8f06f3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e80ccc70 con 0x7fc8e80ce510 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8f06f3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e806dc40 con 0x7fc8e8144ca0 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8f06f3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cc2b0 con 0x7fc8e81451f0 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc8e80ce510 0x7fc8e80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:59254/0 (socket says 172.21.15.82:59254) 2024-04-25T10:14:13.799 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 -- 172.21.15.82:0/3368395501 learned_addr learned my addr 172.21.15.82:0/3368395501 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2423421416 0 0) 0x7fc8e80ccc70 con 0x7fc8e80ce510 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8d4003530 con 0x7fc8e80ce510 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8eec90640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8e8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993055443 0 0) 0x7fc8d4003530 con 0x7fc8e80ce510 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 msgr2=0x7fc8e8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8e8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 >> v1:172.21.15.196:6789/0 conn(0x7fc8e8144ca0 legacy=0x7fc8e806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e806df20 con 0x7fc8e80ce510 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc8e4001680 con 0x7fc8e80ce510 2024-04-25T10:14:13.800 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 >> v1:172.21.15.82:6789/0 conn(0x7fc8e80ce510 legacy=0x7fc8e80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8d4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d4004da0 0x7fc8d4005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cc2b0 con 0x7fc8d4003a60 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d400bc50 con 0x7fc8d4004da0 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d400be50 con 0x7fc8e81451f0 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8d4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8edc8e640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8eec90640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d4004da0 0x7fc8d4005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 -- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 msgr2=0x7fc8d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81451f0 0x7fc8d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 -- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d4004da0 msgr2=0x7fc8d4005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d4004da0 0x7fc8d4005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8ee48f640 1 -- 172.21.15.82:0/3368395501 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4003530 con 0x7fc8d4003a60 2024-04-25T10:14:13.801 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.798+0000 7fc8eec90640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8d4004da0 0x7fc8d4005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8d4003e80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c210 tx=0x7fc8e401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc9e22e011c4dd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007d40 con 0x7fc8d4003a60 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e4005b00 con 0x7fc8d4003a60 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ed48d640 1 -- 172.21.15.82:0/3368395501 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007070 con 0x7fc8d4003a60 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/3368395501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 msgr2=0x7fc8d4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/3368395501 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8d4003e80 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400c210 tx=0x7fc8e401f3f0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8eec90640 1 -- 172.21.15.82:0/3368395501 reap_dead start 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/3368395501 shutdown_connections 2024-04-25T10:14:13.805 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/3368395501 >> 172.21.15.82:0/3368395501 conn(0x7fc8e80b11a0 msgr2=0x7fc8e80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/3368395501 shutdown_connections 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/3368395501 wait complete. 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 Processor -- start 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- start start 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81f4b00 0x7fc8e81f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e81f4f80 0x7fc8e81f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e80cc900 con 0x7fc8d4003a60 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e806e290 con 0x7fc8e81f4f80 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e8142680 con 0x7fc8e81f4b00 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:41974/0 (socket says 172.21.15.82:41974) 2024-04-25T10:14:13.806 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 -- 172.21.15.82:0/2700143176 learned_addr learned my addr 172.21.15.82:0/2700143176 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8eec90640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e81f4f80 0x7fc8e81f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8edc8e640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81f4b00 0x7fc8e81f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 -- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81f4b00 msgr2=0x7fc8e81f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81f4b00 0x7fc8e81f7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 -- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e81f4f80 msgr2=0x7fc8e81f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e81f4f80 0x7fc8e81f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 -- 172.21.15.82:0/2700143176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e80a9dd0 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8ee48f640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e401f220 tx=0x7fc8d4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40047f0 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e4004990 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e80b26b0 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.802+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4004d60 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e80b2c70 con 0x7fc8d4003a60 2024-04-25T10:14:13.807 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e804efa0 con 0x7fc8d4003a60 2024-04-25T10:14:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc8e4007070 con 0x7fc8d4003a60 2024-04-25T10:14:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8deffd640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 0x7fc8cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:13.808 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc8e41229b0 con 0x7fc8d4003a60 2024-04-25T10:14:13.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8edc8e640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 0x7fc8cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:13.809 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.806+0000 7fc8edc8e640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 0x7fc8cc069450 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80b4780 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:13.810 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.810+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc8e4019100 con 0x7fc8d4003a60 2024-04-25T10:14:13.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc8e8073150 con 0x7fc8d4003a60 2024-04-25T10:14:13.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8deffd640 1 -- 172.21.15.82:0/2700143176 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v104) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc8e40f3650 con 0x7fc8d4003a60 2024-04-25T10:14:13.993 INFO:teuthology.orchestra.run.smithi082.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:13.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 msgr2=0x7fc8cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 0x7fc8cc069450 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80b4780 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 msgr2=0x7fc8e815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7fc8e401f220 tx=0x7fc8d4005840 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 shutdown_connections 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc8cc066f90 0x7fc8cc069450 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc8e81f4b00 0x7fc8e81f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc8e81f4f80 0x7fc8e81f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 --2- 172.21.15.82:0/2700143176 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc8d4003a60 0x7fc8e815b230 unknown :-1 s=CLOSED pgs=2217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 >> 172.21.15.82:0/2700143176 conn(0x7fc8e80b11a0 msgr2=0x7fc8e813ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 shutdown_connections 2024-04-25T10:14:13.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:13.994+0000 7fc8f06f3640 1 -- 172.21.15.82:0/2700143176 wait complete. 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi082.front.sepia.ceph.com 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:14:14.011 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:14:14.012 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:14.012 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:14.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.012 DEBUG:teuthology.orchestra.run.smithi082:> ip addr 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.018 INFO:teuthology.orchestra.run.smithi082.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c6 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f0 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:d8:da:c7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: altname enp7s0f1 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:fe brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0c:c4:7a:8f:d3:ff brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.21.15.82/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft 41816sec preferred_lft 41816sec 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::ec4:7aff:fe8f:d3ff/64 scope link 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.019 INFO:teuthology.orchestra.run.smithi082.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 02:42:93:ba:70:a7 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi082.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run.smithi082.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run.smithi082.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run.smithi082.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:14.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.021 DEBUG:teuthology.orchestra.run.smithi082:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:14.069 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:14:14.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.071 DEBUG:teuthology.orchestra.run.smithi082:> route 2024-04-25T10:14:14.125 INFO:teuthology.orchestra.run.smithi082.stdout:Kernel IP routing table 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T10:14:14.126 INFO:teuthology.orchestra.run.smithi082.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:14.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> (cd / && exec bash -c ' 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> set -e 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:14:14.127 DEBUG:teuthology.orchestra.run.smithi082:> ') 2024-04-25T10:14:14.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.199 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:14:14.245 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:14.245 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:14.246 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:14.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.246 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link set brx.0 up 2024-04-25T10:14:14.301 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T10:14:14.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:14.301 DEBUG:teuthology.orchestra.run.smithi082:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi082.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:14.350 DEBUG:teuthology.orchestra.run.smithi082:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:14.397 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:14:14.454 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:14:14.508 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:14.509 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:14.510 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:14.510 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:14.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.511 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:14:14.557 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:14:14.557 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T10:14:14.602 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:14:14.617 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:14.665 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:14:14.665 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:14:14.665 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:14:14.665 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:14:14.666 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:14.667 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:14.668 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:14.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.669 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:14:14.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:2024-04-25T10:14:14.686+0000 7fd26fab5580 -1 init, newargv = 0x55fa62b308d0 newargc=15 2024-04-25T10:14:14.685 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[56611]: starting ceph client 2024-04-25T10:14:14.691 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[56611]: starting fuse 2024-04-25T10:14:15.790 DEBUG:teuthology.orchestra.run.smithi082:> sudo modprobe fuse 2024-04-25T10:14:15.808 DEBUG:teuthology.orchestra.run.smithi082:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/sys 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/proc 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/dev 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/pts 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/run 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/ 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/security 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/shm 2024-04-25T10:14:15.861 INFO:teuthology.orchestra.run.smithi082.stdout:/run/lock 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/cgroup 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/pstore 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/bpf 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/hugepages 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/dev/mqueue 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/debug 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/tracing 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/proc/fs/nfsd 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/fs/fuse/connections 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/sys/kernel/config 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/run/rpc_pipefs 2024-04-25T10:14:15.862 INFO:teuthology.orchestra.run.smithi082.stdout:/run/user/1000 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:15.863 INFO:teuthology.orchestra.run.smithi082.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:15.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:15.864 DEBUG:teuthology.orchestra.run.smithi082:> ls /sys/fs/fuse/connections 2024-04-25T10:14:15.914 INFO:teuthology.orchestra.run.smithi082.stdout:45 2024-04-25T10:14:15.914 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> import glob 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> import re 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> import subprocess 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:15.914 DEBUG:teuthology.orchestra.run.smithi082:> def _find_admin_socket(client_name): 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> files = glob.glob(asok_path) 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> # Given a non-glob path, it better be there 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> if "*" not in asok_path: 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> assert(len(files) == 1) 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> return files[0] 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> for f in files: 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> contents = proc_f.read() 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> if mountpoint in contents: 2024-04-25T10:14:15.915 DEBUG:teuthology.orchestra.run.smithi082:> return f 2024-04-25T10:14:15.916 DEBUG:teuthology.orchestra.run.smithi082:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:14:15.916 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:15.916 DEBUG:teuthology.orchestra.run.smithi082:> print(_find_admin_socket("client.0")) 2024-04-25T10:14:15.916 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.038 INFO:teuthology.orchestra.run.smithi082.stdout:/var/run/ceph/ceph-client.0.56611.asok 2024-04-25T10:14:16.042 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.051 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56611.asok 2024-04-25T10:14:16.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.051 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56611.asok status 2024-04-25T10:14:16.208 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:16.208 INFO:teuthology.orchestra.run.smithi082.stdout: "metadata": { 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "entity_id": "0", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "hostname": "smithi082", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "pid": "56611", 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "root": "/" 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_count": 0, 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "dentry_pinned_count": 0, 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "id": 10191, 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "inst": { 2024-04-25T10:14:16.209 INFO:teuthology.orchestra.run.smithi082.stdout: "name": { 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "client", 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "num": 10191 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1979800425 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: } 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": { 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "type": "v1", 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: "nonce": 1979800425 2024-04-25T10:14:16.210 INFO:teuthology.orchestra.run.smithi082.stdout: }, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "inst_str": "client.10191 192.168.0.1:0/1979800425", 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "addr_str": "192.168.0.1:0/1979800425", 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "inode_count": 1, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "mds_epoch": 104, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch": 123, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "blocklisted": false, 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout: "fs_name": "cephfs" 2024-04-25T10:14:16.211 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:16.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.223 DEBUG:teuthology.orchestra.run.smithi082:> stat --file-system '--printf=%T 2024-04-25T10:14:16.224 DEBUG:teuthology.orchestra.run.smithi082:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:16.231 INFO:teuthology.orchestra.run.smithi082.stdout:fuseblk 2024-04-25T10:14:16.231 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:16.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.231 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:16.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.286 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./ 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./parent 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/charlie 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/alpha 2024-04-25T10:14:16.351 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/flushed/bravo 2024-04-25T10:14:16.352 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/unflushed 2024-04-25T10:14:16.352 INFO:teuthology.orchestra.run.smithi082.stdout:./parent/unflushed/jfile 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T10:14:16.353 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.463 INFO:teuthology.orchestra.run.smithi082.stdout:1 2024-04-25T10:14:16.466 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.473 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.474 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.474 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.474 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.474 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T10:14:16.474 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.575 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627776 2024-04-25T10:14:16.578 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T10:14:16.585 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.687 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627777 2024-04-25T10:14:16.690 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.697 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.698 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.698 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.698 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.698 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T10:14:16.698 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.801 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628280 2024-04-25T10:14:16.804 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.811 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.811 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.812 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.812 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.812 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T10:14:16.812 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:16.912 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627778 2024-04-25T10:14:16.915 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T10:14:16.922 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:17.024 INFO:teuthology.orchestra.run.smithi082.stdout:1099511628279 2024-04-25T10:14:17.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:17.034 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:17.034 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:17.034 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:17.035 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:17.035 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T10:14:17.035 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:17.141 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627779 2024-04-25T10:14:17.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> stdin-killer -- python3 -c ' 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> import os 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> import stat 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T10:14:17.151 DEBUG:teuthology.orchestra.run.smithi082:> ' 2024-04-25T10:14:17.251 INFO:teuthology.orchestra.run.smithi082.stdout:1099511627780 2024-04-25T10:14:17.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:14:17.261 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:17.261 DEBUG:teuthology.orchestra.run.smithi082:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:17.355 DEBUG:teuthology.orchestra.run.smithi196:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:17.417 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:17.418 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:14:17.418 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:17.424 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:17.596 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 Processor -- start 2024-04-25T10:14:17.597 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 -- start start 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2d4159ea0 0x7ff2d40b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff2d415a280 0x7ff2d415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2d415ae30 con 0x7ff2d40718c0 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d40b2990 con 0x7ff2d4159ea0 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d93bd640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2d415b110 con 0x7ff2d415a280 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d2ffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff2d4159ea0 0x7ff2d40b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d37fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff2d40718c0 0x7ff2d406d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60726/0 (socket says 172.21.15.82:60726) 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d37fe640 1 -- 172.21.15.82:0/902034881 learned_addr learned my addr 172.21.15.82:0/902034881 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d27fc640 1 --2- 172.21.15.82:0/902034881 >> v2:172.21.15.196:3300/0 conn(0x7ff2d415a280 0x7ff2d415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.599 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d1ffb640 1 -- 172.21.15.82:0/902034881 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141157135 0 0) 0x7ff2d415ae30 con 0x7ff2d40718c0 2024-04-25T10:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d1ffb640 1 -- 172.21.15.82:0/902034881 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2b8003450 con 0x7ff2d40718c0 2024-04-25T10:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/902034881 >> v2:172.21.15.196:3300/0 conn(0x7ff2d415a280 msgr2=0x7ff2d415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d2ffd640 1 --2- 172.21.15.82:0/902034881 >> v2:172.21.15.196:3300/0 conn(0x7ff2d415a280 0x7ff2d415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/902034881 >> v1:172.21.15.82:6789/0 conn(0x7ff2d40718c0 legacy=0x7ff2d406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.600 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.598+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/902034881 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d40b2620 con 0x7ff2d4159ea0 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d2ffd640 1 --2- 172.21.15.82:0/902034881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d4159ea0 0x7ff2d40b2e20 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40728e0 tx=0x7ff2c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82080a71b3ad6a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d1ffb640 1 -- 172.21.15.82:0/902034881 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b1070 con 0x7ff2d4159ea0 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d1ffb640 1 -- 172.21.15.82:0/902034881 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40ac5f0 con 0x7ff2d4159ea0 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d1ffb640 1 -- 172.21.15.82:0/902034881 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40ac9c0 con 0x7ff2d4159ea0 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- 172.21.15.82:0/902034881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d4159ea0 msgr2=0x7ff2d40b2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/902034881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d4159ea0 0x7ff2d40b2e20 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40728e0 tx=0x7ff2c401c480 comp rx=0 tx=0).stop 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- 172.21.15.82:0/902034881 shutdown_connections 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/902034881 >> v2:172.21.15.196:3300/0 conn(0x7ff2d415a280 0x7ff2d415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/902034881 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d4159ea0 0x7ff2d40b2e20 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.602 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- 172.21.15.82:0/902034881 >> 172.21.15.82:0/902034881 conn(0x7ff2d40b11a0 msgr2=0x7ff2d40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- 172.21.15.82:0/902034881 shutdown_connections 2024-04-25T10:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- 172.21.15.82:0/902034881 wait complete. 2024-04-25T10:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 Processor -- start 2024-04-25T10:14:17.603 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- start start 2024-04-25T10:14:17.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.605 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2d4159ea0 0x7ff2d414a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d415a280 0x7ff2d4151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4166b30 con 0x7ff2d40718c0 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d41669b0 con 0x7ff2d415a280 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d93bd640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2d4166cb0 con 0x7ff2d4159ea0 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d2ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46382/0 (socket says 172.21.15.82:46382) 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/2009934378 learned_addr learned my addr 172.21.15.82:0/2009934378 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.602+0000 7ff2d37fe640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d415a280 0x7ff2d4151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d27fc640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2d4159ea0 0x7ff2d414a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2d4159ea0 msgr2=0x7ff2d414a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2d4159ea0 0x7ff2d414a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d415a280 msgr2=0x7ff2d4151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.606 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d415a280 0x7ff2d4151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 -- 172.21.15.82:0/2009934378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d4155c20 con 0x7ff2d40718c0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d2ffd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af040 tx=0x7ff2c4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b1040 con 0x7ff2d40718c0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c40bb070 con 0x7ff2d40718c0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c40b5410 con 0x7ff2d40718c0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2d4153710 con 0x7ff2d40718c0 2024-04-25T10:14:17.607 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2d414a6a0 con 0x7ff2d40718c0 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2d404efa0 con 0x7ff2d40718c0 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2c40b55b0 con 0x7ff2d40718c0 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 0x7ff2bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff2c411ede0 con 0x7ff2d40718c0 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d27fc640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 0x7ff2bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.608 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.606+0000 7ff2d27fc640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 0x7ff2bc069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40cee90 tx=0x7ff2c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.610+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2c40a9070 con 0x7ff2d40718c0 2024-04-25T10:14:17.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.774+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2d406e6b0 con 0x7ff2d40718c0 2024-04-25T10:14:17.776 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.774+0000 7ff2cb7fe640 1 -- 172.21.15.82:0/2009934378 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17221 (secure 0 0 0) 0x7ff2c4016070 con 0x7ff2d40718c0 2024-04-25T10:14:17.776 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:17.776 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":123,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:14:07.178089+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 msgr2=0x7ff2bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 0x7ff2bc069420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff2d40cee90 tx=0x7ff2c0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 msgr2=0x7ff2d41541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7ff2c40af040 tx=0x7ff2c4019220 comp rx=0 tx=0).stop 2024-04-25T10:14:17.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 shutdown_connections 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff2bc066f60 0x7ff2bc069420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff2d4159ea0 0x7ff2d414a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff2d415a280 0x7ff2d4151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 --2- 172.21.15.82:0/2009934378 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff2d40718c0 0x7ff2d41541f0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 >> 172.21.15.82:0/2009934378 conn(0x7ff2d40b11a0 msgr2=0x7ff2d4152de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 shutdown_connections 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.778+0000 7ff2d93bd640 1 -- 172.21.15.82:0/2009934378 wait complete. 2024-04-25T10:14:17.796 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:17.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:17.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.922+0000 7f30a46c1640 1 Processor -- start 2024-04-25T10:14:17.924 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.922+0000 7f30a46c1640 1 -- start start 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a46c1640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f309c15b950 0x7f309c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a46c1640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f309c0cf420 0x7f309c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a46c1640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f309c15bd30 con 0x7f309c0cfb80 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a46c1640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c0cc0d0 con 0x7f309c15b950 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a46c1640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c09ab10 con 0x7f309c0cf420 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f309c15b950 0x7f309c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f309c15b950 0x7f309c09a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46394/0 (socket says 172.21.15.82:46394) 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 -- 172.21.15.82:0/1288381959 learned_addr learned my addr 172.21.15.82:0/1288381959 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a2c5e640 1 --1- 172.21.15.82:0/1288381959 >> v1:172.21.15.82:6789/0 conn(0x7f309c0cfb80 0x7f309c0cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60730/0 (socket says 172.21.15.82:60730) 2024-04-25T10:14:17.926 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a1c5c640 1 --2- 172.21.15.82:0/1288381959 >> v2:172.21.15.196:3300/0 conn(0x7f309c0cf420 0x7f309c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 -- 172.21.15.82:0/1288381959 >> v2:172.21.15.196:3300/0 conn(0x7f309c0cf420 msgr2=0x7f309c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 --2- 172.21.15.82:0/1288381959 >> v2:172.21.15.196:3300/0 conn(0x7f309c0cf420 0x7f309c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 -- 172.21.15.82:0/1288381959 >> v1:172.21.15.82:6789/0 conn(0x7f309c0cfb80 legacy=0x7f309c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 -- 172.21.15.82:0/1288381959 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0cc370 con 0x7f309c15b950 2024-04-25T10:14:17.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a145b640 1 -- 172.21.15.82:0/1288381959 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708361079 0 0) 0x7f309c15bd30 con 0x7f309c0cfb80 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a245d640 1 --2- 172.21.15.82:0/1288381959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c15b950 0x7f309c09a480 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f309c072710 tx=0x7f309801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f33a736f3af90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a145b640 1 -- 172.21.15.82:0/1288381959 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980aa650 con 0x7f309c15b950 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a145b640 1 -- 172.21.15.82:0/1288381959 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980af070 con 0x7f309c15b950 2024-04-25T10:14:17.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.926+0000 7f30a145b640 1 -- 172.21.15.82:0/1288381959 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980aaa10 con 0x7f309c15b950 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/1288381959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c15b950 msgr2=0x7f309c09a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- 172.21.15.82:0/1288381959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c15b950 0x7f309c09a480 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f309c072710 tx=0x7f309801c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/1288381959 shutdown_connections 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- 172.21.15.82:0/1288381959 >> v2:172.21.15.196:3300/0 conn(0x7f309c0cf420 0x7f309c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- 172.21.15.82:0/1288381959 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c15b950 0x7f309c09a480 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/1288381959 >> 172.21.15.82:0/1288381959 conn(0x7f309c0b30b0 msgr2=0x7f309c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/1288381959 shutdown_connections 2024-04-25T10:14:17.929 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/1288381959 wait complete. 2024-04-25T10:14:17.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 Processor -- start 2024-04-25T10:14:17.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- start start 2024-04-25T10:14:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 0x7f309c1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f309c15b950 0x7f309c1f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0cd390 con 0x7f309c0cf420 2024-04-25T10:14:17.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c06faa0 con 0x7f309c0cfb80 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0c6f00 con 0x7f309c15b950 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46402/0 (socket says 172.21.15.82:46402) 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 -- 172.21.15.82:0/4214595999 learned_addr learned my addr 172.21.15.82:0/4214595999 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a2c5e640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f309c15b950 0x7f309c1f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a1c5c640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 0x7f309c1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 -- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f309c15b950 msgr2=0x7f309c1f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f309c15b950 0x7f309c1f9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 -- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 msgr2=0x7f309c1f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 0x7f309c1f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 -- 172.21.15.82:0/4214595999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c0c71e0 con 0x7f309c0cf420 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a1c5c640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 0x7f309c1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:17.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a245d640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3098019100 tx=0x7f309801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980b3600 con 0x7f309c0cf420 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30980a6040 con 0x7f309c0cf420 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c0c74c0 con 0x7f309c0cf420 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30980b3a00 con 0x7f309c0cf420 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.930+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c1f5ed0 con 0x7f309c0cf420 2024-04-25T10:14:17.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309c04efa0 con 0x7f309c0cf420 2024-04-25T10:14:17.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30980b3ba0 con 0x7f309c0cf420 2024-04-25T10:14:17.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f3092ffd640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 0x7f3080069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:17.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f309811f9b0 con 0x7f309c0cf420 2024-04-25T10:14:17.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f30a1c5c640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 0x7f3080069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:17.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.934+0000 7f30a1c5c640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 0x7f3080069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f308c006fd0 tx=0x7f308c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:17.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:17.938+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30980c3060 con 0x7f309c0cf420 2024-04-25T10:14:18.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.098+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f309c0d13a0 con 0x7f309c0cf420 2024-04-25T10:14:18.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.098+0000 7f3092ffd640 1 -- 172.21.15.82:0/4214595999 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3098016110 con 0x7f309c0cf420 2024-04-25T10:14:18.098 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 msgr2=0x7f3080069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 0x7f3080069450 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f308c006fd0 tx=0x7f308c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 msgr2=0x7f309c1f5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f3098019100 tx=0x7f309801ad30 comp rx=0 tx=0).stop 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 shutdown_connections 2024-04-25T10:14:18.101 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3080066f90 0x7f3080069450 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f309c15b950 0x7f309c1f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f309c0cfb80 0x7f309c1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 --2- 172.21.15.82:0/4214595999 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f309c0cf420 0x7f309c1f5990 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 >> 172.21.15.82:0/4214595999 conn(0x7f309c0b30b0 msgr2=0x7f309c073830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 shutdown_connections 2024-04-25T10:14:18.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.102+0000 7f30a46c1640 1 -- 172.21.15.82:0/4214595999 wait complete. 2024-04-25T10:14:18.118 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:18.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 Processor -- start 2024-04-25T10:14:18.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 -- start start 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d00af280 0x7fa6d015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6d006c3b0 0x7fa6d006df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d0066c30 con 0x7fa6d00b2c30 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d006daa0 con 0x7fa6d006c3b0 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d84ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d00af660 con 0x7fa6d00af280 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6d006c3b0 0x7fa6d006df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.247 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d5a85640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa6d00b2c30 0x7fa6d006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60740/0 (socket says 172.21.15.82:60740) 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fa6d006c3b0 0x7fa6d006df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46418/0 (socket says 172.21.15.82:46418) 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d5a85640 1 -- 172.21.15.82:0/3959273170 learned_addr learned my addr 172.21.15.82:0/3959273170 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6286640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d00af280 0x7fa6d015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 -- 172.21.15.82:0/3959273170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d00af280 msgr2=0x7fa6d015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d00af280 0x7fa6d015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 -- 172.21.15.82:0/3959273170 >> v1:172.21.15.82:6789/0 conn(0x7fa6d00b2c30 legacy=0x7fa6d006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d6a87640 1 -- 172.21.15.82:0/3959273170 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d015e640 con 0x7fa6d006c3b0 2024-04-25T10:14:18.248 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.246+0000 7fa6d5284640 1 -- 172.21.15.82:0/3959273170 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567521803 0 0) 0x7fa6d0066c30 con 0x7fa6d00b2c30 2024-04-25T10:14:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d006c3b0 0x7fa6d006df30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009d90 tx=0x7fa6cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169c2276eec5343e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d5284640 1 -- 172.21.15.82:0/3959273170 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0b2070 con 0x7fa6d006c3b0 2024-04-25T10:14:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d5284640 1 -- 172.21.15.82:0/3959273170 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0a5830 con 0x7fa6d006c3b0 2024-04-25T10:14:18.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d5284640 1 -- 172.21.15.82:0/3959273170 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0a5c00 con 0x7fa6d006c3b0 2024-04-25T10:14:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- 172.21.15.82:0/3959273170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d006c3b0 msgr2=0x7fa6d006df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d006c3b0 0x7fa6d006df30 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc009d90 tx=0x7fa6cc01c970 comp rx=0 tx=0).stop 2024-04-25T10:14:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- 172.21.15.82:0/3959273170 shutdown_connections 2024-04-25T10:14:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d006c3b0 0x7fa6d006df30 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/3959273170 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d00af280 0x7fa6d015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- 172.21.15.82:0/3959273170 >> 172.21.15.82:0/3959273170 conn(0x7fa6d0071810 msgr2=0x7fa6d0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- 172.21.15.82:0/3959273170 shutdown_connections 2024-04-25T10:14:18.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- 172.21.15.82:0/3959273170 wait complete. 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 Processor -- start 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- start start 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d006c3b0 0x7fa6d014c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00af280 0x7fa6d00acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d006f010 con 0x7fa6d00b2c30 2024-04-25T10:14:18.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d006ee90 con 0x7fa6d00af280 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d84ea640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d006f190 con 0x7fa6d006c3b0 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46428/0 (socket says 172.21.15.82:46428) 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 -- 172.21.15.82:0/4170243017 learned_addr learned my addr 172.21.15.82:0/4170243017 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d5a85640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00af280 0x7fa6d00acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 -- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d006c3b0 msgr2=0x7fa6d014c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6286640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d006c3b0 0x7fa6d014c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d006c3b0 0x7fa6d014c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 -- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00af280 msgr2=0x7fa6d00acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00af280 0x7fa6d00acf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.250+0000 7fa6d6a87640 1 -- 172.21.15.82:0/4170243017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0150100 con 0x7fa6d00b2c30 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d6a87640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0b0040 tx=0x7fa6cc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0bd020 con 0x7fa6d00b2c30 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6cc0b8670 con 0x7fa6d00b2c30 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d014fb40 con 0x7fa6d00b2c30 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6cc0b2040 con 0x7fa6d00b2c30 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d015faf0 con 0x7fa6d00b2c30 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d004efa0 con 0x7fa6d00b2c30 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6cc0c0020 con 0x7fa6d00b2c30 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 0x7fa6b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa6cc11fd50 con 0x7fa6d00b2c30 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d6286640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 0x7fa6b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.254+0000 7fa6d6286640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 0x7fa6b4069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cee10 tx=0x7fa6c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.258+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6cc123030 con 0x7fa6d00b2c30 2024-04-25T10:14:18.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.418+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa6d00b0440 con 0x7fa6d00b2c30 2024-04-25T10:14:18.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.418+0000 7fa6c6ffd640 1 -- 172.21.15.82:0/4170243017 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa6cc0c8020 con 0x7fa6d00b2c30 2024-04-25T10:14:18.419 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 msgr2=0x7fa6b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 0x7fa6b4069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00cee10 tx=0x7fa6c8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 msgr2=0x7fa6d00ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fa6cc0b0040 tx=0x7fa6cc019710 comp rx=0 tx=0).stop 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 shutdown_connections 2024-04-25T10:14:18.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa6b4066f90 0x7fa6b4069450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa6d00b2c30 0x7fa6d00ad450 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa6d00af280 0x7fa6d00acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 --2- 172.21.15.82:0/4170243017 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa6d006c3b0 0x7fa6d014c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 >> 172.21.15.82:0/4170243017 conn(0x7fa6d0071810 msgr2=0x7fa6d00af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 shutdown_connections 2024-04-25T10:14:18.422 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.422+0000 7fa6d84ea640 1 -- 172.21.15.82:0/4170243017 wait complete. 2024-04-25T10:14:18.438 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.558+0000 7fca0acf3640 1 Processor -- start 2024-04-25T10:14:18.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.558+0000 7fca0acf3640 1 -- start start 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca0acf3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca0415ecb0 0x7fca040c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca0acf3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fca040c9ef0 0x7fca040ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca0acf3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca040ca4c0 con 0x7fca040cf590 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca0acf3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca040ca780 con 0x7fca0415ecb0 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca0acf3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca0415f0f0 con 0x7fca040c9ef0 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca0415ecb0 0x7fca040c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca08a8f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fca040cf590 0x7fca040cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60742/0 (socket says 172.21.15.82:60742) 2024-04-25T10:14:18.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fca0415ecb0 0x7fca040c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46438/0 (socket says 172.21.15.82:46438) 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca08a8f640 1 -- 172.21.15.82:0/288759119 learned_addr learned my addr 172.21.15.82:0/288759119 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca09290640 1 --2- 172.21.15.82:0/288759119 >> v2:172.21.15.196:3300/0 conn(0x7fca040c9ef0 0x7fca040ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 -- 172.21.15.82:0/288759119 >> v2:172.21.15.196:3300/0 conn(0x7fca040c9ef0 msgr2=0x7fca040ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 --2- 172.21.15.82:0/288759119 >> v2:172.21.15.196:3300/0 conn(0x7fca040c9ef0 0x7fca040ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 -- 172.21.15.82:0/288759119 >> v1:172.21.15.82:6789/0 conn(0x7fca040cf590 legacy=0x7fca040cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca037fe640 1 -- 172.21.15.82:0/288759119 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643343588 0 0) 0x7fca040ca4c0 con 0x7fca040cf590 2024-04-25T10:14:18.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 -- 172.21.15.82:0/288759119 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040cd690 con 0x7fca0415ecb0 2024-04-25T10:14:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca03fff640 1 --2- 172.21.15.82:0/288759119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0415ecb0 0x7fca040c99b0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fca040b38c0 tx=0x7fc9ec01eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ae13ded0805371 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca037fe640 1 -- 172.21.15.82:0/288759119 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0b4070 con 0x7fca0415ecb0 2024-04-25T10:14:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca037fe640 1 -- 172.21.15.82:0/288759119 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0a7b30 con 0x7fca0415ecb0 2024-04-25T10:14:18.564 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.562+0000 7fca037fe640 1 -- 172.21.15.82:0/288759119 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0af440 con 0x7fca0415ecb0 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/288759119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0415ecb0 msgr2=0x7fca040c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- 172.21.15.82:0/288759119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0415ecb0 0x7fca040c99b0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fca040b38c0 tx=0x7fc9ec01eb50 comp rx=0 tx=0).stop 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/288759119 shutdown_connections 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- 172.21.15.82:0/288759119 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca0415ecb0 0x7fca040c99b0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- 172.21.15.82:0/288759119 >> v2:172.21.15.196:3300/0 conn(0x7fca040c9ef0 0x7fca040ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/288759119 >> 172.21.15.82:0/288759119 conn(0x7fca0406ae60 msgr2=0x7fca0406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/288759119 shutdown_connections 2024-04-25T10:14:18.565 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/288759119 wait complete. 2024-04-25T10:14:18.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 Processor -- start 2024-04-25T10:14:18.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- start start 2024-04-25T10:14:18.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca040c9ef0 0x7fca04152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.568 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0415ecb0 0x7fca040c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040cc170 con 0x7fca040cf590 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca040cbff0 con 0x7fca0415ecb0 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca040cc2f0 con 0x7fca040c9ef0 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46448/0 (socket says 172.21.15.82:46448) 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 -- 172.21.15.82:0/1320953717 learned_addr learned my addr 172.21.15.82:0/1320953717 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca09290640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0415ecb0 0x7fca040c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 -- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca040c9ef0 msgr2=0x7fca04152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca040c9ef0 0x7fca04152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 -- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0415ecb0 msgr2=0x7fca040c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0415ecb0 0x7fca040c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 -- 172.21.15.82:0/1320953717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040c2e90 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca03fff640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0b2040 tx=0x7fc9ec01b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0b4040 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec0c1070 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0415d510 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec0ba470 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.566+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca040c3150 con 0x7fca040cf590 2024-04-25T10:14:18.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0404efa0 con 0x7fca040cf590 2024-04-25T10:14:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9ec0ba610 con 0x7fca040cf590 2024-04-25T10:14:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca017fa640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 0x7fc9e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc9ec122720 con 0x7fca040cf590 2024-04-25T10:14:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca08a8f640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 0x7fc9e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.570+0000 7fca08a8f640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 0x7fc9e4069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001570 tx=0x7fc9fc007cb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.573 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.574+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc9ec018740 con 0x7fca040cf590 2024-04-25T10:14:18.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.730+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca040ca4c0 con 0x7fca040cf590 2024-04-25T10:14:18.731 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.730+0000 7fca017fa640 1 -- 172.21.15.82:0/1320953717 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc9ec0f34f0 con 0x7fca040cf590 2024-04-25T10:14:18.731 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 msgr2=0x7fc9e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 0x7fc9e4069420 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc001570 tx=0x7fc9fc007cb0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 msgr2=0x7fca040c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec0b2040 tx=0x7fc9ec01b8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 shutdown_connections 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc9e4066f60 0x7fc9e4069420 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fca0415ecb0 0x7fca040c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fca040cf590 0x7fca040c5370 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 --2- 172.21.15.82:0/1320953717 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fca040c9ef0 0x7fca04152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 >> 172.21.15.82:0/1320953717 conn(0x7fca0406ae60 msgr2=0x7fca040afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 shutdown_connections 2024-04-25T10:14:18.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.734+0000 7fca0acf3640 1 -- 172.21.15.82:0/1320953717 wait complete. 2024-04-25T10:14:18.750 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 Processor -- start 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 -- start start 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3260066e30 0x7f326006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f32600ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326006ec30 con 0x7f326006bca0 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32600669a0 con 0x7f3260066e30 2024-04-25T10:14:18.870 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3267153640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326006c5c0 con 0x7f32600af370 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3260066e30 0x7f326006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f32656f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f32600ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3264eef640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f326006bca0 0x7f32600cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:56256/0 (socket says 172.21.15.82:56256) 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f3264eef640 1 -- 172.21.15.82:0/788599363 learned_addr learned my addr 172.21.15.82:0/788599363 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f3260066e30 0x7f326006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:39382/0 (socket says 172.21.15.82:39382) 2024-04-25T10:14:18.871 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 -- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 msgr2=0x7f32600ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 --2- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f32600ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 -- 172.21.15.82:0/788599363 >> v1:172.21.15.196:6789/0 conn(0x7f326006bca0 legacy=0x7f32600cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.872 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.870+0000 7f325ffff640 1 -- 172.21.15.82:0/788599363 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326006f230 con 0x7f3260066e30 2024-04-25T10:14:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f325ffff640 1 --2- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326006c080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32540097c0 tx=0x7f325401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fcf08ebc36b1327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f325f7fe640 1 -- 172.21.15.82:0/788599363 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b0070 con 0x7f3260066e30 2024-04-25T10:14:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f325f7fe640 1 -- 172.21.15.82:0/788599363 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32540ab600 con 0x7f3260066e30 2024-04-25T10:14:18.873 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f325f7fe640 1 -- 172.21.15.82:0/788599363 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540ab9d0 con 0x7f3260066e30 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 msgr2=0x7f326006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 --2- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326006c080 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f32540097c0 tx=0x7f325401c350 comp rx=0 tx=0).stop 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- 172.21.15.82:0/788599363 shutdown_connections 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 --2- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f32600ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 --2- 172.21.15.82:0/788599363 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326006c080 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- 172.21.15.82:0/788599363 >> 172.21.15.82:0/788599363 conn(0x7f3260071810 msgr2=0x7f3260071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- 172.21.15.82:0/788599363 shutdown_connections 2024-04-25T10:14:18.875 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- 172.21.15.82:0/788599363 wait complete. 2024-04-25T10:14:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 Processor -- start 2024-04-25T10:14:18.876 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.874+0000 7f3267153640 1 -- start start 2024-04-25T10:14:18.877 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326014bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f3260134d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32601687b0 con 0x7f326006bca0 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3260168630 con 0x7f3260066e30 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3260168930 con 0x7f32600af370 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46454/0 (socket says 172.21.15.82:46454) 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 -- 172.21.15.82:0/2590637836 learned_addr learned my addr 172.21.15.82:0/2590637836 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3264eef640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326014bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 -- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 msgr2=0x7f3260134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f3260134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.878 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 -- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 msgr2=0x7f326014bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326014bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 -- 172.21.15.82:0/2590637836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32601357e0 con 0x7f326006bca0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3264eef640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326014bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325ffff640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f3254009710 tx=0x7f32540190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b8020 con 0x7f326006bca0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32540b4410 con 0x7f326006bca0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3260135b20 con 0x7f326006bca0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b0040 con 0x7f326006bca0 2024-04-25T10:14:18.879 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326014b100 con 0x7f326006bca0 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326014b330 con 0x7f326006bca0 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32540bb020 con 0x7f326006bca0 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f325411ebd0 con 0x7f326006bca0 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3264eef640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:18.880 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.878+0000 7f3264eef640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 0x7f324c069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3250005890 tx=0x7f3250009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:18.883 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:18.882+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f32540c2030 con 0x7f326006bca0 2024-04-25T10:14:19.046 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.046+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f326004efa0 con 0x7f326006bca0 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.046+0000 7f325d7fa640 1 -- 172.21.15.82:0/2590637836 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v123) v1 ==== 80+0+242 (secure 0 0 0) 0x7f3260135b20 con 0x7f326006bca0 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/3303693719 2024-04-26T10:14:07.178065+0000 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6837/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6835/3303693719 2024-04-26T10:14:07.178065+0000 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stdout:172.21.15.82:6835/781412207 2024-04-26T10:13:47.565011+0000 2024-04-25T10:14:19.047 INFO:teuthology.orchestra.run.smithi082.stderr:listed 4 entries 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 msgr2=0x7f324c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 0x7f324c069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3250005890 tx=0x7f3250009040 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 msgr2=0x7f32601478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7f3254009710 tx=0x7f32540190f0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 shutdown_connections 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f324c066f60 0x7f324c069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f32600af370 0x7f3260134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f326006bca0 0x7f32601478f0 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 --2- 172.21.15.82:0/2590637836 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3260066e30 0x7f326014bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 >> 172.21.15.82:0/2590637836 conn(0x7f3260071810 msgr2=0x7f3260152640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 shutdown_connections 2024-04-25T10:14:19.050 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.050+0000 7f3267153640 1 -- 172.21.15.82:0/2590637836 wait complete. 2024-04-25T10:14:19.064 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi082.front.sepia.ceph.com... 2024-04-25T10:14:19.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.065 DEBUG:teuthology.orchestra.run.smithi082:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:19.075 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi082.stderr:ceph-fuse[56611]: fuse finished with error 0 and tester_r 0 2024-04-25T10:14:19.076 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:14:19.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.077 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:19.243 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:14:19.243 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:19.250 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:19.250 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:14:19.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.250 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:14:19.296 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T10:14:19.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:19.297 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:14:19.297 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:19.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:19.304 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:14:19.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.304 DEBUG:teuthology.orchestra.run.smithi082:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:14:19.350 INFO:teuthology.orchestra.run.smithi082.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T10:14:19.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:19.351 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:14:19.351 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:19.357 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:19.357 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:14:19.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.357 DEBUG:teuthology.orchestra.run.smithi196:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:14:19.404 INFO:teuthology.orchestra.run.smithi196.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T10:14:19.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:14:19.405 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 Processor -- start 2024-04-25T10:14:19.537 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 -- start start 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f68940cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f68940cbea0 0x7f68940cc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68940d0050 con 0x7f6894099e50 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68940cfe20 con 0x7f68940cbea0 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689c28a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940d0330 con 0x7f6894099a70 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899825640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f6894099e50 0x7f689406c9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60746/0 (socket says 172.21.15.82:60746) 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899825640 1 -- 172.21.15.82:0/1905741812 learned_addr learned my addr 172.21.15.82:0/1905741812 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 --2- 172.21.15.82:0/1905741812 >> v2:172.21.15.196:3300/0 conn(0x7f68940cbea0 0x7f68940cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a026640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f68940cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174003156 0 0) 0x7f68940d0050 con 0x7f6894099e50 2024-04-25T10:14:19.540 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6880003530 con 0x7f6894099e50 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165201596 0 0) 0x7f6880003530 con 0x7f6894099e50 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 msgr2=0x7f68940cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f68940cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 >> v2:172.21.15.196:3300/0 conn(0x7f68940cbea0 msgr2=0x7f68940cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 --2- 172.21.15.82:0/1905741812 >> v2:172.21.15.196:3300/0 conn(0x7f68940cbea0 0x7f68940cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940d0610 con 0x7f6894099e50 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f6884001680 con 0x7f6894099e50 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 >> v1:172.21.15.82:6789/0 conn(0x7f6894099e50 legacy=0x7f689406c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880003a30 0x7f6880003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f6880004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f68800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940d0330 con 0x7f6880004d50 2024-04-25T10:14:19.541 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688000b9d0 con 0x7f6880003a30 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688000bba0 con 0x7f6894099a70 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f68800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a026640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880003a30 0x7f6880003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899825640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f6880004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 -- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 msgr2=0x7f6880004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f6880004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 -- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880003a30 msgr2=0x7f6880003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6880003a30 0x7f6880003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f689a827640 1 -- 172.21.15.82:0/1905741812 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880003530 con 0x7f6880004d50 2024-04-25T10:14:19.542 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.538+0000 7f6899825640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f6894099a70 0x7f6880004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689a827640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f68800051d0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fc0 tx=0x7f688c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69ed30857e660bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c0b2070 con 0x7f6880004d50 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688c0a5bd0 con 0x7f6880004d50 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f6899024640 1 -- 172.21.15.82:0/1905741812 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688c009c10 con 0x7f6880004d50 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- 172.21.15.82:0/1905741812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 msgr2=0x7f68800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 --2- 172.21.15.82:0/1905741812 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f68800051d0 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f688c009fc0 tx=0x7f688c01ccc0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689a827640 1 -- 172.21.15.82:0/1905741812 reap_dead start 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- 172.21.15.82:0/1905741812 shutdown_connections 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- 172.21.15.82:0/1905741812 >> 172.21.15.82:0/1905741812 conn(0x7f68940b3100 msgr2=0x7f68940b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- 172.21.15.82:0/1905741812 shutdown_connections 2024-04-25T10:14:19.544 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- 172.21.15.82:0/1905741812 wait complete. 2024-04-25T10:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 Processor -- start 2024-04-25T10:14:19.545 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.542+0000 7f689c28a640 1 -- start start 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 0x7f6894158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689414dad0 0x7f6894159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689409a2b0 con 0x7f6880004d50 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68940d0bd0 con 0x7f6894156ba0 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68940b2da0 con 0x7f689414dad0 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46470/0 (socket says 172.21.15.82:46470) 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 -- 172.21.15.82:0/3409278869 learned_addr learned my addr 172.21.15.82:0/3409278869 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f6899825640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 0x7f6894158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a827640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689414dad0 0x7f6894159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 -- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689414dad0 msgr2=0x7f6894159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689414dad0 0x7f6894159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 -- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 msgr2=0x7f6894158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 0x7f6894158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 -- 172.21.15.82:0/3409278869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894157e00 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f6899825640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 0x7f6894158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689a026640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f6890011560 tx=0x7f6890011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890015930 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6890015ad0 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68941580e0 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689001d400 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68941586a0 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689404efa0 con 0x7f6880004d50 2024-04-25T10:14:19.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f689001d5a0 con 0x7f6880004d50 2024-04-25T10:14:19.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f6890085b10 con 0x7f6880004d50 2024-04-25T10:14:19.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f6899825640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 0x7f6878069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.549 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.546+0000 7f6899825640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68840017f0 tx=0x7f6884001440 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.550+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6890019030 con 0x7f6880004d50 2024-04-25T10:14:19.729 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.730+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68940b3cc0 con 0x7f6880004d50 2024-04-25T10:14:19.730 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.730+0000 7f688affd640 1 -- 172.21.15.82:0/3409278869 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4954 (secure 0 0 0) 0x7f68900568e0 con 0x7f6880004d50 2024-04-25T10:14:19.730 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:19.730 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:11.226477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:19.730 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 104 2024-04-25T10:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 msgr2=0x7f6878069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68840017f0 tx=0x7f6884001440 comp rx=0 tx=0).stop 2024-04-25T10:14:19.733 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 msgr2=0x7f6894157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f6890011560 tx=0x7f6890011a20 comp rx=0 tx=0).stop 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 shutdown_connections 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f6878066f60 0x7f6878069420 secure :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f68840017f0 tx=0x7f6884001440 comp rx=0 tx=0).stop 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f689414dad0 0x7f6894159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f6894156ba0 0x7f6894158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 --2- 172.21.15.82:0/3409278869 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f6880004d50 0x7f6894157770 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 >> 172.21.15.82:0/3409278869 conn(0x7f68940b3100 msgr2=0x7f68940ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 shutdown_connections 2024-04-25T10:14:19.734 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.734+0000 7f689c28a640 1 -- 172.21.15.82:0/3409278869 wait complete. 2024-04-25T10:14:19.749 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.854+0000 7fea9b249640 1 Processor -- start 2024-04-25T10:14:19.856 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.854+0000 7fea9b249640 1 -- start start 2024-04-25T10:14:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea940ce510 0x7fea940ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fea941451f0 0x7fea94142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.857 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940cee30 con 0x7fea94144ca0 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea940cc2b0 con 0x7fea940ce510 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9406e290 con 0x7fea941451f0 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fea940ce510 0x7fea940ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea93fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fea94144ca0 0x7fea9406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60762/0 (socket says 172.21.15.82:60762) 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea93fff640 1 -- 172.21.15.82:0/3875452839 learned_addr learned my addr 172.21.15.82:0/3875452839 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea997e6640 1 --2- 172.21.15.82:0/3875452839 >> v2:172.21.15.196:3300/0 conn(0x7fea941451f0 0x7fea94142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 -- 172.21.15.82:0/3875452839 >> v2:172.21.15.196:3300/0 conn(0x7fea941451f0 msgr2=0x7fea94142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 --2- 172.21.15.82:0/3875452839 >> v2:172.21.15.196:3300/0 conn(0x7fea941451f0 0x7fea94142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 -- 172.21.15.82:0/3875452839 >> v1:172.21.15.82:6789/0 conn(0x7fea94144ca0 legacy=0x7fea9406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.858 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 -- 172.21.15.82:0/3875452839 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea940ccc70 con 0x7fea940ce510 2024-04-25T10:14:19.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea98fe5640 1 --2- 172.21.15.82:0/3875452839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea940ce510 0x7fea940ce8f0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fea940b48f0 tx=0x7fea8401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6297b37220e2f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea937fe640 1 -- 172.21.15.82:0/3875452839 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea840b5070 con 0x7fea940ce510 2024-04-25T10:14:19.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea937fe640 1 -- 172.21.15.82:0/3875452839 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea840a8970 con 0x7fea940ce510 2024-04-25T10:14:19.859 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea937fe640 1 -- 172.21.15.82:0/3875452839 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea840a8d40 con 0x7fea940ce510 2024-04-25T10:14:19.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- 172.21.15.82:0/3875452839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea940ce510 msgr2=0x7fea940ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 --2- 172.21.15.82:0/3875452839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea940ce510 0x7fea940ce8f0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fea940b48f0 tx=0x7fea8401ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- 172.21.15.82:0/3875452839 shutdown_connections 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 --2- 172.21.15.82:0/3875452839 >> v2:172.21.15.196:3300/0 conn(0x7fea941451f0 0x7fea94142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 --2- 172.21.15.82:0/3875452839 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea940ce510 0x7fea940ce8f0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- 172.21.15.82:0/3875452839 >> 172.21.15.82:0/3875452839 conn(0x7fea940b11a0 msgr2=0x7fea940b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- 172.21.15.82:0/3875452839 shutdown_connections 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.858+0000 7fea9b249640 1 -- 172.21.15.82:0/3875452839 wait complete. 2024-04-25T10:14:19.861 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 Processor -- start 2024-04-25T10:14:19.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- start start 2024-04-25T10:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 0x7fea940cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea94144ca0 0x7fea941e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406ee30 con 0x7fea941451f0 2024-04-25T10:14:19.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9406df20 con 0x7fea940ce510 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea940cc900 con 0x7fea94144ca0 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46490/0 (socket says 172.21.15.82:46490) 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 -- 172.21.15.82:0/1568031154 learned_addr learned my addr 172.21.15.82:0/1568031154 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea98fe5640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 0x7fea940cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea93fff640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea94144ca0 0x7fea941e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 -- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea94144ca0 msgr2=0x7fea941e1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea94144ca0 0x7fea941e1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 -- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 msgr2=0x7fea940cb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 0x7fea940cb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 -- 172.21.15.82:0/1568031154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea941e24a0 con 0x7fea941451f0 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea98fe5640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 0x7fea940cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea997e6640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00cc80 tx=0x7fea8c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0039a0 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c003b60 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea941e2780 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c004020 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.862+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea941e2d40 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9404efa0 con 0x7fea941451f0 2024-04-25T10:14:19.867 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea8c0041c0 con 0x7fea941451f0 2024-04-25T10:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea917fa640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 0x7fea80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fea8c085720 con 0x7fea941451f0 2024-04-25T10:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea98fe5640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 0x7fea80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:19.868 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.866+0000 7fea98fe5640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 0x7fea80069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fea840b2000 tx=0x7fea840aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:19.869 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:19.870+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea8c014030 con 0x7fea941451f0 2024-04-25T10:14:20.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.022+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea941e3a80 con 0x7fea941451f0 2024-04-25T10:14:20.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea917fa640 1 -- 172.21.15.82:0/1568031154 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17221 (secure 0 0 0) 0x7fea8c0564f0 con 0x7fea941451f0 2024-04-25T10:14:20.026 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:20.026 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":123,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:14:07.178089+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:20.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 msgr2=0x7fea80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 0x7fea80069420 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fea840b2000 tx=0x7fea840aa070 comp rx=0 tx=0).stop 2024-04-25T10:14:20.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 msgr2=0x7fea941e3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00cc80 tx=0x7fea8c00a5c0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 shutdown_connections 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fea80066f60 0x7fea80069420 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fea94144ca0 0x7fea941e1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fea941451f0 0x7fea941e3540 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 --2- 172.21.15.82:0/1568031154 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fea940ce510 0x7fea940cb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 >> 172.21.15.82:0/1568031154 conn(0x7fea940b11a0 msgr2=0x7fea940c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 shutdown_connections 2024-04-25T10:14:20.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.026+0000 7fea9b249640 1 -- 172.21.15.82:0/1568031154 wait complete. 2024-04-25T10:14:20.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:55.331959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:19.785837+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:20.465294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:20.045 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.162+0000 7f33d8759640 1 Processor -- start 2024-04-25T10:14:20.164 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.162+0000 7f33d8759640 1 -- start start 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d8759640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33d00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d8759640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f33d0073010 0x7f33d00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d8759640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d00cbd20 con 0x7f33d00cf590 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d8759640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d015ba90 con 0x7f33d0073010 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d8759640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00cff30 con 0x7f33d015b6b0 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d5cf4640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f33d00cf590 0x7f33d015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60768/0 (socket says 172.21.15.82:60768) 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d5cf4640 1 -- 172.21.15.82:0/3878219045 learned_addr learned my addr 172.21.15.82:0/3878219045 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d6cf6640 1 --2- 172.21.15.82:0/3878219045 >> v2:172.21.15.196:3300/0 conn(0x7f33d0073010 0x7f33d00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.167 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33d00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232215619 0 0) 0x7f33d00cbd20 con 0x7f33d00cf590 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33bc003530 con 0x7f33d00cf590 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3545472416 0 0) 0x7f33bc003530 con 0x7f33d00cf590 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 msgr2=0x7f33d00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33d00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 >> v2:172.21.15.196:3300/0 conn(0x7f33d0073010 msgr2=0x7f33d00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 --2- 172.21.15.82:0/3878219045 >> v2:172.21.15.196:3300/0 conn(0x7f33d0073010 0x7f33d00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d00cc000 con 0x7f33d00cf590 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f33c0001680 con 0x7f33d00cf590 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 >> v1:172.21.15.82:6789/0 conn(0x7f33d00cf590 legacy=0x7f33d015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc004d50 0x7f33bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d015ba90 con 0x7f33bc003a30 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc00b9d0 con 0x7f33bc004d50 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc00bb50 con 0x7f33d015b6b0 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d5cf4640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d6cf6640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc004d50 0x7f33bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 -- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 msgr2=0x7f33bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d015b6b0 0x7f33bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 -- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc004d50 msgr2=0x7f33bc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc004d50 0x7f33bc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d64f5640 1 -- 172.21.15.82:0/3878219045 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc003530 con 0x7f33bc003a30 2024-04-25T10:14:20.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.166+0000 7f33d6cf6640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33bc004d50 0x7f33bc0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33bc003e50 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009ae0 tx=0x7f33cc01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c644540846c64eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0b2070 con 0x7f33bc003a30 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0a5610 con 0x7f33bc003a30 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d54f3640 1 -- 172.21.15.82:0/3878219045 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0096b0 con 0x7f33bc003a30 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- 172.21.15.82:0/3878219045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 msgr2=0x7f33bc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 --2- 172.21.15.82:0/3878219045 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33bc003e50 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009ae0 tx=0x7f33cc01ca40 comp rx=0 tx=0).stop 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d6cf6640 1 -- 172.21.15.82:0/3878219045 reap_dead start 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- 172.21.15.82:0/3878219045 shutdown_connections 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- 172.21.15.82:0/3878219045 >> 172.21.15.82:0/3878219045 conn(0x7f33d006ae60 msgr2=0x7f33d006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- 172.21.15.82:0/3878219045 shutdown_connections 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- 172.21.15.82:0/3878219045 wait complete. 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 Processor -- start 2024-04-25T10:14:20.172 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- start start 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d01f54f0 0x7f33d01f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 0x7f33d01f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00cadb0 con 0x7f33bc003a30 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00cba30 con 0x7f33d01f79d0 2024-04-25T10:14:20.173 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d8759640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d00caf30 con 0x7f33d01f54f0 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46512/0 (socket says 172.21.15.82:46512) 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 -- 172.21.15.82:0/1516250664 learned_addr learned my addr 172.21.15.82:0/1516250664 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d6cf6640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 0x7f33d01f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 -- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d01f54f0 msgr2=0x7f33d01f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d01f54f0 0x7f33d01f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 -- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 msgr2=0x7f33d01f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 0x7f33d01f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 -- 172.21.15.82:0/1516250664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d0154f90 con 0x7f33bc003a30 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d6cf6640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 0x7f33d01f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.170+0000 7f33d64f5640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009ae0 tx=0x7f33bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0bc030 con 0x7f33bc003a30 2024-04-25T10:14:20.174 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0a5e40 con 0x7f33bc003a30 2024-04-25T10:14:20.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d0155270 con 0x7f33bc003a30 2024-04-25T10:14:20.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0b2040 con 0x7f33bc003a30 2024-04-25T10:14:20.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d00c48a0 con 0x7f33bc003a30 2024-04-25T10:14:20.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d004efa0 con 0x7f33bc003a30 2024-04-25T10:14:20.175 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f33cc0bf070 con 0x7f33bc003a30 2024-04-25T10:14:20.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f33cc11f1d0 con 0x7f33bc003a30 2024-04-25T10:14:20.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33d5cf4640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.176 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.174+0000 7f33d5cf4640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 0x7f33b4069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000fc0 tx=0x7f33c00013c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.178 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.178+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f33cc016780 con 0x7f33bc003a30 2024-04-25T10:14:20.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.334+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33d006b730 con 0x7f33bc003a30 2024-04-25T10:14:20.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33c6ffd640 1 -- 172.21.15.82:0/1516250664 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33cc0f0030 con 0x7f33bc003a30 2024-04-25T10:14:20.336 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 1 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 msgr2=0x7f33b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 0x7f33b4069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f33c0000fc0 tx=0x7f33c00013c0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 msgr2=0x7f33d01558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009ae0 tx=0x7f33bc005840 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 shutdown_connections 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f33b4066f90 0x7f33b4069450 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f33d01f54f0 0x7f33d01f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f33d01f79d0 0x7f33d01f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 --2- 172.21.15.82:0/1516250664 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f33bc003a30 0x7f33d01558a0 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 >> 172.21.15.82:0/1516250664 conn(0x7f33d006ae60 msgr2=0x7f33d00ac6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 shutdown_connections 2024-04-25T10:14:20.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.338+0000 7f33d8759640 1 -- 172.21.15.82:0/1516250664 wait complete. 2024-04-25T10:14:20.355 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:20.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.474+0000 7fd2be9f4640 1 Processor -- start 2024-04-25T10:14:20.476 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.474+0000 7fd2be9f4640 1 -- start start 2024-04-25T10:14:20.477 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2be9f4640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fd2b8161c80 0x7fd2b80b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2be9f4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2b80c2a40 0x7fd2b80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2be9f4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b8162060 con 0x7fd2b80c9f70 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2be9f4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80ca480 con 0x7fd2b80c2a40 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2be9f4640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80ca740 con 0x7fd2b8161c80 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2b80c2a40 0x7fd2b80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fd2b80c2a40 0x7fd2b80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46528/0 (socket says 172.21.15.82:46528) 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 -- 172.21.15.82:0/1592706593 learned_addr learned my addr 172.21.15.82:0/1592706593 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2b7fff640 1 --2- 172.21.15.82:0/1592706593 >> v2:172.21.15.196:3300/0 conn(0x7fd2b8161c80 0x7fd2b80b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 -- 172.21.15.82:0/1592706593 >> v2:172.21.15.196:3300/0 conn(0x7fd2b8161c80 msgr2=0x7fd2b80b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 --2- 172.21.15.82:0/1592706593 >> v2:172.21.15.196:3300/0 conn(0x7fd2b8161c80 0x7fd2b80b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.478 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 -- 172.21.15.82:0/1592706593 >> v1:172.21.15.82:6789/0 conn(0x7fd2b80c9f70 legacy=0x7fd2b80c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.479 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 -- 172.21.15.82:0/1592706593 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80caa20 con 0x7fd2b80c2a40 2024-04-25T10:14:20.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2bcf91640 1 --2- 172.21.15.82:0/1592706593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80ccf30 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80cf790 tx=0x7fd2b001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b682e8c1bf1731 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2b77fe640 1 -- 172.21.15.82:0/1592706593 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00a5880 con 0x7fd2b80c2a40 2024-04-25T10:14:20.480 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2b77fe640 1 -- 172.21.15.82:0/1592706593 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b00a5a20 con 0x7fd2b80c2a40 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.478+0000 7fd2b77fe640 1 -- 172.21.15.82:0/1592706593 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b00a5df0 con 0x7fd2b80c2a40 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/1592706593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c2a40 msgr2=0x7fd2b80ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/1592706593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80ccf30 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80cf790 tx=0x7fd2b001c9c0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/1592706593 shutdown_connections 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/1592706593 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80ccf30 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/1592706593 >> v2:172.21.15.196:3300/0 conn(0x7fd2b8161c80 0x7fd2b80b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/1592706593 >> 172.21.15.82:0/1592706593 conn(0x7fd2b806ae60 msgr2=0x7fd2b806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/1592706593 shutdown_connections 2024-04-25T10:14:20.481 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/1592706593 wait complete. 2024-04-25T10:14:20.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 Processor -- start 2024-04-25T10:14:20.482 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- start start 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2b8161c80 0x7fd2b8159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80cdc80 con 0x7fd2b80c9f70 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80b3be0 con 0x7fd2b80c2a40 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80cde00 con 0x7fd2b8161c80 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46544/0 (socket says 172.21.15.82:46544) 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 -- 172.21.15.82:0/212821404 learned_addr learned my addr 172.21.15.82:0/212821404 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:20.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 -- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2b8161c80 msgr2=0x7fd2b8159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2b8161c80 0x7fd2b8159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 -- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b80c2a40 msgr2=0x7fd2b80c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2b7fff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 -- 172.21.15.82:0/212821404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b815a820 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2affff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c420 tx=0x7fd2a000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00052d0 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0005470 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b815ab00 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0007260 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b815b550 con 0x7fd2b80c9f70 2024-04-25T10:14:20.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.482+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b804efa0 con 0x7fd2b80c9f70 2024-04-25T10:14:20.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.486+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2a0007400 con 0x7fd2b80c9f70 2024-04-25T10:14:20.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.486+0000 7fd2b57fa640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 0x7fd298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.486+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd2a00846e0 con 0x7fd2b80c9f70 2024-04-25T10:14:20.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.486+0000 7fd2b7fff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 0x7fd298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.486 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.486+0000 7fd2b7fff640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 0x7fd298069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80096f0 tx=0x7fd2a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.489 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.490+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2a0055540 con 0x7fd2b80c9f70 2024-04-25T10:14:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2b80ce620 con 0x7fd2b80c9f70 2024-04-25T10:14:20.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2b57fa640 1 -- 172.21.15.82:0/212821404 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2a0054e40 con 0x7fd2b80c9f70 2024-04-25T10:14:20.642 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 32 2024-04-25T10:14:20.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 msgr2=0x7fd298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 0x7fd298069420 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80096f0 tx=0x7fd2a8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:20.644 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 msgr2=0x7fd2b80c2e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000c420 tx=0x7fd2a000c8e0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 shutdown_connections 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fd298066f60 0x7fd298069420 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fd2b8161c80 0x7fd2b8159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fd2b80c9f70 0x7fd2b80c2e20 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 --2- 172.21.15.82:0/212821404 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fd2b80c2a40 0x7fd2b80c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 >> 172.21.15.82:0/212821404 conn(0x7fd2b806ae60 msgr2=0x7fd2b80aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 shutdown_connections 2024-04-25T10:14:20.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.642+0000 7fd2be9f4640 1 -- 172.21.15.82:0/212821404 wait complete. 2024-04-25T10:14:20.659 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:20.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.766+0000 7f05b7c7d640 1 Processor -- start 2024-04-25T10:14:20.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.766+0000 7f05b7c7d640 1 -- start start 2024-04-25T10:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b7c7d640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f05b015ecb0 0x7f05b00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b7c7d640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b00ca4c0 con 0x7f05b00c9ef0 2024-04-25T10:14:20.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b7c7d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b00cd690 con 0x7f05b00cf590 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b7c7d640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b00ca780 con 0x7f05b015ecb0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b621a640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f05b00c9ef0 0x7f05b00ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60792/0 (socket says 172.21.15.82:60792) 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b621a640 1 -- 172.21.15.82:0/3714184015 learned_addr learned my addr 172.21.15.82:0/3714184015 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348646721 0 0) 0x7f05b00ca4c0 con 0x7f05b00c9ef0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5218640 1 --2- 172.21.15.82:0/3714184015 >> v2:172.21.15.196:3300/0 conn(0x7f05b015ecb0 0x7f05b00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f059c003530 con 0x7f05b00c9ef0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965290787 0 0) 0x7f059c003530 con 0x7f05b00c9ef0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 >> v2:172.21.15.196:3300/0 conn(0x7f05b015ecb0 msgr2=0x7f05b00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 --2- 172.21.15.82:0/3714184015 >> v2:172.21.15.196:3300/0 conn(0x7f05b015ecb0 0x7f05b00c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 >> v1:172.21.15.196:6789/0 conn(0x7f05b00cf590 legacy=0x7f05b00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00cd970 con 0x7f05b00c9ef0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f05a8001a00 con 0x7f05b00c9ef0 2024-04-25T10:14:20.772 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 >> v1:172.21.15.82:6789/0 conn(0x7f05b00c9ef0 legacy=0x7f05b00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f059c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059c004810 0x7f059c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059c0051d0 0x7f059c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c00c0d0 con 0x7f059c003a30 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c00c280 con 0x7f059c004810 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c00c480 con 0x7f059c0051d0 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f059c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 -- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059c0051d0 msgr2=0x7f059c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5218640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059c004810 0x7f059c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b621a640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059c0051d0 0x7f059c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f059c0051d0 0x7f059c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 -- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059c004810 msgr2=0x7f059c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f059c004810 0x7f059c004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.773 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b5a19640 1 -- 172.21.15.82:0/3714184015 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c003530 con 0x7f059c003a30 2024-04-25T10:14:20.774 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.770+0000 7f05b621a640 1 -- 172.21.15.82:0/3714184015 reap_dead start 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f059c003e50 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f05ac001780 tx=0x7f05ac01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a974e991d20b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0013a0 con 0x7f059c003a30 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0052f0 con 0x7f059c003a30 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b4a17640 1 -- 172.21.15.82:0/3714184015 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac003840 con 0x7f059c003a30 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 msgr2=0x7f059c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f059c003e50 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f05ac001780 tx=0x7f05ac01c400 comp rx=0 tx=0).stop 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/3714184015 shutdown_connections 2024-04-25T10:14:20.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/3714184015 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f059c003e50 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/3714184015 >> 172.21.15.82:0/3714184015 conn(0x7f05b006ae60 msgr2=0x7f05b006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/3714184015 shutdown_connections 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/3714184015 wait complete. 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 Processor -- start 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- start start 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b00c5250 0x7f05b01f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 0x7f05b01f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00b2d50 con 0x7f059c003a30 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00b2bd0 con 0x7f05b01f9110 2024-04-25T10:14:20.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00b2ed0 con 0x7f05b00c5250 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46550/0 (socket says 172.21.15.82:46550) 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 -- 172.21.15.82:0/110701237 learned_addr learned my addr 172.21.15.82:0/110701237 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b621a640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 0x7f05b01f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5218640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b00c5250 0x7f05b01f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 -- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b00c5250 msgr2=0x7f05b01f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b00c5250 0x7f05b01f8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 -- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 msgr2=0x7f05b01f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 0x7f05b01f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 -- 172.21.15.82:0/110701237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b0153970 con 0x7f059c003a30 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b621a640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 0x7f05b01f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b5a19640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c10 tx=0x7f059c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.779 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0b8020 con 0x7f059c003a30 2024-04-25T10:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac0054a0 con 0x7f059c003a30 2024-04-25T10:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b01f6390 con 0x7f059c003a30 2024-04-25T10:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac007a40 con 0x7f059c003a30 2024-04-25T10:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.774+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b0153c50 con 0x7f059c003a30 2024-04-25T10:14:20.780 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b004efa0 con 0x7f059c003a30 2024-04-25T10:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f05ac0bb020 con 0x7f059c003a30 2024-04-25T10:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05a67fc640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 0x7f0594069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f05ac11f4f0 con 0x7f059c003a30 2024-04-25T10:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05b5218640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 0x7f0594069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:20.781 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.778+0000 7f05b5218640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 0x7f0594069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05b00724b0 tx=0x7f05a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:20.782 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.782+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f05ac0c2030 con 0x7f059c003a30 2024-04-25T10:14:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.926+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f05b00c2730 con 0x7f059c003a30 2024-04-25T10:14:20.927 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.926+0000 7f05a67fc640 1 -- 172.21.15.82:0/110701237 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f05ac016110 con 0x7f059c003a30 2024-04-25T10:14:20.928 INFO:teuthology.orchestra.run.smithi082.stdout:pg_num: 64 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 msgr2=0x7f0594069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 0x7f0594069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f05b00724b0 tx=0x7f05a0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 msgr2=0x7f05b01f8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000c10 tx=0x7f059c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 shutdown_connections 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0594066f90 0x7f0594069450 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.930 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f05b00c5250 0x7f05b01f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f05b01f9110 0x7f05b01f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 --2- 172.21.15.82:0/110701237 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f059c003a30 0x7f05b01f8690 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 >> 172.21.15.82:0/110701237 conn(0x7f05b006ae60 msgr2=0x7f05b0151e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 shutdown_connections 2024-04-25T10:14:20.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:20.930+0000 7f05b7c7d640 1 -- 172.21.15.82:0/110701237 wait complete. 2024-04-25T10:14:20.945 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:21.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.062+0000 7fb9b7384640 1 Processor -- start 2024-04-25T10:14:21.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.062+0000 7fb9b7384640 1 -- start start 2024-04-25T10:14:21.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b7384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb9b006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b7384640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b00b2090 con 0x7fb9b00b3f50 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b7384640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b00b43e0 con 0x7fb9b015a700 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b7384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b46c0 con 0x7fb9b015ecc0 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b491f640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb9b00b3f50 0x7fb9b00b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60800/0 (socket says 172.21.15.82:60800) 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b491f640 1 -- 172.21.15.82:0/1498248740 learned_addr learned my addr 172.21.15.82:0/1498248740 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb9b006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019056937 0 0) 0x7fb9b00b2090 con 0x7fb9b00b3f50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb99c003530 con 0x7fb9b00b3f50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504606152 0 0) 0x7fb99c003530 con 0x7fb9b00b3f50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 msgr2=0x7fb9b006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb9b006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 >> v1:172.21.15.196:6789/0 conn(0x7fb9b015a700 legacy=0x7fb9b00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00cd6f0 con 0x7fb9b00b3f50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb9a00016a0 con 0x7fb9b00b3f50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 >> v1:172.21.15.82:6789/0 conn(0x7fb9b00b3f50 legacy=0x7fb9b00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb99c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c003fe0 0x7fb99c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb99c004d50 0x7fb99c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b46c0 con 0x7fb99c004d50 2024-04-25T10:14:21.068 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c00bc60 con 0x7fb99c003fe0 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c00be60 con 0x7fb9b015ecc0 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb99c004d50 0x7fb99c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b491f640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c003fe0 0x7fb99c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5120640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb99c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 -- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 msgr2=0x7fb99c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b015ecc0 0x7fb99c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 -- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c003fe0 msgr2=0x7fb99c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c003fe0 0x7fb99c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.069 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.066+0000 7fb9b5921640 1 -- 172.21.15.82:0/1498248740 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c003530 con 0x7fb99c004d50 2024-04-25T10:14:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b5921640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb99c004d50 0x7fb99c0051d0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009fc0 tx=0x7fb9a801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b59cee43d0cf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a80b2070 con 0x7fb99c004d50 2024-04-25T10:14:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a80a5ba0 con 0x7fb99c004d50 2024-04-25T10:14:21.070 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9a7fff640 1 -- 172.21.15.82:0/1498248740 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a8009c10 con 0x7fb99c004d50 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- 172.21.15.82:0/1498248740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb99c004d50 msgr2=0x7fb99c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 --2- 172.21.15.82:0/1498248740 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb99c004d50 0x7fb99c0051d0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8009fc0 tx=0x7fb9a801cc50 comp rx=0 tx=0).stop 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b5921640 1 -- 172.21.15.82:0/1498248740 reap_dead start 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- 172.21.15.82:0/1498248740 shutdown_connections 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- 172.21.15.82:0/1498248740 >> 172.21.15.82:0/1498248740 conn(0x7fb9b00b11a0 msgr2=0x7fb9b00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- 172.21.15.82:0/1498248740 shutdown_connections 2024-04-25T10:14:21.071 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- 172.21.15.82:0/1498248740 wait complete. 2024-04-25T10:14:21.073 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 Processor -- start 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- start start 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 0x7fb9b014b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 0x7fb9b015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015f220 con 0x7fb9b01501e0 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b015f0a0 con 0x7fb99c004d50 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b7384640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00cd310 con 0x7fb9b014b9a0 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b491f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b491f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46564/0 (socket says 172.21.15.82:46564) 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b491f640 1 -- 172.21.15.82:0/2347486068 learned_addr learned my addr 172.21.15.82:0/2347486068 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b5120640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 0x7fb9b014b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.074 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.070+0000 7fb9b5921640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 0x7fb9b015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 -- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 msgr2=0x7fb9b015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 0x7fb9b015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 -- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 msgr2=0x7fb9b014b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 0x7fb9b014b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 -- 172.21.15.82:0/2347486068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b014dd50 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b5120640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 0x7fb9b014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b5921640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 0x7fb9b015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b491f640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000e800 tx=0x7fb9a000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a000c8d0 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a0018070 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b01533e0 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a000ccb0 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01539a0 con 0x7fb9b01501e0 2024-04-25T10:14:21.075 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b004efa0 con 0x7fb9b01501e0 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9a0013420 con 0x7fb9b01501e0 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 0x7fb994069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b5120640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 0x7fb994069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb9a0084e30 con 0x7fb9b01501e0 2024-04-25T10:14:21.076 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.074+0000 7fb9b5120640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 0x7fb994069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0016e0 tx=0x7fb9ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.079 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.078+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9a0055cc0 con 0x7fb9b01501e0 2024-04-25T10:14:21.259 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.258+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9b00aeb50 con 0x7fb9b01501e0 2024-04-25T10:14:21.261 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.258+0000 7fb9a5ffb640 1 -- 172.21.15.82:0/2347486068 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fb9a00555c0 con 0x7fb9b01501e0 2024-04-25T10:14:21.262 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:21.262 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:11.226477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 104 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 msgr2=0x7fb994069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 0x7fb994069450 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac0016e0 tx=0x7fb9ac009040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 msgr2=0x7fb9b015c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fb9a000e800 tx=0x7fb9a000ecc0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 shutdown_connections 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb994066f90 0x7fb994069450 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb9b014b9a0 0x7fb9b015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.263 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb9b01501e0 0x7fb9b015c360 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 --2- 172.21.15.82:0/2347486068 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb99c004d50 0x7fb9b014b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 >> 172.21.15.82:0/2347486068 conn(0x7fb9b00b11a0 msgr2=0x7fb9b0155790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 shutdown_connections 2024-04-25T10:14:21.264 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.262+0000 7fb9b7384640 1 -- 172.21.15.82:0/2347486068 wait complete. 2024-04-25T10:14:21.280 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 Processor -- start 2024-04-25T10:14:21.391 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 -- start start 2024-04-25T10:14:21.392 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c7c0cf420 0x7f3c7c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c7c15bd30 con 0x7f3c7c0cfb80 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c7c0cc0d0 con 0x7f3c7c0cf420 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c827e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c09ab10 con 0x7f3c7c15b950 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c7b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c7c0cf420 0x7f3c7c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c7b7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f3c7c0cf420 0x7f3c7c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46576/0 (socket says 172.21.15.82:46576) 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c7b7fe640 1 -- 172.21.15.82:0/2414882950 learned_addr learned my addr 172.21.15.82:0/2414882950 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.390+0000 7f3c7b7fe640 1 -- 172.21.15.82:0/2414882950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 msgr2=0x7f3c7c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7b7fe640 1 --2- 172.21.15.82:0/2414882950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7b7fe640 1 -- 172.21.15.82:0/2414882950 >> v1:172.21.15.196:6789/0 conn(0x7f3c7c0cfb80 legacy=0x7f3c7c06f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.393 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7b7fe640 1 -- 172.21.15.82:0/2414882950 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0cc370 con 0x7f3c7c0cf420 2024-04-25T10:14:21.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7b7fe640 1 --2- 172.21.15.82:0/2414882950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c0cf420 0x7f3c7c0cce50 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009870 tx=0x7f3c7001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0e7d151048b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7affd640 1 -- 172.21.15.82:0/2414882950 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700b1070 con 0x7f3c7c0cf420 2024-04-25T10:14:21.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7affd640 1 -- 172.21.15.82:0/2414882950 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c700ac620 con 0x7f3c7c0cf420 2024-04-25T10:14:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c7affd640 1 -- 172.21.15.82:0/2414882950 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700ac9f0 con 0x7f3c7c0cf420 2024-04-25T10:14:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 -- 172.21.15.82:0/2414882950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c0cf420 msgr2=0x7f3c7c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 --2- 172.21.15.82:0/2414882950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c0cf420 0x7f3c7c0cce50 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009870 tx=0x7f3c7001c4f0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 -- 172.21.15.82:0/2414882950 shutdown_connections 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 --2- 172.21.15.82:0/2414882950 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c0cf420 0x7f3c7c0cce50 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 --2- 172.21.15.82:0/2414882950 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 -- 172.21.15.82:0/2414882950 >> 172.21.15.82:0/2414882950 conn(0x7f3c7c0b30b0 msgr2=0x7f3c7c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 -- 172.21.15.82:0/2414882950 shutdown_connections 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 -- 172.21.15.82:0/2414882950 wait complete. 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.394+0000 7f3c827e3640 1 Processor -- start 2024-04-25T10:14:21.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- start start 2024-04-25T10:14:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c7c0cfb80 0x7f3c7c0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0cd390 con 0x7f3c7c165570 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c0cf880 con 0x7f3c7c0cfb80 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c164ab0 con 0x7f3c7c15b950 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46590/0 (socket says 172.21.15.82:46590) 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 -- 172.21.15.82:0/1974756837 learned_addr learned my addr 172.21.15.82:0/1974756837 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c7b7fe640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c7bfff640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c7c0cfb80 0x7f3c7c0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 -- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 msgr2=0x7f3c7c0aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 -- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c7c0cfb80 msgr2=0x7f3c7c0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c7c0cfb80 0x7f3c7c0ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 -- 172.21.15.82:0/1974756837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c0cd130 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c80d80640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b970 tx=0x7f3c7400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74019070 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7400cb40 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c7c099cd0 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74014440 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c7c164d70 con 0x7f3c7c165570 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c7c04efa0 con 0x7f3c7c165570 2024-04-25T10:14:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c7401c070 con 0x7f3c7c165570 2024-04-25T10:14:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f3c74084e90 con 0x7f3c7c165570 2024-04-25T10:14:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c7bfff640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.401 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.398+0000 7f3c7bfff640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00a790 tx=0x7f3c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.402+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c74055c60 con 0x7f3c7c165570 2024-04-25T10:14:21.563 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.562+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c7c0cef70 con 0x7f3c7c165570 2024-04-25T10:14:21.566 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.566+0000 7f3c78ff9640 1 -- 172.21.15.82:0/1974756837 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17221 (secure 0 0 0) 0x7f3c74055560 con 0x7f3c7c165570 2024-04-25T10:14:21.566 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:21.566 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":123,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:14:07.178089+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:21.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 msgr2=0x7f3c68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 0x7f3c68069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f3c6c00a790 tx=0x7f3c6c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.569 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 msgr2=0x7f3c7c06bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400b970 tx=0x7f3c7400be30 comp rx=0 tx=0).stop 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 shutdown_connections 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f3c68066f60 0x7f3c68069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f3c7c15b950 0x7f3c7c0aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f3c7c165570 0x7f3c7c06bda0 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 --2- 172.21.15.82:0/1974756837 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f3c7c0cfb80 0x7f3c7c0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 >> 172.21.15.82:0/1974756837 conn(0x7f3c7c0b30b0 msgr2=0x7f3c7c06f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 shutdown_connections 2024-04-25T10:14:21.570 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.570+0000 7f3c827e3640 1 -- 172.21.15.82:0/1974756837 wait complete. 2024-04-25T10:14:21.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:21.586 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.698+0000 7fb4163d7640 1 Processor -- start 2024-04-25T10:14:21.700 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.698+0000 7fb4163d7640 1 -- start start 2024-04-25T10:14:21.702 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb4163d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 0x7fb410142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb4163d7640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100ccc70 con 0x7fb4100ce510 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb4163d7640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41006dc40 con 0x7fb410144ca0 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb4163d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100cc2b0 con 0x7fb4101451f0 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fb4100ce510 0x7fb4100ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60816/0 (socket says 172.21.15.82:60816) 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 -- 172.21.15.82:0/1502784738 learned_addr learned my addr 172.21.15.82:0/1502784738 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828810222 0 0) 0x7fb4100ccc70 con 0x7fb4100ce510 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f8003530 con 0x7fb4100ce510 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb414974640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 0x7fb410142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2753313828 0 0) 0x7fb3f8003530 con 0x7fb4100ce510 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 msgr2=0x7fb410142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 0x7fb410142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 >> v1:172.21.15.196:6789/0 conn(0x7fb410144ca0 legacy=0x7fb41006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41006df20 con 0x7fb4100ce510 2024-04-25T10:14:21.703 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fb400001680 con 0x7fb4100ce510 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 >> v1:172.21.15.82:6789/0 conn(0x7fb4100ce510 legacy=0x7fb4100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb3f8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 0x7fb3f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3f8004da0 0x7fb3f8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100cc2b0 con 0x7fb3f8003a60 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f800bc50 con 0x7fb3f8004da0 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3f800be50 con 0x7fb4101451f0 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb3f8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 -- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 msgr2=0x7fb3f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4101451f0 0x7fb3f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb414974640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3f8004da0 0x7fb3f8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 -- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3f8004da0 msgr2=0x7fb3f8005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb3f8004da0 0x7fb3f8005220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.704 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 -- 172.21.15.82:0/1502784738 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f8003530 con 0x7fb3f8003a60 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb3f8003e80 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c210 tx=0x7fb40001f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b959c76886bedc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400007d40 con 0x7fb3f8003a60 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400005b00 con 0x7fb3f8003a60 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.702+0000 7fb40effd640 1 -- 172.21.15.82:0/1502784738 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400007070 con 0x7fb3f8003a60 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1502784738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 msgr2=0x7fb3f8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1502784738 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb3f8003e80 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fb40000c210 tx=0x7fb40001f3f0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb414974640 1 -- 172.21.15.82:0/1502784738 reap_dead start 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1502784738 shutdown_connections 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1502784738 >> 172.21.15.82:0/1502784738 conn(0x7fb4100b11a0 msgr2=0x7fb4100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1502784738 shutdown_connections 2024-04-25T10:14:21.706 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1502784738 wait complete. 2024-04-25T10:14:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 Processor -- start 2024-04-25T10:14:21.707 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- start start 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 0x7fb4100c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4100c85c0 0x7fb4101f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100cc900 con 0x7fb3f8003a60 2024-04-25T10:14:21.709 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41006e290 con 0x7fb4100c85c0 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb410142680 con 0x7fb4100c5710 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46608/0 (socket says 172.21.15.82:46608) 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 -- 172.21.15.82:0/1577769789 learned_addr learned my addr 172.21.15.82:0/1577769789 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb414974640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4100c85c0 0x7fb4101f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40f7fe640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 0x7fb4100c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 -- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 msgr2=0x7fb4100c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 0x7fb4100c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 -- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4100c85c0 msgr2=0x7fb4101f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4100c85c0 0x7fb4101f4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 -- 172.21.15.82:0/1577769789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100c7e30 con 0x7fb3f8003a60 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40f7fe640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 0x7fb4100c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40ffff640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fb400001980 tx=0x7fb3f8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.710 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4000047f0 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb400004990 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4101f5b10 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb400004d60 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.706+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4101e43f0 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41004efa0 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb400007070 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb40cff9640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fb400122850 con 0x7fb3f8003a60 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb40f7fe640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:21.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.710+0000 7fb40f7fe640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 0x7fb3fc069450 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb41006ebe0 tx=0x7fb404009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:21.713 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.714+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb400019100 con 0x7fb3f8003a60 2024-04-25T10:14:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.894+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4100c89a0 con 0x7fb3f8003a60 2024-04-25T10:14:21.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.894+0000 7fb40cff9640 1 -- 172.21.15.82:0/1577769789 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fb4000f3600 con 0x7fb3f8003a60 2024-04-25T10:14:21.895 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:21.895 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:11.226477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:21.895 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 104 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.894+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 msgr2=0x7fb3fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.894+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 0x7fb3fc069450 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb41006ebe0 tx=0x7fb404009040 comp rx=0 tx=0).stop 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.894+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 msgr2=0x7fb41013ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7fb400001980 tx=0x7fb3f8005840 comp rx=0 tx=0).stop 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 shutdown_connections 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fb3fc066f90 0x7fb3fc069450 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fb4100c5710 0x7fb4100c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.897 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fb4100c85c0 0x7fb4101f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 --2- 172.21.15.82:0/1577769789 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fb3f8003a60 0x7fb41013ffa0 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:21.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 >> 172.21.15.82:0/1577769789 conn(0x7fb4100b11a0 msgr2=0x7fb4100c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:21.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 shutdown_connections 2024-04-25T10:14:21.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:21.898+0000 7fb4163d7640 1 -- 172.21.15.82:0/1577769789 wait complete. 2024-04-25T10:14:21.912 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:22.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 Processor -- start 2024-04-25T10:14:22.019 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 -- start start 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcb600ce510 0x7fcb600ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcb60144ca0 0x7fcb6006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb600cee30 con 0x7fcb6006ec60 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600cc2b0 con 0x7fcb60144ca0 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb68023640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6006e290 con 0x7fcb600ce510 2024-04-25T10:14:22.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcb60144ca0 0x7fcb6006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcb60144ca0 0x7fcb6006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46610/0 (socket says 172.21.15.82:46610) 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 -- 172.21.15.82:0/2983709519 learned_addr learned my addr 172.21.15.82:0/2983709519 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 -- 172.21.15.82:0/2983709519 >> v2:172.21.15.196:3300/0 conn(0x7fcb600ce510 msgr2=0x7fcb600ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb65dbf640 1 --2- 172.21.15.82:0/2983709519 >> v2:172.21.15.196:3300/0 conn(0x7fcb600ce510 0x7fcb600ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 --2- 172.21.15.82:0/2983709519 >> v2:172.21.15.196:3300/0 conn(0x7fcb600ce510 0x7fcb600ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 -- 172.21.15.82:0/2983709519 >> v1:172.21.15.196:6789/0 conn(0x7fcb6006ec60 legacy=0x7fcb60142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.018+0000 7fcb655be640 1 -- 172.21.15.82:0/2983709519 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600ccc70 con 0x7fcb60144ca0 2024-04-25T10:14:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb655be640 1 --2- 172.21.15.82:0/2983709519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb60144ca0 0x7fcb6006e720 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fcb60144c50 tx=0x7fcb5001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30605245462f578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb64dbd640 1 -- 172.21.15.82:0/2983709519 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500b5070 con 0x7fcb60144ca0 2024-04-25T10:14:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb64dbd640 1 -- 172.21.15.82:0/2983709519 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb500a89d0 con 0x7fcb60144ca0 2024-04-25T10:14:22.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb64dbd640 1 -- 172.21.15.82:0/2983709519 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500a8da0 con 0x7fcb60144ca0 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- 172.21.15.82:0/2983709519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb60144ca0 msgr2=0x7fcb6006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 --2- 172.21.15.82:0/2983709519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb60144ca0 0x7fcb6006e720 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fcb60144c50 tx=0x7fcb5001ecd0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- 172.21.15.82:0/2983709519 shutdown_connections 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 --2- 172.21.15.82:0/2983709519 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb60144ca0 0x7fcb6006e720 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 --2- 172.21.15.82:0/2983709519 >> v2:172.21.15.196:3300/0 conn(0x7fcb600ce510 0x7fcb600ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- 172.21.15.82:0/2983709519 >> 172.21.15.82:0/2983709519 conn(0x7fcb600b11a0 msgr2=0x7fcb600b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- 172.21.15.82:0/2983709519 shutdown_connections 2024-04-25T10:14:22.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- 172.21.15.82:0/2983709519 wait complete. 2024-04-25T10:14:22.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 Processor -- start 2024-04-25T10:14:22.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.022+0000 7fcb68023640 1 -- start start 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb600ce510 0x7fcb600a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb60144ca0 0x7fcb600ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600cc900 con 0x7fcb6006ec60 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6006df20 con 0x7fcb60144ca0 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60142d90 con 0x7fcb600ce510 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46620/0 (socket says 172.21.15.82:46620) 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 -- 172.21.15.82:0/2318682162 learned_addr learned my addr 172.21.15.82:0/2318682162 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 -- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb600ce510 msgr2=0x7fcb600a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb600ce510 0x7fcb600a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 -- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb60144ca0 msgr2=0x7fcb600ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb60144ca0 0x7fcb600ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 -- 172.21.15.82:0/2318682162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600a8a10 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb65dbf640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c7f0 tx=0x7fcb5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c015070 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5c004240 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600a8c90 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5c003620 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600a9250 con 0x7fcb6006ec60 2024-04-25T10:14:22.027 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6004efa0 con 0x7fcb6006ec60 2024-04-25T10:14:22.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcb5c0039d0 con 0x7fcb6006ec60 2024-04-25T10:14:22.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 0x7fcb44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fcb5c0852c0 con 0x7fcb6006ec60 2024-04-25T10:14:22.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb655be640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 0x7fcb44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.026+0000 7fcb655be640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 0x7fcb44069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcb500b2000 tx=0x7fcb50004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.030+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcb600a8a10 con 0x7fcb6006ec60 2024-04-25T10:14:22.207 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.206+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb600cbac0 con 0x7fcb6006ec60 2024-04-25T10:14:22.208 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.206+0000 7fcb567fc640 1 -- 172.21.15.82:0/2318682162 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4954 (secure 0 0 0) 0x7fcb5c056120 con 0x7fcb6006ec60 2024-04-25T10:14:22.208 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:22.208 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":104,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:21.485141+0000","modified":"2024-04-25T10:14:11.226477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":123,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10068":{"gid":10068,"name":"c","rank":0,"incarnation":101,"state":"up:active","state_seq":4,"addr":"172.21.15.82:6837/2659325772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":2659325772},{"type":"v1","addr":"172.21.15.82:6837","nonce":2659325772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:22.209 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 104 2024-04-25T10:14:22.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 msgr2=0x7fcb44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 0x7fcb44069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcb500b2000 tx=0x7fcb50004740 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 msgr2=0x7fcb600ae490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c00c7f0 tx=0x7fcb5c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 shutdown_connections 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcb44066f60 0x7fcb44069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcb600ce510 0x7fcb600a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcb60144ca0 0x7fcb600ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 --2- 172.21.15.82:0/2318682162 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcb6006ec60 0x7fcb600ae490 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 >> 172.21.15.82:0/2318682162 conn(0x7fcb600b11a0 msgr2=0x7fcb600b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 shutdown_connections 2024-04-25T10:14:22.211 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.210+0000 7fcb68023640 1 -- 172.21.15.82:0/2318682162 wait complete. 2024-04-25T10:14:22.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:22.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 Processor -- start 2024-04-25T10:14:22.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 -- start start 2024-04-25T10:14:22.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 0x7fe1440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe144144ca0 0x7fe14406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1440cee30 con 0x7fe14406ec60 2024-04-25T10:14:22.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1440cc2b0 con 0x7fe144144ca0 2024-04-25T10:14:22.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14c168640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406e290 con 0x7fe1440ce510 2024-04-25T10:14:22.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe144144ca0 0x7fe14406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149f04640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 0x7fe1440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe144144ca0 0x7fe14406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:39562/0 (socket says 172.21.15.82:39562) 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe14a705640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fe14406ec60 0x7fe144142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:56316/0 (socket says 172.21.15.82:56316) 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 -- 172.21.15.82:0/977341058 learned_addr learned my addr 172.21.15.82:0/977341058 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 -- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 msgr2=0x7fe1440ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 0x7fe1440ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 -- 172.21.15.82:0/977341058 >> v1:172.21.15.196:6789/0 conn(0x7fe14406ec60 legacy=0x7fe144142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149703640 1 -- 172.21.15.82:0/977341058 --> v2:172.21.15.196:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440ccc70 con 0x7fe144144ca0 2024-04-25T10:14:22.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.334+0000 7fe149f04640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 0x7fe1440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:22.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe149703640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144144ca0 0x7fe14406e720 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe134009870 tx=0x7fe13401ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5c1fa01cd0518cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe148f02640 1 -- 172.21.15.82:0/977341058 <== mon.1 v2:172.21.15.196:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1340b5070 con 0x7fe144144ca0 2024-04-25T10:14:22.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe148f02640 1 -- 172.21.15.82:0/977341058 <== mon.1 v2:172.21.15.196:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1340a8a50 con 0x7fe144144ca0 2024-04-25T10:14:22.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe148f02640 1 -- 172.21.15.82:0/977341058 <== mon.1 v2:172.21.15.196:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1340a8e20 con 0x7fe144144ca0 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 -- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144144ca0 msgr2=0x7fe14406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144144ca0 0x7fe14406e720 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe134009870 tx=0x7fe13401ed50 comp rx=0 tx=0).stop 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 -- 172.21.15.82:0/977341058 shutdown_connections 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe144144ca0 0x7fe14406e720 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 --2- 172.21.15.82:0/977341058 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe1440ce510 0x7fe1440ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 -- 172.21.15.82:0/977341058 >> 172.21.15.82:0/977341058 conn(0x7fe1440b11a0 msgr2=0x7fe1440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 -- 172.21.15.82:0/977341058 shutdown_connections 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.338+0000 7fe14c168640 1 -- 172.21.15.82:0/977341058 wait complete. 2024-04-25T10:14:22.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 Processor -- start 2024-04-25T10:14:22.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- start start 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe14406ec60 0x7fe1440b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1440ce510 0x7fe1440cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1440cc900 con 0x7fe144144ca0 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14406df20 con 0x7fe1440ce510 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe144142d90 con 0x7fe14406ec60 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46622/0 (socket says 172.21.15.82:46622) 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 -- 172.21.15.82:0/608904445 learned_addr learned my addr 172.21.15.82:0/608904445 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe149703640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1440ce510 0x7fe1440cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe149f04640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe14406ec60 0x7fe1440b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 -- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe14406ec60 msgr2=0x7fe1440b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe14406ec60 0x7fe1440b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 -- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1440ce510 msgr2=0x7fe1440cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1440ce510 0x7fe1440cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 -- 172.21.15.82:0/608904445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1440b3fa0 con 0x7fe144144ca0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14a705640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c7d0 tx=0x7fe13c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c017070 con 0x7fe144144ca0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c00a640 con 0x7fe144144ca0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c00a9f0 con 0x7fe144144ca0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1440b3700 con 0x7fe144144ca0 2024-04-25T10:14:22.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1440b3cc0 con 0x7fe144144ca0 2024-04-25T10:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14404efa0 con 0x7fe144144ca0 2024-04-25T10:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.342+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe13c01a070 con 0x7fe144144ca0 2024-04-25T10:14:22.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.346+0000 7fe13a7fc640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 0x7fe128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.346+0000 7fe149f04640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 0x7fe128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.346+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fe13c0847f0 con 0x7fe144144ca0 2024-04-25T10:14:22.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.346+0000 7fe149f04640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 0x7fe128069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe140009580 tx=0x7fe140009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.348 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.350+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe13c054f70 con 0x7fe144144ca0 2024-04-25T10:14:22.504 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.502+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe144142460 con 0x7fe144144ca0 2024-04-25T10:14:22.506 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.506+0000 7fe13a7fc640 1 -- 172.21.15.82:0/608904445 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17221 (secure 0 0 0) 0x7fe13c059010 con 0x7fe144144ca0 2024-04-25T10:14:22.507 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:22.507 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":123,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:14:07.178089+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:19.785837+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:20.465294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6837/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 msgr2=0x7fe128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 0x7fe128069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fe140009580 tx=0x7fe140009190 comp rx=0 tx=0).stop 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 msgr2=0x7fe14413fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00c7d0 tx=0x7fe13c00cc90 comp rx=0 tx=0).stop 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 shutdown_connections 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe128066f60 0x7fe128069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe144144ca0 0x7fe14413fe60 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe1440ce510 0x7fe1440cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.509 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 --2- 172.21.15.82:0/608904445 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe14406ec60 0x7fe1440b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 >> 172.21.15.82:0/608904445 conn(0x7fe1440b11a0 msgr2=0x7fe1440c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 shutdown_connections 2024-04-25T10:14:22.510 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.510+0000 7fe14c168640 1 -- 172.21.15.82:0/608904445 wait complete. 2024-04-25T10:14:22.525 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 Processor -- start 2024-04-25T10:14:22.637 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 -- start start 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7f1415da80 0x7f7f140d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f140d0620 0x7f7f140ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1406c440 con 0x7f7f140d0620 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1406be60 con 0x7f7f1415da80 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f1933f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1415de60 con 0x7f7f140d1cd0 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f140d0620 0x7f7f140ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7f140d0620 0x7f7f140ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46624/0 (socket says 172.21.15.82:46624) 2024-04-25T10:14:22.639 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 -- 172.21.15.82:0/1080247053 learned_addr learned my addr 172.21.15.82:0/1080247053 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f127fc640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f12ffd640 1 --2- 172.21.15.82:0/1080247053 >> v2:172.21.15.196:3300/0 conn(0x7f7f1415da80 0x7f7f140d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 -- 172.21.15.82:0/1080247053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 msgr2=0x7f7f140d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 -- 172.21.15.82:0/1080247053 >> v2:172.21.15.196:3300/0 conn(0x7f7f1415da80 msgr2=0x7f7f140d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 --2- 172.21.15.82:0/1080247053 >> v2:172.21.15.196:3300/0 conn(0x7f7f1415da80 0x7f7f140d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.640 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.638+0000 7f7f137fe640 1 -- 172.21.15.82:0/1080247053 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140d2450 con 0x7f7f140d0620 2024-04-25T10:14:22.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f137fe640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f140ce3b0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f7f1408d0a0 tx=0x7f7f0c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3f38630423616e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f11ffb640 1 -- 172.21.15.82:0/1080247053 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0a5690 con 0x7f7f140d0620 2024-04-25T10:14:22.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f11ffb640 1 -- 172.21.15.82:0/1080247053 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0a5830 con 0x7f7f140d0620 2024-04-25T10:14:22.641 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f11ffb640 1 -- 172.21.15.82:0/1080247053 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0a5c00 con 0x7f7f140d0620 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- 172.21.15.82:0/1080247053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 msgr2=0x7f7f140ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f140ce3b0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7f7f1408d0a0 tx=0x7f7f0c01c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- 172.21.15.82:0/1080247053 shutdown_connections 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f140ce3b0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- 172.21.15.82:0/1080247053 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- 172.21.15.82:0/1080247053 >> v2:172.21.15.196:3300/0 conn(0x7f7f1415da80 0x7f7f140d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.642 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- 172.21.15.82:0/1080247053 >> 172.21.15.82:0/1080247053 conn(0x7f7f1408b000 msgr2=0x7f7f140d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- 172.21.15.82:0/1080247053 shutdown_connections 2024-04-25T10:14:22.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- 172.21.15.82:0/1080247053 wait complete. 2024-04-25T10:14:22.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 Processor -- start 2024-04-25T10:14:22.643 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- start start 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1415da80 0x7f7f140b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406ca30 con 0x7f7f140d0620 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406c8b0 con 0x7f7f1415da80 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f1933f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406cbb0 con 0x7f7f140d1cd0 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46638/0 (socket says 172.21.15.82:46638) 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 -- 172.21.15.82:0/2048324937 learned_addr learned my addr 172.21.15.82:0/2048324937 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.645 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 -- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 msgr2=0x7f7f140b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 -- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1415da80 msgr2=0x7f7f140b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1415da80 0x7f7f140b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.642+0000 7f7f12ffd640 1 -- 172.21.15.82:0/2048324937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1415ee50 con 0x7f7f140d0620 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f12ffd640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400d5f0 tx=0x7f7f0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040125f0 con 0x7f7f140d0620 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04019070 con 0x7f7f140d0620 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1415f130 con 0x7f7f140d0620 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040129b0 con 0x7f7f140d0620 2024-04-25T10:14:22.646 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1415f6f0 con 0x7f7f140d0620 2024-04-25T10:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1404efa0 con 0x7f7f140d0620 2024-04-25T10:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f04012b50 con 0x7f7f140d0620 2024-04-25T10:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 0x7f7f00069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7f040856b0 con 0x7f7f140d0620 2024-04-25T10:14:22.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f127fc640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 0x7f7f00069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.646+0000 7f7f127fc640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 0x7f7f00069450 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f08006eb0 tx=0x7f7f08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.650+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f04011340 con 0x7f7f140d0620 2024-04-25T10:14:22.827 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.826+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7f140d0380 con 0x7f7f140d0620 2024-04-25T10:14:22.828 INFO:tasks.ceph.mon.a.smithi082.stderr:2024-04-25T10:14:22.830+0000 7ff489d91640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:22.831 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.830+0000 7f7efb7fe640 1 -- 172.21.15.82:0/2048324937 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7f04056480 con 0x7f7f140d0620 2024-04-25T10:14:22.831 INFO:teuthology.orchestra.run.smithi082.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 msgr2=0x7f7f00069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 0x7f7f00069450 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f7f08006eb0 tx=0x7f7f08008040 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 msgr2=0x7f7f14156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400d5f0 tx=0x7f7f0400dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 shutdown_connections 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7f00066f90 0x7f7f00069450 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7f140d1cd0 0x7f7f140b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7f1415da80 0x7f7f140b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 --2- 172.21.15.82:0/2048324937 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7f140d0620 0x7f7f14156dd0 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 >> 172.21.15.82:0/2048324937 conn(0x7f7f1408b000 msgr2=0x7f7f140cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 shutdown_connections 2024-04-25T10:14:22.836 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.834+0000 7f7f1933f640 1 -- 172.21.15.82:0/2048324937 wait complete. 2024-04-25T10:14:22.837 INFO:tasks.ceph.mds.c.smithi082.stderr: -1461> 2024-04-25T10:14:13.198+0000 7fa0a5943640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:14:22.851 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:22.872 INFO:tasks.ceph.mds.c.smithi082.stdout:starting mds.c at 2024-04-25T10:14:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 Processor -- start 2024-04-25T10:14:22.961 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 -- start start 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 0x7fe9bc0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9bc0d04d0 con 0x7fe9bc15b950 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc15bd30 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c4781640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9bc0cc0d0 con 0x7fe9bc09a480 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46672/0 (socket says 172.21.15.82:46672) 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 -- 172.21.15.82:0/776612461 learned_addr learned my addr 172.21.15.82:0/776612461 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c1d1c640 1 --2- 172.21.15.82:0/776612461 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 0x7fe9bc0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 -- 172.21.15.82:0/776612461 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 msgr2=0x7fe9bc0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 --2- 172.21.15.82:0/776612461 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 0x7fe9bc0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 -- 172.21.15.82:0/776612461 >> v1:172.21.15.196:6789/0 conn(0x7fe9bc15b950 legacy=0x7fe9bc0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c2d1e640 1 -- 172.21.15.82:0/776612461 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc09ab10 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.963 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.962+0000 7fe9c1d1c640 1 --2- 172.21.15.82:0/776612461 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 0x7fe9bc0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:22.964 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c2d1e640 1 --2- 172.21.15.82:0/776612461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc072710 tx=0x7fe9b401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cf429750ffe217 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c151b640 1 -- 172.21.15.82:0/776612461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40a5580 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c151b640 1 -- 172.21.15.82:0/776612461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b40a5720 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.965 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c151b640 1 -- 172.21.15.82:0/776612461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40a5af0 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/776612461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 msgr2=0x7fe9bc0cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- 172.21.15.82:0/776612461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc072710 tx=0x7fe9b401c7d0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/776612461 shutdown_connections 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- 172.21.15.82:0/776612461 >> v2:172.21.15.196:3300/0 conn(0x7fe9bc09a480 0x7fe9bc0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- 172.21.15.82:0/776612461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc0cff90 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/776612461 >> 172.21.15.82:0/776612461 conn(0x7fe9bc0b30b0 msgr2=0x7fe9bc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/776612461 shutdown_connections 2024-04-25T10:14:22.966 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/776612461 wait complete. 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 Processor -- start 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- start start 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc09a480 0x7fe9bc0c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc15b950 0x7fe9bc06d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0cd390 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc06faa0 con 0x7fe9bc15b950 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9bc0c6390 con 0x7fe9bc09a480 2024-04-25T10:14:22.968 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46684/0 (socket says 172.21.15.82:46684) 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 -- 172.21.15.82:0/1176320066 learned_addr learned my addr 172.21.15.82:0/1176320066 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 -- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc09a480 msgr2=0x7fe9bc0c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc09a480 0x7fe9bc0c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 -- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc15b950 msgr2=0x7fe9bc06d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc15b950 0x7fe9bc06d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 -- 172.21.15.82:0/1176320066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc0c6670 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c1d1c640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c7c0 tx=0x7fe9ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac007a30 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac007bd0 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc0c6950 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac0113f0 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.969 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.966+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9bc0c6f10 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9bc04efa0 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9ac011590 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9b2ffd640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 0x7fe9a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:22.970 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9c251d640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 0x7fe9a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7fe9ac085500 con 0x7fe9bc0cfb80 2024-04-25T10:14:22.971 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.970+0000 7fe9c251d640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 0x7fe9a0069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009670 tx=0x7fe9b8006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:22.974 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:22.974+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9ac015020 con 0x7fe9bc0cfb80 2024-04-25T10:14:23.152 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.150+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe9bc0d13a0 con 0x7fe9bc0cfb80 2024-04-25T10:14:23.835 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.834+0000 7fe9b2ffd640 1 -- 172.21.15.82:0/1176320066 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe9ac056280 con 0x7fe9bc0cfb80 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 msgr2=0x7fe9a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 0x7fe9a0069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009670 tx=0x7fe9b8006bd0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 msgr2=0x7fe9bc154b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac00c7c0 tx=0x7fe9ac00cc80 comp rx=0 tx=0).stop 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 shutdown_connections 2024-04-25T10:14:23.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe9a0066f60 0x7fe9a0069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe9bc15b950 0x7fe9bc06d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe9bc0cfb80 0x7fe9bc154b10 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 --2- 172.21.15.82:0/1176320066 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe9bc09a480 0x7fe9bc0c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 >> 172.21.15.82:0/1176320066 conn(0x7fe9bc0b30b0 msgr2=0x7fe9bc157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 shutdown_connections 2024-04-25T10:14:23.839 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:23.838+0000 7fe9c4781640 1 -- 172.21.15.82:0/1176320066 wait complete. 2024-04-25T10:14:23.861 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:24.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 Processor -- start 2024-04-25T10:14:24.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 -- start start 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f70dc0ae6c0 0x7f70dc0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f70dc0cf510 0x7f70dc0cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc15e0f0 con 0x7f70dc15a3c0 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0cfe80 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70e16fc640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0b24a0 con 0x7f70dc0cf510 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70daffd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f70dc15a3c0 0x7f70dc15a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:60824/0 (socket says 172.21.15.82:60824) 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70daffd640 1 -- 172.21.15.82:0/1578575865 learned_addr learned my addr 172.21.15.82:0/1578575865 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70d27fc640 1 --2- 172.21.15.82:0/1578575865 >> v2:172.21.15.82:3300/0 conn(0x7f70dc0ae6c0 0x7f70dc0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70db7fe640 1 --2- 172.21.15.82:0/1578575865 >> v2:172.21.15.196:3300/0 conn(0x7f70dc0cf510 0x7f70dc0cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70da7fc640 1 -- 172.21.15.82:0/1578575865 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4019936264 0 0) 0x7f70dc15e0f0 con 0x7f70dc15a3c0 2024-04-25T10:14:24.037 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.034+0000 7f70da7fc640 1 -- 172.21.15.82:0/1578575865 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c8003530 con 0x7f70dc15a3c0 2024-04-25T10:14:24.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70d27fc640 1 -- 172.21.15.82:0/1578575865 >> v2:172.21.15.196:3300/0 conn(0x7f70dc0cf510 msgr2=0x7f70dc0cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:24.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70d27fc640 1 --2- 172.21.15.82:0/1578575865 >> v2:172.21.15.196:3300/0 conn(0x7f70dc0cf510 0x7f70dc0cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70d27fc640 1 -- 172.21.15.82:0/1578575865 >> v1:172.21.15.82:6789/0 conn(0x7f70dc15a3c0 legacy=0x7f70dc15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70d27fc640 1 -- 172.21.15.82:0/1578575865 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc15b0b0 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.038 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70d27fc640 1 --2- 172.21.15.82:0/1578575865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0b27a0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0cdd00 tx=0x7f70c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1453df90cba6296b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70da7fc640 1 -- 172.21.15.82:0/1578575865 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40b0070 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70da7fc640 1 -- 172.21.15.82:0/1578575865 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c40ab600 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70da7fc640 1 -- 172.21.15.82:0/1578575865 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40ab9b0 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 -- 172.21.15.82:0/1578575865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0ae6c0 msgr2=0x7f70dc0b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 --2- 172.21.15.82:0/1578575865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0b27a0 secure :-1 s=READY pgs=2267 cs=0 l=1 rev1=1 crypto rx=0x7f70dc0cdd00 tx=0x7f70c401c480 comp rx=0 tx=0).stop 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 -- 172.21.15.82:0/1578575865 shutdown_connections 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 --2- 172.21.15.82:0/1578575865 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0b27a0 unknown :-1 s=CLOSED pgs=2267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 --2- 172.21.15.82:0/1578575865 >> v2:172.21.15.196:3300/0 conn(0x7f70dc0cf510 0x7f70dc0cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 -- 172.21.15.82:0/1578575865 >> 172.21.15.82:0/1578575865 conn(0x7f70dc06ae60 msgr2=0x7f70dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 -- 172.21.15.82:0/1578575865 shutdown_connections 2024-04-25T10:14:24.040 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.038+0000 7f70e16fc640 1 -- 172.21.15.82:0/1578575865 wait complete. 2024-04-25T10:14:24.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 Processor -- start 2024-04-25T10:14:24.041 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- start start 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0c48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70dc15a3c0 0x7f70dc072ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0b2e60 con 0x7f70dc0cf510 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0b2ce0 con 0x7f70dc0ae6c0 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc1673f0 con 0x7f70dc15a3c0 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:46700/0 (socket says 172.21.15.82:46700) 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 -- 172.21.15.82:0/2849760097 learned_addr learned my addr 172.21.15.82:0/2849760097 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70daffd640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0c48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.043 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 -- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70dc15a3c0 msgr2=0x7f70dc072ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70db7fe640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70dc15a3c0 0x7f70dc072ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70dc15a3c0 0x7f70dc072ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 -- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc0ae6c0 msgr2=0x7f70dc0c48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0c48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 -- 172.21.15.82:0/2849760097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0c5280 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d27fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f70c401a0b0 tx=0x7f70c4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40ba070 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c40b43f0 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc070e50 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c40b0040 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc071410 con 0x7f70dc0cf510 2024-04-25T10:14:24.044 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc071650 con 0x7f70dc0cf510 2024-04-25T10:14:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.042+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70c40bd070 con 0x7f70dc0cf510 2024-04-25T10:14:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.046+0000 7f70d3fff640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 0x7f70bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.046+0000 7f70daffd640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 0x7f70bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.046+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f70c411e500 con 0x7f70dc0cf510 2024-04-25T10:14:24.045 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.046+0000 7f70daffd640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 0x7f70bc069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007fd0 tx=0x7f70cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:24.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.050+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70dc0c5280 con 0x7f70dc0cf510 2024-04-25T10:14:24.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.194+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f70dc06edf0 con 0x7f70dc0cf510 2024-04-25T10:14:24.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.846+0000 7f70d3fff640 1 -- 172.21.15.82:0/2849760097 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f70c4016070 con 0x7f70dc0cf510 2024-04-25T10:14:24.847 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:24.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 msgr2=0x7f70bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.850 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 0x7f70bc069450 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f70cc007fd0 tx=0x7f70cc009040 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 msgr2=0x7f70dc1f4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f70c401a0b0 tx=0x7f70c4019220 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 shutdown_connections 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f70bc066f90 0x7f70bc069450 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f70dc15a3c0 0x7f70dc072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f70dc0cf510 0x7f70dc1f4300 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 --2- 172.21.15.82:0/2849760097 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f70dc0ae6c0 0x7f70dc0c48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 >> 172.21.15.82:0/2849760097 conn(0x7f70dc06ae60 msgr2=0x7f70dc15a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 shutdown_connections 2024-04-25T10:14:24.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.850+0000 7f70e16fc640 1 -- 172.21.15.82:0/2849760097 wait complete. 2024-04-25T10:14:24.873 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 Processor -- start 2024-04-25T10:14:24.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 -- start start 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e700adc70 0x7f7e7006e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e7006e9f0 0x7f7e700cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e700b28c0 con 0x7f7e700cfa50 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7015bb20 con 0x7f7e7006e9f0 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e752ef640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b2b60 con 0x7f7e700adc70 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e6e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f7e7006e9f0 0x7f7e700cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e6f7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7e700cfa50 0x7f7e7015f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43170/0 (socket says 172.21.15.82:43170) 2024-04-25T10:14:25.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:24.998+0000 7f7e6f7fe640 1 -- 172.21.15.82:0/3835930787 learned_addr learned my addr 172.21.15.82:0/3835930787 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6effd640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e700adc70 0x7f7e7006e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6dffb640 1 -- 172.21.15.82:0/3835930787 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 720582059 0 0) 0x7f7e700b28c0 con 0x7f7e700cfa50 2024-04-25T10:14:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6dffb640 1 -- 172.21.15.82:0/3835930787 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e58003530 con 0x7f7e700cfa50 2024-04-25T10:14:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6e7fc640 1 -- 172.21.15.82:0/3835930787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e700adc70 msgr2=0x7f7e7006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6e7fc640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e700adc70 0x7f7e7006e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6e7fc640 1 -- 172.21.15.82:0/3835930787 >> v1:172.21.15.82:6789/0 conn(0x7f7e700cfa50 legacy=0x7f7e7015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.002 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6e7fc640 1 -- 172.21.15.82:0/3835930787 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e700b2e40 con 0x7f7e7006e9f0 2024-04-25T10:14:25.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6e7fc640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e7006e9f0 0x7f7e700cf510 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7e640097c0 tx=0x7f7e6401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a648597996b5c160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6dffb640 1 -- 172.21.15.82:0/3835930787 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640b0070 con 0x7f7e7006e9f0 2024-04-25T10:14:25.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6dffb640 1 -- 172.21.15.82:0/3835930787 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e640a4e50 con 0x7f7e7006e9f0 2024-04-25T10:14:25.003 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e6dffb640 1 -- 172.21.15.82:0/3835930787 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e640ab800 con 0x7f7e7006e9f0 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 -- 172.21.15.82:0/3835930787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e7006e9f0 msgr2=0x7f7e700cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e7006e9f0 0x7f7e700cf510 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f7e640097c0 tx=0x7f7e6401c4a0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 -- 172.21.15.82:0/3835930787 shutdown_connections 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e7006e9f0 0x7f7e700cf510 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 --2- 172.21.15.82:0/3835930787 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e700adc70 0x7f7e7006e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 -- 172.21.15.82:0/3835930787 >> 172.21.15.82:0/3835930787 conn(0x7f7e7006ae60 msgr2=0x7f7e7006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 -- 172.21.15.82:0/3835930787 shutdown_connections 2024-04-25T10:14:25.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.002+0000 7f7e752ef640 1 -- 172.21.15.82:0/3835930787 wait complete. 2024-04-25T10:14:25.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 Processor -- start 2024-04-25T10:14:25.005 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- start start 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 0x7f7e700c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 0x7f7e7015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700b3c00 con 0x7f7e700cfa50 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e700cfe60 con 0x7f7e700adc70 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e70159960 con 0x7f7e7006e9f0 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60586/0 (socket says 172.21.15.82:60586) 2024-04-25T10:14:25.007 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 -- 172.21.15.82:0/395437217 learned_addr learned my addr 172.21.15.82:0/395437217 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6effd640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 0x7f7e700c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6e7fc640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 0x7f7e7015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 -- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 msgr2=0x7f7e700c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 0x7f7e700c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 -- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 msgr2=0x7f7e7015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 0x7f7e7015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 -- 172.21.15.82:0/395437217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e70159c40 con 0x7f7e700cfa50 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6effd640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 0x7f7e700c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6e7fc640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 0x7f7e7015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e6f7fe640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c390 tx=0x7f7e68001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e68017070 con 0x7f7e700cfa50 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e68005cf0 con 0x7f7e700cfa50 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e70159f20 con 0x7f7e700cfa50 2024-04-25T10:14:25.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e680072f0 con 0x7f7e700cfa50 2024-04-25T10:14:25.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7015a4e0 con 0x7f7e700cfa50 2024-04-25T10:14:25.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e70067c30 con 0x7f7e700cfa50 2024-04-25T10:14:25.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.006+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7e6801a070 con 0x7f7e700cfa50 2024-04-25T10:14:25.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.010+0000 7f7e577fe640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:25.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.010+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f7e68084470 con 0x7f7e700cfa50 2024-04-25T10:14:25.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.010+0000 7f7e6effd640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:25.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.010+0000 7f7e6effd640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 0x7f7e5c069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007330 tx=0x7f7e6000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:25.013 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.014+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7e68055430 con 0x7f7e700cfa50 2024-04-25T10:14:25.171 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.170+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7e7006d760 con 0x7f7e700cfa50 2024-04-25T10:14:25.860 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.858+0000 7f7e577fe640 1 -- 172.21.15.82:0/395437217 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7e68054d30 con 0x7f7e700cfa50 2024-04-25T10:14:25.860 INFO:teuthology.orchestra.run.smithi082.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 msgr2=0x7f7e5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 0x7f7e5c069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f7e60007330 tx=0x7f7e6000b040 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 msgr2=0x7f7e7015cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f7e6800c390 tx=0x7f7e68001e80 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 shutdown_connections 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7e5c066f60 0x7f7e5c069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7e700cfa50 0x7f7e7015cfe0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7e700adc70 0x7f7e7015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 --2- 172.21.15.82:0/395437217 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7e7006e9f0 0x7f7e700c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:25.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 >> 172.21.15.82:0/395437217 conn(0x7f7e7006ae60 msgr2=0x7f7e7006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:25.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 shutdown_connections 2024-04-25T10:14:25.864 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:25.862+0000 7f7e752ef640 1 -- 172.21.15.82:0/395437217 wait complete. 2024-04-25T10:14:25.891 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.018+0000 7f841e09c640 1 Processor -- start 2024-04-25T10:14:26.020 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.018+0000 7f841e09c640 1 -- start start 2024-04-25T10:14:26.021 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f841e09c640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f8418166060 0x7f84181664a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f841e09c640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418165b60 con 0x7f841806ec90 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f841e09c640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418162bd0 con 0x7f84180c2330 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f841e09c640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841806f0f0 con 0x7f8418166060 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84177fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7f84180c2330 0x7f84180c2710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:47852/0 (socket says 172.21.15.82:47852) 2024-04-25T10:14:26.022 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84177fe640 1 -- 172.21.15.82:0/1658999076 learned_addr learned my addr 172.21.15.82:0/1658999076 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554325019 0 0) 0x7f8418165b60 con 0x7f841806ec90 2024-04-25T10:14:26.023 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8400003530 con 0x7f841806ec90 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.1 v1:172.21.15.196:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176731191 0 0) 0x7f8418162bd0 con 0x7f84180c2330 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> v1:172.21.15.196:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8418165b60 con 0x7f84180c2330 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1052520841 0 0) 0x7f8400003530 con 0x7f841806ec90 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f8417fff640 1 --2- 172.21.15.82:0/1658999076 >> v2:172.21.15.196:3300/0 conn(0x7f8418166060 0x7f84181664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 >> v2:172.21.15.196:3300/0 conn(0x7f8418166060 msgr2=0x7f84181664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 --2- 172.21.15.82:0/1658999076 >> v2:172.21.15.196:3300/0 conn(0x7f8418166060 0x7f84181664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 >> v1:172.21.15.196:6789/0 conn(0x7f84180c2330 legacy=0x7f84180c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418162eb0 con 0x7f841806ec90 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f840c001680 con 0x7f841806ec90 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 >> v1:172.21.15.82:6789/0 conn(0x7f841806ec90 legacy=0x7f84181623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8400003bc0 0x7f8400003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8400004810 0x7f8400004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.024 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f840000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841806f0f0 con 0x7f8400005320 2024-04-25T10:14:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840000c260 con 0x7f8400003bc0 2024-04-25T10:14:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840000c460 con 0x7f8400004810 2024-04-25T10:14:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.022+0000 7f8417fff640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f840000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.025 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f84177fe640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8400003bc0 0x7f8400003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 -- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8400004810 msgr2=0x7f8400004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8400004810 0x7f8400004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 -- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8400003bc0 msgr2=0x7f8400003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8400003bc0 0x7f8400003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.026 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 -- 172.21.15.82:0/1658999076 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400003530 con 0x7f8400005320 2024-04-25T10:14:26.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 -- 172.21.15.82:0/1658999076 reap_dead start 2024-04-25T10:14:26.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.026+0000 7f8417fff640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f840000b9d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f8410007f60 tx=0x7f841001ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad978be7ba68a6ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100b2070 con 0x7f8400005320 2024-04-25T10:14:26.028 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84100ad440 con 0x7f8400005320 2024-04-25T10:14:26.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84167fc640 1 -- 172.21.15.82:0/1658999076 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410007b30 con 0x7f8400005320 2024-04-25T10:14:26.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 msgr2=0x7f840000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f840000b9d0 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7f8410007f60 tx=0x7f841001ceb0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.029 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- 172.21.15.82:0/1658999076 shutdown_connections 2024-04-25T10:14:26.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 --2- 172.21.15.82:0/1658999076 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f840000b9d0 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- 172.21.15.82:0/1658999076 >> 172.21.15.82:0/1658999076 conn(0x7f84180b0e90 msgr2=0x7f84180b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- 172.21.15.82:0/1658999076 shutdown_connections 2024-04-25T10:14:26.030 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- 172.21.15.82:0/1658999076 wait complete. 2024-04-25T10:14:26.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 Processor -- start 2024-04-25T10:14:26.031 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- start start 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 0x7f84180723b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8418155130 0x7f84180728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418166620 con 0x7f8400005320 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181664a0 con 0x7f8418155130 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f841e09c640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84181667a0 con 0x7f8418154cb0 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.033 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84177fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60596/0 (socket says 172.21.15.82:60596) 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f84177fe640 1 -- 172.21.15.82:0/844515012 learned_addr learned my addr 172.21.15.82:0/844515012 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f8417fff640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8418155130 0x7f84180728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.030+0000 7f8416ffd640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 0x7f84180723b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 -- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 msgr2=0x7f84180723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 0x7f84180723b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 -- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8418155130 msgr2=0x7f84180728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8418155130 0x7f84180728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 -- 172.21.15.82:0/844515012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180c44e0 con 0x7f8400005320 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f8416ffd640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 0x7f84180723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f84177fe640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f8408001450 tx=0x7f8408001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800f600 con 0x7f8400005320 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408018070 con 0x7f8400005320 2024-04-25T10:14:26.034 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84180c47c0 con 0x7f8400005320 2024-04-25T10:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800f9c0 con 0x7f8400005320 2024-04-25T10:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180c52a0 con 0x7f8400005320 2024-04-25T10:14:26.035 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841804efa0 con 0x7f8400005320 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f840800fb60 con 0x7f8400005320 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 0x7f8404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8408084ce0 con 0x7f8400005320 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f8416ffd640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 0x7f8404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.036 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.034+0000 7f8416ffd640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 0x7f8404069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f840c000f70 tx=0x7f840c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.039 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.038+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84080557f0 con 0x7f8400005320 2024-04-25T10:14:26.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.202+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f84180b9900 con 0x7f8400005320 2024-04-25T10:14:26.206 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.206+0000 7f83fffff640 1 -- 172.21.15.82:0/844515012 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8408059890 con 0x7f8400005320 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 msgr2=0x7f8404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 0x7f8404069420 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f840c000f70 tx=0x7f840c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 msgr2=0x7f84181543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f8408001450 tx=0x7f8408001910 comp rx=0 tx=0).stop 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 shutdown_connections 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8404066f60 0x7f8404069420 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f8418154cb0 0x7f84180723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.209 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f8418155130 0x7f84180728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 --2- 172.21.15.82:0/844515012 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8400005320 0x7f84181543c0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 >> 172.21.15.82:0/844515012 conn(0x7f84180b0e90 msgr2=0x7f84181653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 shutdown_connections 2024-04-25T10:14:26.210 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.210+0000 7f841e09c640 1 -- 172.21.15.82:0/844515012 wait complete. 2024-04-25T10:14:26.226 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T10:14:26.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 Processor -- start 2024-04-25T10:14:26.341 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 -- start start 2024-04-25T10:14:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 0x7f2a3c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f2a3c06ec60 0x7f2a3c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a3c0ccc70 con 0x7f2a3c0ce510 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c0cc2b0 con 0x7f2a3c06ec60 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a436f8640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a3c06e290 con 0x7f2a3c144ca0 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41494640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f2a3c0ce510 0x7f2a3c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43186/0 (socket says 172.21.15.82:43186) 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41494640 1 -- 172.21.15.82:0/587147683 learned_addr learned my addr 172.21.15.82:0/587147683 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41c95640 1 --2- 172.21.15.82:0/587147683 >> v2:172.21.15.82:3300/0 conn(0x7f2a3c06ec60 0x7f2a3c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a40c93640 1 --2- 172.21.15.82:0/587147683 >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 0x7f2a3c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a3bfff640 1 -- 172.21.15.82:0/587147683 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167741186 0 0) 0x7f2a3c0ccc70 con 0x7f2a3c0ce510 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a3bfff640 1 -- 172.21.15.82:0/587147683 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a24003530 con 0x7f2a3c0ce510 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41c95640 1 -- 172.21.15.82:0/587147683 >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 msgr2=0x7f2a3c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41c95640 1 --2- 172.21.15.82:0/587147683 >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 0x7f2a3c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41c95640 1 -- 172.21.15.82:0/587147683 >> v1:172.21.15.82:6789/0 conn(0x7f2a3c0ce510 legacy=0x7f2a3c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a41c95640 1 -- 172.21.15.82:0/587147683 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c06dc40 con 0x7f2a3c06ec60 2024-04-25T10:14:26.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.342+0000 7f2a40c93640 1 --2- 172.21.15.82:0/587147683 >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 0x7f2a3c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:26.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a41c95640 1 --2- 172.21.15.82:0/587147683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c142680 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c070fe0 tx=0x7f2a2c01f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b99d962f4b7400d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a3bfff640 1 -- 172.21.15.82:0/587147683 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0b4070 con 0x7f2a3c06ec60 2024-04-25T10:14:26.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a3bfff640 1 -- 172.21.15.82:0/587147683 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2c0af600 con 0x7f2a3c06ec60 2024-04-25T10:14:26.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a3bfff640 1 -- 172.21.15.82:0/587147683 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c0af9d0 con 0x7f2a3c06ec60 2024-04-25T10:14:26.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- 172.21.15.82:0/587147683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c06ec60 msgr2=0x7f2a3c142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- 172.21.15.82:0/587147683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c142680 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c070fe0 tx=0x7f2a2c01f110 comp rx=0 tx=0).stop 2024-04-25T10:14:26.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- 172.21.15.82:0/587147683 shutdown_connections 2024-04-25T10:14:26.346 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- 172.21.15.82:0/587147683 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c142680 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- 172.21.15.82:0/587147683 >> v2:172.21.15.196:3300/0 conn(0x7f2a3c144ca0 0x7f2a3c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- 172.21.15.82:0/587147683 >> 172.21.15.82:0/587147683 conn(0x7f2a3c0b11a0 msgr2=0x7f2a3c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- 172.21.15.82:0/587147683 shutdown_connections 2024-04-25T10:14:26.347 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- 172.21.15.82:0/587147683 wait complete. 2024-04-25T10:14:26.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 Processor -- start 2024-04-25T10:14:26.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- start start 2024-04-25T10:14:26.349 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c131240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 0x7f2a3c15b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c1451f0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c0cc900 con 0x7f2a3c06ec60 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a436f8640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3c145370 con 0x7f2a3c144ca0 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a40c93640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a40c93640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60610/0 (socket says 172.21.15.82:60610) 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a40c93640 1 -- 172.21.15.82:0/4160778774 learned_addr learned my addr 172.21.15.82:0/4160778774 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a41494640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c131240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.350 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.346+0000 7f2a41c95640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 0x7f2a3c15b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 -- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 msgr2=0x7f2a3c15b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 0x7f2a3c15b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 -- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c06ec60 msgr2=0x7f2a3c131240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c131240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 -- 172.21.15.82:0/4160778774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c0ca2e0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a41c95640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 0x7f2a3c15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a40c93640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d7d0 tx=0x7f2a3000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30018020 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a300096a0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a3c0ca560 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30009a90 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3c0cab20 con 0x7f2a3c0ce510 2024-04-25T10:14:26.351 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a3c04efa0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a3001b020 con 0x7f2a3c0ce510 2024-04-25T10:14:26.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:26.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2a300854b0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a41494640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:26.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.350+0000 7f2a41494640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 0x7f2a28069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005890 tx=0x7f2a34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:26.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.354+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a30022020 con 0x7f2a3c0ce510 2024-04-25T10:14:26.571 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.570+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7f2a3c0b1cc0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.862 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.862+0000 7f2a39ffb640 1 -- 172.21.15.82:0/4160778774 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v562) v1 ==== 144+0+0 (secure 0 0 0) 0x7f2a3c0ca2e0 con 0x7f2a3c0ce510 2024-04-25T10:14:26.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 msgr2=0x7f2a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 0x7f2a28069420 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f2a34005890 tx=0x7f2a34009040 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 msgr2=0x7f2a3c0c9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000d7d0 tx=0x7f2a3000dc90 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 shutdown_connections 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f2a28066f60 0x7f2a28069420 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f2a3c144ca0 0x7f2a3c15b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f2a3c0ce510 0x7f2a3c0c9da0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 --2- 172.21.15.82:0/4160778774 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f2a3c06ec60 0x7f2a3c131240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 >> 172.21.15.82:0/4160778774 conn(0x7f2a3c0b11a0 msgr2=0x7f2a3c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 shutdown_connections 2024-04-25T10:14:26.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:26.866+0000 7f2a436f8640 1 -- 172.21.15.82:0/4160778774 wait complete. 2024-04-25T10:14:26.887 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T10:14:26.887 INFO:tasks.cephfs_test_runner: 2024-04-25T10:14:26.887 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:14:26.887 INFO:tasks.cephfs_test_runner:Ran 7 tests in 689.059s 2024-04-25T10:14:26.888 INFO:tasks.cephfs_test_runner: 2024-04-25T10:14:26.888 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:14:26.888 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:14:26.897 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T10:14:26.905 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T10:14:26.905 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:14:26.905 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:26.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:26.913 DEBUG:teuthology.orchestra.run.smithi082:> set -ex 2024-04-25T10:14:26.913 DEBUG:teuthology.orchestra.run.smithi082:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:26.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16266 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:26.961 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:14:26.961 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:26.968 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:26.969 DEBUG:teuthology.orchestra.run.smithi196:> set -ex 2024-04-25T10:14:26.969 DEBUG:teuthology.orchestra.run.smithi196:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:27.017 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336460k,nr_inodes=4084115,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17298 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:27.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.018 DEBUG:teuthology.orchestra.run.smithi082:> ip netns list 2024-04-25T10:14:27.024 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:27.024 INFO:teuthology.orchestra.run.smithi082.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:27.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.025 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:27.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.083 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:27.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.153 DEBUG:teuthology.orchestra.run.smithi082:> sudo ip link delete ceph-brx 2024-04-25T10:14:27.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.334 DEBUG:teuthology.orchestra.run.smithi196:> ip netns list 2024-04-25T10:14:27.341 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:14:27.341 INFO:teuthology.orchestra.run.smithi196.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:14:27.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.341 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:14:27.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.397 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:14:27.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.452 DEBUG:teuthology.orchestra.run.smithi196:> sudo ip link delete ceph-brx 2024-04-25T10:14:27.606 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:14:27.616 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:14:27.616 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:27.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 Processor -- start 2024-04-25T10:14:27.745 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 -- start start 2024-04-25T10:14:27.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 0x7fc794099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 0x7fc79415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7940b0e10 con 0x7fc7940b0a30 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7940d0d90 con 0x7fc79406f4a0 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc79a86b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940d0180 con 0x7fc794099fb0 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc7937fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 0x7fc794099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc793fff640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7fc7940b0a30 0x7fc7940cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:47856/0 (socket says 172.21.15.82:47856) 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc798e08640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 0x7fc79415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc7937fe640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 0x7fc794099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.196:3300/0 says I am v2:172.21.15.82:48530/0 (socket says 172.21.15.82:48530) 2024-04-25T10:14:27.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc793fff640 1 -- 172.21.15.82:0/2629331216 learned_addr learned my addr 172.21.15.82:0/2629331216 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc798e08640 1 -- 172.21.15.82:0/2629331216 >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 msgr2=0x7fc794099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc798e08640 1 --2- 172.21.15.82:0/2629331216 >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 0x7fc794099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc798e08640 1 -- 172.21.15.82:0/2629331216 >> v1:172.21.15.196:6789/0 conn(0x7fc7940b0a30 legacy=0x7fc7940cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.746+0000 7fc798e08640 1 -- 172.21.15.82:0/2629331216 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940d0460 con 0x7fc794099fb0 2024-04-25T10:14:27.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc798e08640 1 --2- 172.21.15.82:0/2629331216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 0x7fc79415d230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009d90 tx=0x7fc78c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b7aad64f88acd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc792ffd640 1 -- 172.21.15.82:0/2629331216 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b2070 con 0x7fc794099fb0 2024-04-25T10:14:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc792ffd640 1 -- 172.21.15.82:0/2629331216 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c0a5830 con 0x7fc794099fb0 2024-04-25T10:14:27.750 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc792ffd640 1 -- 172.21.15.82:0/2629331216 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0a5c00 con 0x7fc794099fb0 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- 172.21.15.82:0/2629331216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 msgr2=0x7fc79415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 --2- 172.21.15.82:0/2629331216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 0x7fc79415d230 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc78c009d90 tx=0x7fc78c01c970 comp rx=0 tx=0).stop 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- 172.21.15.82:0/2629331216 shutdown_connections 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 --2- 172.21.15.82:0/2629331216 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc794099fb0 0x7fc79415d230 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 --2- 172.21.15.82:0/2629331216 >> v2:172.21.15.196:3300/0 conn(0x7fc79406f4a0 0x7fc794099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- 172.21.15.82:0/2629331216 >> 172.21.15.82:0/2629331216 conn(0x7fc7940b30b0 msgr2=0x7fc7940b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- 172.21.15.82:0/2629331216 shutdown_connections 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- 172.21.15.82:0/2629331216 wait complete. 2024-04-25T10:14:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 Processor -- start 2024-04-25T10:14:27.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.750+0000 7fc79a86b640 1 -- start start 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc79406f4a0 0x7fc79414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794099fb0 0x7fc794153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940cc020 con 0x7fc7940b0a30 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940cbea0 con 0x7fc794099fb0 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7940cc1a0 con 0x7fc79406f4a0 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60622/0 (socket says 172.21.15.82:60622) 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 -- 172.21.15.82:0/240363258 learned_addr learned my addr 172.21.15.82:0/240363258 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc7937fe640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794099fb0 0x7fc794153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc793fff640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc79406f4a0 0x7fc79414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 -- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc79406f4a0 msgr2=0x7fc79414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc79406f4a0 0x7fc79414d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 -- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794099fb0 msgr2=0x7fc794153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794099fb0 0x7fc794153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 -- 172.21.15.82:0/240363258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79406dd10 con 0x7fc7940b0a30 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc798e08640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0b0040 tx=0x7fc78c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0b2050 con 0x7fc7940b0a30 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78c0be070 con 0x7fc7940b0a30 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc794150240 con 0x7fc7940b0a30 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78c0ad890 con 0x7fc7940b0a30 2024-04-25T10:14:27.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc79406cc70 con 0x7fc7940b0a30 2024-04-25T10:14:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79404efa0 con 0x7fc7940b0a30 2024-04-25T10:14:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc78c0ada30 con 0x7fc7940b0a30 2024-04-25T10:14:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 0x7fc780069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:27.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc78c120030 con 0x7fc7940b0a30 2024-04-25T10:14:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc793fff640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 0x7fc780069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:27.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.754+0000 7fc793fff640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 0x7fc780069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc788000f70 tx=0x7fc788009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:27.759 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.758+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc78c016560 con 0x7fc7940b0a30 2024-04-25T10:14:27.917 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.918+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc79406fa80 con 0x7fc780066f60 2024-04-25T10:14:27.918 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.918+0000 7fc790ff9640 1 -- 172.21.15.82:0/240363258 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13512 (secure 0 0 0) 0x7fc79406fa80 con 0x7fc780066f60 2024-04-25T10:14:27.918 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:27.919 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 msgr2=0x7fc780069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 0x7fc780069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc788000f70 tx=0x7fc788009040 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 msgr2=0x7fc7941527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fc78c0b0040 tx=0x7fc78c019710 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 shutdown_connections 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc780066f60 0x7fc780069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc7940b0a30 0x7fc7941527d0 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc794099fb0 0x7fc794153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 --2- 172.21.15.82:0/240363258 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc79406f4a0 0x7fc79414d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 >> 172.21.15.82:0/240363258 conn(0x7fc7940b30b0 msgr2=0x7fc7940b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 shutdown_connections 2024-04-25T10:14:27.921 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:27.922+0000 7fc79a86b640 1 -- 172.21.15.82:0/240363258 wait complete. 2024-04-25T10:14:27.937 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-04-25T10:14:27.499623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279136,"kb_used_data":29696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695584,"statfs":{"total":773094113280,"available":772808278016,"internally_reserved":0,"allocated":30408704,"data_stored":23766064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002387"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":368,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:14:22.877813+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:14:22.877813+0000","last_peered":"2024-04-25T10:14:22.877813+0000","last_clean":"2024-04-25T10:14:22.877813+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:14:22.877813+0000","last_undegraded":"2024-04-25T10:14:22.877813+0000","last_fullsized":"2024-04-25T10:14:22.877813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2158592,"data_stored":1316223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5742592,"data_stored":4919958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4922341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":728059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":725077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:27.938 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 Processor -- start 2024-04-25T10:14:28.053 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 -- start start 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a500cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500cf370 con 0x7f7a5015e1e0 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a500ce070 con 0x7f7a500b2ac0 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a54955640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500ce350 con 0x7f7a5015a600 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4ed9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7a5015e1e0 0x7f7a500b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43202/0 (socket says 172.21.15.82:43202) 2024-04-25T10:14:28.054 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4ed9d640 1 -- 172.21.15.82:0/1133790416 learned_addr learned my addr 172.21.15.82:0/1133790416 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:28.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a500cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705946468 0 0) 0x7f7a500cf370 con 0x7f7a5015e1e0 2024-04-25T10:14:28.055 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a38003530 con 0x7f7a5015e1e0 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2361823586 0 0) 0x7f7a38003530 con 0x7f7a5015e1e0 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 msgr2=0x7f7a500cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a500cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 >> v1:172.21.15.196:6789/0 conn(0x7f7a500b2ac0 legacy=0x7f7a500cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500ce630 con 0x7f7a5015e1e0 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7a48001a00 con 0x7f7a5015e1e0 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 >> v1:172.21.15.82:6789/0 conn(0x7f7a5015e1e0 legacy=0x7f7a500b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a38003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.057 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a38004d50 0x7f7a380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500ce350 con 0x7f7a38003a60 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3800ba80 con 0x7f7a38004d50 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3800de40 con 0x7f7a5015a600 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a38003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4dd9b640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4ed9d640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a38004d50 0x7f7a380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 -- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 msgr2=0x7f7a38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 -- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a38004d50 msgr2=0x7f7a380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a38004d50 0x7f7a380051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.058 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4e59c640 1 -- 172.21.15.82:0/1133790416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38003530 con 0x7f7a38003a60 2024-04-25T10:14:28.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.054+0000 7f7a4dd9b640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5015a600 0x7f7a38004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.059 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a38003e40 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ae0 tx=0x7f7a4001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25226e3cda5d7c60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400b2070 con 0x7f7a38003a60 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a400a5660 con 0x7f7a38003a60 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a4d59a640 1 -- 172.21.15.82:0/1133790416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400096b0 con 0x7f7a38003a60 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- 172.21.15.82:0/1133790416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 msgr2=0x7f7a38003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 --2- 172.21.15.82:0/1133790416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a38003e40 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ae0 tx=0x7f7a4001ca70 comp rx=0 tx=0).stop 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a4ed9d640 1 -- 172.21.15.82:0/1133790416 reap_dead start 2024-04-25T10:14:28.062 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- 172.21.15.82:0/1133790416 shutdown_connections 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- 172.21.15.82:0/1133790416 >> 172.21.15.82:0/1133790416 conn(0x7f7a500b1f10 msgr2=0x7f7a500b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- 172.21.15.82:0/1133790416 shutdown_connections 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- 172.21.15.82:0/1133790416 wait complete. 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 Processor -- start 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.058+0000 7f7a54955640 1 -- start start 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5014c040 0x7f7a501f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 0x7f7a501f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015a9e0 con 0x7f7a38003a60 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a500cc1e0 con 0x7f7a501f3000 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5015ab60 con 0x7f7a5014c040 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60638/0 (socket says 172.21.15.82:60638) 2024-04-25T10:14:28.063 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 -- 172.21.15.82:0/2302338939 learned_addr learned my addr 172.21.15.82:0/2302338939 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4ed9d640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 0x7f7a501f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4dd9b640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5014c040 0x7f7a501f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 -- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5014c040 msgr2=0x7f7a501f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5014c040 0x7f7a501f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 -- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 msgr2=0x7f7a501f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 0x7f7a501f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 -- 172.21.15.82:0/2302338939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a500c4910 con 0x7f7a38003a60 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4ed9d640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 0x7f7a501f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4e59c640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ae0 tx=0x7f7a38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400b2040 con 0x7f7a38003a60 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a40004020 con 0x7f7a38003a60 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a500c4bf0 con 0x7f7a38003a60 2024-04-25T10:14:28.064 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a400043d0 con 0x7f7a38003a60 2024-04-25T10:14:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a500c51b0 con 0x7f7a38003a60 2024-04-25T10:14:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5004efa0 con 0x7f7a38003a60 2024-04-25T10:14:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a400c0030 con 0x7f7a38003a60 2024-04-25T10:14:28.065 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 0x7f7a30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f7a4011f5d0 con 0x7f7a38003a60 2024-04-25T10:14:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4dd9b640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 0x7f7a30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.062+0000 7f7a4dd9b640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 0x7f7a30069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7a500cd830 tx=0x7f7a4400c130 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.066 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.066+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a400c7060 con 0x7f7a38003a60 2024-04-25T10:14:28.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.214+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a500b6380 con 0x7f7a30066f60 2024-04-25T10:14:28.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.214+0000 7f7a3effd640 1 -- 172.21.15.82:0/2302338939 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13512 (secure 0 0 0) 0x7f7a500b6380 con 0x7f7a30066f60 2024-04-25T10:14:28.215 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:28.215 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.214+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 msgr2=0x7f7a30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.214+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 0x7f7a30069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7a500cd830 tx=0x7f7a4400c130 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.214+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 msgr2=0x7f7a501eec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009ae0 tx=0x7f7a38005840 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 shutdown_connections 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7a30066f60 0x7f7a30069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7a5014c040 0x7f7a501f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7a501f3000 0x7f7a501f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 --2- 172.21.15.82:0/2302338939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7a38003a60 0x7f7a501eec40 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 >> 172.21.15.82:0/2302338939 conn(0x7f7a500b1f10 msgr2=0x7f7a5006f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 shutdown_connections 2024-04-25T10:14:28.218 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.218+0000 7f7a54955640 1 -- 172.21.15.82:0/2302338939 wait complete. 2024-04-25T10:14:28.232 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-04-25T10:14:27.499623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279136,"kb_used_data":29696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695584,"statfs":{"total":773094113280,"available":772808278016,"internally_reserved":0,"allocated":30408704,"data_stored":23766064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002387"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":368,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:14:22.877813+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:14:22.877813+0000","last_peered":"2024-04-25T10:14:22.877813+0000","last_clean":"2024-04-25T10:14:22.877813+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:14:22.877813+0000","last_undegraded":"2024-04-25T10:14:22.877813+0000","last_fullsized":"2024-04-25T10:14:22.877813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2158592,"data_stored":1316223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5742592,"data_stored":4919958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4922341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":728059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":725077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:28.233 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:14:28.233 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:28.352 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 Processor -- start 2024-04-25T10:14:28.353 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 -- start start 2024-04-25T10:14:28.354 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1415da80 0x7fec140d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fec140d1cd0 0x7fec140d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1406be60 con 0x7fec140d0620 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec140d0a70 con 0x7fec1415da80 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec18fde640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1406c440 con 0x7fec140d1cd0 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1415da80 0x7fec140d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec12d9d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fec140d0620 0x7fec140ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43210/0 (socket says 172.21.15.82:43210) 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fec1415da80 0x7fec140d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60642/0 (socket says 172.21.15.82:60642) 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec12d9d640 1 -- 172.21.15.82:0/4206196066 learned_addr learned my addr 172.21.15.82:0/4206196066 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec0bfff640 1 --2- 172.21.15.82:0/4206196066 >> v2:172.21.15.196:3300/0 conn(0x7fec140d1cd0 0x7fec140d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 -- 172.21.15.82:0/4206196066 >> v2:172.21.15.196:3300/0 conn(0x7fec140d1cd0 msgr2=0x7fec140d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 --2- 172.21.15.82:0/4206196066 >> v2:172.21.15.196:3300/0 conn(0x7fec140d1cd0 0x7fec140d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 -- 172.21.15.82:0/4206196066 >> v1:172.21.15.82:6789/0 conn(0x7fec140d0620 legacy=0x7fec140ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.355 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec11d9b640 1 -- 172.21.15.82:0/4206196066 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1325515095 0 0) 0x7fec1406be60 con 0x7fec140d0620 2024-04-25T10:14:28.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.354+0000 7fec1259c640 1 -- 172.21.15.82:0/4206196066 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1415de60 con 0x7fec1415da80 2024-04-25T10:14:28.356 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec1259c640 1 --2- 172.21.15.82:0/4206196066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1415da80 0x7fec140d4230 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fec040097c0 tx=0x7fec0401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9445077e2f06c19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec11d9b640 1 -- 172.21.15.82:0/4206196066 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040b0070 con 0x7fec1415da80 2024-04-25T10:14:28.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec11d9b640 1 -- 172.21.15.82:0/4206196066 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec040ab600 con 0x7fec1415da80 2024-04-25T10:14:28.357 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec11d9b640 1 -- 172.21.15.82:0/4206196066 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec040ab9d0 con 0x7fec1415da80 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- 172.21.15.82:0/4206196066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1415da80 msgr2=0x7fec140d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- 172.21.15.82:0/4206196066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1415da80 0x7fec140d4230 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fec040097c0 tx=0x7fec0401c460 comp rx=0 tx=0).stop 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- 172.21.15.82:0/4206196066 shutdown_connections 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- 172.21.15.82:0/4206196066 >> v2:172.21.15.196:3300/0 conn(0x7fec140d1cd0 0x7fec140d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- 172.21.15.82:0/4206196066 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec1415da80 0x7fec140d4230 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- 172.21.15.82:0/4206196066 >> 172.21.15.82:0/4206196066 conn(0x7fec1408b000 msgr2=0x7fec140d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- 172.21.15.82:0/4206196066 shutdown_connections 2024-04-25T10:14:28.358 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- 172.21.15.82:0/4206196066 wait complete. 2024-04-25T10:14:28.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 Processor -- start 2024-04-25T10:14:28.359 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- start start 2024-04-25T10:14:28.360 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec140d0620 0x7fec140c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec1415da80 0x7fec1415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1416a730 con 0x7fec140d1cd0 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1416a5b0 con 0x7fec1415da80 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec18fde640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1416a8b0 con 0x7fec140d0620 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60658/0 (socket says 172.21.15.82:60658) 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 -- 172.21.15.82:0/1380810418 learned_addr learned my addr 172.21.15.82:0/1380810418 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec12d9d640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec1415da80 0x7fec1415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec1259c640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec140d0620 0x7fec140c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 -- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec140d0620 msgr2=0x7fec140c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.361 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec140d0620 0x7fec140c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 -- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec1415da80 msgr2=0x7fec1415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec1415da80 0x7fec1415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 -- 172.21.15.82:0/1380810418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140b42d0 con 0x7fec140d1cd0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.358+0000 7fec0bfff640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7febfc00d830 tx=0x7febfc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0059a0 con 0x7fec140d1cd0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc005b40 con 0x7fec140d1cd0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec140c7720 con 0x7fec140d1cd0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc005d30 con 0x7fec140d1cd0 2024-04-25T10:14:28.362 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1415f240 con 0x7fec140d1cd0 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1404efa0 con 0x7fec140d1cd0 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7febfc01b020 con 0x7fec140d1cd0 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 0x7febf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec1259c640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 0x7febf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7febfc084f30 con 0x7fec140d1cd0 2024-04-25T10:14:28.363 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.362+0000 7fec1259c640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 0x7febf4069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fec040ad000 tx=0x7fec040a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.367 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.366+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec140c7720 con 0x7fec140d1cd0 2024-04-25T10:14:28.511 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.510+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec140b02c0 con 0x7febf4066f60 2024-04-25T10:14:28.512 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec0affd640 1 -- 172.21.15.82:0/1380810418 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13512 (secure 0 0 0) 0x7fec140b02c0 con 0x7febf4066f60 2024-04-25T10:14:28.513 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:28.513 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 msgr2=0x7febf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 0x7febf4069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fec040ad000 tx=0x7fec040a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 msgr2=0x7fec14156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7febfc00d830 tx=0x7febfc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 shutdown_connections 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7febf4066f60 0x7febf4069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fec1415da80 0x7fec1415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fec140d1cd0 0x7fec14156d40 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 --2- 172.21.15.82:0/1380810418 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fec140d0620 0x7fec140c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 >> 172.21.15.82:0/1380810418 conn(0x7fec1408b000 msgr2=0x7fec14073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 shutdown_connections 2024-04-25T10:14:28.515 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.514+0000 7fec18fde640 1 -- 172.21.15.82:0/1380810418 wait complete. 2024-04-25T10:14:28.529 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-04-25T10:14:27.499623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279136,"kb_used_data":29696,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695584,"statfs":{"total":773094113280,"available":772808278016,"internally_reserved":0,"allocated":30408704,"data_stored":23766064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390725,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-267,"num_write":-176,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002387"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":368,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:14:22.877813+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:14:22.877813+0000","last_peered":"2024-04-25T10:14:22.877813+0000","last_clean":"2024-04-25T10:14:22.877813+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:14:22.877813+0000","last_undegraded":"2024-04-25T10:14:22.877813+0000","last_fullsized":"2024-04-25T10:14:22.877813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33288,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338552,"statfs":{"total":96636764160,"available":96602677248,"internally_reserved":0,"allocated":2158592,"data_stored":1316223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575002,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36788,"kb_used_data":5608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335052,"statfs":{"total":96636764160,"available":96599093248,"internally_reserved":0,"allocated":5742592,"data_stored":4919958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575003,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575001,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":5624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":5758976,"data_stored":4922341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1560576,"data_stored":728059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575002,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":725077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36792,"kb_used_data":5612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335048,"statfs":{"total":96636764160,"available":96599089152,"internally_reserved":0,"allocated":5746688,"data_stored":4919403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1315467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:28.530 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:28.647 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.646+0000 7fba88cae640 1 Processor -- start 2024-04-25T10:14:28.648 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.646+0000 7fba88cae640 1 -- start start 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba88cae640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fba8415da80 0x7fba840d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba88cae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba8406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba88cae640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba8406be60 con 0x7fba840d1cd0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba88cae640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8406c440 con 0x7fba8415da80 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba88cae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8415de60 con 0x7fba840d07f0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba81d9b640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fba840d1cd0 0x7fba840d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43222/0 (socket says 172.21.15.82:43222) 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba81d9b640 1 -- 172.21.15.82:0/4153179416 learned_addr learned my addr 172.21.15.82:0/4153179416 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8259c640 1 --2- 172.21.15.82:0/4153179416 >> v2:172.21.15.196:3300/0 conn(0x7fba8415da80 0x7fba840d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba8406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094781285 0 0) 0x7fba8406be60 con 0x7fba840d1cd0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba6c003530 con 0x7fba840d1cd0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 671635262 0 0) 0x7fba6c003530 con 0x7fba840d1cd0 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 msgr2=0x7fba8406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba8406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 >> v2:172.21.15.196:3300/0 conn(0x7fba8415da80 msgr2=0x7fba840d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 --2- 172.21.15.82:0/4153179416 >> v2:172.21.15.196:3300/0 conn(0x7fba8415da80 0x7fba840d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840d2450 con 0x7fba840d1cd0 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fba78001820 con 0x7fba840d1cd0 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 >> v1:172.21.15.82:6789/0 conn(0x7fba840d1cd0 legacy=0x7fba840d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba6c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c003fb0 0x7fba6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8406c440 con 0x7fba6c004d50 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8415de60 con 0x7fba6c003fb0 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c00ba00 con 0x7fba840d07f0 2024-04-25T10:14:28.652 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba8259c640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba6c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba81d9b640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c003fb0 0x7fba6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 -- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 msgr2=0x7fba6c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba840d07f0 0x7fba6c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 -- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c003fb0 msgr2=0x7fba6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba6c003fb0 0x7fba6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.653 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.650+0000 7fba82d9d640 1 -- 172.21.15.82:0/4153179416 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c003530 con 0x7fba6c004d50 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba82d9d640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba6c0051d0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fba8408d8a0 tx=0x7fba7c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd33adb2c2f1e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c0b2070 con 0x7fba6c004d50 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7c0a5c30 con 0x7fba6c004d50 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8159a640 1 -- 172.21.15.82:0/4153179416 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7c009c10 con 0x7fba6c004d50 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/4153179416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 msgr2=0x7fba6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 --2- 172.21.15.82:0/4153179416 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba6c0051d0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7fba8408d8a0 tx=0x7fba7c01cc20 comp rx=0 tx=0).stop 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba82d9d640 1 -- 172.21.15.82:0/4153179416 reap_dead start 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/4153179416 shutdown_connections 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/4153179416 >> 172.21.15.82:0/4153179416 conn(0x7fba8408b000 msgr2=0x7fba840d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/4153179416 shutdown_connections 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/4153179416 wait complete. 2024-04-25T10:14:28.655 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 Processor -- start 2024-04-25T10:14:28.656 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- start start 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba840ca7c0 0x7fba8409e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba8414b9d0 0x7fba840c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840d26f0 con 0x7fba6c004d50 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba840d2120 con 0x7fba840ca7c0 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba840d2870 con 0x7fba8414b9d0 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.658 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60676/0 (socket says 172.21.15.82:60676) 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 -- 172.21.15.82:0/3159426657 learned_addr learned my addr 172.21.15.82:0/3159426657 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba81d9b640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba840ca7c0 0x7fba8409e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba82d9d640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba8414b9d0 0x7fba840c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 -- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba8414b9d0 msgr2=0x7fba840c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba8414b9d0 0x7fba840c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 -- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba840ca7c0 msgr2=0x7fba8409e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba840ca7c0 0x7fba8409e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 -- 172.21.15.82:0/3159426657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba840cac40 con 0x7fba6c004d50 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba8259c640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fba74011690 tx=0x7fba74011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7400f9c0 con 0x7fba6c004d50 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba7400fb60 con 0x7fba6c004d50 2024-04-25T10:14:28.659 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba74015460 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840caec0 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.654+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba840cb480 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8404efa0 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba74015600 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba72ffd640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 0x7fba64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba81d9b640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 0x7fba64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fba74085ae0 con 0x7fba6c004d50 2024-04-25T10:14:28.660 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.658+0000 7fba81d9b640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 0x7fba64069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fba780010f0 tx=0x7fba78000f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:28.662 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.662+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba840cac40 con 0x7fba6c004d50 2024-04-25T10:14:28.823 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.822+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba84073910 con 0x7fba6c004d50 2024-04-25T10:14:28.825 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba72ffd640 1 -- 172.21.15.82:0/3159426657 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+13573 (secure 0 0 0) 0x7fba74019030 con 0x7fba6c004d50 2024-04-25T10:14:28.825 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:28.825 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":126,"fsid":"923528c9-0d4c-4379-be0f-2d1b434d3c6c","created":"2024-04-25T10:01:42.716523+0000","modified":"2024-04-25T10:14:25.850108+0000","last_up_change":"2024-04-25T10:01:53.993091+0000","last_in_change":"2024-04-25T10:01:44.990012+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:55.331959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"246b7925-be82-4cc8-a55d-b2faaec118ea","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6800","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6801","nonce":2082960483}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6802","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6804","nonce":2082960483}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6811","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6813","nonce":2082960483}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6806","nonce":2082960483},{"type":"v1","addr":"172.21.15.82:6808","nonce":2082960483}]},"public_addr":"172.21.15.82:6801/2082960483","cluster_addr":"172.21.15.82:6804/2082960483","heartbeat_back_addr":"172.21.15.82:6813/2082960483","heartbeat_front_addr":"172.21.15.82:6808/2082960483","state":["exists","up"]},{"osd":1,"uuid":"bbb18f09-b87b-47ec-a70e-f53bd22c7e66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6816","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6817","nonce":1010291016}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6818","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6819","nonce":1010291016}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6822","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6823","nonce":1010291016}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6820","nonce":1010291016},{"type":"v1","addr":"172.21.15.82:6821","nonce":1010291016}]},"public_addr":"172.21.15.82:6817/1010291016","cluster_addr":"172.21.15.82:6819/1010291016","heartbeat_back_addr":"172.21.15.82:6823/1010291016","heartbeat_front_addr":"172.21.15.82:6821/1010291016","state":["exists","up"]},{"osd":2,"uuid":"917e2894-19a0-4a70-b172-c039392e5436","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6803","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6805","nonce":3086763100}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6807","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6809","nonce":3086763100}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6814","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6815","nonce":3086763100}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6810","nonce":3086763100},{"type":"v1","addr":"172.21.15.82:6812","nonce":3086763100}]},"public_addr":"172.21.15.82:6805/3086763100","cluster_addr":"172.21.15.82:6809/3086763100","heartbeat_back_addr":"172.21.15.82:6815/3086763100","heartbeat_front_addr":"172.21.15.82:6812/3086763100","state":["exists","up"]},{"osd":3,"uuid":"54695f07-cce4-4a88-b28f-a1d141b52f49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6824","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6825","nonce":3269667522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6826","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6827","nonce":3269667522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6830","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6831","nonce":3269667522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6828","nonce":3269667522},{"type":"v1","addr":"172.21.15.82:6829","nonce":3269667522}]},"public_addr":"172.21.15.82:6825/3269667522","cluster_addr":"172.21.15.82:6827/3269667522","heartbeat_back_addr":"172.21.15.82:6831/3269667522","heartbeat_front_addr":"172.21.15.82:6829/3269667522","state":["exists","up"]},{"osd":4,"uuid":"26f4d1c4-cc52-4b4b-96bd-d6e79f823a3f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6816","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6817","nonce":2164040014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6818","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6819","nonce":2164040014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6824","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6826","nonce":2164040014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6820","nonce":2164040014},{"type":"v1","addr":"172.21.15.196:6821","nonce":2164040014}]},"public_addr":"172.21.15.196:6817/2164040014","cluster_addr":"172.21.15.196:6819/2164040014","heartbeat_back_addr":"172.21.15.196:6826/2164040014","heartbeat_front_addr":"172.21.15.196:6821/2164040014","state":["exists","up"]},{"osd":5,"uuid":"2a95d237-1bf2-424b-b5d4-b3e5ae3ee189","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6800","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6802","nonce":1745719129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6804","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6805","nonce":1745719129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6810","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6812","nonce":1745719129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6807","nonce":1745719129},{"type":"v1","addr":"172.21.15.196:6809","nonce":1745719129}]},"public_addr":"172.21.15.196:6802/1745719129","cluster_addr":"172.21.15.196:6805/1745719129","heartbeat_back_addr":"172.21.15.196:6812/1745719129","heartbeat_front_addr":"172.21.15.196:6809/1745719129","state":["exists","up"]},{"osd":6,"uuid":"da7f4dc6-f8de-4cce-9987-dd7b65760a8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6801","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6803","nonce":1119498737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6806","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6808","nonce":1119498737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6814","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6815","nonce":1119498737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6811","nonce":1119498737},{"type":"v1","addr":"172.21.15.196:6813","nonce":1119498737}]},"public_addr":"172.21.15.196:6803/1119498737","cluster_addr":"172.21.15.196:6808/1119498737","heartbeat_back_addr":"172.21.15.196:6815/1119498737","heartbeat_front_addr":"172.21.15.196:6813/1119498737","state":["exists","up"]},{"osd":7,"uuid":"0c42460a-e6a1-45e0-95a9-2f8ffb3ff016","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":119,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6822","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6823","nonce":2552340661}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6825","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6827","nonce":2552340661}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6830","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6831","nonce":2552340661}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6828","nonce":2552340661},{"type":"v1","addr":"172.21.15.196:6829","nonce":2552340661}]},"public_addr":"172.21.15.196:6823/2552340661","cluster_addr":"172.21.15.196:6827/2552340661","heartbeat_back_addr":"172.21.15.196:6831/2552340661","heartbeat_front_addr":"172.21.15.196:6829/2552340661","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.048796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.510788+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.215026+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.597769+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.549803+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.628694+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.614785+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:50.608995+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.82:6835/2659325772":"2024-04-26T10:14:22.831302+0000","172.21.15.82:6837/2659325772":"2024-04-26T10:14:22.831302+0000","172.21.15.82:6837/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6837/781412207":"2024-04-26T10:13:47.565011+0000","172.21.15.82:6835/3303693719":"2024-04-26T10:14:07.178065+0000","172.21.15.82:6835/781412207":"2024-04-26T10:13:47.565011+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 msgr2=0x7fba64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 0x7fba64069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fba780010f0 tx=0x7fba78000f80 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 msgr2=0x7fba8415b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7fba74011690 tx=0x7fba74011b50 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 shutdown_connections 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fba64066f60 0x7fba64069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fba8414b9d0 0x7fba840c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fba840ca7c0 0x7fba8409e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 --2- 172.21.15.82:0/3159426657 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fba6c004d50 0x7fba8415b8d0 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 >> 172.21.15.82:0/3159426657 conn(0x7fba8408b000 msgr2=0x7fba8406ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 shutdown_connections 2024-04-25T10:14:28.828 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:28.826+0000 7fba88cae640 1 -- 172.21.15.82:0/3159426657 wait complete. 2024-04-25T10:14:29.845 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:14:29.846 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:29.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.982+0000 7f479db2e640 1 Processor -- start 2024-04-25T10:14:29.983 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.982+0000 7f479db2e640 1 -- start start 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f479db2e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f479815eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f479db2e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f47980cef90 0x7f47980b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f479db2e640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479815ab40 con 0x7f479806efd0 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f479db2e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479815f050 con 0x7f47980cef90 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f479db2e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479806f3b0 con 0x7f479806bea0 2024-04-25T10:14:29.986 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f479806efd0 0x7f47980ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43226/0 (socket says 172.21.15.82:43226) 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 -- 172.21.15.82:0/2170871011 learned_addr learned my addr 172.21.15.82:0/2170871011 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 71086706 0 0) 0x7f4788001120 con 0x7f479806efd0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4780003530 con 0x7f479806efd0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f4796ffd640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f479815eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f4797fff640 1 --2- 172.21.15.82:0/2170871011 >> v2:172.21.15.196:3300/0 conn(0x7f47980cef90 0x7f47980b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3537251743 0 0) 0x7f4780003530 con 0x7f479806efd0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 msgr2=0x7f479815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f479815eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 >> v2:172.21.15.196:3300/0 conn(0x7f47980cef90 msgr2=0x7f47980b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 --2- 172.21.15.82:0/2170871011 >> v2:172.21.15.196:3300/0 conn(0x7f47980cef90 0x7f47980b3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980b5a40 con 0x7f479806efd0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f4788001850 con 0x7f479806efd0 2024-04-25T10:14:29.987 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f4796ffd640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f479815eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 >> v1:172.21.15.82:6789/0 conn(0x7f479806efd0 legacy=0x7f47980ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47800039c0 0x7f4780003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4780004810 0x7f4780004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f47800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815f050 con 0x7f47800039c0 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479806f3b0 con 0x7f4780004810 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478000bab0 con 0x7f479806bea0 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47800039c0 0x7f4780003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f4796ffd640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4780004810 0x7f4780004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f4797fff640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f47800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 -- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 msgr2=0x7f47800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f479806bea0 0x7f47800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 -- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4780004810 msgr2=0x7f4780004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f4780004810 0x7f4780004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.988 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 -- 172.21.15.82:0/2170871011 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4780003530 con 0x7f47800039c0 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47977fe640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47800039c0 0x7f4780003de0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f478800c2f0 tx=0x7f478801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a719c9327f7ce73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788007e60 con 0x7f47800039c0 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47880a9020 con 0x7f47800039c0 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.986+0000 7f47967fc640 1 -- 172.21.15.82:0/2170871011 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788005ae0 con 0x7f47800039c0 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/2170871011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47800039c0 msgr2=0x7f4780003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 --2- 172.21.15.82:0/2170871011 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f47800039c0 0x7f4780003de0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f478800c2f0 tx=0x7f478801cba0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f4797fff640 1 -- 172.21.15.82:0/2170871011 reap_dead start 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/2170871011 shutdown_connections 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/2170871011 >> 172.21.15.82:0/2170871011 conn(0x7f47980cc680 msgr2=0x7f47980cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/2170871011 shutdown_connections 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/2170871011 wait complete. 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 Processor -- start 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- start start 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 0x7f479814cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4798150d10 0x7f479814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f479806ac90 0x7f479806b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815b880 con 0x7f479814c060 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479815b410 con 0x7f479806ac90 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479815ba00 con 0x7f4798150d10 2024-04-25T10:14:29.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 0x7f479814cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 0x7f479814cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60706/0 (socket says 172.21.15.82:60706) 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 -- 172.21.15.82:0/427997531 learned_addr learned my addr 172.21.15.82:0/427997531 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f4797fff640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f479806ac90 0x7f479806b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f4796ffd640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4798150d10 0x7f479814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 -- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4798150d10 msgr2=0x7f479814d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4798150d10 0x7f479814d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 -- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f479806ac90 msgr2=0x7f479806b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f479806ac90 0x7f479806b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479814fc70 con 0x7f479814c060 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f4796ffd640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f4798150d10 0x7f479814d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f47977fe640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 0x7f479814cff0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f478801c610 tx=0x7f4780005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788007e60 con 0x7f479814c060 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47880b2030 con 0x7f479814c060 2024-04-25T10:14:29.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479814ff50 con 0x7f479814c060 2024-04-25T10:14:29.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4788004cf0 con 0x7f479814c060 2024-04-25T10:14:29.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.990+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798150510 con 0x7f479814c060 2024-04-25T10:14:29.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f479814fa20 con 0x7f479814c060 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47880a5680 con 0x7f479814c060 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4784066f60 0x7f4784069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f478811dd40 con 0x7f479814c060 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f4796ffd640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4784066f60 0x7f4784069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7f478406d540 0x7f478406f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f47880acec0 con 0x7f478406d540 2024-04-25T10:14:29.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4788120440 con 0x7f479814c060 2024-04-25T10:14:29.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f4797fff640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7f478406d540 0x7f478406f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:29.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f4796ffd640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4784066f60 0x7f4784069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f479814f7e0 tx=0x7f478c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f4797fff640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7f478406d540 0x7f478406f980 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:29.997 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:29.994+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== osd.0 v2:172.21.15.82:6800/2082960483 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f47880acec0 con 0x7f478406d540 2024-04-25T10:14:30.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.006+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 --> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4798150960 con 0x7f478406d540 2024-04-25T10:14:30.008 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.006+0000 7f477ffff640 1 -- 172.21.15.82:0/427997531 <== osd.0 v2:172.21.15.82:6800/2082960483 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4798150960 con 0x7f478406d540 2024-04-25T10:14:30.008 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:30.008 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:30.009 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:30.009 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7f478406d540 msgr2=0x7f478406f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6800/2082960483,v1:172.21.15.82:6801/2082960483] conn(0x7f478406d540 0x7f478406f980 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4784066f60 msgr2=0x7f4784069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f4784066f60 0x7f4784069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f479814f7e0 tx=0x7f478c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 msgr2=0x7f479814cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 --2- 172.21.15.82:0/427997531 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f479814c060 0x7f479814cff0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f478801c610 tx=0x7f4780005840 comp rx=0 tx=0).stop 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f4797fff640 1 -- 172.21.15.82:0/427997531 reap_dead start 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 shutdown_connections 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 >> 172.21.15.82:0/427997531 conn(0x7f47980cc680 msgr2=0x7f47980b09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 shutdown_connections 2024-04-25T10:14:30.010 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.010+0000 7f479db2e640 1 -- 172.21.15.82:0/427997531 wait complete. 2024-04-25T10:14:30.024 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:14:30.137 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 Processor -- start 2024-04-25T10:14:30.138 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 -- start start 2024-04-25T10:14:30.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fe02415ba20 0x7fe0240cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe02406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02415c260 con 0x7fe02406b1d0 2024-04-25T10:14:30.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02406b640 con 0x7fe02415ba20 2024-04-25T10:14:30.140 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe02b6d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0240d1430 con 0x7fe02415fd90 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029c73640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fe02406b1d0 0x7fe0240cf6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43238/0 (socket says 172.21.15.82:43238) 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029c73640 1 -- 172.21.15.82:0/1070236783 learned_addr learned my addr 172.21.15.82:0/1070236783 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe02406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029472640 1 --2- 172.21.15.82:0/1070236783 >> v2:172.21.15.196:3300/0 conn(0x7fe02415ba20 0x7fe0240cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931164337 0 0) 0x7fe02415c260 con 0x7fe02406b1d0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe010003530 con 0x7fe02406b1d0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 580691520 0 0) 0x7fe010003530 con 0x7fe02406b1d0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 msgr2=0x7fe02406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe02406ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 >> v2:172.21.15.196:3300/0 conn(0x7fe02415ba20 msgr2=0x7fe0240cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 --2- 172.21.15.82:0/1070236783 >> v2:172.21.15.196:3300/0 conn(0x7fe02415ba20 0x7fe0240cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02415c540 con 0x7fe02406b1d0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fe020001680 con 0x7fe02406b1d0 2024-04-25T10:14:30.141 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe02406ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 >> v1:172.21.15.82:6789/0 conn(0x7fe02406b1d0 legacy=0x7fe0240cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe010003a30 0x7fe010003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe010004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe0100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02406b640 con 0x7fe010004810 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000b9d0 con 0x7fe010003a30 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01000bb50 con 0x7fe02415fd90 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe010004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029472640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe010003a30 0x7fe010003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029c73640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe0100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 -- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 msgr2=0x7fe0100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.142 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe0100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 -- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe010003a30 msgr2=0x7fe010003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe010003a30 0x7fe010003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe028c71640 1 -- 172.21.15.82:0/1070236783 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe010003530 con 0x7fe010004810 2024-04-25T10:14:30.143 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.138+0000 7fe029c73640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe02415fd90 0x7fe0100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe028c71640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe010004c90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fe018009cb0 tx=0x7fe01801cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbeb096ed7f2078 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0180b2070 con 0x7fe010004810 2024-04-25T10:14:30.144 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0180a56f0 con 0x7fe010004810 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe017fff640 1 -- 172.21.15.82:0/1070236783 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018009880 con 0x7fe010004810 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 -- 172.21.15.82:0/1070236783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 msgr2=0x7fe010004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/1070236783 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe010004c90 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7fe018009cb0 tx=0x7fe01801cc10 comp rx=0 tx=0).stop 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe029c73640 1 -- 172.21.15.82:0/1070236783 reap_dead start 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 -- 172.21.15.82:0/1070236783 shutdown_connections 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 -- 172.21.15.82:0/1070236783 >> 172.21.15.82:0/1070236783 conn(0x7fe024099a70 msgr2=0x7fe0240d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 -- 172.21.15.82:0/1070236783 shutdown_connections 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 -- 172.21.15.82:0/1070236783 wait complete. 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.142+0000 7fe02b6d6640 1 Processor -- start 2024-04-25T10:14:30.145 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- start start 2024-04-25T10:14:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 0x7fe024158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.147 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0240b51c0 0x7fe024158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415cb00 con 0x7fe010004810 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0241601f0 con 0x7fe0240b4d40 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02415be00 con 0x7fe0240b51c0 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60716/0 (socket says 172.21.15.82:60716) 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 -- 172.21.15.82:0/3720472943 learned_addr learned my addr 172.21.15.82:0/3720472943 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe028c71640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 0x7fe024158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 -- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0240b51c0 msgr2=0x7fe024158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0240b51c0 0x7fe024158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 -- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 msgr2=0x7fe024158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 0x7fe024158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 -- 172.21.15.82:0/3720472943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0240b5720 con 0x7fe010004810 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe028c71640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 0x7fe024158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.148 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe029472640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fe01c007f60 tx=0x7fe01c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c01a020 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c016470 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0241e5220 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c016860 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0241542b0 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02404efa0 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe01c01d020 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 0x7fe008069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe01c085ea0 con 0x7fe010004810 2024-04-25T10:14:30.149 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe028c71640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 0x7fe008069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.150 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.146+0000 7fe028c71640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 0x7fe008069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe0180af000 tx=0x7fe0180046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.151 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.150+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe01c024020 con 0x7fe010004810 2024-04-25T10:14:30.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.310+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe02406b7c0 con 0x7fe008066f90 2024-04-25T10:14:30.310 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.310+0000 7fe015ffb640 1 -- 172.21.15.82:0/3720472943 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe02406b7c0 con 0x7fe008066f90 2024-04-25T10:14:30.311 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:14:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 msgr2=0x7fe008069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 0x7fe008069450 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fe0180af000 tx=0x7fe0180046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 msgr2=0x7fe024153d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7fe01c007f60 tx=0x7fe01c007f90 comp rx=0 tx=0).stop 2024-04-25T10:14:30.314 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 shutdown_connections 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fe008066f90 0x7fe008069450 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fe0240b51c0 0x7fe024158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fe0240b4d40 0x7fe024158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 --2- 172.21.15.82:0/3720472943 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fe010004810 0x7fe024153d70 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 >> 172.21.15.82:0/3720472943 conn(0x7fe024099a70 msgr2=0x7fe0240ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 shutdown_connections 2024-04-25T10:14:30.315 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.314+0000 7fe02b6d6640 1 -- 172.21.15.82:0/3720472943 wait complete. 2024-04-25T10:14:30.329 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:14:30.330 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:30.457 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 Processor -- start 2024-04-25T10:14:30.458 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 -- start start 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 0x7f7df4159f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f7df40cd000 0x7f7df40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7df40b3bd0 con 0x7f7df40ae5e0 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7df40b3eb0 con 0x7f7df40cd000 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7dfb718640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40b4190 con 0x7f7df40b36d0 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df9cb5640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f7df40ae5e0 0x7f7df406d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43242/0 (socket says 172.21.15.82:43242) 2024-04-25T10:14:30.461 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df9cb5640 1 -- 172.21.15.82:0/3079416682 learned_addr learned my addr 172.21.15.82:0/3079416682 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3079416682 >> v2:172.21.15.196:3300/0 conn(0x7f7df40cd000 0x7f7df40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 0x7f7df4159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663160674 0 0) 0x7f7df40b3bd0 con 0x7f7df40ae5e0 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7de0003530 con 0x7f7df40ae5e0 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056809702 0 0) 0x7f7de0003530 con 0x7f7df40ae5e0 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 msgr2=0x7f7df4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 0x7f7df4159f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 >> v2:172.21.15.196:3300/0 conn(0x7f7df40cd000 msgr2=0x7f7df40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.462 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 --2- 172.21.15.82:0/3079416682 >> v2:172.21.15.196:3300/0 conn(0x7f7df40cd000 0x7f7df40cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df406cf80 con 0x7f7df40ae5e0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f7dec001a00 con 0x7f7df40ae5e0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 >> v1:172.21.15.82:6789/0 conn(0x7f7df40ae5e0 legacy=0x7f7df406d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00039c0 0x7f7de0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de0004810 0x7f7de0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 0x7f7de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40b3eb0 con 0x7f7de00039c0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40b4190 con 0x7f7de0004810 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de000b9d0 con 0x7f7df40b36d0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00039c0 0x7f7de0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de0004810 0x7f7de0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.463 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 -- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 msgr2=0x7f7de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df40b36d0 0x7f7de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 -- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de0004810 msgr2=0x7f7de0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de0004810 0x7f7de0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df94b4640 1 -- 172.21.15.82:0/3079416682 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de0003530 con 0x7f7de00039c0 2024-04-25T10:14:30.464 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.458+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de0004810 0x7f7de0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00039c0 0x7f7de0003de0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009ae0 tx=0x7f7df001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a38ffc17f5ab174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.469 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00b2070 con 0x7f7de00039c0 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df00a5590 con 0x7f7de00039c0 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7debfff640 1 -- 172.21.15.82:0/3079416682 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df00096b0 con 0x7f7de00039c0 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- 172.21.15.82:0/3079416682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00039c0 msgr2=0x7f7de0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 --2- 172.21.15.82:0/3079416682 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7de00039c0 0x7f7de0003de0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f7df0009ae0 tx=0x7f7df001ca80 comp rx=0 tx=0).stop 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7df9cb5640 1 -- 172.21.15.82:0/3079416682 reap_dead start 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- 172.21.15.82:0/3079416682 shutdown_connections 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- 172.21.15.82:0/3079416682 >> 172.21.15.82:0/3079416682 conn(0x7f7df406ae60 msgr2=0x7f7df406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- 172.21.15.82:0/3079416682 shutdown_connections 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- 172.21.15.82:0/3079416682 wait complete. 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 Processor -- start 2024-04-25T10:14:30.470 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.462+0000 7f7dfb718640 1 -- start start 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de00039c0 0x7f7df4098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 0x7f7df40c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df41f6210 0x7f7df415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40cfbd0 con 0x7f7df41565e0 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40cfa50 con 0x7f7de00039c0 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df40cbbb0 con 0x7f7df41f6210 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 0x7f7df40c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 0x7f7df40c4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60736/0 (socket says 172.21.15.82:60736) 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 -- 172.21.15.82:0/3954140919 learned_addr learned my addr 172.21.15.82:0/3954140919 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de00039c0 0x7f7df4098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df9cb5640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df41f6210 0x7f7df415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.471 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 -- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df41f6210 msgr2=0x7f7df415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f7df41f6210 0x7f7df415c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 -- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de00039c0 msgr2=0x7f7df4098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de00039c0 0x7f7df4098880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df41f6710 con 0x7f7df41565e0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f7de00039c0 0x7f7df4098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df8cb3640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 0x7f7df40c4eb0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7de4012ac0 tx=0x7f7de4012f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de4010d50 con 0x7f7df41565e0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7de4004020 con 0x7f7df41565e0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7de40043d0 con 0x7f7df41565e0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df41f69f0 con 0x7f7df41565e0 2024-04-25T10:14:30.472 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df41f6fb0 con 0x7f7df41565e0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7df4067c30 con 0x7f7df41565e0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7de4003620 con 0x7f7df41565e0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dd8066f60 0x7f7dd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f7de4087ec0 con 0x7f7df41565e0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.466+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dd8066f60 0x7f7dd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7de9ffb640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f7dd806d540 0x7f7dd806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7de4001d40 con 0x7f7dd806d540 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7de4088350 con 0x7f7df41565e0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7df9cb5640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f7dd806d540 0x7f7dd806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.473 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7df94b4640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dd8066f60 0x7f7dd8069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7df00af000 tx=0x7f7df00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7df9cb5640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f7dd806d540 0x7f7dd806f980 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.474 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.470+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== osd.1 v2:172.21.15.82:6816/1010291016 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7de4001d40 con 0x7f7dd806d540 2024-04-25T10:14:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 --> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7df40c6420 con 0x7f7dd806d540 2024-04-25T10:14:30.483 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7de9ffb640 1 -- 172.21.15.82:0/3954140919 <== osd.1 v2:172.21.15.82:6816/1010291016 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7df40c6420 con 0x7f7dd806d540 2024-04-25T10:14:30.483 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:30.483 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:30.483 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f7dd806d540 msgr2=0x7f7dd806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.484 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6816/1010291016,v1:172.21.15.82:6817/1010291016] conn(0x7f7dd806d540 0x7f7dd806f980 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dd8066f60 msgr2=0x7f7dd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f7dd8066f60 0x7f7dd8069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f7df00af000 tx=0x7f7df00046d0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 msgr2=0x7f7df40c4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 --2- 172.21.15.82:0/3954140919 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f7df41565e0 0x7f7df40c4eb0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f7de4012ac0 tx=0x7f7de4012f80 comp rx=0 tx=0).stop 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7df9cb5640 1 -- 172.21.15.82:0/3954140919 reap_dead start 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 shutdown_connections 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 >> 172.21.15.82:0/3954140919 conn(0x7f7df406ae60 msgr2=0x7f7df4071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 shutdown_connections 2024-04-25T10:14:30.485 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.482+0000 7f7dfb718640 1 -- 172.21.15.82:0/3954140919 wait complete. 2024-04-25T10:14:30.499 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:14:30.617 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 Processor -- start 2024-04-25T10:14:30.618 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 -- start start 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb7015b950 0x7fbb7009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb700cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7015bd30 con 0x7fbb700cfb80 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb700cc0d0 con 0x7fbb7015b950 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb76701640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7009ab10 con 0x7fbb700cf420 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb7015b950 0x7fbb7009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb7015b950 0x7fbb7009a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60748/0 (socket says 172.21.15.82:60748) 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 -- 172.21.15.82:0/2116303813 learned_addr learned my addr 172.21.15.82:0/2116303813 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6f7fe640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb700cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.619 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 -- 172.21.15.82:0/2116303813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 msgr2=0x7fbb700cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb700cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 -- 172.21.15.82:0/2116303813 >> v1:172.21.15.196:6789/0 conn(0x7fbb700cfb80 legacy=0x7fbb7006f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.618+0000 7fbb6ffff640 1 -- 172.21.15.82:0/2116303813 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700cc370 con 0x7fbb7015b950 2024-04-25T10:14:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6ffff640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7015b950 0x7fbb7009a480 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009870 tx=0x7fbb6001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efd5d8c7c546a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6effd640 1 -- 172.21.15.82:0/2116303813 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600ab640 con 0x7fbb7015b950 2024-04-25T10:14:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6effd640 1 -- 172.21.15.82:0/2116303813 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb600b0070 con 0x7fbb7015b950 2024-04-25T10:14:30.621 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6effd640 1 -- 172.21.15.82:0/2116303813 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb600aba00 con 0x7fbb7015b950 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- 172.21.15.82:0/2116303813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7015b950 msgr2=0x7fbb7009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7015b950 0x7fbb7009a480 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009870 tx=0x7fbb6001c530 comp rx=0 tx=0).stop 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- 172.21.15.82:0/2116303813 shutdown_connections 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb700cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- 172.21.15.82:0/2116303813 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb7015b950 0x7fbb7009a480 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- 172.21.15.82:0/2116303813 >> 172.21.15.82:0/2116303813 conn(0x7fbb700b30b0 msgr2=0x7fbb700b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- 172.21.15.82:0/2116303813 shutdown_connections 2024-04-25T10:14:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- 172.21.15.82:0/2116303813 wait complete. 2024-04-25T10:14:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 Processor -- start 2024-04-25T10:14:30.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- start start 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb7015a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb7015b950 0x7fbb70154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700cd390 con 0x7fbb700cfb80 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700d03a0 con 0x7fbb7015b950 2024-04-25T10:14:30.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb76701640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb700c7800 con 0x7fbb700cf420 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60758/0 (socket says 172.21.15.82:60758) 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.622+0000 7fbb6f7fe640 1 -- 172.21.15.82:0/4189333469 learned_addr learned my addr 172.21.15.82:0/4189333469 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb74c9e640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb7015b950 0x7fbb70154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6ffff640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb7015a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 -- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 msgr2=0x7fbb7015a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb7015a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 -- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb7015b950 msgr2=0x7fbb70154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb7015b950 0x7fbb70154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 -- 172.21.15.82:0/4189333469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700c7ae0 con 0x7fbb700cfb80 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6f7fe640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400d5f0 tx=0x7fbb6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640125f0 con 0x7fbb700cfb80 2024-04-25T10:14:30.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64019070 con 0x7fbb700cfb80 2024-04-25T10:14:30.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640129b0 con 0x7fbb700cfb80 2024-04-25T10:14:30.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb700c7dc0 con 0x7fbb700cfb80 2024-04-25T10:14:30.627 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb700c5ed0 con 0x7fbb700cfb80 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7004efa0 con 0x7fbb700cfb80 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb64012b50 con 0x7fbb700cfb80 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbb64085290 con 0x7fbb700cfb80 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6ffff640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.626+0000 7fbb6ffff640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 0x7fbb5c069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb600a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.630+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb640563f0 con 0x7fbb700cfb80 2024-04-25T10:14:30.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.790+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb700d13a0 con 0x7fbb5c066f60 2024-04-25T10:14:30.790 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.790+0000 7fbb6cff9640 1 -- 172.21.15.82:0/4189333469 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbb700d13a0 con 0x7fbb5c066f60 2024-04-25T10:14:30.790 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:14:30.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 msgr2=0x7fbb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 0x7fbb5c069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fbb600af000 tx=0x7fbb600a7000 comp rx=0 tx=0).stop 2024-04-25T10:14:30.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 msgr2=0x7fbb700c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400d5f0 tx=0x7fbb6400dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.793 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 shutdown_connections 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb5c066f60 0x7fbb5c069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb7015b950 0x7fbb70154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb700cfb80 0x7fbb700c5990 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 --2- 172.21.15.82:0/4189333469 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb700cf420 0x7fbb7015a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 >> 172.21.15.82:0/4189333469 conn(0x7fbb700b30b0 msgr2=0x7fbb7015aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 shutdown_connections 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.794+0000 7fbb76701640 1 -- 172.21.15.82:0/4189333469 wait complete. 2024-04-25T10:14:30.809 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:14:30.809 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:30.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 Processor -- start 2024-04-25T10:14:30.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 -- start start 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f07dc0d12c0 0x7f07dc0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc15dd60 con 0x7f07dc15d980 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07dc0d45e0 con 0x7f07dc0d21f0 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07e1c99640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07dc06c810 con 0x7f07dc0d12c0 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07db7fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f07dc15d980 0x7f07dc0d3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43254/0 (socket says 172.21.15.82:43254) 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07db7fe640 1 -- 172.21.15.82:0/2677064049 learned_addr learned my addr 172.21.15.82:0/2677064049 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4204840108 0 0) 0x7f07dc15dd60 con 0x7f07dc15d980 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07c4003530 con 0x7f07dc15d980 2024-04-25T10:14:30.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.930+0000 7f07daffd640 1 --2- 172.21.15.82:0/2677064049 >> v2:172.21.15.196:3300/0 conn(0x7f07dc0d12c0 0x7f07dc0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2778589366 0 0) 0x7f07c4003530 con 0x7f07dc15d980 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 >> v2:172.21.15.196:3300/0 conn(0x7f07dc0d12c0 msgr2=0x7f07dc0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 --2- 172.21.15.82:0/2677064049 >> v2:172.21.15.196:3300/0 conn(0x7f07dc0d12c0 0x7f07dc0d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 >> v1:172.21.15.196:6789/0 conn(0x7f07dc0d21f0 legacy=0x7f07dc06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0b02c0 con 0x7f07dc15d980 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f07d0001680 con 0x7f07dc15d980 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 >> v1:172.21.15.82:6789/0 conn(0x7f07dc15d980 legacy=0x7f07dc0d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c40051d0 0x7f07c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc06c810 con 0x7f07c4004810 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400c170 con 0x7f07c40039c0 2024-04-25T10:14:30.935 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c400c370 con 0x7f07c40051d0 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07dbfff640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c40051d0 0x7f07c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07db7fe640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 -- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c40051d0 msgr2=0x7f07c400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c40051d0 0x7f07c400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 -- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c40039c0 msgr2=0x7f07c4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 -- 172.21.15.82:0/2677064049 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c4003530 con 0x7f07c4004810 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07db7fe640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07dbfff640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07c40051d0 0x7f07c400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07dbfff640 1 -- 172.21.15.82:0/2677064049 reap_dead start 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07daffd640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ab0 tx=0x7f07cc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29566b632bef6151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0096b0 con 0x7f07c4004810 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07cc0a6430 con 0x7f07c4004810 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07da7fc640 1 -- 172.21.15.82:0/2677064049 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0a6820 con 0x7f07c4004810 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 -- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 msgr2=0x7f07c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7f07cc009ab0 tx=0x7f07cc01ca50 comp rx=0 tx=0).stop 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 -- 172.21.15.82:0/2677064049 shutdown_connections 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 --2- 172.21.15.82:0/2677064049 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 -- 172.21.15.82:0/2677064049 >> 172.21.15.82:0/2677064049 conn(0x7f07dc08b000 msgr2=0x7f07dc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 -- 172.21.15.82:0/2677064049 shutdown_connections 2024-04-25T10:14:30.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.934+0000 7f07e1c99640 1 -- 172.21.15.82:0/2677064049 wait complete. 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 Processor -- start 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- start start 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c4004810 0x7f07dc1f78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07dc0c7220 0x7f07dc0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 0x7f07dc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0d4210 con 0x7f07dc1549d0 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0d2600 con 0x7f07c4004810 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc0d4390 con 0x7f07dc0c7220 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 0x7f07dc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 0x7f07dc0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60776/0 (socket says 172.21.15.82:60776) 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 -- 172.21.15.82:0/1509489265 learned_addr learned my addr 172.21.15.82:0/1509489265 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07daffd640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07dc0c7220 0x7f07dc0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07db7fe640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c4004810 0x7f07dc1f78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 -- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07dc0c7220 msgr2=0x7f07dc0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f07dc0c7220 0x7f07dc0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 -- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c4004810 msgr2=0x7f07dc1f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f07c4004810 0x7f07dc1f78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07dc0c79e0 con 0x7f07dc1549d0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07dbfff640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 0x7f07dc0cc990 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f07dc08d0d0 tx=0x7f07d400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4018070 con 0x7f07dc1549d0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d4003db0 con 0x7f07dc1549d0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc0c7cc0 con 0x7f07dc1549d0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d4005310 con 0x7f07dc1549d0 2024-04-25T10:14:30.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc159290 con 0x7f07dc1549d0 2024-04-25T10:14:30.946 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.938+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f07dc067c30 con 0x7f07dc1549d0 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07d401b070 con 0x7f07dc1549d0 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07c8066f60 0x7f07c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f07d4084d00 con 0x7f07dc1549d0 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07db7fe640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07c8066f60 0x7f07c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7f07c806d540 0x7f07c806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:30.947 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f07d4016e50 con 0x7f07c806d540 2024-04-25T10:14:30.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f07d4085190 con 0x7f07dc1549d0 2024-04-25T10:14:30.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07daffd640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7f07c806d540 0x7f07c806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:30.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07db7fe640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07c8066f60 0x7f07c8069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f07d0000c10 tx=0x7f07d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07daffd640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7f07c806d540 0x7f07c806f980 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:30.948 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.942+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== osd.2 v2:172.21.15.82:6803/3086763100 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f07d4016e50 con 0x7f07c806d540 2024-04-25T10:14:30.955 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.954+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 --> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f07dc0b84b0 con 0x7f07c806d540 2024-04-25T10:14:30.956 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.954+0000 7f07c3fff640 1 -- 172.21.15.82:0/1509489265 <== osd.2 v2:172.21.15.82:6803/3086763100 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f07dc0b84b0 con 0x7f07c806d540 2024-04-25T10:14:30.956 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:30.956 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:30.956 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:30.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.954+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7f07c806d540 msgr2=0x7f07c806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.954+0000 7f07e1c99640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6803/3086763100,v1:172.21.15.82:6805/3086763100] conn(0x7f07c806d540 0x7f07c806f980 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:30.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07c8066f60 msgr2=0x7f07c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.957 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f07c8066f60 0x7f07c8069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f07d0000c10 tx=0x7f07d0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 msgr2=0x7f07dc0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 --2- 172.21.15.82:0/1509489265 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f07dc1549d0 0x7f07dc0cc990 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7f07dc08d0d0 tx=0x7f07d400d890 comp rx=0 tx=0).stop 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07dbfff640 1 -- 172.21.15.82:0/1509489265 reap_dead start 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 shutdown_connections 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 >> 172.21.15.82:0/1509489265 conn(0x7f07dc08b000 msgr2=0x7f07dc15ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 shutdown_connections 2024-04-25T10:14:30.958 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:30.958+0000 7f07e1c99640 1 -- 172.21.15.82:0/1509489265 wait complete. 2024-04-25T10:14:30.972 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:14:31.086 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 Processor -- start 2024-04-25T10:14:31.087 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 -- start start 2024-04-25T10:14:31.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f923c15a700 0x7f923c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.088 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f923c0b24a0 0x7f923c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c0b2090 con 0x7f923c0b3f50 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c15f200 con 0x7f923c15a700 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f924348b640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0b43e0 con 0x7f923c0b24a0 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f9241227640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f923c15a700 0x7f923c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f9241227640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f923c15a700 0x7f923c15ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60792/0 (socket says 172.21.15.82:60792) 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.086+0000 7f9241227640 1 -- 172.21.15.82:0/576984136 learned_addr learned my addr 172.21.15.82:0/576984136 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241a28640 1 --2- 172.21.15.82:0/576984136 >> v2:172.21.15.196:3300/0 conn(0x7f923c0b24a0 0x7f923c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241227640 1 -- 172.21.15.82:0/576984136 >> v2:172.21.15.196:3300/0 conn(0x7f923c0b24a0 msgr2=0x7f923c0b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241227640 1 --2- 172.21.15.82:0/576984136 >> v2:172.21.15.196:3300/0 conn(0x7f923c0b24a0 0x7f923c0b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241227640 1 -- 172.21.15.82:0/576984136 >> v1:172.21.15.196:6789/0 conn(0x7f923c0b3f50 legacy=0x7f923c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.089 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241227640 1 -- 172.21.15.82:0/576984136 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0b46c0 con 0x7f923c15a700 2024-04-25T10:14:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9241227640 1 --2- 172.21.15.82:0/576984136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c15a700 0x7f923c15ecc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f923c06bd20 tx=0x7f923801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7915834b02691920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9233fff640 1 -- 172.21.15.82:0/576984136 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380b0070 con 0x7f923c15a700 2024-04-25T10:14:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9233fff640 1 -- 172.21.15.82:0/576984136 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92380ab600 con 0x7f923c15a700 2024-04-25T10:14:31.091 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f9233fff640 1 -- 172.21.15.82:0/576984136 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380ab9d0 con 0x7f923c15a700 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 -- 172.21.15.82:0/576984136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c15a700 msgr2=0x7f923c15ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 --2- 172.21.15.82:0/576984136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c15a700 0x7f923c15ecc0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7f923c06bd20 tx=0x7f923801c3a0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 -- 172.21.15.82:0/576984136 shutdown_connections 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 --2- 172.21.15.82:0/576984136 >> v2:172.21.15.196:3300/0 conn(0x7f923c0b24a0 0x7f923c0b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 --2- 172.21.15.82:0/576984136 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c15a700 0x7f923c15ecc0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 -- 172.21.15.82:0/576984136 >> 172.21.15.82:0/576984136 conn(0x7f923c0b11a0 msgr2=0x7f923c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 -- 172.21.15.82:0/576984136 shutdown_connections 2024-04-25T10:14:31.092 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.090+0000 7f924348b640 1 -- 172.21.15.82:0/576984136 wait complete. 2024-04-25T10:14:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 Processor -- start 2024-04-25T10:14:31.093 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- start start 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c15a700 0x7f923c160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c160740 0x7f923c15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0cd310 con 0x7f923c0b24a0 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0ce600 con 0x7f923c15a700 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0cd490 con 0x7f923c160740 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60800/0 (socket says 172.21.15.82:60800) 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 -- 172.21.15.82:0/918370636 learned_addr learned my addr 172.21.15.82:0/918370636 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.097 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 -- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c160740 msgr2=0x7f923c15b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241a28640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c160740 0x7f923c15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9240a26640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c15a700 0x7f923c160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c160740 0x7f923c15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 -- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c15a700 msgr2=0x7f923c160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c15a700 0x7f923c160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 -- 172.21.15.82:0/918370636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c15c060 con 0x7f923c0b24a0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9241227640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f92380ae040 tx=0x7f923801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380ba020 con 0x7f923c0b24a0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92380b6640 con 0x7f923c0b24a0 2024-04-25T10:14:31.098 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f923c15c340 con 0x7f923c0b24a0 2024-04-25T10:14:31.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380b0040 con 0x7f923c0b24a0 2024-04-25T10:14:31.099 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c15c900 con 0x7f923c0b24a0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c160b50 con 0x7f923c0b24a0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.094+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92380bd020 con 0x7f923c0b24a0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.098+0000 7f9231ffb640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 0x7f9220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.098+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f923811f1e0 con 0x7f923c0b24a0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.098+0000 7f9240a26640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 0x7f9220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.098+0000 7f9240a26640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 0x7f9220069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f923c15a370 tx=0x7f922c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.102+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f923c160b50 con 0x7f923c0b24a0 2024-04-25T10:14:31.253 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f923c158420 con 0x7f9220066f60 2024-04-25T10:14:31.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f9231ffb640 1 -- 172.21.15.82:0/918370636 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f923c158420 con 0x7f9220066f60 2024-04-25T10:14:31.254 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 msgr2=0x7f9220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 0x7f9220069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f923c15a370 tx=0x7f922c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 msgr2=0x7f923c154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f92380ae040 tx=0x7f923801abe0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 shutdown_connections 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9220066f60 0x7f9220069420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f923c160740 0x7f923c15b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f923c15a700 0x7f923c160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 --2- 172.21.15.82:0/918370636 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f923c0b24a0 0x7f923c154ca0 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 >> 172.21.15.82:0/918370636 conn(0x7f923c0b11a0 msgr2=0x7f923c158940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 shutdown_connections 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.254+0000 7f924348b640 1 -- 172.21.15.82:0/918370636 wait complete. 2024-04-25T10:14:31.271 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:14:31.271 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:31.394 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 Processor -- start 2024-04-25T10:14:31.395 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 -- start start 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcbc0d12c0 0x7fbcbc0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc15dd60 con 0x7fbcbc0d21f0 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcbc0d45e0 con 0x7fbcbc15d980 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcc1ee4640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc06c810 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbaffd640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbcbc0d12c0 0x7fbcbc0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbbfff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fbcbc0d21f0 0x7fbcbc06cca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43264/0 (socket says 172.21.15.82:43264) 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbbfff640 1 -- 172.21.15.82:0/745937153 learned_addr learned my addr 172.21.15.82:0/745937153 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:31.396 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcba7fc640 1 -- 172.21.15.82:0/745937153 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949400168 0 0) 0x7fbcbc15dd60 con 0x7fbcbc0d21f0 2024-04-25T10:14:31.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcba7fc640 1 -- 172.21.15.82:0/745937153 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbca4003530 con 0x7fbcbc0d21f0 2024-04-25T10:14:31.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbaffd640 1 -- 172.21.15.82:0/745937153 >> v1:172.21.15.196:6789/0 conn(0x7fbcbc15d980 legacy=0x7fbcbc0d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbaffd640 1 -- 172.21.15.82:0/745937153 >> v1:172.21.15.82:6789/0 conn(0x7fbcbc0d21f0 legacy=0x7fbcbc06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.397 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.394+0000 7fbcbaffd640 1 -- 172.21.15.82:0/745937153 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0b02c0 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcbaffd640 1 --2- 172.21.15.82:0/745937153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc0d12c0 0x7fbcbc0d1cb0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097c0 tx=0x7fbcb001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6f81015d427019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcba7fc640 1 -- 172.21.15.82:0/745937153 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00b0070 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcba7fc640 1 -- 172.21.15.82:0/745937153 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb00ab600 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.398 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcba7fc640 1 -- 172.21.15.82:0/745937153 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00ab9b0 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/745937153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc0d12c0 msgr2=0x7fbcbc0d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 --2- 172.21.15.82:0/745937153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc0d12c0 0x7fbcbc0d1cb0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00097c0 tx=0x7fbcb001c410 comp rx=0 tx=0).stop 2024-04-25T10:14:31.399 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/745937153 shutdown_connections 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 --2- 172.21.15.82:0/745937153 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc0d12c0 0x7fbcbc0d1cb0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/745937153 >> 172.21.15.82:0/745937153 conn(0x7fbcbc08b000 msgr2=0x7fbcbc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/745937153 shutdown_connections 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.398+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/745937153 wait complete. 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 Processor -- start 2024-04-25T10:14:31.400 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- start start 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcbc0d12c0 0x7fbcbc1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcbc0d21f0 0x7fbcbc0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 0x7fbcbc0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16b640 con 0x7fbcbc15d980 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16b4c0 con 0x7fbcbc0d21f0 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16b7c0 con 0x7fbcbc0d12c0 2024-04-25T10:14:31.403 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 0x7fbcbc0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 0x7fbcbc0cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60824/0 (socket says 172.21.15.82:60824) 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 -- 172.21.15.82:0/587889915 learned_addr learned my addr 172.21.15.82:0/587889915 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbaffd640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcbc0d21f0 0x7fbcbc0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 -- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcbc0d12c0 msgr2=0x7fbcbc1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbb7fe640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcbc0d12c0 0x7fbcbc1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbcbc0d12c0 0x7fbcbc1598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 -- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcbc0d21f0 msgr2=0x7fbcbc0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcbc0d21f0 0x7fbcbc0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc159fe0 con 0x7fbcbc15d980 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbaffd640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbcbc0d21f0 0x7fbcbc0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbbfff640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 0x7fbcbc0cce90 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400caa0 tx=0x7fbcb400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4017070 con 0x7fbcbc15d980 2024-04-25T10:14:31.404 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb400ac00 con 0x7fbcbc15d980 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc15a2c0 con 0x7fbcbc15d980 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4012400 con 0x7fbcbc15d980 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc15a880 con 0x7fbcbc15d980 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbcbc15ac60 con 0x7fbcbc15d980 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbcb401a070 con 0x7fbcbc15d980 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbca8066f60 0x7fbca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbcb4084d50 con 0x7fbcbc15d980 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbb7fe640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbca8066f60 0x7fbca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7fbca806d540 0x7fbca806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbcb4014ec0 con 0x7fbca806d540 2024-04-25T10:14:31.406 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb4087450 con 0x7fbcbc15d980 2024-04-25T10:14:31.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.402+0000 7fbcbaffd640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7fbca806d540 0x7fbca806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.406+0000 7fbcbb7fe640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbca8066f60 0x7fbca8069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc15b6c0 tx=0x7fbcac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.406+0000 7fbcbaffd640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7fbca806d540 0x7fbca806f980 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.407 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.406+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== osd.3 v2:172.21.15.82:6824/3269667522 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbcb4014ec0 con 0x7fbca806d540 2024-04-25T10:14:31.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 --> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbcbc067c30 con 0x7fbca806d540 2024-04-25T10:14:31.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbca3fff640 1 -- 172.21.15.82:0/587889915 <== osd.3 v2:172.21.15.82:6824/3269667522 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbcbc067c30 con 0x7fbca806d540 2024-04-25T10:14:31.419 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:31.419 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:31.419 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7fbca806d540 msgr2=0x7fbca806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6824/3269667522,v1:172.21.15.82:6825/3269667522] conn(0x7fbca806d540 0x7fbca806f980 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbca8066f60 msgr2=0x7fbca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbca8066f60 0x7fbca8069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc15b6c0 tx=0x7fbcac009040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 msgr2=0x7fbcbc0cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 --2- 172.21.15.82:0/587889915 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbcbc15d980 0x7fbcbc0cce90 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400caa0 tx=0x7fbcb400cf60 comp rx=0 tx=0).stop 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcbbfff640 1 -- 172.21.15.82:0/587889915 reap_dead start 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 shutdown_connections 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 >> 172.21.15.82:0/587889915 conn(0x7fbcbc08b000 msgr2=0x7fbcbc15df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 shutdown_connections 2024-04-25T10:14:31.421 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.418+0000 7fbcc1ee4640 1 -- 172.21.15.82:0/587889915 wait complete. 2024-04-25T10:14:31.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:14:31.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 Processor -- start 2024-04-25T10:14:31.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 -- start start 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff24c0ce510 0x7ff24c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ff24c144ca0 0x7ff24c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24c0cee30 con 0x7ff24c06ec60 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c0cc2b0 con 0x7ff24c0ce510 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff253820640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24c06e290 con 0x7ff24c144ca0 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff2515bc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7ff24c0ce510 0x7ff24c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff251dbd640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7ff24c06ec60 0x7ff24c142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43276/0 (socket says 172.21.15.82:43276) 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff251dbd640 1 -- 172.21.15.82:0/1257065672 learned_addr learned my addr 172.21.15.82:0/1257065672 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff250dbb640 1 --2- 172.21.15.82:0/1257065672 >> v2:172.21.15.196:3300/0 conn(0x7ff24c144ca0 0x7ff24c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff2515bc640 1 -- 172.21.15.82:0/1257065672 >> v2:172.21.15.196:3300/0 conn(0x7ff24c144ca0 msgr2=0x7ff24c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff2515bc640 1 --2- 172.21.15.82:0/1257065672 >> v2:172.21.15.196:3300/0 conn(0x7ff24c144ca0 0x7ff24c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff2515bc640 1 -- 172.21.15.82:0/1257065672 >> v1:172.21.15.82:6789/0 conn(0x7ff24c06ec60 legacy=0x7ff24c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff2515bc640 1 -- 172.21.15.82:0/1257065672 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c0ccc70 con 0x7ff24c0ce510 2024-04-25T10:14:31.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.550+0000 7ff243fff640 1 -- 172.21.15.82:0/1257065672 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 466559664 0 0) 0x7ff24c0cee30 con 0x7ff24c06ec60 2024-04-25T10:14:31.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff2515bc640 1 --2- 172.21.15.82:0/1257065672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c0ce8f0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7ff24c070fe0 tx=0x7ff24801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d13ca97e891043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff243fff640 1 -- 172.21.15.82:0/1257065672 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2480b5070 con 0x7ff24c0ce510 2024-04-25T10:14:31.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff243fff640 1 -- 172.21.15.82:0/1257065672 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2480a8970 con 0x7ff24c0ce510 2024-04-25T10:14:31.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff243fff640 1 -- 172.21.15.82:0/1257065672 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2480a8d40 con 0x7ff24c0ce510 2024-04-25T10:14:31.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/1257065672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 msgr2=0x7ff24c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- 172.21.15.82:0/1257065672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c0ce8f0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7ff24c070fe0 tx=0x7ff24801ec70 comp rx=0 tx=0).stop 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/1257065672 shutdown_connections 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- 172.21.15.82:0/1257065672 >> v2:172.21.15.196:3300/0 conn(0x7ff24c144ca0 0x7ff24c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- 172.21.15.82:0/1257065672 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c0ce8f0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/1257065672 >> 172.21.15.82:0/1257065672 conn(0x7ff24c0b11a0 msgr2=0x7ff24c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/1257065672 shutdown_connections 2024-04-25T10:14:31.554 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/1257065672 wait complete. 2024-04-25T10:14:31.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 Processor -- start 2024-04-25T10:14:31.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- start start 2024-04-25T10:14:31.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 0x7ff24c134c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff24c144ca0 0x7ff24c1e9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c0cc900 con 0x7ff24c0ce510 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c06df20 con 0x7ff24c144ca0 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c142d90 con 0x7ff24c06ec60 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60838/0 (socket says 172.21.15.82:60838) 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 -- 172.21.15.82:0/2816270326 learned_addr learned my addr 172.21.15.82:0/2816270326 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff251dbd640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff24c144ca0 0x7ff24c1e9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff2515bc640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 0x7ff24c134c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 -- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 msgr2=0x7ff24c134c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 0x7ff24c134c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 -- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff24c144ca0 msgr2=0x7ff24c1e9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff24c144ca0 0x7ff24c1e9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 -- 172.21.15.82:0/2816270326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c1e72e0 con 0x7ff24c0ce510 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff2515bc640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 0x7ff24c134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff250dbb640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00d5d0 tx=0x7ff23c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c004240 con 0x7ff24c0ce510 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c0043e0 con 0x7ff24c0ce510 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c1e75c0 con 0x7ff24c0ce510 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.554+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003840 con 0x7ff24c0ce510 2024-04-25T10:14:31.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c1e7b80 con 0x7ff24c0ce510 2024-04-25T10:14:31.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c04efa0 con 0x7ff24c0ce510 2024-04-25T10:14:31.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff23c0039e0 con 0x7ff24c0ce510 2024-04-25T10:14:31.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff241ffb640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 0x7ff230069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff23c085c70 con 0x7ff24c0ce510 2024-04-25T10:14:31.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff2515bc640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 0x7ff230069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.560 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.558+0000 7ff2515bc640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 0x7ff230069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff2480b2000 tx=0x7ff2480aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.562 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.562+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff23c056e20 con 0x7ff24c0ce510 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.706+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff24c06e7c0 con 0x7ff230066f90 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff241ffb640 1 -- 172.21.15.82:0/2816270326 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff24c06e7c0 con 0x7ff230066f90 2024-04-25T10:14:31.709 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 msgr2=0x7ff230069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 0x7ff230069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7ff2480b2000 tx=0x7ff2480aa070 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 msgr2=0x7ff24c1e6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00d5d0 tx=0x7ff23c00da90 comp rx=0 tx=0).stop 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 shutdown_connections 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ff230066f90 0x7ff230069450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ff24c144ca0 0x7ff24c1e9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ff24c0ce510 0x7ff24c1e6870 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 --2- 172.21.15.82:0/2816270326 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ff24c06ec60 0x7ff24c134c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 >> 172.21.15.82:0/2816270326 conn(0x7ff24c0b11a0 msgr2=0x7ff24c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 shutdown_connections 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.710+0000 7ff253820640 1 -- 172.21.15.82:0/2816270326 wait complete. 2024-04-25T10:14:31.726 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:14:31.726 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 Processor -- start 2024-04-25T10:14:31.838 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 -- start start 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f98340ae6c0 0x7f98340b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f98340cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f983415e0f0 con 0x7f983415a3c0 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98340cfea0 con 0x7f98340ae6c0 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f983a149640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b24a0 con 0x7f98340cf510 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f983415a3c0 0x7f983415a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43284/0 (socket says 172.21.15.82:43284) 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 -- 172.21.15.82:0/1866471086 learned_addr learned my addr 172.21.15.82:0/1866471086 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f9832ffd640 1 --2- 172.21.15.82:0/1866471086 >> v2:172.21.15.196:3300/0 conn(0x7f98340ae6c0 0x7f98340b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.841 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f9833fff640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f98340cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407896243 0 0) 0x7f983415e0f0 con 0x7f983415a3c0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f981c003530 con 0x7f983415a3c0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277020117 0 0) 0x7f981c003530 con 0x7f983415a3c0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 msgr2=0x7f98340cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f98340cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 >> v2:172.21.15.196:3300/0 conn(0x7f98340ae6c0 msgr2=0x7f98340b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 --2- 172.21.15.82:0/1866471086 >> v2:172.21.15.196:3300/0 conn(0x7f98340ae6c0 0x7f98340b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983415b0b0 con 0x7f983415a3c0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f9824001680 con 0x7f983415a3c0 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 >> v1:172.21.15.82:6789/0 conn(0x7f983415a3c0 legacy=0x7f983415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f981c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.842 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f981c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c004d50 0x7f981c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340cfea0 con 0x7f981c0039c0 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b24a0 con 0x7f981c004d50 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f981c00dbe0 con 0x7f98340cf510 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f981c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f9833fff640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c004d50 0x7f981c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f9832ffd640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f981c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 -- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 msgr2=0x7f981c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340cf510 0x7f981c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 -- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c004d50 msgr2=0x7f981c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f981c004d50 0x7f981c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.843 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.838+0000 7f98337fe640 1 -- 172.21.15.82:0/1866471086 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f981c003530 con 0x7f981c0039c0 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f98337fe640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f981c003de0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f982400c210 tx=0x7f982401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ac0c82985791c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824007d60 con 0x7f981c0039c0 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98240a9020 con 0x7f981c0039c0 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f98327fc640 1 -- 172.21.15.82:0/1866471086 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005b20 con 0x7f981c0039c0 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- 172.21.15.82:0/1866471086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 msgr2=0x7f981c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 --2- 172.21.15.82:0/1866471086 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f981c003de0 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f982400c210 tx=0x7f982401cca0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f9833fff640 1 -- 172.21.15.82:0/1866471086 reap_dead start 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- 172.21.15.82:0/1866471086 shutdown_connections 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- 172.21.15.82:0/1866471086 >> 172.21.15.82:0/1866471086 conn(0x7f983406ae60 msgr2=0x7f983406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- 172.21.15.82:0/1866471086 shutdown_connections 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- 172.21.15.82:0/1866471086 wait complete. 2024-04-25T10:14:31.847 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 Processor -- start 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.842+0000 7f983a149640 1 -- start start 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f98340ae070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340ac170 0x7f9834155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98340ca7c0 0x7f98340cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b2ce0 con 0x7f981c0039c0 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98340aead0 con 0x7f98340ca7c0 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98340b2e60 con 0x7f98340ac170 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f98340ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f98340ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60862/0 (socket says 172.21.15.82:60862) 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 -- 172.21.15.82:0/3755283936 learned_addr learned my addr 172.21.15.82:0/3755283936 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9833fff640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98340ca7c0 0x7f98340cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9832ffd640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340ac170 0x7f9834155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 -- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340ac170 msgr2=0x7f9834155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.848 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340ac170 0x7f9834155290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 -- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98340ca7c0 msgr2=0x7f98340cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f98340ca7c0 0x7f98340cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340ac8e0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9832ffd640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f98340ac170 0x7f9834155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f98337fe640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f98340ae070 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f98240ab040 tx=0x7f981c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824005dc0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98240047f0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98340acbc0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9824004bc0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9834155bd0 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9834067c30 con 0x7f981c0039c0 2024-04-25T10:14:31.849 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f98240a5780 con 0x7f981c0039c0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9820066f90 0x7f9820069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f982411ea90 con 0x7f981c0039c0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9832ffd640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9820066f90 0x7f9820069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f982006d570 0x7f982006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9820070080 con 0x7f982006d570 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f982411ef20 con 0x7f981c0039c0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9833fff640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f982006d570 0x7f982006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.846+0000 7f9832ffd640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9820066f90 0x7f9820069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f98341fa020 tx=0x7f9828008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.850+0000 7f9833fff640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f982006d570 0x7f982006f9b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.851 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.850+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== osd.4 v2:172.21.15.196:6816/2164040014 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9820070080 con 0x7f982006d570 2024-04-25T10:14:31.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 --> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f98340c6140 con 0x7f982006d570 2024-04-25T10:14:31.863 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f981bfff640 1 -- 172.21.15.82:0/3755283936 <== osd.4 v2:172.21.15.196:6816/2164040014 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f98340c6140 con 0x7f982006d570 2024-04-25T10:14:31.864 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:31.864 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:31.864 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f982006d570 msgr2=0x7f982006f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.196:6816/2164040014,v1:172.21.15.196:6817/2164040014] conn(0x7f982006d570 0x7f982006f9b0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9820066f90 msgr2=0x7f9820069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f9820066f90 0x7f9820069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f98341fa020 tx=0x7f9828008040 comp rx=0 tx=0).stop 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 msgr2=0x7f98340ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 --2- 172.21.15.82:0/3755283936 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f981c0039c0 0x7f98340ae070 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f98240ab040 tx=0x7f981c005840 comp rx=0 tx=0).stop 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f9833fff640 1 -- 172.21.15.82:0/3755283936 reap_dead start 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 shutdown_connections 2024-04-25T10:14:31.865 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.862+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 >> 172.21.15.82:0/3755283936 conn(0x7f983406ae60 msgr2=0x7f983406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.866+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 shutdown_connections 2024-04-25T10:14:31.866 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.866+0000 7f983a149640 1 -- 172.21.15.82:0/3755283936 wait complete. 2024-04-25T10:14:31.879 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:14:31.989 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 Processor -- start 2024-04-25T10:14:31.990 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 -- start start 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 0x7ffa4c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa4c0af370 0x7ffa4c0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c06ec30 con 0x7ffa4c06bca0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c0669a0 con 0x7ffa4c0af370 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa52179640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06c5c0 con 0x7ffa4c066e30 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4affd640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 0x7ffa4c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4bfff640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7ffa4c0af370 0x7ffa4c0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4b7fe640 1 --1- >> v1:172.21.15.196:6789/0 conn(0x7ffa4c06bca0 0x7ffa4c0cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.196:6789/0 says I am v1:172.21.15.82:47904/0 (socket says 172.21.15.82:47904) 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4b7fe640 1 -- 172.21.15.82:0/279728846 learned_addr learned my addr 172.21.15.82:0/279728846 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4affd640 1 -- 172.21.15.82:0/279728846 >> v2:172.21.15.196:3300/0 conn(0x7ffa4c0af370 msgr2=0x7ffa4c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4affd640 1 --2- 172.21.15.82:0/279728846 >> v2:172.21.15.196:3300/0 conn(0x7ffa4c0af370 0x7ffa4c0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4affd640 1 -- 172.21.15.82:0/279728846 >> v1:172.21.15.196:6789/0 conn(0x7ffa4c06bca0 legacy=0x7ffa4c0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.990+0000 7ffa4affd640 1 -- 172.21.15.82:0/279728846 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c06f230 con 0x7ffa4c066e30 2024-04-25T10:14:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa4affd640 1 --2- 172.21.15.82:0/279728846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 0x7ffa4c06c080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097c0 tx=0x7ffa3401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c681e63dda749e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa4a7fc640 1 -- 172.21.15.82:0/279728846 <== mon.2 v2:172.21.15.196:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340ab600 con 0x7ffa4c066e30 2024-04-25T10:14:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa4a7fc640 1 -- 172.21.15.82:0/279728846 <== mon.2 v2:172.21.15.196:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa340b0070 con 0x7ffa4c066e30 2024-04-25T10:14:31.994 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa4a7fc640 1 -- 172.21.15.82:0/279728846 <== mon.2 v2:172.21.15.196:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340ab9c0 con 0x7ffa4c066e30 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 -- 172.21.15.82:0/279728846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 msgr2=0x7ffa4c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 --2- 172.21.15.82:0/279728846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 0x7ffa4c06c080 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa340097c0 tx=0x7ffa3401c3e0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 -- 172.21.15.82:0/279728846 shutdown_connections 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 --2- 172.21.15.82:0/279728846 >> v2:172.21.15.196:3300/0 conn(0x7ffa4c0af370 0x7ffa4c0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 --2- 172.21.15.82:0/279728846 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c066e30 0x7ffa4c06c080 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 -- 172.21.15.82:0/279728846 >> 172.21.15.82:0/279728846 conn(0x7ffa4c071810 msgr2=0x7ffa4c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 -- 172.21.15.82:0/279728846 shutdown_connections 2024-04-25T10:14:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.994+0000 7ffa52179640 1 -- 172.21.15.82:0/279728846 wait complete. 2024-04-25T10:14:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 Processor -- start 2024-04-25T10:14:31.996 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- start start 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa4c066e30 0x7ffa4c1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c06bca0 0x7ffa4c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1687b0 con 0x7ffa4c0af370 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c168630 con 0x7ffa4c066e30 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c168930 con 0x7ffa4c06bca0 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60876/0 (socket says 172.21.15.82:60876) 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 -- 172.21.15.82:0/591509121 learned_addr learned my addr 172.21.15.82:0/591509121 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa4c066e30 0x7ffa4c1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4affd640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c06bca0 0x7ffa4c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 -- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c06bca0 msgr2=0x7ffa4c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c06bca0 0x7ffa4c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 -- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa4c066e30 msgr2=0x7ffa4c1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa4c066e30 0x7ffa4c1466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 -- 172.21.15.82:0/591509121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c14d7d0 con 0x7ffa4c0af370 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa4bfff640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00efc0 tx=0x7ffa3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c018020 con 0x7ffa4c0af370 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c014480 con 0x7ffa4c0af370 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c147100 con 0x7ffa4c0af370 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c014870 con 0x7ffa4c0af370 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1476c0 con 0x7ffa4c0af370 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c04efa0 con 0x7ffa4c0af370 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:31.998+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa3c01b020 con 0x7ffa4c0af370 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.002+0000 7ffa43fff640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 0x7ffa2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.002+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ffa3c085d00 con 0x7ffa4c0af370 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.002+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 0x7ffa2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.002+0000 7ffa4b7fe640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 0x7ffa2c069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000f70 tx=0x7ffa44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.006+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa3c022030 con 0x7ffa4c0af370 2024-04-25T10:14:32.165 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4c147900 con 0x7ffa2c066f60 2024-04-25T10:14:32.166 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa43fff640 1 -- 172.21.15.82:0/591509121 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ffa44001350 con 0x7ffa2c066f60 2024-04-25T10:14:32.166 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:14:32.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 msgr2=0x7ffa2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.168 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 0x7ffa2c069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7ffa44000f70 tx=0x7ffa44009040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 msgr2=0x7ffa4c154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00efc0 tx=0x7ffa3c00c490 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 shutdown_connections 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7ffa2c066f60 0x7ffa2c069420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7ffa4c06bca0 0x7ffa4c147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7ffa4c0af370 0x7ffa4c154050 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.166+0000 7ffa52179640 1 --2- 172.21.15.82:0/591509121 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7ffa4c066e30 0x7ffa4c1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.170+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 >> 172.21.15.82:0/591509121 conn(0x7ffa4c071810 msgr2=0x7ffa4c06d870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.170+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 shutdown_connections 2024-04-25T10:14:32.169 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.170+0000 7ffa52179640 1 -- 172.21.15.82:0/591509121 wait complete. 2024-04-25T10:14:32.184 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:14:32.184 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 Processor -- start 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 -- start start 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400cf510 0x7fbb400cf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb400ac170 0x7fbb400ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb400b28c0 con 0x7fbb400adc70 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400cfe90 con 0x7fbb400ac170 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb4560d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015bb20 con 0x7fbb400cf510 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb400ac170 0x7fbb400ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fbb400ac170 0x7fbb400ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60882/0 (socket says 172.21.15.82:60882) 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3286246482 learned_addr learned my addr 172.21.15.82:0/3286246482 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3286246482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400cf510 msgr2=0x7fbb400cf950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 --2- 172.21.15.82:0/3286246482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400cf510 0x7fbb400cf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3286246482 >> v1:172.21.15.196:6789/0 conn(0x7fbb400adc70 legacy=0x7fbb4006e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3286246482 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb400b2b60 con 0x7fbb400ac170 2024-04-25T10:14:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3f7fe640 1 --2- 172.21.15.82:0/3286246482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400ac170 0x7fbb400ac600 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb3801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84cdfcb5cc141af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.298+0000 7fbb3e7fc640 1 -- 172.21.15.82:0/3286246482 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380a5760 con 0x7fbb400ac170 2024-04-25T10:14:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3e7fc640 1 -- 172.21.15.82:0/3286246482 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb380a5900 con 0x7fbb400ac170 2024-04-25T10:14:32.300 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3e7fc640 1 -- 172.21.15.82:0/3286246482 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380a5cd0 con 0x7fbb400ac170 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3286246482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400ac170 msgr2=0x7fbb400ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3286246482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400ac170 0x7fbb400ac600 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb3801c9f0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3286246482 shutdown_connections 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3286246482 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400cf510 0x7fbb400cf950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3286246482 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400ac170 0x7fbb400ac600 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3286246482 >> 172.21.15.82:0/3286246482 conn(0x7fbb4006ae60 msgr2=0x7fbb4006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3286246482 shutdown_connections 2024-04-25T10:14:32.301 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3286246482 wait complete. 2024-04-25T10:14:32.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 Processor -- start 2024-04-25T10:14:32.302 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- start start 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb400ac170 0x7fbb40097ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400adc70 0x7fbb40156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 0x7fbb40156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400b3c00 con 0x7fbb400cf510 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400b36e0 con 0x7fbb400ac170 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40158dd0 con 0x7fbb400adc70 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 0x7fbb40156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.304 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 0x7fbb40156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60892/0 (socket says 172.21.15.82:60892) 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3338622406 learned_addr learned my addr 172.21.15.82:0/3338622406 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3effd640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb400ac170 0x7fbb40097ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400adc70 msgr2=0x7fbb40156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fbb400adc70 0x7fbb40156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb400ac170 msgr2=0x7fbb40097ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fbb400ac170 0x7fbb40097ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4015def0 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb3f7fe640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 0x7fbb40156f70 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb38019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380b8670 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb380be070 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb40159020 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb380b8a70 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.302+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb401595e0 con 0x7fbb400cf510 2024-04-25T10:14:32.305 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb40067c30 con 0x7fbb400cf510 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb380b8c10 con 0x7fbb400cf510 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb20066f90 0x7fbb20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fbb38120a30 con 0x7fbb400cf510 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb3effd640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb20066f90 0x7fbb20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7fbb2006d570 0x7fbb2006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb380c3e50 con 0x7fbb2006d570 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb38120ec0 con 0x7fbb400cf510 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb3effd640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb20066f90 0x7fbb20069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb40159fb0 tx=0x7fbb30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb367fc640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7fbb2006d570 0x7fbb2006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.307 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.306+0000 7fbb367fc640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7fbb2006d570 0x7fbb2006f9b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.308 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.310+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== osd.5 v2:172.21.15.196:6800/1745719129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbb380c3e50 con 0x7fbb2006d570 2024-04-25T10:14:32.320 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 --> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb400c9670 con 0x7fbb2006d570 2024-04-25T10:14:32.321 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb37fff640 1 -- 172.21.15.82:0/3338622406 <== osd.5 v2:172.21.15.196:6800/1745719129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbb400c9670 con 0x7fbb2006d570 2024-04-25T10:14:32.321 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:32.322 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:32.322 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7fbb2006d570 msgr2=0x7fbb2006f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.196:6800/1745719129,v1:172.21.15.196:6802/1745719129] conn(0x7fbb2006d570 0x7fbb2006f9b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb20066f90 msgr2=0x7fbb20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fbb20066f90 0x7fbb20069450 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbb40159fb0 tx=0x7fbb30009040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 msgr2=0x7fbb40156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 --2- 172.21.15.82:0/3338622406 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fbb400cf510 0x7fbb40156f70 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009e50 tx=0x7fbb38019790 comp rx=0 tx=0).stop 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb3f7fe640 1 -- 172.21.15.82:0/3338622406 reap_dead start 2024-04-25T10:14:32.323 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 shutdown_connections 2024-04-25T10:14:32.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 >> 172.21.15.82:0/3338622406 conn(0x7fbb4006ae60 msgr2=0x7fbb400cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 shutdown_connections 2024-04-25T10:14:32.324 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.322+0000 7fbb4560d640 1 -- 172.21.15.82:0/3338622406 wait complete. 2024-04-25T10:14:32.336 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:14:32.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 Processor -- start 2024-04-25T10:14:32.441 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 -- start start 2024-04-25T10:14:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f804015da80 0x7f80400d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f80400d1cd0 0x7f80400d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.442 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804006be60 con 0x7f80400d0620 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80400d0a70 con 0x7f804015da80 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8047df3640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804006c440 con 0x7f80400d1cd0 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8045b8f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f804015da80 0x7f80400d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8046390640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f80400d0620 0x7f80400ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43286/0 (socket says 172.21.15.82:43286) 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8046390640 1 -- 172.21.15.82:0/304585248 learned_addr learned my addr 172.21.15.82:0/304585248 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f804538e640 1 --2- 172.21.15.82:0/304585248 >> v2:172.21.15.196:3300/0 conn(0x7f80400d1cd0 0x7f80400d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8045b8f640 1 -- 172.21.15.82:0/304585248 >> v2:172.21.15.196:3300/0 conn(0x7f80400d1cd0 msgr2=0x7f80400d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8045b8f640 1 --2- 172.21.15.82:0/304585248 >> v2:172.21.15.196:3300/0 conn(0x7f80400d1cd0 0x7f80400d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8045b8f640 1 -- 172.21.15.82:0/304585248 >> v1:172.21.15.82:6789/0 conn(0x7f80400d0620 legacy=0x7f80400ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8044b8d640 1 -- 172.21.15.82:0/304585248 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565582561 0 0) 0x7f804006be60 con 0x7f80400d0620 2024-04-25T10:14:32.443 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.442+0000 7f8045b8f640 1 -- 172.21.15.82:0/304585248 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804015de60 con 0x7f804015da80 2024-04-25T10:14:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8045b8f640 1 --2- 172.21.15.82:0/304585248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804015da80 0x7f80400d4230 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcedcefccde85e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8044b8d640 1 -- 172.21.15.82:0/304585248 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300b0070 con 0x7f804015da80 2024-04-25T10:14:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8044b8d640 1 -- 172.21.15.82:0/304585248 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80300ab600 con 0x7f804015da80 2024-04-25T10:14:32.444 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8044b8d640 1 -- 172.21.15.82:0/304585248 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80300ab9d0 con 0x7f804015da80 2024-04-25T10:14:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/304585248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804015da80 msgr2=0x7f80400d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.445 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- 172.21.15.82:0/304585248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804015da80 0x7f80400d4230 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f80300097c0 tx=0x7f803001c460 comp rx=0 tx=0).stop 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/304585248 shutdown_connections 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- 172.21.15.82:0/304585248 >> v2:172.21.15.196:3300/0 conn(0x7f80400d1cd0 0x7f80400d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- 172.21.15.82:0/304585248 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f804015da80 0x7f80400d4230 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/304585248 >> 172.21.15.82:0/304585248 conn(0x7f804008b000 msgr2=0x7f80400d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/304585248 shutdown_connections 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/304585248 wait complete. 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 Processor -- start 2024-04-25T10:14:32.446 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- start start 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80400d0620 0x7f80400cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f804015da80 0x7f8040156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804016a730 con 0x7f80400d1cd0 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804016a5b0 con 0x7f80400d0620 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804016a8b0 con 0x7f804015da80 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60898/0 (socket says 172.21.15.82:60898) 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 -- 172.21.15.82:0/4020990460 learned_addr learned my addr 172.21.15.82:0/4020990460 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8045b8f640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80400d0620 0x7f80400cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8046390640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f804015da80 0x7f8040156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 -- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f804015da80 msgr2=0x7f8040156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.449 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f804015da80 0x7f8040156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 -- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80400d0620 msgr2=0x7f80400cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80400d0620 0x7f80400cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 -- 172.21.15.82:0/4020990460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80400c7950 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f804538e640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f803c00d830 tx=0x7f803c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c0059a0 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c005b40 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80400c7c30 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c005d30 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.446+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80400c81f0 con 0x7f80400d1cd0 2024-04-25T10:14:32.450 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804004efa0 con 0x7f80400d1cd0 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f803c01b020 con 0x7f80400d1cd0 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f80367fc640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 0x7f8024069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f803c084f30 con 0x7f80400d1cd0 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f8045b8f640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 0x7f8024069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.451 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.450+0000 7f8045b8f640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 0x7f8024069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80300ad000 tx=0x7f80300a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.454 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.454+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f803c056090 con 0x7f80400d1cd0 2024-04-25T10:14:32.611 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.610+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f80400b1a30 con 0x7f8024066f60 2024-04-25T10:14:32.612 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.610+0000 7f80367fc640 1 -- 172.21.15.82:0/4020990460 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f80400b1a30 con 0x7f8024066f60 2024-04-25T10:14:32.612 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 msgr2=0x7f8024069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 0x7f8024069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80300ad000 tx=0x7f80300a6000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 msgr2=0x7f80400c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f803c00d830 tx=0x7f803c00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 shutdown_connections 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f8024066f60 0x7f8024069420 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f804015da80 0x7f8040156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.614 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f80400d1cd0 0x7f80400c71e0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 --2- 172.21.15.82:0/4020990460 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f80400d0620 0x7f80400cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 >> 172.21.15.82:0/4020990460 conn(0x7f804008b000 msgr2=0x7f80400d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 shutdown_connections 2024-04-25T10:14:32.615 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.614+0000 7f8047df3640 1 -- 172.21.15.82:0/4020990460 wait complete. 2024-04-25T10:14:32.630 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:14:32.630 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 Processor -- start 2024-04-25T10:14:32.743 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 -- start start 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2f80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fa2f80cd000 0x7fa2f80b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f80b3bd0 con 0x7fa2f8159f20 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f80b3eb0 con 0x7fa2f80cd000 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2fe1fa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80b4190 con 0x7fa2f80b36d0 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.742+0000 7fa2f77fe640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fa2f8159f20 0x7fa2f806d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43302/0 (socket says 172.21.15.82:43302) 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f77fe640 1 -- 172.21.15.82:0/2764556729 learned_addr learned my addr 172.21.15.82:0/2764556729 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:32.746 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2f80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f7fff640 1 --2- 172.21.15.82:0/2764556729 >> v2:172.21.15.196:3300/0 conn(0x7fa2f80cd000 0x7fa2f80b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205636167 0 0) 0x7fa2f80b3bd0 con 0x7fa2f8159f20 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e0003530 con 0x7fa2f8159f20 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1753593785 0 0) 0x7fa2e0003530 con 0x7fa2f8159f20 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 msgr2=0x7fa2f80b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2f80b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 >> v2:172.21.15.196:3300/0 conn(0x7fa2f80cd000 msgr2=0x7fa2f80b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 --2- 172.21.15.82:0/2764556729 >> v2:172.21.15.196:3300/0 conn(0x7fa2f80cd000 0x7fa2f80b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f806cf80 con 0x7fa2f8159f20 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fa2e8001a00 con 0x7fa2f8159f20 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 >> v1:172.21.15.82:6789/0 conn(0x7fa2f8159f20 legacy=0x7fa2f806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.747 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e0004d50 0x7fa2e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80b3eb0 con 0x7fa2e0003f40 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80b4190 con 0x7fa2e0004d50 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e000b9d0 con 0x7fa2f80b36d0 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f77fe640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 -- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 msgr2=0x7fa2e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f80b36d0 0x7fa2e00039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 -- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e0004d50 msgr2=0x7fa2e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.748 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2e0004d50 0x7fa2e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.749 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 -- 172.21.15.82:0/2764556729 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e0003530 con 0x7fa2e0003f40 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2e0004810 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009650 tx=0x7fa2ec01ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9673da4e4635ac10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0b4070 con 0x7fa2e0003f40 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec0a7880 con 0x7fa2e0003f40 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f67fc640 1 -- 172.21.15.82:0/2764556729 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec009690 con 0x7fa2e0003f40 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/2764556729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 msgr2=0x7fa2e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 --2- 172.21.15.82:0/2764556729 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2e0004810 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec009650 tx=0x7fa2ec01ec20 comp rx=0 tx=0).stop 2024-04-25T10:14:32.753 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2f7fff640 1 -- 172.21.15.82:0/2764556729 reap_dead start 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/2764556729 shutdown_connections 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/2764556729 >> 172.21.15.82:0/2764556729 conn(0x7fa2f806ae60 msgr2=0x7fa2f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/2764556729 shutdown_connections 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.746+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/2764556729 wait complete. 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 Processor -- start 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- start start 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2f8158bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2f80ca7c0 0x7fa2f815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f815b7c0 0x7fa2f80b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f806de90 con 0x7fa2e0003f40 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f815a300 con 0x7fa2f80ca7c0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f80ae5e0 con 0x7fa2f815b7c0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2f8158bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2f8158bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60926/0 (socket says 172.21.15.82:60926) 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 -- 172.21.15.82:0/829871298 learned_addr learned my addr 172.21.15.82:0/829871298 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2f80ca7c0 0x7fa2f815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f7fff640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f815b7c0 0x7fa2f80b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 -- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f815b7c0 msgr2=0x7fa2f80b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fa2f815b7c0 0x7fa2f80b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 -- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2f80ca7c0 msgr2=0x7fa2f815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fa2f80ca7c0 0x7fa2f815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f80cb180 con 0x7fa2e0003f40 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2f77fe640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2f8158bc0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8001c70 tx=0x7fa2e800e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e801b070 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e8011040 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f80cb460 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e80143a0 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f815ad60 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.750+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2f815d2b0 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2e801e070 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2e4066f90 0x7fa2e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa2e8088100 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2e4066f90 0x7fa2e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7fa2e406d570 0x7fa2e406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2e8016e50 con 0x7fa2e406d570 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2e8088590 con 0x7fa2e0003f40 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2f6ffd640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2e4066f90 0x7fa2e4069450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b1000 tx=0x7fa2ec010040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.756 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2f7fff640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7fa2e406d570 0x7fa2e406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2f7fff640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7fa2e406d570 0x7fa2e406f9b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.757 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.754+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== osd.6 v2:172.21.15.196:6801/1119498737 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa2e8016e50 con 0x7fa2e406d570 2024-04-25T10:14:32.768 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 --> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa2f8067c30 con 0x7fa2e406d570 2024-04-25T10:14:32.769 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2dffff640 1 -- 172.21.15.82:0/829871298 <== osd.6 v2:172.21.15.196:6801/1119498737 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa2f8067c30 con 0x7fa2e406d570 2024-04-25T10:14:32.769 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:32.769 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:32.769 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:32.770 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7fa2e406d570 msgr2=0x7fa2e406f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.196:6801/1119498737,v1:172.21.15.196:6803/1119498737] conn(0x7fa2e406d570 0x7fa2e406f9b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2e4066f90 msgr2=0x7fa2e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fa2e4066f90 0x7fa2e4069450 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec0b1000 tx=0x7fa2ec010040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 msgr2=0x7fa2f8158bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 --2- 172.21.15.82:0/829871298 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fa2e0003f40 0x7fa2f8158bc0 secure :-1 s=READY pgs=2321 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8001c70 tx=0x7fa2e800e100 comp rx=0 tx=0).stop 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2f7fff640 1 -- 172.21.15.82:0/829871298 reap_dead start 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 shutdown_connections 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 >> 172.21.15.82:0/829871298 conn(0x7fa2f806ae60 msgr2=0x7fa2f8159290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 shutdown_connections 2024-04-25T10:14:32.771 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.770+0000 7fa2fe1fa640 1 -- 172.21.15.82:0/829871298 wait complete. 2024-04-25T10:14:32.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:14:32.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 Processor -- start 2024-04-25T10:14:32.889 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 -- start start 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f9a0015aac0 0x7f9a000ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a000cf0e0 0x7f9a000cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a0015aea0 0x7f9a0015b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000ce0d0 con 0x7f9a000cf0e0 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a000ce320 con 0x7f9a0015aac0 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f9a0483f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000ce600 con 0x7f9a0015aea0 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a000cf0e0 0x7f9a000cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f9a000cf0e0 0x7f9a000cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60928/0 (socket says 172.21.15.82:60928) 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 -- 172.21.15.82:0/979693561 learned_addr learned my addr 172.21.15.82:0/979693561 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.891 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fed9d640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a0015aea0 0x7f9a0015b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fe59c640 1 --2- 172.21.15.82:0/979693561 >> v2:172.21.15.196:3300/0 conn(0x7f9a0015aac0 0x7f9a000ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 -- 172.21.15.82:0/979693561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a0015aea0 msgr2=0x7f9a0015b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a0015aea0 0x7f9a0015b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 -- 172.21.15.82:0/979693561 >> v2:172.21.15.196:3300/0 conn(0x7f9a0015aac0 msgr2=0x7f9a000ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/979693561 >> v2:172.21.15.196:3300/0 conn(0x7f9a0015aac0 0x7f9a000ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.892 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.890+0000 7f99fdd9b640 1 -- 172.21.15.82:0/979693561 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000ce8e0 con 0x7f9a000cf0e0 2024-04-25T10:14:32.893 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a000cf0e0 0x7f9a000cd180 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f9a00072f80 tx=0x7f99e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365c76d6078a96eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f99fd59a640 1 -- 172.21.15.82:0/979693561 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80ab5f0 con 0x7f9a000cf0e0 2024-04-25T10:14:32.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f99fd59a640 1 -- 172.21.15.82:0/979693561 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80b0070 con 0x7f9a000cf0e0 2024-04-25T10:14:32.894 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f99fd59a640 1 -- 172.21.15.82:0/979693561 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80ab9b0 con 0x7f9a000cf0e0 2024-04-25T10:14:32.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 -- 172.21.15.82:0/979693561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a000cf0e0 msgr2=0x7f9a000cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a000cf0e0 0x7f9a000cd180 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f9a00072f80 tx=0x7f99e801c520 comp rx=0 tx=0).stop 2024-04-25T10:14:32.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 -- 172.21.15.82:0/979693561 shutdown_connections 2024-04-25T10:14:32.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a0015aea0 0x7f9a0015b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.895 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 --2- 172.21.15.82:0/979693561 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a000cf0e0 0x7f9a000cd180 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 --2- 172.21.15.82:0/979693561 >> v2:172.21.15.196:3300/0 conn(0x7f9a0015aac0 0x7f9a000ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 -- 172.21.15.82:0/979693561 >> 172.21.15.82:0/979693561 conn(0x7f9a00099a70 msgr2=0x7f9a000d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 -- 172.21.15.82:0/979693561 shutdown_connections 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.894+0000 7f9a0483f640 1 -- 172.21.15.82:0/979693561 wait complete. 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 Processor -- start 2024-04-25T10:14:32.896 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- start start 2024-04-25T10:14:32.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a000cf0e0 0x7f9a00163690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a0015aea0 0x7f9a001f9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.898 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015ee10 con 0x7f9a0015aac0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015ec90 con 0x7f9a0015aea0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015ef90 con 0x7f9a000cf0e0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60940/0 (socket says 172.21.15.82:60940) 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 -- 172.21.15.82:0/645520263 learned_addr learned my addr 172.21.15.82:0/645520263 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fe59c640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a000cf0e0 0x7f9a00163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fed9d640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a0015aea0 0x7f9a001f9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 -- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a000cf0e0 msgr2=0x7f9a00163690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a000cf0e0 0x7f9a00163690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 -- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a0015aea0 msgr2=0x7f9a001f9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a0015aea0 0x7f9a001f9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 -- 172.21.15.82:0/645520263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00163dc0 con 0x7f9a0015aac0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fdd9b640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f99e801abd0 tx=0x7f99e801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.899 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b4800 con 0x7f9a0015aac0 2024-04-25T10:14:32.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80b49a0 con 0x7f9a0015aac0 2024-04-25T10:14:32.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a001640a0 con 0x7f9a0015aac0 2024-04-25T10:14:32.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b4d90 con 0x7f9a0015aac0 2024-04-25T10:14:32.900 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a00164660 con 0x7f9a0015aac0 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0004efa0 con 0x7f9a0015aac0 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99e80c7390 con 0x7f9a0015aac0 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 0x7f99e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99fe59c640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 0x7f99e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.898+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f99e811f520 con 0x7f9a0015aac0 2024-04-25T10:14:32.901 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.902+0000 7f99fe59c640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 0x7f99e0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99f8003fd0 tx=0x7f99f80039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.904 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:32.906+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99e80c4060 con 0x7f9a0015aac0 2024-04-25T10:14:33.048 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.046+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a000cdc90 con 0x7f99e0066f60 2024-04-25T10:14:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f99f6ffd640 1 -- 172.21.15.82:0/645520263 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9a000cdc90 con 0x7f99e0066f60 2024-04-25T10:14:33.049 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 msgr2=0x7f99e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 0x7f99e0069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f99f8003fd0 tx=0x7f99f80039a0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 msgr2=0x7f9a000c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f99e801abd0 tx=0x7f99e801ad60 comp rx=0 tx=0).stop 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 shutdown_connections 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f99e0066f60 0x7f99e0069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f9a0015aea0 0x7f9a001f9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f9a0015aac0 0x7f9a000c8680 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.051 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 --2- 172.21.15.82:0/645520263 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f9a000cf0e0 0x7f9a00163690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 >> 172.21.15.82:0/645520263 conn(0x7f9a00099a70 msgr2=0x7f9a000caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 shutdown_connections 2024-04-25T10:14:33.052 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.050+0000 7f9a0483f640 1 -- 172.21.15.82:0/645520263 wait complete. 2024-04-25T10:14:33.066 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:14:33.066 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:14:33.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 Processor -- start 2024-04-25T10:14:33.189 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 -- start start 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7f22b815b840 0x7f22b809a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22b80b1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b80cd400 con 0x7f22b80ccbf0 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b80b08e0 con 0x7f22b815b840 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bffb2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b1180 con 0x7f22b809a180 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f22b80ccbf0 0x7f22b8099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43316/0 (socket says 172.21.15.82:43316) 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 -- 172.21.15.82:0/1847288461 learned_addr learned my addr 172.21.15.82:0/1847288461 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bdd4e640 1 --2- 172.21.15.82:0/1847288461 >> v2:172.21.15.196:3300/0 conn(0x7f22b815b840 0x7f22b809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22be54f640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22b80b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.193 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221037678 0 0) 0x7f22b80cd400 con 0x7f22b80ccbf0 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22a4003530 con 0x7f22b80ccbf0 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3329826662 0 0) 0x7f22a4003530 con 0x7f22b80ccbf0 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 msgr2=0x7f22b80b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22b80b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 >> v2:172.21.15.196:3300/0 conn(0x7f22b815b840 msgr2=0x7f22b809a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 --2- 172.21.15.82:0/1847288461 >> v2:172.21.15.196:3300/0 conn(0x7f22b815b840 0x7f22b809a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b80b0b30 con 0x7f22b80ccbf0 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7f22a8001680 con 0x7f22b80ccbf0 2024-04-25T10:14:33.194 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 >> v1:172.21.15.82:6789/0 conn(0x7f22b80ccbf0 legacy=0x7f22b8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22a40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a4003f40 0x7f22a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4004d50 0x7f22a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b08e0 con 0x7f22a4003f40 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b1180 con 0x7f22a4004d50 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a400ba20 con 0x7f22b809a180 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a4003f40 0x7f22a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.195 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bdd4e640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22a40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22be54f640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4004d50 0x7f22a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 -- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 msgr2=0x7f22a40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22b809a180 0x7f22a40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 -- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4004d50 msgr2=0x7f22a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4004d50 0x7f22a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22bd54d640 1 -- 172.21.15.82:0/1847288461 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4003530 con 0x7f22a4003f40 2024-04-25T10:14:33.196 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.190+0000 7f22be54f640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22a4004d50 0x7f22a40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bd54d640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a4003f40 0x7f22a4004810 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c270 tx=0x7f22a801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b0a98bd7d66bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8007d60 con 0x7f22a4003f40 2024-04-25T10:14:33.200 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a80a9020 con 0x7f22a4003f40 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bcd4c640 1 -- 172.21.15.82:0/1847288461 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80a5b10 con 0x7f22a4003f40 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- 172.21.15.82:0/1847288461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a4003f40 msgr2=0x7f22a4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 --2- 172.21.15.82:0/1847288461 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22a4003f40 0x7f22a4004810 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c270 tx=0x7f22a801cc90 comp rx=0 tx=0).stop 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22be54f640 1 -- 172.21.15.82:0/1847288461 reap_dead start 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- 172.21.15.82:0/1847288461 shutdown_connections 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- 172.21.15.82:0/1847288461 >> 172.21.15.82:0/1847288461 conn(0x7f22b80b3030 msgr2=0x7f22b80b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- 172.21.15.82:0/1847288461 shutdown_connections 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- 172.21.15.82:0/1847288461 wait complete. 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 Processor -- start 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- start start 2024-04-25T10:14:33.201 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a4003f40 0x7f22b815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 0x7f22b80ae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b814bd50 0x7f22b814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b1f80 con 0x7f22b815beb0 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b1e00 con 0x7f22b814bd50 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.194+0000 7f22bffb2640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80b2100 con 0x7f22a4003f40 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 0x7f22b80ae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 0x7f22b80ae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60968/0 (socket says 172.21.15.82:60968) 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 -- 172.21.15.82:0/461166878 learned_addr learned my addr 172.21.15.82:0/461166878 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bdd4e640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a4003f40 0x7f22b815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.202 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22be54f640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b814bd50 0x7f22b814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 -- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a4003f40 msgr2=0x7f22b815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f22a4003f40 0x7f22b815e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 -- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b814bd50 msgr2=0x7f22b814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f22b814bd50 0x7f22b814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b815ea20 con 0x7f22b815beb0 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bd54d640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 0x7f22b80ae3d0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c270 tx=0x7f22a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8001ca0 con 0x7f22b815beb0 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a80a5b10 con 0x7f22b815beb0 2024-04-25T10:14:33.203 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b815ed00 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a80047f0 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b80c8b40 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22b814c740 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22a8004990 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f229c066f60 0x7f229c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f22a811eb50 con 0x7f22b815beb0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bdd4e640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f229c066f60 0x7f229c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7f229c06d540 0x7f229c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22a8008e50 con 0x7f229c06d540 2024-04-25T10:14:33.204 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_get_version_reply(handle=1 version=126) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22a81232c0 con 0x7f22b815beb0 2024-04-25T10:14:33.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22bdd4e640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f229c066f60 0x7f229c069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009a30 tx=0x7f22b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22be54f640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7f229c06d540 0x7f229c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.198+0000 7f22be54f640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7f229c06d540 0x7f229c06f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.205 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.202+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== osd.7 v2:172.21.15.196:6822/2552340661 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f22a8008e50 con 0x7f229c06d540 2024-04-25T10:14:33.213 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 --> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f22b8068340 con 0x7f229c06d540 2024-04-25T10:14:33.214 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22ae7fc640 1 -- 172.21.15.82:0/461166878 <== osd.7 v2:172.21.15.196:6822/2552340661 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f22b8068340 con 0x7f229c06d540 2024-04-25T10:14:33.214 INFO:teuthology.orchestra.run.smithi082.stdout:{ 2024-04-25T10:14:33.215 INFO:teuthology.orchestra.run.smithi082.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:14:33.215 INFO:teuthology.orchestra.run.smithi082.stdout:} 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7f229c06d540 msgr2=0x7f229c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.196:6822/2552340661,v1:172.21.15.196:6823/2552340661] conn(0x7f229c06d540 0x7f229c06f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f229c066f60 msgr2=0x7f229c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f229c066f60 0x7f229c069420 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009a30 tx=0x7f22b4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 msgr2=0x7f22b80ae3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 --2- 172.21.15.82:0/461166878 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f22b815beb0 0x7f22b80ae3d0 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c270 tx=0x7f22a4005840 comp rx=0 tx=0).stop 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22be54f640 1 -- 172.21.15.82:0/461166878 reap_dead start 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 shutdown_connections 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 >> 172.21.15.82:0/461166878 conn(0x7f22b80b3030 msgr2=0x7f22b81567e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.216 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 shutdown_connections 2024-04-25T10:14:33.217 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.214+0000 7f22bffb2640 1 -- 172.21.15.82:0/461166878 wait complete. 2024-04-25T10:14:33.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:14:33.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 Processor -- start 2024-04-25T10:14:33.333 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 -- start start 2024-04-25T10:14:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c90071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.334 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c900b24a0 0x7f0c90071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c900adbd0 con 0x7f0c90071e90 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c900ae860 con 0x7f0c900b24a0 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c95660640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9015e210 con 0x7f0c9015da10 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c900b24a0 0x7f0c90071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f0c900b24a0 0x7f0c90071570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60984/0 (socket says 172.21.15.82:60984) 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/3942364705 learned_addr learned my addr 172.21.15.82:0/3942364705 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8effd640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c90071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/3942364705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 msgr2=0x7f0c90071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c90071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/3942364705 >> v1:172.21.15.196:6789/0 conn(0x7f0c90071e90 legacy=0x7f0c901614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.335 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.334+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/3942364705 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9015e4f0 con 0x7f0c900b24a0 2024-04-25T10:14:33.336 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c90071570 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009870 tx=0x7f0c8401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efaf988423d16a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8dffb640 1 -- 172.21.15.82:0/3942364705 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840b0070 con 0x7f0c900b24a0 2024-04-25T10:14:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8dffb640 1 -- 172.21.15.82:0/3942364705 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c840ab600 con 0x7f0c900b24a0 2024-04-25T10:14:33.337 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8dffb640 1 -- 172.21.15.82:0/3942364705 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840ab9d0 con 0x7f0c900b24a0 2024-04-25T10:14:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/3942364705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 msgr2=0x7f0c90071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c90071570 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009870 tx=0x7f0c8401c440 comp rx=0 tx=0).stop 2024-04-25T10:14:33.338 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/3942364705 shutdown_connections 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c90071570 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- 172.21.15.82:0/3942364705 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c90071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/3942364705 >> 172.21.15.82:0/3942364705 conn(0x7f0c900b11a0 msgr2=0x7f0c900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/3942364705 shutdown_connections 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/3942364705 wait complete. 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 Processor -- start 2024-04-25T10:14:33.339 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- start start 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c90071e90 0x7f0c90157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c900b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9016aa10 con 0x7f0c900b24a0 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9016a890 con 0x7f0c90071e90 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9016ab90 con 0x7f0c9015da10 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:60988/0 (socket says 172.21.15.82:60988) 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/774993830 learned_addr learned my addr 172.21.15.82:0/774993830 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:33.342 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8effd640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c90071e90 0x7f0c90157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8f7fe640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c900b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 msgr2=0x7f0c900b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c900b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c90071e90 msgr2=0x7f0c90157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c90071e90 0x7f0c90157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 -- 172.21.15.82:0/774993830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c901608a0 con 0x7f0c900b24a0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c8e7fc640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f0c840ae040 tx=0x7f0c840191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840b0040 con 0x7f0c900b24a0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c840b4420 con 0x7f0c900b24a0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c90156760 con 0x7f0c900b24a0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840b4810 con 0x7f0c900b24a0 2024-04-25T10:14:33.343 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.338+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c90156d20 con 0x7f0c900b24a0 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9004efa0 con 0x7f0c900b24a0 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c840b49b0 con 0x7f0c900b24a0 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c777fe640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 0x7f0c7c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f0c8411f5d0 con 0x7f0c900b24a0 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c8effd640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 0x7f0c7c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.344 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.342+0000 7f0c8effd640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 0x7f0c7c069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0c900ae460 tx=0x7f0c88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.345 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.346+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c901608a0 con 0x7f0c900b24a0 2024-04-25T10:14:33.495 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.494+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c901685f0 con 0x7f0c7c066f60 2024-04-25T10:14:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.494+0000 7f0c777fe640 1 -- 172.21.15.82:0/774993830 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0c901685f0 con 0x7f0c7c066f60 2024-04-25T10:14:33.496 INFO:teuthology.orchestra.run.smithi082.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:14:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 msgr2=0x7f0c7c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 0x7f0c7c069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f0c900ae460 tx=0x7f0c88008040 comp rx=0 tx=0).stop 2024-04-25T10:14:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 msgr2=0x7f0c9015ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f0c840ae040 tx=0x7f0c840191e0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.498 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 shutdown_connections 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f0c7c066f60 0x7f0c7c069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f0c9015da10 0x7f0c900b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f0c900b24a0 0x7f0c9015ca70 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 --2- 172.21.15.82:0/774993830 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f0c90071e90 0x7f0c90157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 >> 172.21.15.82:0/774993830 conn(0x7f0c900b11a0 msgr2=0x7f0c900b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 shutdown_connections 2024-04-25T10:14:33.499 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.498+0000 7f0c95660640 1 -- 172.21.15.82:0/774993830 wait complete. 2024-04-25T10:14:33.513 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 Processor -- start 2024-04-25T10:14:33.620 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 -- start start 2024-04-25T10:14:33.622 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f84480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84481451f0 0x7f8448142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84480cee30 con 0x7f8448144ca0 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84480cc2b0 con 0x7f84481451f0 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f844d9cb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844806e290 con 0x7f84480ce510 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f84481451f0 0x7f8448142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f843ffff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f8448144ca0 0x7f844806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:43318/0 (socket says 172.21.15.82:43318) 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 --2- 172.21.15.82:0/2800731853 >> v2:172.21.15.82:3300/0 conn(0x7f84481451f0 0x7f8448142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32772/0 (socket says 172.21.15.82:32772) 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f843ffff640 1 -- 172.21.15.82:0/2800731853 learned_addr learned my addr 172.21.15.82:0/2800731853 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f8446ffd640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f84480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 -- 172.21.15.82:0/2800731853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 msgr2=0x7f84480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f84480ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 -- 172.21.15.82:0/2800731853 >> v1:172.21.15.82:6789/0 conn(0x7f8448144ca0 legacy=0x7f844806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.623 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 -- 172.21.15.82:0/2800731853 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480ccc70 con 0x7f84481451f0 2024-04-25T10:14:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84467fc640 1 -- 172.21.15.82:0/2800731853 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1846809814 0 0) 0x7f84480cee30 con 0x7f8448144ca0 2024-04-25T10:14:33.624 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f8446ffd640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f84480ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.622+0000 7f84477fe640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84481451f0 0x7f8448142680 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f8440009d90 tx=0x7f844001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa24e0a00db72ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f84467fc640 1 -- 172.21.15.82:0/2800731853 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400b4070 con 0x7f84481451f0 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f84467fc640 1 -- 172.21.15.82:0/2800731853 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84400af600 con 0x7f84481451f0 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f84467fc640 1 -- 172.21.15.82:0/2800731853 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84400af9d0 con 0x7f84481451f0 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/2800731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84481451f0 msgr2=0x7f8448142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84481451f0 0x7f8448142680 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f8440009d90 tx=0x7f844001f150 comp rx=0 tx=0).stop 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/2800731853 shutdown_connections 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f84481451f0 0x7f8448142680 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.625 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- 172.21.15.82:0/2800731853 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f84480ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/2800731853 >> 172.21.15.82:0/2800731853 conn(0x7f84480b11a0 msgr2=0x7f84480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/2800731853 shutdown_connections 2024-04-25T10:14:33.626 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/2800731853 wait complete. 2024-04-25T10:14:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 Processor -- start 2024-04-25T10:14:33.628 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- start start 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f8448140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 0x7f84480c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844806ee30 con 0x7f8448144ca0 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844806df20 con 0x7f84481451f0 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84480cc900 con 0x7f84480ce510 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:32774/0 (socket says 172.21.15.82:32774) 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 -- 172.21.15.82:0/3493055660 learned_addr learned my addr 172.21.15.82:0/3493055660 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f84477fe640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 0x7f84480c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f8446ffd640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f8448140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 -- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 msgr2=0x7f8448140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f8448140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.629 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 -- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 msgr2=0x7f84480c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 0x7f84480c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 -- 172.21.15.82:0/3493055660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480aa9e0 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f84477fe640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 0x7f84480c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f843ffff640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4e0 tx=0x7f843000c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430018070 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430007500 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84480aacc0 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430004a30 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.626+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84480ab280 con 0x7f8448144ca0 2024-04-25T10:14:33.630 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84480ae090 con 0x7f8448144ca0 2024-04-25T10:14:33.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f843001b070 con 0x7f8448144ca0 2024-04-25T10:14:33.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f8444ff9640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 0x7f842c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:33.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8430084300 con 0x7f8448144ca0 2024-04-25T10:14:33.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f8446ffd640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 0x7f842c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:33.631 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.630+0000 7f8446ffd640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 0x7f842c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8438006fd0 tx=0x7f8438008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:33.633 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.634+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8430055400 con 0x7f8448144ca0 2024-04-25T10:14:33.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.774+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f844804efa0 con 0x7f842c066f60 2024-04-25T10:14:33.775 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.774+0000 7f8444ff9640 1 -- 172.21.15.82:0/3493055660 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13490 (secure 0 0 0) 0x7f844804efa0 con 0x7f842c066f60 2024-04-25T10:14:33.776 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:33.776 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:14:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 msgr2=0x7f842c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 0x7f842c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8438006fd0 tx=0x7f8438008040 comp rx=0 tx=0).stop 2024-04-25T10:14:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 msgr2=0x7f84480aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.777 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7f843000c4e0 tx=0x7f843000c9a0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 shutdown_connections 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f842c066f60 0x7f842c069420 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f84481451f0 0x7f84480c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f8448144ca0 0x7f84480aa090 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 --2- 172.21.15.82:0/3493055660 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f84480ce510 0x7f8448140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 >> 172.21.15.82:0/3493055660 conn(0x7f84480b11a0 msgr2=0x7f84480c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 shutdown_connections 2024-04-25T10:14:33.778 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:33.778+0000 7f844d9cb640 1 -- 172.21.15.82:0/3493055660 wait complete. 2024-04-25T10:14:33.792 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-04-25T10:14:33.500941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262948,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711772,"statfs":{"total":773094113280,"available":772824854528,"internally_reserved":0,"allocated":13803520,"data_stored":7082880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.648305"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":372,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T10:14:26.011141+0000","last_change":"2024-04-25T10:01:57.038085+0000","last_active":"2024-04-25T10:14:26.011141+0000","last_peered":"2024-04-25T10:14:26.011141+0000","last_clean":"2024-04-25T10:14:26.011141+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:14:26.011141+0000","last_undegraded":"2024-04-25T10:14:26.011141+0000","last_fullsized":"2024-04-25T10:14:26.011141+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:56.006456+0000","last_clean_scrub_stamp":"2024-04-25T10:01:56.006456+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:44.750927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575002,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2170880,"data_stored":1328136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2170880,"data_stored":1328136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:33.796 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:01:56.006456+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=1, tm_sec=56, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=14, tm_sec=28, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:14:33.796 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:14:39.229 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:39.233 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:53.796 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:14:53.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.926+0000 7fc5fa369640 1 Processor -- start 2024-04-25T10:14:53.928 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5fa369640 1 -- start start 2024-04-25T10:14:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5fa369640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40b3f50 0x7fc5f415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5fa369640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f40b2090 con 0x7fc5f415a700 2024-04-25T10:14:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5fa369640 1 -- --> v1:172.21.15.196:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f40b43e0 con 0x7fc5f40b24a0 2024-04-25T10:14:53.931 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5fa369640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f415f200 con 0x7fc5f40b3f50 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f3fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7fc5f415a700 0x7fc5f40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41636/0 (socket says 172.21.15.82:41636) 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f3fff640 1 -- 172.21.15.82:0/2015617972 learned_addr learned my addr 172.21.15.82:0/2015617972 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321805506 0 0) 0x7fc5f40b2090 con 0x7fc5f415a700 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5dc003530 con 0x7fc5f415a700 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f37fe640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40b3f50 0x7fc5f415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v1:172.21.15.82:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1244929138 0 0) 0x7fc5dc003530 con 0x7fc5f415a700 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40b3f50 msgr2=0x7fc5f415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40b3f50 0x7fc5f415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 >> v1:172.21.15.196:6789/0 conn(0x7fc5f40b24a0 legacy=0x7fc5f406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 --> v1:172.21.15.82:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f40b46c0 con 0x7fc5f415a700 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v1:172.21.15.82:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1064934806 0 0) 0x7fc5e4001680 con 0x7fc5f415a700 2024-04-25T10:14:53.932 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 >> v1:172.21.15.82:6789/0 conn(0x7fc5f415a700 legacy=0x7fc5f40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5dc003a60 0x7fc5dc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5dc004810 0x7fc5dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5dc0051e0 0x7fc5dc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f415f200 con 0x7fc5dc0051e0 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc00c0a0 con 0x7fc5dc003a60 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5dc00c2a0 con 0x7fc5dc004810 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5dc0051e0 0x7fc5dc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f3fff640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5dc003a60 0x7fc5dc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f37fe640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5dc004810 0x7fc5dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 -- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5dc004810 msgr2=0x7fc5dc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5dc004810 0x7fc5dc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 -- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5dc003a60 msgr2=0x7fc5dc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.933 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5dc003a60 0x7fc5dc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f8906640 1 -- 172.21.15.82:0/2015617972 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc003530 con 0x7fc5dc0051e0 2024-04-25T10:14:53.934 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.930+0000 7fc5f3fff640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5dc003a60 0x7fc5dc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5dc0051e0 0x7fc5dc00b9d0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001e70 tx=0x7fc5ec01c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b8f1e9b742bca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec001a90 con 0x7fc5dc0051e0 2024-04-25T10:14:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec003b80 con 0x7fc5dc0051e0 2024-04-25T10:14:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f2ffd640 1 -- 172.21.15.82:0/2015617972 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec005080 con 0x7fc5dc0051e0 2024-04-25T10:14:53.937 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- 172.21.15.82:0/2015617972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5dc0051e0 msgr2=0x7fc5dc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 --2- 172.21.15.82:0/2015617972 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5dc0051e0 0x7fc5dc00b9d0 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001e70 tx=0x7fc5ec01c750 comp rx=0 tx=0).stop 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 -- 172.21.15.82:0/2015617972 reap_dead start 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- 172.21.15.82:0/2015617972 shutdown_connections 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- 172.21.15.82:0/2015617972 >> 172.21.15.82:0/2015617972 conn(0x7fc5f40b11a0 msgr2=0x7fc5f40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- 172.21.15.82:0/2015617972 shutdown_connections 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- 172.21.15.82:0/2015617972 wait complete. 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 Processor -- start 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- start start 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4147a60 0x7fc5f40ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 0x7fc5f40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40cd310 con 0x7fc5f40c4950 2024-04-25T10:14:53.938 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40ce600 con 0x7fc5f4147a60 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5fa369640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f40cd490 con 0x7fc5f40ad830 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51286/0 (socket says 172.21.15.82:51286) 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 -- 172.21.15.82:0/3339819360 learned_addr learned my addr 172.21.15.82:0/3339819360 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f3fff640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4147a60 0x7fc5f40ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f37fe640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 0x7fc5f40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 -- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 msgr2=0x7fc5f40c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 0x7fc5f40c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 -- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4147a60 msgr2=0x7fc5f40ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4147a60 0x7fc5f40ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f8906640 1 -- 172.21.15.82:0/3339819360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f40adcd0 con 0x7fc5f40c4950 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.934+0000 7fc5f37fe640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 0x7fc5f40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f8906640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec098140 tx=0x7fc5dc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec0b8070 con 0x7fc5f40c4950 2024-04-25T10:14:53.939 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec005610 con 0x7fc5f40c4950 2024-04-25T10:14:53.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f40adfb0 con 0x7fc5f40c4950 2024-04-25T10:14:53.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec007dc0 con 0x7fc5f40c4950 2024-04-25T10:14:53.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f4148410 con 0x7fc5f40c4950 2024-04-25T10:14:53.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f40cbe80 con 0x7fc5f40c4950 2024-04-25T10:14:53.940 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5ec0bb070 con 0x7fc5f40c4950 2024-04-25T10:14:53.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 0x7fc5e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc5ec11dcb0 con 0x7fc5f40c4950 2024-04-25T10:14:53.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f3fff640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 0x7fc5e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.941 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.938+0000 7fc5f3fff640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 0x7fc5e0069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001cd0 tx=0x7fc5e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.942 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:53.942+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5ec0ac020 con 0x7fc5f40c4950 2024-04-25T10:14:54.100 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 --> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5f4148610 con 0x7fc5e0066f60 2024-04-25T10:14:54.102 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5f0ff9640 1 -- 172.21.15.82:0/3339819360 <== mgr.4104 v2:172.21.15.82:6832/1711591888 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7fc5f4148610 con 0x7fc5e0066f60 2024-04-25T10:14:54.102 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:54.102 INFO:teuthology.orchestra.run.smithi082.stderr:dumped all 2024-04-25T10:14:54.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 msgr2=0x7fc5e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 0x7fc5e0069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4001cd0 tx=0x7fc5e4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 msgr2=0x7fc5f414a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.104 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.102+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec098140 tx=0x7fc5dc005840 comp rx=0 tx=0).stop 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 shutdown_connections 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fc5e0066f60 0x7fc5e0069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fc5f40ad830 0x7fc5f40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fc5f40c4950 0x7fc5f414a090 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 --2- 172.21.15.82:0/3339819360 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fc5f4147a60 0x7fc5f40ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 >> 172.21.15.82:0/3339819360 conn(0x7fc5f40b11a0 msgr2=0x7fc5f4160d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 shutdown_connections 2024-04-25T10:14:54.105 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.106+0000 7fc5fa369640 1 -- 172.21.15.82:0/3339819360 wait complete. 2024-04-25T10:14:54.121 INFO:teuthology.orchestra.run.smithi082.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-04-25T10:14:53.506226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":262936,"kb_used_data":13480,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711784,"statfs":{"total":773094113280,"available":772824866816,"internally_reserved":0,"allocated":13803520,"data_stored":7082880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003054"},"pg_stats":[{"pgid":"1.0","version":"126'119","reported_seq":383,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T10:14:34.192533+0000","last_change":"2024-04-25T10:14:34.192612+0000","last_active":"2024-04-25T10:14:34.192533+0000","last_peered":"2024-04-25T10:14:34.192533+0000","last_clean":"2024-04-25T10:14:34.192533+0000","last_became_active":"2024-04-25T10:01:57.035796+0000","last_became_peered":"2024-04-25T10:01:57.035796+0000","last_unstale":"2024-04-25T10:14:34.192533+0000","last_undegraded":"2024-04-25T10:14:34.192533+0000","last_fullsized":"2024-04-25T10:14:34.192533+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"126'119","last_scrub_stamp":"2024-04-25T10:14:34.192351+0000","last_deep_scrub":"126'119","last_deep_scrub_stamp":"2024-04-25T10:14:34.192351+0000","last_clean_scrub_stamp":"2024-04-25T10:14:34.192351+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:30.489709+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575006,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1328136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575006,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575008,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575007,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32724,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339116,"statfs":{"total":96636764160,"available":96603254784,"internally_reserved":0,"allocated":1576960,"data_stored":737768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2170880,"data_stored":1328136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:14:54.122 DEBUG:teuthology.orchestra.run.smithi082:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:14:54.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 Processor -- start 2024-04-25T10:14:54.242 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 -- start start 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdc015da80 0x7fcdc00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 --2- >> v2:172.21.15.196:3300/0 conn(0x7fcdc00d0620 0x7fcdc00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc006c440 con 0x7fcdc015da80 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 -- --> v2:172.21.15.196:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc006be60 con 0x7fcdc00d0620 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc769e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc015de60 con 0x7fcdc00d1cd0 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdc015da80 0x7fcdc00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7fcdc015da80 0x7fcdc00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51302/0 (socket says 172.21.15.82:51302) 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 -- 172.21.15.82:0/560728861 learned_addr learned my addr 172.21.15.82:0/560728861 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc5c3b640 1 --2- 172.21.15.82:0/560728861 >> v2:172.21.15.196:3300/0 conn(0x7fcdc00d0620 0x7fcdc00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc4c39640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.243 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 -- 172.21.15.82:0/560728861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 msgr2=0x7fcdc00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 -- 172.21.15.82:0/560728861 >> v2:172.21.15.196:3300/0 conn(0x7fcdc00d0620 msgr2=0x7fcdc00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 --2- 172.21.15.82:0/560728861 >> v2:172.21.15.196:3300/0 conn(0x7fcdc00d0620 0x7fcdc00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.244 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.242+0000 7fcdc543a640 1 -- 172.21.15.82:0/560728861 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00d2450 con 0x7fcdc015da80 2024-04-25T10:14:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc015da80 0x7fcdc00d4230 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcdc006de60 tx=0x7fcdbc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1daaac1ce968f777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdb7fff640 1 -- 172.21.15.82:0/560728861 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0b0070 con 0x7fcdc015da80 2024-04-25T10:14:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdb7fff640 1 -- 172.21.15.82:0/560728861 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0ab600 con 0x7fcdc015da80 2024-04-25T10:14:54.245 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdb7fff640 1 -- 172.21.15.82:0/560728861 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0ab9d0 con 0x7fcdc015da80 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- 172.21.15.82:0/560728861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc015da80 msgr2=0x7fcdc00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc015da80 0x7fcdc00d4230 secure :-1 s=READY pgs=2335 cs=0 l=1 rev1=1 crypto rx=0x7fcdc006de60 tx=0x7fcdbc01c520 comp rx=0 tx=0).stop 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- 172.21.15.82:0/560728861 shutdown_connections 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- 172.21.15.82:0/560728861 >> v2:172.21.15.196:3300/0 conn(0x7fcdc00d0620 0x7fcdc00ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- 172.21.15.82:0/560728861 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc015da80 0x7fcdc00d4230 unknown :-1 s=CLOSED pgs=2335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- 172.21.15.82:0/560728861 >> 172.21.15.82:0/560728861 conn(0x7fcdc008b000 msgr2=0x7fcdc00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- 172.21.15.82:0/560728861 shutdown_connections 2024-04-25T10:14:54.246 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- 172.21.15.82:0/560728861 wait complete. 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 Processor -- start 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- start start 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc0156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 0x7fcdc0156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc006ca50 con 0x7fcdc00d0620 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc006c8d0 con 0x7fcdc015da80 2024-04-25T10:14:54.249 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc769e640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc006cbd0 con 0x7fcdc00d1cd0 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51308/0 (socket says 172.21.15.82:51308) 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 -- 172.21.15.82:0/2938402870 learned_addr learned my addr 172.21.15.82:0/2938402870 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc5c3b640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 0x7fcdc0156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc4c39640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc0156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 -- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 msgr2=0x7fcdc0156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc0156240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 -- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 msgr2=0x7fcdc0156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 0x7fcdc0156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 -- 172.21.15.82:0/2938402870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc0157a50 con 0x7fcdc00d0620 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc5c3b640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 0x7fcdc0156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc4c39640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.250 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.246+0000 7fcdc543a640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc01abd0 tx=0x7fcdbc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0b0040 con 0x7fcdc00d0620 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0b4620 con 0x7fcdc00d0620 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0b4a10 con 0x7fcdc00d0620 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc0156db0 con 0x7fcdc00d0620 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc0157370 con 0x7fcdc00d0620 2024-04-25T10:14:54.251 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc01575b0 con 0x7fcdc00d0620 2024-04-25T10:14:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdbc0ab4c0 con 0x7fcdc00d0620 2024-04-25T10:14:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 0x7fcda4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcdbc11eb00 con 0x7fcdc00d0620 2024-04-25T10:14:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdc4c39640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 0x7fcda4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.252 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.250+0000 7fcdc4c39640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 0x7fcda4069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0157ca0 tx=0x7fcdb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.254 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.254+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcdbc016110 con 0x7fcdc00d0620 2024-04-25T10:14:54.408 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.410+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fcdc00b0510 con 0x7fcdc00d0620 2024-04-25T10:14:54.415 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.414+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7fcdbc0efc60 con 0x7fcdc00d0620 2024-04-25T10:14:54.417 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdb5ffb640 1 -- 172.21.15.82:0/2938402870 <== mon.0 v2:172.21.15.82:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fcdbc0ef560 con 0x7fcdc00d0620 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 msgr2=0x7fcda4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 0x7fcda4069450 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0157ca0 tx=0x7fcdb0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 msgr2=0x7fcdc0155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.418 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc01abd0 tx=0x7fcdbc0192c0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 shutdown_connections 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7fcda4066f90 0x7fcda4069450 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.419 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7fcdc00d1cd0 0x7fcdc0156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7fcdc015da80 0x7fcdc0156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 --2- 172.21.15.82:0/2938402870 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7fcdc00d0620 0x7fcdc0155970 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 >> 172.21.15.82:0/2938402870 conn(0x7fcdc008b000 msgr2=0x7fcdc00d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 shutdown_connections 2024-04-25T10:14:54.420 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.418+0000 7fcdc769e640 1 -- 172.21.15.82:0/2938402870 wait complete. 2024-04-25T10:14:54.435 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:54.435 DEBUG:teuthology.orchestra.run.smithi082:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 Processor -- start 2024-04-25T10:14:54.548 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 -- start start 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 --2- >> v2:172.21.15.82:3300/0 conn(0x7f58580b3f50 0x7f585815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58580b24a0 0x7f58580b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 -- --> v1:172.21.15.82:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58580b2090 con 0x7f585815a700 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 -- --> v2:172.21.15.82:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585815f200 con 0x7f58580b3f50 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585e9f0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580b43e0 con 0x7f58580b24a0 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5857fff640 1 --1- >> v1:172.21.15.82:6789/0 conn(0x7f585815a700 0x7f58580cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.82:6789/0 says I am v1:172.21.15.82:41644/0 (socket says 172.21.15.82:41644) 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5857fff640 1 -- 172.21.15.82:0/2967745939 learned_addr learned my addr 172.21.15.82:0/2967745939 (peer_addr_for_me v1:172.21.15.82:0/0) 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 --2- 172.21.15.82:0/2967745939 >> v2:172.21.15.82:3300/0 conn(0x7f58580b3f50 0x7f585815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5856ffd640 1 -- 172.21.15.82:0/2967745939 <== mon.0 v1:172.21.15.82:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967174978 0 0) 0x7f58580b2090 con 0x7f585815a700 2024-04-25T10:14:54.550 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5856ffd640 1 -- 172.21.15.82:0/2967745939 --> v1:172.21.15.82:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5844003530 con 0x7f585815a700 2024-04-25T10:14:54.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f585cf8d640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58580b24a0 0x7f58580b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 -- 172.21.15.82:0/2967745939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58580b24a0 msgr2=0x7f58580b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58580b24a0 0x7f58580b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 -- 172.21.15.82:0/2967745939 >> v1:172.21.15.82:6789/0 conn(0x7f585815a700 legacy=0x7f58580cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.551 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 -- 172.21.15.82:0/2967745939 --> v2:172.21.15.82:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58580b46c0 con 0x7f58580b3f50 2024-04-25T10:14:54.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f58577fe640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58580b3f50 0x7f585815ecc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f58400097c0 tx=0x7f584001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770092fc316374cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5856ffd640 1 -- 172.21.15.82:0/2967745939 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400b0070 con 0x7f58580b3f50 2024-04-25T10:14:54.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5856ffd640 1 -- 172.21.15.82:0/2967745939 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f58400ab600 con 0x7f58580b3f50 2024-04-25T10:14:54.552 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.550+0000 7f5856ffd640 1 -- 172.21.15.82:0/2967745939 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400ab9f0 con 0x7f58580b3f50 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/2967745939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58580b3f50 msgr2=0x7f585815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58580b3f50 0x7f585815ecc0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7f58400097c0 tx=0x7f584001c380 comp rx=0 tx=0).stop 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/2967745939 shutdown_connections 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f58580b24a0 0x7f58580b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- 172.21.15.82:0/2967745939 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f58580b3f50 0x7f585815ecc0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/2967745939 >> 172.21.15.82:0/2967745939 conn(0x7f58580b11a0 msgr2=0x7f58580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/2967745939 shutdown_connections 2024-04-25T10:14:54.553 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/2967745939 wait complete. 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 Processor -- start 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- start start 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 0x7f5858152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f585815a700 0x7f5858159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580cd310 con 0x7f5858157450 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- --> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58580ce600 con 0x7f58580b24a0 2024-04-25T10:14:54.556 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- --> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58580cd490 con 0x7f585815a700 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 --2- >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.82:3300/0 says I am v2:172.21.15.82:51316/0 (socket says 172.21.15.82:51316) 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 -- 172.21.15.82:0/829473825 learned_addr learned my addr 172.21.15.82:0/829473825 (peer_addr_for_me v2:172.21.15.82:0/0) 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f5857fff640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 0x7f5858152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f58577fe640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f585815a700 0x7f5858159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 -- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f585815a700 msgr2=0x7f5858159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f585815a700 0x7f5858159bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 -- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 msgr2=0x7f5858152fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 0x7f5858152fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 -- 172.21.15.82:0/829473825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858153820 con 0x7f5858157450 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f5857fff640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 0x7f5858152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585cf8d640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f584800afd0 tx=0x7f584800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848015070 con 0x7f5858157450 2024-04-25T10:14:54.557 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5848004220 con 0x7f5858157450 2024-04-25T10:14:54.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5858153b00 con 0x7f5858157450 2024-04-25T10:14:54.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848003620 con 0x7f5858157450 2024-04-25T10:14:54.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.554+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58581540c0 con 0x7f5858157450 2024-04-25T10:14:54.558 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585804efa0 con 0x7f5858157450 2024-04-25T10:14:54.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58480037c0 con 0x7f5858157450 2024-04-25T10:14:54.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f5854ff9640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 0x7f5838069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5848084e30 con 0x7f5858157450 2024-04-25T10:14:54.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f5857fff640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 0x7f5838069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.559 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.558+0000 7f5857fff640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 0x7f5838069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f585815a6d0 tx=0x7f5850009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.561 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.562+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5848055e60 con 0x7f5858157450 2024-04-25T10:14:54.751 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.750+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 --> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f585806f260 con 0x7f5858157450 2024-04-25T10:14:54.752 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f5854ff9640 1 -- 172.21.15.82:0/829473825 <== mon.0 v2:172.21.15.82:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f5848055760 con 0x7f5858157450 2024-04-25T10:14:54.752 INFO:teuthology.orchestra.run.smithi082.stdout: 2024-04-25T10:14:54.752 INFO:teuthology.orchestra.run.smithi082.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6038,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6835/1981810990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6834","nonce":1981810990},{"type":"v1","addr":"172.21.15.196:6835","nonce":1981810990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":87},{"gid":6958,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.196:6833/3364106837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.196:6832","nonce":3364106837},{"type":"v1","addr":"172.21.15.196:6833","nonce":3364106837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":9489,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6836/2625375483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6834","nonce":2625375483},{"type":"v1","addr":"172.21.15.82:6836","nonce":2625375483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86},{"gid":10305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.82:6837/3407874337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.82:6835","nonce":3407874337},{"type":"v1","addr":"172.21.15.82:6837","nonce":3407874337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[]} 2024-04-25T10:14:54.752 INFO:teuthology.orchestra.run.smithi082.stderr:dumped fsmap epoch 106 2024-04-25T10:14:54.754 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 msgr2=0x7f5838069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 0x7f5838069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f585815a6d0 tx=0x7f5850009040 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 msgr2=0x7f58581578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7f584800afd0 tx=0x7f584800ca20 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 shutdown_connections 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:6832/1711591888,v1:172.21.15.82:6833/1711591888] conn(0x7f5838066f60 0x7f5838069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3301/0,v1:172.21.15.196:6790/0] conn(0x7f585815a700 0x7f5858159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.82:3300/0,v1:172.21.15.82:6789/0] conn(0x7f5858157450 0x7f58581578b0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 --2- 172.21.15.82:0/829473825 >> [v2:172.21.15.196:3300/0,v1:172.21.15.196:6789/0] conn(0x7f58580b24a0 0x7f5858152fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 >> 172.21.15.82:0/829473825 conn(0x7f58580b11a0 msgr2=0x7f58580af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 shutdown_connections 2024-04-25T10:14:54.755 INFO:teuthology.orchestra.run.smithi082.stderr:2024-04-25T10:14:54.754+0000 7f585e9f0640 1 -- 172.21.15.82:0/829473825 wait complete. 2024-04-25T10:14:54.770 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:14:54.770 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:14:54.770 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:14:54.770 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:54.781 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:14:54.781 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:14:54.781 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:54.834 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:14:54.834 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:14:54.834 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:54.855 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:14:54.855 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:14:54.855 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:54.913 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:14:54.913 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:14:54.914 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:14:54.914 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:54.974 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:14:54.974 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:14:54.974 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:55.051 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:14:55.051 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:14:55.051 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:55.123 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:14:55.123 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:14:55.123 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:55.220 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:14:55.221 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:14:55.221 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:14:57.650 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-04-25T10:15:01.323 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:15:01.324 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:15:01.324 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:03.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-25T10:15:03.356 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-04-25T10:15:07.425 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:15:07.426 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:15:07.426 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.490 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:15:07.490 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:15:07.490 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.575 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:15:07.575 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:15:07.575 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:15:07.576 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.604 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:15:07.604 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:15:07.605 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.633 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:15:07.633 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:15:07.633 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:15:07.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.666 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:15:07.667 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:15:07.667 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.691 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:15:07.691 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:15:07.691 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:07.759 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:15:07.759 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:15:07.759 DEBUG:teuthology.orchestra.run.smithi082:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T10:15:07.777 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:15:07.778 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:15:08.201 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:15:08.201 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:15:08.357 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-25T10:15:08.395 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:15:08.395 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:15:08.567 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:15:08.567 DEBUG:teuthology.orchestra.run.smithi082:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:15:08.727 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:15:08.727 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:15:09.071 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:15:09.071 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:15:09.322 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:15:09.322 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:15:09.374 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:09.377 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:09.537 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:15:09.538 DEBUG:teuthology.orchestra.run.smithi196:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:15:09.796 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:15:09.797 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/data/mon.a.tgz 2024-04-25T10:15:09.797 DEBUG:teuthology.orchestra.run.smithi082:> mktemp 2024-04-25T10:15:09.803 INFO:teuthology.orchestra.run.smithi082.stdout:/tmp/tmp.I93TC7kSXa 2024-04-25T10:15:09.803 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.I93TC7kSXa 2024-04-25T10:15:09.956 DEBUG:teuthology.orchestra.run.smithi082:> sudo chmod 0666 /tmp/tmp.I93TC7kSXa 2024-04-25T10:15:10.016 DEBUG:teuthology.orchestra.remote:smithi082:/tmp/tmp.I93TC7kSXa is 450KB 2024-04-25T10:15:10.083 DEBUG:teuthology.orchestra.run.smithi082:> rm -fr /tmp/tmp.I93TC7kSXa 2024-04-25T10:15:10.088 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/data/mon.b.tgz 2024-04-25T10:15:10.088 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T10:15:10.094 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.36KbMWywKB 2024-04-25T10:15:10.094 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.36KbMWywKB 2024-04-25T10:15:10.244 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.36KbMWywKB 2024-04-25T10:15:10.311 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.36KbMWywKB is 472KB 2024-04-25T10:15:10.377 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.36KbMWywKB 2024-04-25T10:15:10.383 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/data/mon.c.tgz 2024-04-25T10:15:10.383 DEBUG:teuthology.orchestra.run.smithi196:> mktemp 2024-04-25T10:15:10.435 INFO:teuthology.orchestra.run.smithi196.stdout:/tmp/tmp.cyZHPw7G6j 2024-04-25T10:15:10.436 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cyZHPw7G6j 2024-04-25T10:15:10.587 DEBUG:teuthology.orchestra.run.smithi196:> sudo chmod 0666 /tmp/tmp.cyZHPw7G6j 2024-04-25T10:15:10.651 DEBUG:teuthology.orchestra.remote:smithi196:/tmp/tmp.cyZHPw7G6j is 472KB 2024-04-25T10:15:10.718 DEBUG:teuthology.orchestra.run.smithi196:> rm -fr /tmp/tmp.cyZHPw7G6j 2024-04-25T10:15:10.723 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:15:10.723 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:15:10.725 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:15:10.786 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:15:10.827 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:15:10.828 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:15:10.829 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi082/crash 2024-04-25T10:15:10.830 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:15:10.844 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi196/crash 2024-04-25T10:15:10.845 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:15:10.884 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:15:10.884 DEBUG:teuthology.orchestra.run.smithi082:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:10.889 DEBUG:teuthology.orchestra.run.smithi196:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:10.969 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:10.984 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:11.024 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:15:11.025 DEBUG:teuthology.orchestra.run.smithi082:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:15:11.029 DEBUG:teuthology.orchestra.run.smithi196:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:15:11.048 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40554.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42176.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40554.log.gz 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39935.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42176.log: /var/log/ceph/ceph-client.admin.46745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56855.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39935.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42176.log.gz 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39935.log.gz 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38198.log 2024-04-25T10:15:11.049 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57805.log 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56855.log: /var/log/ceph/ceph-client.admin.38198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39533.log 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57805.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38198.log.gz 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56855.log.gz 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57805.log.gz 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49563.log 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57883.log 2024-04-25T10:15:11.050 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39533.log: /var/log/ceph/ceph-client.admin.49563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37618.log 2024-04-25T10:15:11.051 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57883.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2024-04-25T10:15:11.051 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49563.log.gz 2024-04-25T10:15:11.051 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41804.log 2024-04-25T10:15:11.051 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57883.log.gz 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44991.log 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37618.log.gz 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41804.log.gz 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44991.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35134.log 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44991.log.gz 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36510.log 2024-04-25T10:15:11.061 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49589.log: /var/log/ceph/ceph-client.admin.35134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57089.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36510.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35134.log.gz 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57089.log: gzip -5 --verbose 88.0% -- -- replaced with /var/log/ceph/ceph-client.admin.36510.log.gz /var/log/ceph/ceph-client.admin.34123.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57089.log.gz 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39773.log: /var/log/ceph/ceph-client.admin.34123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33907.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34123.log.gz 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39156.log 2024-04-25T10:15:11.062 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33907.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54304.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38354.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35924.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54304.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39156.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52826.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38354.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54304.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35924.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38354.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55542.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35924.log.gz 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.52826.log: -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:15:11.063 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55542.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39184.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52826.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55282.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46281.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39184.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35582.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55282.log: /var/log/ceph/ceph-client.admin.46281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54330.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55282.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55984.log 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46281.log.gz 2024-04-25T10:15:11.064 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.35582.log.gz -5 --verbose 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.55394.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53034.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55984.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54330.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39800.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55984.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55394.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54215.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48479.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39800.log: /var/log/ceph/ceph-client.admin.54215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52971.log 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39800.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54215.log.gz 2024-04-25T10:15:11.065 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41246.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27699.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33502.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27699.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27699.log.gz 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.1.33502.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27747.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr: 90.9%/var/log/ceph/ceph-osd.6.log: -- replaced with /var/log/ceph/ceph-client.1.33502.log.gz 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27843.log 2024-04-25T10:15:11.109 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27747.log: /var/log/ceph/ceph-client.admin.27843.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.27747.log.gz 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27723.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27843.log.gz 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33617.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.27723.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.3.33617.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27723.log.gz 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5/var/log/ceph/ceph-mgr.x.log: --verbose -- /var/log/ceph/ceph-client.admin.27795.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.3.33617.log.gz 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27795.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27795.log.gz/var/log/ceph/ceph-mon.c.log: 2024-04-25T10:15:11.110 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27771.log 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27771.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27771.log.gz 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mds.b.log: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:15:11.111 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:15:11.118 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-mon.b.log: 94.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:15:11.119 INFO:teuthology.orchestra.run.smithi196.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:15:11.119 INFO:teuthology.orchestra.run.smithi196.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27819.log 2024-04-25T10:15:11.119 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27675.log 2024-04-25T10:15:11.119 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27819.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27819.log.gz 2024-04-25T10:15:11.119 INFO:teuthology.orchestra.run.smithi196.stderr:/var/log/ceph/ceph-client.admin.27675.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27675.log.gz 2024-04-25T10:15:11.142 INFO:teuthology.orchestra.run.smithi196.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:15:11.193 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48479.log.gz 2024-04-25T10:15:11.193 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40221.log 2024-04-25T10:15:11.193 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35896.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.52971.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41246.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44488.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40221.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35896.log: /var/log/ceph/ceph-client.admin.44488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35896.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44488.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40195.log 2024-04-25T10:15:11.194 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42863.log: /var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36848.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40195.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35340.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44070.log: /var/log/ceph/ceph-client.admin.36848.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.49307.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36848.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47407.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35340.log.gz 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2024-04-25T10:15:11.195 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49307.log: /var/log/ceph/ceph-client.admin.47407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53924.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49307.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1%/var/log/ceph/ceph-client.admin.40273.log: -- replaced with /var/log/ceph/ceph-client.admin.47407.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38744.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51358.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53924.log: /var/log/ceph/ceph-client.admin.38744.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52400.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53924.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38744.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.51358.log.gz 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56959.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52400.log: /var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52005.log 2024-04-25T10:15:11.196 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.52400.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52005.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56959.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45069.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46122.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52005.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55478.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45069.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35138.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55478.log: /var/log/ceph/ceph-client.admin.47943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55478.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-04-25T10:15:11.197 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48177.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39429.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35138.log.gz 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40814.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48177.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44638.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40814.log.gz/var/log/ceph/ceph-osd.0.log: 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53366.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52504.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53366.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44638.log.gz 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51045.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33737.log 2024-04-25T10:15:11.198 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52504.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53366.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44455.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52504.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51045.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40984.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44455.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33737.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44455.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57779.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40984.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35088.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36298.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35088.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57779.log.gz 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35530.log 2024-04-25T10:15:11.199 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.2.36298.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35088.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35132.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35530.log: /var/log/ceph/ceph-client.admin.35132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49772.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35530.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47258.log: 91.5% -- replaced with /var/log/ceph/ceph-client.2.36298.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35132.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53238.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56311.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53238.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.49772.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39024.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42522.log 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39024.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53238.log.gz 2024-04-25T10:15:11.200 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51979.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39024.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51501.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42522.log: /var/log/ceph/ceph-client.admin.51979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42522.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5%/var/log/ceph/ceph-client.admin.51501.log: -- replaced with /var/log/ceph/ceph-client.admin.51979.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.56311.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51501.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37168.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39102.log 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz 2024-04-25T10:15:11.201 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37168.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56611.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37168.log.gz 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51527.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39102.log: /var/log/ceph/ceph-client.0.56611.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39102.log.gz 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35333.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50492.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51527.log.gz 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54617.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35333.log.gz 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-04-25T10:15:11.202 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.54617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50492.log.gz 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38770.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55702.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz -5 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.36406.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35349.log: /var/log/ceph/ceph-client.0.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 93.3% -- replaced with /var/log/ceph/ceph-client.0.56611.log.gz 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.203 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57727.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36406.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37194.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37194.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57727.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42431.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37194.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40958.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49882.log 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55229.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz 2024-04-25T10:15:11.204 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53957.log 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40958.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49882.log: /var/log/ceph/ceph-client.admin.42490.log: gzip -5 --verbose -- 92.9% /var/log/ceph/ceph-client.admin.49667.log 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.0.54617.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36822.log 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53957.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38458.log/var/log/ceph/ceph-client.admin.49667.log: 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.49882.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53957.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.55702.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45474.log 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38458.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36822.log.gz 2024-04-25T10:15:11.205 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42993.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57597.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38458.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45474.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41769.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42993.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50662.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57597.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45017.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.41769.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41942.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43940.log 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50662.log.gz 2024-04-25T10:15:11.206 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47828.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41942.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43940.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47828.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49077.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39483.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57623.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50628.log 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-04-25T10:15:11.207 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37142.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50628.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57623.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 91.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.49077.log.gz --verbose 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.51097.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50628.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45121.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37142.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35347.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45121.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49798.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45121.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:gzip/var/log/ceph/ceph-client.admin.35347.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39721.log 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49798.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35347.log.gz 2024-04-25T10:15:11.208 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55006.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49798.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55853.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3%/var/log/ceph/ceph-client.admin.55006.log: -- replaced with /var/log/ceph/ceph-client.admin.39721.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49954.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55853.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55006.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40037.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55853.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39904.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41064.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40037.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35141.log 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41064.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39904.log.gz 2024-04-25T10:15:11.209 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42941.log 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41064.log.gz 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35870.log 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.49954.log.gz 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42941.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35141.log.gz 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54899.log 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42941.log.gz 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57675.log 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54899.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35870.log.gz 2024-04-25T10:15:11.210 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42230.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5/var/log/ceph/ceph-client.admin.57675.log: --verbose -- /var/log/ceph/ceph-client.admin.34201.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54899.log.gz 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57675.log.gz 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35766.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34201.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42230.log.gz 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51711.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34201.log.gz 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35766.log.gz 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41568.log 2024-04-25T10:15:11.211 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35137.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41568.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51711.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36562.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41568.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37434.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35137.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50022.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42150.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45773.log 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50022.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36562.log.gz 2024-04-25T10:15:11.212 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55360.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50022.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42150.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45773.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55360.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45773.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35818.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.2.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55360.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51868.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38919.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39351.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35818.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.51868.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51813.log 2024-04-25T10:15:11.213 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42124.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51813.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38919.log.gzgzip 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37116.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42124.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51813.log.gz 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37460.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42124.log.gz 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34510.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37116.log: /var/log/ceph/ceph-client.admin.34510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52328.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37116.log.gz 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48203.log 2024-04-25T10:15:11.214 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34510.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.52328.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36952.log 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48203.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48203.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 94.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.0.51868.log.gz -- 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr: /var/log/ceph/ceph-client.admin.46174.log 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45448.log 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36952.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46174.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22748.log 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46174.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54133.log 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.tmp-client.admin.22748.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45448.log.gz 2024-04-25T10:15:11.215 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57753.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22748.log.gz 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37802.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57753.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54133.log.gz 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41742.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57545.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37802.log: -- replaced with /var/log/ceph/ceph-client.admin.57753.log.gz 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55123.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37802.log.gz 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41742.log.gz 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54055.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57545.log: /var/log/ceph/ceph-client.admin.55123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33855.log 2024-04-25T10:15:11.216 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57545.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43464.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55123.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45500.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54055.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33855.log.gz/var/log/ceph/ceph-client.admin.43464.log: 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51953.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34227.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45500.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51953.log: -- replaced with /var/log/ceph/ceph-client.admin.43464.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57415.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51953.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36157.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34227.log: /var/log/ceph/ceph-client.admin.57415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40247.log 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.36157.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34227.log.gz 2024-04-25T10:15:11.217 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50416.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57415.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44200.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.52328.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40247.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50416.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.36157.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52031.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50416.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41517.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.44200.log.gz --verbose 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.40762.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52031.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40762.log: gzip -5 89.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.52031.log.gz /var/log/ceph/ceph-client.admin.56091.log 2024-04-25T10:15:11.218 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mon.a.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40762.log.gz 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33802.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56091.log: /var/log/ceph/ceph-client.admin.33802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56091.log.gz 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4%/var/log/ceph/ceph-client.admin.41908.log: -- replaced with /var/log/ceph/ceph-client.admin.33802.log.gz 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37854.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39377.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.37854.log: /var/log/ceph/ceph-client.admin.54107.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46017.log 2024-04-25T10:15:11.219 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37854.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39377.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38172.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54107.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47915.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46017.log: /var/log/ceph/ceph-client.admin.38172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44377.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47915.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38172.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47800.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46017.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35036.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47915.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44377.log.gz 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44807.log 2024-04-25T10:15:11.220 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47800.log: /var/log/ceph/ceph-client.admin.35036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52899.log 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56717.log 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47800.log.gz 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35036.log.gz 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51466.log 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44807.log.gz 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51685.log 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.52899.log.gz 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56717.log.gz 2024-04-25T10:15:11.221 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53716.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51466.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51633.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51685.log: /var/log/ceph/ceph-client.admin.53716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51685.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51633.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53716.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48862.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51633.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37776.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35556.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48862.log: 89.0%/var/log/ceph/ceph-client.admin.37776.log: -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37403.log 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48862.log.gz 2024-04-25T10:15:11.222 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44746.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37776.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35556.log: /var/log/ceph/ceph-client.admin.37403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9%/var/log/ceph/ceph-client.admin.44746.log: -- replaced with /var/log/ceph/ceph-client.admin.37403.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48669.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44746.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47355.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35556.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52164.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50048.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48669.log: /var/log/ceph/ceph-client.admin.52164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27556.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48669.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45638.log 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52164.log.gz 2024-04-25T10:15:11.223 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57649.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50048.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.27556.log: /var/log/ceph/ceph-client.admin.45638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47746.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.27556.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57649.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55801.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45638.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47746.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57649.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49486.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45681.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47746.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55801.log: /var/log/ceph/ceph-client.admin.49486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55801.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54382.log 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3%/var/log/ceph/ceph-client.admin.45681.log: -- replaced with /var/log/ceph/ceph-client.admin.49486.log.gz 2024-04-25T10:15:11.224 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36374.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43286.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45681.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49641.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54382.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36374.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38998.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43286.log: /var/log/ceph/ceph-client.admin.49641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55958.log 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49641.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 90.7%/var/log/ceph/ceph-client.admin.38998.log: -- replaced with /var/log/ceph/ceph-client.admin.43286.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43019.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38998.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-04-25T10:15:11.225 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55958.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57063.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55775.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48966.log: -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34640.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52273.log 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57063.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55775.log: /var/log/ceph/ceph-client.admin.34640.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55775.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34640.log.gz 2024-04-25T10:15:11.226 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40788.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34614.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52273.log.gz 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44299.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35062.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34614.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40788.log.gz 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34614.log.gz 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43770.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35062.log.gz/var/log/ceph/ceph-client.admin.54847.log: 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44403.log 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-04-25T10:15:11.227 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46854.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43770.log.gz 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38120.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44403.log: /var/log/ceph/ceph-client.admin.46854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48722.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38120.log: 90.1% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.46854.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44403.log.gz 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38120.log.gz 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57310.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.48722.log.gz 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48584.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 89.0%/var/log/ceph/ceph-client.admin.57310.log: -5 -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz --verbose 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.35634.log 2024-04-25T10:15:11.228 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57310.log.gz 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 88.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.48584.log.gz /var/log/ceph/ceph-client.admin.56210.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35634.log: /var/log/ceph/ceph-client.admin.47284.log: gzip 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35634.log.gz -5 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54159.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56210.log: gzip -5 --verbose -- 88.6% /var/log/ceph/ceph-client.admin.38432.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45095.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56210.log.gz 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53690.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54159.log.gz 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38432.log: /var/log/ceph/ceph-client.admin.45095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35792.log 2024-04-25T10:15:11.229 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37644.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45095.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38432.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53690.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38614.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35792.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48762.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37644.log: /var/log/ceph/ceph-client.admin.38614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34253.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37644.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1%/var/log/ceph/ceph-client.admin.48762.log: -- replaced with /var/log/ceph/ceph-client.admin.38614.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38380.log 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.48762.log.gz 2024-04-25T10:15:11.230 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34253.log.gz/var/log/ceph/ceph-client.admin.41038.log: 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49694.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51607.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45745.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38380.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54873.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38867.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45745.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51607.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54873.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41116.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45745.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54873.log.gz 2024-04-25T10:15:11.231 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38867.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35660.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41116.log: /var/log/ceph/ceph-client.admin.47511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36207.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49436.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41116.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36207.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35660.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43906.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36207.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52667.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49436.log: gzip/var/log/ceph/ceph-client.admin.43906.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50792.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52667.log: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57037.log 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43906.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49436.log.gz 2024-04-25T10:15:11.232 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52667.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50792.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50792.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51659.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52635.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57037.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52774.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56386.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51659.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52635.log: /var/log/ceph/ceph-client.admin.52774.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.52635.log.gz -5 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: --verbose -- /var/log/ceph/ceph-client.admin.38146.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35350.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56386.log: /var/log/ceph/ceph-client.admin.38146.log: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33704.log 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.52774.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56386.log.gz 2024-04-25T10:15:11.233 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38146.log.gz 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35350.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35350.log.gz 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53427.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33704.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53427.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33704.log.gz 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34958.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.3.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53427.log.gz 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50818.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34719.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50818.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34958.log.gz 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57011.log 2024-04-25T10:15:11.234 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8%/var/log/ceph/ceph-client.admin.34719.log: -- replaced with /var/log/ceph/ceph-client.admin.50818.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42301.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34719.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56480.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56135.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57011.log: /var/log/ceph/ceph-client.admin.56480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51438.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57011.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56480.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56135.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.56135.log.gz 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49829.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56829.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5%/var/log/ceph/ceph-client.admin.49829.log: -- replaced with /var/log/ceph/ceph-client.admin.51438.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46719.log 2024-04-25T10:15:11.235 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49829.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56829.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56829.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43388.log 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36744.log 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46719.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48401.log 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55453.log 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36744.log: /var/log/ceph/ceph-client.admin.48401.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43388.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36744.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48401.log.gz 2024-04-25T10:15:11.236 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49746.log 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46593.log 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49746.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55453.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53167.log 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49746.log.gz 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46593.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46593.log.gz 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40606.log 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52086.log 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40606.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53167.log.gz 2024-04-25T10:15:11.237 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51737.log 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40059.log 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40606.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52086.log: /var/log/ceph/ceph-client.admin.51737.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.52086.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51737.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39965.log 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39965.log: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40059.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39965.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52195.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-04-25T10:15:11.238 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40015.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3%/var/log/ceph/ceph-client.admin.40015.log: -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49180.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.40015.log.gz 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35844.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45197.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35844.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51581.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54716.log 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35844.log.gz 2024-04-25T10:15:11.239 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45197.log: /var/log/ceph/ceph-client.admin.51581.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45197.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51581.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47100.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49383.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47100.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37750.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54716.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47100.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49383.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49383.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27779.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50154.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7%/var/log/ceph/ceph-client.admin.27779.log: -- replaced with /var/log/ceph/ceph-client.admin.37750.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45526.log 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27779.log.gz 2024-04-25T10:15:11.240 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50154.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50154.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43211.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34906.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47485.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45526.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43211.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43211.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52225.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33828.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5%/var/log/ceph/ceph-client.admin.52225.log: -- replaced with /var/log/ceph/ceph-client.admin.47485.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48151.log 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52225.log.gz 2024-04-25T10:15:11.241 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33828.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33828.log.gz 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47074.log 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44887.log 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47074.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48151.log.gz 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45965.log 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41272.log 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47074.log.gz 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44887.log: /var/log/ceph/ceph-client.admin.45965.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44887.log.gz 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45965.log.gz 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52609.log 2024-04-25T10:15:11.242 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44690.log 2024-04-25T10:15:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52609.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41272.log.gz 2024-04-25T10:15:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-04-25T10:15:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-04-25T10:15:11.243 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52609.log.gz 2024-04-25T10:15:11.243 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44690.log: /var/log/ceph/ceph-client.admin.48229.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44690.log.gz 2024-04-25T10:15:11.328 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39128.log 2024-04-25T10:15:11.328 INFO:teuthology.orchestra.run.smithi082.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57857.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39128.log: /var/log/ceph/ceph-client.admin.57857.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39128.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.57857.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47152.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34097.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56556.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4%/var/log/ceph/ceph-client.admin.34097.log: -- replaced with /var/log/ceph/ceph-client.admin.47152.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34097.log.gz 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45849.log 2024-04-25T10:15:11.329 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42614.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56556.log: /var/log/ceph/ceph-client.admin.45849.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56556.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45849.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56338.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42614.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42614.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39219.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56338.log: /var/log/ceph/ceph-client.admin.39219.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39219.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.56338.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38068.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33636.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34588.log: /var/log/ceph/ceph-client.admin.38068.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38068.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.0.43150.log 2024-04-25T10:15:11.330 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33636.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33636.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56743.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.43150.log: /var/log/ceph/ceph-client.admin.56743.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56743.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.43150.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44835.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37985.log: /var/log/ceph/ceph-client.admin.34880.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44835.log: /var/log/ceph/ceph-client.admin.42640.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44835.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-04-25T10:15:11.331 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50993.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37512.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49615.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52996.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50993.log: /var/log/ceph/ceph-client.admin.49615.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50993.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49615.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49859.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52996.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.52996.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41010.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43591.log 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49859.log: /var/log/ceph/ceph-client.admin.41010.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49859.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41010.log.gz 2024-04-25T10:15:11.332 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48073.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39050.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57362.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48073.log.gz 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43591.log.gz 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39050.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39050.log.gz 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40351.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35976.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57362.log: /var/log/ceph/ceph-client.admin.40351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44226.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57362.log.gz 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7%/var/log/ceph/ceph-client.admin.35976.log: -- replaced with /var/log/ceph/ceph-client.admin.40351.log.gz 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.35976.log.gz 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51208.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53560.log 2024-04-25T10:15:11.333 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51208.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44226.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44965.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47204.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.51208.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53560.log: /var/log/ceph/ceph-client.admin.44965.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53560.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44965.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51019.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47204.log: /var/log/ceph/ceph-client.admin.51019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42835.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47204.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39747.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51019.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-04-25T10:15:11.334 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46989.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42835.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55597.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6%/var/log/ceph/ceph-client.admin.57212.log: -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40476.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41882.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55597.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40476.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40476.log.gz 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53343.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34329.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41882.log: /var/log/ceph/ceph-client.admin.53343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54434.log 2024-04-25T10:15:11.335 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34329.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41882.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53343.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34329.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55827.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48335.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54434.log: /var/log/ceph/ceph-client.admin.55827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41194.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.48335.log: 88.2% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54434.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55827.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51071.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41194.log: /var/log/ceph/ceph-client.admin.51071.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41194.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51071.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53293.log 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50602.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-04-25T10:15:11.336 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36614.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53638.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53293.log: /var/log/ceph/ceph-client.admin.36614.log: 92.5% -- replaced with /var/log/ceph/ceph-client.0.48335.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36614.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53293.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35140.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56881.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53638.log: /var/log/ceph/ceph-client.admin.35140.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53638.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35140.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41142.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56881.log: /var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41490.log 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41142.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56881.log.gz 2024-04-25T10:15:11.337 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41142.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34043.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36458.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41490.log: /var/log/ceph/ceph-client.admin.34043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44018.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36458.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34043.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41490.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36458.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39271.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54821.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44018.log: /var/log/ceph/ceph-client.admin.39271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51332.log 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44018.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54821.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39271.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54821.log.gz 2024-04-25T10:15:11.338 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43438.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6%/var/log/ceph/ceph-client.admin.43438.log: -- replaced with /var/log/ceph/ceph-client.admin.51332.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55201.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43438.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40665.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51240.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55201.log: /var/log/ceph/ceph-client.admin.40665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51240.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55201.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40665.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53898.log 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51240.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39403.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-04-25T10:15:11.339 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27665.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57571.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53898.log: /var/log/ceph/ceph-client.admin.27665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46231.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57571.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53898.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27665.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44174.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57571.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46231.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46231.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38224.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44174.log: /var/log/ceph/ceph-client.admin.38224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44174.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47667.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38224.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-04-25T10:15:11.340 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50688.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57493.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50688.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42457.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53060.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42967.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57493.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53060.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42967.log.gz 2024-04-25T10:15:11.341 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54250.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56011.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40299.log: /var/log/ceph/ceph-client.admin.54250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48505.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56011.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54250.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56011.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55334.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48505.log: /var/log/ceph/ceph-client.admin.38718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40923.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55334.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48505.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55334.log.gz 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56160.log 2024-04-25T10:15:11.342 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40923.log: /var/log/ceph/ceph-client.admin.56160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48453.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40923.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56160.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56452.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55932.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48453.log: /var/log/ceph/ceph-client.admin.56452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46541.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48453.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50442.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55932.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56452.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46541.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55932.log.gz 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36926.log 2024-04-25T10:15:11.343 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43880.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46541.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36926.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50442.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43880.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36926.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38536.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43880.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54408.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2%/var/log/ceph/ceph-client.admin.38536.log: -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41374.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38536.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54408.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54408.log.gz 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37004.log 2024-04-25T10:15:11.344 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41374.log: /var/log/ceph/ceph-client.admin.37004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36692.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39669.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41374.log.gz 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37004.log.gz 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34984.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34771.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36692.log: /var/log/ceph/ceph-client.admin.34984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54690.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34771.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34984.log.gz 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36692.log.gz 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34771.log.gz 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52138.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54690.log: /var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37325.log 2024-04-25T10:15:11.345 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54690.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52138.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52138.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36432.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40895.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36432.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37325.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43828.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.36432.log.gz --verbose 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.45043.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40895.log: /var/log/ceph/ceph-client.admin.43828.log: 89.4% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40895.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43828.log.gz 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46937.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38037.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45043.log: /var/log/ceph/ceph-client.admin.46937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41220.log 2024-04-25T10:15:11.346 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38037.log: 89.2% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45043.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46937.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38037.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46911.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50209.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41220.log: /var/log/ceph/ceph-client.admin.46911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49127.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.50209.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41220.log.gz 89.0% 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46911.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50550.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46414.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49127.log: /var/log/ceph/ceph-client.admin.50550.log: 92.1% -- replaced with /var/log/ceph/ceph-client.0.50209.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50550.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49127.log.gz 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40867.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45395.log 2024-04-25T10:15:11.347 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46414.log: /var/log/ceph/ceph-client.admin.40867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38946.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45395.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40867.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46414.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45395.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48699.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47232.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38946.log: /var/log/ceph/ceph-client.admin.48699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39245.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47232.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38946.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48699.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54081.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47232.log.gz 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53612.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39245.log: /var/log/ceph/ceph-client.admin.54081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39852.log 2024-04-25T10:15:11.348 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53612.log: 88.8% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39245.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54081.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53612.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57701.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39852.log: /var/log/ceph/ceph-client.admin.57701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57467.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39852.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43514.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57701.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36536.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53453.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36536.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57467.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38640.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36536.log.gz 2024-04-25T10:15:11.349 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53453.log: /var/log/ceph/ceph-client.admin.38640.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38640.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53453.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50948.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34932.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45585.log: /var/log/ceph/ceph-client.admin.50948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57833.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34932.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34932.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51306.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49984.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57833.log: /var/log/ceph/ceph-client.admin.51306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47048.log 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49984.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51306.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57833.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49984.log.gz 2024-04-25T10:15:11.350 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37540.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40169.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37273.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37540.log: /var/log/ceph/ceph-client.admin.40169.log: 88.7% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47048.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37540.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40169.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44718.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55256.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37273.log: /var/log/ceph/ceph-client.admin.44718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53482.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55256.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37273.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44718.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36666.log 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55256.log.gz 2024-04-25T10:15:11.351 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54768.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53482.log: /var/log/ceph/ceph-client.admin.36666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46800.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54768.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53482.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36666.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54486.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54768.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56691.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54486.log: /var/log/ceph/ceph-client.0.46800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53742.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56691.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54486.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56691.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.46800.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35740.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56907.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53742.log: /var/log/ceph/ceph-client.admin.35740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53664.log 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56907.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53742.log.gz 2024-04-25T10:15:11.352 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35740.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42783.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56907.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53664.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53664.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37486.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55061.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42783.log: /var/log/ceph/ceph-client.admin.37486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.55061.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42783.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37486.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38692.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57238.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38250.log: /var/log/ceph/ceph-client.admin.38692.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38893.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38692.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57238.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57238.log.gz 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39076.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43854.log 2024-04-25T10:15:11.353 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38893.log: /var/log/ceph/ceph-client.admin.39076.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38893.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39076.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53586.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41090.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43854.log: /var/log/ceph/ceph-client.admin.53586.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43854.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53586.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41090.log: 94.4% -- replaced with /var/log/ceph/ceph-client.0.55061.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41090.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37351.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55647.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38802.log: /var/log/ceph/ceph-client.admin.37351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38406.log 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55647.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37351.log.gz 2024-04-25T10:15:11.354 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55647.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34667.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50844.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34667.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38406.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34667.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50844.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50844.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34452.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36640.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42666.log: /var/log/ceph/ceph-client.admin.34452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36640.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56426.log 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36640.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39455.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz 2024-04-25T10:15:11.355 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.34452.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44664.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56426.log: /var/log/ceph/ceph-client.admin.42889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54460.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44664.log: 88.7% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56426.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44664.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52478.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54460.log: /var/log/ceph/ceph-client.admin.52478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50740.log 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52557.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54460.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52478.log.gz 2024-04-25T10:15:11.356 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: -- /var/log/ceph/ceph-client.admin.48636.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34069.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50740.log: /var/log/ceph/ceph-client.admin.48636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56233.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50740.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48636.log.gz/var/log/ceph/ceph-client.admin.34069.log: 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34069.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34175.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37907.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34175.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56233.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37907.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37907.log.gz 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56069.log 2024-04-25T10:15:11.357 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45552.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40403.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56069.log: /var/log/ceph/ceph-client.admin.45552.log: 89.8% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56069.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45552.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54356.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52583.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40403.log: /var/log/ceph/ceph-client.admin.54356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53989.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52583.log: 88.0% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40403.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54356.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52583.log.gz 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53989.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47969.log: /var/log/ceph/ceph-client.admin.51175.log 2024-04-25T10:15:11.358 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42405.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53989.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45719.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52703.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41429.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45719.log: 88.6% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51175.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52703.log: -- replaced with /var/log/ceph/ceph-client.admin.45719.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52703.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38328.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.41429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44586.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34797.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0%/var/log/ceph/ceph-client.admin.44586.log: -- replaced with /var/log/ceph/ceph-client.admin.38328.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44586.log.gz 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35950.log 2024-04-25T10:15:11.359 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40081.log 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34797.log: /var/log/ceph/ceph-client.admin.35950.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34797.log.gz 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35950.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.41429.log.gz 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40081.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.40081.log.gz 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48532.log 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45615.log 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.48532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45615.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48532.log.gz 2024-04-25T10:15:11.360 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45615.log.gz 2024-04-25T10:15:11.361 INFO:teuthology.orchestra.run.smithi196.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:15:11.374 INFO:teuthology.orchestra.run.smithi082.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:15:11.374 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-04-25T10:15:11.374 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57284.log 2024-04-25T10:15:11.374 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52452.log: /var/log/ceph/ceph-client.admin.36796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54925.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57284.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48125.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57284.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54925.log: /var/log/ceph/ceph-client.admin.48125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36900.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.c.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54925.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48125.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46643.log 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36900.log: /var/log/ceph/ceph-client.admin.44913.log: 88.7% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz -- replaced with /var/log/ceph/ceph-client.admin.36900.log.gz 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.375 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56113.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34149.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46643.log: /var/log/ceph/ceph-client.admin.56113.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46643.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.56113.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43540.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54562.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34149.log: /var/log/ceph/ceph-client.admin.43540.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34149.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52876.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43540.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54562.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54562.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37722.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52876.log: /var/log/ceph/ceph-client.admin.37722.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52876.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37722.log.gz 2024-04-25T10:15:11.376 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53166.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53088.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.53166.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53166.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53088.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53088.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43720.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48558.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37696.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43720.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43720.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48279.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37696.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37696.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50766.log 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48279.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48279.log.gz 2024-04-25T10:15:11.377 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39826.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50766.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50766.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45991.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39826.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39826.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44271.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45991.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45991.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44271.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41692.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39993.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41692.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41692.log.gz 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53820.log 2024-04-25T10:15:11.378 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39993.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.39993.log.gz 2024-04-25T10:15:11.379 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37057.log 2024-04-25T10:15:11.379 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53820.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53820.log.gz 2024-04-25T10:15:11.379 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47459.log 2024-04-25T10:15:11.379 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37057.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37057.log.gz 2024-04-25T10:15:11.379 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52112.log 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47459.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47459.log.gz 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42720.log 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52112.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52112.log.gz 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42720.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42720.log.gz 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54956.log 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45799.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-04-25T10:15:11.382 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56506.log 2024-04-25T10:15:11.383 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54956.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.54956.log.gz 2024-04-25T10:15:11.383 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40325.log 2024-04-25T10:15:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56506.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.56506.log.gz 2024-04-25T10:15:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43644.log 2024-04-25T10:15:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40325.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40325.log.gz 2024-04-25T10:15:11.384 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38276.log 2024-04-25T10:15:11.386 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43644.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.43644.log.gz 2024-04-25T10:15:11.386 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53114.log 2024-04-25T10:15:11.387 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38276.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38276.log.gz 2024-04-25T10:15:11.387 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35142.log 2024-04-25T10:15:11.389 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53114.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.53114.log.gz 2024-04-25T10:15:11.389 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44861.log 2024-04-25T10:15:11.390 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35142.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35142.log.gz 2024-04-25T10:15:11.390 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35345.log 2024-04-25T10:15:11.391 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44861.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44861.log.gz 2024-04-25T10:15:11.391 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2024-04-25T10:15:11.392 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35345.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35345.log.gz 2024-04-25T10:15:11.393 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55879.log 2024-04-25T10:15:11.393 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47995.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-04-25T10:15:11.394 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36874.log 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55879.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55879.log.gz 2024-04-25T10:15:11.395 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48836.log 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36874.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36874.log.gz 2024-04-25T10:15:11.396 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53508.log 2024-04-25T10:15:11.397 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48836.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48836.log.gz 2024-04-25T10:15:11.398 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35686.log 2024-04-25T10:15:11.398 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53508.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53508.log.gz 2024-04-25T10:15:11.399 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54276.log 2024-04-25T10:15:11.400 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35686.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35686.log.gz 2024-04-25T10:15:11.400 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2024-04-25T10:15:11.401 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54276.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54276.log.gz 2024-04-25T10:15:11.401 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35348.log 2024-04-25T10:15:11.402 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38302.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2024-04-25T10:15:11.403 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43238.log 2024-04-25T10:15:11.403 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35348.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35348.log.gz 2024-04-25T10:15:11.404 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34017.log 2024-04-25T10:15:11.405 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43238.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43238.log.gz 2024-04-25T10:15:11.405 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54742.log 2024-04-25T10:15:11.406 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34017.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34017.log.gz 2024-04-25T10:15:11.406 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36588.log 2024-04-25T10:15:11.407 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54742.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54742.log.gz 2024-04-25T10:15:11.408 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37933.log 2024-04-25T10:15:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36588.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36588.log.gz 2024-04-25T10:15:11.409 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50896.log 2024-04-25T10:15:11.410 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37933.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37933.log.gz 2024-04-25T10:15:11.410 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49720.log 2024-04-25T10:15:11.411 INFO:teuthology.orchestra.run.smithi196.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:15:11.412 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50896.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50896.log.gz 2024-04-25T10:15:11.412 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44044.log 2024-04-25T10:15:11.413 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49720.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49720.log.gz 2024-04-25T10:15:11.413 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35124.log 2024-04-25T10:15:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44044.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44044.log.gz 2024-04-25T10:15:11.414 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55516.log 2024-04-25T10:15:11.415 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35124.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35124.log.gz 2024-04-25T10:15:11.416 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50576.log 2024-04-25T10:15:11.416 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55516.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55516.log.gz 2024-04-25T10:15:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46388.log 2024-04-25T10:15:11.417 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50576.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50576.log.gz 2024-04-25T10:15:11.418 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42588.log 2024-04-25T10:15:11.419 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46388.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46388.log.gz 2024-04-25T10:15:11.419 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37083.log 2024-04-25T10:15:11.420 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42588.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42588.log.gz 2024-04-25T10:15:11.420 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50366.log 2024-04-25T10:15:11.421 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37083.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37083.log.gz 2024-04-25T10:15:11.422 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2024-04-25T10:15:11.423 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50366.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50366.log.gz 2024-04-25T10:15:11.423 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43045.log 2024-04-25T10:15:11.424 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44325.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2024-04-25T10:15:11.425 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33959.log 2024-04-25T10:15:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43045.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43045.log.gz 2024-04-25T10:15:11.426 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47889.log 2024-04-25T10:15:11.427 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33959.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33959.log.gz 2024-04-25T10:15:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53794.log 2024-04-25T10:15:11.428 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47889.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47889.log.gz 2024-04-25T10:15:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37670.log 2024-04-25T10:15:11.429 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53794.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53794.log.gz 2024-04-25T10:15:11.430 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38562.log 2024-04-25T10:15:11.431 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37670.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37670.log.gz 2024-04-25T10:15:11.431 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52944.log 2024-04-25T10:15:11.432 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38562.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38562.log.gz 2024-04-25T10:15:11.433 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44148.log 2024-04-25T10:15:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52944.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.52944.log.gz 2024-04-25T10:15:11.434 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46885.log 2024-04-25T10:15:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44148.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44148.log.gz 2024-04-25T10:15:11.435 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34693.log 2024-04-25T10:15:11.436 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46885.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46885.log.gz 2024-04-25T10:15:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57160.log 2024-04-25T10:15:11.437 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34693.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34693.log.gz 2024-04-25T10:15:11.438 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40377.log 2024-04-25T10:15:11.439 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57160.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57160.log.gz 2024-04-25T10:15:11.439 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52426.log 2024-04-25T10:15:11.440 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40377.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40377.log.gz 2024-04-25T10:15:11.440 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57186.log 2024-04-25T10:15:11.441 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52426.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.52426.log.gz 2024-04-25T10:15:11.441 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45343.log 2024-04-25T10:15:11.442 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57186.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57186.log.gz 2024-04-25T10:15:11.443 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48890.log 2024-04-25T10:15:11.444 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45343.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45343.log.gz 2024-04-25T10:15:11.444 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52702.log 2024-04-25T10:15:11.445 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48890.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48890.log.gz 91.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:15:11.445 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46200.log 2024-04-25T10:15:11.446 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44781.log 2024-04-25T10:15:11.447 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52702.log: /var/log/ceph/ceph-client.admin.46200.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46200.log.gz 2024-04-25T10:15:11.447 INFO:teuthology.orchestra.run.smithi082.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52702.log.gz 2024-04-25T10:15:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54512.log 2024-04-25T10:15:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51123.log 2024-04-25T10:15:11.448 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44781.log: /var/log/ceph/ceph-client.admin.54512.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44781.log.gz 2024-04-25T10:15:11.449 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-04-25T10:15:11.449 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51123.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54512.log.gz 2024-04-25T10:15:11.449 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51123.log.gz 2024-04-25T10:15:11.450 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34828.log 2024-04-25T10:15:11.450 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35712.log 2024-04-25T10:15:11.450 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47693.log: /var/log/ceph/ceph-client.admin.34828.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-04-25T10:15:11.452 INFO:teuthology.orchestra.run.smithi082.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34828.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50104.log 2024-04-25T10:15:11.454 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35712.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35712.log.gz 2024-04-25T10:15:11.454 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50714.log 2024-04-25T10:15:11.454 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56042.log 2024-04-25T10:15:11.457 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50714.log: 91.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50714.log.gz 2024-04-25T10:15:11.457 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50104.log.gz 2024-04-25T10:15:11.457 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43966.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50870.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56042.log: /var/log/ceph/ceph-client.admin.43966.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56042.log.gz 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.43966.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50870.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50870.log.gz 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42204.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47772.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42379.log: /var/log/ceph/ceph-client.admin.42204.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gz 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42204.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43261.log 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.458 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47772.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47772.log.gz 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38972.log 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46069.log 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43261.log: /var/log/ceph/ceph-client.admin.38972.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.43261.log.gz 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38972.log.gz 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50263.log 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38484.log: /var/log/ceph/ceph-client.admin.46069.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46069.log.gz 2024-04-25T10:15:11.461 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2024-04-25T10:15:11.462 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37592.log 2024-04-25T10:15:11.462 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40117.log 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50263.log: /var/log/ceph/ceph-client.admin.37592.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37592.log.gz 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.50263.log.gz 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55571.log 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45904.log 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40117.log: /var/log/ceph/ceph-client.admin.55571.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40117.log.gz 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55571.log.gz 2024-04-25T10:15:11.464 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42072.log 2024-04-25T10:15:11.466 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.45904.log: 91.8% -- replaced with /var/log/ceph/ceph-client.0.45904.log.gz 2024-04-25T10:15:11.466 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37031.log 2024-04-25T10:15:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40528.log 2024-04-25T10:15:11.467 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42072.log: /var/log/ceph/ceph-client.admin.37031.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42072.log.gz 2024-04-25T10:15:11.467 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37031.log.gzgzip 2024-04-25T10:15:11.467 INFO:teuthology.orchestra.run.smithi082.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-04-25T10:15:11.468 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52531.log 2024-04-25T10:15:11.468 INFO:teuthology.orchestra.run.smithi082.stderr: 89.2%/var/log/ceph/ceph-client.admin.47433.log: -- replaced with /var/log/ceph/ceph-client.admin.40528.log.gz 2024-04-25T10:15:11.469 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.52531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49927.log 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52531.log.gz/var/log/ceph/ceph-client.admin.45317.log: 2024-04-25T10:15:11.470 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2024-04-25T10:15:11.471 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40143.log 2024-04-25T10:15:11.471 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57441.log 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49927.log: /var/log/ceph/ceph-client.admin.40143.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40143.log.gz 2024-04-25T10:15:11.472 INFO:teuthology.orchestra.run.smithi082.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.49927.log.gz 2024-04-25T10:15:11.473 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37377.log 2024-04-25T10:15:11.474 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55906.log 2024-04-25T10:15:11.474 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57441.log: /var/log/ceph/ceph-client.admin.37377.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57441.log.gz 2024-04-25T10:15:11.474 INFO:teuthology.orchestra.run.smithi082.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37377.log.gz 2024-04-25T10:15:11.474 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36718.log 2024-04-25T10:15:11.475 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55906.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55906.log.gz 2024-04-25T10:15:11.475 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55308.log 2024-04-25T10:15:11.476 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38510.log 2024-04-25T10:15:11.476 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36718.log: /var/log/ceph/ceph-client.admin.55308.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36718.log.gz 2024-04-25T10:15:11.477 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43694.log 2024-04-25T10:15:11.477 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55308.log.gz 2024-04-25T10:15:11.477 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38510.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38510.log.gz 2024-04-25T10:15:11.478 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41324.log 2024-04-25T10:15:11.478 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47381.log 2024-04-25T10:15:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43694.log: /var/log/ceph/ceph-client.admin.41324.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43694.log.gz 2024-04-25T10:15:11.479 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39588.log 2024-04-25T10:15:11.479 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2%/var/log/ceph/ceph-client.admin.47381.log: -- replaced with /var/log/ceph/ceph-client.admin.41324.log.gz 2024-04-25T10:15:11.480 INFO:teuthology.orchestra.run.smithi082.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47381.log.gz 2024-04-25T10:15:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37880.log 2024-04-25T10:15:11.480 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42046.log 2024-04-25T10:15:11.481 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.0.39588.log: /var/log/ceph/ceph-client.admin.37880.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37880.log.gz 2024-04-25T10:15:11.484 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50313.log 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42046.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42046.log.gz 2024-04-25T10:15:11.485 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42353.log 2024-04-25T10:15:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.50313.log: 91.3% 93.1% -- replaced with /var/log/ceph/ceph-client.0.39588.log.gz 2024-04-25T10:15:11.487 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50313.log.gz 2024-04-25T10:15:11.487 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47126.log 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37566.log 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0%/var/log/ceph/ceph-client.admin.47126.log: -- replaced with /var/log/ceph/ceph-client.admin.42353.log.gz 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47126.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56795.log 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41994.log 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9%/var/log/ceph/ceph-client.admin.56795.log: -- replaced with /var/log/ceph/ceph-client.admin.37566.log.gz 2024-04-25T10:15:11.489 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56795.log.gz 2024-04-25T10:15:11.491 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56769.log 2024-04-25T10:15:11.491 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56985.log 2024-04-25T10:15:11.491 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41994.log: /var/log/ceph/ceph-client.admin.56769.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41994.log.gz 2024-04-25T10:15:11.491 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38588.log 2024-04-25T10:15:11.491 INFO:teuthology.orchestra.run.smithi082.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56769.log.gz 2024-04-25T10:15:11.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56985.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56985.log.gz 2024-04-25T10:15:11.492 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55175.log 2024-04-25T10:15:11.492 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38588.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38588.log.gz 2024-04-25T10:15:11.493 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48021.log 2024-04-25T10:15:11.493 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-04-25T10:15:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.55175.log: /var/log/ceph/ceph-client.admin.48021.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55175.log.gz 2024-04-25T10:15:11.494 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48021.log.gz 2024-04-25T10:15:11.494 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48610.log 2024-04-25T10:15:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-04-25T10:15:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36978.log: /var/log/ceph/ceph-client.admin.48610.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48610.log.gz 89.0% 2024-04-25T10:15:11.495 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-04-25T10:15:11.495 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45252.log 2024-04-25T10:15:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44096.log 2024-04-25T10:15:11.496 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42327.log: /var/log/ceph/ceph-client.0.45252.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-04-25T10:15:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45147.log 2024-04-25T10:15:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.44096.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44096.log.gz 2024-04-25T10:15:11.497 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-04-25T10:15:11.499 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45147.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45147.log.gz 2024-04-25T10:15:11.500 INFO:teuthology.orchestra.run.smithi082.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.0.45252.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37220.log 2024-04-25T10:15:11.500 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.500 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42809.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-04-25T10:15:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48916.log 2024-04-25T10:15:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-04-25T10:15:11.501 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37220.log: /var/log/ceph/ceph-client.admin.48916.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37220.log.gz 2024-04-25T10:15:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34279.log 2024-04-25T10:15:11.502 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48916.log.gz 2024-04-25T10:15:11.502 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38666.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-04-25T10:15:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-04-25T10:15:11.503 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-04-25T10:15:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34279.log: /var/log/ceph/ceph-client.admin.42020.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-04-25T10:15:11.504 INFO:teuthology.orchestra.run.smithi082.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34279.log.gz 2024-04-25T10:15:11.504 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46669.log 2024-04-25T10:15:11.505 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57388.log 2024-04-25T10:15:11.505 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42692.log: /var/log/ceph/ceph-client.admin.46669.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-04-25T10:15:11.505 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47863.log 2024-04-25T10:15:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57388.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46669.log.gz 2024-04-25T10:15:11.506 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.57388.log.gz 2024-04-25T10:15:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40448.log 2024-04-25T10:15:11.506 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35336.log 2024-04-25T10:15:11.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47863.log: /var/log/ceph/ceph-client.admin.40448.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47863.log.gz 2024-04-25T10:15:11.508 INFO:teuthology.orchestra.run.smithi082.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40448.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51384.log 2024-04-25T10:15:11.508 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.508 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40697.log 2024-04-25T10:15:11.509 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3%/var/log/ceph/ceph-client.admin.51384.log: -- replaced with /var/log/ceph/ceph-client.admin.35336.log.gz 2024-04-25T10:15:11.509 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51384.log.gz 2024-04-25T10:15:11.510 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-04-25T10:15:11.510 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49333.log 2024-04-25T10:15:11.510 INFO:teuthology.orchestra.run.smithi082.stderr: 88.6%/var/log/ceph/ceph-client.admin.47719.log: -- replaced with /var/log/ceph/ceph-client.admin.40697.log.gz 2024-04-25T10:15:11.510 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-04-25T10:15:11.510 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41830.log 2024-04-25T10:15:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53534.log 2024-04-25T10:15:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.49333.log: /var/log/ceph/ceph-client.admin.41830.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41830.log.gz 2024-04-25T10:15:11.512 INFO:teuthology.orchestra.run.smithi082.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49333.log.gz 2024-04-25T10:15:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46043.log 2024-04-25T10:15:11.512 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51553.log 2024-04-25T10:15:11.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53534.log: /var/log/ceph/ceph-client.admin.46043.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.53534.log.gz 2024-04-25T10:15:11.514 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46464.log 2024-04-25T10:15:11.514 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46043.log.gz 2024-04-25T10:15:11.514 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.51553.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.51553.log.gz 2024-04-25T10:15:11.515 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51410.log 2024-04-25T10:15:11.515 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57519.log 2024-04-25T10:15:11.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46464.log: /var/log/ceph/ceph-client.admin.51410.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46464.log.gz 2024-04-25T10:15:11.516 INFO:teuthology.orchestra.run.smithi082.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51410.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40502.log 2024-04-25T10:15:11.516 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.516 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56284.log 2024-04-25T10:15:11.516 INFO:teuthology.orchestra.run.smithi082.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57519.log.gz 2024-04-25T10:15:11.517 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40502.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40502.log.gz 2024-04-25T10:15:11.517 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53872.log 2024-04-25T10:15:11.518 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.56284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-04-25T10:15:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.53872.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53872.log.gz 2024-04-25T10:15:11.519 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57336.log 2024-04-25T10:15:11.519 INFO:teuthology.orchestra.run.smithi082.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.56284.log.gz 2024-04-25T10:15:11.520 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48798.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz 2024-04-25T10:15:11.520 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48047.log 2024-04-25T10:15:11.520 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40580.log 2024-04-25T10:15:11.521 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57336.log: /var/log/ceph/ceph-client.admin.48047.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57336.log.gz 2024-04-25T10:15:11.521 INFO:teuthology.orchestra.run.smithi082.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48047.log.gz 2024-04-25T10:15:11.521 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48099.log 2024-04-25T10:15:11.521 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40580.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40580.log.gz 2024-04-25T10:15:11.523 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34536.log 2024-04-25T10:15:11.523 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36770.log 2024-04-25T10:15:11.523 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48099.log: /var/log/ceph/ceph-client.admin.34536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48099.log.gz 2024-04-25T10:15:11.523 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34536.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54187.log 2024-04-25T10:15:11.523 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.524 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.36770.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36770.log.gz 2024-04-25T10:15:11.524 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55149.log 2024-04-25T10:15:11.524 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33881.log 2024-04-25T10:15:11.526 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54187.log: /var/log/ceph/ceph-client.admin.55149.log: 88.7% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54187.log.gz 2024-04-25T10:15:11.526 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55149.log.gz 2024-04-25T10:15:11.526 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34562.log 2024-04-25T10:15:11.526 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46491.log 2024-04-25T10:15:11.526 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33881.log: /var/log/ceph/ceph-client.admin.34562.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33881.log.gz 2024-04-25T10:15:11.527 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2024-04-25T10:15:11.527 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34562.log.gz 2024-04-25T10:15:11.527 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46491.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46491.log.gz 2024-04-25T10:15:11.528 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48427.log 2024-04-25T10:15:11.528 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43095.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2024-04-25T10:15:11.529 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39878.log 2024-04-25T10:15:11.530 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57115.log 2024-04-25T10:15:11.530 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.48427.log: /var/log/ceph/ceph-client.admin.39878.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48427.log.gz 2024-04-25T10:15:11.530 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39878.log.gz 2024-04-25T10:15:11.530 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38094.log 2024-04-25T10:15:11.531 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.57115.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57115.log.gz 2024-04-25T10:15:11.531 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47570.log 2024-04-25T10:15:11.531 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45369.log 2024-04-25T10:15:11.532 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.38094.log: /var/log/ceph/ceph-client.admin.47570.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38094.log.gz 2024-04-25T10:15:11.532 INFO:teuthology.orchestra.run.smithi082.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47570.log.gz 2024-04-25T10:15:11.532 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2024-04-25T10:15:11.534 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-04-25T10:15:11.534 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.45369.log: /var/log/ceph/ceph-client.admin.41856.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45369.log.gz 2024-04-25T10:15:11.534 INFO:teuthology.orchestra.run.smithi082.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2024-04-25T10:15:11.534 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49256.log 2024-04-25T10:15:11.534 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-04-25T10:15:11.535 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.41639.log: /var/log/ceph/ceph-client.admin.49256.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-04-25T10:15:11.535 INFO:teuthology.orchestra.run.smithi082.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.49256.log.gz 2024-04-25T10:15:11.535 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-04-25T10:15:11.536 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35010.log 2024-04-25T10:15:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43992.log: /var/log/ceph/ceph-client.admin.34854.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-04-25T10:15:11.537 INFO:teuthology.orchestra.run.smithi082.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-04-25T10:15:11.537 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-04-25T10:15:11.538 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35010.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35010.log.gz 2024-04-25T10:15:11.538 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49021.log 2024-04-25T10:15:11.538 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46096.log 2024-04-25T10:15:11.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.39695.log: /var/log/ceph/ceph-client.0.49021.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-04-25T10:15:11.539 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46148.log 2024-04-25T10:15:11.539 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46096.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46096.log.gz 2024-04-25T10:15:11.540 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2024-04-25T10:15:11.541 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46148.log: 93.9% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46148.log.gz 2024-04-25T10:15:11.541 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.0.49021.log.gz 2024-04-25T10:15:11.541 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52060.log 2024-04-25T10:15:11.542 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46338.log 2024-04-25T10:15:11.542 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47178.log: /var/log/ceph/ceph-client.admin.52060.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2024-04-25T10:15:11.542 INFO:teuthology.orchestra.run.smithi082.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52060.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47022.log 2024-04-25T10:15:11.542 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.543 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.46338.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.46338.log.gz 2024-04-25T10:15:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50922.log 2024-04-25T10:15:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35608.log 2024-04-25T10:15:11.544 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.47022.log: /var/log/ceph/ceph-client.admin.50922.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47022.log.gz 2024-04-25T10:15:11.545 INFO:teuthology.orchestra.run.smithi082.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37246.log 2024-04-25T10:15:11.545 INFO:teuthology.orchestra.run.smithi082.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50922.log.gz 2024-04-25T10:15:11.545 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.35608.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35608.log.gz 2024-04-25T10:15:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53768.log 2024-04-25T10:15:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54794.log 2024-04-25T10:15:11.546 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37246.log: /var/log/ceph/ceph-client.admin.53768.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37246.log.gz 2024-04-25T10:15:11.547 INFO:teuthology.orchestra.run.smithi082.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53768.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33933.log 2024-04-25T10:15:11.548 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.54794.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.54794.log.gz 2024-04-25T10:15:11.548 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50524.log 2024-04-25T10:15:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2024-04-25T10:15:11.549 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.33933.log: 87.4%/var/log/ceph/ceph-client.admin.50524.log: -- replaced with /var/log/ceph/ceph-client.admin.33933.log.gz 2024-04-25T10:15:11.549 INFO:teuthology.orchestra.run.smithi082.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50524.log.gz 2024-04-25T10:15:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40632.log 2024-04-25T10:15:11.550 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.43336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-04-25T10:15:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.40632.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40632.log.gz 2024-04-25T10:15:11.551 INFO:teuthology.orchestra.run.smithi082.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2024-04-25T10:15:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36348.log 2024-04-25T10:15:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42748.log 2024-04-25T10:15:11.551 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.37959.log: /var/log/ceph/ceph-client.admin.36348.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-04-25T10:15:11.553 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34745.log 2024-04-25T10:15:11.553 INFO:teuthology.orchestra.run.smithi082.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36348.log.gz 2024-04-25T10:15:11.553 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.42748.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42748.log.gz 2024-04-25T10:15:11.553 INFO:teuthology.orchestra.run.smithi082.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51149.log 2024-04-25T10:15:11.554 INFO:teuthology.orchestra.run.smithi082.stderr:/var/log/ceph/ceph-client.admin.34745.log: /var/log/ceph/ceph-client.admin.51149.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34745.log.gz 2024-04-25T10:15:11.554 INFO:teuthology.orchestra.run.smithi082.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51149.log.gz 2024-04-25T10:15:11.683 INFO:teuthology.orchestra.run.smithi082.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:15:12.199 INFO:teuthology.orchestra.run.smithi082.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:15:12.399 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:15:12.417 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:15:12.448 INFO:teuthology.orchestra.run.smithi196.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:15:12.463 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:15:12.469 INFO:teuthology.orchestra.run.smithi196.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:15:12.471 INFO:teuthology.orchestra.run.smithi196.stderr: 2024-04-25T10:15:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:real 0m1.396s 2024-04-25T10:15:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:user 0m5.881s 2024-04-25T10:15:12.471 INFO:teuthology.orchestra.run.smithi196.stderr:sys 0m0.187s 2024-04-25T10:15:12.585 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:15:12.629 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:15:12.781 INFO:teuthology.orchestra.run.smithi082.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:15:12.783 INFO:teuthology.orchestra.run.smithi082.stderr: 2024-04-25T10:15:12.783 INFO:teuthology.orchestra.run.smithi082.stderr:real 0m1.751s 2024-04-25T10:15:12.783 INFO:teuthology.orchestra.run.smithi082.stderr:user 0m8.029s 2024-04-25T10:15:12.783 INFO:teuthology.orchestra.run.smithi082.stderr:sys 0m0.537s 2024-04-25T10:15:12.783 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:15:12.783 DEBUG:teuthology.misc:Transferring archived files from smithi082:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi082/log 2024-04-25T10:15:12.784 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:15:14.422 DEBUG:teuthology.misc:Transferring archived files from smithi196:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi196/log 2024-04-25T10:15:14.423 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:15:15.190 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:15:15.199 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:15:15.199 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:15:15.202 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:15:15.282 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:15:15.292 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:15:15.293 DEBUG:teuthology.orchestra.run.smithi082:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:15:15.297 DEBUG:teuthology.orchestra.run.smithi196:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout:============================================================================== 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout:+hv01.front.sepi 67.205.162.81 3 u 19 64 377 0.109 -6.085 0.844 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout:*hv02.front.sepi 63.231.80.2 3 u 34 64 377 0.087 -4.160 0.522 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout:+hv03.front.sepi 74.6.168.72 3 u 26 64 377 0.080 -5.011 0.416 2024-04-25T10:15:15.307 INFO:teuthology.orchestra.run.smithi082.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:15:15.309 INFO:teuthology.orchestra.run.smithi196.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T10:15:15.309 INFO:teuthology.orchestra.run.smithi196.stdout:============================================================================== 2024-04-25T10:15:15.309 INFO:teuthology.orchestra.run.smithi196.stdout:+hv01.front.sepi 67.205.162.81 3 u 16 64 377 0.097 -4.406 2.124 2024-04-25T10:15:15.309 INFO:teuthology.orchestra.run.smithi196.stdout:*hv02.front.sepi 63.231.80.2 3 u 37 64 377 0.096 -4.136 1.161 2024-04-25T10:15:15.309 INFO:teuthology.orchestra.run.smithi196.stdout:+hv03.front.sepi 74.6.168.72 3 u 26 64 377 0.080 -4.839 1.001 2024-04-25T10:15:15.310 INFO:teuthology.orchestra.run.smithi196.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T10:15:15.310 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:15:15.320 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:15:15.321 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:15:15.331 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:15:15.343 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:15:15.355 INFO:teuthology.task.internal:Duration was 1439.051492 seconds 2024-04-25T10:15:15.355 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:15:15.365 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:15:15.366 DEBUG:teuthology.orchestra.run.smithi082:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:15:15.368 DEBUG:teuthology.orchestra.run.smithi196:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:15:15.403 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:15:15.404 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi082.front.sepia.ceph.com 2024-04-25T10:15:15.404 DEBUG:teuthology.orchestra.run.smithi082:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:15:15.453 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi196.front.sepia.ceph.com 2024-04-25T10:15:15.453 DEBUG:teuthology.orchestra.run.smithi196:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:15:15.463 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:15:15.464 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:15:15.497 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:15:15.536 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:15:15.536 DEBUG:teuthology.orchestra.run.smithi082:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:15:15.569 DEBUG:teuthology.orchestra.run.smithi196:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:15:15.644 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:15:15.654 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:15:15.654 DEBUG:teuthology.orchestra.run.smithi082:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:15:15.665 DEBUG:teuthology.orchestra.run.smithi196:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:15:15.676 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:15:15.686 DEBUG:teuthology.orchestra.run.smithi082:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:15.709 DEBUG:teuthology.orchestra.run.smithi196:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:15.716 INFO:teuthology.orchestra.run.smithi082.stdout:kernel.core_pattern = core 2024-04-25T10:15:15.729 INFO:teuthology.orchestra.run.smithi196.stdout:kernel.core_pattern = core 2024-04-25T10:15:15.741 DEBUG:teuthology.orchestra.run.smithi082:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:15.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:15.772 DEBUG:teuthology.orchestra.run.smithi196:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:15:15.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:15:15.787 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:15:15.797 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:15:15.797 DEBUG:teuthology.misc:Transferring archived files from smithi082:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi082 2024-04-25T10:15:15.797 DEBUG:teuthology.orchestra.run.smithi082:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:15:15.835 DEBUG:teuthology.misc:Transferring archived files from smithi196:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673040/remote/smithi196 2024-04-25T10:15:15.835 DEBUG:teuthology.orchestra.run.smithi196:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:15:15.855 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:15:15.855 DEBUG:teuthology.orchestra.run.smithi082:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:15:15.873 DEBUG:teuthology.orchestra.run.smithi196:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:15:15.903 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:15:15.914 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:15:15.914 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:15:15.931 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:15:15.931 DEBUG:teuthology.orchestra.run.smithi082:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:15:15.934 DEBUG:teuthology.orchestra.run.smithi196:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:15:15.936 INFO:teuthology.orchestra.run.smithi082.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:15 /home/ubuntu/cephtest 2024-04-25T10:15:15.951 INFO:teuthology.orchestra.run.smithi196.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:15 /home/ubuntu/cephtest 2024-04-25T10:15:15.952 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:15:15.963 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:15:15.997 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1439.0514922142029 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:15:15.998 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:15:16.106 INFO:teuthology.run:pass